Goldstein, 2014

This article describes an evaluation approach that can be used on both group and single-case research designs (SSED). The approach evaluates types of treatments, research designs, quality of studies and effect sizes. The study follows the graphical presentation of products evaluated by Consumer Reports magazine (no, really–and, it’s cool!).

Research studies were evaluated according to the following criteria:

  • Design characteristics and general validity (and internal validity for group methods)
  • Measurement and reliability (assessment methods, implementation fidelity)
  • General characteristics and results (rationale, treatment effects, statistics)
  • Dimensions of external validity (such as social validity)

Advantages of this technique are:  patterns can be detected, the presentation of the data is clear, and the study is accessible to many researchers (not just quant folks).

Disadvantages of this technique are: inter-rater issues (complexity and training), categories are quite general.

Advertisements
| Leave a comment

Barton, 2015

This article is an example of a literature review of SPED articles. The purpose of this review was to examine studies about sensory treatments for students with disabilities. Outcomes from studies about sensory treatments suggest varying levels of evidence for its efficacy. Meanwhile, the number of therapists and interventionists using it have increased. The review was launched to evaluate studies from ethical, financial, and clinical perspectives. Thirty studies were included in the review. The studies differed notably from one another in their implementation, measurement, and rigor. The article concludes that little evidence exists for their usage.

The methods were informed by the Cochrane and Campbell Collaborations, (Higgins & Green, 2008), the What Works Clearinghouse, the PRIMSA. They were registered with PROSPERO (a protocol for systematic reviews). The authors searched particular databases and included articles found in other reviews. Studies were:

  • Experimental design to compare sensory-based interventions to another treatment or control.
  • Used sensory-based treatments as intervention (per sensory integration theory)
  • Were randomized and quasi-randomized control trials, controlled clinical trials (CCT), and SCRD
  • Excluded case studies, SCRD with fewer than 3 attempts to demonstrate an effect, and qualitative reports
  • Participants were under 9-years-old who had a behavioral or developmental disability
  • Written in English
  • Published in peer-reviewed journals

Two independent coders double-coded all studies. Variables were:

  • Participant characteristics
    • age
    • diagnostic characteristics of sample
    • presence of challenging behaviors
  • Intervention characteristics
    • type of treatment
    • density of intervention
    • materials used
    • training /experience of interventionists
    • setting
  • Outcome measures
    • primary outcome
      • academic
      • adaptive
      • attention/engagement
      • in-seat
      • IQ
      • language
      • motor
      • problem behavior
      • sensory
      • social
    • measurement system
      • standardized assessment
      • direct observation
      • rating scale
    • primary assessor
      • parent
      • teacher
      • interventionist
  • Research characteristics
    • sample size
    • research design
    • risk of bias (using Cochrane Collaboration’s risk of bias tool) and an adaptation of a tool for SCRD (Reichow, Barton, & Maggin, 2013)
    • methodological quality (using What Works Clearinghouse Procedures and Standards Handbook, 2013)
  • Study results
    • were extracted and summarized for each study
    • included p-values and/or effect sizes
    • for SCRD, summarized according to Reichow and Volkmar (2010)

The authors coded four tables and used colored figures to present bias. The researchers discussed limitations in the current literature and limitations in this review. Limitations in the literature were: inconsistent findings, methodological concerns, concerns with underlying theories, bias, lack of fidelity measures, measurement inadequacies, lack of neurological processing measures. Limitations in the review were: methodological decisions made early in the study (did not limit, but did create a less flexible analysis), researcher bias (a study included was done by one of the review authors), much time elapsed among the studies (4 decades), and no studies that measured delayed effects were included.

 

 

| Leave a comment

Gersten, 2005

Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional children, 71(2), 149-164.
This article presents quality indicators for group experimental research. The authors encourage the use of many types of research designs in education–as is seen in many fields in the social sciences. They do note, however, that randomized control trials (RCT) are underutilized in education science. Indicators are presented/grouped as essential or desirable. Indicators for research proposals are listed as well as research studies. For an acceptable quality rating, studies must meet all but one essential quality indicators and at least one desirable indicator. For a high-quality rating, a study must meet all but one essential indicator and at least four desirable indicators.

Indicators for group experimental and quasi-experimental research articles

Essential

For describing participants

  • Sufficient info is provided to determine whether the participant met the requirements (difficulty or disability)
  • Appropriate procedures were used to ensure participants were comparable across conditions
  • Sufficient info about interventionsists/teachers given? Were they comparable across conditions?

For implementation of the intervention and description of comparison conditions

  • Interventions were clearly described and specified
  • Fidelity of implementation described and assessed
  • Nature of services provided in comparison conditions described

For outcome measures

  • Multiple measures closely aligned with intervention were used
  • Outcomes measured at appropriate times

For data analysis

  • Data analysis techniques are closely linked to research questions and hypotheses. Also linked to appropriate unit of analysis in study
  • Research report includes not only inferential stats, but also effect size calculations

Desirable quality indicators

  • Was data available on attrition rates? Severe attrition documented? Less than 30 percent?
  • Internal consistency reliable? Test/re-test reliablity and interrater reliability (when appropriate) for outcome measures? Data collectors/scorers equally (un) familiar to participants?
  • Were outcomes measured beyond immediate post-test?
  • Was evidence of criterion validity and construct validity of the measures provided?
  • Did research team assess surface features of fidelity (procedures) and also examine quality of implementation?
  • Was documentation of instruction provided in comparison conditions?
  • Is there audio/video of implementation?
  • Were results presented in a clear/coherent fashion?

Research to practice

Determining when a practice is evidence-based

  • There are at least 4 acceptable-quality studies and 2 high-quality studies
  • The weighted effect size is significantly greater than zero

Determining when a practice is promising

  • There are at least 4 acceptable-quality studies and 2 high-quality studies
  • There is a 20 percent confidence interval for the weighted effect size that is greater than zero

Practices that lead to lower effect sizes

  • Controlling for teacher effects
  • Using standardized measures
  • using appropriate unit in data analyses
  • reporting sample ethnic composition
  • providing psychometric info
  • using multiple criteria in defining sample

 

 

| Leave a comment

Brantlinger, 2005

Brantlinger, E., Jimenez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative studies in special education. Exceptional children, 71(2), 195-207.

This article defines qualitative studies in SPED, clarifies the goals, suggests techniques for establishing and maintaining credibility in the studies, and offers a list of quality indicators for qualitative studies in SPED.

Definition

Qualitative study is a systematic approach to understanding qualities, or the essential nature, of a phenomenon within a particular context.

Qualitative study involves:

  • Empiricism (a sense experience and observation)
  • Knowledge production
  • Use of particular skills + tools (qualitative methods)
  • Production of scientific evidence
  • A coherent articulation of the results

Goals

  • Producing descriptive or procedural knowledge –what is happening and why is it happening
  • Explores attitudes, opinions, beliefs, personal reactions to SPED contexts and teaching strategies
  • Nature of qualitative study in SPED: can be inductive (reasoning from specific to general) or deductive
  • Types
    • Case study
    • Collective case study
    • Ethnography
    • Action research
    • Collaborative action research
    • Grounded theory
    • Phenomenological
    • Symbolic interactionism
    • Narrative research
    • Life (oral) histor
    • Quasi-life history research
    • Interpretive research
    • Content analysis
    • Conversational analysis
    • Discourse analysis
    • Ideological critique

Interpretation is a necessary stage of qualitative work

  • Critical interrogation of meanings
  • Deconstruction
  • Reflexive and critical analysis
  • Examines cultural constructions and tensions (and labeling of individuals, groups, actions, etc.)
  • Interrogates what we think we know and who we think we are as scholars
  • Discourse analysis
    • Analyzes texts and discourses
    • Looks for embedded assumptions and values

Establishing credibility

Actions which are meant to establish that empirical studies are credible and trustworthy (reliable). Do not use these actions in a rigid or unreflective way in research.

  • Triangulation
  • Disconfirming evidence (aka negative or discrepant case analysis)
  • Researcher reflexivity
  • Member checks
  • Collaborative work (multiple researchers)
  • External auditors
  • Peer debriefing
  • Audit trail
  • Prolonged field engagement
  • Thick, detailed descriptions
  • Particularizability (transferability to other situations)

Quality Indicators

Interviews

  • Careful and thoughtful participant selection
  • Questions are reasonable
  • Recording mechanisms are sound
  • Sensitive and fair representation of participants in account
  • Ensure sound measures of confidentiality

Observation studies

  • Appropriate setting and observer
  • Sufficient time spent in field
  • Research has minimal impact on setting
  • Field notes systematically collected
  • Ensure sound measures of confidentiality

Document analysis

  • Meaningful documents
  • Secure document storage
  • Described and cited
  • Ensure sound measures of confidentiality

Data analysis

  • Results sorted and coded
  • Systematically, meaning fully
  • Rationale for inclusion/exclusion info in the report
  • Methods are documented
  • Reflection of personal perspective is included
  • Connect with related research
  • Ensure sound measures of confidentiality
| Leave a comment

I Read TWO ROMANCES IN A ROW

You should do it, too. It makes ya feel all happy inside (Practical Magic and At the Water’s Edge).

This is my “between semesters” reading pile. Yes, I plan to actually READ Rececca this time, not simply keep her on my night-table for 9 weeks (she likes it here). This is the second time I checked out Driftless and The Other Wes Moore, so I hope to get to those. If not, I’m sure they enjoy visiting me, too.

12509426_10205687164915177_9202279659215231736_n

Posted in books I might read | Leave a comment

The “still between semesters” Book List

Students are back in school, but I’m still between semesters, so I thought I’d squeeze in a list of books. If you’re wondering how books are chosen for this list, you can read about it here.

The first book was described as “this is SUCH a good book!” by readers. It’s a coming-of-age story about life in a tough neighborhood. It’s also about friendship and family–it has all the good stuff. I read it as well and would recommend it. It’s probably for mature 12 year old readers and up, especially those who love basketball. Though anyone can read it, boys might especially enjoy it. The main character is a young man, and he has a girlfriend (who is a strong secondary character in the book). The young man talks “admiringly” about the girlfriend (but, he is also respectful and kind). Just so you know.

Boy 21 Boy 21

The next couple of books were enjoyed by many readers. The first two are graphic novels (always a hit with our reluctant readers). Many students read Nimona 2-3 times, so it must be good. I haven’t read it yet, but I will. I love graphic novels. The next was written by Brian Selznick, who brought us The Invention of Hugo Cabret and Wonderstruck. The next two novels were brought to my attention in a search for historical fiction books with strong female characters. The last novel was added to our collection after the search for historical fiction, but none of the students have read it yet. I’m recommending it anyway because many reviewers I trust also recommended it. I paged through it, and it looks like a pretty quick read. Maybe I’ll tackle it this evening. I’ll let you know.

Nimona Nimona
The Marvels The Marvels
The Stolen One The Stolen One
The True Confessions of Cha... The True Confessions of Charlotte Doyle
Fever 1793 Fever 1793

 

Posted in book lists for young readers | Leave a comment

End of Semester Book List

We’re in the final throes of Fall semester 2015, so how about a study-break book list? This list heavily favors girl readers, but boys might enjoy these suggestions as well (especially Mistwood). As well, it’s never a bad thing for boys to read a book with strong female characters, right? Variety, variety, variety.

The first book in this list is a Newberry Medal winner. I’ll be honest, Newberry books are not heavily favored in this group of students. I’m not sure why that is, and it’s an interesting question. I’ll explore it with readers and get back to you. In the meantime, here’s a good one:

Catherine, Called Birdy Catherine, Called Birdy

The next recommended book is part of a series, but I didn’t know about the series thing when I suggested it to readers. I want to focus on stories with a complete context and closure at the end. But alas, this one sneaked in. No surprise–students found the ending unsatisfying (but that’s generally how a book in a series ends). Readers said the story was “good” despite the ending:

¬†This book was described as “sad.” Before you decide not to recommend it to a small friend, remember that kid-sad is different than grownup-sad.

  • Me: did you like the book?
  • She: Yes! It was sooooooo good. But, it was sad.
  • Me: Tell me more about what made it sad
  • She: People were mean to other people. I don’t know anything else to say
  • Me: Why was it good?
  • She: It was about friendship

Now, get out there and eat/drink/be_merry!

Posted in book lists for young readers | Leave a comment