Research - OSEP
The Methods Behind the OSEP Reporting Feature of CreativeCurriculum.net
Beginning in early 2007, mandatory reports must be submitted by programs receiving funding through Part B/Section 619 and Part C of the Individuals with Disabilities Education Act. Many programs will be using data collected by using CreativeCurriculum.net to generate outcome reports that meet OSEP requirements. Teaching Strategies has worked with Dale Cohen, a psychometrician from the University of North Carolina at Wilmington, to develop a system that allows CreativeCurriulum.net users to generate these reports without additional effort. Below, we discuss how this system works.
The reporting requirements mandate that programs report on children across three outcome areas:
- positive social/emotional skills (including social relationships)
- acquisition and use of knowledge and skills (including early language/communication and, for preschool, early literacy)
- use of appropriate behavior to meet needs
The reporting is somewhat difficult because each program must assess its children in relation to the functioning of a "typical" child of the same age and determine, when the child leaves the program, where the child fits among a number of reporting categories. For each outcome area, programs must first decide whether each child began the program at the level of a typical child or not. If the child was not developing at a typical level at entry, the program must assess, at the child's exit from the program, whether the child progressed to the level of a typical child; progressed nearer to the functioning of a typical child (but did not achieve a typical level); progressed but did not close the gap; or made no progress.
OSEP Outcomes Reporting Categories
|A. Percent of children who did not improve functioning.|
|B. Percent of children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers.|
|C Percent of children who improved functioning to a level nearer to same-aged peers but did not reach it.|
|D. Percent of children who improved functioning to reach a level comparable to same-aged peers.|
|E. Percent of children who maintained functioning at a level comparable to same-aged peers.|
How are CreativeCurriculum.net data used to determine the categorization of a child for the OSEP reports? We used a six-step process to create the reporting system.
Step 1: We determined which items on The Creative Curriculum Developmental Continuum for Ages 3-5 best measured each OSEP reporting outcome area.
The Developmental Continuum consists of 50 assessment items that measure development across four domains. Using exploratory and confirmatory factor analysis, we empirically established groups of objectives that best measured each of the three OSEP outcomes.
Step 2: We created a composite measure for each of the three outcomes using the objectives that best measured each outcome.
To do this, we sum each child's raw scores on the items within each group.
Step 3: We identified the scales that best measure child proficiency in each of the OSEP outcome areas.
Using Rasch Item Response Theory (IRT) analysis for each outcome measure, we estimated a child's latent proficiency for each scale. Both the factor analysis and IRT were conducted on a sample of more than 3000 children from across the country, weighted to approximate the characteristics of the national population of children. From the IRT, we determine each child's scale score for each outcome.
Step 4: With guidance from the Early Childhood Outcomes Center (ECO), we calculated a range that defined typical for each outcome measure.
ECO recommends that typical be defined as children who are at approximately the 10th percentile and above for each of the outcome areas. This is the cutoff we adopted. Accordingly, for each of the outcome areas, we determined the scale score that represents the 10th percentile for children at each month of age from 30-72 months. This was done by estimating three fixed-effects regression models of the scaled scores (one for each outcome) using age and age-squared as predictors. Using the results from the model, we determined the expected value for each month of age and, by using the standard deviation, determined the 10th percentile scale score for each age.
Step 5: Using the same method described in Step 4, we determined where a child falls on the ECO seven-point Child Outcome Summary Form (COSF).
ECO recommends that states use a seven point form to determine age-expected functioning for the purposes of OSEP reporting. Categories six and seven on the form are for children whose functioning is considered typical for their age (i.e., above approximately the 10th percentile). A child would be given a rating of six if his or her functioning was considered "typical" but still of some concern. Categories 1 - 5 are used for children in approximately the bottom 10th percentile of functioning. For each of the seven points on the form, ECO has assigned a percentile range. For example, children in the bottom half percentile of functioning would fall on point 1 of the scale. Children falling in the .52 - 1.97 percentile range would fall on point two of the scale. Using the same method discussed in Step 4, we determined where a child falls on the seven-point COSF.
Step 6: Place the child in the appropriate OSEP reporting category.
To determine the appropriate reporting category for a given child, the system analyzes the child's entry and exit data, compares where the child is on the ECO summary forms at each point in time, and places the child in the appropriate OSEP reporting category. If a child is at either point six or seven at both entry and exit, he or she is placed in OSEP reporting category E ("maintains functioning at a level typical to same-aged peers"). If a child is below point six at entry but at or above point six at exit, he or she is placed in OSEP reporting category D ("improved functioning to reach a level comparable to same-aged peers"). If a child moves up on the ECO form but does not reach point six, he or she is placed in OSEP reporting category C ("improved functioning to a level nearer same-aged peers but did not reach it"). If a child makes progress but does not move up on the form (from point one to point two, for example), he or she is placed in OSEP reporting category B ("improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers"). Finally, if the child makes no progress, he or she is placed in reporting category A ("did not improve functioning").
If you want to learn more about the methods used to create the reporting system, please feel free to call our Director of Public Policy and Research, , at (202) 362-7543, x1742. In addition, a full technical report will be available soon.
 ECO actually recommends that children at or above 1.30 standard deviations below the mean be identified as typical. This corresponds to at or above the 9.68 percentile. While we round to the 10th percentile for simplicity in the text, our calculations use the 9.68 percentile.