MCAS pass rate for special ed students inflated by Massachusetts Department of Education; Discounting thousands of students from the count artificially boosts rates and discourages schools from holding on to all students

May, 2003

A Massachusetts Department of Education (DOE) press release of May 7, 2003 (http://www.doe.mass.edu/news/news.asp?id=1503) states that 5,268 out of 7,307 grade 12 special education students have passed MCAS.  The Department calculates the MCAS pass rate for these students as 72%.  Even according to the DOE report, one-quarter of all Massachusetts seniors with disabilities still have not passed MCAS.

However, DOE's method of calculating MCAS pass rates is based on grade 12 enrollment.  Because it does not account for all members of the class of 2003, including students who have been lost from the class of 2003 since enrolling in grade 9 or taking MCAS in grade 10, the DOE method dramatically inflates the pass rate for Massachusetts special education students and misleads the public into believing that "MCAS is working" for students with disabilities.  (For an analysis of MCAS pass rate inflation for the class of 2003 statewide, by race, and for high-need, upper-income, and additional selected districts, see also, http://www.massparents.org/news/2003/inflated_scores.htm).

Analysis of DOE data for the class of 2003 shows the following:
------ Based on grade 9 special ed enrollment numbers (12,437*), the MCAS pass rate for students with disabilities is 42%, not 72%;
------ Based on grade 10 special ed enrollment numbers (9,434), the MCAS pass rate for students with disabilities is 56%, not 72%;
------ Nearly as many special education students have been lost from the class of 2003 since grade 9 as have reached grade 12 with passing MCAS scores;
------  The decrease in special education students enrolled in the class of 2003 between grade 10 and grade 12 nearly matches the increase in students passing MCAS between grade 10 and grade 12.


Declining enrollments inflate pass rates

The "rise" in MCAS special education pass rates reflects as much a decline in special education enrollments - both between grades 9 and 12 and between grades 10 and 12 - as increased numbers of students reaching "competency determination" through testing or appeals.

When pass rates are calculated to account for all special education students who were members of the orginal class of 2003 and enrolled in ninth grade in October 1999, the real pass rate for students with disabilities from the class of 2003 is an estimated 42%, not 72%.
------ Fewer than half the 12,437* special ed students enrolled with the original class of 2003 have survived to grade 12 and will receive a diploma.

Even when pass rates are calculated based on the number of students taking MCAS for the first time in 10th grade in 2001, the pass rate for students with disabilities from the class of 2003 is 56%, not 72%.
------ Even discounting students who repeat or drop out of ninth grade, 44% of the special education students who were enrolled in the class of 2003 when they first took MCAS in grade 10 have not reached twelfth grade with passing MCAS scores.

MCAS special education pass rates reported by DOE reflect a loss of students with disabilities from the class of 2003, not just increasing numbers reaching "competency determination."  The decline in the number of special education students in the class of 2003 as the class has progressed through the grades artifically props up MCAS pass rates for students with disabilities.

In October 1999, special education student enrollment in the orginal class of 2003 numbered 12,437.* Three years later, special education enrollment had declined, as of May 2003, to 7,307.  Given enrollment declines:
------ The number of special education seniors passing MCAS - 5,268 - is only slightly higher than the 5,130 special education students lost from the class since grade 9. 
------ The decrease in the number of students with disabilites enrolled in the class of 2003 since grade 10 - 2,127 - nearly matches the increase in the number of students who have passed MCAS since the first round of testing - 2,438 - over the same time period. 

A more detailed analysis of DOE enrollment data show:

Although the DOE reports that 5,268 special education seniors have passed MCAS, almost as many -- another 5,130 students with disabilities --- have not reached grade 12 with passing MCAS scores:
------ Enrollment of special education students declined by 3,003 between grades 9 and 10 - from 12,437 to 9,434;
------ An additional 1,609 special ed students were lost between grades 10 and 11 - reflecting a drop from 9,434 to 7,825.
----- Another 518 have been lost between grade 11 and the most recent count of grade 12 students.

Since the first testing session in 2001, after three MCAS retests, and four rounds of appeals, nearly as many students with disabilities have been lost from the class as have become eligible for diplomas:
------ Between grades 10 and 12, the number of students with disabilities eligible for a diploma increased by 2,438 students, from 2,830 passing MCAS in grade 10 to 5,268 passing MCAS in grade 12.
------ During the same period, attrition of students with disabilities lost from the class of of 2003 also increased; the number of special education students lost from the class grew by 2,127 students.


The MCAS graduation requirement is not "working" for special education students

State education officials state that "MCAS is working."  But DOE's own data raise serious policy questions:  For whom is MCAS "working?"  To what purpose is MCAS  "working?"

Education reform was intended to benefit all students.  With fewer than half of ninth graders with disabilities and only 56% of tenth graders with disabilities reaching 12th grade with passing MCAS scores, MCAS is not "working" for a large number of students with disabilities.

Our analysis shows that the "results" claimed for the MCAS graduation requirement, defined in terms of the numbers of students with disabilities passing the test or appeals process, are about the same as "results" defined in terms of numbers of students lost from the class of 2003 during the same period. 

Policy makers should be equally concerned with the status of students lost from the class of 2003 as they are with test results for students who survive to grade 12.  While a small percentage of special ed students are "ungraded" and would be expected to remain in school until age 21 rather than graduate from grade 12, it is likely that the large majority of students with disabilities who have been lost from the class have either dropped out or been held back in grade and are at risk for dropping out.

Given the large numbers of students lost from the class of 2003, claims that MCAS is key to inclusion of special education students in the grade-level curriculum also have limited meaning.  

For the thousands of students with disabilities who have been lost from the class of 2003 between grades 9 and 12 and who are not included in the DOE's calculation of MCAS pass rates, "inclusion" is an illusion.  Faithful commitment to a policy of inclusion requires that lost students also be accounted for when calculating MCAS pass rates.

Conclusion

This analysis of DOE data for special education enrollments and MCAS pass rates raises serious policy questions about the MCAS graduation requirement: To what educational purpose is MCAS "working"?  Which results matter?

If enough students are lost from the class through attrition (whether because of dropping out, repeating a grade, or transferring out of the public schools), pass rates could reach 100%. This would be cause for worry, not celebration.  

The Massachusetts Department of Education should restate all pass rates based on enrollments from  grade 9, not grade 12, for the class of 2003.   Reporting MCAS pass rates based on grade 12 enrollments is misleading and can act as a disincentive to schools' doing all they can to hold on to all students.  Education policy makers should adopt policies that encourage, not discourage, schools to improve holding power for all students, especially the most vulnerable.


Student enrollment and students passing MCAS, Class of 2003, by progressive grade

The following data taken from Massachusetts Department of Education, "Progress Report on the Class of 2003: Percentage of Students Who Have Earned a Competency Determination Statewide and by District," September 2002, and Massachusetts Department of Education, "Progress REport on Students Attaining the Competency Determination Statewide and by District: Classes of 2003 and 2004," March 2003 (see http://www.doe.mass.edu/mcas/2002/results for links to both reports as PDF files) as well as from the most recent DOE press release, show the numbers of special education students enrolled in the Class of 2003 each year and the number passing MCAS with each round of MCAS testing.

Grade 9 (School year 1999-00):
--- Special education students enrolled: 12,437 (estimated at 16% of 77,733)

Grade 10 (School year 2000-01):
--- Special education students enrolled: 9,434
--- Special education students passing MCAS: 2,830
--- Pass rate based on grade 10 enrollment: 30%
--- Pass rate based on grade 9 enrollment: 23%

Grade 11 (School year 2001-02):
--- Special education students enrolled: 7,825
--- Special education students passing MCAS: 4,303
--- Pass rate based on grade 11 enrollment: 55%
--- Pass rate basesd on grade 10 enrollment: 46%
--- Pass rate based on grade 9 enrollment: 35%

Grade 12 (School year 2002-03, reported March 2003):
--- Special education students enrolled: 7,292
--- Special education students passing MCAS: 5,025
--- Pass rate based on grade 12 enrollment: 69%
--- Pass rate based on grade 11 enrollment: 64%
--- Pass rate based on grade 10 enrollment: 53%
--- Pass rate based on grade 9 enrollment: 40%

Grade 12 (School year 2002-03, reported May 2003):
--- Special education students enrolled: 7,307
--- Special education students passing MCAS: 5,268
--- Pass rate based on grade 12 enrollment: 72%
--- Pass rate based on grade 11 enrollment: 67%
--- Pass rate based on grade 10 enrollment: 56%
--- Pass rate based on grade 9 enrollment:  42%

*Calculation of MCAS pass rates should be based on the enrollment of special education students who were originally enrolled in the class of 2003 as ninth graders in October 1999.  Since the Massachusetts DOE did not collect special ed enrollments by grade until October 2001, the number of special education students who were enrolled in the orginal class can only be estimated.  Special education students make up 16% of all students enrolled in Massachusetts schools.  In October 1999, 77,733 students were enrolled in 9th grade in the original class of 2003. The number of grade 9 students enrolled in special education in the orginal class of 2003 is estimated as 12,437.

For further information: 
- Anne Wheelock, Progress Through the Education Pipeline Project, Boston College, 617-524-7324;
- George Madaus, National Board on Educational Testing and Public Policy, Boston College, 617-552-4521;
- Walt Haney, Center for the Study of Testing, Evaluation, and Education Policy, Boston College, 617-552-4521.