Reality Bites: Shock Ripples Across New York State Educational Establishment as Test Scores Plummet

Time to read
4 minutes
Read so far

Reality Bites: Shock Ripples Across New York State Educational Establishment as Test Scores Plummet

July 29, 2010 - 16:52

Chart of Data Falling.jpgThe new team running the State Education system in New York State promised real accountability with tougher, less predictable testing. They appear to have delivered.

In 2009, 86 precent of students in New York State scored a 3 or 4 on the math tests which is considered "at or above grade level" or "passing". This year that figure plummeted to 61%. In English, statewide scores dropped from 77 precent to 53 percent. New Rochelle test scores fell across the board.

“Now that we are facing the hard truth that not all of the gains were as advertised, we have to take a look at what we can do differently,” said Merryl H. Tisch, the chancellor of the State Board of Regents. “These results will finally provide real, unimpeachable evidence to be used for accountability.”

New York Times:

New York State said the tests had become too easy, with some questions varying little from year to year, making it simple for teachers to prepare students because each test is made publicly available after it is given. So this year, the state made the questions less predictable and raised the number of correct answers needed to pass the tests, which are given to every student from the third through the eighth grades.

As usual, Cathey O'Donnell at the Journal News has put together a fabulous database of all of the SED testing data. We are working on culling out just the New Rochelle data and will post that as an update when it is ready.

NOTE: if you would like to volunteer to help extract the New Rochelle specific data contact me ASAP. I could use some help. I am pulling the data from lohud and putting into a Google Doc spreadsheet which I will publish when done.

There are 3 Comments

There's no question that decreasing test result is cause for concern. We--perhaps incorrectly--use these scores as a barometer of growth in our classrooms because it's the hardest data we can access. However, there are a few things to consider:

1) Earlier last week the Board of Regents released plans to lower the "cut marks" for ELA and Math scoring. What this means is that a score that previous qualified as a "3" may, today, qualify as a "2". Which doesn't change the fact that the student in question has not achieved the desired "4", but does call into question how reliably we can compare scores from year to year. Test scores are not reactive to inflation, so a child who accurately answers 80% of the questions this year should get the same score that a child in the next year gets when s/he answers the same portion of questions correctly.

2) When we consider that historically, a student who achieved roughly 18% correct on an ELA exam qualified for a "2" (per Diane Ravitch, The Death and Life of the Great American School System), we can only conclude that the scores in the past have been inflated. Is it so wild to assume that this year's scores have been deflated?

When the state controls the data and the methods used to translate raw score into overall score, they have the ability to generate the results that they want.

Theoretically, next year, should the state decide that our scores are still too low, they could re-adjust those cut marks again to reflect "an explosion of growth" in our classrooms. It is because we don't get access to the raw data, only the "translated" data, that we are really kept in the dark about how our students are performing.

What's even more confounding is that the standards that will be in use for the 2010-2011 school year will be in line with the new Core Standards--a set of guidelines somewhat different from the NY State Standards used in the past. Therefore, the tests offered to our students this year will be testing different material. But you can be that this time next year, when results come out, good or bad, pundits will be looking to make inferences about our teachers and students by comparing two wholly different assessments.

Maybe it's time that we, as a community, look for a new barometer of success. At the high school level, we can hold ourselves to task by examining an accurately disaggregated graduation rate. At the lower levels, we can attempt to gain access to raw testing data, before it goes through the spin machine and comes out a neat one digit number.

Robert Cox's picture

While there is much merit in what you have to say what the community wants is real and meaningful comparisons

Consider that there is a national test as well and that test can serve as a baseline in making state-to-state comparisons as well as making adjustments for longitudinal studies within states. So, while the state itself is not providing it there are means to make apples to apples comparisons.

Further, it is possible to look at deviations from the mean on a longitudinal basis and get a good measure of school-to-school and district-to-district performance.

However it is done, it is false, as is often asserted by school officials in New Rochelle that you cannot compare New Rochelle to any other school district because there is no district that is exactly like New Rochelle. This was said again just the other night at the BoE meeting.

Anyone school official or apologist for the district who asserts that there is no way to compare New Rochelle to any other district is not fit to be part of the discussion.

Actually, I agree entirely with your final point. Since all districts are subject to the mathematical gymnastics performed by the state, we all get painted by the same brush. Further, if, in fact, our students are underperforming, it is cause for concern and solutions need to be investigated.

I simply want to temper what I fear is going to be a bit of fear mongering by certain anti-teacher factions circling the issue.

Consider the implications of re-adjusting the cut mark. When the state recognizes that many districts are nearing re-negotiations, the state can manipulate the most public data to engender negative feelings towards teachers. Similarly, when policy makers need to defend their decisions about state curricula, they can modify cut marks to justify the decisions they have made.

Ultimately, I'm not denying that there may be cause for concern. I'm simply arguing that the recently released data is skewed and spun and should not be taken seriously in its current format. Rather, a push for access to raw data ought to be made so that we can make the accurate longitudinal comparisons that you asked for.