Data, data everywhere, but what does it mean?

Many states are collecting extensive education data, but aren’t training teachers and parents in how to use the information effectively to help students learn, concludes the Data Quality Campaign.

Data should be used to improve student achievement and inform parents, not just for “shame and blame,” said Aimee Guidera, executive director of DQC.

Teaching the quantified student

 “I am a bad teacher” wrote Sujata G. Bhatt in Valerie Strauss’ Washington Post blog in the school test-taking season of 2011.  Education reformers want to use data to drive instruction, reform and accountability, wrote Bhatt. “At what cost? Does this data really represent learning and knowledge?”

Since then, she’s embraced data, Bhatt writes in The Quantified Student.

She teaches in a high-poverty Los Angeles school. Many of her students aren’t fluent in English. In the fall of 2010, her fourth graders were particularly unprepared.

Since California’s standardized test for fourth graders measured skills almost all my students needed, I analyzed its requirements, broke them down into core concepts, and then worked and reworked these concepts with the students until they felt a sense of mastery over them. My daily job consisted of finding different, creative ways of approaching, teaching, and reteaching the same core skills so that most all students could incorporate them into their cognitive toolkits.

It worked. The students succeeded wildly. They returned to me for fifth grade with heightened confidence. They saw something new in themselves: the reward of effort and the joy of success.

They also came back with questions about “how many more points it would take to get to the next level, how many more problems they’d need to get right to get those points.”  They saw the test as a game they wanted to win.

Teaching the same cohort in fifth grade, she looked for ways for her students to explore their interest in data. 

We used math websites like TenMarks that enable students to learn about their own learning even as they practice new skills. We analyzed information graphics and dove into ways of presenting numerical information. We explored how numbers shape our understanding of ourselves and the world. And much of their enthusiasm and curiosity for these tasks came out of their interest in numbers from standardized testing.

She now believes standardized testing can help teachers understand how well they’ve taught and help students become “agents in their own learning.”

Testing — and evaluation systems built on test scores — need to get a lot better, Bhatt writes. But it makes more sense “to work to create better data than to fight data.”

Data analysis is an increasingly significant and empowering way of making sense of the world. All sorts of professions use data to interpret their work and decide upon courses of action. Why shouldn’t we in education?

In the high tech world there’s a growing movement called “The Quantified Self.” With quantified self models, adults use data to change habits and behaviors–to lose weight, exercise more, to calm themselves.

“Why not help our students become makers and masters of their own data, and help them use it to propel their own learning forward?” Bhatt asks.

The Measured Man is a fascinating — and somewhat alarming — Atlantic profile of Larry Smarr, an astrophysicist, computer scientist and highly quantified human.

Miami-Dade wins Broad Prize

Miami-Dade’s school district has won the Broad Prize for Urban Education, after five years as a finalist, reports Ed Week.

More black and Hispanic students are scoring “advanced” on state tests and graduating, the foundation said. In addition, more students are taking the SATs and earning higher scores.

(Superintendent Alberto) Carvahlo drew attention to improvements in some of the district’s lowest-performing schools, which he attributed partly to the Data/COM (short for Data assessment, technical assistance, coordination of management, according to Carvalho) process. During Data/COM, school officials analyze a school’s challenges and debate solutions, Carvahlo said.

. . . The district’s budget has also improved dramatically under Carvalho’s tenure, which was noted by the jury. “This may seem strange, but we actually embraced the economic recession as an opportunity to leverage and accomplish change,” he said. The district found additional government and foundation funding and made sure all spending was directed at improving student achievement, Carvalho said.

Runner-ups were Palm Beach County (Florida), Houston and Corono-Norco (California).

Boston Superintendent Carol R. Johnson was honored as the best urban superintendent by the Council of Great City Schools.

Data myths

Defusing myths about classroom data will help teachers reach all students, argues the Dell Foundation.

Common education data myths stifle progress

The value-added debate

Can a few years’ data reveal bad teachers? The New York Times‘ Room for Debate takes on value-added analysis.

Pomp, circumstance and then what?

Few high schools track graduates to see if they’re succeeding in college or careers. Some states are linking high school and college data to evaluate success rates.

Also on Community College Spotlight:  Community college construction has stopped in Los Angeles. The district has billions in bond money, but can’t afford to pay for building maintenance or for instructors to use the new space.

Jerry Brown: Data is useless

School performance data is a “siren song for school reform,”  (pdf) wrote California Gov. Jerry Brown in vetoing a bill to add “multiple indicators,” such as graduation rates, to the state’s Academic Performance Index.

This bill requires a new collection of indices called the “Education Quality Index” (EQI), consisting of “multiple indicators,”many of which are ill-defined and some impossible to design. These “multiple indicators” are to change over time, causing measurement instability and muddling the picture of how schools perform.

SB547 would also add significant costs and confusion to the implementation of the newly-adopted Common Core standards which must be in place by 2014. This bill would require us to introduce a whole new system of accountability at the same time we are required to carry out extensive revisions to school curriculum, teaching materials and tests. That doesn’t make sense.

Finally, while SB547 attempts to improve the API, it relies on the same quantitative and standardized paradigm at the heart of the current system. The criticism of the API is that it has led schools to focus too narrowly on tested subjects and ignore other subjects and matters that are vital to a well-rounded education. SB547 certainly would add more things to measure, but it is doubtful that it would actually improve our schools. Adding more speedometers to a broken car won’t turn it into a high-performance machine.

Over the last 50 years, academic “experts” have subjected California to unceasing pedagogical change and experimentation. The current fashion is to collect endless quantitative data to populate ever-changing indicators of performance to distinguish the educational “good” from the education “bad.”

. . . SB547 nowhere mentions good character or love of learning. It does allude to student excitement and creativity, but does not take these qualities seriously because they can’t be placed in a data stream. Lost in the bill’s turgid mandates is any recognition that quality is fundamentally different from quantity.

There are other ways to improve our schools — to indeed focus on quality. What about a system that relies on locally convened panels to visit schools, observe teachers, interview students, and examine student work? Such a system wouldn’t produce an API number, but it could improve the quality of our schools.

Actually, I doubt it.  Maybe a state school inspector could evaluate school quality without student performance data by looking for signs of good character and love of learning.  Maybe not. A local committee would be easy to snow.

The vetoed bill, SB 547, had broad support, notes John Fensterwald of Educated Guess. The proposed Education Quality Index could have included “dropout rates, the need for remediation in college, success with career technical education programs, and graduation rates.” Standardized test scores would have counted for no more than 40 percent of the score in high school. While critics “questioned whether the EPI would be too squishy,” Brown complained “it would have demanded more of the same, hard data.”


The secrets of high-performing charter schools

High-performing charter management organizations spend more per student at the school level, using some of that money to fund a more teachers per student, writes James Peyser of New Schools Venture Fund in Education Next. The high flyers also invest more in recruiting and developing talented teachers and “building instructional support systems that are grounded in the use of performance data.”

. . . the most successful organizations strive to create enthusiasm for learning and an expectation of college success for all, with a commitment to hard work and persistence in the face of initial failures or setbacks. They have adopted standards-based curricula, with an intensive focus on literacy and numeracy as the first foundation for academic achievement, which typically manifests itself in extra time for reading and math each day and a relatively heavy reliance on direct instruction and differentiated grouping, especially in the early grades. And they are increasingly focused on developing and deploying comprehensive student assessment and coaching systems to ensure more effective and consistent classroom practice, not just from year to year but during the course of each school year.

The five highest-performing CMOs in NewSchools’ portfolio operate 85 schools with more than 28,000 students. Their low-income students have proficiency rates that are more than 25 percentage points higher than those in their local districts.

On average, NewSchools’ CMOs score 9 points higher on reading and math proficiency than district schools, 12 points higher when low-income students are compared and 14 points higher comparing schools open five years or more.

Critics often suggest that superior performance in the charter sector is a result of high levels of attrition, caused by implicit or explicit efforts on the part of school staff to “counsel out” the students who are hardest to educate. Excluding students who move away, our data show average attrition rates of about 12 percent, compared to many schools in high-poverty urban neighborhoods that have annual attrition rates of close to one-third. Interestingly, the highest performers in our portfolio have below-average attrition rates of approximately 9 percent, while the lowest performers have above-average attrition rates of close to 20 percent.

NewSchools CMO students are more likely to graduate from high school than other low-income, minority students and much more likely to enroll in college, Peyser writes.

The value-added bubble

The rush to evaluate teachers by value-added models reminds Rick Hess of the collateralized mortgage bubble.

Edu-econometricians are eagerly building intricate models stacked atop value-added scores. Yet, today’s value-added measures are, at best, a pale measure of teacher quality. There are legitimate concerns about test quality; the noisiness and variability of calculations; the fact that metrics don’t account for the impact of specialists, support staff, or shared instruction; and the degree to which value-added calculations rest upon a narrow, truncated conception of good teaching. Value-added does tell us something useful and I’m in favor of integrating it into evaluation and pay decisions, accordingly, but I worry when it becomes the foundation upon which everything else is constructed.

Even the best model is only as good as the data, Hess writes. If test scores are ” flawed, biased, or incomplete measures of learning or teacher effectiveness, the models won’t pick that up.”

Sturdy hybrid vigor

“Hybrid” schools that combine face-to-face teaching by teachers with online instruction are the next big thing, reports Education Next. The Rocketship schools in San Jose, School of One in New York City, Denver School of Science and Technology, Carpe Diem in Yuma and San Diego’s High Tech High “use technology intensively and thoughtfully to tailor instruction to individual students’ needs, and provide robust, frequent data on their performance,”  write Jonathan Schorr and Deborah McGriff, NewSchools Venture Fund partners.

In the lab, the 1st graders log in by selecting from a group of images that acts as a personal password, and then race through a short assessment that covers math and reading problems. Faced with the prompt “Put all the striped balls in one basket and all the polka-dotted balls in the other basket,” a student named Jazmine uses her mouse to move the objects to their places. Then it’s on to the core activity of her 90 minutes in the lab: a lesson on counting and grouping using software from DreamBox. . . .  A bit later, she’ll read a book from a box targeted at her exact reading level, and make a return visit to the computer to take a short quiz about what she read.

Hybrid schools realize productivity gains, Schorr and McGriff writes. Rocketship hires an aide to monitor 43 students in the computer lab. The money saved is used to pay teachers more and keep class size down in the face-to-face part of the day.

In the future, Rocketship hopes children will be able to “learn much of their basic skills via adaptive technology like the DreamBox software, leaving classroom teachers free to focus on critical-thinking instruction and extra help where kids are struggling.”

Likewise, teachers will be able to “prescribe” online attention to specific skills. Part of the model involves providing teachers with a steady stream of data that will help them adjust instruction to kids’ specific needs, and to guide afterschool tutors.  overwhelming to teachers.

High Tech High uses ALEKS, “a Web-based, artificially intelligent assessment and learning system,” which provides “a snapshot of a student’s knowledge in a given content area, recognizing which topics he has mastered and which he has not.”