You are here: Home / Archives for NAEP

Attack of the reading tests

August 9, 2011 By

Rachel Levy hoped to teach history and geography while developing her high school students’ reading and writing skills. But the principal of her inner-city D.C. school — pre-Rhee — told social studies teachers to spend one-fifth of class time teaching the reading test, Levy writes on Core Knowledge Blog.

Teachers were told to make a chart for each student showing how well he or she did on each skill, such as “context clues.”

Then I was supposed to target my lesson plans to teach and remedy each student’s individual weaknesses. . . . such instruction and data collection had to be documented in our lesson plan books and during classroom observations.

Teach and remedy each student’s individual weaknesses?

While testing doesn’t require such stupidities, few educators have the patience to rely on a “well-rounded and knowledge-rich curriculum” to raise scores gradually, Levy writes.

She tried to persuade colleagues that the way to raise test scores was to “teach content and have students read and write as much as possible.”  No one agreed.

Now raising three children, Levy blogs at All Things Education.

Update:  You need to know how to teach but you also need to know your subject very well, writes Michael Bromley, a social studies teacher who guest-blogged for Rick Hess on Ed Week.  “No matter the teaching strategy, if you don’t have something valid, interesting, and important to teach there will be no learning.”

In June, the National Assessment of Educational Progress released a report showing core historical illiteracy among American school children. In response, famed historian David McCullough told the Wall Street Journal, “People who come out of college with a degree in education and not a degree in a subject are severely handicapped in their capacity to teach effectively because they’re often assigned to teach subjects about which they know little or nothing.”

Wait a minute, there, David, hold on: modern pedagogy states that qualified, education-proficient teachers can teach anything, so long as the correct strategies for student engagement are followed. Isn’t that the problem? David replies, “You can’t love something you don’t know any more than you can love someone you don’t know.” Amen, brother . . .

If you don’t know the subject, your students won’t either, Bromley concludes.

Filed Under: Education, Testing Tagged With: , , , , , , ,

Florida leads in gains by low-income kids

August 7, 2011 By

Florida’s low-income students made impressive gains from 2003 to 2009 on NAEP reading and math tests for fourth and eighth grade, writes Matthew Ladner on Jay Greene’s blog. In fact, Florida also made large gains from 1998 to 2002, Ladner writes, but there isn’t data for all 50 states going back that far. (FRL stands for free and reduced lunch eligibility, the standard measure of student poverty.)

Low-income students lost ground in West Virginia, which is a leader in teaching “21st-century skills.” Coincidence?

Filed Under: Education Tagged With: , , , , ,

Few are proficient in geography

July 19, 2011 By

Fourth graders know more geography, eighth graders are about the same and 12th graders are losing ground, according to the Nation’s Report Card: Geography 2010 by the National Assessment of Educational Progress (NAEP). Fewer than one third of students are proficient.

In geography, civics and U.S. history, achievement is stagnating or declining, NAEP advises.

“In particular, the pattern of disappointing results for our twelfth graders’ performance across all three social science subjects should be of great concern to everyone,” said David P. Driscoll, chairman of the National Assessment Governing Board, which sets policy for NAEP.

The lowest-scoring students made gains at all three grade levels, and some racial/ethnic achievement gaps narrowed.

A proficient fourth grader can recognize what prevents soil erosion, a proficient eighth grader can explain the effects of a monsoon in India and a proficient 12th grader can explain why Mali is considered overpopulated.

Some of the 12th grade questions are challenging:

The diagram shown is a profile of a content from West to East (shown in miles) and elevation (showed in feet or height off of the ground). In the West, the elevation starts at 0, and quickly rises to its maximum elevation of 23,000 feet approximately 750 miles inland. After the this high point, the elevation drops steadily back to 0 at the most easternmost point of the continent, approximately 3,260 miles from the westernmost point of the continent.

The diagram above shows a profile of which continent?

  1.   Europe
  2.   South America
  3.   Antarctica
  4.   Africa

I got it right, but it was an educated guess.

Go here for sample questions.

This map of The World According to Americans represents the way many of us were taught as children, writes Lynne Diligent.

These maps are about feelings rather than knowledge, writes Diligent, who lives overseas.

I used to play a geography game with my father. I’d close my eyes, spin the globe and point to a place. He’d tell me about it. I loved the sound of “Addis Ababa” and “Haile Selassie,” the Lion of Judah.  Later, I played a German game, Weltreise, that taught me the best air, rail and shipping routes. My favorite was Montevideo to Kapstadt (Capetown) to Adelaide by boat.

Filed Under: Education, Testing Tagged With: , , , ,

Hispanic students aren't catching up

June 24, 2011 By

Hispanic fourth and eighth graders didn’t catch up in math and reading from 1990 to 2009, concludes Achievement Gaps, a National Assessment of Educational Progress (NAEP) report issued yesterday. Hispanic students improved, but so did non-Hispanic whites.

Nationwide, the average Hispanic student is working two or more grade levels below the average white student, notes the Christian Science Monitor.  (Ten points on NAEP is the equivalent of one grade level.)

In fourth-grade math in 2009, the average Hispanic score of 227 corresponds with the “basic” skill level, and it indicates that students can make a pictograph of given information, and can determine, in a multiple-choice question, how many given pieces cover a shape.

The white average score of 248, on the other hand, is just one point shy of reaching the “proficient” skill level, and it indicates that these students can subtract a two-digit number from a three-digit number and solve a word problem involving quarts and cups.

Hispanic school enrollment in grades 4 and 8 tripled in the last two decades, growing from 7 percent to 22 percent by 2009.  Some 77 percent of Hispanic students come from low-income families.

Thirty-seven percent in fourth grade and 21 percent in eighth grade are not fully proficient in reading English. Not surprisingly, Hispanic students who’ve achieved proficiency — which is measured by scoring well on tests — do much better than those who aren’t proficient.

For Hispanics who already know English, the gaps with whites have narrowed. That gap was 15 points in Grade 8 reading, for instance, while ELL Hispanics scored 39 points lower than non-ELL Hispanics.

Among low-income students, the gaps between Hispanics and whites have narrowed in reading and eighth-grade math since 2003.

Florida boasts a significantly smaller Hispanic-white achievement gap. Other school systems with smaller-than-average gaps are Kentucky, Missouri, Wyoming and the Department of Defense schools. California, sadly, has a larger-than-average achievement gap.

Filed Under: Education Tagged With: , , , , ,

Don't know much about history

June 14, 2011 By

What’s important about that bearded guy with the stovepipe hat? What advantage did American colonists have over British troops during the Revolutionary War? What country allied with North Korea during the Korean War? American students don’t know much about U.S. history, concludes the Nation’s Report Card 2010. History is the weakest of all subjects tested.

Eighth graders made some progress from 2006 to 2010, while scores were flat at the fourth an 12th-grade level. However, only 17 percent of students score at or above the proficient level.

Since 1994, scores have risen at all three levels. Black and Hispanic fourth and eighth graders have made significant gains since 1994. That’s the good news.

Not so good: Only a third of fourth graders can identify the purpose of the Declaration of Independence.

Two percent of 12th graders can name the social problem — school segregation — that Brown vs. the Board of Education was supposed to correct, even after reading: “We conclude that in the field of public education, separate but equal has no place, separate educational facilities are inherently unequal.”

While some questions are multiple choice, others require students to write a short response. Frequently, they are asked to interpret maps, pictures, posters, graphs, original documents and quotations, such as explaining the historical context of a slave letter, using a map to explain the purpose of the Lewis and Clark expedition or analyzing a graph of the declining number of farms.

Questions about minorities and women are common: Identify a role of women during the American Revolution. Explain how World War II affected African-Americans’ struggle for civil rights. Study an 1849 picture of a Sioux encampment and identify three ways the Sioux used natural resources.

I had to think about this eighth-grade question:

For centuries, a young man who wanted to learn a craft was apprenticed to a master craftsman who taught him the necessary skills. Why did the apprenticeship system begin to decline in the first half of the 1800′s?

A. The apprenticeship system was considered unsuitable for the increased number of women working outside the home.

B. The growth of the factory system led to a decreased need for skilled labor.

C. Many young men chose to become farmers instead of craftsmen.

D. Craftsmen began to use unskilled immigrant labor in their shops.

(The answer is B.)

You can test yourself at all three grade levels.

Filed Under: Education Tagged With: , ,

Old enough to vote — whatever that is

May 5, 2011 By

Civics knowledge has declined for 12th graders, many of whom are old enough to vote, but climbed for fourth graders, according to the new civics report card from the National Assessment of Educational Progress (NAEP). Eighth-grade scores stayed about the same.

Hispanic students made gains, though the racial/ethnic achievement gap remains large.

Students were asked both multiple-choice and constructed-response questions to measure their knowledge of civic life, politics and government and their understanding of citizens’ role in a democracy and the relationship of the U.S. to other nations.

At grade 4, students who scored at or above the Basic level (77 percent) were likely to identify a method used to select public office holders, students scoring at Proficient (27 percent) could identify a purpose of the U.S. Constitution, and students at Advanced (2 percent) could explain two ways a country could deal with a shared problem.

At grade 8, the 72 percent of students who performed at or above the Basic level were likely to identify a right protected by the First Amendment, the 22 percent who performed at or above the Proficient level could recognize a role performed by the Supreme Court, and the 1 percent who scored at the Advanced level could name two actions that citizens could take to encourage Congress to pass a law.

At grade 12, the 64 percent of students who performed at or above the Basic level were likely to interpret a political cartoon, the 24 percent scoring at or above Proficient could define “melting pot” and argue whether or not the phrase applied to the U.S., and the 4 percent scoring at Advanced could compare U.S. citizenship requirements to those of other countries.

Some of the questions appear to test reading and reasoning skills rather than civics knowledge.

Grade 4 sample question:

Cartoon strip. A boy named Calvin says to his father, "I think it's time we had a new dad around here. When does your term of office expire?" Dad responds, "Sorry, Calvin. I was appointed dad for life." Calvin screams, "For life?! What about a recall vote? What about impeachment?" Dad responds, "There are no provisions for either." Calvin asks, "Did you write this constitution yourself, or what?" Dad replies, "Well, your mom helped some, too."

Calvin and Hobbes © 1986 Watterson. Dist. By Universal Press Syndicate.
Reprinted with permission. All rights reserved.

The child in the cartoon strip above is a six-year-old boy named Calvin. What is the main point of the cartoon?

1. Constitutions have rules about how long someone can stay in office.
2. Families and governments are not run the same way.
3. The term of office for elected and appointed officials is different.
4. Calvin does not know how a constitutional government works.

Grade 8 sample question:

In the United States, which civil right belongs only to American citizens?

  1. Freedom of speech
  2. Freedom of assembly
  3. The right to legal representation if charged with a crime
  4. The right to vote in local, state, and national elections

Grade 12 sample question:

Political cartoon showing a man pulling the handle of a large box labeled “The Kick the Bum Out Ballot Box Company.” A large spring extends from the box with a shoe at its end. The shoe is kicking a man holding a briefcase through the air. The caption underneath the cartoon says, “Still the best congressional term-limiting device.

© Pat Oliphant/Universal Press Syndicate

Which of the following best captures the meaning of the cartoon above?

  1. Voters can limit the term of any member of Congress by simply exercising their right to vote.
  2. Term limits can be put in place only through an amendment to the Constitution.
  3. Term limits are needed to prevent incumbents from staying in office for life.
  4. Voters too often throw good people out of office.

No Child Left Behind-mandated reading and math tests are supposed to be crowding out instruction in civics, science and everything else, notes Quick and the Ed’s Kevin Carey. But NAEP shows a different pattern: NAEP scores are improving or holding steady in elementary and middle school, where there’s lots of  NCLB testing, and falling in high school, where there’s only one NCLB-mandated test. He speculates, “it’s hard to learn civics if you can’t read.”

Filed Under: Education Tagged With: , ,

Test-based accountability: Time to wobble?

April 23, 2011 By

Minority and special-ed students made significant gains once states and then No Child Left Behind began holding schools accountable for their performance, argues Bush adviser Sandy Kress in a New York Daily News op-ed. Kress accuses President Obama of going “wobbly” on  accountability.

Under the framework being proposed for the reform of the law, the administration would require that, unless a school is among the very worst in the nation, it would no longer be required to improve even if it continues to fail its black, Hispanic and other disadvantaged kids. Further, in the case of schools that do not improve, special tutoring and public school choice would no longer be required.

In Fact-checking Sandy Kress, Fordham’s Mike Petrilli argues that nearly all the improvement occurred by 2004, less than two years after NCLB was implemented.

For instance, according to the NAEP, the average reading score for Black 9-year olds rose from 186 in 1999 to 204 in 2008–an increase of 18 points. (At 10 points per grade level that comes close enough to the “two grade levels” of progress Kress claims.) Hispanic 9-year olds increased their average reading scores from 213 in 1999 to 234 in 2008–an increase of 21 points. Fourth-grade students with disabilities increased their reading scores from 167 in 2000 to 189 in 2009.

. . . For Black 9-year-olds, 78 percent of the improvement took place in the five years between 1999 and 2004, compared to 22 percent in the four years between 2004 and 2008. For Hispanic students, 81 percent of the gains occurred between 1999 and 2004, compared to 19 percent between 2004 and 2008. For fourth-grade students with disabilities, 91 percent of the gains occurred in just two years: between 2000 and 2002.

While there’s “plausible evidence to credit accountability-based reforms,” writes Petrilli, NCLB can’t claim much credit since it didn’t start till fall of 2002.

Petrilli thinks the states’ accountability measures boosted student achievement in the late 1990s to early 2000s. NCLB jumped on a moving bandwagon. “To me, the evidence shows that NCLB and test-based accountability had their day in the sun, and made a big difference, but now it’s time to try something else if we want to see progress continue.”

Kress responds in an e-mail with more NAEP data:

Let’s deal with the easy part – gains on the Long Term Trend for students with disabilities (SWDs) and English Language Learners (ELLs) from 2004-2008. This is squarely within NCLB time.

9 year old SWDs improved a half grade level (5 points) in reading.

9 year olds ELLs improved almost a grade level (8 points) in reading.

9 year old SWDs improved over a half grade level (6 points) in math.

9 year old ELLs improved 3 points in math.

13 year old SWDs improved 3 points in math.

13 year old ELLs improved over a half grade level (7 points) in math.

13 year old SWDs improved almost a full grade level (9 points) in reading.

13 year old ELLs improved 2 points in reading.

Now, since Mike is enamored of the Main NDE, let’s look at that data:

For 4th grade math, it is true that SWDs had an incredible jump from 2000 to 2003, from 200 to 216. I don’t want to argue this was due to NCLB, but, since there’s almost a full academic year since the summer of 2002 in this data, I would suggest that this bridge period probably shouldn’t be used for a pre and post analysis.

In any event, 4th grade SWDs have gone up 7 points since 2003, which is a gain of over a half a grade level. 4th grade ELLs had that nice pop in 2003, too, but also have grown an additional half grade level since.

It is incontestable that something unusual happened in NAEP testing between the late 1990s and 2002 and 2003, first a drop and then an unusual increase. I can’t explain it, and I suspect Mike can’t either. I invite thoughts from any and all of you on that topic.

Nevertheless, it is heartening to see that 4th grade ELLs have improved over a half grade level in reading since 2002, and SWDs have improved almost a half grade level as well.

The same pattern of a pop in 2003 occurs in 8th grade math with further gains for SWDs and ELLs after 2003. Reading at the 8th grade level is stagnant.

“Consequential accountability, which began in many states in the mid-1990s and was extended and deepened by NCLB, works!” Kress writes. “Any weakening of its pillars threatens the progress we’ve made.”

Filed Under: Education, Testing Tagged With: , , , , , ,

13% take rigorous classes

April 14, 2011 By

More students are taking rigorous classes and — surprise! — scoring higher on achievement tests, according to the National Assessment of Educational Progress High School Transcript Study.

Thirteen percent of students complete rigorous courses, including high-level math and science, up from only 5 percent in 1990.  Another 46 percent take a “midlevel” curriculum,  increased from 26 percent. Ed Week reports:

The report classifies student coursetaking by three curriculum levels: “standard” (at least four credits of English and three credits each in social studies, math, and science); midlevel (standard requirements plus geometry and Algebra 1 or 2; at least two courses from biology, chemistry, and physics; and at least one credit of a foreign language); and rigorous (all the midlevel requirements plus an additional credit in precalculus or higher math; courses in biology, chemistry, and physics; and at least three credits in a foreign language).

“Rigorous” graduates typically score proficient on NAEP exams, while midlevel and standard graduates score at the basic level.

 . . . The proportion of black graduates attaining a rigorous-curriculum level increased from 2 percent in 1990 to 6 percent in 2009. For Hispanic students, the number rose from 2 percent to 8 percent in the same stretch, and for Asian/Pacific Islanders, it went from 13 percent to 29 percent. Whites, meanwhile, increased from 5 percent to 14 percent.

Blacks and Latinos who take more rigorous courses don’t perform as well on achievement tests as whites and Asians, said Cornelia Orr, executive director of the National Assessment Governing Board, which sets policy for the NAEP.

“It makes me want to question: Are the courses the same?” she said. “Perhaps the pace of instruction is different. There are any number of variables.”

In addition, boys outscore girls in math and science.  

Missing science classes were responsible for keeping many students, especially girls, below the rigorous level.  (As a 1970 graduate, I wouild have been midlevel with no physics and no math past advanced algebra/trig.)

Two-thirds of  “rigorous” graduates took algebra in eighth grade.

In light of the discussion here, I was surprised to learn that 76 percent of high school graduates take Algebra II. That seems high, though the sample excludes those who don’t earn a diploma.  In 1990, 53 percent of graduates had taken Algebra II.

Filed Under: Education Tagged With: , , , , ,

Rhee’s record

April 13, 2011 By

The case against Michelle Rhee is full of holes, writes Paul Peterson of Harvard’s Program on Education Policy and Governance in the Washington Times. Ed Next has his full analysis.

Rhee was more effective than her predecessors, he writes, contradicting a recent study (pdf) by Alan Ginsburg, a former director of Policy and Program Studies in the U.S. Department of Education.  And, contrary to a National Research Council (NRC)  committee’s preliminary analysis, which downplays progress, there’s reason to believe Rhee’s reforms made a difference.  

Like Ginsburg and the NRC committee, Peterson looks at NAEP data, since it’s a low-stakes test with no incentive to cheat. He excludes the scores of charter schools beyond Rhee’s control, which caused a blip in the data in 2007, inflating pre-Rhee progress. He finds progress accelerated after Rhee took over as chancellor.

 Once the data are corrected and adjusted for national trends, it becomes evident that during the Rhee years, fourth-grade students gained at a pace twice that seen under her predecessors in both reading and math. The gains in math by eighth-grade students were nearly as much, although no eighth-grade reading gains are detected.

Gains are not enormous in any one year, but over time, they add up. In 2000, the gap between the District and the nation in fourth-grade math was 34 points. Had students gained as much every year between 2000 and 2009 as they did during the Rhee era, that gap would have been just 7 points in 2009. Three more years of Rhee-like progress and the gap would have been closed. In eighth-grade math, the gap in 2000 was 38 points. Had Rhee-like progress been made over the next nine years, the gap in 2009 would have been just 14 points, with near closure in 2012. In fourth-grade reading, the gap was 30 points in 2003; if Rhee-like gains had taken place over the next six years, the gap in 2009 would have been cut in half.

The NRC committee claims that District gains “were similar” to those in 10 “other urban districts” for which comparable data is available.

In fact, D.C. students gained 6 points between 2007 and 2009 in both math and reading, while the average gain for the other 10 cities was just 1 point in reading and 2 points in math. In eighth-grade math, D.C. gains were 7 points, as compared to an average of three points for 10 other cities. Only in eighth-grade reading did the District lag behind, dropping a point while elsewhere, students gained 2 points.

The committee also admits that student and teacher attendance improved significantly during Rhee’s tenure, but questions the significance of the change.

Rhee said she wanted to change the culture, Peterson notes.  When students show up to learn and teachers show up to teach, that’s considered a very good sign. But Rhee’s enemies don’t want to give her credit for anything.

Filed Under: Education Tagged With: , , , , , ,

Longhorns 17, Badgers 1

March 6, 2011 By

In “low-tax, low-spending Texas, graduation rates are low, writes New York Times columnist Paul Krugman. SAT scores are low in the five states without collective bargaining for teachers, reports The Economist. Texas ranks 47th, while Wisconsin is second.

“The point being, I suppose, is that unionized teachers stand as a thin chalk-stained line keeping Wisconsin from descending into the dystopian non-union educational hellscape of Texas,” writes Iowahawk. Actually, Texas is out-educating Wisconsin, according to the National Assessment of Education Progress, which breaks down test scores by grade, state, subject and ethnicity.

“A state’s ‘average ACT/SAT’ is, for all intents and purposes, a proxy for the percent of white people who live there,” writes Iowahawk, who attributes the test gap to differences in socioeconomic status, racism and family structure. Wisconsin (4% black, 4% Hispanic) will have higher average scores than Texas (12% black, 30% Hispanic). When scores are disaggregated by race and ethnicity, “brokeass, dumbass, redneck Texas” does better than “progressive unionized Wisconsin” for whites and blacks and Hispanics.

2009 4th Grade Math

White students: Texas 254, Wisconsin 250 (national average 248)

Black students: Texas 231, Wisconsin 217 (national 222)

Hispanic students: Texas 233, Wisconsin 228 (national 227)

2009 8th Grade Math

White students: Texas 301, Wisconsin 294 (national 294)

Black students: Texas 272, Wisconsin 254 (national 260)

Hispanic students: Texas 277, Wisconsin 268 (national 260)

2009 4th Grade Reading

White students: Texas 232, Wisconsin 227 (national 229)

Black students: Texas 213, Wisconsin 192 (national 204)

Hispanic students: Texas 210, Wisconsin 202 (national 204)

2009 8th Grade Reading

White students: Texas 273, Wisconsin 271 (national 271)

Black students: Texas 249, Wisconsin 238 (national 245)

Hispanic students: Texas 251, Wisconsin 250 (national 248)

2009 4th Grade Science

White students: Texas 168, Wisconsin 164 (national 162)

Black students: Texas 139, Wisconsin 121 (national 127)

Hispanic students: Wisconsin 138, Texas 136 (national 130)

2009 8th Grade Science

White students: Texas 167, Wisconsin 165 (national 161)

Black students: Texas 133, Wisconsin 120 (national 125)

Hispanic students: Texas 141, Wisconsin 134 (national 131)

Whites, blacks and Hispanics do better in Texas than in Wisconsin in 17 comparisons; Hispanics score insignificantly higher in science in Wisconsin in fourth grade.

Texas students exceeded the national average for their ethnic cohort in all 18 comparisons; Wisconsinites were below the national average in 8, above average in 8.

In addition, the racial achievement gap is much wider in Wisconsin than in Texas.

Non-union Georgia also does well in comparison to Wisconsin, though not as well as Texas, writes Kyle Wingfield.

The Economist’s SAT scores are both out of date and meaningless, writes Angus Johnston. Using current data, Wisconsin ties for 17th on the ACT. Very few Wisconsin students take the SAT. Texas ranks “45th on the SAT with 53% participation, 33rd on the ACT with 33% participation.”

In a follow-up post that serves as a statistics primer, Iowahawk breaks out ACT scores by race and ethnicity for Wisconsin and Texas and explains Simpson’s Paradox.

He also links to Michael Pollard’s NAEP analysis: “After controlling for ethnicity, compared to the running-dog Gang of Five non-collective bargaining states (TX, VA, SC, NC, GA), Wisconsin is a (1) middling performer for white students; (2) below middling for Hispanic students, and (3) an absolute disaster for black students.”

Filed Under: Education Tagged With: , , , , , , , , , ,
« Older Posts
Newer Posts »