Okay, so you’re engaging in a discussion about the quality/effectiveness of the education systems. What’s the point though of engaging in this discussion if you don’t care whether your opinions are right or not? Even in the context of this discussion you are happy to be skeptical about the quality/effectiveness of standardised tests, but oddly not about your own potentially-biased observations. Why not? Why the one-way skepticism?

I made an educated decision to raise and educate my children here because I believe it is the best place to do so…

You are free to do so. However, I do not consider comparing merely two countries’ education systems a valid basis for claiming that the top third of American students are the best in the world. You may choose not to be skeptical about your own beliefs, but I am most certainly skeptical about your beliefs, particularly when you expand them beyond the two systems you know about to the rest of the world.

Or, from the standpoint of Fareed Zakaria in “The Post-American World,” this not about the demise of the US, but about the “rise of the rest.” Certainly, both our universities and corporations are going to seek the best people, and those “best” won’t always be Americans. In fact, it would be rather ego-centric to think so, though many countries and cultures often have. American society it simply more open to opportunities for all. Additionally, I argued that American companies “predominantly” employ Americans, and that is, of course, true. Tracy W. pointed out that I have implied, or even stated, that Americans students are “best” across the board, though I merely meant to imply our best can and do compete with everyone else’s “best.” Perhaps I’ve overstated my optimism about our top students, but that only qualifies overstating the significance of American students trailing other countries’ students in international standardized tests.

Hmm. Want to try again? Because that statement just isn’t true. Look into the public debate on the need for HB-1 visas. Increasingly, the top students in other countries are the top students in US universities. The US depends upon the K-12 systems of other countries to meet our economy’s need for competent graduates in science and engineering.

“In many fields of science and engineering, foreign students make up the majority of doctorate recipients,” Finn said. “Universities, research labs, and other high-tech employers have become dependent on these scientists and engineers.” (http://www.orau.org/media-center/news-releases/2008/fy08-16.aspx)

This is not an endorsement of a wonderful, creative US system. To me, this means that many students are not adequately prepared after high school to complete an S&E degree at the college level, nor to pursue a graduate degree in science & engineering.

Because I am not in the midst of conducting educational research, but merely engaging in a discussion about the quality/effectiveness of the education systems, I don’t seek to control for my biases, though that doesn’t mean I don’t understand or acknowledge them. The criticism of American society and the American system, based on these tests, is, in my opinion, extremely limited, and it overstates the validity of these tests. While you apparently place great significance on these results, I know of no economic models which would do the same.

If there is no “statistical data” you know of to satisfy your concerns, why would you not look outside of this limited measurement. I am quite “satisfied” by the quality of education my children and students receive. So are 3/4 of Americans who, when polled, are “satisfied” or “very satisfied” with their children’s schools. Additionally, 80% of Americans are satisfied with their own education. If I weren’t I would make a change. Not being averse to living, teaching, and raising my children in the U.S. or abroad, I made an educated decision to raise and educate my children here because I believe it is the best place to do so. Certainly, some of that is cultural, but that bias works both ways.

What I am curious about is how *you* could be satisfied making the claim:

“our top third are still the best in the world

without using any statistical evidence at all, and only based on an anecdotal comparison of two countries, without you using any methods to control for your possible biases.

I know that there’s not the sort of statistical evidence that would satisfy me, but how can you be satisfied without any either?

Additionally, you argue that America’s greatest assets might be indicative of cultural, socio-economic factors, and despite our education system. That is absolutely valid, but you can’t definitively parse it out, and it is indicative of the strengths of American students. The areas in which America leads the world – production, business administration, technical innovation, teamwork, critical thinking – are reflective of American society, which includes (but cannot be exclusively linked to) its education system. Most European and Asian systems, which rank higher than we, do not, at the high school level, offer the diversity of extra-curricular activities – athletics, clubs, music, student government, internships – that develop the qualities which lead to our strengths in innovation. Therein lies much of your difference, which belies the validity of using these tests to compare systems.

As you noted, there are many ways to measure “best” (GDP, production, health, investment, “happiness,” etc.). The evidence you seek is American society – production, technology, medicine, engineering, global leadership, citizenship, and social/civic stability.

Neither of us have defined “best in the world”, so I wouldn’t say we have gotten to the stage of disagreement. All I have been doing is casting doubt on your assertion that the top third of American students are the best in the world.

In terms of “best in the world,” I would simply argue that our “top” students are as effectively serving our society as the Japanese and Hungarian and English and Taiwanese are serving theirs.

Okay then, argue that. Present your evidence.

I’m simply pointing out that American companies still “hold their own” and even dominate many areas

You have set yourself another problem here. If American companies are “holding their own” this may be for reasons entirely independent of the education system, such as the USA’s system of business law, or accidents of history (as I stated earlier, Nazi persecution leading to an exodus of many of the best scientific minds in Europe to the USA). You haven’t supplied any evidence that the performance of American business is because of the performance of American schools with the top third of students.

I guess I might ask what your criteria are for determining that these tests/rankings are indicative of anything other than the ability to do the test.

The assessment framework developed for the 2007 TIMSS is available at their website.

The tests on creativity are fascinating, though I don’t see them being used currently in the type of ratings and rankings which prompted this discussion.

This may be because you didn’t use any rankings in your initial statement in this discussion. I used the TIMSS to question your assertion because it was conveniently to hand. I don’t know of any sources of international standardised tests of creativity, so I don’t think we can say anything one way or another about the relative international performance of American students in terms of creativity.

I don’t put our “top third” on top of any others, but simply note that it’s possible they only thing trail in is the test ranking.

Michael Mazenko, to quote you:

Our top third are still the best in the world, and the United States is still leading the world in innovation.

You most definitely did put the USA’s “top third” on top of any others.

It may be possible that the only thing that American students trail the world in is test ranking. All sorts of things are possible. I note however that American students were about the top 10 or 11 in the eighth grade tests I linked to, well above the average of the countries’ tested, which does cast some doubt on this possiblity you describe here.

I see your point, though it seems we simply disagree on what “best in the world” means and how these tests measure it. Are they the best students? Are they the best test takers? Are they the fastest readers? The best at math with a calculator? The best at math without a calculator?

The more important question, for me, is will they become the best doctors and lawyers and engineers and scientists and producers and programmers and technicians and historians and writers and musicians and humanitarians and citizens. I don’t believe the international tests are great predictors of this. The market/workplace is the predictor of this, and I’m simply pointing out that American companies still “hold their own” and even dominate many areas, and they predominantly employ American citizens who were educated in American schools. That’s the comparison I am making – it’s one outside of the test. If our students perform poorly on these tests in comparison, is there a correlation to a negative impact on American business and society. I’m not seeing it. Where is the impact?

The tests on creativity are fascinating, though I don’t see them being used currently in the type of ratings and rankings which prompted this discussion. In terms of “best in the world,” I would simply argue that our “top” students are as effectively serving our society as the Japanese and Hungarian and English and Taiwanese are serving theirs. I don’t put our “top third” on top of any others, but simply note that it’s possible they only thing trail in is the test ranking.

I completely agree that their “rankings” should be a concern, just not the primary concern. I guess I might ask what your criteria are for determining that these tests/rankings are indicative of anything other than the ability to do the test.

As for standardised tests, if you want to use success-in-life as your basis for international comparison, then to do so you need some standardised test of what success-in-life is. Perhaps GDP per capita, or happiness on those global surveys, or rate of mental illnesses, or some combination of all of those and other things. People have attempted to construct many standardised tests aimed at measuring sucess-in-life, for example the Human Development Index. I am in favour of being skeptical about every standardised test, but if you don’t use some sort of standardised test you can’t make any sort of reliable international comparison. That’s why I asked if you are crazy.

American students can go to a four-year college after graduating with a D average in high school, or not graduating at all, and performing below average on ACT/SAT. That is true nowhere else in the industrialized world.

You use this to argue that American students will do worse on standardised tests. Okay. But this argument also implies that American students on average will do worse at their schoolwork overall – they lack the same incentive. So therefore this argument undermines your claim that the top third of American students are the best in the world.

I also note that in the link I provided to the maths international comparison, only 6% of American students made the top level on the test. Do only 6% of American students care about getting into selective high schools?

The people they are looking for are the ones who “ask the good questions” and use their “imagination to solve problems.” That is not quantifiable by a test.

Then the school system shouldn’t waste its time teaching it. If we can’t test the difference between someone who has been educated to “ask the good questions” and someone who hasn’t, or between someone who is creative, and someone who isn’t, it’s foolish to waste money on teaching it. Spend the time on something where schools can make a detectable difference.

However, I don’t actually believe your assertion that creativity or question-asking is not quantifiable. Options for creativity testing – give students a box and ask them to write down as many different uses for it as possible. Next item, give them a variety of shapes and ask them to draw an object including as many of those shapes as possible. Etc. Test on a test population to develop inter-marker reliability.
Want to test question-asking? Set up a set of scenarios and ask students to write down as many questions as occurr to them. Use experts in the fields related to those scenarios to assess the questions – allow them to discuss anomalous results. Develop from this a standardised marking protocol.
People have already made standardised tests of creativity.

Ouch!

While I don’t put extensive faith in the significance of international standardized test rankings, I certainly wouldn’t say I have no faith in or use for the tests. In fact, in my district and state, I am generally one of the more vocal defenders of testing – CSAP and NCLB, as well as AP/ACT/SAT/PSAT. And I do apply my skepticism to my claims about the top third, knowing many of my AP students arrive ready to get a 4 on the exam, though I am pretty effective at giving my borderline students an edge. Additionally, I wasn’t conducting an experiment to establish a position on these rankings – I was using anecdotal evidence to provide perspective on the issue.

I can only speak personally of two systems, as I did, and then I use that perspective to evaluate the rest. There are numerous other variables to consider, such as the restrictive nature of higher education in Europe and Asia. The English – and, in fact, many European schools – use these tests as gates to higher education. They have far greater incentive to do well. That’s much less true in the US. American students can go to a four-year college after graduating with a D average in high school, or not graduating at all, and performing below average on ACT/SAT. That is true nowhere else in the industrialized world. Thus, the stakes on these tests are much different for US and foreign students. As I said, if the ACT/SAT forbid calculator usage on their tests, it wouldn’t take long for US students to catch up to that expectation. But to what end? To do better on a test and a ranking? How does it correlate to success in the marketplace?

The foundation of my skepticism about the test rankings is the increasingly dubious correlation between those tests/rankings and success in work/life. And to answer the question, no, that’s not “crazy.” Colleges and businesses are seeing less correlation between high standardized test scores and actual success in college and the workplace. I don’t have any article titles at the point, but I could look some up. Of course, the top tier of test takers are most likely assured of success. When I point to the top third, I’m also noting that America currently has about 29% of its population with a bachelor degree. However, I continue to read anecdotes like that at the start of Wagner’s “The Global Achievement Gap,” in which he conveys the comments of CEOs who argue they will teach “the skills” they want their workers to have. The people they are looking for are the ones who “ask the good questions” and use their “imagination to solve problems.” That is not quantifiable by a test. Many CEOs have spoken of their lack of interest in what their candidates learned in college. They just want to know what college they got into and that they graduated. Thus, the skill on the ACT math section is of little considerable value or interest to many employers.

That difference is to what I attribute my skepticism about the rankings. The ultimate test, at least from my point of view, is what I see happening in the marketplace. Where is the innovation and the progress? That quality, which is omnipresent and foundational, in the US, is not at risk – at least not from what I hear and read of corporate America. The tests matter to me – they really do – but I am as interested by what’s happening at our engineering firms and NIH and CDC and NASA and IBM and Google, etc.