2009.11.08: Poll of Oklahoma High School Students was likely fabricated
Peace Corps Online:
Directory:
Swaziland:
Special Report: RPCV Journalist Chris Matthews:
Chris Matthews: Newest Stories:
2009.09.18: Chris Matthews says just 23% of High School Students in Oklahoma know the name of the first President of the United States :
2009.11.08: Poll of Oklahoma High School Students was likely fabricated
Poll of Oklahoma High School Students was likely fabricated
"Cannaday's survey however, found his students doing just fine: They answered an average of 7.8 out of the 10 questions correctly. By comparison, the high school students that were purportedly surveyed by Strategic Vision had gotten just 2.8 out of the items correct. 98 percent of the students on Cannaday's survey -- not 23 percent -- knew that George Washington was the first President. 81 percent -- not 14 percent -- knew that Thomas Jefferson had written the Declaration of Independence. 95 percent -- not 43 percent -- knew that the Democrats and Republicans are the major political parties. There was just no comparison between the two."
Poll of Oklahoma High School Students was likely fabricated
11.08.2009
Real Oklahoma Students Ace Citizenship Exam; Strategic Vision Survey Was Likely Fabricated
by Nate Silver @ 7:30 AM
In detailing some of the evidence against Strategic Vision LLC, a pollster I am now almost certain is disreputable and fraudulent, I pointed in particular to a poll that they conducted on behalf of the Oklahoma Council of Public Affairs, an conservative-leaning educational thinktank. The poll purported to show that Oklahoma's high school citizens were deficient in some of the most basic aspects of citizenship. Only 23 percent of them knew that George Washington was the first president, the poll claimed! Just 43 percent knew that the Democrats and Republicans are the two major political parties!
These conclusions seemed dubious to me on their face. Several years ago, at my old consulting job, I participated in a project for the State of Ohio's public schools which involved sitting down in a third or fifth grade classroom for the better part of a day and seeing how the students were learning. Most of these observations took place in poor, post-industrial towns, which were still suffering the effects of the steel mill or the axle plant that had long ago left town. What struck me, most of all, was how smart the kids were, relative to my expectations. These kids might not have been the highest achievers -- but I'm pretty sure that more than 90 percent of them would have known who George Washington was. And these were third and fifth graders.
There were other hints too, that Strategic Vision's poll may have been fake. The scores that Strategic Vision claimed the kids had gotten, for instance, were strangely underdispersed. And they seemed to contradict results from Oklahoma's own standardized testing, which asked much more difficult citizenship questions and found most of the students doing just fine.
It turns out that I was not the only person who had doubts about the survey. So did Ed Cannaday, the State Representative from Oklahoma's 15 House District.
In a telephone interview yesterday, Cannaday told me he was shocked when he heard of the results, which had received widespread media attention. "When I saw the statistics, I was just flabbergasted and said it cannot be true," he told me.
There were two items in particular that sent up warning flags for him: the one claiming that only 23 percent of the students knew the identity of George Washington, and another that claimed that about one in every ten students had listed the two major political parties as "Republican and Communist".
"Given the dialog of today, if they had said Republican and socialist, then maybe," Cannaday told me. "But communist -- that's just not something that you throw out there any more. I don't think Sarah Palin even used that term."
Cannaday, age 69, would be in a position to know. Before entering the State Legislature three years ago, he had spent decades in education, first as a teacher in a large public school in Kenosha, Wisconsin, and then in Oklahoma where he set up an alternative school. After a stint in private business, Cannaday returned to classroom, first as a teacher and then as a principal, and then -- finding he missed the one-on-one interaction with his students -- as a teaching principal at a small school in House District 15. He now serves on the House's education committee in Oklahoma City, and continues to pay regular visits to the schools in his district. "Most schools like to have me once a month," he says, to talk about legislation pending before the state.
Cannaday therefore had little difficulty setting up an experiment: he arranged to have all the seniors in the 10 secondary schools in his district take the Strategic Vision/OCPA survey. Cannaday tried to replicate the Strategic Vision survey to the greatest extent possible. The same exact questions were used, and as in the case of the original survey, the answers were open-ended rather than multiple choice. The survey was administered to a total of 325 seniors, including special education students.
Cannaday's survey however, found his students doing just fine: They answered an average of 7.8 out of the 10 questions correctly. By comparison, the high school students that were purportedly surveyed by Strategic Vision had gotten just 2.8 out of the items correct. 98 percent of the students on Cannaday's survey -- not 23 percent -- knew that George Washington was the first President. 81 percent -- not 14 percent -- knew that Thomas Jefferson had written the Declaration of Independence. 95 percent -- not 43 percent -- knew that the Democrats and Republicans are the major political parties. There was just no comparison between the two.
Cannaday distributed his results via e-mail to the constituents on his mailing list, including Karina Henderson, who published his findings in a dairy at Daily Kos. He also sent hard copies to each of the schools in his district, as well as all of Oklahoma's state legislators. The reaction so far has been entirely positive -- "even from the Republicans," said Cannaday, a Democrat.
Cannaday also sent his results to OCPA, the thinktank that had commissioned the survey, but has yet to receive a response. In October, before the results of Cannaday's survey had surfaced, OCPA had told the Oklahoma Gazzette that they were taking "a closer look at the raw data and the methodology," behind the Strategic Vision survey but were not yet ready to "toss out" the results.
House District 15 is generally quite representative of Oklahoma, especially its Eastern portion, but is somewhat poorer than the state as a whole. "Rural" was the first adjective that came to mind when I asked Cannaday to describe his district -- no town has more than 3,000 people. Most of the residents make their living in the natural gas industry, commute to service-sector jobs in the comparatively large towns of Muskogee, Oklahoma or Fort Smith, Arkansas, or are engaged in what Cannaday calls "cow/calf operations". The five counties that make up the district range from middle-class to impoverished. Haskell County, for instance, where the town of Stigler is located, has one of the highest unemployment rates in the state and one of the largest proportions of its students on free and reduced lunch programs, the preferred benchmark of socioeconomic status in public education. House District 15 has no private schools.
Cannaday is proud of the achievements of his students -- particularly their low drop-out rate, which is about five percent, and their success in the state's mock trial tournaments, where they've frequently finished in the top 5 in the state competing against much larger schools. He has seen his students become doctors, attorneys, optometrists and accountants, he told me. "Any time you can have one of your former students in your district who's on speed dial in Oklahoma City as a physician, that's not too bad," he said.
But the schools in House District 15, which sends 40-50 percent of its students to college and sees 20-25 percent compete it -- are not exceptional in any obvious way. The students at Haskell High School, for instance, received below-average scores in 5 of the 7 categories tested by Oklahoma's standard exam, including in U.S. History.
There is no reason to think, in other words, that the students in House District 15 should have gotten such profoundly superior results to the "students" in Strategic Vision's survey. Nor could Strategic Vision's results have been the result of any sort of mathematical or methodological oddity. Consider their claim that literally none of the 1,000 students they surveyed were able to answer more than 7 of the 10 questions correctly -- lower than the average score achieved in Cannaday's test.
There are, rather, only two possibilities. Either the Strategic Vision survey was entirely fabricated -- or Cannaday's was.
I would put every dollar to my name on Cannaday, who has kept the surveys and is happy to show them to them to anyone who comes asking.
Next week is Celebrate Freedom Week in Oklahoma, with public schools students to be taught from a special curriculum highlighting the Declaration of Independence. "If were going to be pass education reform then we need to be out in the classroom demonstrating it," Cannaday said of his fellow legislators. "I will be in a classroom Friday," he told me. "I enjoy it."
I e-mailed to David E. Johnson, the CEO of Strategic Vision, a draft of this article and asked for any comments. The entirety of his comments were as follows:
"Thank you for the opportunity to respond. Our company did survey the Oklahoma students grades 9-12 for the Oklahoma Council of Public Affairs. Our client has all of the raw data, cross tabs, methodology from the survey."
Johnson did not reply to a second e-mail asking for his interpretation of the substantial difference between his results and those found by Cannaday.
Links to Related Topics (Tags):
Headlines: November, 2009; RPCV Chris Matthews (Swaziland); Figures; Peace Corps Swaziland; Directory of Swaziland RPCVs; Messages and Announcements for Swaziland RPCVs; Journalism; Television; Oklahoma
When this story was posted in April 2010, this was on the front page of PCOL:
Peace Corps Online The Independent News Forum serving Returned Peace Corps Volunteers
| Memo to Incoming Director Williams PCOL has asked five prominent RPCVs and Staff to write a memo on the most important issues facing the Peace Corps today. Issues raised include the independence of the Peace Corps, political appointments at the agency, revitalizing the five-year rule, lowering the ET rate, empowering volunteers, removing financial barriers to service, increasing the agency's budget, reducing costs, and making the Peace Corps bureaucracy more efficient and responsive. Latest: Greetings from Director Williams |
Read the stories and leave your comments.
Some postings on Peace Corps Online are provided to the individual members of this group without permission of the copyright owner for the non-profit purposes of criticism, comment, education, scholarship, and research under the "Fair Use" provisions of U.S. Government copyright laws and they may not be distributed further without permission of the copyright owner. Peace Corps Online does not vouch for the accuracy of the content of the postings, which is the sole responsibility of the copyright holder.
Story Source: 538
This story has been posted in the following forums: : Headlines; Figures; COS - Swaziland; Journalism; Television
PCOL45170
23