2009.11.08: Poll of Oklahoma High School Students was likely fabricated

Peace Corps Online: Directory: Swaziland: Special Report: RPCV Journalist Chris Matthews: Chris Matthews: Newest Stories: 2009.09.18: Chris Matthews says just 23% of High School Students in Oklahoma know the name of the first President of the United States : 2009.11.08: Poll of Oklahoma High School Students was likely fabricated

By Admin1 (admin) (98.188.147.225) on Friday, April 30, 2010 - 10:34 am: Edit Post

Poll of Oklahoma High School Students was likely fabricated

Poll of Oklahoma High School Students was likely fabricated

"Cannaday's survey however, found his students doing just fine: They answered an average of 7.8 out of the 10 questions correctly. By comparison, the high school students that were purportedly surveyed by Strategic Vision had gotten just 2.8 out of the items correct. 98 percent of the students on Cannaday's survey -- not 23 percent -- knew that George Washington was the first President. 81 percent -- not 14 percent -- knew that Thomas Jefferson had written the Declaration of Independence. 95 percent -- not 43 percent -- knew that the Democrats and Republicans are the major political parties. There was just no comparison between the two."

Poll of Oklahoma High School Students was likely fabricated

11.08.2009

Real Oklahoma Students Ace Citizenship Exam; Strategic Vision Survey Was Likely Fabricated

by Nate Silver @ 7:30 AM

In detailing some of the evidence against Strategic Vision LLC, a pollster I am now almost certain is disreputable and fraudulent, I pointed in particular to a poll that they conducted on behalf of the Oklahoma Council of Public Affairs, an conservative-leaning educational thinktank. The poll purported to show that Oklahoma's high school citizens were deficient in some of the most basic aspects of citizenship. Only 23 percent of them knew that George Washington was the first president, the poll claimed! Just 43 percent knew that the Democrats and Republicans are the two major political parties!

These conclusions seemed dubious to me on their face. Several years ago, at my old consulting job, I participated in a project for the State of Ohio's public schools which involved sitting down in a third or fifth grade classroom for the better part of a day and seeing how the students were learning. Most of these observations took place in poor, post-industrial towns, which were still suffering the effects of the steel mill or the axle plant that had long ago left town. What struck me, most of all, was how smart the kids were, relative to my expectations. These kids might not have been the highest achievers -- but I'm pretty sure that more than 90 percent of them would have known who George Washington was. And these were third and fifth graders.

There were other hints too, that Strategic Vision's poll may have been fake. The scores that Strategic Vision claimed the kids had gotten, for instance, were strangely underdispersed. And they seemed to contradict results from Oklahoma's own standardized testing, which asked much more difficult citizenship questions and found most of the students doing just fine.

It turns out that I was not the only person who had doubts about the survey. So did Ed Cannaday, the State Representative from Oklahoma's 15 House District.

In a telephone interview yesterday, Cannaday told me he was shocked when he heard of the results, which had received widespread media attention. "When I saw the statistics, I was just flabbergasted and said it cannot be true," he told me.

There were two items in particular that sent up warning flags for him: the one claiming that only 23 percent of the students knew the identity of George Washington, and another that claimed that about one in every ten students had listed the two major political parties as "Republican and Communist".

"Given the dialog of today, if they had said Republican and socialist, then maybe," Cannaday told me. "But communist -- that's just not something that you throw out there any more. I don't think Sarah Palin even used that term."

Cannaday, age 69, would be in a position to know. Before entering the State Legislature three years ago, he had spent decades in education, first as a teacher in a large public school in Kenosha, Wisconsin, and then in Oklahoma where he set up an alternative school. After a stint in private business, Cannaday returned to classroom, first as a teacher and then as a principal, and then -- finding he missed the one-on-one interaction with his students -- as a teaching principal at a small school in House District 15. He now serves on the House's education committee in Oklahoma City, and continues to pay regular visits to the schools in his district. "Most schools like to have me once a month," he says, to talk about legislation pending before the state.

Cannaday therefore had little difficulty setting up an experiment: he arranged to have all the seniors in the 10 secondary schools in his district take the Strategic Vision/OCPA survey. Cannaday tried to replicate the Strategic Vision survey to the greatest extent possible. The same exact questions were used, and as in the case of the original survey, the answers were open-ended rather than multiple choice. The survey was administered to a total of 325 seniors, including special education students.

Cannaday's survey however, found his students doing just fine: They answered an average of 7.8 out of the 10 questions correctly. By comparison, the high school students that were purportedly surveyed by Strategic Vision had gotten just 2.8 out of the items correct. 98 percent of the students on Cannaday's survey -- not 23 percent -- knew that George Washington was the first President. 81 percent -- not 14 percent -- knew that Thomas Jefferson had written the Declaration of Independence. 95 percent -- not 43 percent -- knew that the Democrats and Republicans are the major political parties. There was just no comparison between the two.



Cannaday distributed his results via e-mail to the constituents on his mailing list, including Karina Henderson, who published his findings in a dairy at Daily Kos. He also sent hard copies to each of the schools in his district, as well as all of Oklahoma's state legislators. The reaction so far has been entirely positive -- "even from the Republicans," said Cannaday, a Democrat.

Cannaday also sent his results to OCPA, the thinktank that had commissioned the survey, but has yet to receive a response. In October, before the results of Cannaday's survey had surfaced, OCPA had told the Oklahoma Gazzette that they were taking "a closer look at the raw data and the methodology," behind the Strategic Vision survey but were not yet ready to "toss out" the results.

House District 15 is generally quite representative of Oklahoma, especially its Eastern portion, but is somewhat poorer than the state as a whole. "Rural" was the first adjective that came to mind when I asked Cannaday to describe his district -- no town has more than 3,000 people. Most of the residents make their living in the natural gas industry, commute to service-sector jobs in the comparatively large towns of Muskogee, Oklahoma or Fort Smith, Arkansas, or are engaged in what Cannaday calls "cow/calf operations". The five counties that make up the district range from middle-class to impoverished. Haskell County, for instance, where the town of Stigler is located, has one of the highest unemployment rates in the state and one of the largest proportions of its students on free and reduced lunch programs, the preferred benchmark of socioeconomic status in public education. House District 15 has no private schools.

Cannaday is proud of the achievements of his students -- particularly their low drop-out rate, which is about five percent, and their success in the state's mock trial tournaments, where they've frequently finished in the top 5 in the state competing against much larger schools. He has seen his students become doctors, attorneys, optometrists and accountants, he told me. "Any time you can have one of your former students in your district who's on speed dial in Oklahoma City as a physician, that's not too bad," he said.

But the schools in House District 15, which sends 40-50 percent of its students to college and sees 20-25 percent compete it -- are not exceptional in any obvious way. The students at Haskell High School, for instance, received below-average scores in 5 of the 7 categories tested by Oklahoma's standard exam, including in U.S. History.

There is no reason to think, in other words, that the students in House District 15 should have gotten such profoundly superior results to the "students" in Strategic Vision's survey. Nor could Strategic Vision's results have been the result of any sort of mathematical or methodological oddity. Consider their claim that literally none of the 1,000 students they surveyed were able to answer more than 7 of the 10 questions correctly -- lower than the average score achieved in Cannaday's test.

There are, rather, only two possibilities. Either the Strategic Vision survey was entirely fabricated -- or Cannaday's was.

I would put every dollar to my name on Cannaday, who has kept the surveys and is happy to show them to them to anyone who comes asking.

Next week is Celebrate Freedom Week in Oklahoma, with public schools students to be taught from a special curriculum highlighting the Declaration of Independence. "If were going to be pass education reform then we need to be out in the classroom demonstrating it," Cannaday said of his fellow legislators. "I will be in a classroom Friday," he told me. "I enjoy it."

I e-mailed to David E. Johnson, the CEO of Strategic Vision, a draft of this article and asked for any comments. The entirety of his comments were as follows:

"Thank you for the opportunity to respond. Our company did survey the Oklahoma students grades 9-12 for the Oklahoma Council of Public Affairs. Our client has all of the raw data, cross tabs, methodology from the survey."

Johnson did not reply to a second e-mail asking for his interpretation of the substantial difference between his results and those found by Cannaday.




Links to Related Topics (Tags):

Headlines: November, 2009; RPCV Chris Matthews (Swaziland); Figures; Peace Corps Swaziland; Directory of Swaziland RPCVs; Messages and Announcements for Swaziland RPCVs; Journalism; Television; Oklahoma





When this story was posted in April 2010, this was on the front page of PCOL:




Peace Corps Online The Independent News Forum serving Returned Peace Corps Volunteers RSS Feed

 Site Index Search PCOL with Google Contact PCOL Recent Posts Bulletin Board Open Discussion RPCV Directory Register

Feb 10, 2010: Senator Dodd to Retire Date: February 19 2010 No: 1433 Feb 10, 2010: Senator Dodd to Retire
Dodd retires from Senate 6 Jan
Cameron Hume named US Ambassador to Pakistan 8 Feb
Florida RPCVs sponsor Everglades Experience 6 Feb
Jeff Hall brings aid to Sierra Leone 1 Feb
Peace Corps to reach 11,000 PCVs in 2016 1 Feb
Hugh Pickens writes: Standing Bear Looks to the Future 27 Jan
Ann Varghese survives 55 hours in Haiti rubble 26 Jan
John Guy LaPlante at 80 was oldest PCV 17 Jan
Steve Radelet to advise Hilary Clinton on Development 15 Jan
Obituary for Co-Author of ‘The Ugly American' 14 Jan
Peace Corps Establishes Program in Indonesia 11 Dec
What Happened to Obama's Promise? 3 Dec
George Packer writes: Obama's Troubles 24 Nov
PC Mourns Loss of Morocco PCV So-Youn Kim 17 Nov
Peace Corps volunteers return to Madagascar 16 Nov
PC to grow by several thousand over next 2 years 15 Nov
Former Hostage John Limbert named to Iran Bureau 11 Nov
Carrie Hessler Radelet named PC Deputy Director 9 Nov
Garamendi Sworn into Congress 9 Nov
Jesse Lonergan writes graphic novel "Joe and Azat" 4 Nov
David Macaray writes: Hearts and Minds in Afghanistan 29 Oct
Dustin Hogenson writes: Sauna in Kazakstan 26 Oct


Memo to Incoming Director Williams Date: August 24 2009 No: 1419 Memo to Incoming Director Williams
PCOL has asked five prominent RPCVs and Staff to write a memo on the most important issues facing the Peace Corps today. Issues raised include the independence of the Peace Corps, political appointments at the agency, revitalizing the five-year rule, lowering the ET rate, empowering volunteers, removing financial barriers to service, increasing the agency's budget, reducing costs, and making the Peace Corps bureaucracy more efficient and responsive. Latest: Greetings from Director Williams

Join Us Mr. President! Date: June 26 2009 No: 1380 Join Us Mr. President!
"We will double the size of the Peace Corps by its 50th anniversary in 2011. And we'll reach out to other nations to engage their young people in similar programs, so that we work side by side to take on the common challenges that confront all humanity," said Barack Obama during his campaign. Returned Volunteers rally and and march to the White House to support a bold new Peace Corps for a new age. Latest: Senator Dodd introduces Peace Corps Improvement and Expansion Act of 2009 .

Meet Aaron Williams - Our Next Director Date: July 30 2009 No: 1411 Meet Aaron Williams - Our Next Director
Senator Dodd's Senate Subcommittee held confirmation hearings for Aaron Williams to become the 18th Peace Corps Director. "It's exciting to have a nominee who served in the Peace Corps and also has experience in international development and management," said Dodd as he put Williams on the fast track to be confirmed by the full Senate before the August recess. Read our exclusive coverage of the hearings and our biography of Peace Corps Director Aaron Williams.



Read the stories and leave your comments.








Some postings on Peace Corps Online are provided to the individual members of this group without permission of the copyright owner for the non-profit purposes of criticism, comment, education, scholarship, and research under the "Fair Use" provisions of U.S. Government copyright laws and they may not be distributed further without permission of the copyright owner. Peace Corps Online does not vouch for the accuracy of the content of the postings, which is the sole responsibility of the copyright holder.

Story Source: 538

This story has been posted in the following forums: : Headlines; Figures; COS - Swaziland; Journalism; Television

PCOL45170
23


Add a Message


This is a public posting area. Enter your username and password if you have an account. Otherwise, enter your full name as your username and leave the password blank. Your e-mail address is optional.
Username:  
Password:
E-mail: