Wednesday, April 28, 2010

The Liberty Times Should Publish Its Raw Polling Data

The Liberty Times Should Publish Its Raw Polling Data
China Times editorial (Taipei, Taiwan, ROC)
A Translation
April 28, 2010

Ruling and opposition leaders recently held a debate on ECFA. The next day a number of media organizations published their poll results. Their results were nearly identical, differeing by at most one or two percentage points. The overall picture was consistent. A majority of the public felt President Ma Ying-jeou fared better in the debate than Chairman Tsai Ing-wen. Public understanding of ECFA and public support for ECFA showed substantial increases. Only the Liberty Times' poll results differed. Once in a blue moon, media polls will result in large discrepancies. But the Liberty Times published its numbers on the front page, in banner headlines, in a clear violation of professional conventions, not to mention professional ethics. The Liberty Times should immediately publish the raw data behind its recent poll results, allowing neutral parties to verify its results, and confirm its credibility. Otherwise it will be misleading or deceiving its readers.

Polls are an important part of contemporary democratic society. Using objective data, citizens can track public support for their political leaders and their policy proposals. The numbers will speak for themselves. This is why politicians in democratic nations dare not ignore the polls. They believe polls are accurate reflections of public opinion. That is why polls must reflect public opinion in fact, and not just in name. That is why polls must be conducted in accordance with strict professional procedures. That is why they must be scientific and free from bias, to prevent special interests from misusing them or even perpetrating frauds. Put simply, a polling organization can survive only if it maintains its credibility.

Every modern polling association, including the American Association for Public Opinion Research (AAPOR), the National Opinion Polls Council (NCPP), and the World Association for Public Opinion Research (WAPOR) has strict standards for poll results released to the media. News reports on poll results must include the sponsor of the poll, the conductor of the poll, a complete list of the poll questions, the poll sample, the sampling methods used, the sample size, the response rates, the polling technique, the time frame, the margin of error, and the sample weighting. The poll published on the front page of the Liberty Times on the 26th of this month violated virtually every one of these standards for professional procedures and professional ethics.

A polling organization will normally conduct over a hundred polls a year. The Liberty Times "Polling Center" has conducted only three polls over the past five years. Each of these polls was conducted at a critical juncture, for example, just before an election. Each time its results were the exact opposite of all other polling organizations. Why did the Liberty Times set up its "Polling Center" only at these critical junctures? When exactly did the Liberty Times set up its "Polling Center?" How many polls has it conducted over the years? Who was in charge? Who were its key members? What kind of polling expertise and polling experience did they have? Outsiders have no idea. Professional pollsters would also like to know. After all, the poll included at least 10 questions. It was conducted on a Sunday, when finding respondents is the most difficult. It was conducted between 5:00pm and 8:30pm, at dinner time. In order to poll at least 1,300 individuals, one needs at least 50 pollsters. We would like to know just exactly where the Liberty Times "Polling Center" is located. What kind of telephone equipment does it have? How large is it? How many pollsters does it have in its employ? How much polling experience do they have? These are concrete and specific questions concerning manpower and equipment. The Liberty Times should have no difficulty answering these questions in order to establish its bona fides to a skeptical public.

We of course are curious why the Liberty Times poll results diverged so drastically from those reached by other media organizations. We are not implying that the results were faked. What we want to know is why the Liberty Times chose not to publish the poll questions? Isn't this the most basic requirement for any polling organization when it publishes polling data? If the poll questions were omitted due to space considerations, can't they provide a copy of the questionnaire? Can't they tell us what questions were asked, in what order, to establish their credibility? It is bad enough the poll questions are nowhere to be found. What's worse is that the Liberty Times has raised so many questions with its "news poll," yet it is choosing not to provide any of the procedural details. For example, what was the structural composition of the respondents? What proportion of the respondents were blue or green camp supporters? How was the poll weighted? How many respondents refused to answer? What was the success rate? These are all "standard operating procedure." If the Liberty Times "Polling Center" is sufficiently professional, how can it omit this information?

Even more interesting was the question of public support for ECFA. As many as 36% answered "don't know / have no opinion," far higher than for other media organizations. As many as 50% of respondents between the ages of 20 and 29 were undecided. As many as 45% of respondents 30 and over were undecided. Meanwhile, only 29% of those over 60 were undecided. Experience and common sense tell us this poll was substandard. Normally the older the respondents, the higher the undecided rate. The results of the Liberty Times poll were just the opposite. Would the Liberty Times care to offer an explanation in response to public doubts?

The best way to allay public doubts is for the Liberty Times "Polling Center" to immediately submit the telephone numbers and recorded telephone conversations, along with the original data, to scholars and professionals, for neutral and objective review, allowing them to confirm their authenticity and reliability. This the very least any "polling organization" can do when its credibility is brought into question. Did the Liberty Times "Polling Center" in fact conduct a poll? If it did, it should have no trouble providing this raw data.

中時電子報 新聞
中國時報  2010.04.28



這亦就是當代所有民調協會組織,包括美國民意研究協會(AAPOR)、全國民意調查評議會(NCPP) 、世界民意研究協會(WAPOR)等,都曾針對媒體在發布民調訂有一套相當嚴格的標準,責求所有公開發布「民調新聞」都必須要完整交待民調的贊助者是誰、執行者是誰、完整問卷題目、調查的母群體及抽樣方法、樣本大小及完成率、施測的方式及時間、調查結果的精確度如抽樣誤差的估計、加權或推估程度等。從上述標準看來,我們必須要說,自由時報在本月二十六日在該報頭版所發布的民調數據,幾乎背離了所有民調發布專業程序與倫理。





No comments: