The Liberty Times Should Publish Its Raw Polling Data

The Liberty Times Should Publish Its Raw Polling Data
China Times editorial (Taipei, Taiwan, ROC)
A Translation
April 28, 2010

Ruling and opposition leaders recently held a debate on ECFA. The next day a number of media organizations published their poll results. Their results were nearly identical, differeing by at most one or two percentage points. The overall picture was consistent. A majority of the public felt President Ma Ying-jeou fared better in the debate than Chairman Tsai Ing-wen. Public understanding of ECFA and public support for ECFA showed substantial increases. Only the Liberty Times' poll results differed. Once in a blue moon, media polls will result in large discrepancies. But the Liberty Times published its numbers on the front page, in banner headlines, in a clear violation of professional conventions, not to mention professional ethics. The Liberty Times should immediately publish the raw data behind its recent poll results, allowing neutral parties to verify its results, and confirm its credibility. Otherwise it will be misleading or deceiving its readers.

Polls are an important part of contemporary democratic society. Using objective data, citizens can track public support for their political leaders and their policy proposals. The numbers will speak for themselves. This is why politicians in democratic nations dare not ignore the polls. They believe polls are accurate reflections of public opinion. That is why polls must reflect public opinion in fact, and not just in name. That is why polls must be conducted in accordance with strict professional procedures. That is why they must be scientific and free from bias, to prevent special interests from misusing them or even perpetrating frauds. Put simply, a polling organization can survive only if it maintains its credibility.

Every modern polling association, including the American Association for Public Opinion Research (AAPOR), the National Opinion Polls Council (NCPP), and the World Association for Public Opinion Research (WAPOR) has strict standards for poll results released to the media. News reports on poll results must include the sponsor of the poll, the conductor of the poll, a complete list of the poll questions, the poll sample, the sampling methods used, the sample size, the response rates, the polling technique, the time frame, the margin of error, and the sample weighting. The poll published on the front page of the Liberty Times on the 26th of this month violated virtually every one of these standards for professional procedures and professional ethics.

A polling organization will normally conduct over a hundred polls a year. The Liberty Times "Polling Center" has conducted only three polls over the past five years. Each of these polls was conducted at a critical juncture, for example, just before an election. Each time its results were the exact opposite of all other polling organizations. Why did the Liberty Times set up its "Polling Center" only at these critical junctures? When exactly did the Liberty Times set up its "Polling Center?" How many polls has it conducted over the years? Who was in charge? Who were its key members? What kind of polling expertise and polling experience did they have? Outsiders have no idea. Professional pollsters would also like to know. After all, the poll included at least 10 questions. It was conducted on a Sunday, when finding respondents is the most difficult. It was conducted between 5:00pm and 8:30pm, at dinner time. In order to poll at least 1,300 individuals, one needs at least 50 pollsters. We would like to know just exactly where the Liberty Times "Polling Center" is located. What kind of telephone equipment does it have? How large is it? How many pollsters does it have in its employ? How much polling experience do they have? These are concrete and specific questions concerning manpower and equipment. The Liberty Times should have no difficulty answering these questions in order to establish its bona fides to a skeptical public.

We of course are curious why the Liberty Times poll results diverged so drastically from those reached by other media organizations. We are not implying that the results were faked. What we want to know is why the Liberty Times chose not to publish the poll questions? Isn't this the most basic requirement for any polling organization when it publishes polling data? If the poll questions were omitted due to space considerations, can't they provide a copy of the questionnaire? Can't they tell us what questions were asked, in what order, to establish their credibility? It is bad enough the poll questions are nowhere to be found. What's worse is that the Liberty Times has raised so many questions with its "news poll," yet it is choosing not to provide any of the procedural details. For example, what was the structural composition of the respondents? What proportion of the respondents were blue or green camp supporters? How was the poll weighted? How many respondents refused to answer? What was the success rate? These are all "standard operating procedure." If the Liberty Times "Polling Center" is sufficiently professional, how can it omit this information?

Even more interesting was the question of public support for ECFA. As many as 36% answered "don't know / have no opinion," far higher than for other media organizations. As many as 50% of respondents between the ages of 20 and 29 were undecided. As many as 45% of respondents 30 and over were undecided. Meanwhile, only 29% of those over 60 were undecided. Experience and common sense tell us this poll was substandard. Normally the older the respondents, the higher the undecided rate. The results of the Liberty Times poll were just the opposite. Would the Liberty Times care to offer an explanation in response to public doubts?

The best way to allay public doubts is for the Liberty Times "Polling Center" to immediately submit the telephone numbers and recorded telephone conversations, along with the original data, to scholars and professionals, for neutral and objective review, allowing them to confirm their authenticity and reliability. This the very least any "polling organization" can do when its credibility is brought into question. Did the Liberty Times "Polling Center" in fact conduct a poll? If it did, it should have no trouble providing this raw data.

中時電子報 新聞
中國時報 　2010.04.28
社論－自由時報應公布民調「原始資料檔」
本報訊

朝野政黨領袖日前就ＥＣＦＡ舉行政策辯論會後，多家媒體次日立即公布相關民調數據，基本上各媒體所反應的民眾意向差別並不大，縱有一兩個百分點落差，總體趨勢倒是一致的，即占多數民眾傾向認為馬總統的表現較蔡英文主席佳，同時對ＥＣＦＡ內容的知曉程度、乃至對ＥＣＦＡ的支持程度，都較以往大幅上升。這中間唯獨自由時報一家媒體所公布的數據，與其它機構完全不同。儘管媒體公布民調出現彼此數據鉅幅落差者不是沒有，但自由時報這篇刊登在頭版的民調新聞，不僅明顯違反諸多民調專業程序，更違反諸多報導民調新聞應有的專業倫理。我們認為該報應立即提供此次民調的「原始資料檔」，供中立機構查核以昭公信，否則就是在誤導讀者甚至欺騙讀者。

誰都知道，民調是當代民主社會的重要機制之一，透過一組客觀的數據，不論是一時一刻民眾對政治領袖的支持程度、對政策取向的偏好程度等，全數都一覽無遺。這亦是為何所有民主國家的政治人物都不敢忽略民調，因為他們相信這就是民意的真實反應。但也因為這樣，民調也被要求必須要反應「真實」的民意；換言之，任何民調的執行都得遵守一套嚴格的專業程序，也都必須堅持科學與誠信的原則，以防其被有心人士私心濫用甚至惡意偽造。講得再直接一點，任何民調機構能夠存活的第一原則，就是「公信力」！

這亦就是當代所有民調協會組織，包括美國民意研究協會（AAPOR）、全國民意調查評議會（NCPP） 、世界民意研究協會（WAPOR）等，都曾針對媒體在發布民調訂有一套相當嚴格的標準，責求所有公開發布「民調新聞」都必須要完整交待民調的贊助者是誰、執行者是誰、完整問卷題目、調查的母群體及抽樣方法、樣本大小及完成率、施測的方式及時間、調查結果的精確度如抽樣誤差的估計、加權或推估程度等。從上述標準看來，我們必須要說，自由時報在本月二十六日在該報頭版所發布的民調數據，幾乎背離了所有民調發布專業程序與倫理。

一個正常運作的民調機構，一年至少也要執行上百次以上的民調，然而自由時報的「民調中心」過去五年有紀錄可查的只發布過三次民調，而且都是選在關鍵時刻（如選前），公布與所有民調機構調查結果完全相反的數據。至於這個只在重要時刻才上班的「民調中心」，究竟何時成立？每年進行幾次民調？負責人為何？主要成員為何？具有何種民調專業與經歷等，外界一直諱莫如深，民調界也一直都很好奇。畢竟一份連同基本資料至少十題的民意調查，在最難找到受訪者的星期天執行，選在五點至八點半又是用餐時間，要完成一千三百多份的民調，最起碼得要有一個五十人以上的訪員規模才能執行，我們很想知道的是：自由時報民調中心的地點在哪裡？電訪用的設備是什麼？規模有多大？訪員有多少人？有多少訪問經驗？這些實體的人力與設備，自由時報對外公布一下以昭公信，應該不困難吧！

當然，我們對自由時報這次發布的民調數據，為何與其他媒體民調落差那麼大感到很好奇！我們並沒有暗示這份民調是否可能做假，我們想追問的是：為什麼不公布完整的問卷題組？這不是任何民調機構公布民調數據時最起碼的前提嗎？如果先前因為版面考量漏登了，可以提供全份問卷、題目的問法與順序以昭公信嗎？讓我們更感到好奇的，還不只是「問卷題目」不見蹤跡，自由時報這則「民調新聞」還有諸多該交待，但卻未交待的程序細節。例如這份調查受訪民眾的樣本結構為何？支持藍綠的受訪者比例各有多少呢？加權方法為何？拒訪率有多少？成功率又有多少呢？這都是所有民調新聞發布的「基本作業程序」，自由時報民調中心如果夠專業，怎麼漏掉了呢？

更耐人尋味的是，對於民眾是否支持ＥＣＦＡ的調查上，這份民調新聞回答「不知道/無意見」的比例竟高達三六％，比其他媒體都還高出許多，同時二十至二十九歲受訪者未表態比例高達五成，三十餘歲者未表態的也有四五％，但六十餘歲的銀髮族，未表態比例只有二九％，明顯與民調實務與常理不合。所有民調均可發現，年齡愈高者未表態的比例就愈高，本次民調卻恰好相反！針對這一點，自由時報可否給個說法，以釋外界的疑問呢？

我們以為，要澄清外界質疑最好的方法，就是請自由時報「民調中心」立即提供成功樣本的電話號碼與電話錄音，以及調查過程的原始資料檔，供中立客觀的學術專業機構進行複查，以確認其數據的真實性與可靠性，這是所有「民調機構」面對質疑時，「昭公信」最起碼的動作，自由時報「民調中心」如果確實做了民調，針對外界種種質疑，公布這些「原始資料」，應該一點都不困難吧！