TECHNOLOGY AND CITIZENS AN ANALYSIS OF CITIZENS' JURY ON THE KOREAN NATIONAL PANDEMIC RESPONSE SYSTEM Abstract Due to technical complexity, most public policies in technological society are dominated by expert-centrism and technocracy (an institutional form of expert-centrism), based on the belief that they should be the exclusive realm of technical experts. But globally, expert-led and technocratic policy-making culture is faced with challenges. We analyse the democratic implications of the Korean experience of the citizens'jury, a form of citizens' deliberative participation. We document and examine the citizens' jury on the National Pandemic Response System in 2008, which was the first case of the citizens' jury in Korea. We conclude that such characteristics of citizens' jury present positive implications in realising deliberative democracy. YOUNG HEE LEE DAL YONG JIN Young Hee Lee is Professor of Sociology at the Catholic University of Korea; e-mail: leeyoung@catholic.ac.kr. Dal Yong Jin is Associate Professor of Communication at Simon Fraser University; e-mail: yongjin23@gmail.com. CO m m rN £± cp rn o o rN rN S A 3 a 0 £ * 1 * (fl 0 c ! m fN Introduction In the 21st century, technological development has been occurring at such a phenomenal rate that citizens and governments are having difficulty in understanding and coping with the changes. As Ellul (1964) already pointed out, technologies have a significant impact on people and society, and citizens have given up control over human affairs to technology and technological imperative. People feel bewildered, alienated and disempowered because their social world seems to be constructed outside their control on issues such as food irradiation, animal biotechnology, and infertility (Davison et al 1997; McKenna and Kasteren 2006). Public policies that are directly related to the citizens' social lives also become more technical, because both the content and means of public policies are technically specialised, and public policies are decided by experts and technocrats with special knowledge. The decision-making process in contemporary society, therefore, has been almost monopolised by experts and technocrats with no assurance of participation by citizens, even on issues that have a direct impact on people's lives. Such practice has been backed by the argument that only the experts are capable of understanding the technical content of public policies, as well as the technocratic justification that ordinary citizens without expert knowledge have neither the capability nor the qualifications to take part in such decision-making processes. However, an expert-led and technocratic policy-making culture is faced with challenges. In many countries, such as the U.S., Canada, and Australia, attempts are being made to expand the room for participation by ordinary citizens in the policy-making process. The types of such attempts are varied, and they include participatory actions similar to social movements such as protests, rallies and picketing, as well as institutional participation such as round tables, consensus conferences and citizens' jury. In Korea, in which technological breakthroughs happening are remarkable, there have been a number of attempts at citizens' participation in technical public issues in the past few years. On the one hand, there are several examples of social movement-type participation, including anti-nuclear, anti-GMO (genetically modified organism), and candlelight vigils against the import of US beef.1 On the other hand, cases of institutional participation are membership in government commissions and deliberative civic engagement such as in consensus conferences or citizens' jury. While social movement-type participation is a highly important form of citizens' involvement, in this article, we examine institutional participation and deliberative civic engagement. Of the different forms of deliberative civic engagement, we analyse the citizens' jury that was newly attempted by the Centre for Democracy in Science and Tech- 3 a map out whether participants in the citizens' jury facilitate deliberate democracy with the case of the 2008 National Pandemic Response System (NPRS). We examine whether citizens' jury based on stratified random sampling represents ordinary people's voices. In order to do this, we discuss the outcomes of the recommendations made by the citizen's jury and the group process of the citizen's jury. Finally, we investigate whether citizens' jury as a form of deliberative democracy is effective ^ in resolving the gap between technocracy and citizenry. Emergence of the Theory of Technological Citizenship and Deliberative Civic Engagement Technological citizenship mainly refers to the right to be enjoyed by members of a society in relation to the determination of science and technology policies in a technological society (Frankenfeld 1992). The conventional concept of citizenship established since the people's revolution that ushered in the modern society is based on the basic rights in civic life, such as entitlements, participation, and status within certain realms governed by the state, that individuals should be able to enjoy as members of a society. The concept of technological citizenship is based on the importance of technology in our society. As technology wields an enormous influence over most of a society's members, it is more urgently necessary than ever to subject the orientation and content of technological development to democratic control based on citizens' participation. Thus it can be seen as an extension of the conventional concept of citizenship, better suited to our technological society. According to Frankenfeld (1992), technological citizenship is made up of four rights: rights to knowledge or information; rights to participation; rights to guarantees of informed consent; and rights to the limitation on the total amount of endangerment of collectivities and individuals. These components are obviously interlinked, but the most important is to allow citizens to participate in important public policy-making processes on technological issues to influence the direction of technological development to be more democratic. In this light, compared to ordinary citizens, the right of access to knowledge or information is a sub-category of the right to participate in the technological policy-making process, the right to argue that decision-making be based on consensus which serves as the basis to the right to participate, and the right to limit the risk of endangering groups or individuals which is implicit in the intended goal of the right to participate. The theory of technological citizenship can be traced back to the Frankfurt school scholars such as Herbert Marcuse who criticised the "one-dimensional man" in developed industrialised societies (Marcuse 1964), or Jürgen Habermas (1968) who feared "colonisation of life world by systems." But more directly, STS (science, technology and society) scholars who study the interactions between science/technology and the society in a more practical perspective have developed a theory regarding the attempts at democratisation of technology through civic participation that had spread in the West since the 1960s.2 There have been various methods of citizens' participation in technological policy-making. It can be participation by ordinary citizens or by NGO (non-governmental organisation) representatives. It can be a simple and instant collection of participants' preferences (like a poll) or participants' determination of preferences after a long period of deliberation. The methods of institutional participation can be grouped into the following four categories depending on the participant (ordinary citizens vs. elite citizens (NGO leaders)) and mechanism of participation (preference gathering vs. deliberation) (Table 1).3 Of course, the institutional participation can be categorised by diverse standards; however, we select and develop only four types of participation addressed in Table 1, because we make a distinction of the civic jury system from the general forms of civic participation (A, B and D in Table 1). In addition, in relation to technological citizens, in type B and D, only elite citizens, not ordinary citizens, participate in the process; therefore, their practice of technological citizenship is limited. The type A represents ordinary citizens' participation; however, it relies on instantaneous gathering instead of deliberation; therefore, it is limited in informed consent. Contrast to this, both consensus conference and citizen's jury reflect technological citizenship's characteristics addressed above because participants in these two types are ordinary citizens and they participate in the process based on impartial information and enough deliberation (Lee 2009). Table 1: Methods of Institutional Participation Ordinary citizens Elite citizens Preference gathering A (polling, voting) B (public hearing, National Assembly hearing, polling) Deliberation C (consensus conference, citizens' jury) D (round table) For example, NGO leaders' participation in public hearings, committees and round tables (B and D) and polling and voting by ordinary citizens (A) are all types of institutionalised participation. NGO leaders' participation, although it is meaningful itself, cannot be called citizens' participation in the sense as it is participation as elite citizens. Meanwhile, polling and voting have the major limitation of merely collecting citizens' preferences at a certain point in time, although they have the advantage of encouraging participation in a large number.4 Due to these limitations, citizens' participation in deliberation (C) has recently gained more importance (Lee 2009). Deliberation is a dynamic process where participants can exchange their judgments, preferences, and perspectives through learning, discussion, and self-reflections. One of its biggest characteristics is that changes in preferences occur through persuasion and mutual learning based on debates and discussions, not through coercion, threat, image manipulation or deceit (Cho 2006). Thus citizens' participation through deliberation is clearly different from participation through voting or polling, which is intended to collect static preferences at a certain point in time. Deliberative democracy here is a theory on democracy that focuses on the possibility to expand and deepen democracy by going beyond the boundaries of representative democracy and participatory democracy through citizens' participation based on deliberation (Elster 1998; Dryzek 2000; Jeong 2005; Oh 2007; Isabel 2011). Under such a backdrop, when discussing technical issues it would be more advisable to use the deliberative process to ensure meaningful participation by non-expert ordinary citizens as it is preceded by supply of balanced information, learning, and pondering. This is the case primarily because it will not be easy for citizens to immediately determine their preferences on the issues presented to them for discussion as they would involve technical details not familiar to them. Therefore, participation in deliberation would be a more desirable format of citizens' participation, at least for technical issues, than conventional methods, including preference gathering. A Citizens' Jury as Deliberative Civic Engagement The two best-known types of citizens' participation through deliberation are consensus conference and citizens' jury. Developed in Denmark in the late 1980s, U S 3 a 0 £ 1 * (I) 0 c I KD fN consensus conference is defined as a forum where a selected group of laypersons unify their opinion regarding scientific or technological issues that are socially controversial or of social interest after raising questions to and heeding the answers ^ from experts and then announce such opinions through a press conference (Joss and Durant 1995; Einsiedel and Eastlick 2000; Goven 2003; Nishizawa 2005; Seifert 2006; Fan 2013). In other words, "the consensus conference involves the formation of a small panel of persons drawn from the general public, followed by a number of briefing weekends on issues raised by the technology being investigated, and culminating in a public conference at which the lay panel is able to control the agenda and to interrogate various invited experts" (Davison et al 1997, 339). The participants would be provided with knowledge and information on the subject matter through such formats as documents or expert lectures, based on which they would go through learning and discussions to judge the different expert views presented to them. They would finally present a list of actions in the name of citizens that should be undertaken by the government. The process concludes with the preparation of a report by the lay panel (Davison et al 1997). However, the consensus conference typically shows its problems as deliberative civic engagement due to its unstructured selection process of a lay panel of around 15 members. Typically, the consensus conference selects participants out of the citizens volunteering after seeing the advertisement in newspapers, as Einsiedel and Eastlick (2000, 330) explained with the issue of food biotechnology in the Canadian context. Of course, the consensus conference is able to randomly choose participants by stratified random sampling. For example, the consensus conference, which was organised by Danish Board of Technology, randomly selected citizen panels. However, existing consensus conferences in several countries, including the U.S., Canada, and Japan, selected citizen panels among volunteers who applied after seeing the advertisement in newspapers. In Korea, the consensus conference has become known since it was attempted by the CDST in 1998 (GMO), 1999 (cloning technology), and 2004 (nuclear energy). It was then used as a model for the Open Citizens' Forum implemented by the Korean Institute of Science and Technology Evaluation and Planning (KISTEP) in 2006 and 2007, as a form of participatory technological impact assessment (Lee 2009). However, as in many previous cases in other countries, most of them selected the participants from volunteers using newspapers. What is problematic the most is that if volunteers are recruited through newspaper advertisement, such self-selection would result in participation by only those who are interested in the subject matter, not ordinary citizens. As Rowe and Frewer (2000) argue, public participants in consensus conferences must have no knowledge on the topic; however, the volunteering process of consensus conference cannot prevent false volunteers from participating in the deliberation process. The selection process in several consensus conferences held in Korea did not secure both representation and ordinary citizens' views. Although the conclusion from such gathering cannot necessarily be an appropriate representation of lay citizens' views, it could be a fatal limitation as in some cases when socially delicate issues are addressed, those with interests in the issues could volunteer for the citizens' jury while hiding their purpose or intent. Such a case of false volunteers actually happened in 2004 in the consensus conference on nuclear power generation. A member of a housewives group in favour of nuclear energy volunteered for a citizens' panel without disclosing her membership in the group. The fact was discovered during the interview process. It was a lucky case of prior discovery, but such a problem of false volunteers remains as long as citizen panels are organised with volunteers. Consequently, Korea has recently developed the citizens' jury, which is a systematic program designed in the early 1970s by the Jefferson Center (2004) - an American non-profit organisation - to ensure citizens' participation in public policies. The citizens' jury is a form of deliberative democracy that allows a small panel of participants drawn from the general public to investigate, and make determinations about, significant social issues, including technological ones (McKenna and Kasteren 2006; Tutui 2011). Citizens' jury is different from consensus conference in that the participants are randomly selected, and the modalities of opinion collection and presentation well illustrate the differences and non-alignment between the participating citizens. The jury consists of around 15 members who are randomly selected, who then work on behalf of ordinary citizens. The deliberation process is also another key factor that Korea has widely begun to use the citizens' jury over the consensus conference. In the citizens' jury, it is held as a process of careful deliberation by a group of randomly selected citizens on publicly important issues for four-five days. The citizen jurors receive some compensation for their participation as they listen to testimonies by expert witnesses, discuss and deliberate on the possible solutions. Expert testimonies offer varying perspectives and arguments, and the jurors participate in a testimonial process conducted as a question-and-answer session. It is important that the testimonies be designed to reach a balance between contrasting opinions to address all the relevant aspects of the issue in a fair manner. The final opinion of the citizens' jury produced after such process would be submitted as non-binding policy recommendations (Smith and Wales 2000).5 Korea also selects the citizens' jury due to the advantages in the method of finalising the participants' opinions. At the consensus conference, the citizen panel y generally gets together for a meeting on the night before the last day and the members write the report themselves. But at the citizens' jury, the jury vote on a list of opinions compiled during their deliberation, and the final opinion is written into a report produced by the secretariat. For the consensus conference, it emphasises that the citizens' panel "agreed" on the opinion which can help stimulate public opinion but the small differences inside the panel may be overridden. In comparison, the citizens' jury's method makes it possible to illustrate even the detailed views of the participants but it could actually be detrimental to forming a public opinion as it only shows the distribution of various views. 0 £ i * (I) Citizens' Jury on the National Pandemic Response g System Technology Assessment and National Pandemic Response System (Avian Influenza) According to Korea's Basic Act on Science and Technology of 2001, the Ministry of Education, Science and Technology must select new technologies that might be socially controversial and undertake technology assessments every year through the government-invested research centre, known as KISTEP. The assessment should be performed by experts and ordinary citizens, and the outcome must be reflected CT> in national policies.6 Such assessment by citizens was conducted by KISTEP in ^ 2006 and 2007 under the title of "Open Citizens' Forum." It was modelled after the consensus conference widely used in Western countries. In 2008, KISTEP commissioned the assessment project to the CDST, a non-profit NGO that has been active in improving citizens' participation in the field of science and technology.7 Upon being commissioned, the CDST began to look for a model that is more advanced than the Open Citizens' Forum based on the consensus conferences held in the previous two years. In the end they decided to try the citizens' jury for the first time in Korea, for which members are randomly selected, unlike the consensus conference for which they are mainly self-selected. The selection committee chose the NPRS as the target for the technology assessment in 2008. Since the scope would be overwhelmingly large, the Committee narrowed it down to zoonosis (avian influenza) - infectious diseases by biological terrorism using anthrax and new infectious diseases from climate change, and the administration team decided to limit the topic to national pandemic caused by avian influenza. Avian influenza (AI) is generally called the "bird flu" or "bird influenza." It is an acute infectious disease that occurs through infection by the avian influenza virus, a devastating disease with almost 100 percent mortality rate that causes acute respiratory symptoms in chickens, turkeys and other poultry. The problem became even more serious as it was recently discovered that it infects not only poultry but also human beings. Since the first case of human mortality from AI type-A H5N1 virus occurred in Hong Kong in 1997, there have been growing concerns of a pandemic from a new influenza. In fact, the avian influenza H5N1 that occurred in East Asia and Southeast Asia has been jumping geographic and species boundaries since late 2003. From late 2003 to June 2007, there were officially 317 cases of human infection of H5N1 in 12 countries, of which 191 ended in deaths (60.3 percent). There are even reports of suspected human-to-human infection, although these are very limited (Chun 2007). If the AI virus keeps evolving through gene mutation and becomes capable of efficient human-to-human infection, it could lead to Pandemic Influenza (PI), which could cause up to 100 million deaths around the world (Davis 2005). In fact, the 1918 Spanish Flu, one of the biggest catastrophes in human history, that took 40 million lives or 1 percent of the global population at the time, was recently found to have been caused by the AI virus. As the most recent AI problem grew in scale, the World Health Organisation announced a guideline on planning against PI in 1999 and 2005, urging each country to create a specific and doable step-by-step national contingency plan suitable to their own circumstances. The Korean government also has a PI response system of its own led by the Disease Control Centre of the National Institute of Health. Composition of the Citizens' Jury The citizens' jury is largely made up of the advisory committee, expert witnesses, and the jury. The citizens' jury on the NPRS was organised based on the following frameworks. First, the project management team (three members) created an advisory committee made up of experts who could advise them on the administration of the project and recruitment of experts.8 The committee was made up of members with technical expertise in the topic of "NPRS against AI" as well as experts in social sciences.9 Participants for the jury were selected through a stratified random sampling by a professional survey organisation. The project management team outsourced the selection process to Media Research, a consulting company, to come up with a list of candidates, men and women over the age of 19 living in Seoul or Gyeonggi Do (Province: hereafter Gyeonggi). It was the first attempt for citizen jurors in Korea although the population had to be limited to Seoul and Gyeonggi due to budgetary reasons. Media Research contacted 5,500 people among randomly extracted phone numbers. Of them, 118 expressed willingness to participate in the citizens' jury.10 Media Research stratified them into homogenous sub-groups in order to improve the representations of the sample. In other words, the 118 were grouped by demographic characteristics, and a final list of 59 was sent to the project management team. The team randomly selected 16 candidates out of this list. The final citizens' jury consisted of 14 out of the 16. Demographics of the final 14 citizen jurors were as follows (Table 2): eight women and six men,11 ages from early 20s to 70s, residing in large cities, small cities or rural areas, with occupations including unemployed, housewife, student, self-owned business and professional (Lee 2009). Table 2: List of the Citizens' Jury £ 3 a 0 £ * 1 * (I) 0 c I o ro Gender Age Occupation Region M 47 Self-owned business (interior decoration) Anyang, Gyeonggi F 25 Civil servant (contract position at a public clinic) Guro, Seoul F 44 Nursery teacher Dobong, Seoul M 31 Internet shopping mall Youngdeungpo, Seoul M 27 Hospital physiotherapist Bucheon, Gyeonggi F 51 Housewife Paju, Gyeonggi F 53 Counselor at Private Study Institute Songpa, Seoul F 45 Health food business Guri, Gyeonggi M 66 Self-owned business (real estate) Dongdaemun, Seoul F 62 Freelancer (English tutor) Seongnam, Gyeonggi M 55 Self-owned business (mail-delivered study aid) Gimpo, Gyeonggi F 70 Unemployed Suwon, Gyeonggi M 40 Financial institution Paju, Gyeonggi F 22 University student Gangbuk Seoul Process and Outcome of the Citizens' Jury Members of the citizens' jury listened to the presentations by different experts, asked questions, held their own discussions and came to their final assessment and policy recommendations. To help the jurors draw their conclusion and policy recommendations, the management team produced, with the help of the advisors, a list of questions to be answered by the jury. These questions were designed to clearly illustrate the jurors' views on the NPRS. The questions are largely grouped into four categories. The first group of questions are regarding the possibility of a national pandemic occurring from AI; the second group are on the state of readiness of Korea against such a national pandemic; the third and fourth are on the policy recommendations to improve the response system. These questions are as follows: What is the likelihood of a national pandemic occurring in Korea due to the Avian Influenza?, How would you rate Korea's response system against a possible outbreak of a national pandemic?, What are the areas of improvement necessary to ensure effective readiness and response against a national pandemic?, and What are the ways of enhancing citizens' understanding and confidence in the National Response System? To form a well thought-out response to the four questions, the citizens' jury was convened at a hotel meeting room in Seoul on four weekend days across two weeks: The first was the period August 30-31 (Saturday and Sunday) and the second was the period September 6-7, 2008.12 For four days, there were testimonies by eight experts, questions and answers, and discussions by the whole jury or by sub-groups. On the last day, the opinions of the citizen jurors were collected. Opinion gathering was conducted through the following process. First, regarding questions 1 and 2 that are about current conditions, they were asked to give a rating out of a scale of five. Secondly, for questions 3 and 4, a multitude of views were expressed through jury discussions. After several rounds of discussion, similar views were consolidated, other views were modified, and the final opinions were put together in a list. Lastly, they voted on the list of final opinions to identify the jurors' differing preferences. Each juror had votes in the number that was half of the number of opinions. But to ensure that various views are expressed, the number of votes that can be given for one opinion was limited. For example, they were able to give up to five votes for one opinion under question three, and up to three votes under question four. And they were also given veto power to be able to express themselves clearly on sharply contested opinions. The number of veto power was in proportion to voting rights: up to five under question 3 and up to three under question 4. But they were reminded that they were not required to exercise the veto power, unlike the voting right.13 The citizens' jury's assessment of the NPRS is as follows. They saw the likelihood of a national pandemic occurring due to AI to be relatively low, giving it a score of 1.79 out of a scale of 0-4 (higher score indicating a higher risk). But many voiced the view that much work has to be done to prevent it because if it does occur, the damage would be quite extensive. Regarding Korea's readiness in the areas of forecasting, human resources/equipment, vaccine/treatment and compensation, their evaluation was unfavourable, giving it a score of 1.5 out of a scale of 0-4, with 4 being the most favourable and 0 being the least. They thought that the readiness was especially insufficient in the area of vaccine or treatment (0.86). The best score was given to forecasting, but it was still a very low 1.5 (Figure 1). Regarding questions on improvement of the response system, a total of 25 opinions were collected through jury discussion.14 Thus each juror was given 13 votes and 5 vetoes, which made the total number of votes 182. The number of vetoes used was 11. As for this question, the following opinions were expressed. However, due to the limit of space, we only selected seven major responses that showed a high degree of agreements in order for this paper. "Stronger regulation against Figure 1: The Distribution of Readiness in the National Pandemic Response System overuse of antibiotics and growth hormones used on poultry" (17 votes), "To secure manpower dedicated to quarantine and disposal of poultry, provide specialised training, and strengthen follow-up monitoring" (13), "Ongoing monitoring/supervision and communication to prevent secondary damages from poultry disposal such as environmental contamination" (13), "To implement early blockade (access restriction) to prevent spreading of the AI and strengthen quarantine system" (12), "To install an organisation dedicated to developing and producing AI vaccine and treatment, and encourage private investment" (12), "To strengthen surveillance on sites with history of AI outbreak or likelihood of future outbreak (migratory bird sites, animal farm sites)" (11), "To secure a stock of treatment drugs (Tamiflu, etc.) enough for at least 20 percent of the population" (10). Lastly, regarding the question on how to enhance understanding and confidence in the NPRS, a total of 11 opinions were generated through jury discussion. Each juror was thus given six votes and three vetoes. There were 84 total votes and 10 total vetoes used. The expressed opinions were: "To use more public communication through cinemas or TV for the purpose of prevention education" (16 votes), "To create an institutional environment enabling the media to provide sufficient Q information without over/under-reporting" (13), "For the local media and local authorities to provide active education and communication targeting local residents" (11), "To increase citizens' online/offline participation in developing the national pandemic forecasting and response system and in the process of public communication" (10). Conclusion: Implications of the Citizens' Jury on Democracy £ 3 a > This article has analysed deliberative civic engagement with the case of Korea's first citizens' jury. It has examined whether the citizens' jury is effective in resolving the gap between technocracy and citizenry in our technological society. The first PQ citizens' jury in Korea has provided a new insight on deliberative democracy in several different ways, including the selection process of jurors and their role in deliberative democracy. PO There are several significant implications of Korea's first ever citizens' jury. ^ To begin with, random selection, in this case a stratified random sampling, as opposed to self-selection, is important in that the jurors would be more similar to ordinary citizens: this increases the demographic representation of the jurors. In fact, citizens' participation through random selection was already gaining attention as a viable option against the limitations of representative democracy (Manin 1997; Carson and Martin1999). Fishkin (1991) even called for "deliberative polling," a model of citizens' participation based on random selection, to address the lack of representation in representative democracy. The citizens' jury on the NPRS was organised through random selection in order to preclude several problems in self-section or volunteering process from the beginning; therefore, its composition was quite varied in terms of places of residence, occupations, education, and age. The nature of random selection makes it difficult to induce the utmost dedication from the participants, and the random selection conducted in the citizen's jury is not without areas of concern. Arbitrariness and bias in the jury composition could be avoided to a certain extent through random selection, but a membership of around 15 would be too small to have full demographic representation. Thus for the citizens' jury to become a truly powerful decision-making unit, it should be carefully designed to have the size that is sufficient to claim representation without undermining in-depth deliberation. Only then will the citizens' jury become a powerful institutional basis for public policy-making that can usher in deliberative participatory democracy, not opinion-gathering democracy. Secondly, this citizens' jury implies the possibility of deliberative democracy in the realm of science and technology. We were concerned about the lack of proactive-ness among the jurors in their attitude toward expert testimonies and discussion. However, their attitude changed visibly from the second of the four day sessions. Through the process of small group discussion (among five members) followed by general discussion, the members became friendlier with each other and gained higher understanding of the subject matter. They became much more active in their approach toward expert testimonies and internal discussion. When we asked the participants to fill out a survey form regarding the various aspects of the citizens' jury program during the last day, the result showed that 9 out of 14 responded "Very much so" to the question on whether they were satisfied with the attitude displayed by other citizen jurors, while the remaining 5 answered "Generally so." This is an indirect indicator of the positive evaluation of the citizens themselves of the active participation of fellow citizen jurors. One juror indeed said, "I was not sure about the quality of the citizens' jury because I was selected randomly; however, I was very impressed because participants worked hard during the process." Thirdly, the flip side of the same token proves the significance of the citizens' jury. As time goes by, some of the expert witnesses expressed their surprise at the sharp and to-the-point questions raised by the citizen jurors.15 If such randomly selected citizens lack the ability to deliberate on public policy issues, especially those of technical complexity, and are thus unable to make rational judgments on the topic, the citizens' jury cannot take root as an institution that can strengthen democracy. However, although it would be clearly impossible for ordinary citizens to acquire expert-level technical understanding in a short period of time, the jury's experience proves that even ordinary citizens can participate, with a certain basis of knowledge, in the learning, discussion and decision-making process of highly technical and complex issues if the process of deliberation is designed in a way that can pique their interest. Results of the aforementioned evaluation form appear to support such an assessment. To the question on whether they were familiar with the topic before joining the citizens' jury, only one person responded "Generally so," while nine answered, "Not at all" and four "Generally not so." This shows that they were initially lacking in knowledge on the topic. To the question on whether their knowledge on the topic was enriched through this experience, twelve responded "Very much so," while two answered, "Generally so." This shows that their basis of knowledge was dramatically strengthened through the discussions. Meanwhile, to the question on whether their views regarding the topic have changed after the discussion, nine answered, "Very much so," three responded, "Generally so," while only two picked "Generally not so." This means that there have been considerable changes to the citizen jurors' preferences regarding the issue. The deliberative procedures affect in a significant and positive manner the character of the jury in which they take place. Throughout the meetings, the jurors became more informed than before so that they were willing to shift their opinions in light of new knowledge. It is not dicey to conclude that the changing of their view is connected with the information they learned. In the 21st century, citizens used to giving up control over human affairs to science and technology because they do not understand the complexity of scientific and technological information and evidence. However, the citizens' jury on the NPRS proves that ordinary citizens are capable of dealing with these issues through deliberative democracy. As Isabel correctly observes (2011, 50), "in Habermas's normative model of communication, to deliberate is to engage in society's reason-based dialogue, y oriented toward common understanding, held among all citizens, and free from strategic action (i.e., from the influence of power and money)," and the current form of the citizens' jury exemplifies the high potential to serve deliberative democracy. As Valkenberg (2012, 477-478) argues, "citizens' abilities to exert influence must not depend on their level of science and technology education," because they learn knowledge through civic engagement. Experts are in an advantaged position in technology-related decisions; however, when the system, such as citizens' jury, provides an arrangement for expert to explain and train lay citizens, participants are able to make reliable decisions. Regardless of the fact that the citizens' jury's report is a non-binding policy suggestion, therefore, this form of deliberative democracy implies that informed citizens could provide meaningful policy alternatives. Last but not least, it implies that through their participation as civic jury, par- l-t/tit^'^m^p a^vm nun/^ rl n /-»«a /~v4- tvi/^pl- nrvmi c > ticipants earn civic pride - one of the most significant parts of technological citizenship - and this is very important in conjunction with democracy in our modern society in which science and technology become further significant. We spent days and nights with the citizen jurors and were able to observe them in formal and informal settings and listen to what they say with full attention, and we perceived that their level of understanding, concentration and discussion ability went up considerably over time. Indeed, the jurors themselves were seen to feel quite proud of such change. One of the questions in the evaluation form was whether there have LH been any changes in their views regarding citizens' participation in the process of ^ national policy discussion. Only one responded, "my views changed more toward the negative than positive," while seven answered, "my views became fully positive," and six chose, "my views changed more toward the positive than negative." This can be understood as an expression of their pride in the development of their deliberative ability. This point is well illustrated by the following statement written by a participant in his evaluation of the citizens' jury. When I decided to participate in the citizens' jury, which is a new and unfamiliar concept to me, I was worried and skeptical about what I can do with no expert knowledge or whether I can have any influence on something as big as policy recommendations. However, when we produced our policy recommendations to the government after listening to expert presentations, asking questions, and discussing with other ordinary citizens like myself, I felt proud as a citizen of a nation. I believe that being able to voice our views through such opportunity would be one shortcut to future development, and I hope that the government and the private sector would more actively develop such a program (Lee 2009). This means that the citizens' jury has demonstrated that through a systematic deliberation process, citizens with no expert knowledge can develop deliberative ability to make judgments on somewhat complex technical issues. Meanwhile, it is not perfect, we believe that this citizens' jury is a relatively good democratic system because it proves the display and understanding of differences and non-alignment among the participants that are revealed through deliberations. The goal of this citizens' jury was not to drive toward a unified opinion, but it was designed to highlight even the minor differences in the jurors' views through surveys, list of opinions, discussions and final voting. When we asked the question on how to improve the NPRS, they came up with numerous ideas, and even after filtering them out through intensive discussions, 25 independent views still remained. This is a good case in point of the diverging views of the jurors being fully respected. As Barnes (1999) argued, the success of the program should be judged not by whether decisions were made by the participants' agreement, but on whether the deliberative mechanism was designed in a way to help reveal and understand the differences and non-alignment between them. This is a cause to beware of on reaching an agreement, as it can end up hampering the deliberative process by creating tacit pressure that could suppress differences between the participants. From this point of view, the citizens' jury we analyse appears to have democratic implications, because the citizen's jury is designed to reveal the differences and non-alignment among the participating citizens . Acknowledgement This work was supported by the National Research Foundation of Korea Grant funded by the Korean Government (NRF-2013S1A3A2054849). Notes: 1. Hundreds of thousands of Koreans rallied daily in Seoul in May 2008, because the government decided to reimport beef from the U.S.in April. The Korean market had been shut for the past four and a half years following the first US case of mad cow disease in 2003. 2. Fuller (2000) calls such science and technology studies based on practical approaches "Low Church," in differentiation from "High Church" that focuses more on the academic approach to science and technology such as epistemology on scientific and technological knowledge. Generally the Low Church school sees STS as Science, Technology, and Society while the High Church school uses it as Science and Technology Studies. The concept of technological citizenship in this paper would be in line with the traditions of the Low Church STS. 3. Parentheses indicate the leading examples of each method. 4. Although polling and voting involve provision of information to help citizens determine their preferences, they are categorised as "preference gathering" since the given information does not guarantee a process of deliberation. 5. Citizens' jury is actively used in many parts of the world in order to review some issues, such as water quality in agriculture and bioethics. For the experience in the UK, see Barnes (1999) and Rogers-Hayden and Pidgeon (2006). For Australian cases, see Goodin and Niemeyer (2003), and for a Canadian case, see Johns and Einsiedel (2011). For a general assessment of citizens' jury, see Smith and Wales (2000). 6. Technology assessment is one of the technology policies for social integration where the positive and negative impact of technology on human beings, society, culture, politics, and economy is assessed beforehand to minimise any negativity. It is more regulatory toward technology than enabling, and its institutionalisation indicates a process of technology politics. For the history of technology assessment in Korea see Lee (2007). 7. In the Korean context, there had been a number of cases of citizens' participation under the title of "citizens' jury," such as the "Citizens' jury on protection of human genetic information" held in 2001, "Citizens' jury on facility to turn food waste into resources" held in 2004, and the "Citizens' jury on late-night electricity regime" held in 2007. But the citizens' jury in all three cases cannot be called a true citizens' jury, given that the recruitment method was not random sampling: it was by nomination or volunteering. The citizens' jury on the NPRS in 2008 was the first of its kind in Korea to be based on random selection. 8. When the government started this citizens' jury project, one of the two authors of this article, took a role as project manager and organised the jury, led the discussions, collected the data, and made the report to the government. Therefore, he was in a position to observe the entire process of the country's first citizens' jury. In this first citizens' jury, the researcher who was a project manager participated in the process as the observer, hence, generated more complete understanding of the group's activities. Since validity is stronger with the use of additional strategies used with observation, such as interviewing, surveys, or questionnaires (Kawulich 2005), the project management team also used survey research with participants at the end of the work. The major questions are about their experience as a jury, their understanding of the process, and their recommendations to the future forms of the citizens' jury. 9. A total of five members were appointed in the advisory committee (1 social sciences expert, 2 medical experts, 1 healthcare NGO expert and 1 KISTEP member). Eight experts who deliver lectures to the jury and answer their questions during deliberation were selected out of the advisory committee's nominations. They were experts representing government's health authorities, academia, and NGOs, including Doctors' Council for Humanitarianism. 10. Since there were only a few empirical studies showing the selection process by the random sampling, we cannot explain whether the reply rate here (2.1 percent) is too small or not; however, one particular empirical study conducted in the Netherlands in 2007 also showed only a 3 percent of reply rate in the first attempt out of 2,000 samples, and 6 percent out of 4,700 samples in the second attempt (Huitema 2007). Regardless of its low rate, therefore, we are sure that it can certainly be a part of random sampling, because the poll of participants were selected by the researchers, instead of volunteers. In random sampling, potential participants contacted select u S 3 a 0 £ * 1 * (I) 0 c I KD m themselves whether they participate in the process or not, which is unavoidable; however, since they were firstly contacted by the researchers, it was not self-selection. 11. Both of those who pulled out of the final citizens' jury were men, and participants were paid US$400. 12. In the U.S. or UK the citizens' jury is normally held during the week; however, Media Research was concerned that if it was held during the week almost no one would be able to participate. 13. This method of creating a list of opinions on the subject, going through deliberations, then converging the opinion through voting was used by the Danish Board of Technology (2005) at the citizens' jury on genetically modified crops. 14. The total number of opinions submitted by the jurors was 47 in the beginning. Through mutual discussion, they narrowed them down to 25, having consolidated similar ones. The process of narrowing down the opinions itself could be seen as the process of deliberation. 15. One of the factors that enabled such sharpness of the citizen jurors' questions appears to be the meeting format where expert witnesses with opposing views regarding the same issue conducted presentations. It seems that through such competing presentations, the jurors were able to get a better understanding of the subject and attained the ability to conduct a type of cross examination on expert witnesses. References: Awad, Isabel. 2011. Critical Multiculturalism and Deliberative Democracy: Opening Spaces for More Inclusive Communication. Javnost - The Public 18, 3, 39-54. Barnes, Marian. 1999. Building a Deliberative Democracy: An Evaluation of Two Citizens' Juries. Institute for Public Policy Research. Carson, Lyn, and Brin Martin. 1999. Random Selection in Politics. London: Praeger. Chun, Byung Chul. 2007. Epidemiology of the Avian Influenza and New Influenza Pandemic. Korean Journal of Public Health 44, 1, 27-40. The Danish Board of Technology. 2005. New GM Plants: The Final Document of the Citizens' Jury. Davis, Mike. 2005. The Monster at Our Door: The Global Threat of Avian Flu. New York: The New Press. Davison, Aidan, Ian Barns and Renato Schibeci. 1997. Problematic Publics: A Critical Review of Surveys of Public Attitudes to Biotechnology. Science, Technology and Human Values 22, 3, 317-348. Dryzek, John. 2000. Deliberative Democracy and Beyond. Oxford: Oxford University Press. Einsiedel, Edna, and Eastlick Deborah. 2000. Consensus Conferences as Deliberative Democracy: A Communications Perspective. Science Communication 21, 4, 323-343. Ellul, Jacques. 1964. The Technological Society. Vintage Books. Elster, Jena, ed. 1998. Deliberative Democracy. Cambridge: Cambridge University Press. Fan, Mei Fang. 2013 (online First). Evaluating the 2008 Consensus Conference on Genetically Modified Foods in Taiwan. Public Understanding of Science. Fishkin, James. 1991. Democracy and Deliberation. New Haven: Yale University Press. Frankenfeld, Philip. 1992. Technological Citizenship: A Normative Framework for Risk Studies. Science, Technology & Human Values 17, 4, 459-484. Fuller, Steve. 2000. Thomas Kuhn: A Philosophical History for Our Times. Chicago: The University of Chicago Press. The Jefferson Center. 2004. Citizens Jury Handbook. Johns, Mavis, and Edna Einsiedel. 2011. Institutional Policy Learning and Public Consultation: The Canadian Xenotransplantation Experience. Social Science & Medicine 73, 5, 655-662. Goodin, Robert, and Simon Niemeyer. 2003. When Does Deliberation Begin? Internal Reflection versus Public Discussion in Deliberative Democracy. Political Studies 51, 4, 627-649. Goven, Joanna. 2003. Deploying the Consensus Conference in New Zealand: Democracy and De- problematisation. Public Understanding of Science 12, 4, 423-440. Habermas, Jürgen. 1968. Toward a Rational Society: Student Protest, Science and Politics. Boston: Beacon Press. CO £ 3 a 0 £ * 1 * » 0 c B 00 m Huitema, Dave, Marleen Kerkhof and Udo Pesch. The Nature of the Beast: Are Citizens' Juries Deliberative or Pluralist? Policy Sciences 40, 4, 287-311. Jeong, Won Gyu. 2005. Two Faces of Democracy: Participatory Democracy and Deliberative Democracy. Society and Philosophy 10, 281-329. Joss, Simon, and John Durant. 1995. Public Participation in Science: The Role of Consensus Conferences in Europe. London: Science Museum. Kawulich, Barbara. 2005. Participant Observation as a Data Collection Method. Forum: Qualitative Social Research 6, 2, Art. 43. Lee, Young Hee. 2007. Social Control and Acceptance of Technology: Politics of Technology Assessment. Economy and Society [Korean Journal] 73, 246-268. Lee, Young Hee. 2009. Technology and Citizens: A Case Study of Citizen's Jury on Pandemic Influenza. Economy and Society [Korean Journal] 82, 216-239. Manin, Bernard. 1997. The Principles of Representative Government. London: Cambridge University Press. Marcuse, Herbert. 1964. One-dimensional Man. Boston: Beacon Press. McKenna, Bernard, and Yasmin Kasteren. 2006. The Micro Processes of Citizen Jury Deliberation: Implications for Deliberative Democracy. International Communication Association, 2006 Annual Meeting, 1-32. Nishizawa, Mariko. 2005. Citizen Deliberations on Science and Technology and Their Social Environments: Case Study on the Japanese Consensus Conference on GM Crops. Science and Public Policy 32, 6, 479-489. Oh, Hyun Chul. 2007. Deliberative Democracy and Political Participation. In S.S. Joo (ed.), Democracy and Citizens' Participation in Korea, 83-116. Seoul: Arche. Rogers-Hayden, Tee, and Nick Pidgeon. 2006. Reflecting Upon the UK's Citizens' Jury on Nanotechnologies: NanoJury UK. Nanotechnology Law & Business 3, 2, 167-178. Rowe, Gene, and Lynn Frewer. 2000. Public Participation Methods: A Framework for Evaluation. Science, Technology & Human Values 25, 1, 3-29. Seifert, Franz. 2006. Local Steps in an International Career: A Danish-style Consensus Conference in Austria. Public Understanding of Science 15, 1, 73-88. Smith, Graham, and Corinne Wales. 2000. Citizens' Juries and Deliberative Democracy. Political Studies 48, 1, 51-65. Tutui, Viorel. 2011. Democratic Deliberation Procedures: Theoretical and Practical Issues. Argumentum: Journal the Seminar of Discursive Logic, Argumentation Theory & Rhetoric 9, 1, 121-134. Valkenberg, Govert. 2012. Sustainable Technological Citizenship. European Journal of Social Theory 15, 4, 471-487.