DOI: 10.17573/cepar.2018.2.04 1.01 Original scientific article Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law1 Christoph E. Mueller German Research Institute for Public Administration, Germany mueller@foev-speyer.de Bettina Engewald German Research Institute for Public Administration, Germany engewald@foev-speyer.de ABSTRACT Freedom of information acts (FOIA) aim to improve the public's opportunities to access official information from public authorities and hence to increase the level of transparency. Thus, it is important to know whether and to what degree the effects intended by establishing FOIAs are achieved and how their implementation could be improved. Hence, this article presents the evaluation of the Hamburg Transparency Law (HmbTG) - Germany's first FOIA that binds authorities to disclose government information proactively. The purpose of the paper is to provide a valuable example of how evaluating FOIA might produce useful information for policymakers and public authorities. The analysis results, based on a mixed set of methods (i.e. standardised surveys, statistical secondary data, qualitative expert interviews, and criteria-driven document analysis), lead to the conclusion that the HmbTG was very effective in providing the direct access. On the other hand, it was found that strategies for implementing the law varied considerably between authorities, yet proactive disclosure was overall implemented effectively. Moreover, this law shows some weaknesses to be improved in the future. Besides providing practitioners with valuable insights into how a transparency law may be implemented, the evaluation of the HmbTG also provides researchers with ideas how FOIA evaluation might be conducted comprehensively. Keywords: access to information, proactive transparency, FOIA, Hamburg transparency law, evaluation JEL: K23 1 Parts of this article have already been published in German, see Herr, M., Müller, C.E., Engewald, B. and Ziekow, J. (2018a). Transparenzgesetzgebung in Deutschland in der Bewährung: Erfahrungen einer Gesetzesevaluation. Die öffentliche Verwaltung, 2018(5), pp. 165-174, as explicitly allowed by the publisher. Mueller, C. E., Engewald, B. (2018). Making Transparency Work: Experiences from the 69 Evaluation of the Hamburg Transparency Law Central European Public Administration Review, 16(2), pp. 69-90 Christoph E. Mueller, Bettina Engewald 1 Introduction Providing the public with open access to official information by means of freedom of information (FOI) laws - also referred to as transparency laws - is often considered to be a well suited approach for decreasing public sector corruption, improving governance, increasing bureaucratic efficiency and accountability, constraining politicians, or empowering citizens and journalists (e.g., Vadlamannati and Cooray, 2016; Berliner, 2010; Escaleras, Lin, and Register, 2010; Worthy, 2010; Piotrowski and Rosenbloom, 2002). Therefore it is not surprising that many countries have already passed FOI laws in order to "institutionalize transparency by creating legal guarantees of the right to request government information" (Berliner, 2010, p. 479). A common feature of all FOI laws is that they are supposed to increase the level of transparency by improving the public's opportunities to access government information from public authorities. Yet there are also substantial differences concerning the details of FOI legislation. For example, most FOI laws only cover the public's right to access government information by filing a request, while only a minority require the authorities to disclose official information proactively. Transparency laws also differ with regard to aspects of enforcement, ease of access, coverage, and exemptions (e.g., Ackerman and Sandoval-Ballesteros, 2006). Regardless of the contents and formulation of FOI laws, for the public and for policymakers it is crucial to know whether the effects intended by establishing a transparency law are achieved. Although previous research has already investigated potential effects with regard to public sector corruption (e.g., Cuc-ciniello, Porumbescu, and Grimmelikhuijsen, 2018; Bauhr and Grimes, 2014; Escaleras et al., 2010; Lindstedt and Naurin, 2010), governance (e.g., Cucciniel-lo et al., 2018; Islam, 2006), accountability (e.g., Cucciniello et al., 2018; Riddell, 2013), bureaucratic efficiency (e.g., Cucciniello et al., 2018; Vadlamannati and Cooray, 2016), perceived transparency (e.g., Worthy, 2010; Relly and Sabhar-wal, 2009), perceived government performance (e.g., Porumbescu, 2017a), and trust in government (e.g., Cucciniello et al., 2018; Porumbescu, 2017b; Grimmelikhuijsen et al., 2013; Worthy, 2010), most of these studies involved cross-national comparisons for assessing the effects of transparency laws. Yet comparing transparency laws between countries can be problematic because of "differences from one country to another, including varying political systems, disclosure procedures, and measures of public use and awareness" (Michener, 2011, p. 149). Thus, cross-national comparisons are not well suited to assessing the individual effects of a transparency law, and they do not provide policymakers or the public with specific knowledge about how the FOI legislation in a given country could be adjusted and improved. A fair assessment of a country's FOI legislation needs to take into account the individual formulation of the transparency law as well as its implementation - a unique and very country-specific process - which is crucial for its effectiveness (Relly and Sabharwal, 2009; Islam, 2006). Therefore, the current state of 70 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law research calls for comprehensive evaluations of individual transparency laws at national or even federal state level. An FOI law evaluation which serves as a valuable example of how such an endeavour might produce useful information for policymakers and public authorities is the evaluation of the Hamburg Transparency Law (HmbTG) from 2016 to 2017 (Herr et al., 2018b; see also Herr et al. 2017 for an on-line version of the final evaluation report). The evaluation was commissioned by the Hamburg Ministry of Justice and comprised various evaluation tasks, including the assessment of the effectiveness of the HmbTG and the quality of its implementation as well as an in-depth analysis of the legal document itself. Besides providing practitioners with valuable insights into how a transparency law may be implemented in order to achieve its intended effects, the evaluation of the HmbTG also provides researchers with ideas about how an FOI law evaluation might be conducted in evaluation practice. This article provides readers with a brief description of the contents of the HmbTG, the methodological approach of its evaluation, and the main findings of the evaluation. Before that, however, the historical background of the HmbTG is briefly explicated. 2 The Hamburg Transparency Law Germany is a federation, which is why FOI legislation has taken and still takes place at different levels and different speeds. It is crucial to differentiate between federal and federal state ('Land) level because each entity can only bind its own governmental bodies and administration. At the moment, Germany ranks 12 th in the 2017 Corruption Perceptions Index and 24th in the Global Open Data Index. At federal level, FOI legislation started in 1994 when Germany implemented the Council Directive 90/313/EEC by passing the Environmental Information Act (UIG). The UIG guarantees access to environmental information without prerequisites such as stating a reason for the interest in the information requested. Consequently, one might say that at federal level, German FOI legislation was initiated by EU legislation (Mecklenburg and Poppelmann, 2007, p. 16). In 1998, the Federal Ministry of the Interior drafted a federal FOI law, whose legislation took three legislative periods (the 14th, 15th, and 16th) (Mecklenburg and Poppelmann, 2007, p. 15). In 2002, an alliance of representatives of journalist and civil rights organisations joined the discussion with their own draft (Mecklenburg and Poppelmann, 2007, p. 15). The German Bundestag passed the Federal Act Governing Access to Information held by the Federal Government (IFG) at the end of 2005, and the IFG entered into force in 2006. Under the IFG, anyone is entitled to access official information from the authorities of the Federal Government. The IFG only applies to federal bodies and contains an obligation to release government information after a request has been filed. Yet the IFG does not force the authorities bound by the law to disclose government information proactively. Central European Public Administration Review, Vol. 16, No. 2/2018 71 Christoph E. Mueller, Bettina Engewald At Land level, FOI legislation is more diverse. Four Länder passed FOI laws several years before the Federation. The first was Brandenburg, whose FOI law dates back to 1998. It was followed by Berlin (1999), Schleswig-Holstein (2000) and North Rhine-Westphalia (2001). During the next two years, seven other Länder - Hamburg, Bremen, Mecklenburg-Western Pomerania, Saarland, Thuringia, Rhineland-Palatinate, and Saxony-Anhalt - followed. In 2018, Hesse included ten sections regulating the access to information upon request in its data protection law. Like the IFG, all these laws entitled everyone to access official information from the authorities of the respective Land. All the applicant needs to do is request the information. In 2012, Hamburg replaced its Freedom of Information Act with the HmbTG. The HmbTG was the first FOI act in Germany that included an obligation on the authorities to disclose information proactively on line. By 2017, three other Länder - Rhine-land-Palatinate, Bremen and Schleswig-Holstein - had revised their respective FOI laws and installed an obligation to disclose official information proactively too. Baden-Württemberg installed this obligation in its first FOI legislation in 2015. Therefore, in the autumn of 2018, the landscape of German FOI legislation is still fragmented. Five Länder force their authorities to disclose information proactively, eight Länder and the Federation give applicants the possibility to request information, whilst three Länder have not provided the public with access to official information by means of an FOI act so far. 2.1 Genesis of the HmbTG In 2006, Hamburg's parliament - the 'Bürgerschaft - passed the Hamburg Freedom of Information Act (HmbIFG). Its core feature was the obligation on public authorities to release official information on request. Yet the HmbIFG was not the end of Hamburg's efforts to make its administration transparent, because six years later, a more rigorous transparency law entered into force in an unusual manner. As in many other Länder, in Hamburg it is possible to pass a law via referendum. (See Article 50 of the Constitution of the Free and Hanseatic City of Hamburg). To initiate a referendum a people's initiative has to collect 10,000 signatures from citizens entitled to vote for the Bürgerschaft. Then the Bürgerschaft has to tackle the issue. If they do not make a positive decision on the matter, the people's initiative can move on to a referendum and ultimately have the citizens pass the bill. In 2011, the not-for-profit organisations Transparency International, Mehr Demokratie, and the Chaos Computer Club Hamburg founded the people's initiative 'Transparenz schafft Vertrauen' ('transparency creates trust') (Maatsch and Schnabel, 2015, p. 1). The initiative aimed to pass an FOI law containing an obligation to the proactive disclosure of information (Maatsch and Schnabel, 2015, p. 1) in order to impede corruption, prevent the waste of taxpayers' money, reduce mistrust, strengthen trust in politics and the administration, simplify administrative processes, and facilitate co-determination 72 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law (Bürgerschaft der Freien und Hansestadt Hamburg, 2011, p. 5). They intended to overturn the system of FOI legislation by creating a central information register (Bürgerschaft der Freien und Hansestadt Hamburg, 2011, p. 5) in which official information had to be published. The citizens were to have the possibility to inform themselves before political decisions were made (Bürgerschaft der Freien und Hansestadt Hamburg, 2011, p. 6). The initial draft of the bill of the people's initiative was written with the help of an on-line tool (Humborg et al., 2012, p. 56). By the end of 2011, the initiative had gathered the 10,000 necessary signatures (Maatsch and Schnabel, 2015, p. 1). The Bürgerschaft discussed the draft in early 2012 and the people's initiative 'Transparenz schafft Vertrauen' revised it in order to make it the basis for a referendum (Maatsch and Schnabel, 2015, p. 1). This revised draft was the basis for the bill of the HmbTG made by all fractions of the Bürgerschaft (Kleindiek, 2013, p. 188). Unlike the initial draft of the people's initiative 'Transparenz schafft Vertrauen', this draft was elaborated by only six to eight people (Kleindiek, 2013, p. 194). On 12 June 2012, the people's initiative and all fractions of the Bürgerschaft held a press conference and announced that the bill of the HmbTG would be passed the next day (Maatsch and Schnabel, 2015, p. 1f.). It would not be an exaggeration to say that the legislative process was very high-speed, particularly towards the end (Maatsch and Schnabel, 2015, p. 2). 2.2 Content of the HmbTG The HmbTG was the first FOI law in Germany that forces public authorities to disclose official information proactively. It is divided into four sections -the principle of transparency, information on request, the Hamburg Commissioner for data protection and freedom of information (HmbBfDI), and final provisions. The principle of transparency consists of the Sections 1-10. This includes the purpose of the HmbTG, terms and definitions, the scope of application, the protection of personal data, the exemptions, and the technical details of the obligation to proactive disclosure. The information on request is dealt with in three sections that regulate application, access to information, and the procedure. The final provisions govern the relationship between the HmbTG and other FOI legislation, state treaties, treaties older than the HmbTG itself, and the evaluation. With regard to the goals of the law, the HmbTG [in Section 1 (1)] first aims to facilitate the democratic formation of opinion and enable the monitoring of government action beyond the existing possibilities of information. To achieve this, information held by the authorities mentioned in Section 2 (3) HmbTG is to be made accessible to the general public. Nevertheless, the protection of personal data is guaranteed. The justification of the law adds that the proactive disclosure facilitates democratic opinion-making, enables the public to monitor the actions of the state and prevent corruption (Bürgerschaft der Freien und Hansestadt Hamburg, 2012, p. 12). The HmbTG is fur- Central European Public Administration Review, Vol. 16, No. 2/2018 73 Christoph E. Mueller, Bettina Engewald ther designed to support trust in the actions of politics and the administration (Bürgerschaft der Freien und Hansestadt Hamburg, 2012, p. 13). At the same time, the law is intended to raise cost awareness within the administration inasmuch as the possibility to raise questions creates pressure to justify the costs (Bürgerschaft der Freien und Hansestadt Hamburg, 2012, p. 13). Finally, Section 1 (2) HmbTG provides everyone with the right to direct access to information under the provisions of the HmbTG. Section 2 HmbTG contains various definitions. These include the very important terms 'obligation to proactive disclosure' and 'obligation to provide information upon request'. According to this section, all files - no matter in which form they are stored - are considered to be information. The information has to be published in a central, electronic, publically available register. Moreover, not only the public administration but also companies which perform public tasks (particularly public services) and are under the control of Hamburg have an obligation to proactive disclosure. In addition, direct and indirect public administration have to give access to information on request. Indirect public administration consists of legal entities under administrative law created by the state to fulfil the duties of the state. Section 3 HmbTG governs the scope of application. The areas that come under the obligation to proactive disclosure include public service contracts, budgets, administrative regulations, official statistics, studies carried out on behalf of the authority, geodata, the tree cadastres, urban land use plans and landscape plans, the main provisions of granted building permits, and subsidies and grant awards [Section 3 (1) HmbTG]. Contracts whose publication is of public interest should be published as long as this does not negatively affect the economic interests of Hamburg [Section 3 (2) HmbTG]. All information that falls under the obligation to proactive disclosure also falls under the obligation to provide information on request [Section 3 (3) HmbTG]. Sections 4-7 and Section 9 HmbTG deal with the exemptions. Exemptions are mainly made for personal data, some authorities, the protection of public interest, and trade secrets. Section 4 HmbTG regulates the protection of personal data. This means a general obliteration of personal data as in Section 4 (1) sentence 1 HmbTG. The remaining parts of Section 4 govern the exemptions from the non-disclosure of personal data. Section 5 HmbTG deals with exempted authorities. Among others, the courts and the Court of Auditors (as long as they operate in judicial independence), the State Office for the Protection of the Constitution, matters of tax assessment and collection, and broadcasters are exempt from the information obligation. Furthermore, Section 6 HmbTG exempts some public interests. These include the decisionmaking process of the Senate and draft decisions as well as protocols and consultations protected by special laws. In addition to Section 4, Section 7 (1) HmbTG exempts business and trading secrets insofar and for as long as the interest in non-disclosure outweighs the interest in disclosure. It is the first legal definition of the term 'business and trading secret'. Finally, Section 74 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law 9 HmbTG contains some further restrictions of the obligation to inform. For example, contracts with an intrinsic value of less than €100,000 do not need to be published. Sections 8 and 10 HmbTG are technical regulations. The authorities have to take measures so that exempted data can be separated easily from the rest of the information [Section 8 HmbTG]. Section 10 regulates the information register and its use and the further use and distribution of the information. This includes a special regulation for contracts. All contracts must be concluded in such a way that they become effective only one month after having been made public, and in such a way that they can be revoked by the authority within that period [Section 10 (2) HmbTG]. Sections 11 to 13 HmbTG contain the application procedure for information that is not subject to proactive disclosure. Section 11 regulates the request. The request should be filed in written form, but an electronic or oral request is also possible [Section 11 (1) HmbTG]. The claimant has to name the information required [Section 11 (2) sentence 1 HmbTG]. The authority has to counsel the claimant [Section 11 (2) sentence 2 HmbTG]. The authorities have to provide the claimant with the name of the authority which has the information if they do not have it themselves [Section 11 (2) sentence 3 HmbTG]. Sections 12 and 13 HmbTG govern the access to information. The authority has to give the claimant access to the information or provide the information depending on what the claimant is requesting. If exemptions pursuant to Section 4 or Section 7 HmbTG hinder the provision of access, the authority has to ask the respective third party for its consent in order to provide claimants with access to the information requested. With regard to time frames, the access to information requested has to be provided within one month but the deadline can be prolonged by another month. Equally, the rejection has to be made within a month too. The rejection has to be in written form, but an oral request can be rejected orally. Finally, costs and fees apply. Section 14 HmbTG regulates the HmbBfDI, who assumes the role of an ombudsman. Anyone who believes their rights under the HmbTG have been violated can appeal to the HmbBfDI. The HmbBfDI can only counsel and provide the advice-seeking individual with an informed opinion. In addition, the HmbBfDI informs the public about the right to access information and counsels the authorities. Furthermore, if the HmbBfDI identifies deficiencies in the way the authorities treat the citizen's right to access official information, the HmbBfDI can ask the authorities to remedy those deficiencies. Finally, Section 16 HmbTG governs the relation between the HmbTG and state treaties. The HmbTG also applies to treaties older than the HmbTG itself [Section 17 HmbTG]. Section 18 (2) stipulates that the HmbTG has to be evaluated four years after it entered into force. The Senate (the government of Hamburg) has to inform the Bürgerschaft about the results of the evaluation. Central European Public Administration Review, Vol. 16, No. 2/2018 75 Christoph E. Mueller, Bettina Engewald 3 Evaluation of the HmbTG 3.1 Objectives of the evaluation According to the evaluation clause in Section 18 (2) HmbTG, the overall purpose of the evaluation was to assess the law's implementation and effectiveness. With regard to the latter, the evaluation team was to check whether the objectives stated in Section 1 HmbTG - providing official information to the public, supporting democratic will- and opinion-forming processes, and enabling the public to monitor government actions - were achieved. Moreover, the evaluation team had to assess whether the objectives specified in the justification of the law (Bürgerschaft der Freien und Hansestadt Hamburg, 2012, pp. 12f.) were achieved, namely increasing the public's acceptance of administrative action and enhancing trust in government actions. The second central evaluation task was to provide the client of the evaluation with empirically founded insights into the law's implementation by the public authorities bound by the law and into the implementation of the Hamburg Transparency Portal, the information register specified in Section 10 HmbTG. The evaluation team was expected to uncover implementation problems and identify potentials for optimisation. In this context, not only implementation processes had to be investigated but also the legal document itself, which means that the evaluation team also had to assess whether or not the formulation of the HmbTG supported or hampered the implementation of the law. In order to comply with the evaluation goals, both the obligation of the public authorities to disclose official information proactively and their obligation to release government information on request were considered by the evaluation team. Since the HmbTG was the first transparency law in Germany that forced authorities to disclose government information proactively, however, the evaluation put special emphasis on the implementation and effects of this special feature and the infrastructure - namely the Hamburg Transparency Portal - that was necessary for implementing the proactive disclosure of information. 3.2 Methodology Basically, the evaluation of the HmbTG was conceptualised as an ex-post regulatory impact assessment (Böhret & Konzendorf, 2001), which means that the evaluation of the legal measure did not start until the regulation had already come into force. In the present case, the evaluation started four years after the HmbTG had entered into force. It started in July 2016 and was completed in July 2017. In conducting the study, the evaluation team availed itself of an integrative mixed-methods approach that combined the use of quantitative and qualitative methods of empirical research as well as social scientific and legal perspectives. Moreover, the team followed a participative approach to evaluation (Stockmann, 2008), which means that relevant stakeholders were 76 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law involved in the evaluation process - for example, in the course of instrument development and sampling - in order to exploit their domain-related expert knowledge for evaluative purposes. To ensure a study of high quality, the evaluation team followed the standards of the German and Austrian Evaluation Association (DeGEval, 2016). The evaluation of the HmbTG required the collection of empirical data from various sources. In cooperation with the Hamburg Ministry of Justice, the evaluation team identified four relevant stakeholder groups from which data had to be gathered: (1) public authorities bound by the HmbTG who have to implement the law; (2) users of the Transparency Portal who are the main target group of the HmbTG's obligation to proactive disclosure of information; (3) the employees of the city of Hamburg who have to deal with the HmbTG in their daily work; (4) members of the advisory board of the former project 'Implementation of the Hamburg Transparency Law' - a committee mainly composed of representatives of civil society organisations that was intended to accompany and control the implementation of the HmbTG. In the course of data collection, the evaluation team followed different strategies. First, all the stakeholder groups mentioned above were subjected to an online survey. Depending on the recipients of the survey, a broad range of information was gathered. The survey of the public authorities bound by the law included various questions concerning the amount, type, and quality of information published and released, about the organisational structures and processes adjusted for implementing the HmbTG, and about the quality, manageability, and effectiveness of the law. The survey was conducted in two waves in order to collect a sufficient amount of information on objective data (i.e., on information published and released). The first wave started in October 2016 and aimed to collect objective data for the period from 2012 to September 2016. The second wave started in March 2017 and was intended to collect objective data for the period from October 2016 to February 2017. In total, 93 authorities participated in the first wave and 83 in the second. Because the evaluation team was not provided with a full list of all authorities bound by the HmbTG, a response rate could not be calculated. However, according to the evaluation client the response rate was very high. The users of the Transparency Portal were mainly asked about their user behaviour when using the portal and about their perceptions of the effects of proactive information disclosure. Because it was not possible to recruit survey participants randomly, the evaluation team had to rely on a specific type of convenience sample. More specifically, survey participants were recruited via flash-layers - a non-intrusive type of pop-up - while surfing on the Transpar- Central European Public Administration Review, Vol. 16, No. 2/2018 77 Christoph E. Mueller, Bettina Engewald ency Portal. The survey was on line from October 2016 to February 2017. In total, 412 visitors to the Transparency Portal participated in the survey. The survey of the employees of the city of Hamburg mainly contained questions about the law's practicability and its potential effectiveness. The on-line questionnaire was distributed via the joint portal of Hamburg's administration. The survey was on line from October 2016 to December 2016. In total, 896 employees of the city of Hamburg participated. Finally, the survey of the advisory board also contained questions relating to the implementation and effectiveness of the HmbTG. All 17 members of the advisory board were requested to participate in the survey from October 2016 to December 2016. Only five institutions took up the offer, which equals a response rate of 29%. Second, as an accompanying measure, the evaluation team conducted 13 semi-structured interviews (see list in annex) with experts and representatives from different institutions - namely, from the authorities bound by the law, indirect public administration, the above-mentioned advisory board, the office of the HmbBfDI, and the Technical Control Center for the Transparency Portal. These interviews were primarily used as an instrument to get deeper insights into the implementation of the HmbTG and obtain expert assessments about the practicability of the law, its effects, and any revisions that might be necessary. Third, statistical secondary data relating to the Transparency Portal - containing information on the amount, subject, date, and type of information published by authorities and on usage patterns (e.g. monthly click numbers; search terms; use of help function) from September 2014 to February 2017 -were analysed to provide the evaluation client both with a detailed overview of the official information proactively published on the portal by the public authorities and with an assessment of the level of public demand for government information in the city of Hamburg. Finally, the evaluation team conducted a criteria-led content analysis of the Transparency Portal by screening documents published on the portal in order to assess whether they met the requirements specified by the HmbTG (exemplary criteria: accuracy, completeness, and comprehensibility of information provided; access to information provided; timely provision of information; etc.) and whether they were prepared in a user-friendly and comprehensible way (exemplary criteria: quality of information on the purpose and the site operator; quality of presentation of the page contents; provision of examples; quality/degree of visualisation/graphical presentation; quality of search function; provision of a help function; etc.). Half the documents were selected based on their relevance, whilst the other half were selected randomly. In total, 40 documents published on the Transparency Portal were analysed. 78 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law While the evaluation team collected and analysed empirical data, its members also conducted a thorough assessment of the legal text of the HmbTG against the background of the jurisprudence relating to it, the legal commentary of the HmbTG, the internal comments on how to implement the HmbTG developed by the Hamburg Ministry of Justice, the activity reports of the HmbBfDI, and the available articles by legal scholars relating to the HmbTG. The results of the legal assessment were combined with the empirical findings in order to paint a comprehensive picture of the quality of the HmbTG. 3.3 Results The following sections contain selected evaluation findings with regard to the effectiveness of the HmbTG, its implementation, and the formulation of the law. A complete and detailed overview of the findings - including the numbers and statistical data - can be found in Herr et al. (2018b; 2017). 3.3.1 Effectiveness of the HmbTG First, assessing the effectiveness of the HmbTG included an analysis of the outputs induced by the passing of the law. Typical output indicators of an FOI law like the HmbTG are the amount of information that is published proac-tively, the numbers of FOI requests filed, granted, and refused, and the extent of delays of information disclosure. The evaluation of the HmbTG provided detailed information on all these outputs and a lot more. For example, it was found that about 66,000 data files were published on the Transparency Portal from September 2014 to February 2017. Almost two thirds of the data published on the Portal referred to the subject areas 'infrastructure, construction and housing' and 'politics and elections', the majority of which were published without any delay. With regard to the demand side, the Transparency Portal was accessed more than 22.5 million times from April 2015 to February 2017, mostly by private citizens. In addition to that, the evaluation found that during the period from October 2012 to February 2017, Hamburg's authorities received more than 4,000 requests for accessing official information that had not been published on the Transparency Portal. In more than 75% of these cases, complete access to the information requested was granted. In contrast to that, considerably fewer than 10% of the requests were fully denied. It is worth noting that only a small fraction - about 13% - of the requests with a positive decision needed more time for processing than is allowed by the HmbTG. Finally, in the majority of cases, the authorities charged claimants only moderate fees or provided the information requested free of charge. Second, evaluating the effectiveness of the HmbTG involved an assessment of whether or not it was a suitable instrument for achieving the goals specified in Section 1 HmbTG. For this purpose, the authorities bound by the HmbTG, the employees of the city of Hamburg, and several experts were asked to provide assessments in the on-line surveys and expert interviews conducted. The results of the evidence collected suggested that the expediency of the Central European Public Administration Review, Vol. 16, No. 2/2018 79 Christoph E. Mueller, Bettina Engewald HmbTG was assessed ambiguously. Whereas the authorities bound by the law and employees of the city of Hamburg were, on average, slightly critical of the HmbTG's expediency for achieving the goals stated in Section 1 HmbTG, the advisory board and several of the experts interviewed were rather more positive on this issue. More specifically, 73.6% of the authorities bound by the HmbTG and 71.8% of Hamburg's employees believed that the HmbTG did not fulfil its purpose or only partially fulfilled it. Further, in a follow-up correlational analysis, it was found that the degree of expediency as assessed by the bound authorities and Hamburg's employees was positively associated with their assessments of the manageability (authorities: r = .42, p < .01; employees: r =.54, p < .001) and comprehensibility of the HmbTG (authorities: r = .29, p < .05; employees: r = .40, p < .001). This means that the perceived expediency of the law at least partially depends on aspects of its formulation and wording. Finally, determining the effectiveness of the HmbTG included assessments by different stakeholder groups as to whether the HmbTG had impacts on a variety of specific aspects. In this context, a majority of the users of the Transparency Portal believed that the disclosure of government information increases trust in government actions (66.3%), opportunities for political participation (78.0%), and opportunities for monitoring government actions (55.3%). With regard to the latter, however, this was assessed differently by the majority of the Hamburg employees surveyed who - on average - did not see an increased potential for public monitoring of government actions. More specifically, only 32.6% of the employees believed that the HmbTG increases the opportunities to monitor government actions. In addition to that, the majority of the employees (72.6%) were negative about whether or not the introduction of the HmbTG led to increased cost awareness within the public authorities bound by the law. 3.3.2 Implementation of the HmbTG The effective implementation of an FOI act is an important prerequisite for its impacts because "without effective implementation, an access to information law - however well drafted - will fail to meet the public policy objectives of transparency" (Neuman and Calland, 2007, p. 182). Consequently, analysing the implementation of the HmbTG had to be an essential part of the evaluation. Due to the vast amount of data collected, however, the results of the implementation analysis cannot be presented in full detail here. Instead, the following paragraphs provide a brief overview of selected findings. Basically, one can distinguish between two areas of implementation of the HmbTG, namely the implementation that takes place within the authorities bound by the law and the implementation that takes place outside those authorities. Both areas were considered by the evaluation team. First, with regard to the law's implementation within the bound authorities, the evaluation team put some emphasis on changes in organisational structures and processes. In this context, it was found that the authorities bound by the law 80 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law introduced diverse types of organisational structural change - for example, new positions within authorities managing the proactive disclosure of information and handling the processing of incoming requests were created or specific responsibilities assigned - in order to respond to the requirements made by the HmbTG. Furthermore, the evaluation team observed the introduction and adjustment of processes and workflows in respect of reviewing, recording, and monitoring incoming requests and the data which have to be published. The evaluation team also found that the authorities established new and adjusted existing processes of quality management, data protection, and complaint management, and they observed that specific routines for handling exceptions and preparing the information for publication were developed. At the end of the day, only a small fraction of authorities stated that they did not make any adjustments in response to the introduction of the HmbTG. In this context, an important finding was that there was no generalis-able 'best practice' in respect of how authorities responded to the introduction of the HmbTG in terms of organisational and procedural adaptation. On the contrary, it became clear that the strategies for implementing the law varied considerably, depending on a number of factors such as the nature, size, and administrative culture of the individual authorities. Second, with regard to the implementation of the HmbTG outside the authorities bound by the law, the evaluation team mainly investigated aspects relating to the technical infrastructure required for implementing the obligation to proactive disclosure, namely the Hamburg Transparency Portal. It goes without saying that the functionality of such an information register depends on the technical implementation of the portal and the quality of the workflows employed by the authorities to make their data available on the portal. While the technical implementation was considered to be good by the majority of the authorities, many of them reported minor problems with the developed workflows. However, almost two thirds of the authorities stated that technical problems with the workflows are usually solved in a very timely manner, which guarantees the functionality of the portal. This brings us to an important aspect, namely the existence of a sufficiently well staffed and experienced technical support unit. And indeed, the work and advice provided by the responsible support unit in Hamburg was assessed as very important and helpful by the authorities. Moreover, the effectiveness of the portal depends not only on technical aspects but also on its clarity and usability and the preparation and comprehensibility of the information published on the portal. Neither the clarity nor the usability were rated as negative or positive by several groups of stakeholders (e.g., the surveyed users of the portal and the employees of the city of Hamburg). One of the main reasons for these mediocre assessments is the search engine embedded on the portal, which still has to be improved. A correlational analysis showed that the better the search engine was rated, the more positive the assessments of the clarity (r = .64; p < .001) and usability of the portal as a whole (r = .72; p < .001), and the greater the satisfaction of the users with the time needed to find the information being sought (r = .59; p < Central European Public Administration Review, Vol. 16, No. 2/2018 81 Christoph E. Mueller, Bettina Engewald .001). Yet if the information being sought is eventually found by the searchers, it fulfils its purpose in most of the cases. The users of the Transparency Portal - and some representatives from other stakeholder groups in the expert interviews - stated that the information obtained on the portal was mostly comprehensible, complete, and useful. 65.3% of the portal users surveyed stated that the information obtained met their information needs (in contrast to 7.1% who claimed that it did not meet their needs), 69% had the impression that the information was complete (in contrast to 12.1% who considered it incomplete), and 79.3% found that the information was comprehensible (in contrast to fewer than 3% who found it not to be comprehensible). Finally, the evaluation team collected data on whether existing advisory services and support measures provided by the Hamburg Ministry of Justice, the HmbBfDI, and other actors played an important role in implementing the law. In this context, the authorities bound by the HmbTG mainly referred to the internal comments of the Ministry of Justice that were indispensable for handling the law in everyday practice. Similarly, the authorities stated that it was important to have their employees trained for working with the HmbTG by qualified training institutions. On the contrary, the advisory services of the HmbBfDI were only rarely used by the authorities, although the existence of such an offer was considered to be important. 3.3.3 Formulation of the HmbTG The effectiveness of a transparency law depends not only on its implementation, but also on its formulation. The more comprehensible a transparency law is to those bodies that have to work with it, the higher the degree of manageability and as a consequence, the easier its implementation. Because of these relationships, the evaluation team took a close look into the formulation of the HmbTG. From an empirical perspective, this mainly meant that those who deal with the law in everyday practice were asked whether the HmbTG is comprehensible and manageable in practice. With regard to comprehensibility, representatives of the authorities bound by the law and employees of the city of Hamburg who had to deal with the HmbTG in their daily work rated its comprehensibility to be only average. More specifically, more than 80% of the authorities and employees surveyed did not rate the comprehensibility of the HmbTG as good or very good. Therefore, it is not surprising that the manageability of the law was also rated as mediocre. Here too, more than 80% of the authorities and employees surveyed did not believe that the manageability of the HmbTG was good or very good. Besides these rather general aspects, the legal analysis showed that the HmbTG poses several specific legal problems. The most complicated problem revolves around indirect public administration and its mention in Section 2 (3) and (5) HmbTG. The German administrative doctrine refers to 'indirect public administration' if the state creates a legal entity under administrative law to fulfil its duties. Those legal entities are not to be confused with legal entities 82 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law under civil law. According to Section 2 (3) sentence 1 HmbTG, an authority is what Section 1 (2) of the Hamburg Administrative Procedure Act defines as an authority. Section 1 (2) of the Hamburg Administrative Procedure Act says that any entity which performs tasks of the public administration is an authority. This is the functional definition of 'authority' (Maatsch and Schnabel, 2015, p. 141). This definition includes direct as well as indirect public administration (Maatsch and Schnabel, 2015, p. 141). Unfortunately, Section 2 (5) HmbTG refers to the authorities pursuant to Section 2 (3) HmbTG and indirect public administration separately, which raises a question: does indirect public administration fall under the obligation to proactive disclosure or not? A lengthy discussion among legal scholars followed. Some believe that indirect public administration is under the obligation to proactive disclosure, whilst others claim that it is not. The HmbTG has its own definition of authority. The latter group has to answer a second question: if indirect public administration does not fall under the obligation to proactive disclosure found in Section 2 (3) HmbTG, does it fall under the obligation to publish as stated in Section 3 (2) HmbTG? Some deny this because Section 3 (2) only relates to Section 3 (1) HmbTG. After the evaluation had been completed, both the Administrative Court Hamburg and the Hamburg Higher Administrative Court subsequently and unanimously ruled that the indirect public administration does not fall under the obligation to proactive disclosure (VG Hamburg, Urteil vom 18.9.2017 - 17 K 273/15 and Hamburgisches OVG, Beschluss vom 16.4.2018 -3 Bf 271/17.Z). Personally, we believe that indirect public administration does fall under the obligation to proactive disclosure. The wording of both Section 2 (3) and (5) HmbTG is of no help. Although they are both easy to understand and clear, they contradict each other. The definition of 'authority' under German administrative law is the same in the Federation and the Lander. It is fairly old and has so far proved to be a clear, easily understandable, working definition. Given that good regulation is clear and easy to understand, it seems very unwise to change this in favour of a second term for 'authority' which only applies to one single law. A law which requires the direct administration entities controlled by the state to publish information but does not apply the same requirement to the indirect administration entities is unsystematic. Moreover, it was the intention of the people's initiative to have the whole administration fall under the obligation to proactive disclosure. Therefore, in order to solve the problem, we recommend a revision of Section 2 HmbTG. Another important legal problem is found in Section 3 HmbTG: many of the law's terms and definitions are unclear. For example, Section 3 (1) No. 1 provides for publication of the petitions of the decisions of the Senate. While petitions exist, they are not part of the decisions. Moreover, according to Section 3 (1) No. 8 HmbTG, reports and studies which were commissioned by an authority, affected the authority's decision, or served the purpose of preparing a decision, have to be published. It is unclear under which conditions a report or study is 'commissioned' by an authority. It is equally unclear whether the reports and studies have to be commissioned and used or if the use of a Central European Public Administration Review, Vol. 16, No. 2/2018 83 Christoph E. Mueller, Bettina Engewald study by a third party is enough to warrant publication. Another example can be found in Section 3 (1) No. 15 HmbTG, which provides for publication of the salaries of the CEOs of the businesses under the control of the city of Hamburg. Yet it is conceivable that the legislative power for regulating this subject is held not by the city of Hamburg but by the Federation. Furthermore, this may or may not impair the CEO's right to informational self-determination. Another legal problem has to do with Section 4 (1) sentence 1 HmbTG, which concerns the obliteration of all personal data. The obliteration of all personal data sounds easy in theory, but it is complicated in practice because it does not provide for an exemption for ministers. The purpose of Section 4 (1) sentence 1 HmbTG is that the author of a published document is unrecognisable. This extends to the names of ministers since there is no exception for them. It is, however, easy to find out who held which post in which government. An obliteration of the minister's name is not enough to obliterate the author. Even the name of the ministry, the date, and the content of the document may provide hints on who the author is. As a consequence the document cannot be published. Personally, we recommend a revision of Section 4 (1) sentence 1 HmbTG. Finally, Section 10 (2) HmbTG, which regulates the possibility to revoke a contract, is unique in German law. Unfortunately, it is not problem-free. Dogmatically, it is unclear where it fits in to German civil law which provides the leges generale for the revoking of contracts. Only the Federation has the legislative capacity to change civil law. Additionally, the Civil Code prohibits a right to withdrawal written in standard terms and conditions if the withdrawal is unconditional. Section 10 (2) HmbTG does not provide for any conditions for the withdrawal. It is unclear whether or not this violates the Civil Code. 4 Discussion Taken together, the evaluation findings suggest that the intended goals of the HmbTG were widely achieved. Particularly with regard to the outputs, the findings show that one of the main goals of the law - namely, providing the public with free and direct access to government information - was largely met. We believe that this is true despite the fact that no target attainment criteria were specified prior to the evaluation, which would have made it easier to assess whether goals were achieved or not. Further, since the results acquired by the output analysis are based on objective secondary data, they can be considered as quite reliable. In contrast to that, the findings with regard to the expediency and the impacts of the law are less reliable for two reasons. First, there were no baseline, time-series, or control group data available (see Mueller 2018), which is why the evaluation team had to rely exclusively on stakeholders' perceptions and subjective assessments of potential outcomes. Although these ratings provide valuable hints about whether or not the HmbTG is effective in changing various aspects - such as increasing 84 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law the acceptance of administrative action, trust in government actions, the opportunities for political participation, and the opportunities for monitoring government actions - the findings should not be interpreted as causal effects of the law. Second, the expediency of the HmbTG and the impacts as perceived by different groups of stakeholders were rated ambiguously. Whereas some stakeholder groups saw neither a high expediency nor any positive outcomes of the HmbTG, others were confident that the commencement of the law has led to positive changes regarding several aspects. As stated by Herr et al. (2018b), these ambiguous perceptions are presumably caused by different viewpoints between those required to implement the law and deal with it in everyday practice and those who benefit from the law by gaining access to official information. Hence, it is concluded that it is likely that the HmbTG has at least had some positive effects on its intended target groups, but that future research should further investigate its causal effects. When it comes to the implementation of the law, the evaluation findings provide practitioners with several important implications. First of all, it became clear that there is no best way of implementing the law, but that any authority bound by the HmbTG should find its own individual strategy for implementation. Second, the evaluation showed that the introduction of the law did not lead to a cost explosion as expected by some of its critics. This is an important message to all those countries or federal states that still do not have a transparency law containing an obligation to proactive disclosure. Third, the findings also clearly showed that accompanying measures - such as detailed comments about how the law is applied and training and consulting measures - are indispensable in order to support the authorities by implementing the law. Consequently, such measures should be developed even before the law enters into force so that they can provide authorities with the necessary support right from the beginning of the implementation process. Moreover, it has become clear that developing a working technical infrastructure - including the existence of a sufficiently well equipped and experienced technical support unit - is indispensable for guaranteeing the smooth execution of the obligation to proactive information disclosure. Finally, the evaluation findings suggest that the quality of the formulation of the HmbTG should be improved in future revisions because the compre-hensibility of the law determines its manageability for practitioners, which in turn affects its effectiveness in respect of producing outputs and inducing impacts. Hints on how the HmbTG might be revised are provided by the results of the legal analysis of the HmbTG (Herr et al., 2018b) and by more than 50 specific recommendations made by the authorities bound by the HmbTG (Herr et al., 2018b, pp. 340-350). Recommendations are, for example, to restrict the amount of information that has to be published, to specify and concretise legal terms and definitions, to revise the regulations concerning the charging of fees, and to introduce instruments for sanctioning if authorities do not comply with the legal requirements. Central European Public Administration Review, Vol. 16, No. 2/2018 85 Christoph E. Mueller, Bettina Engewald 5 Conclusion This article was devoted to presenting the background and results of the evaluation of the HmbTG, Germany's first FOI law that forced public authorities to disclose government information proactively. The evaluation findings showed that the introduction of a transparency law can lead to advantages as well as disadvantages in various stakeholder groups. In general, however, the evaluation team concluded that the advantages which have accompanied the introduction of the law outweigh the disadvantages. Because of this overall conclusion, introducing transparency laws forcing public authorities to publish government information proactively and release information on request seems to be a reasonable approach for increasing the level of transparency in Germany. The present article also showed how valuable a comprehensive FOI law evaluation can be. Besides the fact that it contributes to the scientific literature by providing researchers with unique insights into the effectiveness and functionality of a transparency law, it also provides the legislative power with useful information for revising the law and practitioners with valuable information about how to deal with the law in everyday practice. In order to ensure the quality of future FOI law evaluations, they require their clients to make various types of preparation. First, when commissioning an evaluation, one should be clear about the goals pursued by it (e.g., assessing the effectiveness or implementation of an FOI law) and communicate these to the evaluators in a clear manner. Second, evaluation clients should provide evaluators with sufficient data, particularly when it comes to causal impact assessment, because this evaluation task depends crucially on the availability of certain types of data (e.g., longitudinal or control group data). Thus, it is reasonable for public authorities to start collecting data even before an FOI law comes into force and to continue with data collection until the evaluation starts. Third, since the evaluation of FOI laws takes time and involves costs, evaluation clients should provide sufficient temporal and monetary resources so that evaluators can conduct the evaluation appropriately. This is particularly important for estimating the causal impacts of an FOI law, which often consumes more resources than other evaluation tasks (e.g., White, 2006). Fourth, evaluation clients should provide FOI law evaluators with organisational assistance, for example by giving them access to the relevant information carriers (e.g., public authorities which are responsible for implementing an FOI law, lawmakers, or civil society actors). Finally, it seems reasonable for clients to ensure the publication of evaluation reports. This is not only relevant for various kinds of actor in order to learn from the evaluation and work with the evaluation results, but also enables researchers to summarise the results of FOI law evaluations and provides the basis for 'meta-evaluations', studies that "check evaluations for problems such as bias, technical error, administrative difficulties, and misuse" (Stufflebeam, 2010, p. 99). 86 Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law References Ackerman, J. M. and Sandoval-Ballesteros, I. E. (2006). The global explosion of freedom of information laws. Administrative Law Review, 58(1), pp. 85-130. Bauhr, M. and Grimes, M. (2014). Indignation or resignation: The implications of transparency for societal accountability. Governance, 27(2), pp. 291-320. Berliner, D. (2010). The political origins of transparency. Journal of Politics, 76(2), pp. 479-491. Böhret, C. and Konzendorf, G. (2001). Handbuch Gesetzesfolgenabschätzung (GFA): Gesetze, Verordnungen, Verwaltungsvorschriften. Baden-Baden: Nomos. Bürgerschaft der Freien und Hansestadt Hamburg (2011). Unterrichtung durch die Präsidentin der Bürgerschaft. Betr.: Volksinitiative „Transparenz schafft Vertrauen". At , accessed 27 August 2018. Bürgerschaft der Freien und Hansestadt Hamburg (2012). Antrag der Abgeordneten Dr. Andreas Dressel, Urs Tabbert, Hansjörg Schmidt, Gabi Dobusch, Peri Arndt, Jan-Hinrich Fock, Prof. Dr. Loretana de Libero, Olaf Steinbiß, Sabine Steppat, Carola Veit (SPD) und Fraktion, der Abgeordneten Viviane Spethmann, Dietrich Wersich, Christoph Ahlhaus, Heiko Hecht, Ralf Nied-mers, André Trepoll (CDU) und Fraktion, der Abgeordneten Antje Möller, Farid Müller, Olaf Duge, Dr. Till Steffen, Jens Kerstan (GAL) und Fraktion, der Abgeordneten Finn-Ole Ritter, Katja Suding, Dr. Thomas-Sönke Kluth, AnnaElisabeth von Treuenfels, Robert Bläsing (FDP) und Fraktion, der Abgeordneten Christiane Schneider, Dora Heyenn, Norbert Hackbusch, Tim Golke, Cansu Özdemir (DIE LINKE) und Fraktion. Betr.: Erlass eines Hamburgischen Transparenzgesetzes (HmbTG). At , accessed 27 August 2018. Cucciniello, M., Porumbescu, G. A. and Grimmelikhuijsen, S. (2017). 25 Years of transparency research: Evidence and future directions. Public Administration Review, 77(1), pp. 32-44. DeGEval (2016). Standards für Evaluation. At , accessed 27 August 2018. Escaleras, M., Lin, S. and Register, C. (2010). Freedom of information acts and public sector corruption. Public Choice, 145(3-4), pp. 435-460. Grimmelikhuijsen, S., Porumbescu, G. A., Hong, B. and Im, T. (2013). The effect of transparency on trust in government: A cross-national comparative experiment. Public Administration Review, 73(4), pp. 562-571. Herr, M., Müller, C.E., Engewald, B. and Ziekow, J. (2018a). Transparenzgesetzgebung in Deutschland in der Bewährung: Erfahrungen einer Gesetzesevaluation. Die öffentliche Verwaltung, 2018(5), pp. 165-174. Herr, M., Müller, C., Engewald, B., Piesker, A. and Ziekow, J. (2018b). Das Hamburgische Transparenzgesetz in der Praxis. Ergebnisse der Gesetzesevaluation. Baden-Baden: Nomos. Herr, M., Müller, C., Engewald, B., Piesker, A. and Ziekow, J. (2017). Abschlussbericht zur Evaluation des Hamburgischen Transparenzgesetzes. [online] transparenz.hamburg.de. At , accessed 9 July 2018. Humborg, C.; Mayer, H.; Spelsberg, A.; Hüsgen, D. (2012). Informationsfreiheit aus zivilgesellschaftlicher Perspektive. Informationsfreiheit und Informationsrecht, Jahrbuch 2012, 39-58 Islam, R. (2006). Does more transparency go along with better governance? Economics and Politics, 18(2), pp. 121-167. Kleindiek, R. (2013). Machen Volksgesetzgebung und Transparenz unsere Demokratie besser? In: M. Bäuerle, P. Dann, A. Wallrabenstein, eds., Demokratie-Perspektiven. Festschrift für Brun-Otto Bryde zum 70. Geburtstag. Tübingen: Mohr-Siebeck, pp. 175-198. Lindstedt, C. and Naurin, D. (2010). Transparency against corruption. International Political Science Review, 31(3), pp. 301-322. Maatsch, A. and Schnabel, C. (2015). Das Hamburgische Transparenzgesetz. Kommentar. Berlin: Lexxion. Mecklenburg, W. and Pöppelmann, B. (2007). Informationsfreiheitsgesetz. Bonn: DJV. Michener, G. (2011). FOI laws around the world. Journal of Democracy, 22(2), pp. 145-159. Mueller, C. E. (2018). Evaluating freedom of information laws: Objectives, approaches, and practical considerations. At , accessed 27 August 2018. Neuman, L. and Calland, R. (2007). Making the law work: The challenges of implementation. In A. Florini, ed., The right to know: Transparency for an open world. New York: Columbia University Press, pp. 179-213. Piotrowski, S. J. and Rosenbloom, D. H. (2002). Nonmission-based values in results-oriented public management: The case of freedom-of-information. Public Administration Review, 62(6), pp. 643-656. Porumbescu, G. A. (2017a). Does transparency improve citizens' perceptions of government performance? Evidence from Seoul, South Korea. Administration & Society, 49(3), pp. 443-468. Porumbescu, G. A. (2017b). Linking transparency to trust in government and voice. American Review of Public Administration, 47(5), pp. 520-537. Relly, J. E. and Sabharwal, M. (2009). Perceptions of transparency of government policymaking: A cross-national study. Government Information Quarterly, 26(1), pp. 148-157. Riddell, P. (2013). Impact of transparency on accountability. In N. Bowles, J. T. Hamilton and D. A. Levy, eds., Transparency in politics and the media: Accountability and open government. London & New York: Tauris, pp. 19-30. Schoch, F. (2016). Informationsfreiheitsgesetz. München: C.H. Beck. Stockmann, R. (2008). Evaluation and quality management: Principles of impact-based quality management. Frankfurt: Peter Lang. Stufflebeam, D. L. (2010). Meta-evaluation. Journal of Multidisciplinary Evaluation, 7(15), pp. 99-158. Vadlamannati, K. and Cooray, A. (2016). Do freedom of information laws improve bureaucratic efficiency? An empirical investigation. Oxford Economic Papers, 68(4), pp. 968-993. SB Central European Public Administration Review, Vol. 16, No. 2/2018 Making Transparency Work: Experiences from the Evaluation of the Hamburg Transparency Law White, H. (2006). Impact evaluation: The experience of the Independent Evaluation Group of the World Bank. Washington D.C.: Independent Evaluation Group, World Bank. Worthy, B. (2010). More open but not more trusted? The effect of the Freedom of Information Act 2000 on the United Kingdom central government. Governance, 23(4), pp. 561-582. Central European Public Administration Review, Vol. 16, No. 2/2018 89 Christoph E. Mueller, Bettina Engewald Annex: List of interview partners Authorities bound by the HmbTG: Senatskanzlei Hamburg (Hamburg Senate Chancellery) Behörde für Umwelt und Energie Hamburg (Hamburg Ministry for the Environment and Energy) Behörde für Stadtentwicklung und Wohnen Hamburg (Hamburg Ministry of Urban Development and Housing) Finanzbehörde Hamburg (Hamburg Ministry of Finance) Bezirksamt Hamburg-Mitte (District Office Hamburg-Mitte) Indirect public administration: Handelskammer Hamburg (Hamburg Chamber of Commerce) Hamburg Port Authority Businesses bound by the HmbTG: Gebäudemanagement Hamburg GmbH Hafencity Hamburg GmbH Members of the advisory board of the former project "Implementation of the Hamburg Transparency Law": Chaos Computer Club Hamburg Transparency International Hamburg Hamburgischer Beauftragter für Datenschutz und Informationsfreiheit (Hamburg Commissioner for Data Protection and Freedom of Information) Fachliche Leitstelle Transparenzportal (Technical Control Center for the Transparency Portal) 90 Central European Public Administration Review, Vol. 16, No. 2/2018