From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall
Efficiency of an Organization
Aurelien CONRAUX, Dr.
Head Records Manager, National Library of France/Bibliotheque nationale de France, Quai Franfois Mauriac,
75706 Paris Cedex 13, France e-mail: aurelien.conraux@bnf.fr
From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization
ABSTRACT
The records managers of the National Library of France (BnF) developed in 2011 a methodology to analyze and compare the information management practices of the Library's departments. All types of data are considered, both paper and digital, yet the method was specifically created to address the huge number of digital information circulating within the staff on a daily basis. When processes in an organization are not readily defined, it is very often the case that the records manager may serve as an excellent analyst and provide useful insider tips on what could be improved if need be. This includes domains where it is not generaly assumed that archivists or records manager may contribute. Why is that so? The more data and documents produced in an organization, the more useful a knowledgeable records manager will be to identify the practices that slow teams or individuals down: a lack of written instructions to define roles and responsibilities, problems in information-sharing or evaluation, cooperation between team-members, security problems or IT needs. It was nevertheless considered difficult for the records manager to cooperate with executives or other support teams such as Human Resources or the IT department, because analysis of different teams wasn't standardized or independent of the functional characteristics of the organizational unit considered. Starting from highly specific questions, the BnF's questionnaire is a practical tool to address those issues. I provides an easy-to-understand, business-independent diagram that can be used by the department to improve its daily operations, while also providing high-quality information useful to optimize knowledge management policies or software/hardware renewals at the Library level.
Dall'informazione al miglioramento: come una verifica sulla gestione dell'informazione elettronica possa contribuire all'efficienza globale di un'organizzazione
SINTESI
I gestori documentali della Biblioteca Nazionale di Francia (BNF) hanno sviluppato nel 2011 una metodologia per analizzare e confrontare le pratiche di gestione delle informazioni dei servizi della Biblioteca. Sono stati presi in considerazione tutti i tipi di dati, sia in formato cartaceo sia digitale, ma il metodo e stato creato appo-sitamente per affrontare l'enorme numero di informazioni digitali che circolano fra il personale su base giorna-liera. Quando i processi di un'organizzazione non sono chiaramente definiti, e molto spesso il gestore documen-tale puo servire da eccellente analista e fornire utili suggerimenti su quello che potrebbe essere migliorato laddove necessario. Cio include settori in cui non e generalmente accettato che gli archivisti o i gestori documentali possano contribuire. Perche e cosi? Piu dati e documenti vengono prodotti in un'organizzazione, e piu utile sara avere un esperto gestore documentale in grado di identificare le pratiche gruppi o individui: mancan-za di istruzioni scritte per definire ruoli e responsabilita, problemi di condivisione delle informazioni e la valu-tazione, cooperazione tra i componenti della squadra, problemi di sicurezza o bisogni informatici. Si e tuttavia ritenuto difficile per il gestore documentale il collaborare con i dirigenti o gruppi di supporto come le risorse umane o il reparto informatico, perche l'analisi dei diversi gruppi non e stata standardizzata o resa indipenden-te dalle caratteristiche funzionali dell'unita organizzativa considerata. A partire da domande molto specifiche, il questionario della BNF costituisce uno strumento pratico per affrontare tali questioni. Fornisce un diagramma di facile comprensione e libero da vincoli economici, che puo essere utilizzato dal dipartimento per migliorare le proprie attivita quotidiane, ed al tempo stesso capace di fornire informazioni di alta qualita utili per ottimiz-zare le politiche di gestione delle conoscenze o i rinnovi del parco software/hardware a livello della biblioteca.
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
Od informacije do napredka: kako lahko revizija upravljanja z elektronskimi informacijami pripomore k splošni učinkovitosti organizacije
IZVLEČEK
Arhivarji Francoske nacionalne knjižnice (BnF) so leta 2011 razvili metodologijo analize in primerjanja praks informacijskega upravljanja med različnimi oddelki knjižnice. Pri tem so upoštevali vse vrste podatkov, tako v papirni kot digitalni obliki, s tem, da je bila metoda posebej ustvarjena za velike količine digitalnih informacij, ki vsakodnevno krožijo med osebjem. V primeru, da procesi v organizaciji niso natančno definirani, je lahko arhivar tisti, ki na podlagi svojih analiz posreduje koristne informacije o tem, kaj je potrebno izboljšati. To vključuje tudi področja, ki na splošno ne veljajo za prodročje pristojnosti arhivistov ali arhivarjev. Zakaj je temu tako? Večja kot je proizvodnja podatkov in dokumentov v organizaciji, tembolj bo izkušen arhivar uporaben pri odkrivanju praks, ki ovirajo skupine ali posameznike: pomanjkanje pisnih navodil za definiranje vlog in odgovornosti, problemi pri izmenjavi informacij ali ocenjevanju, sodelovanje med člani skupin, varnostni problemi ali potrebe informacijske tehnologije. Kljub temu se je zdelo težko, da arhivar sodeluje z odgovornimi ali drugimi podpornimi skupinami, kot so kadrovski oddelek ali oddelek za informacijsko tehnologijo, kajti analiza različnih skupin ni bila standardizirana ali neodvisna od karakteristik delovanja obravnavane organizacijske enote. Začenši z zelo specifičnimi vprašanji predstavlja BnF vprašalnik praktično orodje za razrešitev teh tem. Avtor predstavlja v prispevku lahko razumljiv, poslovno neodvisen diagram, ki ga lahko oddelek uporabi z namenom izboljšanja svojih vsakodnevnih operacij in ki hkrati omogoča visoko kvalitetne informacije, uporabne za opti-miziranje politike upravljanja znanja ali obnavljanja programske/strojne oprema na knjižničnem nivoju.
Le progres par l'information: comment un audit des pratiques de gestion de l'information electronique
peut contribuer a l'efficacite d'une institution
RESUME
Les records managers de la Bibliotheque nationale de France (BnF) ont developpe en 2011 une methodologie d'evaluation et de comparaison des pratiques de gestion de l'information pour les services de la Bibliotheque. Tous les types de donnees sont pris en compte, sur support electronique comme sur papier, meme si la methode visait particulierement a sa creation les masses de documents numeriques geres quotidiennement par le personnel. Lorsqu'une organisation ne peut definir facilement certains de ses processus, il arrive souvent que les records managers servent d'analystes et puissent fournir des conseils utiles sur les ameliorations possibles, dans des domaines meme ou on n'attend generalement aucune contribution des archivistes ou des records managers. Pourquoi? Plus le nombre de documents et donnees produits dans une organisation est eleve, plus un records manager actif peut utilement identifier les pratiques qui ralentissent les equipes ou les individus: un manque de definition ecrit des roles et responsabilites; des difficultes a partager l'information, a evaluer celle qu'on refoit ou a produire un travail collaboratif au sein d'une equipe; des problemes de securite ou des besoins informati-ques. Il restait cependant malaise pour le records manager d'echanger avec l'encadrement ou d'autres equipes supports comme les Ressources humaines ou sa Direction des systemes d'information, parce que son analyse des differents services n'etait ni standardisee ni independante des caracteristiques metiers de l'entite consideree. Le questionnaire de la BnF est un outil pratique destine a repondre a cette problematique. A partir de questions tres specifiques, il produit un diagramme facilement comprehensible et independant du metier du service. Ce-lui-ci peut l'utiliser dans une optique d'amelioration continue, mais il fournit egalement des informations pou-vant etre pertinentes pour optimiser la politique de gestion des connaissances de la Bibliotheque ou sa program-mation d'achat de logiciels ou de materiels informatiques.
introductlon
When an electronic records management program was started at the National Library of France in 2004, few people knew what results exactly it would yield. It was generally assumed -and the future proved it right- that moving documents from local hard drives to the main datacenter would provide better information security. Universal identifiers and state-of-the-art metadata would allow one copy of a document to be kept for the whole library instead of distributing thousands of files or paper copies, while versioning would allow that same document to be updated as regularly as needed by qualified contributors without ever losing the correct order of the records -and which was a valid one, and which only a draft circulated for comments. Centralization would decrease the volume of data stored. It would allow users to effectively have a paperless office, and they would get the same virtual workplace from any computer connected to the network.
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
Besides these expectations, they were few insights into the managerial consequences of electronic records management. There were very few similar programs in France at the time -the Library has more than 2.500 agents in seven locations-, and none was sufficiently mature to provide an extensive feedback. The head records manager and archivist who started the project, Catherine Dherent, assumed that users would master their informational needs and would thus know exactly when a document could pass to the next step in its lifecycle: i.e. deletion of perennial preservation in the dedicated electronic stack of the Library. After a couple of years the number of documents would be stable since new documents would be offset by the removal of the older ones. The records manager would be the controller of the system, but not a major active player.
Five years later
When I became head records manager of the Library, on January first, 2009, the situation was radically different. The electronic documents and records system had been deployed and running for four years, and although staff number has been steadily decreasing, more and more documents were poured into the databases: +18% in 2009, then +18% in 2010, +18% again in 2011. Though unfinished, 2012 promises to be a record year by about the same margin. Moreover, the same questions kept coming from users: "How should I know what to do with the tools I have? Can you come and help me define my needs for information governance?"
Clearly, training wasn't the solution: it was already in place. Newcomers were receiving a two-day special course on the practical use of the documents and records management portal and its related applications. Additional workshops had been created to accommodate special needs: people taking jobs with additional responsibilities or information management demands; personal assistants; top managements Handbooks, several leaflets, dedicated pages on the Intranet were giving answers to every technical know-how possibly needed.
From records manager to coach and auditor
A reassessment of the situation was made with my deputy, Matthieu Angebault. It came to three intriguing conclusions:
•	Although more than 200 interviews had been conducted in 2004 during the early phase of the records management program to define the overall classification scheme for the library, few referred to written processes for documents and records, and our visits to departments gave no clue that it had changed: knowledge sharing within the library was extensive, yet it was build on so many informal exchanges between high-level experts that teams rarely took time to formalize their document management processes.
•	The records' metadata were not precise enough to allow records managers to validate the user's lifecycle choices without questioning them. In many cases, agents would determine the same retention period -from two years to illimited preservation- for all their records, whatever their importance.
•	The new, two-hour workshop we had opened to provide intermediate management with a panorama of the information governance and tips for a better use of mails and mobile devices wasn't doing just that. Executives would come, then immediately afterwards ask us to give the same lecture during their next department team meeting, or to coach a team member assigned to setting up procedures within the department.
Something else became obvious: many targets for improvement could be identified by the records managers in the documents and records management processes those individual users, teams or departments simply couldn't spot. Either redundancies coi ldn't be seen from the inside, or the pace of day-to-day operations was preventing a thorough analysis. Our secondary assessment was that records managers could spot things no one else could, simply looking at the way records and documents were produced, filed, archived. Controls or external audit exists at the library. However they usually focus on one specific topic or process: production levels, management, costs or strategy. They use records to validate their analysis, but never flip the idea upside down : we realized improving information management practices could lead to overall improvement of the organization, and that identifying redundancies, weaknesses or possible upgrades in the use of paper or electronic information could be of use to management and teams, not only to the records managers. It wasn't our job to examine policies or strategies, but we could support the efficiency of their implementation.
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
It then became clear why the financial services of the Library, especially the accountancy team, were by far the best users of the electronic records system, rarely if ever asking for assistance yet always producing perfect records: their yearly audit and the internal control system were so focused about providing written evidence for every operation that work processes and records management practices had been streamlined together.
What we wanted to do was to provide the same benefits to other parts of the organization with a lighter touch: guidance and training rather than control.
The audit project
What we needed was a tool. What we lacked was time. Since it would be a pilot project, we set out to use an Excel spreadsheet, and floated the specifications of the project to the University de Picar-die Jules Vernes, which offers a specialized degree in records management. A senior student, Camille Cauvet, picked up the gauntlet and agreed to a four-month internship as her final credit before graduation. The team was complete.
Together, we refined the specifications of the intended results with our eyes on several aims:
•	Results shouldn't be punitive or single out individuals in an "academic" way. This was the key factor to success: ranking wasn't important, upgrading information governance was. We discarded the French 0 to 20 grading system and looked for another which could be positively associated with "room for improvement" rather than classical control -i.e. lists of errors, mistakes or unnecessary costs. We briefly explored a three-letter grading system to separately evaluate the use of technological tools, teams procedures and management, then decided setting up a credit-rating agency within a library just wouldn't work_ In the end, a presentation by Marie-Dominique Parchas, a preservation expert from the Service interministeriel des Archives de France (SIAF), convinced us than a radar chart was the missing link: it would show graphically the equivalent to a dozen grades in one diagram; it could be an easy support for debate; the shapes of radars from different teams would probably be widely different, meaning they could be juxtaposed without confronting a "good one" versus an "bad one".
•	It was necessary to ask a wide area of questions, even though we knew many wouldn't be applicable to all parts of the organization: we needed to filter out non-answers in order to get comparable graphs from teams as different in their daily jobs as accountants, IT engineers or curators.
•	Grading had to be quick and unbiased: we planned to use a limited number of grades, with fixed criteria associating process and risks for each notch of the scale.
•	Suboptimal, low-risk habits could be tolerated, while any act throwing a vital process into jeopardy had to trigger some alarm and be corrected first.
•	Additional questions or themes could be required in the future : the questionnaire itself had to be flexible and easy to understand, or at least have a handbook.
Methodology
The dream of an automated tool vanished after a few days effort: there was a whole base of statistical data to mine built into the electronic documents and records management system (EDRMS), which could provide many useful answers, but it wouldn't give any indication about neither the quality of management, nor processes, mailboxes, knowledge sharing or future projects. Moreover, quantity wasn't enough. We were determined by then to go for quality. We intended to look for behavioural patterns and for the information that wasn't even into the system.
A self-administered tool wasn't feasible either. Users wouldn't have access to some technical data -the size of their mailbox, the complete list of formats used, the standard deviation of the size of files, etc.- or wouldn't give answers to a written questionnaire if they would be clearly subpar.
In the end, we decided than a combination would be required. For the assessment of a specific team, some data would be extracted from the EDRMS, some would be asked to the IT department. It would then be completed by interviews by a record manager, who would be the one gathering all the answers and aggregating results into the desired categories. The advantage of this compromise was that
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
no limit existed any more to the number of questions we could investigate. Whereas standard questionnaires may approach up to forty questions, ours ended up checking up more than two hundred details. Each time we knew of a specific issue, we included it into the questionnaire even when it would of limited relevance -if any- to the majority. For instance, only a limited number of services use audio or video files as administrative records, and only top management is authorized to access files from the EDRMS from their smartphones. We were able to include those concerns into the standard questionnaire nevertheless, thereby eliminating the need for additional tools to deal with those highly specific issues. Drawbacks of course are the additional complexity of a Swiss army knife, as well as the ength of the questionnaire and the time required to complete a full assessment.
Implementation
Overview of the questionnaire
Each question receives a grade according to the following formula: 0 is a failure in a critical process; 2 means a failure in a non-critical process; 4 and 6 are average grades with low and fairly low associated risks. 8 stands for excellent: procedures are followed to the letter. 10 is perfect: procedures are followed to the letter and non-standard situations are assessed and dealt with according to the spirit of the records management policy, or reported to the person in charge.
If the auditor is grading directly, he's doing so according to explicit criteria laid out in the first tab of the spreadsheet -the question and answer tab. It is also the case when a drop-down menu is used to select the applicable answer. For some questions, one or two figures have to be entered into an "answer" cell because an additional calculation is done by comparison between the two. For instance, the number of documents and records produced by a department will be compared to the expected theoretical number deducted from experience or from equivalent unit. A .5 result will be a 0, .7 means 2, and so on. "Non applicable" answers are accepted.
Grades are then reported to a second tab where a preliminary report is issued. Answers related to a particular tool or issue are grouped into one of the forty categories we identified as relevant for information governance within the technical and managerial context of the National Library of France: use of templates, vital records, access rights management, coaching & training, use of the classification scheme, etc.
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
Non-average grades are highlighted to signal discrepancies or documents and records managements practices that would need immediate action.
Screen view of the preliminary report
For each set, grades receive a secondary calculation that factors in the level of risk associated with that grade, using a multiplier factor scale put in another tab. ^at extra step could have been avoided, yet we found it to be extremely useful for practical uses. It allows in effect to change the multiplier factor for each level of risk according to strategic priorities without having to modify anything else in the spreadsheet. If the records management team wishes to primarily address issues involving serious risks to part of the organization, it only has to set the multiplier for grades 4 and above to null: every deviation from an optimum area in the resulting radar char will signal an endangered process. On the opposite, if the beginning of an audit campaign doesn't reveal any major risks but a certain degree of laisser-faire that could have unforeseen consequences in the future, it is possible to increase the multiplier for average grades.
Results of the secondary calculation are then added up. Each sum represents for one category the uncorrected deviation from optimal information management. It needs correction because the more items checked in a category, the higher that sum will probably be -that is, the final score will be low if uncorrected. When seven questions are needed to evaluate the "use of the classification scheme" and only one for "back up copies", the first category is likely to accumulate minus points. Our next-to-last calculation filters out "non-applicable" answers and averages the results according to the number of answers: we get a figure for each category -on a scale of 0 to 100- that is entirely independent from the activity of the audited department.
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
Only one step remains: forty technical categories being too many, a final calculation aggregates them into thirteen clusters and presents the results in a "summary" tab, which is also a shortcut into the whole questionnaire process. It is possible to go into minute details for two or three of the clusters -those scores will be calculated by the spreadsheet- and enter by hand estimates of what the scores of the others would be, had there been time to complete the entire version of the assessment. Results won't be exact, but they might be accurate enough.
Scores in the summary generate a radar chart.
Screen view of a radar chart (random answers)
Clusters and their names have been chosen to represent users' needs:
•	"Context" is a control cluster that takes into account the size of the audited department, whether it operates on one or several sites, etc. It indicates whether interpersonal communication and coordination may be easily achieved among staff or not.
•	"Local guidance" represents the designation and availability of a network of correspondents for IT issues and filing, as well as the degree of guidance provided by management.
•	"General statistics" is more about quantity than quality.
•	"Adequate use of available storage spaces" examines whether documents, records and data are in their right place.
•	"Correct use of electronic mailboxes". The National Library of France has a strict policy about the use of mailboxes, and no professional information should stay there for long, because they are deleted when the agent leaves the Library for retirement or for a job in another organization.
•	"Correct use of applications and software" verifies whether staff knows the purpose of available applications for data or file management and how to use them efficiently.
•	"Quality of filing" checks whether the classification scheme of the Library is understood, whether names of files have been harmonized within a team and whether correct metadata have been added.
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
•	"Quality of documents management" is about templates, copies, versions, etc., and whether they are used to gain time and guarantee an adequate lifecycle management for records.
•	"Use of adequate electronic formats" is a cluster dedicated to the shape, weight and manoeuvrability of information: are text, picture or video files encoded in a format that we know how to preserve or manipulate, is the definition level -especially for pictures- too high or too low for circulating the information easily or re-using it as evidence ater on ?
•	"Knowledge sharing" checks the parts of the Knowledge management program sponsored by Human Resources that are based on documents.
•	"Collaborative work" is essential to both team leaders and records managers. It indicates whether team members or participants to a project share files and documents as needed, with sufficient metadata to retrieve the information later on.
•	"Security and confidentiality issues".
•	"Technological prospects" has been a late addition to the questionnaire. This indicator shows whether a team is likely to adopt new methods or devices in the near future that should be reported to the IT department for consideration. Its intention is to reduce the number of "rogue processes" that users deploy for excellent reasons but without considering their relation to existing technology or policies.
What the questionnaire is not is a standard evaluation of IT performance. Neither does it grade teams or management. It is entirely about metadata, files, documents, records and their related workflows. Yet its results have been useful to many because paper or digital information is part of -nearly-every aspect of an organization's life and because it has a concrete quality that other more abstract analysis lack.
Future prospects
Although the two hundred point questionnaire fell short from being the self-administered, easy-to-use tool we dreamed of at the beginning of the project, I consider it to be a small step towards a giant leap. Building a comprehensive generic questionnaire as a proof of concept first and then putting it a work in unexpected ways has already proved most useful in changing the records management team's vision of its role. A more complete understanding of how information governance is linked to management helps us to target more achievable results and to better coordinate our action with executives, who in return get concrete results from reengineering their information flows: better reporting, additional security, more efficiency, streamlined workflows, more agile communication between teams...
Support units such as Human Resources, IT or Records Management are integrating efforts about information management that used to be separate: from a user perspective, it also more efficient to be presented with small things to improve when a team is ready to evolve, one after the other, rather than with fully integrated sets of standards and systems when support is ready.
Although we know a lighter version is required to achieve sustainability in the long run, the questionnaire as it is provides a standardized analysis based on the requirements of the institution considered, yet independent of the functional characteristics of the organizational unit considered. Since it was finalized in the summer of 2011, the BnF spreadsheet and its handbook have been widely circulated outside the library through professional workshops and training sessions, and we expect that similar experiments in other institutions or adaptations of the questionnaire will result in feedback that can be used to improve this project.
SUMMARY
Drawing on the experience of the French National Library, where a comprehensive electronic records management program has been set since 2004, my aim is to present how a thorough examination of documents and data by an experienced records manager may contribute to the greater good, even in domains where it is not generally assumed that archivists may contribute. When processes in an organization are not readily defined, it is very often the case that the archivist may serve as an excellent analyst and provide useful insider tips on what could be improved if need be. Why is that so ? Very often, control or external audit only focus on one specific topic or process: production levels, management, costs or strategy, etc. Its aim usually being some sort of reen-
Aurelien CONRAUX: From Information to Progress: How an Audit of Electronic Information Management May Contribute to the Overall Efficiency of an Organization, 53-61
gineering, it consecutively misses small fragmented details that - were they to be put together- could pave an easy road to immediate improvement in many key areas. ^e more data and documents produced in an organization, the more useful a knowledgeable records manager will be to identify the "worse practices" that slow teams or individuals down: a lack of written instructions to define roles and responsibilities, problems in information-sharing or evaluation, cooperation between team-members, security problems or IT needs. Even when such information is gathered, it is however difficult for the archivist who "owns it" to be identified as a reliable resource of input and operational advice by the producers of by other support teams such as the human resources or the IT department, because his/her analysis is rarely standardized or independent of the functional characteristics of the organizational unit considered. ^e records managers of the French National Library (Bi-bliotheque nationale de France) developed in 2011 a methodology to analyze and compare the information management practices of the library's departments. All types of data are considered, both paper and digital, yet the method was specifically created to address the huge number of digital information circulating within the staff on a daily basis. Starting from highly specific questions answered a simple spreadsheet; this practical tool provides in the end an easy-to-understand, business-independent diagram that can be used by the department to improve its daily operations, while also providing high-quality information to optimize human resources policies or software/hardware renewals at the Library level.
Original scientific article Submitting date: 02.04.2012 Acceptance date: 30.06.2012