LONG- TERM C ARE – A CHALLENGE AND AN OP LONG-TERM CARE – A CHALLENGE AND AN OPPORTUNITY FOR A BETTER TOMORROW POR TUNIT Y F OR A BET Evaluation of pilot projects in the field of long-term care TER T OMORRO W MODEL DOLGOTRAJNE OSKRBE Pilotno preizkušeni pristopi za boljšo integracijo storitev dolgotrajne oskrbe Ljubljana, March 2022 ACKNOWLEDGEMENTS In addition to the authors, many people contributed to the creation of the text in front of you. The authors of the text are therefore extremely grateful to each of them for their valuable assistance in the implementation of the evaluation of the pilot projects of long-term care. We thank the employees of the pilot projects, especially the managers, coordinators and assessors, for their cooperation in providing a considerable amount of information, establishing contact with various stakeholders, connecting with users and informal carers, conducting surveys and, finally, for letting us into your work process believing that your experience will help create better system solutions. Special thanks go to users and informal carers. Without the insight into your personal experience, evaluation in this form would not be possible. Thank you for your openness, willingness and trust. We thank the representatives of municipalities from the project environments for sharing experiences, views and vision, the developer of the information system Aleja Soft, d. o. o., and providers of supportive technologies A.L.P Peca, d. o. o., MKS Electronic Systems, d. o. o., and Telekom Slovenije, d. d., for your participation in the preparation of data and for numerous explanations regarding your work in the project and beyond. We also thank the University of Ljubljana, Faculty of Health Sciences for all the information and explanations related to training for employees in pilot projects as well as the European Centre for Social Welfare Policy for providing valuable expertise and advice in defining and guiding the methodology and evaluation. Special thanks go to the contracting authority of the evaluation, the Ministry of Health, for cooperation, guidance, patience and constructive responses to the course and results of the evaluation. The idea of a publication came about during the Scirocco Exchange project as part of the knowledge transfer programme. We want to thank both the Ministry of Health and the Scirocco Exchange project for their support in developing the idea of disseminating the results of the evaluation, and for their financial support in publishing the evaluation. In addition to the authors of the present text, various associates and researchers from the Social Protection Institute of the Republic of Slovenia, the Faculty of Social Sciences of the University of Ljubljana and the Institute for Economic Research were involved in various evaluation activities of pilot projects in the field of long-term care. We are grateful to you for being a part of the team and that each of you, with your unique role and expertise, have made an important contribution to the text, which we believe is a good foundation for the development of long-term care in our country in the future. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 4 FOREWORD BY THE MINISTER OF HEALTH Long-term care comprises a set of measures, services and activities intended for persons who, due to illness, weakness arising from old age, injuries, disability, lack or loss of intellectual abilities, are for a long period or permanently dependent on the assistance of other people to perform activities of daily living and instrumental activities of daily living. All modern and responsible societies face the challenges of regulating long-term care systems. Systems that will respond flexibly to the needs of users and at the same time be stable in the long term, financially sustainable and will strengthen the development of community forms of care. Slovenia is one of the fastest ageing societies. As the population ages, the need for long-term care services increases. The development of new technologies, new methods of treatment, a better living environment and the awareness of the population about taking care of our health enable us to live better and longer. The ageing of the population is thus a reflection of the development of society, and the search for answers regarding appropriate assistance in periods when, due to illness, injury, old age or disability, we can no longer fully take care of ourselves is a reflection of social responsibility to every citizen. In 2017, the Ministry of Health took over the task of preparing a proposal for the Long-Term Care Act and implementing pilot projects in the field of long-term care. We took full advantage of the opportunity we received in Slovenia with the possibility of implementing a pilot project in the field of long-term care, which was co-financed by the European Social Fund. On one hand, we were able to test the mechanisms and procedures proposed for the future unified systemic regulation of long-term care and upgrade them so that they are as user-friendly and administratively non-burdensome as possible within the solutions provided by the Long-Term Care Act. On the other hand, as part of the activities involved in the “Implementation of pilot projects that will support the transition to the implementation of the systemic long-term care act”, we were able to provide beneficiaries with services they cannot access at home under the current regulation and verify whether these meet their needs and enable them to maintain the highest possible degree of independence. Activities involved in the implementation of pilot projects enable 5 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE beneficiaries to play an active role in the entire process, from planning to the provision of services. At the same time, the project activities confirmed the importance of investing in knowledge and strengthening the competencies of employees in the field of long-term care, not only because of the higher quality and safety of services provided to users but also because of knowledge and skills for protecting employee health. Finally, the project activities also confirmed the exceptional role of informal carers in the field of long-term care, as they represent an important complement to the services to be provided within the future uniform system of long-term care within formal services, so that beneficiaries with comparable needs under the same conditions will receive comparable services regardless of the environment in which they reside. The pilot project in the field of long-term care, coordinated by the Ministry of Health, has been completed. The results of the evaluation of the pilot project in the field of long-term care show that in Slovenia we need new solutions and answers to the needs of citizens in periods of life when they are no longer able to take care of themselves. The challenge of adopting a systemic law in the field of long-term care is behind us. This, however, is only the beginning of a huge amount of work that will enable the law to come to life in practice, and in all environments provide those in need with services that are high quality, safe and tailored to individual needs. The solutions proposed in the Long-Term Care Act (adopted 2021) have been verified within project activities coordinated by the Ministry of Health and provide the beneficiaries with the option to choose where and what services they want. They enable the active role of beneficiaries, strengthen support for informal care providers, strengthen the conditions to link health, social care and long-term care systems with the aim of continuous and integrated care. The solutions bring new services, including services to strengthen and maintain independence, enable citizens with comparable needs to access comparable rights and meet the wishes of the majority, to remain at home and in the circle of their social network woven over many years, despite various disabilities, even during the period of life when they are no longer able to take care of themselves completely, with diverse, high-quality and safe long-term care services provided within the public network. Janez Poklukar, Minister of Health EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 6 FOREWORD BY THE DIRECTOR OF THE SOCIAL PROTECTION INSTITUTE OF THE REPUBLIC OF SLOVENIA In developed western societies, the population is ageing, due to which the proportion of the older adults in the total population is increasing. Slovenia is no exception in this respect. In these societies, the concept of the welfare state has been formed, in accordance with which the state plays an important role in the economic and social protection of citizens. Due to the ageing of the population, the problem of caring for the older adults is becoming more and more acute. This framework also includes long-term care for that segment of the older adults who, for various reasons (illness, disability, mental health problems, etc.), need assistance and support in everyday life. Of course, it would be wrong to limit long-term care only to the medical aspect (how many days we will add to the life of a person), as the social aspect (how good those days will be) is also extremely important. There are many problems in establishing long-term care, from the lack of systemic regulation of the field today to ensuring the sustainability of the financial system tomorrow. Therefore, research in this area is essential. It is important for decision-makers to be aware of this, as only thus will they have the knowledge to establish a fair and sustainable long-term care system. The goal we have committed ourselves to is the realisation of Principle 18 of the European Pillar of Social Rights, which states: “Everyone has the right to affordable long-term care services of good quality, in particular homecare and community-based services.” The text in front of you is the result of monitoring the implementation of pilot projects by various contractors in the period from 2018 to 2020 in Celje, Dravograd and Krško. It was a demanding and large-scale innovation, in the framework of which tools and procedures for assessing eligibility for long-term care, the whole process and new long-term care services for people living at home in their home environment were tested in 7 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE pilot environments. The evaluation, in which researchers from the Social Protection Institute of the Republic of Slovenia, the Faculty of Social Sciences of the University of Ljubljana and the Institute for Economic Research participated, was equally demanding. Monitoring and evaluating the implementation of these projects has yielded numerous results that can be used as a tool in controlling the solutions for long-term care system regulation. Through the pilot project, we obtained a credible and appropriate evaluation tool in Slovenia, which, following the German model, was developed within the framework of the project “Preparation of bases for the implementation of pilot projects that will support the transition to the implementation of the systemic long-term care act” at the Social Protection Institute of the Republic of Slovenia in 2016-2017. The tool has been tested on almost 2,000 people in pilot projects. In the pilot environments, interest in e-care was very high. We believe that it needs to be developed systemically, a position which was actually reinforced by the Covid-19 epidemic. Despite the fact that the effects of social concepts such as the quality of life usually show up in the long term, we find that pilot activities have had a positive effect on users, especially in terms of improved health and well-being. New services have also reduced the workload of informal carers. Cooperation and networking is important both in the provision of services and research in the field of long-term care, as well as in the preparation of legal solutions. Legislation that systematically regulates long-term care is currently being drafted, but I believe that we will have to prepare at least one more study for the financial assessment of long-term care in Slovenia. Once this information is available, it will be up to the politicians to come together and adopt comprehensive, professionally sound and financially sustainable legislation in the field of long-term care and long-term care insurance. Mag. Barbara Kobal Tomc, Director of the Social Protection Institute of the Republic of Slovenia EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 8 REVIEWS Assistant Professor Nikolaj Lipič, doc. dr. Alma Mater Europaea ECM Long-term care is the central research axis of the present scientific monograph with the goal of presenting and updating the scientific and research findings gathered during the evaluation of pilot projects during the “Implementation of pilot projects supporting the transition to the implementation of a systemic law on long-term care” project, which ran between 2018 and 2020, and is designed to support the transition to the implementation of a systemic law on long-term care. Project activities were implemented in three closed pilot environments, namely in Celje, Dravograd and Krško. This monograph portrays long-term care as a multidimensional model with some integral components, which are discussed individually and which form a coherent whole to model systemic solutions at the micro (local), meso (regional) and macro (state) levels. They are the foundation for the formation and implementation of a socially acceptable and economically sustainable long-term care system in Slovenia, while the complexity of demographic, social and political, technological, and social and economic changes in the last three decades have also been taken into consideration as they represent a paradigm shift in long-term care in Slovenia. The monograph also takes into account a set of key milestones of societal development to date, the specificities and capabilities of the Slovenian health and social care system, and the characteristics of the socio-cultural environment in which long-term care is provided. The authors want to emphasise that we should not only observe the global development trends in the field of long-term care, but become key players in outlining future long-term care policies. The monograph presents a project development and innovation study in the field of long-term care in Slovenia. From a methodological point of view, it is worth emphasising that the study has a systematic and rigorous methodological framework, i.e. research approach based on both quantitative and qualitative research paradigms. The research phases follow a logical sequence. A concurrent triangulation design and a concurrent nested design are also used, which ensure a balanced contribution of both quantitative and qualitative research findings. A range of different relevant and up-to-date research methods ensures that the study as a whole is organised in a transparent and structured way. Primary and secondary data were used in the study, which creates a unique and extensive database in the field of long-term care in Slovenia that can be used to carry out a number of analyses that would meaningfully complement the collection of research results and insights in this research area. The data were collected using a variety of research methods, which are described in detail in the monograph both in terms of their 9 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 9 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM C purpose and the scope of the data covered, i.e. the size of the individual samples included in the study at each stage. The advantage of the study lies in its strict adherence to the key principles of scientific research – generalisability, objectivity, verifiability, validity and reliability. The researchers were guided by rigorous professional and ethics research standards in the field of health and social care. Epistemologically, the study describes the interplay between scientific theory, scientific methods and experience. The monograph uses the correct scientific terminology and language usage. The research findings are presented in a way that is comprehensible to scientific and professional communities as well as to the general public. It is suitable for everyone involved in the field of long-term care: decision-makers at all levels of decision-making (local, regional and national), experts and researchers, and coordinators, providers, users and relatives of long-term care services users. This is all the more important as it is the first comprehensive study in the field of long-term care in the Slovenian scientific setting. The comparison between the results of the study and the findings of a number of authors in the domestic and foreign professional and scientific literature deepens the understanding of the field and allows for international comparability. The monograph contributes to the understanding of the needs of the long-term care system in Slovenia and, in doing so, it is based on efficiency. Research innovation is reflected in the efficient implementation of methods, procedures and services. At the level of efficiency of methods, it focuses on the approaches of assess eligibility, on personal planning and the coordination of services, and taking into account the dynamics of teamwork in long-term care. The effectiveness of procedures is checked on the basis of the forms used, the procedures followed to claim the right to long-term care, the procedures for assessing eligibility, for personal planning, coordination and the provision of services, the integration of e-care and e-health, the drawing up of waiting lists and complaints procedures. The effectiveness of services is checked at the level of inclusive care services and services to maintain independence and the integration of support services. The key development objectives for the establishment of the long-term care system are focused on at least two goals at societal level. The first objective is to ensure the quality of life of all groups involved in the long-term care system, such as users, informal carers and employees in the long-term care system, thereby contributing to a more humane society and to social well-being. In this way, health inequalities can be overcome and the social EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 10 11 inclusion of different target groups can be strengthened, including long-term care services users. The second objective targets the transition to community forms of long-term care, which have to have a clear organisational structure, information and communication system, a local project council and a project team, as well as set cooperation protocols for the provision of inclusive care in the community. The monograph emphasises the user-centeredness of long-term care, which must become the goal of long-term care services. All these challenges and changes, according to the authors, can only be achieved by adapting policies and implementing reforms in the field of long-term care. The scientific monograph is an original scientific contribution to the field of long-term care research in Slovenia, as it deals with the research area in an interdisciplinary manner and undoubtedly contributes to the creation of optimal societal solutions in the field of the systemic implementation of long-term care at the national, regional and local levels through concrete research results and findings. It supports deinstitutionalisation and creates opportunities for the development of the concept of active ageing at individual, community and societal levels. The monograph’s integrated approach to long-term care also provides a challenge and an opportunity for closer cooperation between health and social care professionals. 11 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 11 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM C Mircha Poldrugovac, dr. med. Amsterdam UMC location University of Amsterdam, Department of Public and Occupational Health, Amsterdam, the Netherlands; Amsterdam Public Health research institute, Amsterdam, the Netherlands The monograph is the result of a detailed, professional and in-depth evaluation of pilot projects in the field of long-term care that took place between 2018 and 2020. The role of the pilot projects was to support the transition to the implementation of the systemic law in the field of long-term care. The complexity of regulation of this field is also reflected in the complexity of projects. Extensive evaluation is thus necessary so that the experience of pilot projects can be best used to introduce the changes brought by the envisaged law on long-term care. A key element of evaluation is the use of qualitative and quantitative methods. In addition to recognising the success of implementation of individual solutions, such an approach also enables understanding of the elements that were key to the success or failure of solutions and the reasons thereof. It is also exceptionally important to take into account a broad range of perspectives: service users, their informal carers, a broader range of stakeholders and providers of pilot projects, which are numerous, and which play different roles. Additionally, the value of the evaluation also comes from a comparison of data and experience that stem from three very different pilot environments. The results presented in the monograph reflect the methodology in that they are presented in an elaborate and nuanced way. For example, the reader finds that the initial concept of the evaluation and the project itself makes it impossible to evaluate the staffing requirements that changes to how long-term care is provided would bring. At the same time, the reader obtains a considerable amount of valuable information about the workload of pilot project providers and the suitability of the set time frames for the provision of services. The information is further expanded by analyses of interviews with providers that describe the challenges they faced in providing services as part of the pilot project. When reviewing the effectiveness of procedures, it is possible to determine which parts of the established procedures have proven to be useful and to what degree. In addition, we also find out how the procedures in the pilot environments were adapted in practice, which is a valuable starting point for improvements. Perhaps the most important issue from the societal aspect concerns the success of pilot projects in improving the long-term care of users. Readers find the results very encouraging: there is a noticeable improvement in the quality of life of users, as measured by the EQ-5D tool one year after the services piloted by the project started to be used. Satisfaction as expressed in interviews and surveys was also very high. In addition to EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 12 13 these findings, the authors also identified several challenges and the desire of many users for the intensity of services to be increased. This is valuable information for those who use and organise the long-term care system. Also exceptionally interesting are the findings regarding the experience of informal carers. A description of the situation itself is important for planners of the long-term care system: who are the informal carers, how often people in need of long-term care can rely on them, how much time informal carers dedicate to the person they care for, what tasks they perform, etc. However, through the project, the authors also offer us an insight into how long-term care services in a pilot can affect informal carers. An interesting finding is that many factors indicate the objective relief of informal carers through the project, which however did not reflect on the subjective burden of these persons. Such a finding represents a recommendation for long-term care planners to pay extra attention to the role of informal carers. The authors of the monograph also clearly present the limitations of the evaluation and its findings. For example, there are clearly identified cases in which the small sample size limits the possibilities or prevents statistical processing of data, for example in relation to the use of certain support technologies. It is also pointed out that the initial measurements, which are expected to refer to the situation before the start of the project, were actually performed after the very beginning of the project. Regardless of the reasons, which are of an administrative-organisational nature, it is important that the reader is aware of these restrictions. The authors did not limit themselves to a narrow view of the methodological limitations of the research process, but also asked themselves about the broader purpose of the pilot project and its evaluation. In this respect, two types of goals in particular were identified, 13 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 13 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM C which were not envisaged in the evaluation: financial impact assessments and short-term and long-term consequences on total expenditure. The possibility of performing such calculations is largely conditioned by a comprehensive overview of services related to the long-term care received by users. The authors point out that the concept of the project did not envisage monitoring the implementation of existing services that were not part of the pilot, although they will be part of the regulation of long-term care in the future. It is valuable to recognise such shortcomings because this acquaints planners with the existing unknowns and can direct future activities. I would recommend anyone who thinks they can influence the long-term care system in Slovenia to read this monograph. The Ministry of Health commissioned the evaluation and used it to draft the new law. However, this does not mean that it is the only institution for which the results are important and interesting. Any stakeholder who wants to make a constructive contribution to creating solutions for the modernisation of the long-term care system and participate in its organisation and, in particular, its implementation, will find useful messages in this monograph. The monograph is also intended for all experts and, especially, researchers in the field of long-term care, who can better understand the starting points and plans for modernising the long-term care system in Slovenia. Experts are the providers of further research that will make it possible to make decisions that are supported by scientific evidence. In this sense, researchers are also stakeholders who make an important contribution to creating the long-term care system. At a time when opinion polls show that the trust of the general public in science is at a low level, a monograph that elaborately shows the possibilities and limitations of science-based evaluation is an important contribution to a high level of public debate in the field of long-term care. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 14 50 TABLE OF CONTENTS 58 60 2 ACKNOWLEDGEMENTS 4 FOREWORD BY THE MINISTER OF HEALTH 61 6 FOREWORD BY THE DIRECTOR OF THE SOCIAL PROTECTION INSTITUTE OF THE REPUBLIC OF SLOVENIA 8 REVIEWS 64 66 18 LONG-TERM CARE – A CHALLENGE AND AN OPPORTUNITY FOR A BETTER 67 TOMORROW 21 CONTRIBUTION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE WHEN PLANNING SYSTEM SOLUTIONS IN THE REPUBLIC OF SLOVENIA 23 Introduction 74 23 Approaches to the arrangement of long-term care 77 27 Long-term care in the Republic of Slovenia 29 Implementation of pilot projects, which will support the transition to the 79 implementation of the systemic act on long-term care 30 Discussion with key messages 81 32 Bibliography 83 83 33 EVALUATION OF PILOT PROJECTS AND METHODOLOGY 36 Introduction 36 Concept of evaluation of pilot projects Objectives of the evaluation Type of evaluation Research plan and timeline Establishing methodology and important stakeholders in the evaluation process 92 41 Implementing evaluation by means of mixed research methods 96 Quantitative instruments and data Data from the information system 97 Questionnaires for applicants and users 99 Questionnaires for informal carers 101 Questionnaires for stakeholders 102 Questionnaire for employees Quantitative data analysis 15 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 50 Qualitative instruments and data Semi-structured interviews Focus groups Personal plans and annexes to personal plans Reports by employees on activities as part of the pilot projects in the field of long-term care Democratic forum with assessors Minutes and other qualitative material 58 Discussion with key messages 60 Bibliography 61 TRANSITION TO INTEGRATED LONG-TERM CARE BY ESTABLISHING A SINGLE ENTRY POINT, INTEGRATED CARE TEAM AND CONNECTING STAKEHOLDERS 64 Introduction 66 Methodology 67 Results Establishment of single entry points Employing a long-term care coordinator and establishing an integrated care team Networking and cooperation of stakeholders in the environment 74 Discussion with key messages 77 Bibliography 79 FROM APPLICATION TO SERVICE: EXPERIENCE OF PROCEDURES IN PILOT PROJECTS 81 Introduction 83 Methodology 83 Results Exercising the right to long-term care Eligibility assessment Personal planning and coordination of long-term care Provision of long-term care services E-care and e-health Waiting list Complaint procedure 92 Discussion with key messages 96 Bibliography 97 ASSESSMENT OF ELIGIBILITY FOR LONG-TERM CARE 99 Introduction 101 Methodology 102 Results Characteristics of eligibility assessment Assessors’ experience with eligibility assessing EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 16 Suitability of classification in eligibility categories 187 112 Discussion with key messages 115 Bibliography 189 192 117 PERSONAL PLANNING AND COORDINATION IN LONG-TERM CARE: IDENTIFYING NEEDS AND PLANNING CARE TOGETHER WITH THE USER 193 120 Introduction 121 Methodology 195 122 Results 195 Long-term care coordinator and long-term care coordination 196 Personal planning and personal plan Living conditions – a sufficient framework for comprehensive identification of user needs? 207 Goals and implementation plan hand in hand 210 134 Discussion with key messages 139 Bibliography 211 141 TEAMWORK AND INTEGRATION OF STAKEHOLDERS AS THE 213 FOUNDATIONS FOR ENSURING INTEGRATED LONG-TERM CARE 214 143 Introduction 215 144 Methodology 144 Results Team dynamics in pilot environments Organisational climate and employee satisfaction 224 Cooperation and provision of mutual support among the employees 226 Integration with other important stakeholders in the community 155 Discussion with key messages 227 159 Bibliography 229 161 IMPLEMENTING AND STRENGTHENING LONG-TERM HOME CARE SERVICES 229 163 Introduction 230 164 Methodology 166 Results Providing long-term care services in users’ homes 232 User satisfaction with new services to maintain independence 239 Identified benefits and effects of receiving new services 176 Discussion with key messages 241 178 Bibliography 247 179 PERCEPTION OF THE USE OF ASSISTIVE TECHNOLOGIES 181 Introduction 182 Methodology Presentation of a sample of e-care users and the intervention process Presentation of a sample of e-health users and the intervention process Introducing e-care and e-health services 17 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 187 Results Identified effects of e-care use 189 Discussion with key messages 192 Bibliography 193 HOW PILOT PROJECTS CONTRIBUTED TO THE QUALITY OF LIFE AND THE STATE OF HEALTH OF USERS 195 Introduction 195 Methodology 196 Results Who were the users of the services in the pilot long-term care projects? How has the quality of life of users changed after the pilot activities? 207 Discussion with key messages 210 Bibliography 211 CARE FOR THOSE WHO CARE: STUDYING THE QUALITY OF LIFE OF INFORMAL CARERS 213 Introduction 214 Methodology 215 Results Who are the informal carers, how do they provide care and how much are they burdened? How pilot activities have affected the lives of informal carers 224 Discussion with key messages 226 Bibliography 227 ELECTRONIC MANAGEMENT OF PROCEDURES AND SERVICES AND INFORMATION SYSTEM SUITABILITY 229 Introduction 229 Methodology 230 Results The OSKRBA ONLINE web application The OSKRBA MOBILE mobile application 232 Discussion with key messages 239 Bibliography 241 AUTHOR INDEX 247 SUBJECT INDEX EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 18 LONG-TERM CARE – A CHALLENGE AND AN OPPORTUNITY FOR A BETTER TOMORROW A monograph, “Long-term care: A challenge When writing, the authors usually used the and an opportunity for a better tomorrow,” was so-called set terms or concepts, while knowing that drafted as part of the “Evaluation of pilot projects their suitability would have to be considered in the in the field of long-term care,” which was carried future. We tried to use Slovenian terms and avoid out by the Social Protection Institute of the Republic foreign ones as much as possible. For the most part, of Slovenia, the Faculty of Social Sciences of the we succeeded. Certain terms, such as evaluation, University of Ljubljana and the Institute for activities and coordination, were kept either due to Economic Research in the 2019–2020 period. Within the established use within the projects (evaluation, the European cohesion project, i.e. the Community- activities) or because no suitable synonyms existed based long-term care model, the evaluation was in Slovenian (e.g. coordination, integrated care). commissioned by the Long-Term Care Directorate Some terms were used only in places where of the Ministry of Health in order to prepare a original texts were cited. For example, the term general assessment of pilot projects with which “pokretnost” (mobility) was only used when the pilot environments could test new methods, referring to the statements from the validated procedures, mechanisms and services in the field of questionnaire; in other cases, the term “pomičnost” long-term care, while the key general objective is for (mobility) was used. If it was assessed that incorrect the findings of the evaluation to help create better understanding may occur, the term or concept was solutions regarding long-term care and possible further explained (e.g. assistive technologies). projections of the future long-term care system. We also observed the differences between The monograph was prepared in cooperation the terms applicant, beneficiary and user. The between the evaluator and the contracting term “applicant” refers to the person completing authority of the evaluation, each with their roles an application to participate in the project, the and perspectives. During the writing, the authors “beneficiary” is a person who was assessed and again agreed that a dictionary of terms relating is eligible for long-term care services within pilot to long-term care is needed in Slovenia, as was projects as per the assessment, and the “user” suggested by a number of initiatives coming from is a person who actually participates in the various sides. In expert circles, we frequently service implementation within the pilot projects. encounter various uses, ascription of meaning Nevertheless, we sometimes found ourselves in a and understanding of individual terms which, to dilemma about which term was more suitable. a great extent, are the results of long-term care For a number of years now, discussions have fragmentation and its interdisciplinarity. After a been taking place in national and international recent adoption of the Long-Term Care Act, the expert circles about how to address people urgency and opportunity to unify and consider a above the age of 65, so as not to discriminate and uniform and current long-term care terminology is stigmatise. The authors of the monograph decided even more evident. Such a terminology would be to use the phrases “older people” or “older adults,” clear, inclusive and, above all, enable unambiguous which we believe neutrally address the relevant communication of all stakeholders and more population and do not create a distinction on the efficient functioning of the complex and broad field basis of chronological age. Nevertheless, when of long-term care. reading the text, a reader will also come across 19 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE terms such as “an old person” and “the elderly,” description of individual chapters are usually cited particularly in sections where other authors’ works in this chapter. were cited. Although, we wished to maintain a The third chapter, Transition to integrated neutral position, we certainly did not fully succeed long-term care by establishing a single entry point, throughout the entire text. integrated care team and connecting stakeholders, We tried to capture as many themes as addresses the process of establishing single entry possible that were addressed by the evaluation points in pilot projects as one of the starting points of pilot projects regarding long-term care. It was of the integrated approach and describes the course impossible to fully utilise and contextualise all and characteristics of employing key personnel themes or aspects and data in this monograph, in pilot projects, especially the long-term care but they remain a rich source of information for coordinator and the integrated care team. Because further analyses and research. The themes selected the cooperation of various stakeholders at the local were assessed as being the most relevant for the level is important for integrated implementation first publication of results and the monograph was of long-term care, this chapter discusses the forms thus divided into twelve main chapters. As the of organisational cooperation and integration monograph was written before the Long-Term Care established in the pilot projects. In the fourth Act was adopted, we accordingly refer to different chapter, From application to service: Experience variants of the proposed act in individual chapters, of procedures in pilot projects, the procedure is most frequently to the act proposed in 2021. presented as it was developed and tested in pilot In the first chapter, Contribution of pilot projects. All the main steps in the procedure are projects in the field of long-term care when planning described, from completing the application for system solutions in the Republic of Slovenia, the the assessment of eligibility to long-term care to authors present the challenges of several decades of the inclusion in long-term care and the receipt of attempts to provide a uniform system regulation in services. Special emphasis is placed on waiting lists long-term care in the Republic of Slovenia and the and complaints channels. fragmentation of the current arrangement of rights Subsequent chapters refer to work methods or services relating to long-term care and its vision. and techniques in long-term care. In the fifth They also discuss the concept and the model of pilot chapter, Assessment of eligibility for long-term care, projects in the field of long-term care, which were we first present the characteristics of assessing the subject of evaluation and their contribution eligibility, which was tested for the first time in the to the planning of system solutions, as anticipated field of long-term care in Slovenia. The experience in the Long-Term Care Act1. The presentation of of assessors with the eligibility assessment is evaluation design and its implementation with discussed, as a new professional profile of the the help of mixed research methods is provided assessor was tested in the pilot projects. The in the chapter, Evaluation of pilot projects and chapter ends with the assessment of suitability of methodology. All measuring instruments and classifying applicants in the category of eligibility research methods used during the evaluation for long-term care. Personal planning and the are discussed in this chapter and all types of data coordination of services are the topics of the next collected, including that which was not described chapter, Personal planning and coordination in in further detail and used in other chapters of the long-term care: Identifying needs and planning care monograph. This is a comprehensive methodology together with the user. The profile of the long-term review of the evaluation to which the reader care coordinator is introduced and their central returns time and again when reading other role in the project from the aspect of coordinating chapters, as the details in the methodological care and personal planning. A special emphasis is 1 Zakon o dolgotrajni oskrbi (Uradni list RS, št. 196/21). Retrieved from: http:/ www.pisrs.si/Pis.web/pregledPredpisa?id=ZAKO7621 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 20 placed on the personal plan, as the key document chapter, the two target groups monitored within for implementing and receiving long-term care. The the evaluation, i.e. the users and their informal elements of a personal plan are compared to the carers, were also important. It was expected that concepts of the method according to which training the pilot activities would have the greatest impact for the employees in pilot projects took place. In the on these two groups because the services or care chapter, Teamwork and integration of stakeholders were intended for them. In the tenth chapter, How as the foundations for ensuring integrated long-term pilot projects contributed to the quality of life and care, we describe the dynamics of teams established the state of health of users, the service users are in the pilot environments. Furthermore, we present first discussed, followed by a presentation about the organisational climate, employee satisfaction how their quality of life has changed after the pilot and cooperation between the employees in the activities. Similarly, in the chapter, Care for those sense of providing mutual social support, which who care: Studying the quality of life of informal is also linked to the quality of working life. At the carers, we discuss who are the informal carers, end of this chapter, we focus on the cooperation of how they care and what is their burden. Special the employees with other important stakeholders emphasis is placed on the assessment of how the in the local environment which, in addition to pilot activities impacted the lives of informal carers. cooperation within and between the teams, also The monograph ends with the chapter, represents one of the foundations of integrated Electronic management of procedures and services long-term care. and information system suitability, in which is The next two chapters deal with the services presented and assessed the information system provided at the users’ homes by means of the developed and used by the pilot environments pilot projects. In the chapter, Implementing and together with the information system developer for strengthening long-term home care services, all the needs of the pilot project implementation. services are presented, with the emphasis on new The monograph illustrates the complexity of services in the home environment provided by the pilot projects regarding long-term care and their pilot environments within the pilot projects. The extensive evaluation and presents numerous and new services were also examined through the lens significant results. The results presented and their of user satisfaction, usefulness and the recognised evaluation can thus be used as a significant tool for effects of receiving them. As part of the evaluation, political decision-makers and experts in the field we also monitored the introduction of assistive of social care and healthcare when transferring technologies, e-care and e-health in the pilot knowledge and seeking better solutions to bridge projects. The results of monitoring are presented in the gaps and challenges of the current arrangement the ninth chapter, Perception of the use of assistive of the long-term care system. The systemic technologies. We particularly highlighted the effects regulation of long-term care, the foundations of of user inclusion in e-care, in which significantly which were set with the adoption of the Long-Term more users were participating than in e-health. Care Act at the end of 2021, is certainly a challenge In addition to the employees and the quality for Slovenian society and simultaneously an of their working lives mentioned in the seventh opportunity for our joint better tomorrow. Mateja Nagode Social Protection Institute of the Republic of Slovenia Klavdija Kobal Straus Ministry of Health 21 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE CONTRIBUTION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE WHEN PLANNING SYSTEM SOLUTIONS IN THE REPUBLIC OF SLOVENIA Emilija Guštin Ministry of Health Anita Jacović Ministry of Health Klavdija Kobal Straus Ministry of Health Mojca Počič Ministry of Health EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 22 CONTRIBUTION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE WHEN PLANNING SYSTEM SOLUTIONS IN THE REPUBLIC OF SLOVENIA KEY MESSAGES The Long-Term Act has set the foundations for: ▶ uniformly and systematically regulated long-term care in the Republic of Slovenia, ▶ the citizens with comparable needs to have access to comparable rights, ▶ the beneficiaries to have the option to choose in what way they wish to exercise their right to long-term care (in the form of formal home care or formal care in an institution or cash benefits or a carer for a family member (in the case of beneficiaries with severe and the most severe limitation of independence or self- care abilities)), ▶ the beneficiaries to access comparable services in an institution or at home, ▶ the beneficiaries to access new services, services for strenghtening and maintaining independence and e-care services and ▶ ensuring that a higher proportion of public funds will be earmarked for long-term care, which will provide a financial relief to persons in need of long-term care, their family members and local communities. 23 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 22 Introduction Pillar of Social Rights Action Plan, the Member States committed themselves to further work As per the international definition, also in the field of long-term care. In addition to the summarised in the December 2021 adopted foregoing, the European Commission has been Slovenian umbrella act (Long-Term Care Act (Zakon reminding the Republic of Slovenia of the urgency o dolgotrajni oskrbi (Uradni list RS, št. 196/21)), of a systemic arrangement of the field of long-term long-term care encompasses an array of measures, care since 2013 (European Commission, 2020). The services and activities intended for persons who, urgency of a uniform systemic arrangement of due to the consequences of an illness, weakness long-term care has been highlighted by different related to old age, injury, disability, lack or loss of public stakeholders due to changed needs and intellectual ability over a longer period of time wishes of users, and social relations and the role of which is not shorter than three months, or persons family, which have changed over time, including who are permanently dependent on the assistance the expenditure growth linked to long-term care of other persons in performing the basic and (Flaker et al., 2008; Hlebec et al., 2013; OECD, 2017; instrumental activities of daily living. Ciccarelli & Van Soest, 2018; Mali, Flaker, Urek & All EU Member States are facing the challenges Rafaelič, 2018; Wagner & Brandt, 2018; European of a long-lived society and an increase in the need Commission, 2021; Nagode et al., 2021). In regard for long-term care, which are being addressed in to the aforementioned, it is not surprising that in various ways. Irrespective of the tradition and 2021, after more than twenty years of attempting to diversity of organising the field of long-term care, draft systemic solutions by various stakeholders in its financing and the manner of integration in other the field of long-term care, the Government of the social care systems, they nevertheless have certain Republic of Slovenia prepared and submitted to the common objectives, i.e. to ensure: legislative procedure a draft Long-Term Care Act 1. the equal accessibility and availability of long- (Vlada Republike Slovenije, 2021). The fact that long- term care services for everyone who needs it; term care, as an important field, is in need of reform 2. the high quality of long-term care services; is also highlighted in the national Recovery and 3. the long-term financial sustainability of the long- Resilience Plan (Služba Vlade Republike Slovenije za term care system, and razvoj in evropsko kohezijsko politiko, 2021). 4. sufficient numbers of adequately trained staff and conditions for high-quality work and support for informal carers (European Commission, 2015; Approaches to the European Commission, 2021). arrangement of long- Numerous international documents testify term care to the importance of long-term care. In 2010, the EU Charter of Rights and Responsibilities of Older When addressing the challenges relating People in Need of Long-Term Care and Assistance to long-term care, the EU Member States apply (European Commission, 2010) was adopted, which various approaches. This is reflected in the defined the rights and responsibilities of people various organisations, responsibilities in the in need of long-term care in ten articles. The right field of long-term care, methods of financing and to long-term care is determined in Principle 18 provision of long-term care. Certain countries treat of the European Pillar of Social Rights (European long-term care as an independent field of social Commission, 2017), which stipulates that everyone care, while in others the responsibility is divided has the right to affordable long-term care services between the healthcare and social care systems of good quality, in particular homecare and to varying degrees (Spasova et al., 2018; Institute community–based services. Through The European of Macroeconomic Analysis and Development, EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 24 2021; Vlada Republike Slovenije, 2021), which can age group, but the risk of dependency on the help pose a challenge when establishing and ensuring of another person for performing the basic and high-quality and safe long-term care services and instrumental activities of daily living increases may lead to a fragmentation of rights (Cès & Coster, with ageing (Figure 2). 2019), making it difficult to take a holistic approach On average, 10.8 per cent of people aged to a person in need of long-term care and to develop 65 and more receive long-term care in the OECD a long-term care system. In some countries, long- countries (OECD, 2019), which shows a 5-per cent term care is based almost exclusively on informal growth in comparison to 2007. The data reveals care, while there are other countries where formal that the proportion of severely impaired persons care prevails (Institute of Macroeconomic Analysis above the age of 85 in Slovenia is significantly and Development, 2021). Furthermore, significant higher than the average in the EU countries. On differences exist between countries regarding average, 8.8 per cent of all Slovenian citizens were the method of financing and the amount of funds severely impaired when performing activities earmarked for long-term care from public funds of daily living in 2019 (6.9 per cent was the EU and relationships between resources that are average) in all age groups; the proportion was intended for the health and social part of long-term exceptionally high for people above the age of care (MISSOC, 2021; OECD, 2021) As shown in Figure 85 (41 per cent) (Institute of Macroeconomic 1, the LTC costs are for the most part covered from Analysis and Development, 2021). In addition to the budget and social insurance in most countries. demographic changes, such as the rapid growth in For the most part, the EU Member States began the number of people above the age of 80, changed addressing the issue of long-term care in 1997 and social roles also impact the increase in the needs 1998 due to prominent demographic changes which for long-term care, which most frequently include not only affect the quality of life of all generations, the changed role of the family and the social role but also call into question the sustainability of the of women, increasing desires and expectations of existing social security systems as the likelihood of individuals for high-quality long-term care and, the need for long-term care increases with ageing. last but not least, the development and access to The latter is not only a challenge for an individual, various assistive technologies that enable people but the ageing population became a challenge for to stay in their home for the longest time possible the whole of society and it is thus not surprising despite various impairments (Colombo, Llena- that it was also recognised as one of the greatest Nozal, Mercier, & Tjadens, 2011). Considering challenges of the 21st century by the European the foregoing, it is not surprising that the field of Commission (Majcen, Eržen, & Stanovnik, 2015). long-term care has been granted a more visible Questions arise, due to the ageing population, place within the social policies of the economically about how to address the economic consequences developed countries in recent years as a result of resulting from a smaller proportion of the working demographic changes. population and an increasing proportion of persons To make long-term care accessible, attainable who depend on the social protection systems, as and available, a flexible response of countries to well as the growth in the need for health and long- the challenges of the ageing population is urgent, term care. The traditional approach, in which an including the development of new and more extended family took care of an older person, and efficient care models for persons in need of long- the industrial approach, which was composed of term care (Cylus, Figueras, & Normand, 2019). The unrelated sections of domestic and formal care and development of the field of long-term care not unconnected services of healthcare and social care, only results in the improved safety and quality no longer meet modern needs (Ramovš, 2020). The of long-term care services, but also generates need for long-term care is not limited to a specific additional jobs and demand for a broader array 25 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Figure 1: Expenditure structure for the health part of LTC by financing sources, Slovenia and EU countries, 2018. % 0 20 40 60 80 100 Government schemes Voluntary health care payments schemes Compulsory contributory health insurance schemes Household out-of-pocket payment Source: European Commission, 2021. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 26 Figure 2: Recipients of long-term care by age groups, Slovenia and OECD countries, 2017 Hungary 31 43 26 Estonia 42 34 24 Norway 43 22 35 Czech Republic 44 26 30 Netherlands 47 22 32 Slovenia 47 22 31 Sweden 47 24 29 Portugal1 50 33 16 Switzerland 51 27 23 OECD19 51 27 21 Germany 52 26 22 Israel 53 28 19 Luxembourg 53 22 25 Spain 56 17 26 New Zealand 57 25 18 Denmark 58 29 13 Finland 58 25 17 Korea 61 35 4 Austra 63 27 10 Japan 67 30 3 % 0 20 40 60 80 100 0-64 65-79 80+ Source: OECD, 2019. 27 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE of goods and services linked to age, including The regulation of long-term care was written the development and enhancement of the use of down as one of the objectives in numerous strategic information and communication technologies national documents, including the Resolution on (Corselli-Nordblad & Strandell, 2020). In recent the National Health Care Plan 2016–2025 “Together years, most EU Member States have made for a Healthy Society” (hereinafter: Resolution) adjustments to the existing arrangements relating adopted in 2016, which regulates the development to long-term care, particularly by establishing of healthcare at the general level. The adoption of coordination structures between healthcare and the Resolution was one of the important milestones social care, improving infrastructure and local and in establishing an integrated long-term care system, regional system management, and establishing as the Resolution is not only a politically binding enhanced supervision and new tools and standards act, but also a binding legal act based on law and for measuring and monitoring the quality of providing key guidelines for the preparation and long-term care. Reforms of cash benefits, the implementation of activities which will respond introduction of new social benefits, allowances and to the needs of a long-lived society. The Resolution relief, measures to establish a balance between determines general and specific objectives, activities professional and private life, training of long-term and measures relating to long-term care which will care providers and provision of respite care have ensure equal accessibility to high-quality and safe improved the situation of long-term care users and services, and the integrated and comprehensive their family members. The Member States have care of individuals while considering the changing also improved the conditions in the labour market needs of the ageing population. For the approach by increasing funds for staff employment, higher to the introduction of the new paradigm in the wages and better working conditions (European functioning of the long-term care system to be as Commission, 2021). The coronavirus pandemic also thoughtful as possible, the implementation of pilot pointed to the urgency of changes regarding long- projects relating to long-term care with testing of term care and better resilience of social protection the single entry point mechanism and coordinated systems (OECD, 2021, Sagan et al., 2021). discussion of long-term care users in the community was planned among the measures provided in the Resolution. The National Council of the Republic Long-term care in the of Slovenia (2016) expressed its support for the Republic of Slovenia gradual and pilot-tested introduction of changes. In 2016, supported by the European Commission The first initiatives for a uniform systemic and in cooperation with the Ministry of Health arrangement of long-term care in Slovenia and the Ministry of Labour, Family, Social Affairs were launched in 2002. The expert and political and Equal Opportunities, the National Council dialogue on the urgency of reforms regarding organised a wide-ranging exchange of views on long-term care thus took place over twenty years. the further steps required for the arrangement of In this period, the Ministry of Labour, Family, long-term care in Slovenia, in which practically Social Affairs and Equal Opportunities drafted the entire professional and other interested two versions of the Act. One was formed by the public participated, and adopted the conclusions Federation of Pensioners’ Associations, one by the which were taken into account to the greatest Association of Social Institutions of Slovenia, and possible extent when drafting the Long-Term two proposals were prepared by the Ministry of Care Act (2021). Other important documents Health (Nagode, Zver, Marn, Jacović, & Dominkuš, followed the Resolution, which helped the policy- 2014; Ministrstvo za zdravje, 2017; Vlada Republike makers in their efforts to realise the long-standing Slovenije, 2021). development objectives of establishing a uniform EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 28 long-term care system. One of these was the Active the decision of the Government of the Republic of Ageing Strategy (Government of the Republic Slovenia to transfer certain tasks related to long- of Slovenia, Ministry of Labour, Family, Social term care from the Ministry of Labour, Family, Affairs and Equal Opportunities and Institute Social Affairs and Equal Opportunities to the of Macroeconomic Analysis and Development, Ministry of Health, a reason behind the transfer 2017), which was adopted in 2017 and defines being the facilitated harmonisation of further the vision and main objectives when forming development of integrated long-term healthcare responses to the challenges arising from the and social care. With the transfer, the Ministry changed age structure of the Slovenian citizenry of Health also assumed the tasks of drafting and provides strategic guidelines and operational the Long-Term Care Act and coordinating the objectives based on four pillars. Among these is implementation of pilot projects co-financed by the the pillar, Independent, healthy, and safe life of structural funds of the European Union in the field all generations, which includes social protection of long-term care. systems, accessibility of healthcare services and As per the transferred tasks, the Ministry of long-term care, care for health, and a reduction of Health published a public call for the selection of health inequalities. operations, “Implementation of pilot projects that The expert and other interested public have will support the transition to the implementation of more or less intensively highlighted the urgency the systemic act on long-term care” (Official Gazette of establishing a uniform LTC system over the last of the Republic of Slovenia [Uradni list RS], No. twenty years. Furthermore, Slovenia has been 24/2018) in the 2017–2021 period and ensured the receiving country-specific recommendations coordination of the project activities that are the from the European Commission since 2013 with subject of the relevant monograph and the results calls to systemically regulate long-term care as of which contributed significantly to the planning the relevant reform is defined as one of the key of systemic solutions in the field of long-term care structural reforms necessary for the provision in Slovenia. In the relevant period, the Ministry of of the long-term sustainability of public finances Health submitted two proposals of the Long-Term due to an ageing population and unfavourable Care Act for public discussion; one in 2017 and demographic trends (European Commission, the second in 2020. The second draft of the Long- 2022). The Court of Audit, which in 2019 issued Term Care Act was adopted by the Government of the audit report entitled “Care for the elderly and the Republic of Slovenia on 17 June 2021 (Vlada those with physical or mental disabilities who Republike Slovenije, 2021) and the National are in need of assistance”, also drew attention to Assembly adopted it on 9 December 2021 (Zakon o the inappropriate regulation of long-term care dolgotrajni oskrbi (Uradni list RS, št. 196/21))). The (Računsko sodišče Republike Slovenije, 2019) content of the adopted Long-Term Care Act (2021) (hereinafter: Report). In the Report, which referred does not differ significantly from all preceding to the work of the Government of the Republic of drafts in the objectives pursued, but upgrades Slovenia, the Ministry of Labour, Family, Social them, especially in the sections highlighted by the Affairs and Equal Opportunities and the Ministry Court of Audit (2019), the Covid-19 pandemic and of Health between 1 January 2007 and 30 June the experience of other countries when establishing 2018, the Court of Audit of the Republic of Slovenia long-term care systems. concluded that the concern of the country in the The adopted Long-Term Care Act (2021) is audited field to ensure assistance to everyone who thus based on broadly supported baseline, the needs it was not satisfactory. findings of the audit of the Court of Audit (2019), Relating to the arrangement of long-term recommendations of the European Commission care, significant progress was made in 2016 with (2022) and good practices of other countries, 29 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE and addresses the following objectives with its interaction and simultaneously shortens the time solutions: needed by the employees to reach users (Eurofund, 1. define long-term care in detail; 2020). The Ministry of Health approached the 2. unify the legal bases governing the rights in the more flexible training of long-term care staff field of long-term care; with an appeal to the Institute of the Republic of 3. define content and the scope of rights and the Slovenia for Vocational Education and Training selection of long-term care services; by submitting a proposal in July 2021 to prepare 4. establish a uniform assessment mechanism to a new professional qualification of a carer in enter the LTC system; healthcare, social care and long-term care within 5. form a comprehensive, universally accessible, the healthcare professions. geographically and financially sustainable and In addition to the preparation of bases for a available long-term care system; uniform systemic arrangement of long-term care, 6. enable a beneficiary who so wishes to stay in 2021 was also an important year for determining their home environment with suitable support for the foundations of the national model for quality as long as possible; monitoring and service safety of long-term care 7. place individuals who choose the mode of long- providers. As per the commitments of the Resolution term care provision within their rights at the centre to establish a comprehensive system for the of the LTC system; monitoring and continuous improvement of the 8. manage the growing private funding of quality and safety of healthcare, the first national individuals, which increases the risk of poverty, model for monitoring the quality and safety of especially of the older population; healthcare in social care institutions or future long- 9. improve planning, managing and ensuring the term care providers was set up in 2021 (Bolčević quality, safety and efficiency of performing long- et al., 2021; Farkaš Lainščak et al., 2022). Quality term care as a public service; and safety monitoring of the healthcare service 10. establish effective public scrutiny in the field of and long-term care, system assessment and result performing long-term care. measurement are crucial for ensuring high-quality The Long-Term Care Act (2021) is a and safe services, implementing a safe working fundamental building block of the system and environment and recognising possible systemic will enable long-term care in Slovenia to be shortcomings, including the best practices for system accessible, available, safe and of high quality. implementation (NHS, 2017; Duffy, 2018; OECD, Further activities necessary regarding education, 2020a; Chadborn, Devi, Hinsliff-Smith, Banerjee, & labour, suitable rewarding of employees working Gordon, 2021; European Commission, 2021). in long-term care and the establishment of a methodology for monitoring long-term care quality at the national level should not be overlooked. Implementation of pilot To address the global shortage of staff in service projects, which will activities, particularly in healthcare, social care support the transition to and long-term care, opportunities to optimise the implementation of processes, transfer competences between various the systemic act on long- occupational groups and introduce services with term care the application of various assistive technologies or e-care services must be sought, as research shows On 13 April 2018, the Ministry of Health that the use of such services has a positive impact published a public call “Izvedba pilotnih projektov, on user satisfaction, their sense of connection and ki bodo podpirali prehod v izvajanje sistemskega safety in their home environment, promotes social zakona o dolgotrajni oskrbi” (Implementation of EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 30 pilot projects which will support the transition to from the eligibility assessment to the recording of the implementation of the systemic act on long- service implementation (Ministrstvo za zdravje, term care), (Official Gazette of the Republic of 2018). Slovenia [Uradni list RS], No. 24/18) (hereinafter: To test integrated service implementation, project). The implementation of the project, which one of the project’s requirements was that took place in the urban (the selected beneficiary the providers selected in the project form a was the Celje Health Care Centre), semi-rural consortium, i.e. they had to connect at least with (the selected beneficiary was the Koroška Care the providers of social care services assisting Home) and rural (the selected beneficiary was the families at home, providers of institutional care Krško unit of the Posavje Centre for Social Work) for the older people or providers of institutional environments, enabled the testing and calibration care for adults with mental and physical of the assessment scale for assessing eligibility disabilities in the public network, a healthcare for long-term care, the formation or management centre or reference outpatient clinics or a medical of cooperation protocols between various station in the project environment, providers of stakeholders, the identification of knowledge community care in the public network and the necessary for coordinated and comprehensive competent centre for social work. The condition care of users, the search for optimum solutions to form a consortium was set with the aim of addressing the needs and desires of people in need enhancing cooperation and integration between of long-term care, the acquisition of data not being the healthcare, social care and long-term care collected at the national level, and the formation (Ibid.) systems, which proved to be a unique and testing of new services which the users challenge requiring further discussion in the received free of charge during the pilot activities future in order to attain comprehensive and user- due to the project activities being financed from oriented long-term care, in the process of which the budget of the Republic of Slovenia and the the user will be an active partner. The necessity of European Social Fund. further discussion is also one of the conclusions The key objectives of the project were to test of the evaluation of pilot projects in the field of the key tools, mechanisms and services relating to long-term care. the implementation of long-term care by means of a comprehensive approach and a defined model of integrated, coordinated and user-oriented care, i.e.: Discussion with key 1. testing of tools and procedures for the messages assessment of eligibility for long-term care (application, assessment tool, personal and In 2021, Slovenia made significant steps implementation plan, informing of target public); forward regarding a uniform systemic regulation 2. testing of new services and integrated care of a of long-term care. With the adoption of the Long- user in their home environment; Term Care Act (2021), the bases were provided for 3. testing of new services and support mechanisms making long-term care accessible, available, safe for informal and formal care providers to carry out and of high quality, and so that the beneficiaries high-quality and safe care; with comparable needs will be able to access 4. testing of coordination mechanisms and the comparable rights. Several decades of attempts establishment of efficient coordination between to uniformly and systemically regulate the field social care and healthcare providers and the newly of long-term care, proposed acts and numerous established entry points to ensure integrated other materials generated in this period and the services for the user; possibility of pilot projects and their evaluation 5. testing of electronic documenting of procedures connected with implementation, which is also 31 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE the subject of the relevant monograph, represent their physical and mental abilities and thus realise important building blocks in the mosaic of their active role in private and social life to the solutions provided by the Long-Term Care Act greatest extent possible. Long-term care is a story (Ibid.) in order to respond to the needs of persons of new chances and opportunities to form various requiring long-term care. The results of the project new types of institutional care and enable holistic showed important conclusions regarding the care of a person in need of long-term care at selection and application of the assessment tool their home. It is an opportunity for new jobs and for assessing eligibility for long-term care, the a new opportunity for persons requiring long- formation or management of cooperation protocols term care, which is particularly evident from the between various stakeholders, the search for statement of a user participating in the project, “I optimum solutions addressing the needs of people can sometimes be a bit of a workaholic and I like it who require long-term care and their informal when the carer comes and we start working with carers, and the search for solutions enabling the tempo that suits me. She first told me what I persons eligible for long-term care to choose from was doing wrong because I was pursuing goals in various modes of long-term care, including the physiotherapy that worked for me as I was before, option of co-financing the e-care services and and that’s not right. I have to follow the goals access to new services for strenghtening and that are right for me now, she has taught me that maintaining independence. The findings and already, for example. I got excited immediately.” experience arising from the project are invaluable. Let us not forget that long-term care is a matter They confirm that, despite incomplete self-care of the present and of the future. It is a need of an resulting from an illness, old age or disability, individual and a responsibility of society that, in persons who need long-term care can live longer the period of life when we are no longer capable of in their home with suitably organised professional taking care of ourselves, no one is overlooked. long-term care and maintain or even improve EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 32 BIBLIOGRAPHY Bolčevič, S. Gorup, L., Jakac, S., Kobal Straus, K., Limonšek, I., Lipar, T., … Štro- -projektov-ki-bodo-podpirali-prehod-v-izvajanje-sistemskega-zakona-o- majer, D. (2021). Kakovost zdravstvene obravnave v socialnovarstvenih -dolgotrajni-oskrbi-st--303-1201816-ob-198618 zavodih – priročnik. Retrieved from https:/ www.gov.si/assets/ministrstva/ MZ/DOKUMENTI/Dolgotrajna/Kakovost-zdravstvene-obravnave-v-social- MISSOC. (2021). Comparative tables. Retrieved from https:/ www.missoc. novarstvenih-zavodih-prirocnik-december-2021-v2.pdf org/missoc-database/comparative-tables/ Cès, S., & Coster, S. (2019). Mapping long-term care quality assurance Nagode, M., Zver, E., Marn, S., Jacović, A., & Dominkuš, D. (2014). Dolgotrajna practices in the EU. Brussels: European Commission: European Social oskrba – uporaba mednarodne definicije v Sloveniji. Urad RS za mak- Policy Network. roekonomske analize in razvoj. Retrieved from https:/ www.umar.gov.si/ fileadmin/user_upload/publikacije/dz/2014/DZ_02_14.pdf Chadbon, N. H., Devi, R., Hinsliff-Smith, K., Banerjee, J., & Gordon, A.l. (2021). Quality improvement in long-term settings: a scoping review of effective Nagode, M., Črnak Meglič, A., Dremelj, P., Dolničar, V., Hvalič Touzery, S., strategies used in care homes. European Geriatric Medicine; 12: 17–26. Istenič, A., … Žakelj, M. (2021): Dolgotrajna oskrba – izziv in priložnost za boljši jutri. Evalvacija pilotnih projektov s področja dolgotrajne oskrbe. Ciccarelli, N., & Van Soest, A. (2018). Informal Caregiving, Employment Sta- Inštitut RS za socialno varstvo. Retrieved from https:/ www.gov.si/assets/ tus and Work Hours of the 50+ Population in Europe. De Economist 166, ministrstva/MZ/DOKUMENTI/Dolgotrajna/Dolgotrajna-oskrba-povzetek- 363–396 (2018). https:/ doi.org/10.1007/s10645-018-9323-1 -dec21.pdf Colombo, F., Llena-Nozal, A., Mercier, J., & Tjadens, F. (2011). Help Wanted? NHS. (2017). The Good Indicators Guide: Understanding how to use Providing and Paying for Long – Term Care. Paris: OECD. and choose indicators. Retrieved from https:/ www.england.nhs.uk/ improvement-hub/wp-content/uploads/sites/44/2017/11/The-Good-Indi- Corselli-Nordblad, L., & Strandell, H. (ed.). (2020). Ageing Europe: Looking at cators-Guide.pdf the lives of older people in the EU. Luxembourg: Publications Office of the European Union. OECD. (2018). Accounting and mapping of long-term care expenditure under sha 2011. Retrieved from https:/ www.oecd.org/els/health-systems/ Cylus, J., Figueras, J., & Normand, C. (ed.). (2019). Will Population Ageing AccountingMappingofLTC.pdf Spell the End of the Welfare State? : A review of evidence and policy options. Copenhagen: WHO. OECD. (2019). Health at a Glance. Retrieved from https:/ www.oecd-ilibrary. org/docserver/4dd50c09-en.pdf?expires=1644192824&id=id&accname=- Državni svet, Republike Slovenije. (2016). Strokovni posvet dolgotrajna oskr- guest&checksum=247033AA2DA126F45580B2EEB664C969 ba. Retrieved from http:/ www.arhiv.ds-rs.si/2012-2017/node/2319.html OECD. Health at a Glance: Europe 2020. Retrieved from https:/ ec.europa. Duffy, J. (2018). Quality Caring in Nursing and Health Systems: Implications eu/health/system/files/2020-12/2020_healthatglance_rep_en_0.pdf for Clinicians, Educators, and Leaders: Third edition. New York: Springer Publishing Company. OECD. (2020a) . The economics of patient safety Part III: Long-term care: Valuing safety for the long haul. Retrieved from https:/ www.oecd-ilibra- European Commission. (2010). European Charter of rights and responsibi- ry.org/docserver/be07475c-en.pdf?expires=1644204612&id=id&accname=- lities of older people in need of long-term care and assistance. Retrieved guest&checksum=79114C5044A05CF8565CB6C9A7FC9C54 (5 February from https:/ www.age-platform.eu/sites/default/files/European%20 2022). Charter_EN.pdf OECD. (2021). Health at a Glance 2021. Retrieved from https:/ www. European Commission. (2015). Long-term care – the problem of susta- oecd-ilibrary.org/social-issues-migration-health/health-at-a-glance-2021_ inable financing. Synthesis report. Retrieved from https:/ ec.europa.eu/ ae3016b9-en social/BlobServlet?langId=sl&docId=13514& Računsko sodišče Republike Slovenije. (2019). Revizijsko poročilo: Skrb za European Commission. (2017). The European Pillar of Social Rights in 20 tiste, ki zaradi starosti, duševne ali telesne prizadetosti potrebujejo po- principles. Retrieved from https:/ ec.europa.eu/info/strategy/prioriti- moč drugih. Retrieved from https:/ www.rs-rs.si/fileadmin/user_upload/ es-2019-2024/economy-works-people/jobs-growth-and-investment/eu- Datoteke/Revizije/2019/KAM/KAM-2_RSP_RevizijskoP.pdf ropean-pillar-social-rights/european-pillar-social-rights-20-principles_en Ramovš, J. (2020). Integrirana dolgotrajna oskrba. Ljubljana: Inštitut Anto-European Commission. (2022). Country-specific recommendations na Trstenjaka za gerontologijo in medgeneracijsko sožitje provide policy guidance tailored to each EU Member State. Retrieved from https:/ ec.europa.eu/info/business-economy-euro/economic-and- Resolucija o nacionalnem planu zdravstvenega varstva 2016–2025 »Skupaj -fiscal-policy-coordination/eu-economic-governance-monitoring-pre- za družbo zdravja« (Uradni list RS, št. 25/16) Retrieved from http:/ pisrs.si/ vention-correction/european-semester/european-semester-timeline/ Pis.web/pregledPredpisa?id=RESO102 spring-package_en Sagan, A., Webb, E., McKee, M., Greer. S. L., Karanikolos, M., & Williams, G. A. Eurofund. (2020) . Public services. Impact of digitalisation on social services. (2021). Health systems resilience during COVID-19: Lessons for building Retrieved from https:/ www.eurofound.europa.eu/sites/default/files/ back better. Retrieved from https:/ eurohealthobservatory.who.int/ ef_publication/field_ef_document/ef19043en.pdf publications/i/health-systems-resilience-during-covid-19-lessons-for-bu- ilding-back-better European Commission. (2021). 2021 Long-Term Care Report. Trends, chal- lenges and opportunities in an ageing society. Retrieved from https:/ Služba Vlade Republike Slovenije za razvoj in evropsko kohezijsko politiko. ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8396 (2021). Načrt za okrevanje in odpornost. Retrieved from https:/ www. eu-skladi.si/sl/dokumenti/rrf/01_si-rrp_23-7-2021.pdf Farkaš Lainščak, J., Grabar, D., Kobal Straus, K., Marušič, D., Poldrugovac, M., & Simčič, B. (2022). Kakovost in varnost v zdravstvu – priročnik za zdra- Spasova, S., Baeten, R., Ghailani, D., Peña-Casas, R., & Vanhercke, B. (2018). vstvene delavce in sodelavce. Ljubljana: Ministrstvo za zdravje. Challenges in long-term care in Europe. A study of national policies 2018. Brussels: European Commission: European Social Policy Network Flaker, V., Mali, J., Kodele, T., Grebenec, V., Škerjanc, J., & Urek, M. (2008). (ESPN). Dolgotrajna oskrba: očrt potreb in odgovorov nanje. Ljubljana: Fakulteta za socialno delo. Institute of Macroeconomic Analysis and Development. (2021). European Pillar of Social Rights – Slovenia 2000–2020. Retrieved from https:/ www. Hlebec, V., Filipovič Hrast, M., Kump,, S., Jelenc Krašovec, S., Pahor, M., & umar.gov.si/fileadmin/user_upload/publikacije/ESSP/2021/ESSP_splet.pdf Domajnko, B. (2013). Medgeneracijska solidarnost v Sloveniji. Ljubljana: Fakulteta za socialno delo. Government of the Republic of Slovenia, Ministry of Labour, Family, Social Affairs and Equal Opportunities and Institute of Macroeconomic Analysis Majcen, B., Eržen, I., & Stanovnik, P. (2015). Značilnosti starejšega prebi-and Development. (2017). Active Ageing Strategy. Retrieved from https:/ valstva v Sloveniji – prvi rezultati raziskave SHARE. Ljubljana: Inštitut za www.gov.si/assets/ministrstva/MDDSZ/Direktorat-za-starejse-in-deinstitu- ekonomska raziskovanja. cionalizacijo/strategije/Active-Ageing-Strategy-2017.pdf Mali, J., Flaker. V., Urek, M., & Rafaelič, A. (2018). Inovacije v dolgotrajni oskrbi: Vlada Republike Slovenije. (2021). Predlog zakona o dolgotrajni oskrbi. primer domov za stare ljudi. Ljubljana: Fakulteta za socialno delo. Retrieved from https:/ www.gov.si/assets/ministrstva/MZ/DOKUMENTI/ Dolgotrajna/ZDO_parafa.pdf Ministrstvo za zdravje. (2017). Predlog predpisa: Zakon o dolgotrajni oskrbi. Retrieved from https:/ e-uprava.gov.si/drzava-in-druzba/e-demokracija/ Wagner, M., & Brandt, M. (2018). Long-term Care Provision and the predlogi-predpisov/predlog-predpisa.html?id=7885 Well-Being of Spousal Caregivers: An Analysis of 138 European Regions. J Gerontol B Psychol Sci Soc Sci. 2018 Apr 16;73(4):e24-e34. doi: 10.1093/ Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih projektov, ki geronb/gbx133 bodo podpirali prehod v izvajanje sistemskega zakona o dolgotrajni oskrbi«. Retrieved from https:/ www.uradni-list.si/glasilo-uradni-list-rs/ Zakon o dolgotrajni oskrbi (Uradni list RS, št. 196/21). Retrieved from http:/ vsebina/2018002400003/javni-razpis-za-izbor-operacij-izvedba-pilotnih- www.pisrs.si/Pis.web/pregledPredpisa?id=ZAKO7621 33 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE EVALUATION OF PILOT PROJECTS AND METHODOLOGY Mateja Nagode Social Protection Institute of the Republic of Slovenia EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 34 KEY MESSAGES ▶ It is important for the evaluation that its planning starts at the right time: when the model that is the subject of the evaluation is being established, and in any case before the start of the intervention. ▶ It is important that several different stakeholders are involved in the evaluation, each with their own specific role: the contracting authority of the evaluation and the project, the consultative body (e.g. professional, research), project providers or employees, other important stakeholders from the local environment or the national level, participants or users in projects and their relatives, etc. Working and creating together can provide better conditions and circumstances and thus lead to better project results. ▶ Pilot projects are intended for testing the set solutions and creating new answers, pathways and good practices. Because such interventions are demanding, innovative and complex, it is recommended that as many different stakeholders as possible who are relevant to the tested field be involved in expert steering. In this case, in addition to the Ministry of Health and representatives of the pilot projects, at least the Health Insurance Institute of Slovenia, the Pension and Disability Insurance 35 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 34 EVALUATION OF PILOT PROJECTS AND METHODOLOGY Institute, the Ministry of Labour, Family, Social Affairs and Equal Opportunities, municipalities, etc. ▶ The evaluation of the pilot projects was based on mixed research methods (linking qualitative and quantitative methods), which requires more research effort, but at the same time provides the results with greater validity. It also provides a broad range of different types of data that can be processed and displayed in different ways even after the conclusion of the project. ▶ The evaluation enabled the use of a relatively new research method in Slovenia, a democratic forum, which proved to be a very useful tool precisely for such projects, so it is recommended that it also be used and tested in the future. ▶ The course of the evaluation was influenced by both the Covid-19 epidemic and the fact that the pilot projects were not completed at the same time. All this has led to greater flexibility in data collection and processing. Based on the experience of evaluation of the pilot projects, it can be concluded that significantly more time should be devoted to the final phase of the evaluation, and especially to the final coordination of data and data analysis, as well as the drafting of the report. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 36 Introduction with which the pilot environments could test new methods, procedures, mechanisms and services in Evaluation has today become almost an the field of long-term care, while the key general essential component of programmes and projects, goal is that the findings of the evaluation help especially development and pilot projects. create better solutions in the field of long-term However, there is no single answer to the question care and possible projections of the future long- of why evaluations are made at all. Parsons (2017) term care system in Slovenia. states that one of the frequent answers to this In this article, the concept and course of the question is “because we have to” or “because evaluation of the pilot projects in the field of long- evaluation is necessary”, and this is mainly term care is first presented, and then all research because evaluation is often a condition of project instruments that were used in conducting the funding and certain pre-reserved funds or is part evaluation are described in detail. of a contractual obligation. However, this “must do” argument is part of a much larger picture, and behind it is the fact that different objectives Concept of evaluation and needs are related to evaluations. In the case of pilot projects of pilot programmes, initiatives and activities, the reason for evaluation is probably a combination The pilot projects in the field of long-term of the following: preparation of an initial care were a complex intervention, as was their evaluation, checking whether the new idea works evaluation. The evaluation covered a large number (or does it meet the requirements?); assessing of objectives, included different target groups and whether it needs to be scaled up (for example, featured various research methods, instruments pilot programme) and whether it is cost-effective and data of both a qualitative and a quantitative to introduce (and what adjustments would be nature. In this chapter, the objectives of the needed); assessing whether the idea or procedure evaluation, type of evaluation, research plan, of intervention is transferable to other situations procedure for establishing the methodology and and in what circumstances (Parsons, 2017). key stakeholders in the evaluation are presented in A combination of these ideas has also more detail in individual sections. dictated the conditions, reasons and objectives of evaluation of pilot projects in the field of long-term care. As the group of experts and decision-makers Objectives of the evaluation was planning the intervention, i.e. implementation of pilot projects, they were also planning As part of the evaluation, many objectives2 their evaluation. Evaluation was therefore an were evaluated at four research levels: unavoidable and integral part of the pilot projects. It was separately funded, and it was external. A. Effectiveness of new methods As explained in the introductory chapter, it was ▷ To assess the suitability and applicability of the carried out by the Social Protection Institute of the selected assessment tool for assessing eligibility for Republic of Slovenia, the Faculty of Social Sciences long-term care. of the University of Ljubljana, and the Institute for ▷ To prepare proposals for amendments for Economic Research. personal planning and coordination of services in The purpose of the evaluation of pilot long-term care. projects in the field of long-term care was to ▷ To prepare proposals for amendments for prepare a general assessment of pilot projects teamwork in long-term care. 2 The objectives were already envisaged with the public procurement for evaluation. 37 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE B. Effectiveness of procedures in pilot projects Type of evaluation ▷ To prepare proposals for amendments to the procedures for assessing the eligibility for long- Depending on the content, it was a process term care, including an assessment of the options and outcome evaluation. Both types of evaluation for reducing bureaucratic burdens. played an equally important role in the final ▷ To prepare proposals for amendments for the evaluation of the pilot activities. Just as the effects entire long-term care process, from entry to the (and results) of the pilot projects are important, so provision of services. is the process itself, i.e. programme itself. At the same time, programme monitoring was carried C. Effects of the pilot project on people out. It was evaluated whether the planned pilot ▷ To develop guidelines for the provision of services activities were proceeding as planned, following that will enable quality of life for informal carers. the indicators from the public procurement for the ▷ To develop guidelines for greater support for implementation of the evaluation3. informal carers. Structural indicators were used to monitor ▷ To prepare guidelines for the quality working life how the pilot projects were set up and when. It of formal carers. was therefore monitoring of the basic structures in the project that enabled the projects to be D. Preparation of content and financial implemented in the first place: projections for the long-term care system ▷ establishing single entry points in the pilot ▷ To prepare proposals for the addition of new environments and employing an adequate number services in the home environment (integrated teams, of qualified assessors in the field of long-term care services for maintaining an independent life, etc.). (1 February 2019). ▷ To prepare guidelines for the introduction of ▷ Each pilot environment employs at least one assistive technologies in the home environment. qualified long-term care coordinator with precisely ▷ To prepare guidelines for the establishment and prescribed education and years of work experience placement of the long-term care entry point. (1 February 2019). ▷ To prepare proposals for amendments related ▷ Each environment establishes an integrated care to the electronic management of procedures and team consisting of precisely prescribed qualified services in the field of long-term care. professional profiles (1 February 2019). ▷ To prepare a possible projection of financial and ▷ A local project council and a local project team human resources in the long-term care system is established in each pilot environment (30 (with the provision of appropriate input data). November 2018). ▷ To prepare guidelines for the development ▷ Protocols on the participation of all stakeholders of organisational forms of cooperation and in the pilot environment were established (1 networking in the field of long-term care and with February 2019). other areas that will support the transition to ▷ All relevant stakeholders of an individual pilot community forms of care. project were informed about all planned activities as part of the pilot project (introductory meeting by The plan also included the possible 15 September 2018, then updated communication preparation of a projection of financial and human with the environments for the entire duration of resources for the long-term care system, although the project). the pilot environments did not systematically Process indicators were used to measure how monitor and collect the necessary input data, so the pilot activities were conducted. This part is this goal could not be evaluated. important for understanding the final results, as 3 Determined in Appendix I to the tender documentation. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 38 it explains how the processes in the projects were increased due to the proposed solutions (at the conducted and what influences the results: beginning and end). ▷ 70% of the existing adult users of rights in the ▷ Proportion of users with care plans that are field of long-term care in the pilot environments being implemented (continuous monitoring). are assessed by means of an assessment tool in ▷ Proportion of users who transferred to the first nine months (until the end of the pilot institutional care (continuous monitoring). activities). ▷ Proportion of users who were placed into ▷ The proportion of persons assessed within three another eligibility category after a repeated working days of the submission of the application assessment (continuous monitoring). (continuous monitoring). ▷ Proportion of users who have opted for e-care ▷ Proportion of assessed persons who will be services, number of interventions (continuous eligible for long-term care and who will meet with monitoring). a long-term care coordinator within three working Some indicators, mainly due to the lack of days after the assessment of eligibility for long- systematically collected required data, were not term care (continuous monitoring). evaluated during the evaluation: ▷ Proportion of persons who fail to meet the ▷ Proportion of persons who currently do not eligibility threshold and who are informed about exercise their rights in the field of long-term care the existing rights or care options in the field of who were assessed with the assessment tool (it social care and healthcare and about participation can be partly explained by other indicators, e.g. in the evaluation procedure (continuous proportion of users included in services). monitoring). ▷ At least 50% of registered informal carers take ▷ At least 80% of all long-term care beneficiaries part in training sessions organised for the duration are re-assessed after six months if they are still of the pilot activities. included in the pilot activities (every six months, ▷ Proportion of formal care users in terms of the continuous monitoring); place where the service is provided: at home or in ▷ Regular meetings between assessors at entry institutions (at the beginning and end of the pilot points and the expert team of the Ministry of activities; the objective is to increase the number of Health, at which information is exchanged about home care users by at least 10%). possible challenges at work, open issues, etc. (at ▷ Proportion of unplanned hospital admissions or least six times a year). hospitalisations (at the beginning and end of the ▷ Regular supervision of formal carers and pilot activities). employees at the single entry point. ▷ Number of trained long-term care coordinators ▷ Regular meetings between long-term care (at least three) (31 January 2019) (it was evaluated coordinators, care teams and assessors, at by the “Each employment employs at least which information is exchanged about possible one qualified long-term care coordinator with challenges at work, open issues, etc. (at least five precisely prescribed education and years of times a year). service” indicator. The latter is linked to each ▷ Electronic real-time management of logs and environment individually and thus provides better documentation on work tasks, the time required to information about the employment of long-term perform them and the possible workload. care coordinators in projects). The result indicators were used to measure During the evaluation, an initial report was the results of the pilot projects: prepared that, due to objective circumstances4 ▷ Proportion of users whose satisfaction with was not made ex-ante, but at a time when the the quality of life remains unchanged or has pilot activities and intervention were already 4 The evaluation project was initiated relatively late, i.e. after the pilot environments started with the intervention for users. 39 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE under way. Four process reports and one interim projects, perceived usefulness of services, etc.). The report were prepared as part of the intermediate net effects of the intervention could not be accurately (mid-term) evaluation. Process reports were measured, as causality could not be determined based mainly on the monitoring data, i.e. due to the large number of pilot activities. programme monitoring and process evaluation Four key time milestones were determined as of the implementation of pilot activities, while part of the research plan: the interim report was based on the presentation ▷ starting point M0 (January 2019). This is the of the current course of pilot activities and the point when users began to be included in the pilot first intermediate results of these activities by activities. This is the point “before”, i.e. upon entry individual pilot environments. The purpose of the in the project. intermediate evaluation was to become acquainted ▷ intermediate point M6, related to June 2019, with the course of pilot activities and thus the which is an important point of evaluation, as it possibility of reflection and influence on changing represents the period of the first six months of and improving the course of planned activities. user inclusion in the project. The experience of In accordance with the public procurement being included in the pilot activities can already for the evaluation, the final joint evaluation should be observed; be prepared after the end of the intervention for ▷ intermediate point M12, linked to December users and after the completion of all pilot activities 2019, which means one year of implementation (ex-post). Due to the situation related to the and monitoring of the pilot activities. Change can Covid-19 epidemic and the extension of all three already be evaluated, so it can be treated as an pilot projects, the evaluation was completed before “after” point; all pilot activities were completed, i.e. at a time ▷ final point M18/M20, linked to June/August when one pilot environment had already ceased 20205, which means more than a year and a half of implementing pilot activities (September 2020), implementation and monitoring of the pilot activities. while the remaining two were in the final stages. Change can be evaluated, so it can be treated as an “after” point; This is the final point of evaluation. As individuals became included in the project Research plan and timeline gradually, the timeline for each individual was unique. Each user therefore has “their” date for An experimental research plan was not the starting point (M0) and, consequently, also for possible, so the evaluation was based on a all the subsequent ones. For example, one joined comparison of the situations and results before the project in January 2019 (M0), another in May and after the pilot activities, thus identifying the 2019 (M0), and a third in June 2019 (M0). If all change that occurred during the implementation users had joined the project at the same time, for of projects. For example, how the quality of life of example in January 2019, the change for all could the key evaluation target groups has changed. In have been evaluated for a period of a year and a order to rationalise data collection and in order half (until the end of June 2020), but because users to avoid burdening respondents, some effects of were joining the project gradually, one year was the projects were measured only at the end of the determined as the shortest period for the “before- project (for example, what was the experience of after” monitoring, which is the minimum period users, employees and informal carers with the pilot for the monitoring of changes and effects6. The key 5 The pilot projects were originally expected to be completed in June 2020. The Dravograd pilot environment extended the contract until the end of September, and Celje and Krško until the end of December 2020. Accordingly, the observed period was extended until the end of August 2020. 6 Due to a significant delay in the introduction of assistive technologies in the pilot environments, a minimum period of six months was set for the monitoring of changes and effects. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 40 to the evaluation was therefore a comparison of important stakeholder, especially in establishing the situation at the time of inclusion in the project and steering the methodology. The guidelines and (M0) with the situation after one year of inclusion recommendations prepared in the methodological (M12). An important circumstance that arose manual Monitoring and evaluating integrated LTC during the project (between M6 and M18) was the models (Kahlert, Boehler, & Leichsenring, 2018) outbreak of the Covid-19 epidemic. with proposals for standardised questionnaires for individual target groups were mostly followed in the preparation and selection of measuring Establishing methodology and instruments. The centre also organised two important stakeholders in the workshops for the evaluator and the contracting evaluation process authority for the purpose of establishing the methodology and monitoring the evaluation of the Due to objective circumstances related pilot projects. to the public procurement procedure, the first In February and March 2019, measuring measuring instruments began to be prepared only instruments were selected on the basis of in February 20197, after which the evaluation plan recommendations and the procedures for was coordinated with the contracting authority8 obtaining a permit for their use were initiated. and in the period when the intervention in the They were adapted to the circumstances of the pilot environments was already being intensively pilot projects, and they were presented together implemented. The participation of two stakeholders with consents9 and other explanations (e.g. for was important for the evaluation during this period; whom the questionnaires are intended, how the contracting authority and the European Centre to implement them, etc.) in the Instructions for for Social Welfare Policy. surveying. Employees in the pilot environments and Initially, there was no specific protocol the contracting authority were acquainted with of cooperation and communication with the them at the training session on 15 March 201910. contracting authority, and the evaluators were By that time, around 293 people had been assessed not updated on the instructions for the pilot with the assessment tool in Celje, 68 in Dravograd environments, nor was there up-to-date information and 82 in Krško, which meant that the M0 point about all activities and events in the pilot projects. had already been “missed” for these people. The In June 2019, this deficit was bridged with an delay thus affected some baseline data (in cases agreement on further regular meetings with the in which assessors did not survey applicants who contracting authority twice a month. It was then had been evaluated before 15 March 2019, either), that the cooperation, flow of information and i.e. it contains data that is not really covered by M0 role of the contracting authority as an important (for those who were surveyed in March 2019, for stakeholder in the evaluation that is also responsible example, and evaluated in January 2019, the survey for ensuring access to data and information, was was delayed by three months). strengthened and started to perform its function. It would therefore have been ideal if the With its external consultative role, the methodology had been established before the European Centre for Social Welfare Policy was an start of the inclusion of users in the project, i.e. in 7 The evaluation team signed the evaluation contract with the contracting authority on 9 January 2019, and the contracting authority approved the evaluation plan on 4 February 2019. 8 The guidelines from Developing an Effective Evaluation Plan. Setting the course for effective program evaluation (CDC, 2011) were followed in the preparation of the evaluation plan. 9 All respondents in the evaluation also signed a consent that they are willing to participate in the survey for the purposes of evaluation. 10 The instructions for the M0 point were first prepared, after which they were upgraded several times, usually when it was time for a new intermediate point for the evaluation – M6, M12, M18, M24 – and especially at the onset of the Covid-19 epidemic, as some questions related to the new situation were added to the questionnaires. For each adjustment of the instructions, training sessions were organised for employees. 41 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 2018, and if employees in the pilot projects had Implementing also been intensively included in this process. evaluation by means of Employees were another key stakeholder in the mixed research methods evaluation. In addition to being a link between us and the key target groups of the project In order to support the findings and ensure (users, informal carers and stakeholders), they valid results, quantitative and qualitative also performed a major part of the operational approaches, i.e. mixed research methods, evaluation activities. They provided the were used. A concurrent triangulation plan conditions for the implementation of various (in accordance with Creswell and Plano Clark, evaluation activities (for example, rooms for 2007) was used, which is useful primarily in meetings), surveyed users, completed surveys cases in which the results of one method are themselves and wrote reports themselves, used to confirm the results of another method provided data, participated in interviews and as well as the concurrent integrated plan (ibid.) similar. They also participated in establishing an which, unlike the concurrent triangulation plan, encryption system, which was essential for the includes an extensive (predominant) quantitative evaluation. or qualitative phase with an integrated less By establishing encryption, i.e. assigning a qualitative or quantitative part. This adds depth or unique code to each individual, only theoretical breadth to the results of the predominant method. concepts could be included in the questionnaires and thus additional inquiries for the data about users collected by employees by means Quantitative instruments of other forms could be avoided. It turned out and data later that the applicants did not always provide all the information in the application for the The evaluation primarily relied on eligibility assessment, the only document in quantitative data drawn from the information which socio-demographic data was recorded, system, while the questionnaires were developed and the assessors did not subsequently request or adjusted for different target groups (presented it (e.g. education, income, etc.), which caused in Table 5) and for different time points (M0, M6, a significant shortage in data. Otherwise, M12 and M18) and they were linked with the data an advantage of encryption was mainly the from the information system. rationalisation of data collection, less of a burden on users and informal carers and ensuring the flow of data (linking various databases with DATA FROM THE INFORMATION SYSTEM each other), while also enabling the anonymity of the participating users and informal carers. The environments established the These two target groups were also key actors in information system for the updated recording of the evaluation; the evaluators and employees data from the application, eligibility assessment, in the pilot projects conducted a large number personal plan, monitoring of the implementation of interviews with them, and they were also of services and recording of certain personnel constantly included in the survey. The pilot data. They were building it gradually together activities had a direct impact on them, which is with the system developer (for more, see chapter why their experience in the evaluation project is Electronic management of procedures and services extremely valuable. and information system suitability), so aggregated semi-monthly data11, which was received from the 11 Prepared in Excel. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 42 pilot environments twice a month and contained i.e. applicant, and EQ-5D, which assesses their data on the number of employees, applicants medical condition. The survey was conducted by and assessed persons, persons included in the the assessor during the first visit to the applicant, provision of services, the state of information and was intended to assess their eligibility. After support and organisation of e-care, waiting lists one year (M12) the assessor surveyed the user and difficulties in the implementation of the again, this time additionally about the experience pilot activities, was first used for the updated of new services. In June 2020, a set of questions monitoring of the pilot activities. on life changes during the Covid-19 epidemic was Aggregated semi-monthly data in the first added to the questionnaire (M12). In addition to the three process reports was used as the main source mentioned questionnaires, users also answered the of information, and it was also used in May 2020 in questionnaire on the experience of care after six the latest process report, but only for comparison months of inclusion in the project activities (M6), with anonymised micro data at the level of the with the aspect of coordination and the central individual from the information system, because role of users being of main interest. The questions the pilot environments at that time were still were adjusted in accordance with P3CEQ12, which improving the data in the information system. is a questionnaire for measuring the experience of Only a part of the micro data could be used for coordinated care focused on the user, i.e. care and the preparation of the interim report, while the assistance organised and managed based on the data for the preparation of the final report and the needs and preferences of the user. monograph was drawn entirely from anonymised In all three cases, the survey was personal, data at the individual level exported from the and the assessor recorded the answers on an online information system. platform on a mobile phone, which streamlined the A certain part of the data in the information data collection and avoided printing large numbers system remained deficient. For example, when of questionnaires and the subsequent entry of assessing eligibility, the assessors did not examine data in electronic form. In cases where the online all socio-demographic data, such as that pertaining survey was inappropriate or impractical (e.g. in an to education and income, provided by the applicant area with a poor mobile signal), the assessors used in the application, so some data was not included printed questionnaires and entered the answers in in the system. Such specifics are noted in the the online questionnaire later. The M0 questionnaire monograph in places where this data is analysed was implemented from March 2019 to the end of and presented. February 2020, the M12 questionnaire from the An important part of the data collected at the end of January 2020 to August 2020, and the M6 individual level is also the data reported monthly by questionnaire from June 2019 to August 2020. the pilot environments in cooperation with assistive A total of 258 users (59.4% of all assessed technology service providers (more in chapter applicants) answered the M0 questionnaire in Perception of the use of assistive technologies). the Celje pilot environment, 198 users (54.9% of all assessed applicants) in Krško and 257 users in Dravograd (73% of all assessed applicants). The QUESTIONNAIRES FOR APPLICANTS relatively low responsiveness is a consequence of AND USERS several factors. One is that the assessors started the survey later than they started the eligibility The user questionnaire (M0) combines assessment procedure. Applicants may also have two standardised questionnaires: CASP-12, refused to take part in the survey, or it may have which measures the quality of life of the user, been too demanding for some. 12 Person Centred Coordinated Care Experiences Questionnaire. 43 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 1: Survey response rate for users (M0, M6 and M12) Celje Krško Dravograd Total Number of assessed applicants living at home (M0) 434 361 352 1147 Number of assessed applicants (M0) 258 198 257 713 Applicant response rate (M0) 59.4% 54.9% 73.0% 62.16% Number of applicants assessed for a second time (M6) 208 127 100 435 Number of surveyed users with a second eligibility assessment (M6) 101 87 71 259 Response rate for users with a second eligibility assessment (M6) 48.6% 68.5% 71.0% 59.5% Number of users with a third, fourth or fifth eligibility assessment (M12) 102 52 36 190 Number of surveyed users with at least a third eligibility assessment (M12) 92 44 24 160 Response rate for users with at least a third eligibility assessment (M12) 90.2% 84.6% 66.7% 84.2% Number of active e-care users 50 59 31 140 Number of surveyed e-care users 31 35 13 79 Response rate for e-care users 62.0% 59.3% 41.9% 56.4% Number of active e-health users 7 2 12 21 Number of surveyed e-health users 4 1 9 14 Response rate for e-health users 57.1% 50.0% 75.0% 66.7% EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 44 Upon the second eligibility assessment (M6), burden of care (ZBI-22)15, additional questions the questionnaire was answered by 101 users in on demography and the household in which Celje (48.6% of all applicants assessed for a second they live, and on the provision of assistance and time), 87 users in Krško (68.5% of all applicants care. If present, the informal carer completed the assessed for a second time) and 71 users in questionnaire at the first assessment of eligibility Dravograd (71% of all applicants assessed for a of the relative included in the project and, if second time). In the second survey, the response possible, in a room other than the one where the rate decreased in the Celje pilot environment interview between the applicant and the assessor and increased in Krško, while in Dravograd it was conducted. They could also complete the remained at approximately the same level. In the questionnaire later16. After one year (M12), informal third time point (M12), 92 users were surveyed carers again responded to the same questionnaire in Celje, 44 in Krško and 24 in Dravograd. Despite by answering an additional set of questions related the low number of respondents, the response rate to the Covid-19 epidemic. in relation to the number of users with a third The survey in M0 was conducted from March assessment or more is quite high: 90% in Celje, 85% 2019, in M12 from the end of January 2020, and in in Krško and 67% in Dravograd. In addition to the both cases it ended in August 2020. A total of 428 questionnaire related to life during the Covid-19 informal carers took part in the survey. The first epidemic, 65 users from Celje, 21 from Krško and questionnaire was answered by 395 (153 from 23 from Dravograd answered at the M12 point. Celje, 111 from Krško and 131 from Dravograd), the Special emphasis was also placed on users second by 94 (59 from Celje, 16 from Krško and 19 of assistive technologies, for which two special from Dravograd), and both were answered by 58 questionnaires (e-care13 and e-health14) were informal carers (26 from Celje and after 16 from prepared, focusing on measuring the satisfaction, Krško and Dravograd). As well as the addition to perceived usefulness and effects of the use of the questionnaire related to life during the Covid-19 assistive technologies, and a special set was epidemic, 16 informal carers (8 from Celje, 4 from devoted to issues related to changes during the Krško and 4 from Dravograd) answered in the point Covid-19 epidemic. The survey was conducted M0, and 42 (20 from Celje, 8 from Krško and 14 from at one point in time (from May 2020 to August Dravograd) in the M12 point. 2020), in the same way as for other questionnaires. The response rate was high given the type During the survey, the users were included in of survey (self-survey), as 64.8% of all informal e-care for between 58 and 526 days, or for 262 carers registered in the pilot project17 answered days on average. The questionnaire was completed the first questionnaire (M0): 62.7% in Celje, 60% by 79 e-care users (56.4% of active users) and 14 in Krško and 72.4% in Dravograd. In the second e-health users (66.7%). measurement (M12), the response rate was calculated with regard to the number of users assessed for a third time who indicated that they QUESTIONNAIRES FOR INFORMAL CARERS had a relative who helped them. The total response rate in this case was even higher - 79.7% (88.1% in The questionnaire for informal carers (M0) Celje, 55.2% in Krško and 86.4% in Dravograd). includes a standardised Zarit Burden Interview In the period from May 2020 to August 2020, questionnaire on the subjective experience of the informal carers whose relatives used assistive 13 Users of the basic and premium e-care packages and users of the In Life smartwatch were included. 14 Users of the package of vital function monitoring at home and users of telemedicine support were included. 15 More about the questionnaire in chapter Care for those who care: studying the quality of life of informal carers. 16 Each of them received an envelope with paid postage and was able to send the envelope directly to the evaluator. 17 Data from the eligibility assessment application. If the data from the user questionnaire (M0) about whether the user has an informal carer or not was taken into account, the response rate would have been even higher, 80% on average. 45 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 2: Response to the survey on informal carers (M0, M12) Celje Krško Dravograd Total Number of informal carers, data from the information system (M0) 244 185 181 610 Number of surveyed informal carers (M0) 153 111 131 395 Response rate in informal carers (M0) 62.7% 60.0% 72.4% 64.8% Number of informal carers, data from the information system (M12) 67 29 22 118 Number of surveyed informal carers (M12) 59 16 19 94 Response rate in informal carers (M12) 88.1% 55.2% 86.4% 79.7% Number of surveyed informal carers (M0 and M12) 26 16 16 58 Number of surveyed informal carers in e-care 20 16 8 44 Response rate in informal carers in e-care 40.0% 27.1% 25.8% 31.4% Number of surveyed informal carers in e-health 1 2 3 6 Response rate in informal carers in e-health 14.3% 100.0% 25.0% 28.6% 46 Table 3: Survey response rate for stakeholders technologies as part of the project were separately interviewed in the same way with two questionnaires (on e-care and e-health). The e-care questionnaire was completed by 44 informal carers (31.4%), while the e-health Number of contacted stakeholders (M0) questionnaire was completed by six informal CELJE KRŠKO DRAVO- TOTAL carers (28.6%). Due to the very low number of GRAD responses from informal carers, the e-health 45 18 32 95 questionnaires were not analysed in detail. Number of stakeholders who responded to the survey (M0) QUESTIONNAIRES FOR STAKEHOLDERS CELJE KRŠKO DRAVO- TOTAL In the two time points - M0 (March 2019) GRAD and M18 (September 2020) - the positions or 15 16 8 39 sensitivity for long-term care, assessment of availability, access, reach of the continuity Stakeholder response rate (M0) of long-term care services, coordination and integration in the local environment, support CELJE KRŠKO DRAVO- TOTAL GRAD for informal carers, sufficiency of resources in the field of long-term care, quality of services, 33.3% 88.9% 25.0% 41.1% assessment of the strength and impact of the user whose need for long-term care have been Number of contacted stakeholders (M18) satisfied were measured with a questionnaire for stakeholders. The questionnaire consisted of CELJE KRŠKO DRAVO- TOTAL GRAD 41 statements measured by an agreement scale. The questionnaire was followed by four open- 47 19 34 100 ended questions, in which stakeholders were able to explain in detail how they assess certain Number of stakeholders who responded to important segments in the field of long-term care the survey (M18) in the environment in which they live. The online CELJE KRŠKO DRAVO- TOTAL questionnaire was answered by key stakeholders GRAD in the field of long-term care by individual pilot environments: representatives of decision-makers 9 12 14 35 (e.g. municipalities), service and programme providers (e.g. care homes, pharmacies, Stakeholder response rate (M18) community nursing), stakeholder, educational CELJE KRŠKO DRAVO- TOTAL and research organisations (e.g. pensioners’ GRAD association, folk high school, etc.). This also 19.1% 63.2% 41.2% 35.0% refers to stakeholders who were not necessarily involved in the pilot activities in any way. They were identified and invited to complete an online questionnaire by long-term care coordinators. The response to the questionnaire varied by environment. Due to the poor responsiveness 47 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE of stakeholders, a longer period was determined emphasis was placed on the survey conducted in for the survey: the first measurement lasted from the second time point, as it already reflects the 20 March 2019 to 15 June 2019, and the second views on the situation in the field of long-term care from 24 September 2020 to 26 October 2020. At in the environment on which the project has made the first measurement, 15 different stakeholders an impact. responded to the questionnaire in Celje, or 33.3% of those invited, and nine or 19.1% of those invited responded at the second measurement (M18). Of QUESTIONNAIRE FOR EMPLOYEES the 18 stakeholders who participated in the project as partners (lead partner, mandatory consortium A questionnaire for employees was used to partners or additional partners), 12 completed the measure the quality of working life of employees in questionnaire at the M0 point and five at the M18 the pilot projects. It was implemented at two points point. The questionnaire was also completed at of time, specifically in the period of the first half the M0 point by three stakeholders who were not of a year of the project (M0/M6) and in the period involved in the project as partners, and there were between one year and a year and a half (M12/18). four such stakeholders at the M18 point. The questionnaire consisted of several In the Dravograd pilot environment, eight sets of questions. For the purpose of measuring different stakeholders, or a quarter of those who the organisational climate and satisfaction, the had been invited to participate, answered the questions from the SiOK questionnaire (Slovenian questionnaire at the M0 point, while 14 or 41.2% organisational climate) were slightly adjusted18. of the invited ones answered at the M18 point. Of The organisational climate, employee satisfaction the 32 stakeholders who participated in the project with certain aspects of working conditions, as partners, eight completed the questionnaire at reconciliation of work and family life, contacts the M0 point and 12 at the M18 point. Only project with various stakeholders in the field and positive partners were invited to complete the survey at and negative aspects of work as part of the project the M0 point, while two other stakeholders also were measured. The assessors additionally completed the questionnaire at the M18 point. answered a set of questions related to the eligibility In the Krško pilot environment, 16 assessment, evaluation of the usability and stakeholders, or 88.9% of those invited, answered suitability of the assessment tool and the eligibility the questionnaire at the M0 point, while 12 or assessment guidelines. A set of questions for long- 63.2% answered at the M18 point. Of the 11 term care coordinators related to their work and stakeholders who participated in the project as questions related to help and support networks partners, nine completed the questionnaire at among team members were added at the second the M0 point and seven at the M18 point. The point of time. questionnaire was also completed at the M0 point The questionnaire was an online by seven stakeholders who were not involved in questionnaire sent by the project coordinator to all the project as partners, and there were five such employees by e-mail. The survey was conducted stakeholders at the M18 point. for the first time in the second half of June 2019, As a large number of different persons and the second time in August and September from different organisations answered the 2020. At the M0 point, 43 out of 59 employees in the questionnaire at both time points, the samples pilot environments answered the questionnaire: from M0 and M18 are treated as independent the response rate in Celje was 59.1%, in Dravograd samples in the data analysis. In the results, more 76.2% and in Krško 87.5%. 40 employees answered 18 More in the chapter Teamwork and integration of stakeholders as the foundations for ensuring integrated long-term care. 19 Because the surveys were answered partially, the total number of respondents (N) differs in individual results. 48 Table 4: Survey response rate for employees the questionnaire in full, and three of them only partially. 50 employees responded to the survey at the M18 point, which represents an 84.7% response rate (Celje 80%, Krško 87.5% and Dravograd 88.9%). Number of employees (M0) As different persons answered the questionnaire at both points of time (significant CELJE KRŠKO DRAVO- TOTAL staff turnover) and there were no attempts GRAD to identify the respondents, the sample of 22 16 21 59 respondents is considered as independent. Changes in attitudes thus cannot be monitored at the level of the employee, but at the level of all employees Number of employees who responded together, which are not necessarily the same in to the survey (M0) both samples. CELJE KRŠKO DRAVO- TOTAL At the M12 time point, employees also GRAD completed a questionnaire on social support networks, which were used to measure how and 13 14 16 43 in which cases employees offer support, connect and cooperate with each other. Everyone who was employed in the pilot environments during the Employee response rate (M0) implementation of the project, as well as project CELJE KRŠKO DRAVO- TOTAL managers who were not necessarily formally GRAD employed as part of the project, were invited to 59.1% 87.5% 76.2% 72.9% complete the questionnaire. The questionnaire was answered by 18 employees from the Celje pilot environment, 17 from Krško and 14 from Dravograd. Number of employees (M18) CELJE KRŠKO DRAVO- TOTAL GRAD QUANTITATIVE DATA ANALYSIS 25 16 18 59 Quantitative data was analysed in different ways. A descriptive (e.g. presentation of proportions, median, arithmetic mean or Number of employees who responded average, standard deviation, mode), bivariate to the survey (M18) (e.g. chi-square, t-test, Mann-Whitney test, etc.) CELJE KRŠKO DRAVO- TOTAL or multivariate data analysis (e.g. hierarchical GRAD clustering, regression) were performed, depending 20 14 16 50 on the research question and the type of data or sample. Depending on the nature of the data, either parametric or non-parametric methods Employee response rate (M18) were used accordingly. The use of the method is explained with each analysis, in which key CELJE KRŠKO DRAVO- TOTAL GRAD statistical parameters are also assigned. SPSS and STATA were used for data analysis, and the 80.0% 87.5% 88.9% 84.7% Pajek programme for analysis and display of large networks was used in the analysis of networks. 49 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 5: Review of questionnaires and the course of the survey Respondent Time point Interviewer Type of questionnaire Content CASP-12 and EQ-5D (quality of life), Applicants/ face-to-face sur- in M12 also the experience of the users M0, M12 assessor vey, online survey entry new services and, in June 2020, life during Covid-19. P3CEQ, tailored (experience of face-to-face sur- reception of services as part of the Users M6 assessor vey, online survey pilot activities - mainly in terms of entry coordination and the central role of users). self-survey of Informal informal carers, ZBI-22, objective burden, provision carers M0, M12 assessor sub- mits the survey printed question- of care. naire long-term care/ project coor- self-survey of In M12, also the experience of the Stakeholders M0, M18 dinator makes an invitation stakeholders, pilot activities and, in June 2020, to complete a online survey also life during Covid-19. questionnaire long-term care/ project coor- Employees in M0/M6, dinator makes self-survey of Attitudes towards the develop- the project M12/M18 an invitation employees, online ment and quality of long-term to complete a survey care services. questionnaire Quality of working life, experience face-to-face sur- of assessment and assessment E-care users M17-M20 assessor vey, online survey tools, experience of coordination entry and teamwork and social support networks. Experience of service, satisfaction E-health face-to-face sur- and usefulness assessment, intent users M17-M20 assessor vey, online survey for future use, impact assessment entry (PIADS-10), change of positions during Covid-19. Informal self-survey of Experience of service, satisfaction carers for M17-M20 assessor sub- informal carers, and usefulness assessment, intent e-care users mits the survey printed question- for future use, change of positions naire during Covid-19. Informal self-survey of Experience of service, satisfaction carers for informal carers, and usefulness assessment, intent e-health M17-M20 assessor sub- mits the survey printed question- for future use, impact assess- users naire ment, change of positions during Covid-19. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 50 Qualitative instruments applying for the public call for applications and the and data initial organisation of work in the pilot projects: key motivation for applying for the public call, the As part of the qualitative approach, various course of preparations for applying and forming a methods and techniques or instruments were consortium group, the course of activities relative developed and used and materials of various to the set goals, creation of a single entry point, types, both primary and secondary, were collected. cooperation with stakeholders and similar. Semi-structured interviews and focus groups with In the period between April and June 2019, various stakeholders and a democratic forum initial interviews were conducted with long-term with assessors were organised and reports from care coordinators with the purpose of finding employees were collected. Material that was not out how they experience and understand the created in the projects for the needs of evaluation role of long-term care coordinator. The interview but were an integral part of the course of pilot was conducted again at the end of the project projects - personal plans and annexes and various (between August 2020 and November 2020). minutes - were also collected and analysed. All Its content was adjusted and the aspect of the listed instruments are presented below. experience of employment in the pilot project was added. During the pilot projects, quite a few long-term care coordinators were replaced, which SEMI-STRUCTURED INTERVIEWS is why various starting points were prepared for interviews, and those who started performing the Ten types of semi-structured interviews and role of long-term care coordinator in the interim a total of 83 various interviews with stakeholders were asked in detail about their experience relevant for the pilot projects and evaluation were in assuming and concluding this role and the conducted20. The conversations were based on pre- transfer of knowledge and work. In Dravograd, prepared systematic talking points. In agreement an additional interview was conducted with the with the interviewees, the conversations were long-term care coordinator who had a job in the recorded, with the exception of the interview general hospital, in which the conversation also with a user who did not permit recording, and focused on the experience of coordination of transcribed literally or paraphrased. They were discharges from the hospital. conducted mostly in person, and due to measures In September 2019, an interview with related to the Covid-19 epidemic, some were representatives of the Dravograd pilot also conducted online or by phone. Most of the environment and a representative of the interviews were interpreted in the analysis directly smartwatch provider was conducted, with the from the literal transcripts or by thematic sections, conversation focusing on the description of the while a thematic analysis was performed in the functionality of the smartwatch, its functionality case of users and informal carers. in relation to different target groups, use in the A semi-structured interview was conducted pilot project, advantages and disadvantages of the on the first visit to all three pilot environments smartwatch and recommendations regarding its in February 2019. Two people took part in the use. In October 2019, a semi-structured interview interviews - the project manager (Celje, Dravograd, was conducted with a representative of the e-care Krško), also and the project coordinator (Krško) provider on the development and procedures for or long-term care coordinator (Celje, Dravograd). introducing assistive technologies in all three pilot We were interested in the preparatory phase of environments. 20 A semi-structured interview with the contracting authority, the Ministry of Health, had also been planned, but it was not conducted due to objective circumstances on the part of the contracting authority. 51 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 6: Characteristics of the interview procedure Type of interview Time point Method of Number of interviewing interviews Project manager and project/ February 2019 (Krško, Dravograd, long-term care coordinator Celje) face-to-face 3 April 2019 (Dravograd), May 2019 (Krško), June 2019 (2 in Celje), July 2020 (Celje, Krško), August face-to-face, with Long-term care coordinator 2020 (Celje), September 2020 the exception of 12 (Krško, 2 in Dravograd), October two that were 2020 (Celje) done online November 2020 (Krško) Representatives of the Dravograd pilot environment and a representative of the September 2020 online 1 smartwatch provider Representative of the e-care provider October 2019 face-to-face 1 E-care users April and May 2020 by telephone 7 Informal carers for users of assistive technologies April and May 2020 by telephone 9 Users from July to September 2020; three in September 2019 as trial face-to-face 20 Informal carers from July to September 2020; three in September 2019 as trial face-to-face 21 September 2020 (Slovenj Gradec, Representatives of Ravne na Koroškem, Dravograd, municipalities Krško), October 2020 (Celje, Štore, face-to-face 821 Vojnik, Dobrna) Representative of the information system developer November 2020 online 1 21 One interview was not recorded due to technical difficulties, so seven interviews were used in the analysis. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 52 To better understand the experience of and codes were accordingly supplemented with users and informal carers included in assistive a deductive analysis by using Tsertsidis’ (2019) technologies, seven semi-structured interviews coding scheme. The Atlas.ti qualitative analysis with e-care users in April and May 2020 and nine programme was used for coding and analysis. interviews with their informal carers from the In order to gain a comprehensive Krško pilot environment were conducted. The understanding of the experiences of users and purpose was to make a list of the experiences informal carers included in the pilot activities, 20 of users who tested a combination of the basic semi-structured interviews were conducted with package and a fall detector. users and 21 with informal carers. A quota sample The criteria for the inclusion in the was selected, and seven users and seven informal interview were: consent to the conversation carers were systematically selected in each pilot of both the user and the informal carer (dyad) environment with the help of the project staff. and active involvement of the informal carer in Users from institutional care or their relatives care; eloquence and willingness to share one’s were not interviewed. Users not involved in experience, feelings and opinions; use of the basic assistive technologies were not interviewed either package and a fall detector (both). The additional as a sample of the latter, as already mentioned, criteria were that users live alone in their own was interviewed separately. household (they can live in a multi-apartment Based on the criteria, the interviewees were building, e.g. relatives live in the same building, selected by the long-term care coordinators, but in a different household), i.e. that they spend who contacted them and agreed on a date for most of the day alone, and the probability of a fall the visit. In certain cases, given the difficulties was higher. in the selection of interviewees who would Despite the relevant criteria, dyads did not meet all the criteria, the option was allowed of a exclusively participate in the interview; six dyads slight deviation from the sample and the criteria were interviewed, plus an additional two informal were adjusted to the actual situation in the pilot carers and one user. We were interested in the use environments. Four informal carers in a dyad and their experience, changes, e-care during the with users were thus selected as interviewees Covid-19 epidemic, recommendations and (only (i.e. informal carers who provide care to the for informal carers) the burden of care. Interviews interviewed users). were conducted by telephone, and they were The criteria for the selection of users were: relatively short as a result, lasting an average of 27 ability to conduct an interview and diversity minutes. All interviewees were women, who were of the category of eligibility for long-term care, 87 years old on average - three in the first category, gender, age, reception of new services, existence three in the second and one in the third category of informal carer and assessment of eligibility of long-term care. With the exception of one, they by several assessors and, for informal carers, in have not yet had pronounced and serious needs addition to the ability to conduct an interview, for long-term care. Among the informal carers, six diversity of the category of eligibility, gender, women and three men were interviewed - one was employment status and residence. the spouse, and the others were either children or The final sample of users included eight from their partners (i.e. daughter-in-law). the first, ten from the second, third or fourth, and A three-tier coding system (same for two from the fifth eligibility category, six men and interviews with users and with informal cares) 14 women, two adults and 18 older adults, 15 who was created, which initially included five main received independence maintenance services and topics. After an inductive analysis of the interviews five who did not, 11 who had an informal carer was performed, the topics, their categories (factors) and nine who did not. With the exception of one, 53 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE they were assessed by different assessors as the following thematic sets: situation related to part of different assessment. The final sample of issues faced by older people (existing services informal carers included four from the first, 12 and programmes, informal care, analysis of the from the second, third or fourth, and five from the situation, etc.), cooperation of the municipality fifth eligibility category, five men and 16 women, with formal service providers, non-governmental eight employed people, one unemployed and 12 organisations, other municipalities and local retired or homemakers, 16 people who lived in the communities, etc., knowledge of the needs of the same building as the user and five who resided population, involvement and experience of the elsewhere. pilot project, thoughts about the existing system The guidelines for the interview followed and the latest bill dealing with the field of long- the main research objectives of the evaluation, term care. and covered the framework questions related Due to technical difficulties, one interview to the procedures of the entire project, to the was not recorded, and it was excluded from assessment, personal planning, coordination further consideration. The other seven were coded and implementation from the aspect of method in accordance with a pre-prepared multi-level and procedure, to a comparison of the people’s coding system, which followed the questions in lives before and after the project, to the system the guidelines for the interview. In the next step, (satisfaction with the independence maintenance the codes of the municipalities of the same pilot services and proposals for systemic regulation of environment were combined and then all seven long-term care) and to the provision of services municipalities were combined. during the Covid-19 epidemic. The guidelines for At the end of the project, a semi-structured the interview were adapted to the aspect of each interview was conducted with a representative target group, while they otherwise covered the of the developer in November 2020 in order to same key topics. The interviews were conducted in gain better insight into the construction of the person, mostly at users’ homes. information system. The following topics were The interviews were thematically analysed in focused on: a review of the chronology of the a deductive way (Boyatzis, 1998; Braun & Clarke, construction of the information system as part 2006; Hayes, 1997) in accordance with the principle of the pilot projects, information on possible of systematic coding as proposed by Saldana differences in the developed information system (2012). Two coding systems were developed for an individual pilot environment and access (separately for informal carers and users) that to data, and an assessment of suitability of the followed the logic of the interview guidelines and information system developed as part of the pilot were hierarchically divided into several levels. In projects as a basis for developing a new long-term the analysis of codes, mixed methods were used, care system throughout Slovenia. and the content was compared on the basis of the characteristics of users and informal carers in accordance with the previously mentioned criteria. FOCUS GROUPS The MaxQDA qualitative analysis programme was used for this. For the purpose of the evaluation, three In the autumn of 2020, eight semi-structured types of focus groups, and a total of seven focus interviews were conducted with representatives groups were conducted: one on the eligibility of the municipalities in the areas where the assessment, three on the introduction of assistive pilot activities were carried out: Slovenj Gradec, technologies and three on the work in the Ravne na Koroškem, Dravograd, Krško, Celje, independence maintenance unit. All seven were Štore, Vojnik, Dobrna. The interview included recorded in agreement with the participants, 54 Table 7: Characteristics of focus groups literally transcribed, interpreted by thematic sets, and quotes were used in the analysis to illustrate Type of focus group the findings. Prior to the start of the pilot projects, the ▷ ON THE ELIGIBILITY ASSESSMENT Ministry of Health trained a team of expert PARTICIPANTS IN FOCUS GROUP assessors to support the newly employed assessors expert assessors (six of seven) in the pilot projects. The focus group that was conducted in June 2019 was focused on how expert TIME POINT assessors evaluate the training that they have June 2019 received, how they evaluate the transfer of their LOCATION OF IMPLEMENTATION knowledge to assessors in the pilot environments, Ministry of Health and how they evaluate assessment as a method and procedure, and as a tool for the eligibility NUMBER OF FOCUS GROUPS 1 assessment. The focus groups were attended by six of the seven expert assessors. For the purposes of analysing the introduction ▷ ON THE INTRODUCTION OF ASSISTIVE of assistive technologies, three focus groups were TECHNOLOGIES conducted in October and November 2019, one in each pilot environment. They were attended PARTICIPANTS IN FOCUS GROUP by the long-term care coordinators, project long-term care coordinators, project coordinator and assessors coordinator and assessors. The topic of discussion was the process of the introduction of assistive TIME POINT technologies, the process related to users (finding oktober 2019 (Krško, Celje), suitable candidates, user responses, concerns, november 2019 (Dravograd) delays, complications in providing services), LOCATION OF IMPLEMENTATION the advantages and disadvantages of assistive in the pilot environments technologies and proposals related to the provision of such services to users after the conclusion of the NUMBER OF FOCUS GROUPS project. Three more focus groups were conducted 3 in the pilot environments on the same day on the topic of independence maintenance services in the ▷ ON THE INDEPENDENCE MAINTENANCE home environment. They were attended by the SERVICES long-term care coordinators, project coordinator and employees of the independence maintenance PARTICIPANTS IN FOCUS GROUP unit. The discussion focused on the process long-term care coordinators, project coordinator and employees of the independence of recruitment of staff and the challenges of maintenance unit. acquiring staff for the independence maintenance unit. Each of the ten new services (description of TIME POINT service, key service provider, proposal of service oktober 2019 (Krško, Celje), november 2019 (Dravograd) according to user types, responses from users), the work process in their pilot environment LOCATION OF IMPLEMENTATION (distribution of responsibilities, distribution of in the pilot environments hours) and a proposal to reorganise the provision of new services was systematically discussed with NUMBER OF FOCUS GROUPS 3 the participants. 55 Table 8: Characteristics of personal plans and annexes to personal plans PERSONAL PLANS AND ANNEXES TO PERSONAL PLANS Number of beneficiaries who live at home A total of 576 anonymised personal plans (181 from Celje, 159 from Dravograd and 236 from Krško) CELJE KRŠKO DRAVO- TOTAL GRAD and 71 annexes to personal plans were received. The rate of created personal plans in terms of the 378 289 230 897 number of beneficiaries was 64.2% - it was the highest in Krško (81.7%), followed by Dravograd (69.1%), and the lowest in Celje (47.9%). The gap Number of users with a personal plan between the number of beneficiaries and the CELJE KRŠKO DRAVO- TOTAL number of users with personal plans was mainly a GRAD result of some deciding not to use the service despite 181 236 159 576 being eligible, or because death occurred before the plan was created. In Celje22, a long waiting list for inclusion in services also contributed to this, as personal plans were not created in advance. Rate of personal plans made Most of the personal plans and annexes were CELJE KRŠKO DRAVO- TOTAL received in the form of a spreadsheet in Excel, GRAD while a part of the plans, especially those the 47.9% 81.7% 69.1% 64.2% environments prepared before the information support for creating personal plans was established, were received as a Word or pdf document. The latter, Number of users with an annex to a which accounted for more than a tenth of all, were personal plan converted into the xls format and subsequently, in CELJE KRŠKO DRAVO- TOTAL cooperation with the long-term care coordinator, GRAD additional data was added (such as the date of the 35 27 9 71 plan, because the printout from the information system showed the date of entry and last changes in the system and not the actual date of production). More about the sample and analysis of personal plans is available in the chapter Personal planning and coordination in long-term care: identifying needs and planning care together with the user. REPORTS BY EMPLOYEES ON ACTIVITIES AS PART OF THE PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE For the purposes of the evaluation, employees prepared reports with: (1) a chronology of the pilot project in terms of their work and role in 22 In the Dravograd and Celje pilot environments, the reasons why the pilot activities were discontinued were not consistently recorded. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 56 Table 9: List of received semi-annual reports in the three reporting environments by pilot environment Reports by employees Celje Dravograd Krško Total number of reports 22 16 14 52 1st reporting number of period employees 22 21 14 57 response rate 100.0% 76.2% 100.0% 91.2% number of reports 9 12 11 32 2nd reporting number of period employees 23 21 14 58 response rate 39.1% 57.1% 78.6% 55.2% number of reports 11 9 15 35 3rd reporting number of period employees 21 21 15 57 response rate 52.4% 42.9% 100.0% 61.4% Total 42 37 40 119 57 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE the project, (2) their thoughts on the events and Reporting by employees varied considerably, positive and negative aspects of their work, (3) which is why caution was needed in interpreting a vision of their role, work and profession and the results. The records by employees from one pilot proposals for improvement and (4) an assessment environment are generally much more detailed of the scope of their work by pre-determined and extensive when it comes to work tasks (first items. They submitted reports three times and question) compared to the other two environments, reported on the content for the period from which somewhat hinders the comparison of August 2018 to July 2019, from August 2019 to results on the presence of topics at the level of January 2020 and from February 2020 to April environment. Some employees recorded the same 2020. A total of 119 reports were received. The content in both periods, and in some cases the response rate was the highest in the first reporting reports within the teams were (almost) identical. period, at 91.2%, it declined to 55.2% in the second, Despite the mentioned limitations, the reports are and in the third reporting period it was 61.4%. The an important source of information, as they clearly largest proportion of employees who prepared the support quantitative data and highlight topics that report was in Krško at all points in time. were not detected in other measuring instruments The received reports were technically during the evaluation. processed and, based on the code and the initial letter of the field of work in the project, it was possible to link the reports by the same person DEMOCRATIC FORUM WITH ASSESSORS from different periods and compare their development and changes over time (two reports The democratic forum called “Assessing are an exception, as the employees did not record eligibility for long-term care as a method and this information). A qualitative analysis was procedure” was organised on 3 September 2020 performed by means of the MaxQDA qualitative for the needs of the evaluation. The purpose of the analysis and mixed method programme. The democratic forum was to have an argument-based coding system was initially designed in an discussion on various aspects related to the work inductive way and the three umbrella codes of the assessor and the eligibility assessment. Four (identical to the three main questions) for each thematic fields were discussed: the single entry reporting period were assigned a different point and assessors, assessing in accordance with number of sub-codes depending on the recurring the new paradigm and consideration of the context, content. In the next phase, the structure of ensuring objectivity of the tool and the role of the the coding system was partially changed in a life story, and assessment as a procedure. deductive way so that the codes (and their content) Assessors from all pilot environments and all followed the objectives of the evaluation, while expert assessors were invited to the event. It was the quotations were redistributed in accordance attended by ten of the fifteen assessors (five of the with the new codes. The content of the codes seven from Celje, all four from Krško, one of the was compared in accordance with the pilot four from Dravograd) and three expert assessors. environment of the employee, unit of employment At the democratic forum, with the help of a and their field of work (and in accordance with moderator, the participants exchanged views and the employment profile, where possible) and opinions by presenting arguments and sought to reporting period and changes and similarities of reach a consensus on each topic discussed. For individual content over time were observed by each topic, the evaluators first presented all the means of the entered code (with the exception of a data they had collected during the evaluation and few persons who did not enter the mentioned data the results of an online questionnaire that was in the report form). completed by the assessors and expert assessors EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 58 for the purpose of conducting the democratic after research plan, we were able to provide an forum a week before the event. The presentation of appropriate basis for updated, process-oriented the data was followed by a plenary discussion, and steering of the pilot projects, as well as for this was followed by a discussion in two groups, providing guidance to the contracting authority in and group reporting and a final joint discussion finding system solutions. in which a consensus was reached with the Due to objective circumstances, the participants on the discussed topic. At the end, the methodology began to be established at a time participants once again completed the same online when the pilot projects were already being questionnaire as before the democratic forum. intensively implemented, which is considered The first online questionnaire was answered by 17 one of the weak spots of the evaluation, as the assessors and expert assessors (out of 21 assessors), starting point of the pilot projects was missed, and the second by all 13 participants in the which is the key observation point from the aspect democratic forum. of monitoring of changes. It would be ideal if the methodology was established before the inclusion of users in the project, and if employees in the MINUTES AND OTHER QUALITATIVE pilot projects were also intensively included in this MATERIAL process. It was crucial for the evaluation that it An important source of qualitative data took place in intensive cooperation with other are the notes and minutes made during the stakeholders, the main ones being employees in implementation of pilot projects. For the purposes the pilot environments, users and their relatives of evaluation, we relied mainly on the minutes of or informal carers, the Ministry of Health as evaluators from visits to the environments (five the contracting authority of the evaluation, and in Celje, seven in Dravograd, four in Krško), with the European Centre for Social Welfare Policy. the contracting authority (22), on the minutes of Each of them, with their specific role, has made the contracting authority with coordinators from an important contribution to creating better the environments (13) and on internal minutes conditions and circumstances for the evaluation. prepared in the project environments (52 in Celje, As the pilot projects were demanding and 91 in Dravograd, 82 in Krško). We have a total of innovative, in addition to the listed stakeholders, 225 such pieces of material. an important advisory or steering role could also be played by a broader expert group that would monitor the progress of the pilot projects Discussion with key and evaluation results and provide expert bases messages and proposals for solutions to the challenges that constantly arise during such projects. For While the pilot projects were complex in example, in preparation for the pilot projects terms of organisation and implementation, just in the field of long-term care, the establishment as complex was their evaluation, as part of which has been planned of a national project council a number of evaluation objectives was evaluated for long-term care, which would comprise and by using various research methods and representatives of the Ministry of Health, techniques. An experimental research plan was Ministry of Labour, Family, Social Affairs and not possible, and it was not possible to determine Equal Opportunities, Health Insurance Institute causality due to the large number of pilot activities. of Slovenia, Pension and Disability Insurance We were therefore careful in attributing the effects Institute, Association of Municipalities and Towns of the pilot projects. However, by using a before- of Slovenia, Slovenian Federation of Pensioner 59 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Associations, local project council and project that had been originally planned. Several factors managers from the pilot environments. had an impact on this, including the Covid-19 The evaluation was based on mixed research epidemic, which has strongly marked the course methods in order to support the findings and of the project activities and people’s lives. In ensure valid results. Such an approach also addition, the evaluation was marked by the fact allowed depth and breadth to be added to the that all three pilot projects were not completed at results of the prevailing method. In cooperation the same time and that in two pilot environments with the employees in the pilot projects, a large they ended simultaneously with the completion variety of material and data has been collected that of the evaluation. This has led to greater enables very detailed and extensive quantitative flexibility in both data collection and processing. and qualitative analyses. A democratic forum Our experience is that for evaluations of such with assessors, a method that is not yet very well- innovations, it is necessary to ensure more time for known and widespread in research in Slovenia, coordination of data with the providers after the was also conducted, contributing to the further completion of their activities, and then also more development of the method in our country. time for a thorough preparation of analyses and It was not possible to collect all the desired final results. data as part of the evaluation in the way that had It was not possible to analyse all opinions initially been anticipated, or they were collected and all obtained data, but they certainly represent in an unsystematic way (for example, insufficient a wealth of experience and a wealth of data that number of units of analysis, data unevenly or could provide support and important guidance to deficiently entered into the information system), planners of the long-term care system even after so it was not possible to verify and evaluate all the the completion of the projects and the evaluation. evaluation objectives and indicators in the way EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 60 BIBLIOGRAPHY CDC (2011). Developing and Effective Evaluation Plan. Atlanta, Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih Georgia: Centres for Disease Control and Prevention, National projektov, ki bodo podpirali prehod v izvajanje sistemskega Center for Chronic Disease Prevention and Health Promotion, zakona o dolgotrajni oskrbi«. Retrieved from https:/ www. Office on Smoking and Health; Divison of Nutrition, Physical uradni-list.si/glasilo-uradni-list-rs/vsebina/2018002400003/ Activity, and Obesity. javni-razpis-za-izbor-operacij-izvedba-pilotnih-projektov-ki-bodo- podpirali-prehod-v-izvajanje-sistemskega-zakona-o-dolgotrajni- Creswell, J. W., & Plano Clark, W. L. (2007). Designing and oskrbi-st--303-1201816-ob-198618 conducting mixed methods research. Thousand Oaks, California: Sage Publications, Inc. Ministrstvo za zdravje. (2018). Javno naročilo za Evalvacijo pilotnih projektov s področja dolgotrajne oskrbe. Retrieved from https:/ Boyatzis, R. E. (1998). Transforming Qualitative Information: www.enarocanje.si/Obrazci/?id_obrazec=280003 Thematic Analysis and Code Development. Thousand Oaks, CA: Sage. Parsons, D. (2017). Demistifying evaluation. Practical approaches for researchers and users. Bristol: Policy Press. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3, 2, 77–101. Saldana, J. (2012). The Coding Manual for Qualitative Researchers. London: Sage. Hayes, N. (1997). Theory-Led Thematic Analysis: Social Identification in Small Companies. V Hayes N. (ed.). Doing Qualitative Analysis Tsertsidis, A., Kolkowska, E., & Hedström, K. (2019). Factors in Psychology. Hove, UK: Psychology Press. influencing seniors’ acceptance of technology for ageing in place in the post-implementation stage: A literature review. Kahlert, R., Boehler, C., & Leichsenring, K. (2018). Monitoring International Journal of Medical Informatics, 129, 324–333. and Evaluating Integrated LTC Models. Vienna: Euro Centre doi:10.1016/j.ijmedinf.2019.06.027 Publication. Kustec Lipicer, S. (2009). Vrednotenje javnih politik. Ljubljana: Fakulteta za družbene vede. 61 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE TRANSITION TO INTEGRATED LONG-TERM CARE BY ESTABLISHING A SINGLE ENTRY POINT, INTEGRATED CARE TEAM AND CONNECTING STAKEHOLDERS Aleš Istenič Social Protection Institute of the Republic of Slovenia Mateja Nagode Social Protection Institute of the Republic of Slovenia EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 62 KEY MESSAGES ▶ The pilot projects were implemented in different types of environments (urban, semi-rural and rural environment), with different social contexts and organisation of work: the leading project partners varied as regards type of organisation (health care centre, centre for social work and care home), and consortia were also composed differently. The pilot projects thus provided insight into different structures and methods of operation and organisation of long-term care in different environments and circumstances. ▶ Based on the results of the project, none of the organisations was singled out as the only one suitable for establishing a single entry point. In no pilot environment was a single entry point established at the regional unit of the Health Insurance Institute of Slovenia, which would be an important contribution to testing appropriate solutions, given the long-term care bill (2021), which envisages this. ▶ The establishment of the pilot projects and all planned structures and teams has drawn attention to the already recognised problem of staff shortages in the field of long-term care and provided insight into the challenges that will be faced in Slovenia in the future. All environments have faced employment challenges to a lesser or greater extent, as some profiles were 62 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 63 TRANSITION TO INTEGRATED LONG-TERM CARE BY ESTABLISHING A SINGLE ENTRY POINT, INTEGRATED CARE TEAM AND CONNECTING STAKEHOLDERS more difficult to employ or have not been employed at all (e.g. nursing carer, master of kinesiology). It will be necessary to make care professions more attractive. ▶ The turnover of staff, which was also due to the limited time of implementation of the project and thus the inability to maintain the sustainability of employment, was a special challenge of the pilot projects. ▶ Integrated care contributes to the easier access to services, their higher quality and efficiency and, consequently, greater user satisfaction, which is why it is necessary for various stakeholders in this field to connect and cooperate. ▶ Cooperation between stakeholders must reflect a common interest in identifying and satisfying the needs of the local population, which shows in active and effective cooperation in providing long-term care. ▶ The key stakeholders in the field of long-term care in the pilot environments are aware that integrated and coordinated social care and healthcare is important for care, and at the same time they perceive that the various organisations that provide long-term care are not yet well connected and that transfer of information between them is not efficient enough. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 64 Introduction to services in one place, is being increasingly pronounced (Črnak Meglič et al., 2014). Integrated care is a concept that is widely How to overcome the mentioned challenges accepted and used throughout the world (Goodwin, and find related solutions in the direction of better 2016), although there is no universal definition integration in the field of long-term care was also of integrated care (Goodwin, Sonola, Thiel, & explored by pilot projects in the field of long-term Kodner, 2013; World Health Organisation, 2016), care. The article focuses mostly on the following as different definitions of integrated care are important elements that have been tested: being used. This diversity is mainly the result of establishing single entry points and an integrated the different intentions attributed to integrated care team, employing a long-term care coordinator care by various stakeholders as part of integrated and connecting various stakeholders in the field of care systems. Goodwin (2016) notes that integrated long-term care in pilot environments. care cannot be defined even narrowly, and that it The single entry point as an important part should be seen as a general term for a broad and of integrated care is mentioned by many authors multidimensional set of ideas and principles that in their work (e.g. Pan, 1995; Flaker et al., 2007; aim to better coordinate care in accordance with Flaker, Nagode, Rafaelič, & Udovič, 2011; Črnak people’s needs. It could be said in the simplest Meglič et al., 2014; Lebar et al., 2017). As early as terms that integrated care is an approach to the 1990s, Pan comprehensively discussed the bridging fragmentation, especially where such importance of a single entry point and wrote that it fragmentation leads to detrimental effects for the was a local or regional point that facilitated access care experience and for the results of care itself. to long-term care services. At this point, interested At the core is the commitment to improving the parties receive information on long-term care, quality and safety of services through permanent and this is also where a needs assessment and a and co-productive partnership (ibid.). care plan are made. The procedure at the single In Slovenia, coordination between entry point can run completely independent institutions that provide services in the field of from the further process of provision of service. long-term care is not good, which makes it difficult This means that a procedure is carried out at a for people to access services and reduces their single entry point by means of which persons are quality (Nagode, Zver, Marn, Jacović, & Dominkuš, assessed, advised and referred to appropriate 2014). Services are not integrated in practice, services, regardless of what services these are users are not fully provided with quality, equal or who provides them. From the point of view of and necessary access to services, administrative users, a single entry point brings easier and faster procedures are complex and access to information access to services, and from the point of view is not centralised, as authors of various articles of the state, it can mean a more efficient way of have been pointing out for many years (e.g. managing services. The use of uniform eligibility Dominkuš & Peternelj 2006; Ministrstvo za delo, assessment and referring users to services družino socialne zadeve in enake možnosti, 2008; provides a better overview of the use of services Dominkuš, Zver, Trbanc, & Nagode, 2014; Črnak and their costs. The operation of a single entry Meglič et al., 2014; Ministrstvo za zdravje, 2021). point increases the chances of interested persons In Slovenia, the effort is being made to overcome receiving information about different types of this by adopting an act that would uniformly and assistance and care. Raising people’s awareness of systemically regulate the field of long-term care the possibilities of community-based care can thus (MH, 2021). The need for an integrated method make an important contribution to delaying entry of long-term care in which healthcare and social into institutional care. The concept of a single entry care services are connected, and users have access point implies some degree of integration of the 65 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE long-term care system. However, its establishment Slovenia is high, given the above-mentioned data does not imply an integrated long-term care system and compared to the average of fourteen OECD in its entirety, as it is only one component of that countries, which amounts to 3.7. system (Pan, 1995). If we focus, for example, only on staff One of the important discussions and providing home help, as was the focus of the decisions in the introduction of integrated long- pilot projects, the research shows that staff term care is thus certainly where and what kind providing home help are ageing (Kovač, Orehek, of a single entry point to establish, i.e. which & Černič, 2020) and that this results in a higher organisation will take over this task and what will proportion of sick leave of social carers and be its role in the long-term care system. Various greater difficulty in performing more demanding, discussions on this topic have been published (e.g. physically strenuous tasks. There is also a shortage Flaker et al., 2007; Flaker et al., 2011; Črnak Meglič of staff for the provision of home help, which is et al., 2014; Lebar et al., 2017), and the latest draft overburdened, and interest in such employment of the act on long-term care (MH, 2021) provides is especially low among the younger population for the organisation of entry points at regional (Nagode, Kovač, Lebar, & Rafaelič, 2019). The units of the Health Insurance Institute of Slovenia. situation is also similar in the field of community What is also important for integrated nursing; data from Džananović Zavrl (2021) care is that an integrated team is established for example show that the staffing norm for for the implementation of care which includes community nursing in Slovenia (2,500 inhabitants professional groups in the field of healthcare and per one registered nurse) is exceeded by an social care. The only comprehensive analysis average of 8%. The norm has been exceeded in of staff in long-term care that has so far been two-thirds of statistical regions in Slovenia (data made for the situation in Slovenia (Smolej Jež, for January 2020). This is especially important Nagode, Jacovič, & Dominkuš, 2016) has shown given the fact that an effort is being made in that about 70% of staff in long-term care provide Slovenia to promote community-based care or care in institutions, 30% at home or in community home care, which was also the subject of the pilot forms, which roughly mirrors the ratio of total projects, and which means an even greater need expenditure on long-term care (77.7% vs 22.3%). to employ staff in these two long-term home care By occupational groups, the most staff in long-term services. It is necessary to think about how to care are nursing carer (22.4%), followed by nursing acquire staff at the national level, especially in assistant or state enrolled nurses (17.7%), family the sense of incentives for (potential) interested assistants and recipients of partial compensation persons from Slovenia and in the sense of for lost income (14.7%), registered nurses (13.4%), attracting a labour force from the neighbouring (social) carers (11.4%) and custodians (8.8%), countries; quality data support will be required to providers of community nursing (4.9%) and prepare projections. personal assistance at home (2.9%), homemakers Coordination must be established between all (2.8%), persons employed in housing groups (0.7%) teams involved in providing integrated long-term and staff in psychiatry (9.4%). care. The coordinating role is performed by the Smolej Jež et al. (2016) also report that in long-term care coordinator, who also prepares 2015, for every 100 people aged 65 or older, personal plans with the beneficiaries. In addition 3.1 formally employed persons provided long- to the coordination of the teams involved in term care in Slovenia, and that for every person providing long-term care, in order to ensure the providing long-term care in Slovenia there most integrated care possible it is also necessary were 5.3 recipients of long-term care services. that all stakeholders working in the field of long- According to the authors, the workload on staff in term care in the environment are connected and EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 66 that they cooperate. Their common interest in cooperation was established in the environments. identifying and satisfying the needs of the local It is therefore the concern with monitoring the population for long-term care and the vision and basic structures in the project that enabled the commitment in providing quality long-term care is projects to be implemented in the first place. needed, which was precisely the purpose of the pilot The following structural indicators were projects. In order to ensure quality and efficient monitored: integrated care, it is also necessary to coordinate ▷ establishment of a single entry point by 1 and unify a number of procedures. To this end, it February 2019, is necessary to establish protocols and standards ▷ employment of an adequate number of qualified that are created and used by all those involved in assessors in the field of long-term care by 1 providing long-term care. On the one hand, it is February 2019, therefore important to formalise the cooperation ▷ employment of at least one qualified long- process, which ensures that all stakeholders work term care coordinator with precisely prescribed in unison, and on the other hand, the process of education and years of work experience by 1 creating protocols that encourages stakeholders February 2019, to find appropriate solutions and methods of ▷ establishment of an integrated care team cooperation is also important. consisting of precisely prescribed qualified professional profiles by 1 February 2019, ▷ all relevant stakeholders of the pilot project Methodology informed about all planned activities as part of the pilot project, an introductory meeting by 15 One of the goals of the evaluation of the pilot September 2018, then updated communication project was to evaluate the establishment and with the environments for the entire duration of placement of an entry point for long-term care. the project, In doing so, we determined whether the single ▷ a local project council is established in each pilot entry point for long-term care was established as environment by 30 November 2018, planned in the public call for applications, and ▷ protocols on the participation of all stakeholders whether its placement was appropriate from in the pilot environment established by 1 February various aspects. The establishment of a team for 2019. integrated care and networking of stakeholders Data and information on the establishment in the pilot environments was also crucial for of single entry points, employment of staff in the the establishment and launch of projects in project and networking of stakeholders in the the environments. To this end, we monitored pilot environments was obtained through various whether organisational forms of cooperation and research methods. We first conducted in February networking in the field of long-term care and other 2019 semi-structured interviews with project areas suitable for the integrated implementation managers and long-term care coordinators in all of long-term care have developed in the pilot environments in which we were interested environments. in what the main reasons were for them to apply In this part, the project was evaluated mainly for the call for applications and how this took by means of structural indicators. We were place, how the consortium group was formed in therefore interested in when the single entry point the environment, how the initial organisation was set up, how the employment of professional and implementation of the project took place and workers (assessors, long-term care coordinators how the computerisation of processes took place. and service providers in the integrated care team) To monitor and study the establishment of single was conducted and how and with which partners entry points, we also used some findings or results 67 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE of the deliberative discussion at the democratic Establishment of single entry forum that featured assessors employed at single points entry points and expert assessors23 and findings from the focus group with expert assessors. In the Celje pilot environment, it was Employment in the pilot environments was initially planned that the single entry point monitored through a list of employees, to which would be in the premises of the Health Insurance the pilot environments added information Institute of Slovenia, but because cooperation about new employments or terminations of has not materialised, the single entry point was employment. The evaluation team obtained established in the premises of the leading partner the updated lists from the pilot environments - the Celje Health Care Centre. In the Dravograd at its own request. The data for this part of the pilot environment, the single entry point was evaluation and for monitoring of the cooperation established at the Ravne na Koroškem Health between stakeholders was also obtained during Care Centre and not at the premises of the leading the project by means of visits to the environments partner in the project, i.e. the Koroška Care Home. and communication via e-mail or telephone. This It was established at the Ravne na Koroškem communication was particularly intensive before Health Care Centre with the aim of continuing the drafting of all, and especially the process with the construction of a long-term care centre reports. The participation of stakeholders in the after the completion of the project in Ravne na environment was also studied by means of a Koroškem. The leading partner in the project later survey that included all relevant stakeholders estimated that it would be better to place the single in the field of long-term care in the pilot entry point in a unit of the Koroška Care Home in environments (i.e. not only consortium and Slovenj Gradec, as it faced organisational problems additional partners of the pilot projects) and that due to the locations (care providers, long-term was completed at the end of the project (more in care coordinators and the single entry point) being the Evaluation of pilot projects and methodology dispersed. In the Krško pilot environment, the chapter). single entry point was established as part of the Krško unit of the Posavje Centre for Social Work, which was the leading partner of the project, Results although not in the same location as the centre for social work, but in new premises in its immediate We present below how the establishment vicinity. The single entry points were therefore not of the single entry point took place in the pilot placed in the existing infrastructure, as was done environments, in particular when and where in Celje and Dravograd, but the premises for the it was established, and how the employment of project were completely refurbished and adapted. assessors at the single entry points took place. We In Krško, the single entry point was describe the process of establishing integrated established a few months later than in the other care teams and employment of long-term care two environments (Dravograd in October 2018 and coordinators, and at the end we present how Celje in November 2018), in early March 2019, as different stakeholders in the field of long-term the new premises were approved for use only in care were integrated in the pilot environments. February 2019. The structural indicator that was monitored as part of the evaluation determined, as a criterion for fulfilling the indicator, that single entry points should be established by 1 February 23 Prior to the start of the pilot activities, the Ministry of Health trained a team of expert assessors to serve as support to the newly employed assessors in the pilot projects. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 68 2019, which means that, if the official opening problems in Dravograd, and they were especially of single entry points is taken into account, the pronounced in Celje. In the Krško and Celje criteria were met by the Celje and Dravograd pilot pilot environments, all four different profiles environments, while the criteria of the indicator of assessors as envisaged in the public call for was not met in Krško. It should be taken into applications (graduate social worker, registered account that applications were accepted, and nurse, graduate occupational therapist and eligibility assessments were performed in Krško graduate physiotherapist) were employed, while in even before the opening of the single entry point Dravograd, a graduate occupational therapist and in the new premises, so the process of inclusion of a graduate physiotherapist were not employed, applicants in the project was not delayed for this despite efforts being made, as such staff did not reason. As in Celje and Dravograd, applications apply for the job vacancy. were accepted as early as at the end of 2018. In Krško and Dravograd, an appropriate Based on the experience of the project, the number of assessors were employed by 1 February participants in the democratic forum agreed at the 2019 and thus the criterion of the structural end of the project that it is important for a single indicator was met. All assessors were employed entry point to be established as part of the same in these two environments for the entire duration type of organisation throughout Slovenia. This will of the project. In Celje, the challenges in ensuring ensure that people, regardless of where in Slovenia an adequate number of assessors were significant they will need care or information on care, will and lasted throughout the project. Despite the fact know where the single entry point is located. that there was a great need for assessors in the However, the participants in the democratic environment, the plan to employ seven assessors forum did not identify one type of organisation as envisaged in the public call for applications by that would be the most appropriate in which to 1 February did not materialise. There were seven establish a single entry point. They thought that it employed assessors in the environment only in could be established either in the municipality, at the periods from 1 August 2019 to 30 November the regional unit of the Health Insurance Institute 2019 and from 8 February 2020 to 31 August 2020, of Slovenia or at a new location, independent of which means that for most of the duration of the other organisations. The participants agreed that project there were not enough assessors, and a single entry point should be as independent the criteria of the structural indicator were not as possible from long-term care providers, i.e. met. Contributing to this were the challenges in autonomous, although it should be noted that employing an adequate number of assessors, while single entry points in the pilot environments were five assessors left their jobs during the project. not completely separate from the implementation team, as assessors and care providers in Celje and Krško were situated at the same location, Employing a long-term care and they also had joint meetings. The employees coordinator and establishing an in the project also noted the positive side of the integrated care team cooperation between the assessors and providers, as they also benefited from the mutual exchange of The central profile in integrated care is the information in certain situations. long-term care coordinator, for whom it was The course of employment and the provision envisaged in the call for applications that they of the estimated number of assessors at the single will, among other things, coordinate the care entry points varied considerably between the team, the newly established single entry point and environments during the project. While there informal carers, including organised volunteers (JR were no such problems in Krško, there were such 2018) (more in the chapter Personal planning and EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 69 Table 1: Employees in the integrated care team on 1 February 2019 Celje Krško Dravograd Graduate social worker 3 1 1 Graduate physiotherapist 1 1 0 Single entry point Graduate occupational therapist 0 1 0 Registered nurse 0 1 3 Total 4/7* 4/4* 4/4* Graduate social worker 0 1 0,5 Long-term care coordinator Registered nurse 1 0 0,5 Total 1/1* 1/1* 1/1* Social carer 2 0 3 Care team Nursing carer 1 0 0 Nurse assistant 3 4 4 Total 6/9* 4/7* 7/7* Graduate occupational therapist 1 1 1 Graduate physiotherapist 1 1 1 Independence maintenance team Graduate social worker 0 1 2 Master of kinesiology 0 1 0 Total 2/6* 4/4* 4/4* * number of jobs envisaged in the public call for applications EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 70 coordination in long-term care: identifying needs field of organisation. A long-term care coordinator and planning care together with the user). was employed in all pilot environments by 1 In the pilot environments, one post was February 2019, which means that the indicator planned for this work. In the Dravograd pilot has been fulfilled. environment, two people were employed for this In order to provide long-term care services purpose, both of them part-time: a graduate social in the project, an integrated long-term care team worker and a registered nurse. A graduate social was established in each environment, consisting worker was employed at the Koroška Care Home, of a care team and an independence maintenance while a graduate social worker was employed at team. As in the employment of assessors, there the Slovenj Gradec General Hospital. Both long- were significant challenges in the Celje project term care coordinators performed their job from environment in regard to ensuring staff in these the beginning to the end of the project. two teams, and there were also some problems In the Celje pilot environment, three long- in ensuring adequate staff in the Dravograd term care coordinators were replaced during the pilot environment, while in Krško there were project; all of them were full-time employees. no major challenges in the employment of an The first of these, who had a master’s degree in integrated team for long-term care. In Krško, nursing care, was in the post from 1 October 2018 the staff structure was adjusted from the very to 31 March 2019, the second, a social worker, beginning, and the care team did not employ the worked from 15 March 2019 to 31 December 2019, entire staff as envisaged in the public tender. Four while the third, also a social worker, worked from nurse assistants were employed, while a nursing 1 January 2020 onwards. carer position and a social worker position were In the Krško pilot environment, a social not filled. It was noted already in the application worker was employed as a long-term care for the public call for applications that only four coordinator at the beginning of the project, instead of seven professional workers will be but stopped working on 30 November 2019. employed, as home help had already been well The environment was without a long-term developed in the environment and there was care coordinator until 21 May 2020, and their a sufficient number of social carers (25). Later, duties were performed by a graduate social in November 2019 and in February 2020, an worker, who otherwise performed the work of additional two nurse assistants were employed an assessor at a single entry point, and who also because a need for this emerged and in order to met the requirements for the post of a long-term prevent a waiting list from being created. The care coordinator. On 21 May 2020, a new social independence maintenance team employed four worker was hired in the post of a long-term care professional workers, specifically a graduate coordinator, who worked until the end of the occupational therapist, a graduate physiotherapist, project. a graduate social worker and a master of The structural indicator criterion envisaged kinesiology. All worked in both teams until the end that at least one qualified long-term care of the project. coordinator will be employed in each pilot In Dravograd, the care team employed seven environment by 1 February 2019. The long-term professional workers, as envisaged in the public care coordinator had to hold a valid licence for call for applications, but despite efforts being independent provision of nursing care or have the made, it was not possible to employ a nursing education level of a graduate social worker with a carer, as none applied for the job vacancies. professional examination and have at least three During the project, a total of nine professional years of work experience in the field of healthcare workers were employed in the team, specifically or social care and additional knowledge in the seven nurse assistants and two social carers. One 71 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE registered nurse changed jobs and continued to project, specifically six graduate physiotherapists, work as an assessor at a single entry point, while two masters of kinesiology and one graduate three nurse assistants stopped working on the occupational therapist. Three employees in the project, which left five professional workers on team (one master of kinesiology, one graduate the team in the environment in the last months physiotherapist and one graduate occupational of the implementation of the project, instead of therapist) stopped working on the project during seven. A total of six professional workers were its implementation. employed in the independence maintenance The indicator criterion envisaged that one team, specifically three graduate social workers, integrated care team consisting of precisely two graduate physiotherapists and one graduate prescribed qualified professional profiles will occupational therapist. Two employees in the be established in each pilot environment by 1 team stopped working on the project during its February 2019. The indicator was not met in any of implementation. A master of kinesiology was not the environments. In Krško, all the planned profiles employed despite efforts being made, as no one and the appropriate number of professional applied for the job vacancy. workers were employed in the independence In the Celje pilot environment, there were maintenance unit, while four instead of seven significant challenges in ensuring an adequate professionals were employed in the care team number of professional workers in the integrated by 1 February 2019. In the Dravograd pilot care team. The number of employees in the care environment, an adequate number of professional team envisaged in the call for applications was workers were employed both in the independence reached only in April 2019. Due to the perceived maintenance team and in the care team, although need for care, the care team was increased to 11 the latter did not employ a nursing carer, who was employees in May 2019 (the call for applications envisaged as mandatory staff in the team. In the envisaged nine). The goal was to employ 12 of Celje pilot environment, an appropriate number them in the environment, and to employ relatively of professional workers were not employed either fewer professional workers (than envisaged in the in the independence maintenance team or in the public call for applications) in the independence care team by 1 February 2019, and both teams maintenance team (three instead of six). The combined had eight employees instead of the higher number of employees in the care team than envisaged fifteen. was envisaged in the public call for applications was maintained until July 2020, after which the team was reduced to nine professional workers in Networking and cooperation accordance with the instructions of the contracting of stakeholders in the authority of the project. During the project, a environment total of 19 professional workers were employed in the team, specifically nine social carers, six An important element in providing integrated nurse assistants and four nursing carers. Ten long-term care is networking anf cooperation of employees in the team (four social carers, three various stakeholders in this field. It was envisaged nursing carers and three nurse assistants) stopped in the project that, in addition to the project working on the project during its implementation. applicant (leading partner), other long-term care The envisaged number of employees in the providers in the local environment would be independence maintenance team was reached included in the consortium in an individual pilot in the environment only in May 2020. A total of environment as mandatory partners (home help nine professional workers were employed in providers, institutional care providers, health care the independence maintenance team during the centres, community nursing service providers in EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 72 the public network and the competent centre for indicator criterion that all relevant stakeholders social work). In addition, it was possible to involve must be informed about all planned activities as other stakeholders in the field of long-term care in part of the pilot project by 15 September 2018. the environments as additional partners. As part of During the project, the leading partner in networking and cooperation between stakeholders, the Celje pilot environment had regular monthly the project envisaged an introductory event meetings with the consortium partners, resulting at which all stakeholders in the environment in a total of 15 meetings. In the first months of the would be acquainted with project activities, the project until February 2019, four meetings of the establishment of a local project council and the consortium partners with the Ministry of Health creation of stakeholder cooperation protocols in also took place. Seven meetings of formal long- the field of long-term care. term care providers in the local community with In the Celje pilot environment, the leading representatives of the pilot project, three meetings partner in the project was the Celje Health Care of services in the field of care for older people Centre, which provides community nursing, while at home in the environment, and two meetings also included in the consortium as mandatory of representatives of the pilot project with the partners were the Celje unit of the Celje Centre for community nursing were also held. Social Work, the Dom Sveti Jožef Celje Educational Nine meetings with the consortium partners and Pastoral Centre and the Public Institute Socio. were held in the Dravograd pilot environment Another 14 partners participated in the pilot during the project. Five meetings were held in project. The leading partner in the Dravograd pilot the environment with representatives of the environment was the Koroška Care Home, which smart watch developer, and two with the e-care also provides home help in the environment. provider. Two meetings were held with home Also included in the consortium as mandatory help providers and one each with the Paraplegics partners were the Ravne na Koroškem Health Care Association and the Credit Point Club. The pilot Centre, the Slovenj Gradec General Hospital, the environment often cooperated with municipalities Koroška Intergenerational Centre and the Ravne in the environment, especially in the preparation na Koroškem unit of the Koroška Centre for Social of training sessions for informal carers. Work. In addition, another 28 stakeholders from In the Krško pilot environment, many the Mežica Valley, Dravograd and Slovenj Gradec stakeholders in the environment had one meeting were involved in the project. In the Krško pilot each with most of the support partners. They met environment, the leading partner in the project was twice with all project partners, and twice with the Krško unit of the Posavje Centre for Social Work, all consortium partners. In addition to meetings, which provides home help in the environment, the environment was also often in contact with while also included in the consortium as mandatory stakeholders by telephone, and also cooperated partners were the Municipality of Krško, Krško with them in organising various activities and Health Care Centre, Brežice General Hospital, Krško implementing long-term care, which are listed in Care Home and the Krško-Leskovec Special Social Table 2. Care and Employment Centre. Another five partners A local project council was established in participated in the pilot project. all pilot environments, consisting of all partners In the Celje pilot environment, stakeholders involved in the project, and in the Krško pilot in the field of long-term care were informed about environment, 12 other stakeholders from the all planned activities in the project in October environment were included. The indicator 2018, in Dravograd in November 2018 and in Krško criterion envisaged that local project councils in September 2018. The Krško pilot environment would be established in the environments by is also the only environment that has met the 30 November 2018, which was achieved only 73 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 2: Other forms of cooperation with stakeholders in the Krško pilot environment Stakeholder Type or content of cooperation Regular cooperation in the creation of informative material with which Various stakeholders potential users were informed about the services provided by individual stakeholders in the environment. Brežice General Contacts were frequent in order to ensure safe discharge from the hospital, Hospital which was also defined in the protocol on cooperation. Krško Care Home Cooperation in the absence of informal carers - it temporarily took in users of long-term care services at home (in a limited number). Krško Health Care Cooperation with the community nursing took place on a daily basis. Centre Occasionally, direct contact with their personal physicians was also needed for certain users. Health Insurance Together with the HIIS, single entry point informed users about the rights Institute of Slovenia that individuals have in the field of health or social insurance. An example of (HISS) such cooperation is the leaflet entitled The Right to Assistive Devices Aristotel Health Centre Cooperation was established with the centre's nurses and doctors as needed. Posavje Adult Mental The pilot project represented a link between the Mental Health Centre and Health Centre the Brežice General Hospital in regard to treatment of users. Pharmacies Carers brought the necessary medications to the homes of some users. Organised a "Lifelong Camp", which was also attended by an employee of the Sožitje Association care unit in the pilot project, who accompanied three people with intellectual disabilities at the camp. Brežice Secondary Together, these worked to open a department for the training of medical School of Commerce technicians. A joint visit to the Ministry of Education, Science and Sport was and Economics also planned. Brežice Health Care Centre: Health Cooperation was established in the field of prevention in user healthcare. Promotion Centre: Tačke pomagačke The user was provided with assistance by means of therapy with dogs. Association The occupational therapist employed in the project also attended a 2-day professional training session entitled: Basics of Work with a Therapy Dog. Posavski obzornik newspaper Informed the public about the activities of the project. Sonček Posavje Association for Organisation of workshops in which a kinesiologist employed in the project Cerebral Palsy also participated. Krško-Leskovec Special Social Care A kinesiologist performed exercises with protégés of the special social care and Employment and employment centre twice a week. Centre Source: Data from the Krško pilot environment EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 74 in the Dravograd pilot environment, while the Discussion with key local project council in Krško was established in messages December 2018 and in Celje in February 2019. The local project council met twice in Celje, three For many years, Slovenia has been striving to times in Krško and four times in Dravograd. regulate the field of long-term care in an integrated The Covid-19 epidemic affected the number of way and thus provide people with quality and meetings, as no meeting took place in the Celje accessible services. One of the important elements pilot environment in 2020, and only one meeting of how to ensure this is primarily to ensure a single each was held in Dravograd and Krško. In Krško, entry or single entry points. this meeting took place online. The environments As regards the form of organisation as also noted that it was challenging to coordinate part of which it would be most appropriate the dates of meetings with numerous to establish a single entry point, taking into stakeholders. account the experience of the project, it is not Protocols on the participation of possible to determine what organisation is most stakeholders involved in the project were also suitable for this. It can be established either developed in the environments. In the Celje pilot in the municipality, at the Health Insurance environment, a shorter protocol was developed Institute of Slovenia (local unit), within another in the second half of 2019 and in Dravograd at the organisation (e.g. health care centre or centre for beginning of 2020, which means that the indicator social work) or at a new location, independent criteria had not been met, as it envisaged that the of other organisations. While the long-term care protocol would be developed by 1 February 2019. bill (2021) envisages that single entry points In the Krško pilot environment, the protocol was will be established at the regional units of the established as early as in October and November Health Insurance Institute of Slovenia, this has 2018, so the indicator was met. not been tested in the pilot projects. If any of At the end of the project, the stakeholders24 the environments managed to test a single entry of key importance for the field of long-term point at a regional unit of the Health Insurance care in the pilot environments who responded Institute of Slovenia, this would be an important to the questionnaire (39 stakeholders from all contribution to testing the solutions provided for environments participated) agreed in the vast in the bill (2021). It is important that the single majority (92.6%) that, in order to provide long- entry point is recognised in the environments and term care, healthcare and social care services be accessible to potential users or applicants and should be combined and coordinated. Fewer than their relatives, i.e. that it is not located too far from half (43.6%) of the participating stakeholders them, that free parking spots are provided in its agreed that the various organisations involved vicinity and that it is also accessible to people who in the provision of long-term care are well may have difficulties with access due to disability. connected. Less than a third (30.7%) of The premises of the single entry point must be stakeholders also agreed that the transfer of arranged in such a way as to ensure discretion information between different long-term care in communication between assessors and the providers is effective. Responses of stakeholders applicant, their relative or another person who indicate that they are aware that integrated and visits the single entry point. coordinated long-term care services are essential, In the project, the single entry point was while networking and cooperation between separated from the service providers in the stakeholders in the existing system is not yet Dravograd pilot environment, while the single satisfactory. entry points in Celje and Krško were located at 24 For more information about the questionnaire for stakeholders, see the chapter Evaluation of pilot projects and methodology. 75 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE the same location as the care providers. There networking in the field of long-term care have were also many joint meetings, so in these developed in the environments, the project environments the single entry points were not monitored how the pilot environments were more completely separate from the care providing team. broadly connected with the local environment The long-term care coordinator and the care team and key stakeholders and how they were included also faced certain challenges in the organisation in the project. The leading organisations of the of work in Dravograd, due to the single entry pilot projects were those that in the existing point being dislocated. Although it is important system of long-term care operate as part of that the single entry point and the care providing healthcare or social care and provide various team are separate and that the assessors do their services – community nursing, institutional care work completely autonomously, positive aspects of and home help. All pilot environments had a cooperation between all employees were perceived health care centre, a care home and a centre for in the environments. Sharing of information about social work in the consortium, which is crucial users proved to be useful, so they participated in from the point of view of providing integrated some meetings. care, as it includes key organisations from both The pilot environments faced various healthcare and social care. An event at which challenges in employing the envisaged number of the project was presented to local stakeholders staff, as for example they failed to employ certain in the field of long-term care was held in all profiles (e.g. in Dravograd) or faced a significant environments in the first months of the project. turnover of employees (e.g. in Celje). The challenges Local project councils were also established of employment in long-term care are not specific in all pilot environments, in which all project to this project alone. These challenges have been partners were involved; other stakeholders (12) increasingly noted by long-term care providers at were later involved in it at a later stage. Both the national level, which is why problems in this the introductory events at which the project field must be addressed strategically, on the one was presented to local stakeholders and the hand in terms of incentives to increase recruiting local project council were mostly organised or power, and on the other hand by reflecting on established later than envisaged in the indicators, whether staff in Slovenia can be secured from the which indicates that it takes more time than domestic labour force alone or if the labour force envisaged for cooperation between stakeholders for this work will have to be imported. What also to be established, i.e. at least a few months. There contributed to the challenges related to employment were not many meetings of local project councils, was the fact that it was employment on a project with one of the reasons for this being the Covid-19 for a definite period. This problem is particularly epidemic, as meetings in person were not possible pronounced in such projects towards the end of most of the time in 2020, and decisions to adopt projects, when employees are looking for new virtual ones were not made except in Krško. jobs, while it is difficult to recruit new employees From the aspect of overcoming the difficulties for this shorter period at the end of projects. Staff of involving a large number of participants in turnover is also a challenge from the aspect of meetings, the use of technology should also be appropriate training of employees, as it is necessary considered, as this proved to be an effective to continuously implement the training provided mechanism for cooperation in certain cases to employees at the beginning of the project during the Covid-19 epidemic. throughout the project for all new employees who The purpose of local project councils in enter the project at different times. the environment was mainly to acquaint all With the aim of evaluating whether partners with the activities in the project, and organisational forms of cooperation and it was noted in Krško that the purpose of the EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 76 local project council was to coordinate and attention was paid to protocols in the other two determine professional guidelines for long-term environments. care development strategy at the local level, As part of the project, stakeholders were which probably also contributed to the project connected into consortia and partnerships, and they activities being partially maintained even after also created local project councils and cooperation the end of the project. Regular communication protocols; however, at the end of the project, in a and cooperation in the pilot environments of Celje survey involving not only partners included in the and Dravograd took place mainly with obligatory project, they nevertheless assessed that stakeholder partners, while in Krško a broad range of other participation in the environment is not yet at the stakeholders from various fields were involved appropriate level. This means that efforts should be in various activities, although fewer stakeholders made to improve stakeholder cooperation that has than in other environments were involved in also been established within projects. This is also the project. Protocols on the participation of an indicator that probably reaffirms at a broader all stakeholders in a pilot environment were level in the country that coordination between established in the pilot environments. In Krško services providing long-term care services is not in particular, protocols were recognised as an good in Slovenia and that a considerable amount important element in the organisation and of attention should be paid to this in light of the implementation of long-term care, while little transition to more integrated care. 77 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE BIBLIOGRAPHY Črnak Meglič, A., Drole, J., Kobal Tomc, B., Koprivnikar, B., Ministrstvo za delo, družino socialne zadeve in enake možnosti. Lebar, L., Nagode, M., Peternelj, A., … Toth, M. (2014). Podpora (2008). Nacionalno poročilo o strategijah socialne zaščite in samostojnemu bivanju v domačem okolju in dolgotrajne oskrbe. socialnega vključevanja za obdobje 2008-2010. Analitsko poročilo DP5 projekta AHA.SI. Retrieved from http:/ www.staranje.si/files/upload/images/aha.si_dolgotrajna_oskrba_ Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih porocilo_v1.pdf projektov, ki bodo podpirali prehod v izvajanje sistemskega zakona o dolgotrajni oskrbi«. Retrieved from https:/ www. Dominkuš, D., & Peternelj, A. (2006). Dolgotrajna oskrba v Sloveniji, uradni-list.si/glasilo-uradni-list-rs/vsebina/2018002400003/ Seminar ob pripravi Nacionalnega poročila o strategijah za javni-razpis-za-izbor-operacij-izvedba-pilotnih-projektov-ki-bodo- socialno zaščito in socialno vključevanje 2006 – 2008: Ministrstvo podpirali-prehod-v-izvajanje-sistemskega-zakona-o-dolgotrajni- za delo, družino in enake možnosti. oskrbi-st--303-1201816-ob-198618 Dominkuš, D., Zver, E., Trbanc M., & Nagode, M. (2014). Long-term Nagode, M., Zver, E., Marn, S., Jacović, A., & Dominkuš, D. (2014). care – the problem of sustainable financing. Host country paper. Dolgotrajna oskrba – uporaba mednarodne definicije v Sloveniji. Peer review on financing of long term care. Ljubljana, 18-19 Urad RS za makroekonomske analize in razvoj. Retrieved from November 2014. https:/ www.umar.gov.si/fileadmin/user_upload/publikacije/ dz/2014/DZ_02_14.pdf Džananović Zavrl, D. (2021). Patronažno zdravstveno varstvo Slovenije. Zaposleni in analiza opravljenega dela. Letna Nagode, M., Kovač, N., Lebar, L., & Rafaelič, A. (2019) . Razvoj socialne publikacija, 2020/2019. Ljubljana: Nacionalni inštitut za javno oskrbe na domu: od besed k dejanjem. Ljubljana: Fakulteta za zdravje. Retrieved from https:/ www.nijz.si/sites/www.nijz.si/files/ socialno delo. uploaded/podatki/podatkovne_zbirke_raziskave/patronaza/ patronazno_zdravstveno_varstvo_slovenije_2020.pdf Pan, G. P. (1995). Long-term care: A single entry point for three populations. Retrieved from https://lrb.hawaii.gov/wp-content/ Flaker, V., Nagode, M., Rafaelič, A., & Udovič N. (2011). Nastajanje uploads/1995_LongTermCare.pdf dolgotrajne oskrbe: Ljudje in procesi – eksperiment in sistem. Ljubljana: Fakulteta za socialno delo. Smolej, Jež, S., Nagode, M., Jacovič, A., & Dominkuš, D. (2016). Analiza kadra v dolgotrajni oskrbi. Ljubljana, Inštitut RS za Flaker, V., Cigoj-Kuzma, N., Grebenc, V., Kodele, T., Kranjc, B., Pirnat, socialno varstvo. Retrieved from https:/ irssv.si/upload2/analiza_ T., … Žnidarec Demšar, S. (2007). Krepitev moči v teoriji in praksi. kadra_v_do.pdf Ljubljana: Fakulteta za socialno delo WHO. (2016) . Integrated care models: an overview. Working Goodwin, N., Sonola, L., Thiel, V., & Kodner, D. (2013). Co-ordinated document. Retrieved from https:/ www.euro.who.int/__data/ care for people with complex chronic conditions. Key lessons assets/pdf_file/0005/322475/Integrated-care-models-overview.pdf and markers for success. The King’s Fund 11–13 Cavendish Square London W1G 0AN. Vlada Republike Slovenije. (2021). Predlog Zakona o dolgotrajni oskrbi. Retrieved from https:/ www.gov.si/assets/ministrstva/MZ/ Kovač, N., Orehek, Š. in Černič, M. (2020). Analiza izvajanja pomoči DOKUMENTI/Dolgotrajna/ZDO_parafa.pdf na domu v letu 2019, končno poročilo. Ljubljana: Inštitut RS za socialno varstvo. Retrieved from: https:/ www.irssv.si/upload2/ Analiza%20izvajanja%20PND%20za%20leto%202019_30.9.2020_ pop2.pdf Lebar, L., Dremelj, P., Flaker, V., Rode, N., Mali, J., Peternelj, A., Smolej Jež, S., ... Kobal Tomc, B. (2017). Priprava podlag za izvedbo pilotnih projektov, ki bodo podpirali prehod v izvajanje sistemskega zakona o dolgotrajni oskrbiAktivnost 1: Priprava orodij za ugotavljanje potreb uporabnikov, metodika postopka ter ugotavljanje upravičenosti do storitev dolgotrajne oskrbe. Ljubljana: Inštitut RS za socialno varstvo. 79 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE FROM APPLICATION TO SERVICE: EXPERIENCE OF PROCEDURES IN PILOT PROJECTS Magdalena Žakelj Social Protection Institute of the Republic of Slovenia Mateja Nagode Social Protection Institute of the Republic of Slovenia EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 80 FROM APPLICATION TO SERVICE: EXPERIENCE OF PROCEDURES IN PILOT PROJECTS KEY MESSAGES ▶ We believe that the procedures as part of the implementation of the pilot project were appropriate and that the activities ran mostly undisrupted. ▶ Forms used in long-term care procedures should be as adapted as possible to the understanding and abilities of all groups of users of long-term care services (Braille, easy-to-read format, audio recording). ▶ Assessing eligibility at home is a practice that needs to be maintained and encouraged. ▶ What needs to be ensured in a future long-term care system is that waiting lists are generated as rarely as possible. ▶ Users should be acquainted with the complaint procedures and be provided with support in the event that they wish to lodge a complaint. 81 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Introduction comprehensive, integrated and continuous, outcome oriented and evidence based, and transparent, Today, long-term care in Slovenia is regulated while being gender and culture sensitive. The within the framework of various regulations and is Charter of Fundamental Rights of the European provided through several separate social security Union (2010) recognises and respects the right to (protection) systems that are regulated by various live in dignity and independence and to be included laws. The methods for exercise of rights and in social and cultural life. It is therefore necessary to provision of services thus do not follow the same ensure that people in need of care receive support criteria and are therefore diverse and inconsistent and assistance in a way that preserves their dignity with each other. Systematic regulation of long-term and prevents their abuse - so all of this is the subject care would ensure a uniform and comprehensive of the procedure. procedure for exercising rights in the field of The process must, to the greatest possible long-term care, from submitting an application to extent, enable people to become active, to be inclusion in long-term care and receiving services. provided a basis for achieving their goals and Coordination of procedures is recognised as one become subjects and contractual partners in long- of twelve key components in the Scirocco Maturity term care processes. In establishing the procedure, Model for Integrated Care (Scirocco, 2021). In old patterns of behaviour that make people passive, addition to the methods and provision of services, objectify them, blame them for their condition the uniform procedure is precisely what was tested and discredit them as contractual individuals, thus by pilot environments as part of long-term care excluding them from decision-making, should be pilot projects. Coordination and unification of the avoided (Lebar et al., 2017; Flaker, 2017). numerous procedures is necessary if we want to When creating the procedure, it is important ensure quality, safe and effective long-term care. to clearly define and coordinate all individual parts Procedures are a key element in establishing of the procedure: submission of an application for long-term care, in fact they are the framework that the right to services and other rights as part of long- regulates long-term care, determines the access term care, determining eligibility, procedures for and right to care, methods for exercising this right defining assistance and support and, consequently, and providing and monitoring long-term care. services to be received, and procedures that Procedures are important because they determine determine the provision of long-term care itself. the framework within which the needs for long- Included in these main parts of the procedure term care are met and services are provided, and are various actions and procedures that should they are also important because they determine the contribute to the quality provision of services manner in which individual parts of the procedure and thus user satisfaction, and through which the are implemented and in which care is provided. various mechanisms of the entire long-term care Therefore, it is not only important that a person procedure could be measured and improved (Seys receives care, it is also important how, when and et al., 2019). in what way they receive it. In this regard, both At the same time, the procedures should be general rights and long-term care rights, as set out conducted in such a way that the user understands in national and international guidelines, must be them, can follow them and implement them, and so respected in long-term care procedures. that they receive support if they need it. Throughout The European Quality Framework for long- the process, from the first contact with the services term care services (2012) clearly stipulates that to exiting the care system, the user must be at the quality long-term care must respect human rights centre of the planning and provision of care, and and dignity, be individual-centred, preventive and the task of the experts is to focus on respecting rehabilitative, available, accessible, affordable, their rights for the entire duration of the process EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 82 Figure 1: Procedure for exercising long-term care rights in pilot projects APPLICATION IS REJECTED phone call with explanation APPLICATION FOR THE EXCERCISE OF THE RIGHT SEP APPLICATION IS COMPLETE TO LONG - TERM CARE invitation to visit APPLICATION IS NOT COMPLETE call for supplementation COMPLETION OF PROJECT ACTIVITIES ELIGIBILITY ASSESSMENT in institutional care ELIGIBILITY ASSESSMENT at the applicant’s home MEETING LCC + IMT ELIGIBLE INELIGIBLE review of the eligibility / receives an eligibility receives an eligibility living circumstances assessment and an assessment and assessment announcement of the information about LCC visit other services FIRST VISIT LCC AND PROFESSIONAL WORKERS IMT making a personal plan, agreement on inclusion in e-care REASSESSMENT OF ELIGIBILITY REGULAR: MEETING LCC + IMT after six months goal setting, final EXTRAORDINARY: definition of services, when the user’s record of personal plan status changes PROVISION OF SERVICES PERSONAL PLAN IMT + CT E-care ELIGIBLE E-health receives an eligibility Telemedicine assessment MINOR CHANGE: Key: adjustment of services SEP - single entry point LCC - long-term care coordinator PERSONAL PLAN MAJOR CHANGE: IMT - independence maintenance team REVISION announcement CT - care team Annex to the personal plan of the LCC visit 83 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE (European Network of National Human Rights system of the pilot environments for updated Institutions, 2017). recording of data from the application, eligibility This article describes the course of the entire assessment, personal plan and monitoring of the procedure of the exercise and use of long-term implementation of the service. In order to obtain care rights, which was tested in pilot projects, better insight and better understand the entire with a greater emphasis being placed on the main procedure, different types of information, data and parts of the procedure as part of pilot projects documents were used. We systematically reviewed (exercising the right, assessing eligibility, personal the entire documentation and forms related to the planning and coordination of long-term care, procedure provided by all three pilot environments provision of services, e-care and e-health, waiting and reviewed the minutes of 13 meetings between lists and complaint channels). Certainly, in the pilot the contracting authority, pilot project coordinators environments, the various details of the procedure and long-term care coordinators. Based on this were also implemented in ways that we will not be documentation and discussions with the long-term able to cover in this article - on the one hand due care coordinators, a description of all phases of the to limited space, and on the other due to the lack procedure was prepared for each pilot environment of accurate data, as we were unable to delve into separately, and the long-term care coordinators obtaining such data due to the large scale of the were asked to review, elaborate on and supplement entire evaluation. the description. In addition, the evaluation of the procedure also took into account the opinions of employees, users and informal carers about Methodology the entire process and its individual phases. We reviewed employee reports on activities as part The objective of the evaluation was to evaluate of the pilot projects, interviews with users and both the procedure for assessing eligibility for informal carers, interviews with long-term care long-term care and the entire long-term care coordinators and the minutes with the conclusions procedure, from the submission of the application of the democratic forum of assessors. Univariate to the provision of services. With the evaluation, we and bivariate statistical methods were used in the wanted to evaluate whether the procedure tested in analysis of quantitative data from the information the pilot projects is suitable for the transition to the system, in which the days between individual parts systematic implementation of long-term care. Two of the procedure were calculated (for more detailed process indicators were also monitored at all times information about these measuring instruments during the evaluation of the procedure: and data, see Chapter Evaluation of pilot projects ▷ the proportion of persons assessed within three and methodology). working days of the submission of the application, and ▷ the proportion of persons eligible for long-term Results care who met with the long-term care coordinator within three working days after the eligibility The pilot environments described the assessment was performed. procedures for the promotion and implementation In order to evaluate both indicators, and and the method of termination of the provision of in order to assess the duration of procedures long-term care as part of the pilot project in the in general, we used data from the information protocol25. The draft protocol was prepared by 25 Protocol on the promotion and implementation and the method of termination of the provision of long-term care service as part of the implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care (hereinafter referred to as: protocol). EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 84 Table 1: Number of working days from the submission of the application to the first visit by the assessor (left) and from the preparation of the first eligibility assessment to the first visit by the long-term care coordinator Working Submission of the application - first visit First eligibility assessment – first visit days by the assessor by the long-term care coordinator Environ- ment Total Celje Krško Dravograd Total Celje Krško Dravograd N 1887 815 461 611 512 158 208 146 Average 7.1 4.2 1.8 14.9 24 41 11 23 Median 2 0 0 9 12 15 9 18 Modus 0 0 0 0 7 10 7 8 Standard deviation 13.7 7 4.8 20.1 40 60 11 31 Minimum 0 0 0 0 0 0 0 0 Maximum 250 71 48 250 311 311 105 307 First quarter 0 0 0 4 7 8 5 8 (Q1) Third quarter 9 7 1 20 25 44 14 29 (Q3) 85 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE the contracting authority of the pilot projects, their applications in person or by mail directly to and in agreement between the contracting the single entry point. They were often assisted authority, project coordinators and long-term care in completing the applications by home help coordinators, it was supplemented and changed employees or community nursing, who could during the implementation of the pilot projects. deliver the application to the single entry point. The course of the entire procedure during the It could be concluded from the interviews that pilot projects is shown in Figure 1 and described the application procedure seemed easy enough in the remainder of this article. for the users and their relatives, and that they received sufficient support from the employees in this part of the procedure. Exercising the right to The following outcomes were possible long-term care in processing the application: the application was complete, incomplete or rejected. The Eligible for long-term care services in the application was complete if it contained all the pilot projects were persons aged 18 or older who, information required and the applicant met due to the consequences of an illness, weakness all the aforementioned criteria, which were a related to old age, injury, disability, lack or loss condition for the eligibility assessment to be of intellectual ability, were dependent on the performed. In the Dravograd pilot environment, assistance of other persons in activities of daily all applications received were first discussed living and instrumental activities of daily living by the application team, except those for which for a longer period of time, and not shorter than the eligibility assessment had to be performed three months, and persons who were not included as soon as possible. After the team discussion, in personal assistance and who were classified in they were sent back to the single entry point, and one of the five categories of eligibility for long- the assessor then began the process of making term care as part of the eligibility assessment an eligibility assessment. In the Celje and Krško (Ministrstvo za zdravje, 2018). pilot environments, applications were reviewed The key document in exercising the right directly by the assessors and, in cases where these to long-term care in pilot projects was the were complete, an appointment was agreed for application26 that interested persons could obtain the first eligibility assessment. in various ways: on the websites of leading An application was incomplete if it did pilot project organisations, from pensioners’ not contain all the required information, and associations, from home help providers and in these cases the assessor called the applicant community nursing, from centres for social work and obtained the missing information, or the and, above all, from the single entry point. Always application was taken to be supplemented by enclosed with the application was the Consent home help or community nursing employees. for the collection and processing of personal In many cases, the application was completed information form, which was signed and submitted together with the applicant by the assessors by the applicant together with the application. themselves during the eligibility assessment visit. At the request of the applicants, the assessors There were very few cases where sent or brought the application to their home, applications were rejected - only in Dravograd, and they were also able to complete and submit where some applications were received from it at the single entry point. The pilot projects applicants from municipalities where the pilot showed that the applicants mostly submitted project was not implemented, or the applicant 26 Application for exercising the right to long-term care as part of the project »Implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care« (hereinafter referred to as: application). EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 86 was already in possession of a decision on the in the care homes on the day of the eligibility right to personal assistance. In such cases, the assessment and both were dated on the same day. applicant was called by the assessor, the reasons Meanwhile, in Dravograd, this span is the smallest for the rejection were explained and other options and even reversed, as slightly less than a tenth for services that could be used were suggested. of applicants from institutional care, and more than a tenth of applicants who lived at home were visited the same day. This balance is also a result Eligibility assessment of the work of the application processing team, which treated all applications received equally. After the application was received, the In addition, for applicants in institutional care, applicant and their carer or relative were visited assessors usually made eligibility assessments by an assessor, who initiated the eligibility when there were not many applications from assessment procedure. A visit by the assessors domestic environments in the single entry point. should take place as soon as possible, and practice The assessment of eligibility for long-term has shown that half of the applicants were visited care had two possible outcomes: the applicant by the assessor on the second working day after was either eligible for long-term care services receiving the application (Me=2) (Table 1). In the or not eligible. They were informed about the Celje and Krško pilot environments, half of the results with the Eligibility Assessment form28, applicants were visited on the same working day which, in addition to the eligibility category and (Me=0), and in the Dravograd pilot environment other information, contained a description of within nine working days (Me=9). Within three the applicant’s living circumstances. In Celje, in working days, which was the criterion followed in agreement with the contracting authority, this the evaluation, 51.9% of applicants were assessed practice was terminated after a few complaints in pilot projects: most of them in Krško (79.9%), were received regarding the written content of followed by Celje (58.5%), the least in Dravograd living circumstances of applicants who were not (20.6%). The longer period in Dravograd is likely eligible for services. In general, according to the and mostly a consequence of the application long-term care coordinators, the assessors decided processing team extending the application how much content regarding living circumstances processing procedure by a few days. to record at their own discretion, in particular The assessors made multiple visits within when this included content to which the applicant a short period of time to applicants from could react negatively (e.g. excessive alcohol institutional care in order to make an eligibility consumption, causes of violence, descriptions of assessment; to more than 60.0% as soon as the the applicant or their living environment being first working day, and only a quarter of applicants unkempt, etc.). The assessors kept such statements were visited at home (Q1=0). The difference is separately, and they could be accessed by the statistically significant (U=308258, p=0.00027). In long-term care coordinator and, if necessary, by Celje, about 90.0% of applicants from institutional others who worked with a specific user. care and about 10.0% of those living at home The applicant was not eligible for long-term were visited on the same working day. The span is care services if, on the basis of the eligibility also relatively large in Krško (over 80.0% of visits assessment, they did not reach the threshold for on the same day in institutional care and just inclusion in any of the five categories of long-term under 60.0% at home). This is attributed to the care eligibility. In such a case, the assessors in the fact that the assessors took on many applications Dravograd pilot environment usually revisited the 27 Due to the large dispersion of data, non-parametric bivariate tests were used to calculate statistically significant differences. 28 Assessment of eligibility for long-term care as part of the project »Implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care« (hereinafter referred to as: eligibility assessment). 87 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE applicant at home, handed them a letter with an assessment directly, but the employees filed them eligibility assessment and forms for regulating the in their personal files in the institutions where rights under existing regulations or documents they lived or also in the file in the single entry with information on what kind of assistance, point. For them, their involvement in the pilot depending on their needs, they can turn to in the project ended here29. local environment. In such cases in the Krško After they acquired the right to long-term pilot environment, the assessors sent a letter to care services, those who resided at home were the applicants with an eligibility assessment and re-assessed at regular, six-month intervals a copy of the signed Consent for the collection (regular assessment) or in the case of a change and processing of personal data form. The letter in the ability of self-care due to, for example, was accompanied by leaflets with information improvement or deterioration of the health or about other services and services for which it functional condition of the user, immediately was assessed, based on an interview with the after the change occurred, i.e. before the applicant during the assessment, were of potential expiration of six months (extraordinary benefit to them. In the Celje pilot environment, assessment). The change was communicated the ineligible applicants were sent a letter with to the long-term care coordinator either by the an eligibility assessment at the beginning of the service providers or by the user themselves or project, and later only a letter informing them their relatives, and the long-term care coordinator of their ineligibility and listing the organisations further communicated it to the assessors in the they can still turn to for help. single entry point. An applicant was entitled to long-term When the user’s condition improved and care services if they were placed in one of the they were no longer entitled to long-term care five eligibility categories. The assessors sent services after a repeated, regular or extraordinary the eligibility assessment to the long-term care assessment, the assessor informed them of the coordinator, and a letter with the eligibility outcome as in the first assessment. At the same assessment to the applicants; in Krško this time, they also informed the long-term care included a photocopy of the Consent for the coordinator, who closed the user file. In the event collection and processing of personal data form, that the user continued to be eligible for long- and in Celje they were sent, in addition to the term care services, the assessor also informed eligibility assessment, a letter informing them of them in the same way as in the first eligibility their inclusion on a waiting list once a waiting assessment. The provision of services continued, list was compiled, and they were also informed and the scope of services was adjusted to the about the possibility of being included in e-care increased/decreased needs. The long-term care services and presented with a list of organisations coordinator was also informed about everything. they can still turn to for help. Resulting from the In the interviews, users expressed different Dravograd pilot environment, cases were cited in opinions about the assessment procedure, and an which the beneficiaries did not want to receive analysis of the interviews shows that users were services immediately after the assessment, as generally aware of the procedure: they said how they submitted applications “as a reserve”. Some many assessments they had and who conducted joined later, and some did not do so at all. them, how long the interview lasted and what Applicants from institutional care did the outcome was of the assessment. On the other not receive the forms with the final eligibility hand, some users were not able to say anything 29 The participation of users of institutional care services was limited to the preparation of the eligibility assessment, as the purpose was to test the assessment tool and categorisation, which otherwise had no effect on their eligibility for institutional care services under current regulations. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 88 about the assessment and did not remember this it was necessary to obtain an opinion of the part of the procedure, or did not name it as such. personal physician at this stage of the procedure31. After conducting the eligibility assessment with the In all three pilot environments, personal applicant, in cases in which the assessors did not physicians were asked to provide such opinions receive enough information from them, they also by the long-term care coordinators themselves. turned to the present relatives or informal carers. In cases where this was absolutely necessary Assessors and other staff in the pilot projects (discharge from hospital, sudden change in the were generally of the opinion that the eligibility provision of services by other providers in the assessment procedure was designed in a way that local environment), long-term care coordinators allows them to obtain sufficient information on ensured that the provision of services began the applicant’s condition relevant to long-term immediately, before the personal plan was care planning. finalised. On average, the long-term care coordinator visited the beneficiary within 24 working days Personal planning and after making the eligibility assessment, with coordination of long-term care individual cases ranging from the same day to over 311 working days (Table 1). Most visits After receiving the eligibility assessment, were made within seven days (M0=7), and half of the long-term care coordinator agreed on the them were made within 12 days (Me=12). There date of the first home visit with the beneficiary by are statistically significant differences between telephone. The purpose of this visit was mainly the pilot environments (K-W=8.199, p=0.0177): to discuss the needs of the beneficiary, to obtain in Krško, the long-term care coordinator visited information for the preparation of the personal half of the beneficiaries within nine working plan30 and to agree on the implementation plan days (Me=9), in Celje within 15 (Me=15) and in as part of the personal plan.Usually, the long- Dravograd within 18 days (Me=18). We find that term care coordinators dedicated one visit to 11.7% of the beneficiaries met with the long-term this, rarely more, as they were instructed by the care coordinator within three working days after contracting authority to obtain information for the eligibility assessment: 13.9% in Krško, 11.0% the personal plan in one visit. in Dravograd and 9.5% in Celje. In the first visit, the long-term care According to the long-term care coordinator informed the beneficiary about the coordinators, various activities (e.g. telephone manner in which services are provided and the conversations, home visits) had already taken provisions of the protocol. If the beneficiary place with the beneficiary after the eligibility decided to use e-care and e-health services, they assessment, although they could not be recorded were given all the forms related to the inclusion to in the information system because the user was be signed and informed about further procedures not yet active in terms of receiving services, or related to these services. they could not be recorded because employees did In cases where the personal plan provided not have this option in the selection of services or for the implementation of the “monitoring of in the code list. In some cases, these services were prescribed therapy” service or the “measuring of not entered retroactively after the information vital functions” service, or when the beneficiary’s system was established and upgraded, so there health condition was of more complex nature, may be a discrepancy between the data from the 30 Personal plan for the provision of long-term care as part of the pilot activities »Implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care« (hereinafter referred to as: personal plan). 31 Opinion of the personal physician or treating specialist on the relevant health condition of the insured person (hereinafter referred to as: opinion of the personal physician). 89 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE information system and the actual situation in the with the provision of services or in other cases pilot environments. The large gap in the number when personal intervention of the long-term of days between the eligibility assessment and care coordinator was necessary (for example, the visit by the long-term care coordinator was intervention in a certain case where, despite mainly due to the fact that, as already mentioned, growing problems due to dementia and the risk applicants also submitted applications “as a brought about by their health condition, the user reserve”, just in case they happen to need services still wanted to continue driving their car and in the future, while they currently have support disregarded warnings from their carer). ensured by relatives and/or informal carers. Some In order to have the personal plan signed, simply changed their minds after the assessment the pilot environments resorted to other practices and did not yet want to receive services. in addition to the above-mentioned: the first The personal plan was signed by the long- employee who visited the user gave them the term care coordinator and the beneficiary, with personal plan to sign it, or the personal plan was each keeping their own copy. By doing so, they sent by mail, and the employees who provided confirmed that they had been acquainted with services brought it back signed. and that they agreed with the content and would Otherwise, the long-term care coordinator take their share of responsibility, while and at the communicated with the user or their relatives same time the signing of the personal plan formed by telephone, most often in cases of a change the basis for the start of the provision of services. (temporary or permanent) in the provision If the eligibility category was changed after of services, which is also stated by users in the re-assessment and it transpired that a major interviews: “She also keeps me informed of change in the scope of services was needed, the when she’s coming” , “Yes, she calls if there is long-term care coordinator prepared an annex any change. For instance, if someone who visits, to the personal plan32 in which they recorded the say a physiotherapist, goes on an annual leave. change, new goals and services to achieve the She comes on Thursdays, for instance, and if she goals. In these cases, they visited the user, and in couldn’t, then she would come on a specific day, cases of minor changes in the scope of services, if I agree” . The long-term care coordinator kept they only informed the user about them by phone. the schedule of the provision of service for the When the annex to the personal plan was made, care unit, and the employees of the independence the user received two copies to sign and kept one maintenance unit agreed themselves on the dates for their personal records. of the visits with the users. The long-term care coordinator visited the user for the first time to create a personal plan. In Dravograd and Krško they were usually joined by Provision of long-term care an employee from the maintenance independence services unit, who also discussed the user’s needs and goals from their point of view. Later, on the first Long-term care services started to be visit at the beginning of the provision of services, implemented immediately after the personal the long-term care coordinator accompanied the plan was signed. Long-term care coordinators employee from the care unit, introduced them from all three environments said that they tried and gave the user a personal plan to sign. After to organise the provision of long-term care that, they visited the users in cases when the services in such a way that as few different people personal plan was revised, if there were problems as possible would come to the users’ homes. 32 Annex to the personal plan for the provision of long-term care as part of the pilot activities »Implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care« (hereinafter referred to as: annex), EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 90 Employees and deadlines for the provision of when the support service equipment was being services were changed mainly during annual installed. At the end, if necessary, they also leaves and sick leaves. In these cases, users were arranged for the equipment to be returned to the notified of the changes in advance. provider. In Dravograd, everything necessary The services provided were initially related to e-health was arranged by a coordinator recorded by the service providers manually in the of long-term care with working position placed in service log, and later via a mobile app and an NFC a hospital, while a doctor from the Slovenj Gradec tag. What some relatives missed was real-time General Hospital also informed the users about insight into the content of the services provided: the options for inclusion. “I would like, for example, when the girls come, In Celje, beneficiaries from the waiting list that they have a notebook like they have in care who expressed the wish to be included in support homes, where they would write down what they services were sent an application for inclusion. have done, because we are at work and when we The beneficiaries completed the application and come home, the parents are not really sure and sent it to the single entry point, which in turn sent they are not able to tell what has been done”. it to the service provider. From there onwards, As for the complete procedure, almost all the provider arranged everything directly with interviewed users and informal carers assess service users. that the procedure from the submission of the In Krško, 24-hour on-call duty service was application to the to the moment when the service provided for users who lived alone and did started to be provided was fast enough (“For me not have anyone to respond in the event of an it was fast, they came really fast to assess me, and emergency. It was carried out by an employee all, and then you get a decision and a provider from the independence maintenance unit, who came just like that.” and, what is more, most in the event of an emergency checked what users stated that they did not deal with excessive had happened and reacted accordingly. Due to paperwork as part of the procedure. The assessors difficulties in providing financial resources for the at the democratic forum agreed that the entire implementation of the 24-hour duty service, this procedure that they followed in the pilot projects practice was discontinued after some time. is suitable for transfer to other environments or for a systemic solution. Waiting list E-care and e-health How to act in the event that it is not possible to provide services to beneficiaries due to the Beneficiaries of long-term care were limited capacity of the long-term care provider also able to enjoy various forms of assistive was defined in the protocol, which envisaged technologies, which differed slightly between that the waiting list is kept by the long-term environments (more about this in Chapter care coordinator who, in addition to the name Perception of the use of assistive technologies). and surname of the beneficiary, also keep the They were primarily informed about these date of receipt of the assessment report for options by the assessors. the exercise of long-term care rights. Later, The Dravograd and Krško pilot environments the pilot environments agreed the following assisted those interested in all steps, from the with the contracting authority: “Beneficiaries submission of the application for inclusion to are included in the project in the order in which signing a contract with the provider. If necessary, their applications were submitted, and in the an employee from the pilot project was present event of a different choice of inclusion (e.g. 91 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE inclusion of a person who only needs the services needs assistance, and you have to tell them that of the independence maintenance unit which unfortunately we cannot help them at the moment are available), an official note is written.” 33 In as there is a waiting list”. Celje, where the waiting list was kept, the latter instruction was considered. Some employees also pointed to the importance of the degree of Complaint procedure urgency of the need for services when placing people on the waiting list: “The priority aspect has Complaints or objections expressed by service not been taken into account. Those users who do users are an important part of the provision of not receive home help and have no other organised services, with the right to object being provided assistance should be treated as a priority”. They to applicants, beneficiaries and users of the pilot argued that there were situations in which project. The course of the objection was briefly immediate assistance was unavoidable and that defined in Article 20 of the protocol, which was in such exceptional cases the criterion of urgency used in Dravograd and Krško: “In the event of should be taken into account. unprofessional conduct of formal care providers, Otherwise, the pilot environment in Celje the user is acquainted with the official complaint was the only pilot environment34 that kept a channels under the applicable law”. In Krško, added waiting list practically for the entire duration of to the basic protocol was a protocol for the case of the project. The waiting list in Celje started to fill a complaint against the assessment of eligibility up as early as February 2019 and was kept until for long-term care services, where the complaint the end of the pilot project. The inclusion of a user channel was described and the Appeal Note from the waiting list in the provision of services form was added. The option of complaint in the was possible only with the departure or cessation protocol that was prepared on the basis of the draft of the provision of care to another user. Many protocol of the contracting authority and which people on the waiting list were thus left without the evaluators received for inspection was not project services as the project concluded. In part, mentioned in Celje. the situation was resolved by providing only According to the information from the independence maintenance services, meaning environments, we find that the long-term care that the user received only those services that coordinator acquainted the users with the could be provided as part of the project, and not content of the protocol during their first visit, all the services they needed. while in none of the environments the users The need to be included in the project in said that they were specifically acquainted with Celje was therefore significantly greater than the option and method of complaint; in one of the pilot project was able to satisfy with its the environments they said the long-term care resources. In addition to the disadvantage for coordinator did not specifically explain to users beneficiaries, the waiting lists also caused distress the right to complain during their first visit. to employees, especially assessors, who reported According to our data, the pilot environments in employee reports that the waiting lists were did not present the option of complaint to users one of the negative aspects of their work: “The in any other way (for example: they did not work of an assessor in the field is interesting, but receive the full or at least the draft protocol, the at the same time difficult, especially when you are information about the option of complaint was making an assessment with a person who urgently not included in any other document, e.g. in the 33 Meeting on 17 September 2019. 34 In Krško, this was avoided with the employment of two additional healthcare technicians when there was an increase in the number of users. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 92 eligibility assessment or personal plan). Thus, we activities, research instruments and conversations did not receive information about what exactly, as with users or their relatives that the option of far as the complaint procedure is concerned, the complaint was not sufficiently explained to users long-term care coordinators told the beneficiaries and that, despite the pilot environments having and whether they presented both internal and received no formal complaints, they dealt with external complaint channels to them. quite a few complaints received in other ways Upon our inquiry, the long-term care (telephone, through service providers), on the coordinators said they did not report on basis of which they took action. However, to our complaints from applicants or users received knowledge, these have not been systematically in the formal way defined in the protocol or in recorded in the pilot environments. What is any other way. The contracting authority did not more, the complaint channel was not defined in warn us of any complaints received during the sufficient detail in the protocol to clearly present evaluation, either. to users all the steps of the complaint procedure. Nevertheless, in reviewing the data, we detected cases of complaints and, consequently, measures being taken. For example, it was Discussion with key reported from the Dravograd pilot environment messages that they had received a letter from a lady who did not agree with being included in a specific As part of the pilot activities, the pilot category of eligibility for long-term care. The projects tested the entire long-term care eligibility was re-assessed, with the same result. procedure - from the submission of an application According to the statement by the user (possibly for exercising the right to long-term care to the the same one) in the interview, it is evident use of long-term care services. The procedure that she complained twice and both times was tentatively determined by the contracting unsuccessfully: “It came in writing that I was not authority through a public tender, and the pilot eligible for the second category. It is important that environments supplemented and upgraded it I get at least something, even though I had thought during the pilot activities. Such a procedure, I was entitled to the second one. I was waiting for which has been tested on such a comprehensive their reply for one month. Then my daughter-in-law scale in Slovenia for the first time, also brings wrote to them again, but the request to put me in quite a few new features. the second category was rejected again. I told them An important new feature is the transfer of a that I could not believe it. At first I kept persisting, large part of activities from various organisations but then I left it alone.” We also noticed in the to the home of (potential) users. For example, interviews that users may not have been aware the assessment of eligibility for long-term care in that they have the option of complaint ( “No, no, the pilot projects was mostly carried out at the you have nowhere to complain to” ). applicant’s homes. By doing so, the professionals As we already mentioned in the section on came significantly closer to their living space, the assessment of eligibility for long-term care, which is something that had not thus far been the Celje pilot environment received complaints applied in the field of social and health care in from some ineligible users about the content Slovenia to such a large extent. The eligibility of the summary of living circumstances in the assessment at the applicant’s home allowed the eligibility assessment. assessor to observe how the applicant functions We did not systematically cover the channels in the environment where they felt the most for objection and the option of complaint in the comfortable. This is the practice that needs evaluation, while we noticed during various to be maintained and encouraged. The same 93 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE applies to personal planning, which also took We monitored how quickly the pilot place in the home environment as the centre environments managed to secure the first visit of the user’s daily life, and where professionals of the long-term care coordinator after the find it easiest to identify their needs together eligibility assessment was carried out as part with them and find answers to these needs. The of the second process indicator. The indicator evaluation showed that it is important to transfer criterion envisaged three working days for this as much of the procedure as possible to the home step, with the results showing that in this time environment, and also that the professionals are span the coordinators of long-term care in all flexible in all parts of the procedure and that pilot environments visited only slightly more than they adapt to the circumstances and needs of a tenth of beneficiaries, the most in Krško and (potential) users (e.g. flexibility and assistance the least in Celje. The reasons for this result are in completing and supplementing applications, attributed to various factors. The main one is that assessing eligibility at home, at a single entry there were several assessors, and they were able point, in an institution or elsewhere, and similar). to prepare several eligibility assessments at once, It was important for the evaluation to evaluate while there was one long-term care coordinator the time frame of the long-term care procedure, in and, consequently, they were not able to carry particular the two key steps (speed of the creation out such a large number of visits in a comparable of the eligibility assessment and the first visit of period of time. The second reason was related the long-term care coordinator); in doing so, we to the submission of applications “as a reserve”, relied on two process indicators. The practice which was a practice that was also detected in in Dravograd, where applications for eligibility the environments, so the visit of the long-term assessments were initially processed twice a month care coordinator also depended on the readiness by a team specially appointed for this purpose (it and will of the beneficiaries. Not insignificant is later held meetings if necessary), has shown that the fact that the information system, in which such an arrangement significantly prolongs this the data on visits was recorded as the basis for part of the procedure. Compared to Krško (79.9%) our analysis, was still under development and and Celje (58.5%), the lowest number of applicants was being upgraded during the project; in the was assessed within three days of reception of the part related to personal planning in particular application, which was the criterion of the first it was upgraded quite late, so the coordinators process indicator, in Dravograd (20.6%). Translating of long-term care were not able to record all the this practice to the system would not be effective, data in the information system on time, nor did as it would prevent applicants from acquiring they necessarily enter and correct all of them rights as soon as possible, while depriving retroactively. assessors of autonomy in processing applications. From the point of view of the procedure, The medium-sized proportion of assessments the results of the evaluation showed that it is performed within three days of reception of important for the time span of personal plan the application in Celje is most likely due to the preparation and related visits of the long-term waiting list, as it was not possible to immediately care coordinator that the coordinator is flexible include beneficiaries in services, so speeding up and adapted to the user’s more or less complex this part of the procedure was probably not such needs and specific situations. a priority. The experience of Krško is the one that Although the article does not pay particular is the most representative for planning the system attention to the structure and content of the procedure. This shows that assessors can make the forms used by employees in their work in the vast majority of assessments within three days of pilot projects, we must emphasise that it is reception of the application. important that these be made easier and not EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 94 more difficult for them to edit, monitor and store as otherwise users would be left to their own data; it is therefore necessary to computerise data devices and, in many cases, they would not be management, transparency and optimisation able to use these services. The contribution of of the quantity of data collected at the highest pilot projects in this regard was exceptionally possible level. One piece of information should important, and the practice should be singled out therefore be collected only once and in one in the Krško pilot environment in which the lack place. This can be ensured by having the of a family member of a user was compensated employees review the entire documentation for in a certain period of the implementation of on the applicant before the interview, or by pilot projects with a 24-hour duty service of an determining the central document in which the employee on the project. This was a practice that most information about the user is collected, also requires understanding and support at the e.g. the eligibility assessment or personal plan, systemic level. and which, with the user’s knowledge, can be We can also welcome the practice that in viewed by different employees. It is important emergency cases, the long-term care coordinators that users communicate each piece of information made sure that services started to be provided only once, and that forms are sufficiently immediately, even if the personal plan had not understandable and adaptable to people with yet been finalised. All the described examples different needs (e.g. Braille, easy-to-read format, show that flexibility as part of procedures and audio recording). in the provision of services is possible, and this We recorded the different ways in which can further prevent the distress of users and of documents created as part of the pilot projects their relatives as well as employees. Obtaining were handled. For example, in one pilot an opinion of the personal physician, which took environment, eligibility assessments were not place through the long-term care coordinator, also sent to those assessed as ineligible, and in all proved to be effective. environments, the notes on living circumstances Waiting lists, which of course are not were adjusted in accordance with the judgement a specific feature of the pilot project, but a of the assessors. This was not a good practice nor constant in various long-term care services, is it in accordance with the social model of the pose a general challenge for political policy view of the user, which should be the guidance in makers. It is necessary to anticipate any possible modern long-term care systems (Flaker, Nagode, pitfalls of these challenges (preparation of Rafaelič, & Udovič, 2011). Violations of the user’s appropriate staffing standards, detailed planning right to access information about themselves must of procedures, etc.) and to develop criteria for the be prevented and they must be acquainted with inclusion of beneficiaries from the waiting list in all steps as part of the long-term care procedure. the provision of services. The experience of pilot In order to avoid uncomfortable situations and projects has shown that it was more sensible to to adjust records on users, it would be good if put people on a waiting list based on the date assessors and other professionals who cooperate of receipt of the application and not on the date with the applicant check what information they of the assessment of the eligibility of long-term want to have included in the description of living care, while at the same time the results of the conditions and discuss what is important in the evaluation show that the principle of the need for context of long-term care to have written down services must also be reconsidered. and in what way. The complaint procedure is also an important As a good practice, we can certainly part of any procedure. As part of the evaluation, point out the great involvement of the pilot we have covered this part only to a lesser extent, environments in arranging support services, but we can nevertheless say that in the future it 95 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE will be necessary to define the user complaint As part of the implementation of the pilot procedure in more detail and transparently. projects, the principles that keep the user at The user must be acquainted with all internal the centre and encourage the defining of their and external complaint channels, provided with needs in cooperation with them, as they see them information and complaint forms, and empowered themselves, were thus used in the methods for to lodge a complaint and be provided support. It eligibility assessment and personal planning as transpired during the evaluation that the pilot well as from the procedural aspect. The extent environments did not detect or at least did not to which employees actually succeeded in this report on user complaints. This can be most likely as part of the pilot project is difficult to assess attributed to the fact that, in most cases, users were unambiguously; given the different data that we not sufficiently acquainted with the complaint obtained during the evaluation, we find that there channels. Complaints are not necessarily only an is still room for strengthening such an approach indication of poor performance of the long-term and placing the user at the centre of integrated care providers. Employees should understand long-term care. However, we must also take into complaints as user feedback on their work. account the fact that such changes, which require We highlighted the main deviations and a completely different view of one’s work and of recommendations that we observed in relation to the user, take more time and such effects would the course of procedures as part of the evaluation. probably be more visible if the pilot projects Despite all the above-mentioned, we recognise the lasted longer. entire procedure of the provision of long-term care Finally, we can conclude that procedures in as suitable for being translated into the systematic the future long-term care system, if carried out in implementation of long-term care, as it allowed the the same manner as in the pilot project and taking employees in the pilot projects, from first to last into account the mentioned restrictions and contact, to greatly support the user in all acts as part proposals, could be sufficiently fast and efficient. of the procedure, so that they felt safe and respected. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 96 BIBLIOGRAPHY AGE Platform Europe. (2012). Evropski okvir kakovosti storitev Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih dolgotrajne oskrbe Načela in smernice za dobro počutje in projektov, ki bodo podpirali prehod v izvajanje sistemskega dostojanstvo starejših, potrebnih oskrbe in podpore. Retrieved zakona o dolgotrajni oskrbi«. Retrieved from https:/ www. from https:/ www.age-platform.eu/sites/default/files/24171_WeDo_ uradni-list.si/glasilo-uradni-list-rs/vsebina/2018002400003/ brochure_A4_48p_SL_WEB.pdf javni-razpis-za-izbor-operacij-izvedba-pilotnih-projektov-ki-bodo- podpirali-prehod-v-izvajanje-sistemskega-zakona-o-dolgotrajni- European Network of National Human Rights Institutions. (2017). oskrbi-st--303-1201816-ob-198618 Applying a Human Rights-Based Approach to Long-term Care for Older Persons: A toolkit for Care Providers. Retrieved from Scirocco exchange, Maturity model (2021). Retrieved from https:/ http:/ ennhri.org/IMG/pdf/respect_my_rights_hrba_in_ltc_care_ www.sciroccoexchange.com/digital-infrastructure providers.pdf Seys, D., Panella, M., VanZelm, R., Sermus, W. ,Aeyels, D., European Parliament, European Council and European Bruyneel, L., Coeckelberghs, E., & Vanhaecht, K. (2019). Care Commission. (2010). Charter of fundamental rights of the pathways are complex interventions in complex systems: New Europwean Union. (2010)C 83/389. European Pathway Association framework. doi: https:/ doi. org/10.1177/2053434519839195 Flaker, V. (2017). Teze ob diskusiji o zakonu o dolgotrajni oskrbi in ob vzpostavljanju sistema dolgotrajne oskrbe. Kakovostna starost, revija za gerontologijo in medgeneracijsko sožitje, 20, 38-42. Flaker, V., Nagode, M., Rafaelič, A., & Udovič, N. (2011). Nastajanje dolgotrajne oskrbe: Ljudje in procesi – eksperiment in system. Ljubljana: Fakulteta za socialno delo. Lebar, L., Dremelj, P., Flaker, V., Rode, N., Mali, J., Peternelj, A., Smolej Jež, S., ... Kobal Tomc, B. (2017). Priprava podlag za izvedbo pilotnih projektov, ki bodo podpirali prehod v izvajanje sistemskega zakona o dolgotrajni oskrbiAktivnost 1: Priprava orodij za ugotavljanje potreb uporabnikov, metodika postopka ter ugotavljanje upravičenosti do storitev dolgotrajne oskrbe. Ljubljana: Inštitut RS za socialno varstvo. 97 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE ASSESSMENT OF ELIGIBILITY FOR LONG-TERM CARE Polona Dremelj Social Protection Institute of the Republic of Slovenia Boris Majcen Institute for Economic Research Mateja Nagode Social Protection Institute of the Republic of Slovenia Lea Lebar University of Ljubljana, Faculty of Social Sciences EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 98 ASSESSMENT OF ELIGIBILITY FOR LONG-TERM CARE KEY MESSAGES ▶ The assessment of eligibility for long-term care is a novelty in Slovenia and was tested for the first time within the framework of pilot projects. The experience and results of the pilot projects based on the assessment of eligibility are thus exceptionally important. ▶ Statistical analyses showed that the assessment scale was suitable and the statistical adequacy of proposed adjustments for scoring the modules was also approved. ▶ The findings that the structure of applicants as regards the category of eligibility was very similar irrespective of the type of the environment and that various profiles of assessors did not affect the classification of applicants in certain categories were also important. ▶ The assessors also confirmed that the assessment scale was suitable, as they recognised the concept of assessing people’s independence as an appropriate method for assessing applicant eligibility. ▶ The eligibility assessment must take place in the same manner irrespective of where it is being implemented. The assessor must consider the fact that a person lives alone and assesses on this basis how much assistance they require for implementing individual activities. In the event of a significant change in their living circumstances which may affect the amount of assistance the person requires, it can be proposed that the eligibility assessment is carried out again. ▶ The assessor’s educational profile may encompass healthcare or social care; it is also desirable that both of these profiles possess prior or additional knowledge of the other profile. It is advisable that teams at single entry points should be as heterogeneous as possible regarding the assessors’ education. ▶ When introducing new profiles, such as the assessor, continuous theoretical and practical training is important. ▶ The assessors working on pilot projects can form an important learning basis for further training in eligibility assessment. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 99 Introduction been increasingly recognising the importance of incorporating a broader selection of aspects One of the most important challenges in of a person’s life, especially the cognitive and the systemic arrangement of long-term care is psychological aspect, which is being incorporated, the introduction of a new or uniform practice for example, in Germany, France and Spain, but of establishing eligibility for long-term care, i.e. not yet in Great Britain and Austria (Ranci, Österle, a procedure with which it would be possible Arlotti, & Parma, 2019). to determine which social groups are eligible The selection of a tool for assessing eligibility for rights regarding long-term care and to for long-term care, which was used in pilot what extent. A lack of a uniform procedure for projects, took place in 2016 when Slovenia became determining eligibility in Slovenia is highlighted engaged in the preparation of a tool for eligibility in numerous policy documents at the EU level assessment for long-term care.35 Based on the (i.e. Rodrigues, 2014; Social Protection Committee examination of various tools and results of an and European Commission, 2014). Carrino and international workshop on needs assessment and Orso (2014) define the procedure of implementing due to its good testing results, comprehensiveness the eligibility assessment as a preparation of a and methodological justification, the working “vulnerability profile” which must be compared group selected the German tool, NBA. The choice to the “requirements for objective vulnerability” of tools was based on the criteria that the ideal tool determined by legislation in each country (Carrino should meet, i.e.: high-level of tool standardisation, & Orso, 2014). The result of preparing such a simple assessment procedure, which is not time- profile is the classification of applicants in groups consuming, suitability for use in the domestic of beneficiaries in which beneficiaries with a environment and in institutions, flexibility and smaller scope of needs are usually included in orientation towards an individual in the process lower categories and thus entitled to fewer services of personal planning, the observance of all aspects and benefits, while the beneficiaries with a larger of a person’s life and a focus on the person’s scope of needs are in higher categories. wishes and needs when planning care. The Various approaches and tools for determining proposal of the tool thus consisted of two sections: eligibility are used in the European area. The tools an assessment scale for eligibility assessment are roughly divided into two groups; the first group (hereinafter: assessment scale) and the proposal includes those intended only for the assessment of for implementing personal planning, which is a person’s functional abilities, which include tools analysed in more detail in Chapter Personal for the assessment of activities of daily living (ADL) planning and coordination in long-term care: and instrumental activities of daily living (IADL), identifying needs and planning care together with while the second group includes more complex the user. Members of the working group translated tools which enable a comprehensive needs and adjusted the tool in a way to suit the Slovenian assessment, i.e. assessment of functional abilities, context and verified it in the field from the cognitive and psychological aspect, and also the viewpoint of comprehensibility and applicability social aspect. The latter capture more aspects of a (Lebar et al., 2017). person’s life and are thus more extensive (Nagode, The assessment scale focuses on perosn’s Lebar, & Dremelj, 2018; Nagode, Lebar, & Kovač, activities in everyday life in which they may 2014). In recent years, European countries have need other people’s assistance. It comprises eight 35 Within the operation (project) of Preparation of bases for the implementation of pilot projects that will support the transition to the implementation of the systemic act on long-term care. The developer or beneficiary was the Social Protection Institute of the Republic of Slovenia (IRSSV), which set up a working group that included experts from all fields of long-term care in order to ensure an integrated approach to the preparation of the tool. When drafting their proposal, the working group also cooperated with expert Monika Gabanyi (Lebar et al., 2017). 100 Table 1: Variants of scoring adjustments in Slovenia (NBA-SLO (V1) and NBA-SLO (V2)) modules (fields of life, see Table 1) and each module consists of several items (questions). The assessor assesses the user’s degree of independence in everyday life. In doing so, it is presumed that the person wishes to implement 36 these activities. Eligibility (for services, benefits) FIELDS 1) is established on the basis of the assessment OF LIFE e) tion of a or O (V TION of O (V2): TION of results. For each field (module), points are added (MODULES) -SL -SL up in accordance with the prescribed calculation opor rules (Lebar et al., 2017; Wingenfeld, Büscher, & NBA ORIGINAL (pr total sc NBA REDUC Module 4 NBA REDUC Module 5 Gansweid, 2008). The assessment scale changes the paradigm Module 1 10% 10% 10% in Slovenia, as it diverts from the “dependency Mobility profile” (Carrino & Orso, 2014) and establishes the paradigm of “self-dependency” as the basis for Module 2 + 3 evaluating eligibility for long-term care services Cognitive and and benefits. The need for long-term care is thus communication 15% 15% 15% abilities + defined on the basis of providing assistance to Behaviour and ensure independence and the maximum utilisation mental health of the person’s abilities. As part of the pre-pilot project, the Module 4 40% 35% 40% participants proposed that, in addition to the Self-care original, two additional variants of scoring (NBA- SLO) would be tested in the pilot projects. The Module 5 original scale for eligibility assessment (Buescher, Ability to deal Wingenfeld, & Schaeffer, 2011; Wingenfeld et with illness-/ 20% 20% 15% therapy-related al., 2008) does not include modules 7 and 8 in demands and the scoring, but because Slovenia has a strong burden tradition of social care and an already existing Module 6 + 7 infrastructure or developed network of home help Managing everyday life and 15% providers and as the activities of these modules 10% 10% social contacts + (only M6) maintain a person’s higher quality of life, it is Activities outside the house sensible that its significance is highlighted by adjusting the assessment scale, i.e. by including modules 7 and 8 in the scoring (Lebar et al., 2017). Module 8 Household 0% 10% 10% The proposed method of scoring anticipates maintenance that activities outside the house (module 7) are combined with managing everyday life and social contacts (module 6), in a similar manner as the modules cognitive and communication abilities (module 2) and behaviour and mental health (module 3) are combined in the German system (NBA ORIGINAL). Accordingly, the combination of modules 6 and 7 makes up 10% of the total 36 Buescher, Wingenfeld, & Schaeffer (2011), Wingenfeld et al. (2008). score and module 8 an additional 10%. Due to the 101 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE resulting five per cent surplus, one of the strong As part of determining the suitability of the modules is reduced, i.e. module 4 (M4): Self-care assessment scale, the following indicators were – (NBA-SLO (V1)) or module 5 (M5): Ability to deal further assessed: with illness-/therapy-related demands and burden ▷ 70% of the existing beneficiaries in pilot – (NBA-SLO (V2)) (Lebar et al., 2017). environments are assessed by means of the Based on the drafted eligibility assessment, assessment scale in the first nine months; the applicant is classified in one of the five ▷ at least 80% of all LTC beneficiaries are assessed categories of eligibility for long-term care as per again after six months if they are still part of the the score they receive.37 An adult eligible for pilot activities; long-term care must receive at least 12.5 weighted ▷ proportion of users who were grouped into points in the eligibility assessment procedure and a different eligibility category after the second must have been dependant on a third person’s assessment; assistance for at least three months or permanently ▷ proportion of users who transferred to when performing the activities of daily living and institutional care; instrumental activities of daily living over a longer ▷ proportion of persons who fail to meet the period of time. Classification in category “0” means eligibility threshold but are informed about the that the applicant failed to exceed the set threshold existing rights or care options regarding social – their established degree of independence and and health care and about participation in the thus their need for assistance is not sufficient to be evaluation procedure. eligible for long-term care. Data from the information system of pilot As part of the pilot projects in the field of environments was used to determine the suitability long-term care, the eligibility assessment was of the assessment scale, which mostly referred implemented by specifically trained assessors. The to the data from the application, the eligibility latter thus form a foundation for creating a new assessment and the recording of certain HR data. professional profile in Slovenia. A special section of the questionnaire for employees (under points M0 and M18) was intended for assessors, in which we asked about the course Methodology of the assessment of applicants and the suitability of the assessment scale and the instructions for One of the objectives of the pilot project assessment in order to obtain a subjective opinion evaluation regarding long-term care was to assess about the suitability of the eligibility assessment. We the suitability of the selected assessment scale. also used certain findings or results of a deliberative The latter was assessed from the viewpoint of discussion at the democratic forum. See more time needed for assessment (time consumption on individual measuring instruments in Chapter of assessment), the method of scoring individual Evaluation of pilot projects and methodology. modules of the assessment scale (proposal of When analysing quantitative data, univariate, adjustment for Slovenia) and the suitability of bivariate and multivariate statistical methods were classifying applicants in eligibility categories, used. The ordered logit model was used among assessors’ subjective assessments regarding the the latter. This is also known as an ordered logistic suitability of the assessment scale, the objectivity regression or proportional odds model, which of assessment and assessors’ qualifications for is a regression model for the ordinal dependent assessing. variable.38 37 In the latest proposal of the act on long-term care (2021), the eligibility categories are defined on the basis of the applicant’s degree of independence and their abilities, i.e. category 1 denotes minor limitation of independence and abilities, category 2 moderate limitation, category 3 severe limitation, category 4 more severe limitation and category 5 the most severe limitation of independence and abilities. 38 McCullagh (1980). EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 102 Results latter were thus not incorporated in the service implementation. The results referring to the eligibility Following the first assessment at home, 378 assessment for long-term care (hereinafter: the of the assessed applicants were eligible in Celje. eligibility assessment) are displayed in three The second assessment was carried out for 192 sets. Initially, we highlight the characteristics beneficiaries, which is 51.8% of all beneficiaries of eligibility assessment from different aspects. after the first assessment.39 The data on the Then, we present the experience of assessors with date of the first assessment is available for 370 the eligibility assessment, and we complete the beneficiaries and the date of the second assessment chapter with an assessment of the suitability of for 148 beneficiaries. Data on both dates is classification into eligibility categories. available for a total of 148 beneficiaries, which is 39.2% of those eligible at the first assessment. Among these, only 29.7% received the second Characteristics of eligibility assessment within five to seven months, while the assessment remaining ones received the assessment before (35.8%) or later (34.5%). The objective of at least In all three pilot environments, a total of 2,031 80% of all beneficiaries of long-term care being applications were recorded in the information assessed again in six months was achieved in eight system. Some 1,972 persons were assessed by means months in the pilot environment among those of the assessment scale, i.e. 885 in Celje, 475 in Krško assessed twice. and 612 in Dravograd. The discrepancies between Following the first assessment at home, 289 the number of applications and the number of of the assessed applicants were eligible in Krško. assessments occurred for various reasons, e.g. The second assessment was carried out for 128 applicant’s death, inclusion in a service within the beneficiaries, which is 44.3% of all beneficiaries existing legislation, rights and other reasons (e.g. the after the first assessment.40 The number of units applicant changed their mind after completing the of analysis regarding date equals the number application). Applications for eligibility assessment of assessment units. Among 128 of the assessed were completed by 834 persons from institutional at both time points, the second assessment was care (41.1%) and 1,197 persons who lived at home made within five to seven months for 61.7% of the (58.9%). Of all the applicants living at home, almost assessed ones. Prior to five months, the assessment everyone was assessed (95.8%). was made for 27.3% of beneficiaries and for 10.9% The proportion of beneficiaries among all of beneficiaries the assessment was repeated after assessed applicants amounted to 81.1% in all pilot seven months. The objective of at least 80% of all environments. Although it was somewhat lower in beneficiaries of long-term care being assessed again Dravograd (76.1%) and somewhat higher in Celje in six months was achieved in seven months in the (84.0%), the differences between environments are pilot environment among those assessed twice. not very great. Following the first assessment at home, 230 of The eligibility threshold was not attained the assessed applicants were eligible in Dravograd. by 18.9% of persons; among those living at The second assessment was carried out for 96 home, this proportion amounted to 21.8%. The beneficiaries, which is 41.7% of all beneficiaries 39 For 70 users lacking the second assessment, there is information of an early termination of participation in the project, i.e. more than half of them died (51.4%) and one third (32.9%) were admitted to institutional care. The remaining ones no longer wanted to receive services (4.3%), obtained personal assistants (2.9%) or left for other reasons (8.5%). It is impossible to determine reasons for the lack of the second assessment for 116 users. 40 For 142 users who lack the second assessment, we have information on early termination of cooperation in the project, i.e. less than one third died (29.6%), one quarter (25.4%) no longer wanted to receive services, some 14.8% were admitted to institutional care, 7.7% no longer required relevant care, 4.2% received a personal assistant, and other reasons were provided for the remaining ones. For 19 users, the reason for the missing values is unknown. 103 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 2: All applicants, assessed, beneficiaries of long-term care as per the first assessment, total and by individual environments – number and proportion Total Celje Krško Dravograd Applicants (N) 2031 899 501 631 Applicants in institutional care (N) 834 452 114 268 Applicants living at home (N) 1197 447 387 363 Assessed (N) 1972 885 475 612 Assessed living at home (N) 1147 434 361 352 Beneficiaries among the assessed (N) 1599 743 390 466 Beneficiaries among the assessed living at home (N) 897 378 289 230 Applicants from institutional care (%) 41.1 50.3 22.8 42.5 Applicants from home environment (%) 58.9 49.7 77.2 57.5 Assessed (in %) 97.1 98.4 94.8 97.0 Assessed from home environment (%) 95.8 97.1 93.3 97.0 Beneficiaries among the assessed (in %) 81.1 84.0 82.1 76.1 Beneficiaries among the assessed in home environment (%) 78.2 87.1 80.1 65.3 Ineligible among the assessed (in %) 18.9 16.0 17.9 23.9 Ineligible among the assessed in home environment (in %) 21.8 12.9 19.9 34.7 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 104 after the first assessment.41 The number of units environment (little over an hour) and in the of analysis regarding date equals the number of institution (one hour and more than two minutes); assessment units. Among 96 assessed at both time the median value was lower for the assessment in points, 42.7% received the second assessment the home environment. within five to seven months. The assessment was Indicative total time used for preparing the done for 20.8% of beneficiaries before five months eligibility assessment amounted to five hours and and after seven months for 36.5% of beneficiaries. included transport, visit to a home and interview The objective of at least 80% of all beneficiaries with the applicant, consultation with other of long-term care being assessed again in six assessors and draft of the assessment, preparation months was achieved in eight months in the pilot of the report, completion of the assessment scale environment among those assessed twice. and entry of data in the table template for the The average time of evaluation during the calculation of the eligibility category. The drafting first assessment in all pilot environments totalled of the assessment is extended when assessors 67.9 minutes or approximately one hour and eight are still in training (time shortens with gained minutes and it was shortened to a little over an experience), when arranging documentation, hour during the second and third assessments. when only one person is assessing, when the During the first and second assessments, half of applicant experiences mental health problems and these were performed in less than one hour and when assessing an applicant who lives at home half of them in more than one hour; the shortest (assessments in an institution are usually shorter). time of assessment was 20 minutes and the longest 480 minutes (8 hours). The latter refers to the assessment in two cases and it most likely includes Assessors’ experience with the entire duration of the assessment procedure eligibility assessing and not just the assessment. In one case, the assessment lasted 240 minutes (4 hours) and 210 Opinions about the suitability of the minutes (3 hours and a half) in three cases, which assessment scale for assessing eligibility were was the longest time of assessment. The latter took collected from the assessors by means of a survey place in the applicant’s home environment. questionnaire. The assessor in all pilot environments Similar time frames were also recorded agreed for the most part (average assessment in regarding second assessments; the average, M0 was 3.75 and 3.81 in M18)42 with the concept minimum and maximum time of assessment of assessing a person’s independence as being a were somewhat reduced upon each subsequent suitable method for assessing eligibility. In both assessment. survey points, overall, the assessors also indicated The assessment of applicants in their home their agreement with the fact that the assessment environment was on average (74.7 minutes or scale encompasses all fields relevant for the one hour and some 15 minutes) longer than the eligibility assessment. A similar trend can be seen in assessment of users in institutional care (less than the statements that the assessment scale enables the one hour). The median value was also higher provision of a realistic assessment (regarding this when the assessment took place in the home statement, most assessors at both time points agreed: environment (70 minutes) than in the institutional AS = 4) and that the modules of the assessment scale care (60 minutes). The average time of the second are appropriately weighted (average value in the assessment reduced in the applicant’s home M0 point is somewhat lower than in M12, but the 41 For 67 users who lacked the second assessment, we have information on early termination of cooperation in the project, i.e. more than one third died (37.3 %), one quarter (25.4%) no longer wanted to receive services, less than one quarter (23.9%) were admitted to institutional care, and other reasons were provided for the remaining ones. For 67 users, the reason for the missing values is unknown. 42 On the scale from 1 (I do not agree at all) to 5 (I completely agree). 105 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE median value and the mode value increased at the As per their educational profile, the assessors second assessment, i.e. from 3 to 3.5 and from 3 to in the pilot project were either registered 4, respectively. nurses, registered physiotherapists, occupational Certain assessors claimed problems when therapists or social workers. Which profile is assessing persons with disabilities or health issues, suitable for an assessor was also one of the especially when dealing with persons with sensory important discussion topics at the democratic disabilities and dementia patients. Irrespective of the forum at which the participants agreed that all above, they assessed that they were relatively well of the educational profiles stated above can be trained to assess persons with disabilities or health independent assessors, but it is vital that the issues. During the first survey, they felt competent team of assessors at the single entry point is to assess persons with sensory disabilities to a heterogeneous. Previous or additional healthcare lesser extent, while the average self-assessment of education for profiles coming from social care and competence for assessing this target group increased prior or additional social education for profiles in or improved (from 2.9 to 3.4) during the second healthcare are advised. survey. 43 The difference is statistically significant (t = 1.743, p = 0.093). The increase in self-assessment of competence at the second survey is also revealed Suitability of classification in in the increase of minimum assessment provided eligibility categories by the assessors , i.e. from 1 to 3, which means that no assessor felt incompetent to assess persons Regarding the results of eligibility assessment with sensory disabilities after a year and a half of according to three scoring variants, we found that implementing pilot projects. some 76% of assessed persons would be eligible The majority of assessors stated upon the for long-term care as per the method of scoring first survey that it would be better if an individual adopted from the German assessment model applicant was assessed by two assessors (in a pair). (NBA-original), which does not include modules At the democratic forum, the assessors also agreed 7 and 8 in the scoring. According to both scoring that assessment in a pair has more advantages. variants proposed for Slovenia, some 79% of These are also shown in the assessor’s safety persons (reduced M4) or a little less than 79% of (in cases of violence in the applicant’s family, persons (reduced M5) would be eligible for long- emotional blackmail, cognitive impairments and term care. The results are also similar at the level difficult relatives) and also in the technical sense of the individual pilot environment. of assessment implementation (prompt taking We also observed changes in the categories of notes, interview with the applicant, separate of eligibility assessed during the implementation interview with relatives, etc.). As necessary of the pilot project. The objective was to determine training which they should receive, the assessors what changes occurred in the category of state: training in the field of healthcare and eligibility during the second assessment. For social care, workshops and training regarding 61.9% of beneficiaries, the category of eligibility communication, training on dementia, mental did not change during the second assessment; disabilities, addiction, use of assistive devices, etc. the situation worsened for more than one fifth Furthermore, they highlighted the need for the (21.2%) and improved for 16.8% of beneficiaries. implementation of joint assessments with expert If comparing the category of eligibility of the first assessors (at least ten joint assessments), regular assessment with the latest one, similar conclusions evaluation and supervisory meetings and the can be drawn: the category has not changed for option of peer and additional expert consulting. 58.1% of users, 24.1% were classified in a higher 43 On the scale from 1 (I do not agree at all) to 5 (I completely agree). 106 Table 3: Results of assessing applicants by three scoring variants (total of all pilot environments) category and 17.7% in a lower category, i.e. their condition improved. Differences in the changes of categories of eligibility were seen among the pilot environments. The category of eligibility did not change from the first to the latest assessment for 64% of users in NBA – original Celje, 49% of users in Krško and 58% of users in Dravograd. The largest proportion of transitions to CATEGORY OF ELIGIBILITY a higher category was recorded in Dravograd (32%) and the largest proportions of transitions to a lower 0 1 2 3 4 5 category in Krško (31%). With regard to transitions, N 445 328 356 353 258 152 the data on the users’ transition to institutional care % 23.52 17.34 18.82 18.66 13.64 8.03 and subsequent termination of pilot activities is also important. A total of 11.1% of such transitions was TOTAL NBA – original: 1892 recorded: 12.1% in the Celje pilot environment, 10% in Krško and 10.8% in Dravograd. Below, we analyse the suitability of applicants’ classification in certain categories Reduced M4 of eligibility and also which factors impact the classification. The dependent variable, “category of CATEGORY OF ELIGIBILITY eligibility”, is the ordinal variable. Classification of 0 1 2 3 4 5 applicants in categories 0, 1, 2, 3, 4 and 5 represents N 384 307 362 388 298 153 the assessment of preserved abilities to carry % 20.30 16.23 19.13 20.51 15.75 8.09 out the activities of daily living and instrumental activities of daily living, and healthcare services in TOTAL Reduced M4: 1892 long-term care. The ordered logit model was used for the analysis. We first present basic information about the dependent and explanatory variables. In the continuation, our objective was to determine the net effect of an individual explanatory variable Reduced M5 on the dependent one. Table 6 displays results of the ordered logit CATEGORY OF ELIGIBILITY model, whereby it is possible to interpret the 0 1 2 3 4 5 direction and characteristic, but not the size of the N 399 297 358 364 314 160 coefficients obtained for each explanatory variable. % 21.09 15.70 18.92 19.24 16.60 8.46 The number of observations used in the model totalled 1,385. The model chi-square distribution TOTAL Reduced M5: 1892 with 24 degrees of freedom amounts to 371.29 and is highly significant, which means that the used explanatory variables significantly affect the classification in the categories of eligibility for long-term care. A negative, but only lowly significant coefficient (P>|z|=0.095) for the “gender” variable means that if a person is a woman, the probability of classification 107 Table 4: Changes in the category of eligibility for users in the community in a higher eligibility category almost does not change on average with regard to a person who is a man. It may be determined for the “age” variable that with the increase of age the probability of being classified in a higher eligibility category also increases on average (positive and highly significant coefficient, P>|z|=0.000). The result obtained for WORSE SITUATION the “education” variable was also expected as the First assessment – First assessment – transfer from lower to higher education decreases second assessment latest assessment on average the probability of being placed in the N % N % higher eligibility category (negative and significant coefficients, P>|z|=0.023 in P>|z|=0.000, for the +3 4 1.0 4 1.0 second or third level of education). +2 10 2.4 20 4.8 An interesting result can be seen for the +1 74 17.8 76 18.3 “marital status” variable in which we determined whether the probability of being classified in a higher eligibility category changes significantly for persons who are widowed, single or separated UNCHANGED SITUATION in comparison to persons who are married. The First assessment – First assessment – results revealed that the coefficients obtained second assessment latest assessment for all three marital statuses were significantly N % N % negative, i.e. if a person is not married and has a different status, the probability of their 0 257 61.9 241 58.1 classification in a higher category is on average reduced (negative and highly significant coefficients, P>|z|=0.000, P>|z|=0.003 in P>|z|=0.000). BETTER SITUATION The expected result of the “assistance and First assessment – First assessment – attendance allowance” variable (persons receiving second assessment latest assessment this allowance have serious problems with N % N % carrying out basic activities of daily living and require suitable care) is that these persons would -1 59 14.2 62 14.9 be classified in higher eligibility categories. The -2 4 1.0 5 1.2 results and the positive and significant coefficient -3 6 1.4 6 1.4 confirmed this (P>|z|=0.000). -4 1 0.2 1 0.2 We further wanted to know if the probability of being placed in eligibility categories also differs between pilot environments as they have different TOTAL content and represent urban, semi-rural and rural environments (pilot environment variable). First assessment – First assessment – The Celje urban pilot environment served as the second assessment latest assessment basis for comparison. The results showed that the N % N % probability of being classified in a higher category of eligibility for people living in a semi-rural or 415 100.0 415 100.0 rural environment does not on average differ EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 108 Table 5: Explanatory variables and their effect on the dependent variable Name Effect on the dependent variable Value domain of variable Gender Does gender affect the probability of classification in a higher category of eligibility? 0 = man, 1 = woman Does an increase in age affect the probability 0 = primary school or secondary Age of classification in a higher category of vocational education eligibility? 1 = secondary school education 2 = higher or more 0 = married or cohabitation Marital Does a marital status different to the "married" 1 = widowed status status affect the probability of classification in a higher category of eligibility? 2 = single 3 = separated Assis- tance and Does the receipt of assistance and attendance 0 = does not receive assistance and atten-allowance affect the probability of classification attendance allowance dance in a higher category of eligibility? 1 = receives assistance and attendance allowance allowance Does another pilot environment, in Pilot envi- comparison to the Celje pilot environment, 0 = Celje pilot environment ronment affect the probability of classification in a 1 = Krško pilot environment higher category of eligibility? 3 = Dravograd pilot environment 0 = Old age 1 = Disease Do other reasons in comparison to old age 2 = Disease, old age NEED affect the probability of classification in a 3 = Dementia and disease or old age higher category of eligibility? 4 = Mental disabilities 5 = Injury 6 = Other reasons, usually a combination of several reasons Does classification in the group of applicants TYPE in institutional care in comparison to the group 0 = Others of other applicants affect the probability of 1 = In institutional care classification in a higher category of eligibility? Does a different profile of the assessor in 0 = Social worker 1 = Occupational therapist PROFILE-2 comparison to the “social worker” profile affect the probability of classification in a higher 2 = Physiotherapist category of eligibility? 3 = Nurse 4 = Two assessors 0 = period of assessment until the end Does a period of preparing the eligibility of April 2019 Experi- assessment in comparison to the starting 1 = period of assessment between 1 May ence period affect the probability of classification in 2019 and 28 February 2020 a higher category of eligibility? 2 = period of assessment starting on 1 March 2020 Does the extension of time (measured in minutes) used for the preparation of the OC1 eligibility assessment by means of the assessment scale affect the probability of classification in a higher category of eligibility? EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 109 significantly from the probability of classification Various profiles of assessors participated in an urban environment (highly insignificant in the implementation of the assessment of coefficients in both pilot environments, eligibility for long-term care. For the most part, the P>|z|=0.544, P>|z|=0.453). assessment was carried out by one assessor and The structure of applicants differed as per two assessors performed the assessment in certain the individual pilot environment; the applicants cases (the “profile_2” variable). For the purpose of had different characteristics as per the services ensuring a suitable quality of assessment and the and benefits which they had already received establishment of possible differences in assessing before completing the application. All applicants as per the preliminary education of assessors, it were thus divided into two groups; a group which was necessary to determine whether the different consisted of applicants in institutional care and a profiles of assessors within the pilot projects made group which consisted of the remaining applicants for variations in assessments regarding individual (the “type” variable). We wanted to know applicants and whether the assessments thus made whether there were significant differences in the were also the result of the assesor’s profile. The probability of being classified in higher categories assessments were made by four different profiles of eligibility for applicants in institutional care or two assessors of identical or different profiles. if compared to the group with the remaining The “social worker” profile was used as the control applicants. Highly significant and positive profile, to which a possible significant change coefficients (P>|z|=0.000) confirmed the expected in probability due to various other profiles was result that the probability of being placed in compared. higher eligibility categories increased on average Average values of the assessed eligibility for people in institutional care. Similar results categories indicated lower assessments in were revealed when classifying applicants into the assessor’s “nurse” profile, which was also four groups (0: does not receive any form of long- confirmed with the application of the ordered logit term care; 1: receives only informal assistance; 2: model and the observance of the relevant variable receives assistance and attendance allowance and (significant and negative coefficient for the “nurse” services at home; 3: is in institutional care). Group profile, (P>|z|=0.004) and highly insignificant 0 was a control group and the coefficients obtained coefficients for other profiles), which could have for the remaining three groups were significant been a result of the characteristics of applicants and positive, i.e. the probability of classification in assessed by a nurse (they could have been younger, a higher eligibility category was thus higher in all women, with lower education, etc.; see the results three groups if compared to the control group. of the model below). By incorporating this variable We also wished to know whether the in a broader model and thus controlling numerous reason written in the filed application affects other characteristics, we tried to obtain the actual, the probability of classification in a higher net effect of this variable. The results thus obtained category of eligibility (the “need” variable). The reveal that the probability of classification in a enumerated reasons were divided into seven higher category of eligibility for all other profiles, groups and the “old age” reason was set as the including the “nurse” profile in comparison to control group. Do other reasons, which reflect the “social worker” profile was not significantly more concrete problems (disease, injury, dementia different (highly insignificant coefficients for all or a combination of several problems), on average profiles). affect the probability of being classified in a In the initial phase of project implementation higher eligibility category more than old age? The and eligibility assessment, it could have been expected positive response was confirmed in all expected that the assessors were still adjusting to groups by means of calculations. the assessment scale and perhaps the assessments EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 110 Table 6: Results of the ordered logit model 111 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 7: Ordered logit model – results for the “profile_2” variable EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 112 from this period on average varied from the attainment was established on the basis of data assessments in the continuation. Furthermore, collected in the information system. the first wave of the Covid-19 epidemic was The indicator, which stipulated that at least experienced in 2020, which could have also 70% of the existing users of long-term care services affected the assessing itself. Therefore, the above the age of 18 must be included by the duration of the entire period of assessment was pilot environment in the assessment procedure, divided into three sub-periods (the “experience” was somewhat changed during the evaluation. variable): a) from the project start to the end Because no prescribed method for preparing the of April 2019; b) from May 2019 to the end of assessment of the existing population of long-term February 2020, and c) from March 2020 onwards. care users was in place for the pilot environments The results obtained show no statistically (as Slovenia has no validly applicable definition significant differences in the probability of of long-term care, the environments could have classification in a higher category of eligibility approached the assessment in various ways), we for the group of assessments implemented in pursued the realisation or the attainment of the another period. However, the highly significant criterion as per the call for project evaluation and positive coefficient for the third period (period within the implementation of pilot projects, i.e. after the outbreak of the epidemic) means that the at least 600 people assessed in the Celje pilot probability of being placed in higher eligibility environment and 300 people assessed in the Krško categories was on average increased for people and Dravograd pilot environments each). being assessed in this period (P>|z|=0.002). The results showed that the target indicator Finally, we tested the possible effect of the was attained in all pilot environments and length of the eligibility assessment, which proved also exceeded (before schedule), i.e. some 885 to be insignificant (P>|z|=0.186). More time spent applicants were assessed in the Celje pilot on the assessment apparently has no effect on environment by the end of the project and 612 changing the probability of classification in higher in Dravograd and 475 in Krško. We find that the categories of eligibility. target values of the indicator were set too low, which had already been established at the start of implementing pilot projects. This was revealed by Discussion with key the estimate of the number of potential long-term messages care users prepared by the pilot environments despite insufficient existing data in this field and The assessment of eligibility for long-term also the assessment drafted by the Social Protection care is a method which was tested for the first time Institute of the Republic of Slovenia based on the in the field of long-term care within pilot projects collected and calculated data. We assessed that in Slovenia. This was thus a novelty, which is why some 2,145 users of long-term care (of services the experience and results of the pilot projects for and cash benefits) were found in the Celje pilot the evaluation of suitability of the assessment scale environment in 2016, 1,050 in the Dravograd for assessing eligibility for long-term care are of pilot environment and 841 in the Krško pilot exceptional importance. environment.44 In their bids to the public call, the To monitor the assessment of eligibility, pilot environments themselves stated that there we formed several indicators (provided in the were many more long-term care users in their introduction of this contribution), and their environments than anticipated by the criterion 44 To assess the number of long-term care users in pilot environments, the methodology prepared by the working group at the Statistical Office of the Republic of Slovenia in 2014 was used (Nagode et al., 2014). We proceeded on the basis of the data at the national level for 2016 (latest published data), which is collected and published annually by the Statistical Office of the Republic of Slovenia. 113 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE set in the public call for individual types of The assessors also think that the scale is environments: “The anticipated number of existing suitable for assessing eligibility for long-term care and potential users of the project concerned and consider the concept of assessing people’s considerably exceeds the minimum indicators independence, on which the assessment scale is set...” (Celje Health Care Centre, 2018); “We based, as a suitable method of evaluation. When understand that we by far exceed the condition of assessing eligibility, the assessor must assess the providing 70%...” (Residential Home for the Elderly person with regard to their current condition, Koroška in Dravograd, 2018). whereby they do not observe the broader context The evaluation results support the proposal of receiving assistance (e.g. assistance by informal for the adjustment of the assessment scale from the carers in their home environment and assistance pre-pilot project (Lebar et al., 2017) to include in by formal carers in an institution). They must the scoring all eight modules and not just the first consider the fact that a person lives alone and six as was seen in the German model (Wingenfeld assess on this basis how much assistance they et al., 2008). It was specifically revealed that the require for implementing individual activities. adjustment of scoring did not have a significant The starting point for assessment is the person’s impact on the proportion of people eligible for needs, so if they do not wish to carry out a certain long-term care as only 76% of applicants were activity this is noted in the assessment. In the entitled to long-term care if only the first six event of a significant change in the user’s living modules were observed, and 79% if we observed circumstances which may affect the amount all modules. As stated initially, the activities of assistance the user needs, it is necessary to defined in modules 7 and 8 maintain a higher implement the eligibility assessment again. quality of life. The assessors perform the eligibility From the viewpoint of time consumption assessment on their own, which is why we were when completing the assessment scale, it was unable to determine the level of compliance of determined that the scale was suitable for several assessments (compliance of scoring and assessing eligibility for long-term care. The results classification in the eligibility category) and thus revealed that the average time of assessment at the the objectivity of the assessment scale from the applicant’s home amounts to little over one hour viewpoint of a multiple assessment of the same and does thus not present a great time load for the user. assessor or the applicant. Improved objectivity of the assessment scale The results obtained through the ordered was only one of the reasons why the assessors probit model confirm the expected effects proposed that the eligibility assessment should of individual explanatory variables and be implemented by two assessors. The advantage simultaneously reveal the suitability of classifying of two assessors lies in the technical aspect of applicants in eligibility categories. We can thus the assessment implementation (decreased use conclude that the assessment scale is suitable of service vehicles, less time required to draft for assessing eligibility for long-term care, and the assessment), improved professionalism different profiles of assessors do not make a when assessing (if assessors represent different difference when classifying applicants in eligibility expert profiles, there is no need for additional categories. The results further show that the consultation with colleagues at the single entry probability of being classified in a higher category point) and improved safety of assessors. They of eligibility in semi-rural and rural environments further claimed that when assessing on their own, does not on average differ significantly from they seldom use a computer for prompt taking the probability of classification in an urban of notes or even entering of assessments. When environment. assessing they have to focus on the applicant EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 114 (“aspect of humanity”) and avoid practices which dementia patients and persons with mental would lead them to become “insurance agents”, disorders. When introducing new profiles such as one assessor expressed themselves vividly at as the assessor, continuous theoretical and the democratic forum. If the assessors implement practical training is important in addition to the eligibility assessment in a pair, they could initial training. Joint assessments with expert produce two to a maximum of three assessments assessors proved to be exceptionally useful a day, while one assessor completes one to two among practical training courses, and so were eligibility assessments in a day and visits one the peer and inter-professional consultations. to two applicants at their homes. The proposed Together with expert assessors, the assessors from standard is the result of a deliberative discussion at the pilot environments represent an important the democratic forum. professional group which obtained valuable In addition to the method, the eligibility practical experience during the project through assessment also resulted in another novelty, i.e. this new method being introduced in Slovenia, and the proposal of a new professional profile in thus present an indispensable learning base for Slovenia – the assessor. The assessors trained further steps in the systematic introduction of the for their work during the pilot project. They eligibility assessment in the Slovenian long-term acquired good assessment skills. Problems arose care system. when assessing persons with sensory disabilities, 115 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE BIBLIOGRAPHY Büscher, A., Wingenfeld, K., & Schaeffer, D. (2011) Determining Nagode, M., Zver, E., Marn, S., Jacović, A., & Dominkuš, D. (2014). eligibility for long-term care – lessons from Germany. Dolgotrajna oskrba – uporaba mednarodne definicije v Sloveniji. International Journal of Integrated Care. 11(2). Urad RS za makroekonomske analize in razvoj. Retrieved from https:/ www.umar.gov.si/fileadmin/user_upload/publikacije/ Carrino, L., & Orso, C. E. (2014) Eligibility and Inclusiveness of Long- dz/2014/DZ_02_14.pdf term Care Institutional Frameworks in Europe: a cross-country comparison. Ca' Foscari University of Venice, Dept. of Economics Ranci, C., Österle, A., Arlotti, M., & Parma, A. (2019) ‘Coverage versus Research Paper Series No. 28/WP/2014. generosity: Comparing eligibility and need assessment in six cash-for care programmes’, Social Policy & Administration, vol. Lebar, L., Dremelj, P., Flaker, V., Rode, N., Mali, J., Peternelj, A., 53, pp. 551–566. Smolej Jež, S., ... Kobal Tomc, B. (2017). Priprava podlag za izvedbo pilotnih projektov, ki bodo podpirali prehod v izvajanje Rodrigues, R. (2014). Long-term care: the problem of sustainable sistemskega zakona o dolgotrajni oskrbiAktivnost 1: Priprava financing, Discussion paper, Peer Review on financing of long- orodij za ugotavljanje potreb uporabnikov, metodika postopka term care, Slovenia. ter ugotavljanje upravičenosti do storitev dolgotrajne oskrbe. Ljubljana: Inštitut RS za socialno varstvo. Social Protection Committee & the European Commission. (2014). Adequate social protection against long-term care needs in an McCullagh, P. (1980). Regression Models for Ordinal Data. Journal ageing society. of the Royal Statistical Society. Series B (Methodological). 42 (2): 109–142. Wingenfeld K., Büscher A., & Gansweid B. (2008). Das neue Begutachtungsassessment zur Feststellung Nagode, M., Lebar, L., & Dremelj, P. (2018). Orodja za ocenjevanje von Pflegebedürftigkeit. [The new assessment tool for potreb po dolgotrajni oskrbi v evropskem prostoru: izkušnje in determining the need for nursing care], Bielefeld, Institut für izzivi za Slovenijo. Socialni izziv, 23(39), 46–54. Pflegewissenschaft an der Universität Bielefeld. Nagode, M., Lebar, L., & Kovač, N. (2014). Pregled izvajanja socialne oskrbe na domu po evropskih državah s poudarkom na ocenjevanju potreb uporabnikov, Končno poročilo. Ljubljana: Inštitut RS za socialno varstvo. 117 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE PERSONAL PLANNING AND COORDINATION IN LONG-TERM CARE: IDENTIFYING NEEDS AND PLANNING CARE TOGETHER WITH THE USER Magdalena Žakelj Inštitut RS za socialno varstvo Mateja Nagode Inštitut RS za socialno varstvo EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 118 KEY MESSAGES ▶ The personal plan should be a central document in which the user’s data is covered broadly enough to avoid the practice of the user answering the same questions asked by several different experts. ▶ The use of the personal planning method would require multiple training sessions and continuous monitoring of the development of personal plans in order to provide support to long-term care coordinators in even more user-oriented and broad-based personal plans. ▶ Personal plans should record the user’s life story, which provides a wide awareness of their context and clearly reflects their desires and goals, from which the necessary services are derived. ▶ In order to avoid fragmentation of individual care, we propose that all goals from the life story should be written in a personal plan with an arrangement about who will implement them, or which other services/organisations will be involved in their implementation. ▶ Annexes (“changes” or “revisions”) of the personal plan should show changes in the scope of services, as they follow chronologically. ▶ For the future use of the personal planning method in long- term care, it should be defined as to what form of personal plan 119 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE PERSONAL PLANNING AND COORDINATION IN LONG-TERM CARE: IDENTIFYING NEEDS AND PLANNING CARE TOGETHER WITH THE USER and life story writing should be used - a broad-based personal plan or a personal plan focused on long-term care services. Depending on the decision, it will be necessary to adapt the forms and instructions and train the employees. ▶ The pilot projects have confirmed that the long-term care coordinator is the central profile of long-term care, and their work is highly team-based. ▶ The role of long-term care coordinator has proven to be meaningful and crucial for further work with users. We propose that the norm of the number of users with whom an individual long-term care coordinator should cooperate be set low enough that they are able to follow the concepts of the personal planning method and coordination of services in their work. ▶ Considering the warnings of long-term care coordinators from pilot environments, we propose that the possibility be introduced in the future of providing a “transitional service package” or “initial service package” that the user would receive after the assessment of eligibility, during the creation and adaptation of the personal and, with it, the implementation plan regarding their needs. ▶ We propose that long-term care coordinators be systematically acquainted with the social model of cooperation with the user, as included in the principles of the personal planning method. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 120 Introduction In the practice of creation of personal plans, and otherwise, we often or exclusively focus The personal planning method used in on the health condition or deficits of the older pilot projects of long-term care is one of the people and people who need support, which methods that began to develop in the world in we see as the main source of their problems the mid-1980s. In Canada, it was known as case – in this respect we speak about acting within management, and in the United Kingdom, care the medical model of disability or cooperation management (planning and implementation with the user. Consequently, this leads to the of care) was used in addition to this term. This creation of a personal plan that contains mainly personal planning method is not new in Slovenia medical and corrective measures. In order to either, as it started to be introduced in social care avoid medicalisation, as the Common European in the mid-1990s, after it was first presented by Guidelines on the Transition from Institutional to David Brandon in 1993 (Brandon, D. & Brandon, A., Community-based Care call the manner in which 1994; Videmšek & Mali, 2018). a person is perceived as part of the medical model, In Slovenia, the method had different names personal plans must be comprehensive, based in different periods. Initially, it was the care plan as on the social model of disability or cooperation a direct translation of care planning (ibid.), then the with the user, or on the theory of social exclusion individual plan for independent living (Zaviršek, (Škerjanc, 2004; Škerjanc, 2006). This includes Zorn, & Videmšek, 2002), followed by individual identifying barriers in the environment that planning with realisation of goals (Škerjanc, 2006; are the main factor that restricts people, as well Škerjanc, 2010) and then individual planning and as recognising that people with disabilities are service provision (Flaker, Nagode, Rafaelič, & eligible for full and equal participation in all Udovič, 2011). Today the most broadly used term is aspects of society (Common European Guidelines individual planning and implementation of services on the Transition from Institutional to Community- (Flaker, Mali, Rafaelič, & Ratajc 2013). based Care, 2012). As with the mentioned models Despite the fact that personal planning is not of disability or cooperation, the aspect of health is new in Slovenia, the method is still considered also approached from a medical or social model. innovative. For example, “in institutional care From the aspect of the use of nursing for older people, it is mentioned as a condition diagnoses in healthcare, Ščavničar (1998: 32) notes for innovation, because according to Mali et al. that “pre-prepared models for nursing diagnoses (2017), the needs of residents, sustainable changes are only a guideline that is complemented by in institutional care cannot be ensured without those special features that are obvious in the client individual planning and monitoring.” (Videmšek & as a unique being”. She also emphasised that Mali, 2018). these should be based on health and should not In Slovenia, the described personal planning be exclusively focused on disease, and that they method was developed, adapted and used mainly should be developed and supplemented (ibid.). by professionals in the field of social work, and Roper, Logan and Tierney (2001) have created a the key new feature that it brought was the shift model for assessing needs and planning care in in the treatment of the user, from the user as an nursing care that should enable the assessment of object of treatment to the user as the central subject all human needs based on the life cycle of a person. in defining their needs and creating solutions to The purpose of this model is to plan the greatest satisfy them. The key tool of personal planning is possible independence and quality of life of the thus the personal plan, which consists of a life story, user or patient; when planning it is necessary goals and a plan for the implementation of goals to cooperate with them at all times. In assessing (implementation plan). needs and planning care, it is first necessary to get 121 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE to know the person and their needs, plan how to of the individual, in addition to their needs and the compensate for deficits and advance human health problems they face. and other areas of life (Roper et al., 2001). Modern trends and legislation in the field of In the document Preparation of bases for long-term care (e.g. AGE Strategy 2022-2025, Care the implementation of pilot projects that will Act 2014, Common European Guidelines on the support the transition to the implementation of Transition from Institutional to Community-based the systemic act on long-term care (Lebar et al., Care, 2012) show that tailored care and personal 2017), drafted by a group of professionals in social planning should be focused on a person’s life as a and health care and commissioned by the Ministry whole. The goals arising from the personal plan or of Labour, Family, Social Affairs and Equal life story of the user, therefore, concern not only the Opportunities, guidelines are provided on how to healthcare and social care services or goals that are prepare a personal plan in accordance with the expected to be implemented as part of long-term “method of personal planning of long-term care care services, but also other goals of this person. based on life-world research” and how to set goals The purpose of a personal plan is to achieve the and plan the use of both the resources already user’s goals with the help of a personal story. In the available to a person and those that are still to be implementation plan, the goals are then “broken gained in order to achieve their goals. down” into tasks that lead to their realisation or The creators of the bases for the implementation. Goals are thus an important implementation of pilot projects in the field of part of the personal plan, as “goal-oriented” care long-term care have proposed a personal planning is a response to the limitations of care that is method for use in pilot environments that, in their oriented towards problem-solving, especially when opinion, best encompasses a person’s life in a way facing the growing complex needs of people with that enables their needs to be identified and the numerous chronic conditions and challenges in the services required to satisfy them to be selected. socio-economic situation.” (Boeckxstaens, Boeykens, Of course, this method is not the only possible Macq, & Vandenbroeck, 2020). method, as we could use another one that would meet the criteria from the Common European Guidelines on the Transition from Institutional Methodology to Community-based Care (2012). These state that there is no uniform answer as to what The goal of the evaluation was to evaluate the methodology should be used for determining needs implementation of the personal planning method and for personal planning, as long as it follows the and the work of the long-term care coordinator following principles: in pilot projects, and to prepare proposals for ▷ involving users or their relatives or advocates, amendments that would be important in the where appropriate, in deciding on their future and transition to the systemic provision of long- support services. The assessment cannot be made term care. To this end, we focused on reviewing solely by viewing the file and talking to the user’s personal plans45, with special emphasis on carers or professionals; monitoring the result indicator ‘proportion ▷ true involvement of the family or advocates of users with personal plans that are being throughout the entire process (depending on needs); implemented’. In reviewing personal plans, we ▷ an integrated approach that takes into account focused on the recorded living conditions, goals the person as a whole, not only their disability; and implementation plans, and at the same time ▷ taking into account the advantages and resources on the observance of the principles of the personal 45 Personal plan for the provision of long-term care as part of the pilot activities »Implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care« (hereinafter referred to as: personal plan). EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 122 planning method both in the mentioned records To gain insight into the work and role of the and in the work of the long-term care coordinator long-term care coordinator, we conducted several as part of personal planning. At the same time, semi-structured interviews with the long-term our goal was to evaluate the role of the long-term care coordinators at two points of time. We first care coordinator both in light of cooperation conducted four interviews between April and June with the user and other stakeholders as part of 2019 (one each in Krško and Dravograd and two the provision of services, as well as in the field of in Celje), and an additional seven as the project coordinating the provision of services. was being concluded: three in Celje, three in Krško In order to be able to evaluate the set goals and one in Dravograd. In the second interview, in and indicator, we used several types of data and addition to the experience of the role of long-term information. care coordinator, we were also interested in their One of the richer sources of information is reflection on the experience in the pilot project the personal plans of users. We have received a and the vision of the long-term care coordinator’s total of 576 such plans: 181 from Celje, 159 from profile in the future. Those who assumed the role Dravograd and 236 from Krško. We included 60 in of long-term care coordinator later or while the the random sample for the systematic analysis of project was already being implemented were personal plans46 - 20 from each pilot environment. asked in more detail about their experience of the We reviewed and included in the analysis all 71 commencement and conclusion of the role and received revisions of the plan or annexes to the transfer of knowledge and work. In Dravograd, personal plan: 35 from Celje, 9 from Dravograd we conducted an additional interview with the and 27 from Krško. We analysed the records of all long-term care coordinator who performed elements of both documents. their work at the general hospital, focusing on In order to be able to evaluate the user the experience of coordinating discharges from experience with personal planning and the hospital. We also reviewed the reports of coordination of services, we included the users employees on activities as part of the pilot projects in surveys and interviews. After six months and relied on some of the findings or results of of involvement in the project activities, they deliberative discussion in the democratic forum responded to a questionnaire on the experience and reviewed the minutes of 13 meetings between of care, with an emphasis on coordination and the the contracting authority, pilot project coordinators central role of users. We received a total of 259 and long-term care coordinators. More on responses (101 from Celje, 87 from Krško, 71 from individual measuring instruments in Chapter Dravograd), which represents 59.5% of all of those Evaluation of pilot projects and methodology. assessed for the second time47: The quantitative data was then enhanced with the qualitative data, with interviews with users and informal Results carers being conducted for this purpose. We were interested in their experience of being involved in In the remainder of the article, we will the pilot projects. The guidelines for the interview focus first on the profile of the long-term care were adjusted to the aspect of an individual target coordinator and their role in the project from the group, while they otherwise covered the same aspect of coordination of long-term care services, key topics, including the experience of personal and then on the field of personal planning and in planning and coordination. more detail on the personal plan and its individual 46 We first included in the selection the fourteenth in order from the list of all plans (they followed in the order of user codes - first the Celje pilot environment, then Krško and Dravograd), and then included in the sample every fifth one that contained all the data that was the subject of analysis. 47 The questionnaire was conducted by the assessors when making the second eligibility assessment with the user. 123 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE elements, which are compared with the concepts of relationship, knowledge of the structure of the the method used. personal plan, researching the user’s life-world, recording the personal plan and the importance of revising the personal plan. Long-term care coordinator and The long-term care coordinators who started long-term care coordination working on pilot projects at a later date did not receive this type of training, and where this was According to the public call for applications possible, they were taught how to perform the (2018), the post of a long-term care coordinator work by their predecessor. The transfer of work could be assumed by a graduate social worker or a and knowledge took place in different ways, registered nurse. In the pilot projects, the structure from a few hours to a few days, depending on was dominated by the social worker profile: five when the new long-term care coordinator started were social workers and two were registered working and whether the previous long-term nurses. The experience of pilot projects has care coordinator was still employed in the pilot confirmed that this role can be performed by both project. Some long-term care coordinators handed profession profiles, although, as the long-term care over their notes with detailed instructions to their coordinators pointed out themselves, it is essential successors and showed and explained all the main that they have knowledge of both professions - and most important elements. One long-term healthcare and social care. What leads us to such a care coordinator noted in an interview that they conclusion is also the following statement from an went through one role and personal plan with interview with a long-term care coordinator: “It is the predecessor, and one mentioned that they good that a long-term care coordinator comes from handed over to the new long-term care coordinator both the healthcare and social care staff. Because material and literature from the initial training for the medical staff, as I said earlier, are focused on the long-term care coordinators. implementing service, while the social staff are more In the Krško and Celje pilot environments, the ones who listen and, I would say, have a little three different employees held the position of bit broader social sensitivity than the health staff, long-term care coordinator during the pilot project, but it’s not that healthcare workers don’t have it. In which meant significant fluctuation for the project, order to cover this field comprehensively, it would multiple interventions in the group dynamics also be good to ensure that these two professions and, last but not least, an impact on knowledge really complement each other in the future.” . transfer between long-term care coordinators. In As new methods and approaches were used in the Dravograd pilot environment, this role was the pilot projects of long-term care, the long-term performed for the duration of the project by two care coordinators participated at the beginning long-term care coordinators employed for half of of their implementation in special education and the full time. training sessions specifically intended for their As part of the pilot projects of long-term care, role in the project – implementation of personal the first task of the long-term care coordinator in planning and coordination of pilot activities. The cooperation with the user was to create a personal training sessions were carried out by experts in plan (more in the next sub-chapter). In addition to the field of health and social work, authorised by this role, the role of connecting and coordinating the contracting authority of the pilot projects. The all stakeholders involved in care, with the common training sessions empowered them to work with goal of ensuring good and safe integrated long- users and other stakeholders. Of key importance term care for users, was also important. The long- for their work was knowledge of the principles term care coordinator thus led the care unit and of personal planning, establishing a work the independence maintenance unit and worked EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 124 closely with the single entry point. They also arranged presentations of the pilot project and also connected and cooperated with other stakeholders implemented them in various organisations in the from the environment: centres for social work, pilot environment. Throughout the pilot project, home help providers, community nursing and they were the central person and also collaborated others. The long-term care coordinators also with the evaluator. agreed in the interviews that their work was The following statement from an interview highly team-based and that they played a strong indicates that the work of the long-term care role in connecting with all employees within coordinator in the pilot projects was very broad: and outside the project (more on this in Chapter “The coordinator has a lot, a lot of work to do. Teamwork and integration of stakeholders as the This is perhaps also because the work of formal foundations for ensuring integrated long-term care). providers who are on the project needs to be As part of the coordination, the long-term care coordinated, to adapt to them in some way, and coordinator compiled care provision schedules of course to adapt to the user and their relatives, and coordinated the providers and thus took care who had their own expectations and wishes, of the organisation of care. They were available who required time for conversations, so there for phone calls from employees who had various were many telephone conversations. There was questions related to care or reported absence, considerable coordination of schedules, even after as well as from users when they cancelled the the implementation plan was signed. In short, implementation of services for a certain day, even much coordination, and adjustments of sorts to this outside working hours – in the afternoons and at person and that person. There is a lot of work.” weekends. This meant that when receiving a call Even when the pilot projects were coming to from an employee reporting absence, they had an end, when the activities stabilised and ran more to arrange a substitute and inform users of any smoothly, there was still a considerable amount changes in the provision of services and vice versa. of work for long-term care coordinators, as one of It follows from the monitoring of the work them said in an interview: of the long-term care coordinator that, in addition »… the work is still totally varied, even to personal planning and coordination of long- strenuous, because you have to be ready to term care, they performed many other tasks as communicate every day; both with users on the one part of their job description, including (optionally) hand and with representatives of the system/mayors participating in establishing a pilot project in a on the other … Because you fight for their rights, pilot environment and contributing to creating organise training, sessions, and for employees; if teams by selecting staff and working with the their car breaks down, you arrange a new car or contracting authority and other pilot environments a company car … So, it is precisely because of this in creating the forms used in the procedures for different/diverse work and the 24-hour presence that exercising and implementing the right to long- it is, well, demanding.”. term care. They also participated in the creation We ascertained that all these activities did of the procedures themselves and establishing not leave the long-term care coordinator much methods of cooperation with the teams as part time left for direct contact with users. Given the of the pilot environment. They were in charge of number of those involved in the provision of involving and coordinating informal carers and services, one can imagine how small the amount of volunteers. This meant that they organised various contact with an individual user actually was. This education and training sessions for them, which is also confirmed by the following statements by they attended themselves. For the duration of the interviewed long-term care coordinators: entire pilot project, and especially at the beginning, “I basically embarked on this with an idea in some cases the long-term care coordinator of being a long-term care coordinator, and I 125 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE imagined it to be more teamwork, work with One pointed out that the work of the long- users, and coordinating work in the field; that term care coordinator was “multifunctional”, the user essentially gets the services they require as in addition to social work (for example, …. However, it was actually also promotion, and personal planning), it also required a lot of informing, and municipalities/partners ….” organisational knowledge and skills (networking “I may have had a distorted idea. I badly wanted and coordination). However, the nature of the to do more social work. And I did, but it was mostly work of long-term care coordinators changed those first visits, contacts in the family, when you get during the pilot project, as they initially spent to know the situation and see what they need.”. more time educating employees, setting up the During the evaluation, we noticed in several project itself (e.g. preparing documentation) and different places that the scope of work of the long- focusing on planning of work (e.g. establishing term care coordinator was in fact set very broadly. procedures and protocols), and only later were They were also the only ones to carry out personal they able to focus more on the work with users, planning, as opposed to the eligibility assessment, in the field. Only one long-term care coordinator which was carried out by several assessors. These had such a comprehensive experience - the one were able to assess a large number of applicants who performed this work throughout the entire in a short period of time, of whom all eligible pilot project. ones then had to be visited by the (sole employed) long-term care coordinator. For this reason, in the Dravograd pilot environment, two social workers Personal planning and from the independence maintenance unit assisted personal plan the long-term care coordinator in preparing personal plans and establishing and managing Following the eligibility assessment carried services, in agreement with the contracting out by the assessors with the applicants, the authority of the project. long-term care coordinators started personal The scope of work for the post of a long-term planning with those beneficiaries who wanted to care coordinator was set too broadly in the pilot be included in the services. The basic document of project and therefore resulted in coordinators the personal planning method was the personal being overburdened, and for some this was plan48, which in the introduction contained the reason why they left the post. One of the personal and contact information of the user49 and employees wrote in the evaluation report that: their guardian, legal representative or informal “I assess that there is simply too much work carer, followed by elements that we focus on in for a single coordinator, with the number of people this sub-chapter: living conditions, short-term being so large.”. and long-term goals and an implementation plan In interviews, long-term care coordinators with services defined as the user is expected to expressed their reservations and concerns in the receive them. The form for the personal plan following way: also envisaged the following items: additional “…it always seems to me that there should be proposals for professional goals and measures, more coordinators given the number of assessors. required connection or inclusion of other services/ But this is how the public call for tenders was set up”. providers or desired involvement of organised “It seemed to me that I couldn’t make it, that volunteers, and the method of inclusion of other I’m not able to. It was such a large burden on me, healthcare and social care services, volunteers even outside work. It was difficult, I don’t know.” or informal carers, and the item of possible 48 The template of the personal plan form was prepared by the contracting authority and coordinated for use with the pilot environments. 49 In this part, the term user is also used for a person who has been assessed as eligible and for whom a personal plan was prepared. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 126 additional comments relevant for consideration lowest (47.9%), the situation was somewhat more in long-term care. At the bottom of the form was a specific - the problem of long waiting lists soon space for the signature of the user and coordinator emerged, as not all beneficiaries could be covered of long-term care and a space for entering the by the available staff, so personal plans have not place and date of the created plan. been drawn up for these beneficiaries. In general, Before paying a visit at home, the long-term the “proportion of users with personal plans that care coordinators were able to obtain information are being implemented” for all environments relevant to the personal plan from the eligibility combined was 64.2%. assessment and, in many cases, from conversations with employees from the independence maintenance team, who provided their opinions LIVING CONDITIONS – A SUFFICIENT on the basis of a review of the eligibility FRAMEWORK FOR COMPREHENSIVE assessment even before the scope of their services IDENTIFICATION OF USER NEEDS? was determined as part of the personal plan. The long-term care coordinator then obtained other In accordance with the personal planning important information at the first visit to the user. method, the long-term care coordinator, in These visits were usually attended by relatives or collaboration with the beneficiary, records their other informal carers who could participate in the life story, from which they identify together the discussion and development of the personal plan. beneficiary’s needs and determine the goals on In a conversation with the user and their relatives, the basis of which the services will be determined. the long-term care coordinator further anticipated In the personal plan, the life story was renamed which services would be appropriate for their “living conditions” in the pilot projects, and in situation and presented these services to them. the created personal plans these are usually After the visit, they made a final agreement with represented as an abbreviated version of the the employees of the independence maintenance record of “living circumstances”, as recorded unit regarding all the services they are supposed by the assessors in the Eligibility Assessment to receive. After that, the long-term care form. The same instruction for recording was coordinator wrote down the personal plan with the provided in both forms – “living conditions and implementation plan and sent it or brought it to housing conditions, daily life and care, assessment the user for their signature. They entered the new of the condition (physical and psychosocial).” The services in the service implementation schedule long-term care coordinators therefore made and provided all information to the employees similar notes to those of the assessors, only in an from the care unit and reminded them of any abbreviated way, and in some cases, they directly possible special features. copied the record from life circumstances, as one In the Krško pilot environment, where the of the interviewees pointed out: “I was told the rate of created personal plans was the highest following: here you make it out of this assessment, (81.7%)50, personal plans were not made mainly in which is a biography of that person, it’s basically cases where beneficiaries did not want to use the copy-paste, you copy it and make it shorter. That service, and often the reason was also the death is, what is not important for care you take out, of the beneficiary. The situation was similar in you copy the services that were proposed by the Dravograd (69.1%)51. In the Celje pilot environment, assessor, and then you can make additions. But it’s where the rate of created personal plans was the not quite like that. […]. For future use, I set it out 50 Calculated as a proportion of personal plans created for those applicants who were eligible for long-term care in the first assessment. 51 In the Dravograd and Celje pilot environments, the reasons for the interruption of pilot activities were not recorded in the information system in all cases. 127 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE differently, more broadly and more in line with what emerged of the user’s access to records about their I talked about with the user.” . living conditions or circumstances: should these A comparison of the records made by the records be shown to the user at all, or whether it is assessors and coordinators of long-term care, i.e. more appropriate to create two types of records - one “living circumstances” and “living conditions”, to be given to the user and the other to be used only shows that there were almost no substantive by employees. For example, one environment has differences between the records. The content in adopted the practice of not sending a description both records included data describing the user’s of life circumstances in the Eligibility Assessment status that intertwined with a description of the form to applicants who were not eligible for long- help the user needed and the information on who term care services. In the other two environments, provided it, and it was also indicated what else according to the long-term care coordinators, the they were able to do on their own. The record assessors recorded the content regarding living placed an emphasis on the health condition of circumstances that was sent to the applicant in the user and the help they needed relative to the the scope and content at their own discretion, in health condition. Interviews with long-term care particular when this included content to which coordinators showed why there were no major the applicant could react negatively (e.g. excessive differences between the records made by assessors alcohol consumption, causing violence, descriptions and long-term care coordinators: of the applicant or their living environment being “I always got a life story from our assessors, unkempt, etc.). The assessors kept such statements in fact you let them (users) know already that separately, and they could be accessed by the long- you know the situation, that they don’t need to term care coordinator and, if necessary, by others talk about such painful things, because there are who worked with a specific user. many different things that may have already been Here are examples of records from personal established by assessors and we basically focused plans that, according to the principles of the more on the services that they receive. Because personal planning method, should not appear in you could basically see in the eligibility assessment personal plans or other user records: what he can do, where he is still independent, where “Speech is difficult to understand.”, “She is his strengths are, and where he needs help. So, we obtrusive for other residents”, “Does not look good”, focused more on that – their feeling of what they “Participates appropriately in a conversation”, “He would accept and where they feel they need help”. is oriented”, “Cognitive abilities are intact”, “The It transpired that there were no major person has motivation”, “The lady does not know substantive differences between the records made what month it is, whether she has taken her therapy by individual long-term care coordinators from or not.”. different pilot environments. In addition, the Such sentences are problematic or ethically long-term care coordinators consulted with other questionable, regardless of whether they are persons participating in the project before making written in a positive or negative sense, as they the final record of the user’s living conditions and devalue and confuse the person. These are the content of their eligibility assessment, and sensitive topics, and such notes can hurt a person added their observations to the description of the because they do not understand on what basis they living conditions. were created. The following example, in which In various discussions, as evidenced by the a user who pointed out in an interview that he minutes of meetings of various teams, long-term was affected when he received a personal plan in care coordinators and the contracting authority, which the long-term care coordinator wrote that interviews with long-term care coordinators and the he had no motivation, shows the negative impact discussion in the democratic forum, the question of such recording: EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 128 Interviewer: The gentleman helped you or you the user needs, while it is not clear from what did that on your own? User: No, I did it on my is written that he would have said during the own. After she wrote that I have no motivation or conversation that this is exactly what he wants: anything. Maybe I was in such a mood that day, so … “ […] has suffered a stroke, […]. His speech Interviewer: [...] remained the most affected, which he cites as User: Yes, it affected me a little bit, because very annoying, as conversations and socialising motivation is one thing and you thinking about the have always meant a lot to him. GOAL: Improving meaning of life is another, let’s say. communication skills”. It also happens that the long-term care coordinators fail to record the set goal in the form GOALS AND IMPLEMENTATION PLAN as it was written in the living conditions, or that HAND IN HAND it is not evident from the record on the living conditions at all or it may have been set by the Long-term care coordinators wrote down user’s relatives: goals in personal plans in a variety of ways. As “The relatives and the user want the transfer to already stated, the long-term care coordinators the wheelchair to be carried out safely so that they in all three pilot environments usually made can bring him to the car and go on a trip together. the record of living circumstances with a strong GOAL: Establishing basic motor functions that emphasis on the health condition of the individual would enable him to be independent in performing and on what help the user needed, and less often as many basic daily activities as possible.”. they recorded actual, specific preferences of the “The lady lives […]. The diagnosis is user (what a life story, for example, should contain), dementia,[…] The lady’s blood pressure fluctuates which is why it is noticed that the goals in personal greatly, […]. The lady does not want to be cared plans are quite structured: very short, written in for by her relatives, she does not refuse the help a similar way for different users, tied to help from of strangers […]. The lady is tiny and of medium the point of view of the user’s health condition. height. GOAL: The relatives want morning care and In most personal plans, the goals follow well physiotherapy.”. the story written in the living conditions and relate The long-term care coordinators created the to it. For example, if a person needs any form of objectives of the implementation plan primarily care, help with physical tasks or company, this is as descriptions of services : “Preservation of reflected in the goals: motor functions, strength, mobility”, “Obtaining “The lady would like to socialise, as she feels physiotherapy/occupational therapy,” “Measuring lonely. GOAL: Socialising”. blood sugar and controlling medication”, In rare cases, when the user, or relatives “Assistance in the provision of personal care”. on their behalf, expresses a specific wish, this is In the personal plans, the long-term care consistently stated in the goals: coordinators in all three pilot environments mostly “Last autumn, they were able to accompany recorded the short-term goals of users, while him up/down the (steep) stairs so that he could go long-term ones were recorded less often. For many for a walk in a wheelchair, outside the house. He users, it was evident from the records of living would like this to be so this year. GOAL: Improving conditions that, due to their health condition, the physical condition, ability of transportation certain services need to be arranged for them - while being helped up/down the stairs, on the immediately. Services recorded in personal plans, wheelchair and outside the house.”. e.g.: physiotherapy, personal care, assistance in In some examples, it is clear from the nutrition, were therefore usually listed as short- description that the goal represents exactly what term goals. Under the long-term goals, the long- 129 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 1: An example of an implementation plan with goals and an implementation plan Goal 1: Assistance in care Possible special features in Ser- Date of implementation implemen- vice Name of ser- of service (what days, tation of code vice Provider Start date End date from – to) service Mon: 1:30 p.m. – 2:30 p.m. O3 Dressing and Tue: 1:30 p.m. – 2:30 p.m. undressing SO 17/05/2019 Wed: 7:15 p.m. - 8:15 p.m. Thu: 1 p.m. – 2:30 p.m. Mon: 1:30 p.m. – 2:30 p.m. O5 Hair washing SO 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. Wed: 7:15 p.m. - 8:15 p.m. Thu: 1 p.m. – 2:30 p.m. Healthy nail Mon: 1:30 p.m. – 2:30 p.m. O6 care/fingernail SO 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. trimming Wed: 7:15 p.m. - 8:15 p.m. Thu: 1 p.m. – 2:30 p.m. Healthy nail Mon: 1:30 p.m. – 2:30 p.m. O7 care/toenail SO 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. trimming Wed: 7:15 p.m. - 8:15 p.m. Thu: 1 p.m. – 2:30 p.m. Mon: 1:30 p.m. – 2:30 p.m. O8 Skin care SO 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. Wed: 7:15 p.m. - 8:15 p.m. Thu: 1 p.m. – 2:30 p.m. Morning full Mon: 1:30 p.m. – 2:30 p.m. O11 body wash, Tue: 1:30 p.m. – 2:30 p.m. bed bath or SO 17/05/2019 Wed: 7:15 p.m. - 8:15 p.m. bathing Thu: 1 p.m. – 2:30 p.m. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 130 Goal 2: Care for maintaining health Possible special features in Ser- Date of implementation implemen- vice Name of ser- of service (what days, tation of code vice Provider Start date End date from – to) service Monitoring of vital functions Mon: 1:30 p.m. – 2:30 p.m. Z14 in a stable BN 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. chronic Wed: 7:15 p.m. - 8:15 p.m. disease Thu: 1 p.m. – 2:30 p.m. Assistance in eating and drinking by mouth, including Mon: 1:30 p.m. – 2:30 p.m. O12 serving food SO 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. and drink Wed: 7:15 p.m. - 8:15 p.m. to people Thu: 1 p.m. – 2:30 p.m. without swallowing disorders Goal 3: Assistance in household Possible special features in Ser- Date of implementation implemen- vice Name of ser- of service (what days, tation of code vice Provider Start date End date from – to) service Only Assistance Mon: 1:30 p.m. – 2:30 p.m. cleaning of P4 in running SO 17/05/2019 Tue: 1:30 p.m. – 2:30 p.m. floors and household Wed: 7:15 p.m. - 8:15 p.m. cleaning of Thu: 1 p.m. – 2:30 p.m. windows, if necessary. 131 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Goal 4: Maintenance of independence and greater mobility Possible Date of imple- special mentation of features in Ser- service (what implemen- vice days, from – tation of code Name of service Provider Start date End date to) service Assessment and evalu- S1 ation of the state of the FT/K 17/05/2019 user – initial and final S3 Advice for environment adaptation FT/K 17/05/2019 Prevention, counselling S5 and empowerment of the user for independent FT/K 17/05/2019 living Services to maintain motor independence: S6 flexibility, strength, stabili- FT/K 17/05/2019 ty, mobility, endurance and fall prevention EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 132 term care coordinators mentioned, for example: as services that the long-term care coordinators improving and maintaining health condition, listed in the living conditions or under other items maintaining independence, maintaining social of the personal plan, were not systematically skills, i.e. conditions that are expected in the long recorded in the implementation plan itself, as this run with all the services provided. Also recorded exclusively recorded the services provided by among the long-term goals were e.g.: home care, employees in pilot projects. staying in the home environment, helping to obtain The implementation plan was written in a device such as a hearing aid/denture - in short, the form of a table, above which the goal was goals that cannot be achieved in the short term, as clearly addressed, and a code (e.g. O11, S5, S6) and they take some time and/or funding to be achieved. the name of the service in question and which It can be observed from an analysis of goals provider visited the user recorded in the table – in that the use of short-term goals was of primary some cases with the name and surname and the importance, i.e. that most of the recorded goals professional profile, and in most cases only with were short-term. the job or only with abbreviations of the job (FT, It can be concluded from the statement of the SO, etc.) or with abbreviations of the unit (EO, long-term care coordinator that it was not easy EOS). The start date, the end date of the service, to set goals as part of personal planning: “Some which most of the long-term care coordinators did users know exactly what they want, what their goals not enter, and the date of the service (from - to) are, and some get lost in this process because they were also stated, and the note “by agreement” have never thought about it”, “They looked at me was also found. It was therefore usually recorded bemusedly when I asked them about short-term and from which date the service would be provided, long-term goals, about what I’m doing here”. The on which days and at what time. Possible special following statement, which touches on the boundary features were added at the end (e.g. “the lady wants between objectives and services, tells a similar instructions and advice for exercises that she can story: “For example, regular help in personal care later perform on her own” ). or that the person will have regular control over It was evident from the records and stories medications. This is where I was getting lost. It is that regarding some users, it became clear important that these terms are clarified; that you only after the services were determined in the know what the goal is and what the service is.”. implementation plan and the first home visits, The implementation plan is that part of the especially by professional workers from the personal plan that contains an inventory of all independence maintenance unit, exactly which services that the user receives, and states who, services the individual needed and to what degree when and for how long they are provided. In and how much they actually used them. This all three pilot environments, the long-term care was often due to the fact that users needed time coordinators recorded in the implementation plan to accept and become accustomed to receiving only the long-term care services that they provided services and the new dynamic brought by new as part of the pilot project. It was evident from the people entering their homes. rare living conditions in the personal plans that If, in addition to the content of the described the user also receives some services from other main parts of the personal plan, we also look persons or providers, outside the consortia of pilot at the other items of the personal plan listed at projects – for example: “The occupational therapist the beginning of this chapter, we notice that in got connected with Tačke pomagačke and together the item on the manner of inclusion of other they go on visits to the user. Tačke pomagačke is healthcare and social care services, volunteers a volunteer association that performs therapeutic or providers of informal care, the long-term care work with the help of dogs.” . Such services, as well coordinators of all three pilot environments 133 Figure 1: To what extent does a personal plan for long-term care make your life easier? (N=86) recorded all types of services that the person also received outside the pilot project. They mentioned the inclusion of assistance to the family at home, community nursing care, delivery of meals, information on whether the person received various cash benefits, whether they were included in a special social care and employment centre, CELJE % day or intergenerational centres and whether they received any other service outside the pilot 4 10 11 16 15 project; physiotherapy, cleaning, delivery of meals; information on the frequency of the service per week and the tasks performed by the contractor KRŠKO was also entered in some places. The long-term care coordinators also recorded informal care in this space, i.e. whether help was provided by a 2 3 5 10 son, daughter, partner, etc. They did not record, however, what these informal carers do. A special section, “informal assistance”, was dedicated to DRAVOGRAD this, although the long-term care coordinators did not mention its scope to any great extent there because they had already provided a description 4 4 2 of the assistance given by relatives in the previous section on other services. The scope of informal care is an important piece of information about how burdened relatives are by care, i.e. how much TOTAL they are involved in it. It could be seen from the personal plans that contained this information that the number of hours of informal care ranged from 6 10 18 25 27 four to 140 hours per week. It was evident in the personal plans in the spaces where the long-term care coordinator recorded data on informal care tasks that these were mainly tasks and services such as shopping, hygiene care, household chores, meal preparation (the role of informal carers was also observed with additional measuring instruments, which are presented in more detail in Chapter Care for those who care: studying the quality of life of informal carers). As part of personal planning, the long-term care coordinators also performed a revision of the personal plan. The revision was regular, at six monthly intervals or, in extraordinary circumstances, before six months had elapsed, Not at all if a major change to the scope of long-term To a small extent Neither to a small extent nor to a large extent care services was needed. The long-term care To a large extent To a very large extent EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 134 coordinator performed both on the basis of a condition, in the event of the sudden absence of an reassessment of the user’s eligibility for long- informal carer and in similar situations. term care. In the event of a change in the scope of Finally, we shall look at how users have services, an annex was thus concluded with the assessed the role of a personal plan in their lives53. user52. At the beginning of the pilot project, the In the questionnaire, they were asked to what form was identical in content to the personal plan, extent a personal plan for long-term care makes while the contracting authority later prescribed their life easier. Almost a third of the surveyed a special form for the annex. In the annex, the users (31.4%) said that it makes their life easier to long-term care coordinators briefly described the a very large extent, 29.1% said to a large extent, a circumstances that led to the need to increase fifth (20.9%) said neither to a small nor to a large or decrease the scope of services and entered extent, a tenth (11.6%) said to a small extent, and additions or changes to the implementation plan 7% said not at all. It could be said that, in general on this basis. In some cases, the long-term care or on average (average=3.7), that users perceive coordinators only entered the service that had the personal plan as a means of making their life been changed or added, and not the service that easier, although there were also those among the the user received unchanged, which is why the respondents who do not attribute a special role difference in the scope of services received and to the personal plan in terms of making their life the full range of services that the user received easier. Perhaps this information can be connected was not evident from the annex. In addition, there to the fact that the personal plan should cover the was also space in the annex for information on the whole life of a person and thus all the services they assistance the user was still receiving and for any need, and not just the services of the pilot project. other relevant comments. This means that they could manage their entire In addition to the described findings of lives with it. an analysis of personal plans, an analysis of various minutes of meetings has also shown that instead of “personal plan”, the term “agreement” Discussion with key appeared in some places, which significantly messages changes its actual meaning. Considering the experience of their work, the Personal planning in long-term care is highly long-term care coordinators from all three pilot oriented towards the user and their empowerment, environments noted that it would be necessary to which for professionals in the field of long-term introduce in the future the option of providing a care can represent a new perspective and a “transitional service package” or “initial service different way of working with users than they may package” that the user would receive after the be used to from the existing healthcare and social eligibility assessment, during the creation and care systems. adaptation of the personal plan to their needs. This The personal planning method used in the should be especially true for beneficiaries to whom pilot projects therefore places the user into social the provision of services should be ensured as soon life and does not focus solely on their body. The as possible in order to enable them to stay at home long-term care coordinator therefore makes or who have no other care options, e.g. in the event sure that the services required are coordinated of discharge from hospital, in the event of a sudden between the different providers. In doing so, they significant deterioration in health and functional follow the content of the personal plan that they 52 Annex to the personal plan for the provision of long-term care as part of the pilot activities »Implementation of pilot projects that will support the transition to the implementation of the systemic law on long-term care« (hereinafter referred to as: annex). 53 In the survey presented in the chapter on methodology, we asked the respondents who had previously said that they had a personal plan about this. They were asked to rate this on a scale from 1 (not at all) to 5 (very much). 135 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE have prepared together with the user. The user from the perspective of power, in a friendly, thus takes an active role in deciding on the type positively evaluated and user-friendly way, with and scope of services and activities they want to as much information as possible provided by the receive, and the professional becomes a co-creator users themselves and contributing to the quality of and recorder and is no longer in the role of their care. The life story is not just a place in which someone who, from their position, independently to describe an individual’s health condition and, determines the needs of the user and the services especially, to make judgements and impersonal they believe the user needs. It is a collaboration observations. However, we notice that the method between the user and the professional in building of personal planning in the pilot projects in solid, sufficiently flexible and personally selected this respect departed from the principles of the support for the user. In order to strengthen such method and this is why all the content that the life an attitude, in addition to new methods and story is supposed to cover in accordance with the approaches in long-term care it is necessary to method was not fulfilled as expected. In the given acquaint the long-term care coordinators and all context of personal planning, the term “living others who cooperate with users with the social conditions” in the personal plan itself dictated a model of cooperation with the user. narrower record and not a record in the sense of Practice in pilot projects has shown that a life story, despite the fact that the long-term care even someone who has no past experience with coordinators had quite extensive conversations this method can be trained for personal planning. with users and therefore gained a considerable However, as the personal planning method in amount of information about their lives. Also, the field of long-term care in the form as used the focus on the health condition of the user in in the pilot projects has been implemented to a these interviews and records was probably more lesser extent, it is still considered a new feature, meaningful and important for planning the goals despite the fact that it has been known and and services offered as part of the pilot projects. developed in Slovenia since the 1990s. Therefore, The next important element of the personal in the future it will be necessary to provide long- plan are the goals, which are the “link between term and in-depth training sessions on personal the narrative and the implementing part of the planning, and above all to provide the long-term personal plan. They are an excerpt from the care coordinators with continuous support and analysis of the situation and its conclusion, the supervision in personal planning. It also transpires consequence of the narrative and its projection that long-term care coordinators can be both social into the future. At the same time, the goals are the workers and registered nurses, while it would be basic framework for planning the implementation good if these profiles were to complement each of the plan. In an operationalised form, they are other and combine the principles of operation of actually the implementing part of the plan” (Lebar both professions. The long-term care coordinator et al., 2017). needs knowledge from both the healthcare We have established that the goals recorded and social care systems, as this enables them to in the personal plans were usually short, written perform their role more comprehensively and in a similar way for different users, regardless of cohesively. the different problems they faced, and they were In order for the personal plan to encompass mainly related to help from the aspect of the user’s the entire life context of the user, it is important health condition. The reason for this may have that the life story that is the foundation of the been in the structured record of living conditions. personal plan is broadly captured and that In accordance with the concept of the personal it reflects the individual’s life and their own planning method, the goals should always be set in storytelling. It is also important that it is written a descriptive form, in the user’s words, in order to EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 136 find out from them what they are striving for, and can acquire only through practice and with the what plans they have for the future. support of an professional who is trained in this By analysing personal plans, we have found method and has experience in personal planning. that only the services provided by employees on It would therefore be unfair to simply conclude the project were recorded in the implementation that the practice of setting goals or any other plan, while the services that the user received practice in the context of personal planning in pilot from other sources were recorded in the personal environments was not good enough, as the long- plan by different items. As it has already been term care coordinators, after introductory training noted, in order to ensure comprehensive care, sessions on personal planning in the creation of it is important that all services received by the personal plans, no longer received professional user are recorded in one place, in this case in the support in creating and reviewing personal plans. implementation plan. Although the pilot projects When providing long-term care, users should did not envisage this, we think that it would be also be provided with the option of being gradually sensible for the long-term care system that it integrated into services and their personal is consistently recorded in the implementation and implementation plans should be adapted plan what, when and to what extent assistance is accordingly, as some people may be distrustful provided by formal providers, informal carers and of the new or unaware of their needs in full, and others (e.g. volunteers). This way, the personal plan therefore need more time to become accustomed could become a universal right of long-term care to new things, which personal planning has turned and a method of comprehensive care as proposed out to be. People, especially the older adults, are by the creators of the bases for the implementation often not accustomed to the fact that the expert of the pilot projects in the field of long-term care who visits them is interested in their entire life (Lebar et al. 2017) story, that they will explore with them their wishes It is also important to respect the principle of and goals, and that they are able to participate in inclusion of users and their relatives in the entire deciding on the content of the care provided to process of personal planning. As was the practice them. This is certainly a new practice for the users, in the pilot environments, the long-term care and one with which, as the data presented as part coordinators and staff in the care unit present the of the implementation of personal planning in the service options to the user and agree with them or pilot environments shows, they were generally their relatives on the choice of services. All those satisfied and which they evaluated positively. who work with the user must also ensure that the Also important for the planning and user is acquainted with everything related to their implementation of long-term care is information care and other activities and that they participate from relatives and foresight, observations and in deciding on the content and scope of services. other aspects that may be noticed in the field In addition, it is important that the long-term by both the assessor and the long-term care care coordinator ensures that the content of the coordinator, so it is important that all those who entire personal plan, including the implementation work with the user as part of the provision of plan, is written in a way that is understandable to long-term care are informed about them. The the user and their relatives, without abbreviations answer to the question of whether the user should and with clear information on persons and dates have access to all the documentation and records related to the provision of individual services. that employees make about them is certainly not Writing an in-depth life story together with simple, as evidenced by the experience of the the user, identifying and setting specific goals and pilot projects, as this topic has been the subject operationalising them well requires a long-term of various discussions. From the point of view care coordinator to possess many skills, which they of the user’s central role and their perspective 137 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE of power, it is essential that they receive the are expected to be gradually built, and their eligibility assessment and the personal plan as implementation and all the resources that can a whole. Especially where sensitive information contribute to this are expected to be envisaged is concerned, the user should be interviewed in in terms of time and implementation, more visits regard to obtaining their permission about what and interviews with the user and their relatives to include in, and how to write, the personal plan, are required to make a more widely applicable and be informed about what information is to personal plan. In particular, more visits by the be recorded and to whom it will be provided in long-term care coordinator to the user’s home order to ensure the good and safe provision of would be necessary in cases where users have long-term care. As is well known, the user has the complex life situations and particularly specific right to access their entire documentation, so from needs. It is equally important that the long-term this point of view it is even more important that care coordinator, together with the user, draws descriptions and notes about them are compiled in up a personal plan that provides sufficient basic a respectful manner. information to every expert who comes into It is also worth mentioning at this point the contact with the user and serves to avoid the user naming of the personal plan and the annex itself, or their relatives repeatedly having to answer the as we usually limit and determine its role and use same questions. by naming the document. It used to happen that, We also assess that it is sensible to establish instead of “personal plan”, the term “agreement” a “transitional package of services” or an “initial appeared in some places, which implies a much package of services” for beneficiaries immediately narrower meaning of this document, and probably after the eligibility assessment, as in this case an a different, more rigid relationship between immediate response to the user’s needs would be the two “signatories”. A similar narrowness ensured with long-term care services and the user and rigidity also stems from the term “annex”, would not have to wait for these services to begin, which is understood as a kind of addition to the while at the same time a document written in “contract” (i.e. personal plan) that has already been this way could be a legal basis for services to start concluded. The personal plan is, in fact, a product being provided and costs to be claimed until the that can be revised appropriately and constantly to personal plan is drawn up and signed. reflect changes in the condition of users. It can be seen from the description of the The annex to the personal plan should main works and tasks of the long-term care contain a record of the entire context of changes coordinator that they performed a large number in the provision of services and not be just a of different tasks. Among these tasks, cooperation record of changes in the scope of services. For this with the user took up only a small part of the reason, it would make sense to rename it from whole, although direct cooperation and care for the annex to the personal plan to a “change of the user is supposed to be its central task. One of personal plan”, which is what the annex actually the proposals in interviews with long-term care is. The term “revision of the personal plan” has coordinators was “… that someone take over only already become established in social work, so this matters such as schedules and paperwork”. term could also be used. In order to perform quality work, the long- The long-term care coordinators usually term care coordinator can only take over a certain obtained information for the creation of a personal number of users, as interventions and work are plan while having a conversation with the user usually very extensive, and communication, visits during a home visit. Considering the concept of and arranging various matters for the user take the personal planning method used, in which a lot of time. Given that the responsibilities of the life story and the resulting goals and needs the long-term care coordinator are supposed to EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 138 encompass both work on the case of an individual drafting of personal plans would be most suitable and work related to services provided to them for systemic use. The evaluation has shown that (case and care management), the range of tasks a personal plan made in accordance with the pertaining to an individual can be very extensive, described method could be the foundation for all especially when it comes to users with complex persons involved in user care and support, even life situations. It is precisely because of this, and beyond long-term care, so that the personal plan for the sake of quality work with the user, that could become a universal right of an individual. attention should be paid to how low the number In this way, the user and the long-term care of users (and how complex their needs are) should coordinator would become equal partners who be for the volume of work of an individual long- together discover the possibilities and resources term care coordinator to be manageable. The for building a flexible, albeit sufficiently strong, long-term care coordinator should have enough support system for the user, consisting of services time to explore additional resources to meet the and activities offered by various professions needs of the user in the local environment or to and other resources in the community. With identify shortcomings that must be addressed or such a work relationship and a wide range of advantages in the user’s home environment that services, a future long-term care system could must be strengthened. even more consistently follow the user’s needs It could be concluded that it will be necessary and new approaches to the participation of users to thoroughly consider and coordinate well with and all those who work in the field of long-term decision-makers the decision on what form of care would in this way be deeply rooted and personal planning and what method for the consolidated. 139 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE BIBLIOGRAPHY AGE Platform Europe . A Society for ALL Ages AGE Strategy, 2022- Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih 2025. Retrieved from https:/ www.age-platform.eu/sites/default/ projektov, ki bodo podpirali prehod v izvajanje sistemskega files/AGE_Strategy_22-25_FINAL.pdf zakona o dolgotrajni oskrbi«. Retrieved from https:/ www. uradni-list.si/glasilo-uradni-list-rs/vsebina/2018002400003/ Boeckxstaens,P., Boeykens, D., Macq J., & Vandenbroeck, P. (2020). javni-razpis-za-izbor-operacij-izvedba-pilotnih-projektov-ki-bodo- Goal-Oriented-Care. Brussels: King Baudouin Foundation. podpirali-prehod-v-izvajanje-sistemskega-zakona-o-dolgotrajni- oskrbi-st--303-1201816-ob-198618 Brandon, D., & Brandon, A. (1994). Jin in jang načrtovanja psihosocialne skrb i. Ljubljana: Visoka šola za socialno delo. Roper, N., Logan, W., & Tierney, A. J. (2001). The Roper–Logan– Tierney Model of Nursing. Edinburgh: Churchill Livingstone. Evropska skupina strokovnjakov in strokovnjakinj za prehod iz institucionalne v skupnostno oskrbo (2012). Skupne evropske Ščavničar, E. (1998). Negovalna diagnoza. Obzornik zdravstvene smernice za prehod iz institucionalne v skupnostno oskrbo nege, 32, 167-172. (Smernice za uveljavitev in podporo trajnostnemu prehodu iz institucionalne v družinsko in skupnostno oskrbo otrok, ljudi Škerjanc, J. (2004). I ndividualizacija socialnovarstvenih storitev in z oviranostjo, ljudi s težavami z duševnim zdravjem in starih njen vpliv na kakovost življenja uporabnikov. Magistrsko delo. ljudi v Evropi). Brussels. Retrieved from https:/ di.irssv.si/skupne- Ljubljana: Fakulteta za socialno delo. evropske-smernice Škerjanc, J. (2006). Individualno načrtovanje z udejanjanjem Flaker, V., Nagode, M., Rafaelič, A., & Udovič, N. (2011). Nastajanje ciljev Pomen uporabniškega vpliva pri zagotavljanju socialno- dolgotrajne oskrbe: Ljudje in procesi – eksperiment in sistem. varstvene storitve. Ljubljana: Center RS za poklicno izobraževanje. Ljubljana: Fakulteta za socialno delo. Škerjanc, J. (2010). Prispevek uporabnikov socialnovarstvenih Flaker, V., Mali, J., Rafaelič, A., & Ratajc, S. (2013). Osebno načrtovanje storitev k oblikovanju konceptov za praktični študij socialnega in izvajanje storitev. Ljubljana: Fakulteta za socialno delo. dela (doktorska disertacija). Ljubljana: Fakulteta za socialno delo Kahlert, R., Boehler, C., & Leichsenring, K. (2020). Expert monitoring UK Legislation. Care Act. (2014). Retrieved from https:/ www. and evaluation of pilot projects to establish onestop shops and legislation.gov.uk/ukpga/2014/23/enacted integrated LTC delivery in Slovenia, Final report. Viena: European Centre for Social Welfare Policy and Research. Videmšek, P., & Mali, J. (2018). Od načrta skrbi in programa oskrbe do osebnega načrtovanja in izvajanja storitev. Socialno delo, 56, Lebar, L., Dremelj, P., Flaker, V., Rode, N., Mali, J., Peternelj, A., 27 – 42. Smolej Jež, S., ... Kobal Tomc, B. (2017). Priprava podlag za izvedbo pilotnih projektov, ki bodo podpirali prehod v izvajanje Zaviršek, D., Zorn, J., & Videmšek, P. (2002). Inovativne metode v sistemskega zakona o dolgotrajni oskrbiAktivnost 1: Priprava socialnem delu. Opolnomočenje ljudi, ki potrebujejo podporo za orodij za ugotavljanje potreb uporabnikov, metodika postopka samostojno življenje. Ljubljana: Študentska založba. ter ugotavljanje upravičenosti do storitev dolgotrajne oskrbe. Ljubljana: Inštitut RS za socialno varstvo. Mali, J., Flaker, V., Peršič, M. L., Rafaelič, A., Rode, N., Urek, M., & Žitek, N. (2017). Inovativne oblike oskrbe v domovih starejših občanov: končno poročilo. Ljubljana: Fakulteta za socialno delo. 141 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE TEAMWORK AND INTEGRATION OF STAKEHOLDERS AS THE FOUNDATIONS FOR ENSURING INTEGRATED LONG-TERM CARE Polona Dremelj Social Protection Institute of the Republic of Slovenia Aleš Istenič Social Protection Institute of the Republic of Slovenia EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 142 1 TEAMWORK AND INTEGRATION OF STAKEHOLDERS AS THE FOUNDATIONS FOR ENSURING INTEGRATED LONG- TERM CARE KEY MESSAGES ▶ The employees in pilot environments were satisfied with individual aspects of their work life, but they were somewhat less satisfied with working conditions and their direct superiors. They are more satisfied with the work they did, working hours and their position in the project. ▶ They were particularly dissatisfied with the payment for the work performed. The pilot environments classified their employees in salary grades in different ways, usually in respect of whether they were employed within healthcare or social care. The latter had a poor effect on the motivation to work, including mutual cooperation and integration between the colleagues of various professions. The result revealed that dispersion of long- term care between various sub-systems (especially healthcare and social care) had a negative impact on cooperation and integration, and that service integration is also necessary from this point of view as joint competence of various sectors will thus be enhanced. ▶ The providers of social care at home expressed the greatest dissatisfaction with the payment for the work they do among all employees in the pilot environments. ▶ Supervision proved to be an important element of working life and a significant advantage for the employees in the pilot environments. ▶ The team functioning dynamics varied among the pilot environments. While mutual cooperation between the employees within individual teams and among the employees in different teams was noticed in the Dravograd and Krško pilot environments at the end of the project, such cooperation was not established in the Celje pilot environment. The reasons for that can also be sought in fewer formal meetings held between the employees in Celje in comparison to the other two pilot environments. ▶ The LTC coordinator has the central and connecting role in all three pilot environments when it comes to work-related information exchange, work process, and the provision of expert advice when resolving work challenges. 143 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 142 Introduction who express their job satisfaction view their superiors as supporting and appreciating their The well-being of employees, their work, and employees who mutually trust and cooperation, success and dedication to their work encourage each other evaluate teamwork as and how the organisation works as an entity positive and simultaneously report job satisfaction. is very important for the efficient functioning The establishment of cooperation between of organisations. By measuring organisational employees is crucial for successful teamwork. climate, it is possible to determine the functioning Cooperation in healthcare, social care and long- of an organisation or a work environment. term care is composed of two key elements, i.e. the The organisational climate is defined as a set generation of collective action, which responds of measurable characteristics of the work to the complex needs of users and the creation environment, as they are directly or indirectly of a team spirit, which combines the knowledge perceived by people living and working in this of the various members and through which each environment and which affect the motivation person feels respected and trusted (D’Amour & and behaviour of employees. Payne et al. (in Oandasan, 2005). Unfortunately, it is revealed in Berberoglu, 2018) defined organisational climate practice that numerous countries experience a as the way in which employees perceive their lack of integrated health and social care. Problems organisation and its purposes, while Churchill et are particularly evident in coordination and at. (in Berberoglu, 2018) defined it as the sum of cooperation between individual professions the social factors which constitute the workplace (interprofessionalism) and competent institutions environment for a worker. in the fields of healthcare and social care. The Organisations ascribing great significance to results of a three-year project that took place in the provision of good organisational climate are Sweden also showed the same; its objective was more efficient and successful. Good organisational to improve social care and healthcare services climate also strengthens the feeling of satisfaction for elderly people living at home. To attain this at work. Research shows that employee satisfaction objective, multi-interprofessional teams were contributes to attaining better productivity established whose knowledge was to fill the gap in and dedication to work and reduces the level cooperation between social care and healthcare. of absenteeism and intention to terminate the The results of the project were not encouraging employment relationship (Hagmaier & Abele, as the organisation of multi-interprofessional 2012). High-performance organisations focus on teams (by competent institutions) failed. The bringing the best out of their human resources, care of elderly people did not improve, and and in so doing create an exceptional team capable coordination and cooperation between social care of delivering outstanding results (McWinner, 2020). and healthcare were also not established because Research carried out on a representative interprofessional competitiveness prevailed over sample of retirement homes in Switzerland the goal of mutual cooperation (Emilsson, Strid, & (Schwendimann, Dhaini, Ausserhofer, Engberg, Söderberg, 2020). & Zúñiga 2016) showed that the job satisfaction Within the framework of pilot projects of employees involved in the healthcare of users regarding long-term care, the pilot environments is significantly related to supportive leadership54, tried to establish foundations for teamwork in teamwork and the resident safety climate,55 long-term care as a lack of integrated healthcare responsive administrative staff and adequate and social care is also a significant problem in staffing resources in the organisation. Employees Slovenia. 54 The most important quality of supportive leadership is care for others and provision of help (Boštjančič, 2009). 55 The terms safety climate or safety culture refer to the positions of the healthcare provider on matters pertaining to user safety. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 144 Methodology organisations in 2001 by a group of consulting companies under the auspices of the Chamber of As part of the evaluation, two general Commerce and Industry of Slovenia.56 objectives were set regarding employee working The questionnaires were not necessarily life, i.e. to evaluate teamwork in long-term care completed by the same persons (considerable (LTC) (whether the method of teamwork in LTC staff turnover) on both occasions, which is why is suitable for the transition to the systematic the samples of surveyed persons are discussed implementation of LTC) and the quality of working independently. Thus, the changes in opinions cannot life of formal care providers (whether the quality be monitored at the level of individual employees, of (working) life of formal care providers improves but at the level of all employees together. during the implementation of pilot projects). From At the end of the pilot projects, we prepared the aspect of monitoring the course of activities an additional questionnaire for the employees by in pilot projects, we examined the attainment means of which we inquired how and in which of the following process indicators by means of cases do employees offer support to one another, evaluation: connect and cooperate. Four questions were ▷ regular meetings between LTC coordinators, care drafted for the respondents, inquiring about which teams and assessors, at which information was colleague they turned to for help and support in exchanged about possible challenges at work, open relevant situations. For data analysis, we used issues, etc. (at least five times a year), the Pajek57 programme, which is intended for the ▷ regular supervision of formal care providers and analysis and visualisation of large networks. employees at the single entry point. Cooperation between employees on the The quality of employee working life in project or teams was also monitored with the pilot environments was examined with an online help of minutes taken at meetings in the pilot questionnaire for employees. We wanted to environments, which were provided by the determine how employees feel while working on environments. To clarify certain results, data from the pilot project and what, in their opinion, are the the user satisfaction survey (M6) was also used. advantages and disadvantages of the work they Descriptive (e.g. presentation of proportions) and perform. The survey was implemented twice in bivariate (t-test) data analyses were also performed. order to determine how the employees’ quality of life changed during their work on the project. The questionnaire included questions to Results measure organisational climate and employee satisfaction with certain aspects of working The results are presented in four separate conditions, work-family life balance, contacts with sub-chapters, i.e. we first present the dynamics various stakeholders in the field, and positive and of the teams which were established in pilot negative aspects of work within the project. To environments, this is followed by the sub- measure organisational climate and satisfaction chapter on organisational climate and employee with individual aspects of working conditions, we satisfaction. In the next sub-chapter, we discuss somewhat adjusted the questions from the SiOK cooperation between the employees from the questionnaire (Slovenian organisational climate), aspect of providing mutual support, and the last which was drafted within the project of research sub-chapter focuses on cooperation with other and monitoring organisational climate in Slovenian important stakeholders in the local environment. 56 We used the questionnaire for 2007 obtained from the diploma thesis by Gorše (2011). 57 Mrvar & Batagelj, 1996–2020. Accessible at: http:/ mrvar.fdv.uni-lj.si/pajek/. 145 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Team dynamics in pilot Organisational climate and environments employee satisfaction All three pilot environments established key Table 1 shows assessments of individual teams important for the implementation of long- categories of organisational climate in the pilot term care (more in Chapter Transition to integrated environments. When reading the results of long-term care by establishing a single entry point, measuring organisational climate, we particularly integrated care team and connecting stakeholders). focus on the comparison of results from two In Krško, all teams were located at one location, different times, i.e. at the start and the end of project while in the other two environments they were implementation. The assessment results of individual dispersed at various locations: three in Celje and six categories of organisational climate represent in Dravograd. a guideline indicating that the organisation of In all environments, the teams were connected processes must be improved accordingly in the areas and coordinated by the long-term care coordinator, in which assessments are low. who also participated in practically all of the Assessments reveal that respondents perceived meetings listed below. With the exception of three their working environment as rather positive meetings of the assessors’ team, no meetings were at the start of the project. “Attitude towards held in Celje, which would be divided by individual quality” was assessed highest, which means that teams. Meetings of all teams were held 23 times. the employees contribute to the quality of work The integrated care team held the most meetings to the best of their abilities and feel responsible (47) in the Krško pilot environment. Meetings for the quality of their work. Assessments in the usually took place on a weekly basis. The assessors “motivation and engagement” category were also held 14 meetings, the care team held six and the very high, which means that the employees are independence maintenance team held five meetings. committed to their work and prepared to invest Numerous meetings were thus organised in this additional effort. Their superiors appreciate work environment, which were divided by individual well done and good work results are duly noticed teams until July 2020. In the last months of the and commended. Attitude towards quality is an project, the employees held joint meetings. The organisational advantage reported on by all three assessors at the single entry point held the most pilot environments. Motivation and engagement are meetings (27) in the Dravograd pilot environment. organisational advantages of the Celje and Krško Some 14 meetings of the integrated care team with pilot environments, while this category received a assessors were also organised. These meetings took lower score in the Dravograd pilot environment. place on a monthly basis, except in the period from “Knowledge of objectives” of the pilot project April 2019 to September 2019. The integrated care received the lowest assessments (but still quite team met eleven times during the project and the high) in the first measurement. In this area, all home help providers in the local environment were three pilot environments tackled the challenge of present at five meetings. improving this aspect of organisation during project The Covid-19 epidemic significantly affected implementation. Poorer aspects of organisations the implementation of meetings in all pilot or the aspects representing a challenge for the environments between mid-March 2020 to mid-May Dravograd pilot environment also included 2020. There were no physical meetings in this period, management, communication and information, and instead the employees exchanged information by organisation. The latter is particularly attributed to phone or e-mail. the various locations at which employees perform their work. The LTC coordinator for Dravograd is, for example, located in Slovenj Gradec and the EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 146 Figure 1: Location of individual teams in pilot environments DRAVOGRAD RESIDENTIAL HOME FOR THE ELDERLY SLOVENJ GRADEC ▷ SEP assessors ▷ Team for activities of daily living and instrumental activities of daily living ▷ Independence maintenance team RESIDENTIAL HOME FOR THE ELDERLY KOROŠKA ČRNEČE ▷ LTC coordinator ▷ Independence maintenance team SLOVENJ GRADEC GENERAL HOSPITAL ▷ LTC coordinator CENTRE FOR SOCIAL WORK, RAVNE NA KOROŠKEM UNIT ▷ SEP assessors KOROŠKA INTERGENERATIONAL CENTRE IN RAVNE NA KOROŠKEM ▷ Team for activities of daily living and instrumental activities of daily living RAVNE HEALTH CARE CENTRE ▷ SEP assessors — CELJE CELJE HEALTH CARE CENTRE ▷ LTC coordinator ▷ SEP assessors ▷ Team for activities of daily living and instrumental activities of daily living ▷ Independence maintenance team PUBLIC INSTITUTION SOCIO ▷ Team for activities of daily living and instrumental activities of daily living ST JOSEPH HOME ▷ Team for activities of daily living and instrumental activities of daily living — KRŠKO CENTRE FOR SOCIAL WORK, KRŠKO UNIT ▷ LTC coordinator ▷ SEP assessors ▷ Team for activities of daily living and instrumental activities of daily living ▷ tim za ohranjanje samostojnosti 147 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE single entry point had its head office in Ravne na of the project in comparison to the other two Koroškem. This posed a great challenge for the environments, the average assessments increased organisation of work and impeded the ongoing in all categories. The highest difference in the communication and information transfer between average assessment was in the category of the LTC coordinator and the single entry point. knowledge of objectives, followed by affiliation, At the end of the pilot project, the assessment and innovation and initiative. of affiliation, internal relations, attitude towards The respondents were also asked about work quality, and innovation and initiative satisfaction regarding individual aspects of their somewhat improved in all project environments, working life. Average satisfaction assessments but the differences between assessments are not are positive in all environments, which means statistically significant. The assessments of the that employees are (more or less) satisfied with knowledge of objectives, which was below average individual aspects; they are less satisfied with (3.4) during the first survey, scored 3.6 at the end of working conditions and their direct superiors, the project and the organisation category increased and they are more satisfied with the work they do, from 3.5 to 3.8. The difference in assessments of working hours and their position in the project. the latter was statistically significant between the If looked at by individual pilot environments, first and second measurements (t = 0.646, p = 0.09). differences between the observed points in time If looking at individual pilot environments, are found only in the Celje pilot environment, the average assessments of organisational climate i.e. the score for work satisfaction somewhat categories, except for organisation, decreased decreased during the second survey (AS = 4.6 in Celje. The assessments in Celje were higher if (M0); AS = 4.1 (M18)). The difference is statistically compared to the remaining two pilot environments significant (t = 2.287, p = 0.03). Satisfaction with the during the first survey, i.e. average scores were work they do dropped statistically significantly (t above 4 in eight out of ten categories. At the start = 1.934, p = 0.09), especially for the assessors in the of the project, the highest scores in Celje were Celje pilot environment, i.e. the assessors were less received by attitude towards quality (AS = 4.9) and pleased with their work at the end of the project motivation and engagement (AS = 4.5), while at implementation than at its onset. Due to high the end of the project, these two scores decreased staff turnover in the Celje pilot environment, the the most, i.e. AS = 4.4 and AS = 4.2, respectively. second survey was mostly completed by different Differences in assessments are statistically assessors, which is why we can say that the group significant (t = 2.564, p = 0.015 or t = 3.308, p = of assessors responding to the first survey were 0.002). At the end of the project, the assessment less satisfied with their work than the assessors regarding the knowledge of objectives also responding to the second survey. decreased, i.e. from AS =3.69 to AS = 3.40, which An important aspect of satisfaction with could be related to a high staff turnover in the Celje working conditions is also satisfaction with the pilot environment. payment for the work done. At the start of the Differences between scores at the start and project, 46.5% of the respondents were satisfied end of the project were not statistically significant with the payment for their work and more in the Krško pilot environment; nevertheless, than half of all respondents (52%) at the end of positive changes are seen in the categories the project. If looked at from the viewpoint of affiliation, internal relations, organisation and individual pilot environments, the proportion of knowledge of objectives. employees who are satisfied with the payment In the Dravograd pilot environment, where for their work increased in Celje (from 38.5% to the lowest scores in individual organisational 45%) and Dravograd (from little over 31% to little climate categories were recorded at the start less than 47%), while this proportion somewhat EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 148 Table 1: Differences in average assessments of organisational climate categories in total and by pilot environments during the survey in points M0 and M18 Y ALIT TION AND A TION E TION ES TION TION AND TION AND V A A V LEDGE OF TI TIONS V GEMENT UDE ETENCE V TI GEMENT C A ARDS QU TI W W ORMA TTIT ROFESSIONAL OMP OMMUNIC AFFILIA INTERNAL REL MO ENGA A TO P C INNO INITIA MANA C INF ORGANISA KNO OBJE M0 3.83 3.93 4.11 4.42 3.90 3.99 3.77 3.63 3.50 3.36 L M18 4.04 4.07 3.96 4.35 3.83 4.01 3.75 3.72 3.82 3.61 A TOT ↑ ↑ ↓ ↓ ↓ ↑ ↓ ↑ ↑ ↑ differ- ence* * M0 4.08 4.42 4.50 4.88 4.27 4.34 4.03 4.04 3.69 3.69 M18 4.03 4.35 4.15 4.43 3.93 4.02 3.80 3.98 4.13 3.40 CELJE ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↓ differ- ence ** ** M0 3.83 3.76 4.13 4.29 4.00 4.06 3.88 3.63 3.64 3.39 O M18 4.07 3.83 3.90 4.22 3.83 3.99 3.78 3.68 3.86 3.81 KRŠK ↑ ↑ ↓ ↓ ↓ ↓ ↓ ↓ ↑ ↑ differ- ence M0 3.60 3.67 3.77 4.13 3.50 3.61 3.44 3.28 3.20 3.03 M18 4.01 3.97 3.78 4.40 3.68 4.03 3.64 3.42 3.37 3.67 OGRADV ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ DRA differ- ence * * ** Key: Green indicates categories with the highest score (above 4) (these are most likely advantages of the organisation). Yellow indicates categories assessed above average (between 3.5 and 4). Red indicates results below average (less than 3.5) (these most likely represent challenges for the organisation). * Statistically significant difference (* p < 0.10; ** p < 0.05) 149 Figure 2: Proportion of employees who are (un)satisfied with the payment for work performed (in %) (N=41 (M0), N=52 (M18)) dropped in Krško (from 71% to almost 65%), but still remains high and is somewhat higher if CELJE compared to the other two pilot environments. If focusing on satisfaction with the payment M0 for work performed between individual 61.5 38.5 workplaces, we can determine that the employees in the care unit (AS = 2.85) were on average least M18 satisfied with the payment for their work and 25 30 45 those most satisfied were the employees of the independence maintenance unit (AS = 3.57). The difference is statistically significant (F = 1.925, p KRŠKO = 0.023). The difference between the employees of the care unit and assessors (AS = 3,33) is also M0 statistically significant (F = 1.925, p = 0,.099). 14.3 14.3 71.4 At the start of the project, administrative work presented more of a burden for the major M18 proportion of employees than at the end. A decline 11.8 23.5 64.7 in the proportion was most visible in the Celje pilot environment, i.e. 23% of employees stated that administrative work presented a burden at the DRAVOGRAD start of the project and only 5% of employees were of this opinion at the end of the project. In Krško, M0 this proportion dropped by more than one third 25 43.8 31.3 (35.7%) to almost one quarter (27.8%), and from one half to one third in Dravograd. M18 One of the aspects of the quality of employee 20 33.3 46.7 working life is also workload. The figure below shows the proportion of employees who returned home from work too tired to do certain necessary TOTAL household chores several times a month, and the proportion of those who had difficulties in meeting M0 their family obligations due to the time they spent 32.6 20.9 46.5 at work. At the onset of project implementation, M18 the latter totalled about 20% of employees in all pilot environments (somewhat less in the Celje 19.2 28.8 51.9 pilot environment), and the number of employees who returned home from work too tired to do certain necessary household chores several times a month amounted to more than one half in Krško and Dravograd, while their proportion was lower in Celje (38%). At the end of the project, this proportion decreased in all pilot environments. In the Celje and Dravograd pilot environments, the proportion of employees who had difficulties (Very) dissatisfied in meeting their family obligations due to the Neither dissatisfied neither satisfied (Very) satisfied 150 Figure 3: Proportion of the respondents who experienced what is stated below several times a month (in %) (N=41 (M0), N=53 (M18)) time they spent at work decreased, while this proportion somewhat increased in Krško. CELJE A comparison of self-assessment of employee M0 health at two points of time reveals a minimally 38.5 lower assessment at the end of the project in the 15.4 Celje and Krško pilot environments, but still the employees in all three pilot environments assessed M18 their health conditions as good at both the start 35.0 and end of the project. 15.0 An important aspect of the employees’ working life is also the possibility of supervision in pilot projects. Supervision was organised in the KRŠKO Celje pilot environment in various teams (team for activities of daily living and instrumental M0 50 activities of daily living had four supervisions, the independence maintenance team had one 21.4 supervision and the assessors had three). In the M18 Krško pilot environment, the supervision took 44.4 place in two teams; one for assessors and both 22.2 coordinators (the project coordinator and the LTC coordinator) and the second one in the integrated care team. Since September 2019 and at the supervisor’s initiative, one group consisted of a DRAVOGRAD LTC coordinator, and the project coordinator and M0 57.1 the second group included the integrated care team and assessors. Certain supervisions involved 21.4 all employees in the project. Supervision was also established in the Dravograd pilot environment, M18 40 but it only included expert workers of the care team. The employees of the independence 13.3 maintenaince unit and assessors at the single entry point expressed no need for supervision, but they did meet at intervision meetings. By means of the TOTAL dynamics and structure of supervision groups, the M0 criterion of the indicator (ten times a year) was 48.8 thus met in the Krško and Celje pilot environments. 19.5 Supervision was established in Dravograd, but it only involved expert workers of the care team and M18 the indicator was thus not met. 39.6 17.0 I returned from work too tired to do certain household chores which should have been done at home. Due to the time spent at work, I had difficulties in meeting my family obligations.. 151 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Cooperation and provision of For expert advice when resolving work mutual support among the challenges, the majority (11) of colleagues turned employees to the LTC coordinator. Several employees (four or five) turned for advice to two employees at The networks of exchange of assistance and the single entry point. On average, the employees support between colleagues in individual pilot turned for expert advice to 1.7 persons and no environments are presented below. Only the one turned to seven persons (five of these were employees who responded to the question set in working at the care unit, one at the independence the survey were included in the analysis, which maintenance unit and one at the single entry point). means that persons who could be mentioned by The most frequent source of support in the others and those to whom others could turn for form of a conversation was the assessor, to whom help were excluded, but they themselves did not eight colleagues turned. Other frequent sources respond to the question. The reason for this was of such support include another employee in the significant staff turnover, especially in the Celje care unit and the LTC coordinator. On average, pilot environment. With the elimination of units the employees discussed work problems with 3.4 from the analysis mentioned above, we obtained colleagues, and no one turned to two employees. a complete network of employees for each pilot For emotional support60 (discussion about environment, i.e. final and completed group of important personal matters), the employees in the employees. pilot environment turned to fewer colleagues than This is a targeted network in which in the case of other types of support. It is revealed connections between two units (employees) that mutual exchanges occur between certain are targeted, thus the direction of connection is employees, i.e. two or more persons speak about important and the relations between persons are important personal matters. presented as asymmetrical connections. If two In the Dravograd pilot environment, the persons select one another, we can also speak questionnaire on the support network was about symmetrical connection (Carrington, Scott, & responded to by 14 persons who were working Wasserman, 2009). in the pilot project during its implementation. Figure 4 shows a complete network of Regarding information related to work, the information exchange related to work in the Celje majority (nine) of employees turned to the LTC pilot environment. There are no isolated units coordinator and the employee in the independence or employees in the network to whom no one maintenance unit. Six employees turned to would turn for information. To the greatest extent the project coordinator, two assessors and two possible, the employees turned for information employees in the care unit. On average, the to one of the LTC coordinators (14 employees); employees turned for work-related information according to the number of entry connections, to more than 4.6 persons; half of them contacted these are followed by the assessor (13 employees more than five and a half persons and one half of turned to them) and two employees from the care employees turned to fewer than five and a half unit (12 employees turned to one of them and 11 to persons. Only two persons were not contacted by the other).59 any of their colleagues for information. The employees turned for information related Narrow teams were also formed within to work on average to 7.4 colleagues or one half of the pilot environment, between which mutual them turned to fewer than seven colleagues and communication or work-related exchange of half of them to more than seven. information took place. The information exchange 59 In the figure, the LTC coordinator is marked by KDO, assessor by EVT, an employee from the care unit by EO, and an employee from the independence maintenance unit by EOS. 60 Emotional support is assistance at a major or minor life crisis (death of a loved one, divorce, problems in the family or at the workplace, etc.). EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 152 1 Figure 4: Network for information exchange related to the work in the Celje pilot environment (entry connections), complete network (N=18) Figure 5: Network for information exchange related to the work in the Dravograd pilot environment (entry connections), complete network (N=14) 153 EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 153 Figure 6: Network for information exchange related to the work in the Krško pilot environment (entry connections), (N=18) Figure 7: Two-way connections (work-related information exchange) between employees in the Krško pilot environment EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 154 took place between the LTC coordinator and implementation. The majority of employees (15) project coordinator, and also between the LTC contacted the project manager for work-related coordinator, one employee in the care unit and one information, somewhat fewer people turned to the assessor. The second triad represents an employee LTC coordinator (13) and the assessor (12). Eleven in the care unit and two assessors. employees sought information from three employees The central person in the network of expert in the independence maintenance unit, the assessor advice exchange when resolving work challenges and the second LTC coordinator. If compared to was the LTC coordinator. The latter was contacted the other two environments, the average number by eight colleagues for expert advice. One of the of colleagues to whom employees turned was the employees in the independence maintenance highest in Krško, i.e. ten people (the median had the unit, to whom four colleagues turned for expert same value), and the minimum number of employees advice, i.e. three in the care unit and one in the (6) who contacted their colleagues for information independence maintenance unit, played a minor exchange was also the highest. central role. On average, the employees turned to The strong connection between the employees 1.5 persons for expert advice and six colleagues in the Krško pilot environment regarding the were not contacted by any of the employees for exchange of work-related information can also expert advice. be seen in Figure 7, which shows the network of The central role regarding discussions about employees between whom at least five two-way problems at work was also played by the LTC connections take place. This means that each coordinator in the Dravograd pilot environment, employee specified at least five persons with whom who was contacted by seven colleagues. Other they exchanged work-related information, and at important sources of such support were two least five persons also indicated that same person as assessors and one employee in the care unit, a source of information exchange. For example: the whom four colleagues turned to. On average, project manager contacted the project coordinator the employees contacted 2.3 persons to discuss for information, and four assessors turned to the work-related problems, and two persons were not LTC coordinator. The latter was a distinctly central contacted by any of the employees. person as mutual exchange of information took Mutual discussions about work-related place between them and twelve employees. issues took place between two assessors and one The majority of employees (14) turned for employee in the care unit. These were also the expert advice when resolving work challenges to employees contacted by three other colleagues. the project manager. Seven employees contacted the Emotional support was sought by the LTC coordinator for expert advice and six employees majority (four) employees in the Dravograd turned to two assessors and one employee in the pilot environment with the LTC coordinator independence maintenance unit. On average, the and an assessor. On average, the employees employees contacted 4.8 persons when seeking turned to 1.5 colleagues for emotional support expert advice regarding work challenges and they and six colleagues were not contacted by any of minimally contacted two persons. the employees for such support. As in mutual The central role in the Krško pilot discussions about work-related issues, a two-way environment was played by the employee of exchange of emotional support took place between the independence maintenance unit regarding three employees, i.e. two assessors and one discussions about work-related problems, as employee in the care unit. they were contacted by nine colleagues. Other In the Krško pilot environment, the important sources of such support include the questionnaire was responded to by 17 persons project manager (contacted by eight employees) who were working in the pilot project during its and both LTC coordinators, who were contacted 155 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE by six colleagues. On average, the employees At the start of the project, they cooperated contacted somewhat fewer than four persons to for the most part with users’ relatives, employees discuss work-related problems, and one employee of centres for social work, home help providers contacted the fewest persons, i.e. one. and employees of the retirement home. In the Mutual discussions about work-related Celje pilot environment, where the single entry issues took place between two employees in the point was located in the health care centre, the care unit and one employee of the independence cooperation with the community nursing service maintenance unit. These were also the employees and doctors was most intensive (if compared with contacted by three other colleagues. other environments). On the other hand, the Krško The largest proportion of employees (8) in pilot environment most intensively cooperated the Krško pilot environment contacted either with home help providers and employees of the the project manager or the employee in the occupational activity centre. In the Dravograd pilot independence maintenance unit for emotional environment, the cooperation was most intensive support. Important sources of emotional support with the employees of the retirement home are also the LTC coordinator, employees in the because the LTC coordinator was located there. independence maintenance unit and two assessors At the end of the project, the Celje pilot who were contacted by five colleagues. On average, environment enhanced cooperation with users’ the employees talked to 3.4 people about important relatives and neighbours and home help providers personal issues, while four employees contacted on the one hand, while on the other hand, the only one colleague. When examining the number cooperation with the employees of the centres of two-way connections regarding emotional for social work was somewhat less intensive. The support between employees, we can determine Krško pilot environment strengthened cooperation that the exchange of such support took place with the community nursing service, doctors particularly between two colleagues. and other stakeholders, including volunteers, employees of the occupational activity centre and NGOs. The cooperation of the Dravograd pilot Integration with other environment with home help providers was on important stakeholders in the one hand less intensive than at the start of the the community project, while the cooperation with the employees of the centres for social work and retirement Integrated long-term care does not only home and other stakeholders was more intensive, denote the integration of various profiles in one particularly with the employees of NGOs. organisation or project, but also the integration of various organisations and actors in the provision of care in a community. The pilot projects particularly Discussion with key integrated various profiles within the project, while messages cooperation with other actors in the community was not established at the level of direct user care, which The establishment of teamwork in pilot was also not one of the project’s objectives. The LTC projects was examined from several aspects, i.e. on coordinators did not incorporate service providers the basis of the dynamics of meetings between and in the personal and implementation plan who were within individual teams, within the framework of outside the scope of the pilot project, i.e. in the measuring organisational climate and from the community. Nevertheless, the employees in pilot viewpoint of exchanging different types of support environments cooperated with the stakeholders in between the employees. the local environments at other levels. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 156 Figure 8: Proportion (in %) of external colleagues with whom the employees cooperate at least once a week in their work (M0 and M18) % 0 5.4 8.6 8.5 9.5 8.5 12.0 15.8 14.5 5.4 10 2.9 4.3 3.6 4.3 5.7 4.0 5.4 4.3 4.3 2.0 7.1 7.2 5.4 20 7.9 6.0 7.8 4.8 7.1 14.3 5.3 6.0 30 8.4 22.7 12.8 23.2 12.5 5.7 18.4 7.2 40 5.7 26.0 19.1 50 17.3 7.9 10.6 20.5 25.7 21.4 60 17.9 22.0 19.1 70 22.5 18.1 12.5 44.7 80 31.4 25.0 90 22.0 24.1 19.3 21.4 19.1 100 Users' relatives Employees at centres for social work Home kelp providers Employees at retirenment home Community nursing service Users' neihgbours Volunteers Others 157 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE The dynamics of meetings in all pilot mentioned above, i.e. that the employees became environments revealed that the condition of the familiar with the entire procedure of long-term care indicator, which anticipated at least five regular implementation, including the work of other teams. meetings between LTC coordinators, care teams The data on the networks showed that and assessors, at which information was exchanged smaller teams also formed between the employees about possible work challenges and open issues, in the Krško and Dravograd pilot environments, was fulfilled. Irrespective of the indicator’s among whom assistance and support were more criterion, we assess that the number of formal intensive. It is encouraging to know that not many meetings in the Celje pilot environment was rather isolated units were in the networks, i.e. employees low, as they mostly took place on a monthly basis to whom no one turned for help. and in certain periods even once per two months. When introducing changes or new solutions, Among the organised meetings in the Dravograd the assessment of organisational climate is pilot environment, a large number of meetings very important because it enables us to identify held between the team of assessors and the LTC weak areas of the organisation and strive coordinator stood out, which was also the result of towards improvements accordingly, in addition the fact that the assessors and the LTC coordinator to identifying areas which are critical for the were based at different locations. All three pilot wellbeing of employees. Numerous studies environments explained that the major part of in healthcare and other fields reveal that the communication and information exchange about employees are more satisfied and deal with less the developments on the project and resolution stress and burnout if they work in environments of possible work challenges was implemented on in which leadership offers support, and mutual a daily basis in the form of informal discussions support, cooperation and consensus building are among the employees. The Krško pilot environment promoted (Stone, Pastor, & Harrison, 2006). can be highlighted as an example of good practice The assessments of individual categories of when establishing teamwork, as great attention organisational climate in the pilot environments was paid at the start of the project to the employees were lowest at the start of the project regarding the getting to know each other and learn as much as knowledge of the project’s objectives, organisation, possible about the entire procedure of long-term communication and management from the care implementation, which included the work viewpoint of all and individual pilot environments. of other teams and that of different organisations The assessments of organisation and knowledge of related to long-term care, even if these did not objectives increased at the end of the project. An participate in the project. increase in the category of knowledge of objectives The data about the provision of mutual failed to occur only in the Celje pilot environment. assistance between the employees testifies that The latter can be attributed to the fact that mutual cooperation between the employees in the numerous employees joined the project during its pilot environments was intensive. The role of the implementation (and in later phases), which is why LTC coordinator proved to be very important in they perhaps received insufficient information and this respect as they stand out in all environments knowledge about the pilot project’s objectives, or they as a person to whom other employees turned for failed to participate in the initial phases of the project information and advice to the greatest extent. High when guidelines and objectives are usually formed. level of cooperation between the employees was Successful establishment of cooperation or seen especially in the Krško pilot environment, teamwork among the employees is also indicated where the average number of colleagues contacted by a high score in the category of internal by the employees was the highest (ten people). relations, which increased during the project in The latter could be the result of the good practice Krško and Dravograd. Supervision also proved EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 158 to be an important element of working life and a We determined that work in pilot projects significant advantage for the employees. was quite tiresome for the employees, especially The quality of employee working life was at the start as almost half of them reported that determined on the basis of satisfaction with they were unable to perform certain household individual aspects of working life. Average chores after work due to tiredness. Some 20% of satisfaction assessments were high in all employees spent so much time at work that they environments, which means that employees were had problems meeting their family obligations. The satisfied with individual aspects; they were less workload somewhat decreased at the end of the satisfied with working conditions and their direct project, whereby it is impossible to determine from superiors, and they are more satisfied with the the data whether the reason for such reduction work they did, working hours and their position lies, for example, in a better workflow of the in the project. The employees in the care unit were project, a lower number of applications filed than on average least satisfied with the payment for at the start of the project, or whether the reduction their work and those working in the independence would be more pronounced if the Covid-19 maintenance unit were the most satisfied. epidemic had not begun during the surveying and As the main reason for dissatisfaction with completing period of the project. the payment, the employees mentioned differences An important finding arising from the survey in salary grades between pilot environments. The on user satisfaction was that, when being asked pilot environments classified their employees in to state at least one thing with which they were salary grades in different ways, usually in respect satisfied in the pilot project, the majority of users of whether they were employed within healthcare mentioned the employees who were visiting them or social care. The latter had a poor effect on the at their homes. They mentioned the employees’ motivation to work, including mutual cooperation personal qualities (kindness, care, good spirits, and integration between the colleagues of various helpfulness), their assistance, professionalism and professions. The employees also highlighted accessibility/humanity (“they ask me things”, “they that salaries were too low considering the give me advice”). Socialising and discussion with complexity and scale of the work, an issue which the employees were also assessed as positive. was associated with the resignations of some The employees’ cooperation with other employees. Least satisfied with the payment stakeholders in the community was not based for their work were the social carers, who, in on the implementation level because the LTC accordance with the uniform wage system of coordinator incorporated in personal and the public sector, are classified in salary grades implementation plans particularly the services that are even lower than those of comparable provided within the pilot projects. At the professions such as personal assistant. implementation level, they were thus unable to Regarding employee workload, it may be establish actual integration between social care deduced from the results that a lot of work in and healthcare into integrated and comprehensive the initial phase of pilot project implementation user care. Nevertheless, the employees cooperated concentrated on the preparation of the project with stakeholders in the local environment in itself and suitable documentation, which other ways. The intensity of employee cooperation resulted in more administrative work. Suitable in individual pilot environments somewhat records and documentation were for the differed, as it depended on, for example, the most part established at the end of the project employee profile, location of the single entry point implementation and, as a result, less time was in the pilot environment, prior cooperation with dedicated to administrative tasks. the relevant stakeholders, etc. 159 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE BIBLIOGRAPHY Berberoglu, A. (2018). Impact of organisational climate on Gorše, A. 2011. Zadovoljstvo zaposlenih v podjetju X: diplomsko delo. organisational commitment and perceived organisational Ljubljana: Fakulteta za družbene vede. performance: empirical evidence from public hospitals. BMC Health Service Research, 18(399). Hagmaier, T., & Abele, A., E., (2012). The multidimensionality of calling: Conceptualisation, measurement and a bicultural Boštjančič, E. (2009). Vpliv nezavednih motivov na stil vodenja. perspective. Journal of Vocational Behaviour 81, p. 39–51. Psihološka obzorja. 18(4), p. 37–57. McWinner, Y. (2020). Organisational Climate and Job Satisfaction: Carrington, J. P., Scott, J., & Wasserman, S. 2009. Models and A Literature Review. International Journal of Current Research. Methods in Social Network Analysis. Retrieved from https:/ www. 12(02), p. 10013–10018. researchgate.net/publication/200465300_Models_and_Methods_ in_Social_Network_Analysis Mrvar & Batagelj, 1996–2020. Pajek, Program for Large Network Analysis. Retrieved from http:/ mrvar.fdv.uni-lj.si/pajek/ D'Amour, D., & Oandasan, I. (2005). Interprofessionality as the field of interprofessional practice and interprofessional education: Schwendimann, R., Dhaini, S., Ausserhofer, D., Engberg, S., & Zúñiga, an emerging concept. Montreal: Faculty of Nursing, University of F. (2016.) Factors associated with high job satisfaction among Montreal. care workers in Swiss nursing homes – a cross sectional survey study. BMC Nurs 15, 37. Emilsson, M. U., Strid, A-L., & Söderberg, M. (2020). Lack of Coordination between Health Care and Social Care in Multi- Stone, P., Pastor. K.,D., & Harrison, I.,M. (2006). Organisational Professional Teamwork – the Obstacle for Coherent Care of Older Climate: Implications for the Home Healthcare Workforce. People Suffering from Multi-Morbidity. Journal of Population Journal of Healthcare Quality. 28(1), p. 4–11. Ageing. doi: 10.1007/s12062-020-09300-8 161 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE IMPLEMENTING AND STRENGTHENING LONG-TERM HOME CARE SERVICES Lea Lebar University of Ljubljana, Faculty of Social Sciences Boris Majcen Institute for Economic Research Valentina Prevolnik Rupel Institute for Economic Research Izidor Natek University of Ljubljana, Faculty of Social Sciences EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 162 IMPLEMENTING AND STRENGTHENING LONG-TERM HOME CARE SERVICES KEY MESSAGES ▶ In three pilot environments, users received a total of more than 100,000 services in more than 37,000 visits. By introducing services to maintain independence in the users’ home environment, the pilot project contributed to the equalisation of the rights of persons in institutional care and at home. ▶ Users expressed very high satisfaction, recognised the usefulness of the new services they received in the pilot projects, and reported positive effects on their independence and quality of life. The introduction of new services from the pilot project is therefore also necessary and desirable at the system level. ▶ The analysis of services and needs in the local environment was hampered by methodological challenges in data collection and recording. Data regarding services already received by users in the local environment are collected unsystematically, which poses a systemic problem that pilot projects have failed to overcome. ▶ The data shows large differences between the services envisaged in the implementation plans and the services actually implemented. Based on the available data, we can conclude that the providers mostly did not follow the implementation plan closely or changed the services during the implementation phase. 163 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Introduction 57 municipalities only on weekday mornings. The price per user varies from 0 EUR to 9.52 EUR per As regards the current provision of rights, hour, which puts the inhabitants of municipalities which, according to the definition, fall within with higher prices in a highly unequal position. the field of long-term care, it can be said that in Not all people who need the service receive it Slovenia they are provided in different ways, due to the heavy workload of the staff; as of 31 in different social protection systems, through December 2019, there were 772 people waiting different social and health care services and for the service in Slovenia, and providers have cash benefits. In Slovenia, home care is provided further estimated that at least 446 more people through health care with the community nursing could be included in assistance at home, but for service and through social care with family various reasons they are not (Kovač, Orehek, & support at home61, with the responsibility for Černič, 2020). The profession therefore points to organising home care being further divided the need to expand existing services and bridge between the state and local authorities. Slovenia, inequalities in access on the ground, as well as to like most European countries, is facing fragmented the necessity of developing new innovative forms provision of services in the field of long-term care, of community-based services and programmes i.e. a lack of integration of health and social work, (e.g. Flaker, Nagode, Rafaelič, & Udovič, 2011; and the slow development of both, especially in Flaker, Mali, Rafaelič, & Ratajc, 2013; Nagode, the user’s home and in the context of demographic Kovač, Lebar, & Rafaelič, 2019). The A-Qu-A project change and the rapid increase in needs. Zavrl of Zavod za oskrbo na domu has shown, inter Džananović (2019) concludes for the community alia, that people living at home need more health nursing sector that the structure of employees services, counselling on health topics and issues, is currently inadequate and that the community nursing care, physiotherapy and occupational nursing sector is not adequately prepared in therapy (Perko, 2016). At the same time, the terms of staffing to meet the challenges and needs development of new home-based services is brought about by demographic trends. Despite the essential if Slovenia is to follow the European proven importance of prevention, it is difficult to Commission guidelines that commit Member upgrade the predominantly curative activity to a States to deinstitutionalisation, which, as Ilinca, preventive one with such staffing levels, although Leichsenring and Rodrigues (2015) point out, according to the Institute of Macroeconomic can only be achieved through a well-coordinated Analysis and Development (Bratuž Ferk et al., combination of informal and formal, user- 2021), they are being strengthened. The increasing centred community care, and through improved demand for community nursing services is linked coordination between different disciplines and to the ageing population, the higher incidence of areas of care. The development of new services chronic diseases and the insufficient capacity of must go hand in hand with the strengthening social care services to support families at home62 of existing services in the user’s home, with the (Bratuž Ferk et al., 2021). For home help, regular common aim of preventing or at least postponing monitoring by the Social Protection Institute of the institutionalisation of users and delaying (or the Republic of Slovenia has shown for many preventing) the emergence of more extensive years that geographical, temporal and price access needs through preventive measures. varies across Slovenian municipalities. In fact, the The need to strengthen services in the home service is only available in 209 municipalities (it environment is also highlighted in strategic is not available in three municipalities), including documents. The Resolution on the National 61 Home care also includes personal assistance, family assistants and residential groups (Nagode et al., 2014; Bratuž Ferk et al., 2021), but these are not analysed in detail in the pilot projects. 62 In the text we also use the term assistance at home. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 164 Social Protection Programme 2013–202063 in its In the team for maintaining independence, second objective highlights the improvement a physiotherapist, an occupational therapist, of the availability and diversity as well as a master in kinesiology and/or a social carer ensuring the accessibility and affordability of provided services to home users in accordance services and programmes. The Resolution on with their professional competences. The services the National Health Plan 2016-2025 “Together they provided are called “new services for for a Healthy Society”64 highlights the need to maintaining independence” (also: new services) raise awareness among decision-makers and the in this chapter and are intended to prevent falls, population for greater societal and individual improve motor independence, raise awareness responsibility for health, to move from disease to of health strengthening, counsel for greater health and to upgrade and attribute importance independence in living spaces, advise informal to prevention activities. Data from the World carers on the correct approaches to working with Health Organisation (WHO, 2014) shows that the user and prevent burnout of informal carers, prevention and health promotion activities are and prevent and manage mental distress. cost-effective in the long term; therefore, the Draft Long-Term Care Act (2021) also addresses the area of a new set of services, i.e. services Methodology to maintain and enhance independence. These are expected to significantly slow down the The chapter focuses mostly on the results of progression to higher levels of dependency on testing new services to maintain independence, the help of another person, and to improve the looking at user satisfaction with the new services, quality and safety of life of long-term care users the perceived usefulness of the new services, the and their relatives, who often take on the role perceived effects of engagement with the new of informal carers. As the Ministry of Health services, and satisfaction with the work of the explains in the introduction to the Draft Long- independence maintenance team. We include the Term Care Act (202165), they want to reduce the views of users and staff in the pilot environments. current inequalities in service provision, as We monitored three key indicators, specifically the the current system does not provide the same proportion of users receiving services relative to all range of services in the community or at home eligible users living at home, the proportion of users compared to institutions, despite the comparable who are satisfied with the new services and the needs of citizens. proportion of users who perceive the new services In response to these initiatives, the public as useful. tender highlighted the second key objective In this chapter, we corroborate the of the pilot projects as “testing new services quantitative data with qualitative data collected and integrated treatment of the user in the in different databases. Quantitative data on the home environment”. In the pilot projects, two services provided was extracted from the central teams provided services: a care team and an information system (frequency and duration independence maintenance team. The former team of each service) and from users’ personal plans consisted of a social carer, a nursing carer and a (analysis of implementation plans, in which we nursing assistant, and provided basic activities of monitor the number of times each service is daily living services (ADL), instrumental activities recorded in the implementation plan). We have of daily living services (IADL) and nursing services. only considered data on the services implemented 63 Resolution on the National Social Protection Programme 2013–2020 (Official Gazette of the Republic of Slovenia [ Uradni list RS], No. 39/13) Resolution on the National Health Care Plan 2016-2025 »Together for a healthy society« (Resolucija o nacionalnem planu zdravstvenega 64 varstva 2016–2025 (ReNPZV16–25)) 65 Vlada Republike Slovenije. (2021). Predlog zakona o dolgotrajni oskrbi 165 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE in the pilot projects. We do not include the remaining of Assistive Devices Scale PIADS-10 translated services under the existing rules in the analyses, as into Slovenian (Day & Jutai, 1996; Hsieh & Lenker, individualised anonymised data is not systematically 2006; Jutai & Day, 2002; Jutai et al., 2007). In order collected at the national level, nor collected by the to make the questionnaire as simple and short as providers in the framework of the project. possible for users, we used only four of the ten User satisfaction with the new services and indicators, which we assumed would be the most with the independence maintenance team and relevant to describe the change in the situation of the perceived usefulness of the new services users. The scale takes into account the effects on were measured in a questionnaire after one year specific aspects of daily life (rated on a scale of -3 to of involvement in the project (M12). This part 3)66. Satisfaction with the work of the independence of the questionnaire was co-designed with the maintenance team was measured using the MRPS independence maintenance teams in the three (The MedRisk Instrument for Measuring Patient environments by transforming the list of ten new Satisfaction with Physical Therapy Care) (Beattie, services into the five key tasks of the team that Turner, Dowda, Michener, & Nelson, 2005), with they perform in their direct work with the users only one of the two factors (i.e. the user-provider and where we expect the highest impact. The main interaction factor). We have omitted the factor aim was to simplify understanding and reduce the related to external factors of care, as this is linked burden on users in the interview process. In the to the institutional environment. following, we relate the quantitative data mainly to We supplemented the data with qualitative the following five key tasks: methods. We analysed the semi-annual staff ▷ provision of services to maintain and improve reports and the open-ended responses of users to motor independence; the M6 and M12 questionnaires. We included the ▷ motivation in learning to live independently; reports of the staff members in the independence ▷ the provision of psycho-social support services; maintenance team and marked their quotes ▷ advice on possible activities in the local in the text with PZ. In order to understand the environment or assistance in exercising rights; and experiences of users and informal carers involved ▷ assistance in adapting the living environment. in the pilot activities, we conducted 21 semi- 156 users responded to the questionnaire structured interviews with users and 21 with on the new services provided to users in the pilot informal carers. The interview guidelines were project. Of these, the largest proportion (two the measuring instrument for the interviews and thirds) reported receiving services to maintain followed the four main research objectives of and improve motor independence, 41.7% services the evaluation (i.e. procedures, methods, people, to motivate them to learn to live independently, system) and also included framework questions 35.5% psychosocial support services, 33.8% regarding satisfaction with the new services. services to advise them on networking activities in The topics covered were the strengths and their environment or to help them exercise their weaknesses of the new services, the user power rights, and 29.9% services to help them adapt their aspect, the adequacy of the number of hours, living environment. the missing services. In the text, quotes from the To measure the effects of the new services, user interviews are denoted by I-UP. Quotes from we used a modified validated Psychosocial Impact interviews with informal carers are marked I-NF. 66 The scale is used to determine whether the use of the service to strengthen and maintain independence greatly decreased (-3), significantly decreased (-2), slightly decreased (-1), neither decreased nor increased (0), slightly increased (1), significantly increased (2) or greatly increased (3) the individual aspect of the user's daily life. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 166 Results carried out activities that could not be recorded within the set of services in the pilot project, which We present the results of the (new) service unduly reduced their effective time. Therefore, provision in the pilot projects in three areas: in in Krško, additional services were defined and the first part, we present a general overview of entered in the information system which were not the provision of long-term care services in users’ originally in the codebook: homes, mentioning all of them, and highlighting S11 – Short telephone conversation: used by the the new services in the home environment. This social carer to inform users and informal carers is followed by an overview of user satisfaction and other stakeholders involved in the user’s with the provision of services and finally the treatment. perceived usefulness and effects of receiving the S12 – Informing formal providers: informing new services. doctors and community nurses. S13 – Placement in a hospital or a care home: includes all communication between the user or Providing long-term care their informal carer and the representatives of the services in users’ homes institution. S14 – Safe discharge. The services provided in the users’ homes S15 – Volunteering: introducing volunteers, under the project can be divided into four keeping their records, introducing them to work groups: basic activities of daily living services, with users. instrumental activities of daily living services, S16 – Longer telephone conversation: i.e. nursing services and independence maintenance psychosocial assistance to users over the telephone services. A total of 549 users67 (178 in Celje, 220 that is of a longer duration than 15 minutes. in Krško and 151 in Dravograd) received services We find that the largest number of nursing under the pilot project. Beneficiaries were those services was performed (36,016 or 36.0%), living at home; almost half of the beneficiaries followed by supportive daily tasks (24,722 or in Celje (47.1%), two thirds of the beneficiaries 24.7%) and basic daily tasks (21,731 or 21.7%), the in Dravograd (65.7%) and three quarters of the smallest number represents the independence beneficiaries in Krško (76.1%) received at least services (17.6%). Access to the latter services one service. Any adult citizen of the Republic of was more time-limited in terms of volume (see Slovenia who was assessed as eligible for long-term Table 2). There are some differences between care on the basis of an application submitted in the the pilot environments, with nursing services prescribed procedure was qualified to become a predominating in Krško and Dravograd, and basic beneficiary of the services. and supportive daily tasks in Celje. Users received a total of 100,028 services in a We find that the vast majority of users who total of 37,182 visits. On average, a user received were included in the services (N=549) received 182.2 services from the project68, with an average at least one new service, 88.8% of users in Celje, of 2.7 services per visit. A total of 43,378 services 95.9% of users in Krško and 95.4% of users in were provided in Celje, 40,423 in Krško and 16,227 Dravograd. The receipt of new services is presented in Dravograd. Of the total, 3,122 services were in the table 1, where we present two data sets from recorded as newly added services, which were the information system: the proportion of those recorded only in the Krško pilot environment. who had an actual service recorded in the field and According to them, they (mainly social carers) the proportion of those who had a service recorded 67 The calculation does not include users who only had a recorded visit from a long-term care coordinator. 68 In the calculation of the average, we do not include users who did not engage in services for various reasons. 167 Figure 1: Structure of services provided by sets in the pilot environments (%) (N1=43,378 (Celje), N2=40,423 (Krško), N3=16,227 (Dravograd) and N4=100,028 (total)) in the implementation plan. There are significant differences between the two figures, with either services listed in the implementation plan not having been implemented or services not recorded in the implementation plan being implemented in the field (see Table 1). We therefore suggested that CELJE the implementation plans be updated accordingly during the implementation of the project. The result indicator that we monitored does not 30.0 31.6 16.6 21.8 precisely define the tolerances for deviations of the proportion of services implemented from the proportion indicated in the implementation plan. However, we consider that the differences are too pronounced. The gap between what is written in KRŠKO the implementation plan and the implementation of services was also pointed out by the long-term care coordinators in interviews (see chapter 15.2 16.0 17.7 51.0 Personal planning and coordination in long-term care: identifying needs and planning care together with the user). The public tender set the expected hours of services in each eligibility category (see Table 2) DRAVOGRAD and severely limited access by users. Based on the available data, we find that the providers visited users in the Celje pilot environment on an average 15.7 28.0 19.7 36.6 of 11 times per month or a little more than 2.5 times per week, in Dravograd on an average of 9.3 times per month or 2.2 times per week and in Krško 16.6 times per month or slightly less than 4 times per week (see Table 2))69. At the same time, SKUPAJ users also received unchanged services under the current legislation; these hours are not included in the analyses. 21.7 24.7 17.6 36.0 The absolute differences between the number of hours performed in the field and the number of hours foreseen in the Draft Long-Term Care Act % 0 20 40 60 80 100 (2020) increase as the eligibility category increases. Although data on the services users receive under current legislation is not systematically collected, we tried to identify possible differences in the number of services provided and the duration of services for two groups of users: the first group included users Activities of daily living 69 To calculate the average, we took into account the number Instrumental activities of daily living of weeks the user received visits and the number of visits per Independence maintenance services team during this period. Nursing care EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 168 Table 1: Proportion of users who received each new service to maintain their independence (% ex.) and the proportion of users who had each service written in their implementation plan (% IP) (n=548) Pilot environment Celje Dravograd Krško Name of service % ex. % IP % ex. % IP % ex. % IP 77.5 64.0 65.6 11.9 91.4 0.5 S1 User assessment and evaluation – initial and final Informing and involving team 16.3 7.9 74.8 10.6 50.9 0.9 S2 members or other stakeholders in the treatment process, reporting to the coordinator 48.9 43.8 39.7 6.6 28.6 12.7 S3 Advice for environment adaptation Advice and training for informal 18.0 9.0 24.5 8.6 42.3 28.6 S4 carers to ensure quality and safe care for the user Prevention, counselling and 60.7 38.2 41.1 9.9 46.8 44.5 S5 empowerment of the user for independent living Services to maintain motor 82.6 61.2 41.7 16.6 58.6 58.2 S6 independence: flexibility, strength, stability, mobility, endurance and fall prevention 6.7 1.1 19.2 2.0 42.3 40.5 S7 Counselling for the management of chronic non-communicable diseases Health and healthy lifestyle 12.4 3.4 51.0 4.6 51.8 42.7 S8 promotion programmes and counselling for the user and the informal provider 54.5 25.3 80.1 17.2 66.8 31.8 S9 Psychosocial support for users and/ or relatives 0.0 0.6 32.5 10.6 12.3 16.8 S10 Assistance in integration of the user into the community *Note: The first number (% ex.) for each pilot environment represents the percentage of users with a service recorded in the IT system (among users who received services in the pilot environments, not among all users who were eligible for services); the second number (% IP) for the pilot environment represents the percentage of users who had a service foreseen in the implementation plan. 169 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 2: Foreseen number of hours in each eligibility category for ADL/IADL and nursing services (hours per month) according to the Draft Long-Term Care Act (2020), foreseen number of hours in the pilot projects and average number of care team visits and hours implemented per environment. Foreseen Foreseen number number of hours of hours in in each each care category catego- of care ry (per defined in Celje (average Dravograd (aver- Krško (average month) – the proj- number of visits age number of number of visits Draft Act, ect (per and hours per visits and hours and hours per 2020 month)* month)* per month)* month)* Visits Hours Visits Hours Visits Hours Category 1 20 7 6.1 3.9 6.9 4.5 13.6 3.1 Category 2 40 10 7.9 4.9 9.6 6.9 22.7 7.6 Category 3 60 16 10.1 7.1 9.5 7.6 16.5 6.8 Category 4 80 19 14.9 9.5 10.3 7.4 13.5 5.2 Category 5 110 24 18.3 12.5 11.4 10.2 17.4 9.2 Total 10.9 7.2 9.3 7.0 16.6 6.2 * Does not include data from the independence maintenance team. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 170 who had received neither formal nor informal care in the day goes by very quickly, I wish there was before the eligibility assessment, and the second more time in one piece” (M6). However, many of group included users who had already received the users who expressed in the survey that they informal care or formal home care services. Given do not receive enough hours of assistance directly the limitation on the number of hours of care relate to a desire for more services provided by a per month, we would expect the second group physiotherapist: “I wish the physiotherapist came to receive fewer services and shorter visit times more often” (M12), “It’s a pity there is not more compared to the first group. However, the analysis physiotherapy!!!” (M6). Staff overload led to long of the results did not reveal significant differences waiting queues (see chapter From application to in the average length of each visit (the average service: experience of procedures in pilot projects length of each visit was a little more than five for more details on waiting lists), to which the minutes longer in the second group compared to the environment also responded by offering only average length of each visit in the first group which e-care to individuals. The latter is reflected in a was 40.3 minutes), nor in the average number of greater degree of dissatisfaction in Celje, as two long-term care services (2.3 services per visit in of the users say: “I am waiting for services, and I the second group and 2.2 services per visit in the only have e-care” and “we did not get the services second group). The existing receipt of formal and/or we wanted” . The pilot environments tell us that informal forms of care does not appear to have had more and more older individuals need services a significant impact on further limiting the length of to maintain their independence, so the demand each visit or reducing the number of services. for these services will continue to grow in the We therefore note that the average number future: “As a physiotherapist, I see problems of hours implemented by providers per user is low, especially in the area of norms. There is a high which is due to the project specificity that limited demand for physiotherapy services in the field. In the number of hours by category (see Table 2))70. our environment, we have two part-time employees, The limits of hours set in the project are higher which is totally insufficient” (PZ). than the limits of hours set in the Draft Long-Term In the context of scope, providers of new Care Act (2021), which results in the number of services point to the inadequate distribution of hours performed per month in each setting being hours according to the eligibility category of users significantly lower than the legal limits. and the lack of flexibility in determining the scope Consequently, it is of course not possible of services for maintaining independence. They to draw conclusions about the actual needs of argue that especially users in the lower categories users on the ground solely on the basis of data of care have too few hours and suggest a more from the pilot projects. The data shows that only individualised approach to each user, with more a good third of the users in the pilot projects discretion for providers in determining the time (35.1%) considered that they had received enough for each user. assistance. The largest proportion (39.6%) would “What and how we work with our care like a little more help, and 22.1% would like a lot recipients is for physiotherapists to decide, within more help than they received in the framework the limits of our competences and health legislation. of new services. They would like “more frequent Those with the lowest rating have the lowest hourly visits” (M6) and “services to last longer” (M12). One rates and those with a higher rating have the user also argues that visits are too short because: highest. I think it should be the other way around. “the carers didn’t manage to arrange everything” It is precisely for this reason that the patients with (M12), while another points out that: “One hour lower rating need more physiotherapy, so that they 70 The employees in the pilot environments were able to apply to the Ministry of Health for an increase in the number of hours for users who were assessed as needing more assistance. The Ministry approved all requests received. 171 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 3: Perceived satisfaction with the new services in pilot environments Celje Dravograd Krško TOTAL N AS SD N AS SD N AS SD N AS SD motivation in learning to live independently 36 4 1.1 10 4.9 0.3 19 4.6 0.5 65 4.3 1.0 assistance in adapting the living environment 27 4.2 0.7 6 4.2 1.6 13 4.4 0.7 47 4.2 0.8 maintaining and improving motor 62 4.6 0.6 28 4.5 1.1 13 4.5 0.5 103 4.5 0.8 independence psychosocial support 31 4.2 0.9 8 4.4 0.7 16 4.3 0.6 55 4.2 0.8 advice on possible activities in the user’s environment or 29 4.1 0.6 3 4.7 0.6 19 4.4 0.7 51 4.3 0.6 assistance in exercising rights Note: N... number of units of analysis; AM... asymmetric mean; SD... standard deviation EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 172 do not get into higher categories of care over the therapy and I do it regularly, and the lady who does years and consequently become more dependent on it with me always tells me how it’s okay not to make our care in all aspects (especially care)” (PZ). mistakes, because sometimes you do something and “For category 1, for example, 55 minutes per you are doing yourself harm” (I-UP). Furthermore, month is absolutely too little and meaningless. in the open-ended questions, the psycho-social Half an hour a week would be optimal. Otherwise, support services are also largely highlighted it varies from person to person – some need less, by the users, more specifically the social aspect others need much more. I think we should have is mentioned as one of the key strengths of the more freedom in determining the time and frequency project: “To have someone come to me and talk, of visits to maintain independence” (PZ). socialise, help” (M6). When asked about their satisfaction with the attitude of the providers of the new services User satisfaction with (the independence maintenance team) towards new services to maintain the users (rated on a scale from 1 to 5), the users independence attribute the highest level of satisfaction to the respectfulness of the team members towards the As part of the satisfaction survey, we focused users (AM=4.8; SD=0.4). The attitude of the team on the overall satisfaction with the new services members is also highlighted by users as a key and satisfaction with the providers. To assess positive aspect of the project in the open-ended overall satisfaction, we asked users to rate each questions (questionnaires M6 and M12). Most of received new service on a scale from 1 – very them consider that the contractors are friendly, dissatisfied to 5 – very satisfied the statement respectful, good humoured and do quality work: “Considering all the experiences you have had “respectful attitude, kindness, willingness to help, with the service up to this point, how satisfied advise...” (M6), “kindness, helpful, all praise for the have you been with it?”. Based on the results of the contractors” (M12). In the interview, one user also survey, we find very high satisfaction among users stresses the importance of trust between service for all groups of new services, with the highest providers and recipients: “Trust. Yes, trust, that level of satisfaction for services to maintain and was fine” (I-UP). The majority of users also felt that improve motor independence (AM=4.5; SD=0.8). the team members described the services they We find that users are slightly more satisfied received in detail (AM=4.5; SD=0.7) and that all with the service “motivation in learning to live the questions they asked were answered (AM=4.5; independently” in Krško (AM=4.9; SD=0.3) and SD=0.6): “To have someone professional to ask Dravograd (AM=4.6; SD=0.5) compared to Celje about my body. I fall asleep in the evening with real (AM=4.0; SD=1.1) (Kruskall-Wallis H=10.8; p=0.005) physical tiredness and not a single muscle hurts, (see Table 3), while for the other services we which means that the exercises are really well did not find statistically significant differences composed, that the whole body is happy” (I-UP). between the environments. There were no other The majority of users would contact the team statistically significant differences by gender, age, members for help in the future (AM=4.6; SD=0.6), environment or eligibility category. and the majority of users are overall quite satisfied The interviews and open-ended responses with the services provided by the team members confirm the identified satisfaction of the users, (AM=4.6; SD=0.6). “Everyone who comes wants to who state, for example, that “it helps me because I do more than 100%, as I see it, as I assess people” am active again due to the new activities... /.../ the (I-UP). Satisfaction with the quality of service is occupational therapist pushes you forward, I get a also reported by users in the survey, in which they new impetus, I also take care of the occupational highlight the efficiency, reliability and generally 173 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Figure 2: User satisfaction with the work of the independence maintenance team (%), all pilot environments The team member treated me respectfully. 22.6% 76.6% I would contact the team members for help in the future. 2.9% 31.4% 64.2% Overall, I am completely satisfied with the service I received from the 3.6% 33.3% 61.6% team members. The team members described the services I received in detail. 3.7% 39.3% 55.6% The team members answered all my questions. 7.4% 37.5% 55.1% The team members gave me detailed instructions regarding my home 5. 13. 1% 2% 33.8% 47.1% programme. The team members advised me on ways to avoid future problems. 6.0% 16.4% 34.3% 41.8% The team members did not listen to my concerns. 28.8% 14.4% 3.8% 25.0% 28.0% The team members did not spend enough time with me. 37.5% 18.3% 5.9% 24.3% 14.0% I completely disagree. I disagree. I neither agree nor disagree. I agree. I completely agree. 174 Figure 3: Identified impacts of new services on users (modified PIADS-10, change) good work. Informal carers also report satisfaction with service providers: “I shouldn’t forget to say this, the kinesiologist showed me a few things, how to make it as easy as possible... not to torture myself and not to make my spine hurt. And he watched me, CHANGE and the kinesiologist was surely there 3 or 4 times” (I-NF). Most of them agreed that the team members had presented all the service options to them and Quality of life answered their questions. 0.7 Identified benefits and effects of receiving new services Independence For the services received, users also answered the question How useful is each of the received 0.7 services to you? Please rate on a scale from 1 to 5, where 1 means that the service is not useful at all to Sense of control over their life you and 5 means that it is very useful. The highest perceived usefulness is found for services designed to maintain and improve motor independence 0.4 (AM=4.6; SD=0.7) and services designed to motivate users to learn to live independently (AM=4.5; SD=0.6). “Those 13 hours were fine, because he came Self-esteem regularly, and it made a difference. /.../ Yes, yes, it was extraordinary to tell you that, I couldn’t believe 0.3 it, he put him on his feet, he put a stick in his hand, and he stood by the bed, I thought the heavens were going to open up to me, let’s say. /.../ I couldn’t do it, -3 -2 - I don’t even know the moves, I’m not such a poor 1 0 1 2 3 amateur, but I tried so hard that I learnt quite a lot of his exercises” (I-NF). We find that the service “motivation in learning to live independently” was more frequently identified as useful by users in Krško (AM=4.8; SD=0.4) and Dravograd (AM=4.6; SD=0.5) compared to Celje (AM=4.3; SD=0.6) (Kruskal- Wallies H=10.8; p=0.005). Similarly, the perceived usefulness of the services of advice on possible activities in the local environment and assistance in exercising rights is higher in Dravograd (AM=4.7; SD=0.6) and lower in Celje (AM=4.1; SD=0.8). In Krško, there were not enough units of analysis for comparison (see Table 4). The service of motivation in learning to live independently is 175 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 4: Perceived usefulness of the new services in pilot environments Celje Dravograd Krško TOTAL N AM SD N AM SD N AM SD N AM SD motivation in learning to live independently 36 4.3 0.6 10 4.8 0.4 19 4.6 0.5 65 4.5 0.6 assistance in adapting the living environment 27 4.2 0.7 6 4.7 0.5 13 4.3 0.6 46 4.3 0.7 maintaining and improving motor 62 4.5 0.7 28 4.7 0.5 13 4.8 0.4 103 4.5 0.7 independence psychosocial support 31 4.2 0.7 8 4.3 0.9 16 4.3 0.6 55 4.3 0.7 advice on possible activities in the user’s environment or 29 4.1 0.8 3 4.7 0.6 19 4.6 0.7 51 4.3 0.8 assistance in exercising rights Note: N... number of units of analysis; AM... asymmetric mean; SD... standard deviation EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 176 perceived as more useful by users in the higher forms (less on institutional care), Celje intensively long-term care categories (categories 3, 4, or 5) develops both types of care, and the Koroška pilot (Mann-Whitney U test Z=-1.287; p=0.022). There environment achieves average results in both were no other statistically significant differences forms but is advancing rapidly in the development by gender, age, environment or eligibility category. of both. The analyses show that users report the However, the municipalities involved in greatest positive change as a result of receiving the project face similar challenges as the others, the new services in the areas of quality of life especially in the area of community services. (AM=+0.7) and independence (AM=+0.7). The other The Institute of Macroeconomic Analysis and two aspects (sense of control over life and self- Development (Bratuž Ferk et al., 2021) notes that esteem) show less pronounced, but still positive, access to the long-term care services has been changes. Only four users report negative changes deteriorating for a number of years for a variety (mostly related to a decrease in independence). of reasons, that formal long-term care services at There were no other statistically significant home are significantly less developed in Slovenia differences by gender, age, environment or than in other EU countries, and that among people eligibility category. over 50 years of age in Slovenia, as many as 5% have unmet needs for long-term care services. A total of 549 users received services from Discussion with key the project, receiving a total of 100,028 services messages from the basic daily tasks, supportive daily tasks, nursing care and new services to maintain The initial analysis of the situation, prepared independence in 37,182 visits. Of the total, 3,122 as part of the evaluation, showed that all three services were recorded as newly added services, pilot project environments at their starting which were recorded only in the Krško pilot point were more successful in developing environment. The pilot projects highlighted the community services than the national average. challenge of urgently determining the adequacy This is important, as it is easier to organise such of the proposed set of services – i.e. whether it innovations when the starting point is better, the covers all the necessary services and, if not, which processes more established and the providers services should be added. It is evident that in more coordinated. In our initial analysis of the Celje a much higher proportion of users received environments, we found that Celje and Krško in basic and supportive daily tasks services, while in particular had a good foundation for further work Krško a much higher proportion received nursing and development, or that they sailed into the services, which is related to the initial potentials pilot project with very good predispositions and and the organisation of the local environment. The developed community services better than the municipal provider of home-based family support, vast majority of other Slovenian municipalities. Dom ob Savinji Celje, was not a project partner and In Dravograd, too, they did not lag far behind. several of these services had to be provided by the The data shows that they have intensively started project providers themselves. This partly explains to develop home help in recent years and have the higher frequency of these services compared increased the number of users by as much as to the other two environments, which are also 77% since 2009, the price of the service being providers of assistance at home themselves. In low compared to the national average. If we contrast, in the context of nursing care services, connect these indicators with the development they were less frequently provided in Celje, where of institutional care, we find that the Krško pilot the lead partner is a health care centre, and more environment is primarily focused on community frequently in Dravograd and Krško. 177 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE New services to maintain independence constitute the second biggest challenge faced by were provided by physiotherapists, occupational older people, after a poor financial situation. It therapists, social carers and masters of kinesiology. therefore makes sense to think carefully about how There was a very high level of satisfaction and to help and involve people who live alone and have recognition of the usefulness of the team’s work weak social or support networks. and the delivery of all services, and the effects The results therefore show that new services included a perceived increase in quality of life to maintain independence should be expanded, and independence among users. Contrary to some strengthened, and funded in a sustainable way, calls from professionals for the new services to as a matter of urgency and sense. It should be be strengthened mainly in the lower categories added that the pilot environments have not, of care due to their preventive nature, the data however, been able to respond to all the long- show higher user satisfaction in categories 3, 4 term care needs in the local environment. The and 5. This suggests that the implementation of public tender limited the number of hours such services makes sense irrespective of the available to users, which meant that the number long-term care eligibility category in which users of hours of care received was relatively low. The are classified. Similar activities were carried out low number of hours cannot and should not be some time ago by the Zavod za oskrbo na domu used as a basis for concluding that the average Ljubljana and its partners in the framework needs of users are (relatively) low; in addition of the Active and Quality Ageing in the Home to this limitation, many of them received other Environment (A-Qu-A) project, with comparable services in the existing system in addition to the results. The A-Qu-A project has also shown that project activities mentioned above, which are not the needs are high and that such services are considered in the calculation. The pilot projects effective; for example, marked improvements in have therefore highlighted a high level of need health status, increased mobility and motivation for long-term care services, but at the same time of users to exercise regularly, higher quality the pilot activities have not been able to fully of life, positive long-term impact on patients’ meet this need. However, the pilot projects have independence, and faster reintegration into the taken an important step towards equalising the community have been reported (Štepic, 2016). In rights of users at home and in institutional care addition to the challenges with movement, the by reinforcing existing services and, in particular, long-term care pilot projects have also importantly by extending the range of community services to addressed the hardships associated with loneliness include physiotherapy, occupational therapy, social of older people. A survey in Domžale (Nagode et al., work and kinesiology. 2021) showed that loneliness and social isolation EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 178 BIBLIOGRAPHY Beattie, P., Turner, C., Dowda, M., Michener, L., & Nelson, R. (2005). Nagode, M., Zver, E., Marn, S., Jacović, A., & Dominkuš, D. (2014). The MedRisk Instrument for Measuring Patient Satisfaction With Dolgotrajna oskrba – uporaba mednarodne definicije v Sloveniji. Physical Therapy Care: a psychometric analysis. The Journal of Urad RS za makroekonomske analize in razvoj. Retrieved from Orthopaedic and Sports Physical Therapy, 35(1), 24–32. https:/ www.umar.gov.si/fileadmin/user_upload/publikacije/ dz/2014/DZ_02_14.pdf Bratuž Ferk, B., Čelebič, T., Gregorčič, M., Kajzer, A., Kokalj, A., Koprivnikar Šušteršič, M., … Zver, E. (2021). Evropski steber Nagode, M., Kovač, N., Lebar, L., & Rafaelič, A. (2019). Razvoj socialne socialnih pravic, Slovenija 2000-2020. Ljubljana: Urad RS za oskrbe na domu: od besed k dejanjem. Ljubljana: Fakultetata za makroekonomske analize in razvoj. socialno delo. Day, H., & Jutai, J. (1996). Measuring the psychosocial impact of Nagode, M., Črnak Meglič, A., Ficko, K., Lebar, L., Pirec, S., & Rafaelič, assistive devices: The PIADS. Canadian Journal of Rehabilitation, A. (2021). Dolgotrajna oskrba: potrebe in izzivi v Domžalah. 9(3), 159–168. Ljubljana: Inštitut RS za socialno varstvo. Džananovič Zavrl, D. (2019). Delovanje patronažnega Perko, L. (2016.). Integrirana oskrba na domu. V L. Batič, R. Pišot, zdravstvenega varstva Slovenije v letih 2014 do 2019 - kadri in M. Tomšič, M. Sedmak, J. Hrovatin, D. Maraž, & M. Preglau (ed.), odziv stroke na demografske spremembe. National Institute of Aktivno in kvalitetno staranje v domačem okolju (A-Qu-A). Public Health. Retrieved from https:/ www.nijz.si/files/publikacije- Ljubljana: Solos. datoteke/zaposleni_v_vzd_2019_koncna.pdf WHO. (2014). The Case for Investing in Public Health – A Public Flaker, V., Nagode, M., Rafaelič, A., & Udovič, N. (2011). Nastajanje Health Summary Report for EPHO 8. Retrieved from http:/ www. dolgotrajne oskrbe: ljudje in procesi - eksperiment in sistem. euro.who.int/__data/assets/pdf_file/0009/278073/Case-Investing- Ljubljana: Fakulteta za socialno delo. Public-Health.pdf Flaker, V., Mali, J., Rafaelič, A., & Ratajc, S. (2013). Osebno načrtovanje Vlada Republike Slovenije. (2021). Predlog zakona o dolgotrajni in izvajanje storitev. Ljubljana: Fakulteta za socialno delo. oskrbi. (2020).. Retrieved from https:/ e-uprava.gov.si/drzava-in- druzba/e-demokracija/predlogi-predpisov/predlog-predpisa. Hsieh, Y.-J., & Lenker, J. A. (2006). The Psychosocial Impact of html?id=7885 Assistive Devices Scale (PIADS): Translation and psychometric evaluation of a Chinese (Taiwanese) version. Disability And Vlada Republike Slovenije. (2021). Predlog zakona o dolgotrajni Rehabilitation. Assistive Technology, 1(1–2), 49–57. oskrbi. Retrieved from https:/ www.gov.si/assets/ministrstva/MZ/ DOKUMENTI/Dolgotrajna/ZDO_parafa.pdf Kovač, N., Orehek, Š., & Černič, M. (2020). Analiza izvajanja pomoči na domu v letu 2019. Retrieved from https:/ www.irssv.si/upload2/ Resolucija o nacionalnem planu zdravstvenega varstva 2016–2025 Analiza%20izvajanja%20PND%20za%20leto%202019_30.9.2020_ »Skupaj za družbo zdravja« (ReNPZV16–25). Retrieved from http:/ pop2.pdf pisrs.si/Pis.web/pregledPredpisa?id=RESO102 Ilinca, S., Leichsenring, K., & Rodrigues, R. (2015). From care in Štepic, M. (2016). Ko celostni pristop pomembno pripomore k homes to care at home: European experiences with (de) izboljšanju življenja uporabnika na domu. V L. Batič, R. Pišot, institutionalisation in long-term care. European Centre, policy M. Tomšič, M. Sedmak, J. Hrovatin, D. Maraž, & M. Preglau (ed.), brief December 2015. Retrieved from https:/ www.euro.centre.org/ Aktivno in kvalitetno staranje v domačem okolju (A-Qu-A). downloads/detail/ Ljubljana: Solos. Jutai, J., & Day, H. (2002). Psychosocial impact of assistive devices Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih scale (PIADS) . Technology and Disability, 14(3), 107–111. doi:10.3233/ projektov, ki bodo podpirali prehod v izvajanje sistemskega tad-2002-14305 zakona o dolgotrajni oskrbi«. Retrieved from https:/ www. uradni-list.si/glasilo-uradni-list-rs/vsebina/2018002400003/ Jutai, J., Day, H., Coulson, S., Demers, L., Fuhrer, M.J., Lenker, J., & javni-razpis-za-izbor-operacij-izvedba-pilotnih-projektov-ki-bodo- DeRuyter, F. (2007). Developing a short form of the Psychosocial podpirali-prehod-v-izvajanje-sistemskega-zakona-o-dolgotrajni- Impact of Assistive Devices Scale (PIADS). RESNA Annual oskrbi-st--303-1201816-ob-198618 Conference, Phoenix, AZ, June 15-19. 179 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE PERCEPTION OF THE USE OF ASSISTIVE TECHNOLOGIES Lea Lebar University of Ljubljana, Faculty of Social Sciences Simona Hvalič Touzery University of Ljubljana, Faculty of Social Sciences Izidor Natek University of Ljubljana, Faculty of Social Sciences Vesna Dolničar University of Ljubljana, Faculty of Social Sciences EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 180 PERCEPTION OF THE USE OF ASSISTIVE TECHNOLOGIES KEY MESSAGES ▶ We estimate that the proportion of users living at home who have chosen to use e-care services is rather high (16.9%), suggesting that there is an interest in this form of care. Based on combinations of various measurement instruments, we find that both e-care services users and informal carers are very satisfied and acknowledge the benefits of these services. We also establish the desire among users for the further use of e-care services on the assumption that they will be co-financed. ▶ Although Slovenia is currently lagging behind in this area, the data shows that the areas of e-care and e-health need to be systematically developed. The importance of assistive technologies has increased due to the Covid-19 epidemic. ▶ Due to all the positive effects reported in relation to assistive technologies, we appeal to policy-makers to make these technologies affordable and accessible, particularly for vulnerable groups, for example by enabling co-financing or exemption from payment, to enable people to remain in their home environment longer and reduce other potential forms of inequality among the older population. 181 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Introduction Supan, 2019). Technology-supported care services, with one exception, are not nationally available Innovation in long-term care is increasingly and are not included in the national health or focused on technology-supported care services social care system as part of formal health and (or “assistive technologies”) as a key component social care services in Slovenia (Dolničar & Nagode, of the integration of health and social care. 2010; Nagode & Dolničar, 2010; Dolničar et al., Assistive technologies (AT) is the collective term 2018a). In the 2017 SHARE international survey, for information and communication technology informal carers reported on the use of a personal (ICT)-based systems designed to support recipients alarm for the persons they care for. Among 2,243 of long-term care services at home and their informal carers, only seven (0.3%) reported that informal carers. They include e-care and e-health the persons they care for use a personal alarm. services. E-care and e-health enable the provision However, a more recent Slovenian survey in 2021 of health and social care services at a distance, in among informal carers aged 40+ who care for an users’ homes71. Modern e-care support systems72 elderly relative (n=612) showed that e-care services refer to a range of smart technologies that are designed to monitor activities remotely (e.g. connected with 24-hour accessible services. These personal emergency alarm, automatic fall detector, include personal alarms (a small device triggered motion sensors, and GPS positioning system) are upon adverse event – the need for assistance), used by 4.9% of carers and their elderly relatives. environmental sensors (e.g. gas leaks, smoke), An additional 2.5% have used these services in the mobility-related devices (e.g. fall detector, motion past (Dolničar, Hvalič Touzery, Trkman, Berzelak, & sensors), and a GPS system for positioning or Bartol, 2021). monitoring movements. They can be connected Various factors inhibit the widespread use to an assistance centre that provides a 24-hour of assistive technologies (Dolničar & Nagode, response to any alarms, or to the mobile phone 2010; Nagode & Dolničar, 2010; Petrovčič, Peek, of an (in)formal carer (Cook et al., 2018). E-health & Dolničar, 2019). One of the key ones is the refers to the exchange of physiological data limited usability of assistive technologies, which between a patient at home and medical staff at a is linked to the lack of systemic involvement of distance in order to facilitate the diagnosis and end-users in their design and development. The monitoring of the disease (Goodwin, 2010). In this usability of such technologies has been identified paper, we distinguish between two types of e-health in the literature as a critical aspect of the end-user services: vital functions monitoring services (in experience (Vermeulen et al., 2013). Additional which monitoring is implemented by trained staff restrictions include: 1. insufficient information at an assistance centre) and telemedicine treatment for potential users and the general public about (in which monitoring is implemented by medical the existence and functionalities of e-care and staff employed by a hospital)73. e-health services; 2. poor understanding of the Slovenia still lags far behind many European needs, fears, wishes, abilities and circumstances countries in the adoption of assistive technologies of potential older users; 3. insufficient exploitation (Börsch-Supan, 2019). In Slovenia, less than 1% of of the existing knowledge; 4. lack of cooperation older people use a personal alarm (Kubitschke & between key stakeholders, operational plans Cullen, 2010; Börsch-Supan, 2019), which places for service implementation, user-friendly us at the bottom of the list of European countries design of assistive technologies, integration of together with Poland, Croatia and Greece (Börsch- services and technological solutions, strategic 71 In this chapter, we use the term user to refer to the primary e-care users. 72 The e-care services tested by users in the pilot projects include the Basic and Premium packages (Telekom Slovenije d. d.) and the In life smartwatch (Jožef Stefan Institute). 73 Vital functions monitoring services were provided by Telekom Slovenije d. d. and telemedicine treatment by MKS Electronic Systems d. o. o. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 182 planning, interdisciplinary and interdepartmental (Dolničar, Müller, & Santi, 2011; Tsertsidis et al., cooperation and integration, business models 2019; Prevodnik, Hvalič Touzery, Dolničar, Laznik, and funding models (Dolničar and Nagode, 2010). & Petrovčič, 2021). Research findings also show that Slovenia has taken a step forward by adopting the assistive technologies promote independence (Peek Active Ageing Strategy (Government of the Republic et al., 2014; Tsertsidis et al., 2019; Jaschinski & Ben of Slovenia, Ministry of Labour, Family, Social Allouch, 2019; Hvalič Touzery et al., 2021) and social Affairs and Equal Opportunities and Institute of interaction and reduce the social isolation of older Macroeconomic Analysis and Development, 2017), people (Tsertsidis et al., 2019; Verloo, Kampel, Vidal, which recognises the positive impacts of assistive & Pereira, 2020) and can contribute to different technologies on a long-lived society, but there dimensions of their quality of life (Dolničar et al., have been too few concrete steps to support such 2011; Dolničar, Petrovčič, Šetinc, Košir, & Kavčič, technology. For example, the results of a recent 2017; Verloo et al., 2020). online survey to assess the situation in smart health The effects of assistive technologies are also and care, conducted among stakeholders (n=544) evident among informal carers, as they can reduce in nine European regions as part of the ITHACA the burden, stress and anxiety of informal carers of project, show that among the nine regions analysed, older people (Andersson, Erlingsson, Magnusson, & Slovenia is the region with the weakest services Hanson, 2017; D’Onofrio et al., 2017; Dolničar et al., supporting innovation and the weakest policy 2017; Smole-Orehek et al., 2019; Hvalič Touzery et framework in this area (Dolničar et al., 2018a). A al., 2020a) and at the same time increase their well- key finding of the national meeting organised after being and peace of mind (Andersson et al., 2017; the survey, attended by stakeholders from business, Smole-Orehek et al., 2019; Hvalič Touzery, Lebar, politics, research and the civil society, was that Petrovčič, Smole-Orehek, & Dolničar, 2020a). politics is a key barrier to the development of new The research conducted in the context of the ideas and the development and use of innovative long-term care pilot projects builds on existing products. Two additional barriers are the reluctance knowledge in this area in a meaningful way. of older people to use ICT and the lack of financial Although we have monitored e-care and e-health support. Stakeholders mentioned that there are services in the pilot projects, in the present paper no clearly defined standards and norms related we focus mainly on the former. We only briefly to assistive technologies; that smart health and introduce e-health services. care is not a political priority; that cooperation In this paper, we present the results of between different stakeholders is poor; and that a survey, complemented by semi-structured there is a lack of financial support. They also failed interviews among users of e-care services. We to recognise the major benefits of being involved in analyse the assessment of overall satisfaction, innovation partnerships (Dolničar et al., 2018b). perceived usefulness and ease of use, and the Researchers agree (Goodwin, 2010; Sanders et impact of using e-care services. In this paper we al., 2012; Melchiorre et al., 2018) that there is a lack present mainly the results for the target group of studies that would systematically analyse and of users, but we also touch on the results among empirically test the benefits of assistive technologies informal carers. for end-users. However, existing research finds that assistive technologies can increase the sense of security (Peek et al., 2014; 2016; Tsertsidis, Methodology Kolkowska, & Hedstrom, 2019; Jaschinski & Ben Allouch, 2019; Hvalič Touzery, Smole-Orehek, The evaluation monitored the implementation & Dolničar, 2021) and freedom, and increase of assistive technologies in the pilot environments, awareness of the condition and signs of illness focusing on the overall satisfaction of the long- 183 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE term care users with e-care and their assessment At the time of the survey, users had been involved of the ease and usefulness of use. During the in the e-care service for between 58 days and evaluation, we monitored three indicators, namely 526 days, with an average of 262 days. A separate the proportion of beneficiaries who lived in the paper survey was also conducted among informal community and used assistive technologies, the carers. In this paper, we refer to these results only number of e-care interventions and the number of exceptionally. 44 informal carers (31.4%) completed e-health interventions. the questionnaire. The survey design was broad: on a monthly In order to obtain an understanding of the basis we collected data on e-care and e-health experiences of e-care users and their informal which was anonymised by service providers and carers, we also conducted seven semi-structured developed questionnaires for e-care and e-health interviews with e-care users and nine with users and their informal carers. At the same time, informal carers. As the scope of the study went semi-structured interviews were conducted with a beyond interviewing all users of the different types sample of users and informal carers in one of the of assistive technologies, we wanted to describe pilot environments. only the experiences of those who had tested In the pilot environments, data on the combination of the basic Telekom Slovenije assistive technologies were reported monthly in package and the fall detector. We decided to do collaboration with assistive technology service this based on a review of personal plans and providers. Reporting started when the first user focus groups among EVT staff and long-term was involved or when the evaluators agreed with care coordinators, in which the main focus was the pilot environments on how to monitor service on the advantage of a quick response to a fall. delivery. Data for the previous month was reported All the users involved were from the Krško pilot in Krško from February 2019, in Celje from July environment, where they suggested candidates for 2019, and in Dravograd from October 2019 (e-care) interview in accordance with our guidelines. The or November 2019 (smartwatch and telemedicine). interview cues included the following key themes: The environments sent the reports until the end use and experience, change, e-care in the context of June 2020 (Dravograd) and August 2020 (Celje of the Covid-19 epidemic, recommendations, and and Krško). The reports included the user code, (for informal carers only) the burden of providing start and end date of use, details of the service care. Due to the complexity of the telephone and equipment received, reasons for any non- interviews and the specificities of the population involvement or early termination, and information involved (the older people found it difficult to talk on interventions. We further distinguished between for long periods on the phone, they were less able users of assistive technologies according to their to hear and sometimes understand), the interviews activity, considering users as “active” if they had were shortened slightly and lasted between 18 and used the assistive technologies for at least 25 days. 43 minutes (27 minutes on average). We surveyed e-care users at one point in In the long-term care pilot project, a total of time, with the majority of users being surveyed 152 users tested assistive technologies, among them between May and August 2020. We used a face-to- 131 tested only e-care services, 9 e-care and e-health face interviewing method in which the evaluator services and 12 telemedicine support services. This recorded the answers in an online platform (on a represents 16.9% of all long-term care beneficiaries mobile phone). The survey was short, lasting on in the pilot projects who were assessed at home. average less than five minutes (excluding three In the first part we present the sample of e-care respondents who stopped the questionnaire for a users, followed by a brief presentation of e-health long period of time), with half of the respondents users. Due to the low number of units of analysis, all completing the survey in less than four minutes. further analyses are prepared for e-care users only. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 184 Presentation of a sample services. Due to the low number of completed of e-care users and the questionnaires (N=14), we do not present the results intervention process in detail. However, we dedicate a specific part of the discussion to users who have been involved Information on experiences with e-care was in e-health services, which we consider relevant obtained from 79 users (56.4% of all active users especially in the context of the Covid-19 epidemic. from all three pilot environments), specifically A small number of users (21 in total) used 31 users from the Celje pilot environment, 35 e-health services in the pilot projects, specifically users from Krško and 13 users from Dravograd. 12 users of telemedicine support in Dravograd and The majority of users are female (70.9%), with an nine users of home-based vital functions monitoring average age of 82.7 years (SD74=10.2) (see Table 1). equipment in Celje or Krško. The latter nine were The majority of users (92.4%) used the E-care also included in the e-care services. In contrast to service (58.2% Basic package and 34.2% Premium e-care users, e-health users are predominantly male package), while the remaining 7.6% used the In (57.1%). The average age of users was 80.4 years Life smartwatch (Dravograd pilot environment). (SD=7.0), which means that e-health users were on Of these, about two thirds of users (65.8%) used average younger than e-care users. The proportion assistive technologies for more than six months of more educated users was also higher, with 42.1% and less than one year, 24.1% for more than one having a vocational or secondary education and just year, 8.9% for up to three months and only one under a third having a short-cycle higher education user for less than three months. 83.8% of the or higher, while only 26.3% of users had primary users answered the questionnaire themselves, education. The statistics obtained from the service and for the remaining either an informal carer providers show that e-health users used the service or another relative answered the questionnaire. very frequently. They are therefore active users. Users were active in their use of e-care. In the During the project period, the telemedicine period between January and August 2020, when a support users performed a total of 6,621 total of 140 active users were involved in all three measurements (7.4% additional measurements) pilot environments, providers report the following out of the planned 5,791 measurements. Persons assistance was provided: with heart failure (10 persons) took 5,970 ▷ 775 alarms triggered (40.5% from Celje, 52.5% measurements (6.3% more) out of the planned from Krško and 7.0% from Dravograd); 5,614 measurements, and two persons with type 2 ▷ 4024 alerts from automatic detection (98.6% from diabetes took 251 measurements (41.8% more) out Celje, 1.0% from Krško and 0.5% from Dravograd); of the planned 177 measurements. In total: ▷ 785 interventions (80.6% from Celje, 17.1% from ▷ 61 telemedicine centre interventions (55 for Krško and 2.3% from Dravograd); people with heart failure and 6 for people with type ▷ 316 social calls (39.2% from Celje, 50.6% from 2 diabetes) were implemented. In total, 5 home Krško and 10.1% from Dravograd). visits, 30 CEZAR calls to users and 3 calls by the user to the centre were made as a result of telemedicine monitoring. 23 general observations were also Presentation of a sample recorded. of e-health users and the ▷ 28 interventions by a specialist doctor (all for intervention process people with heart failure) were made. The specialist doctor advised users 16 times, in 12 cases changing In this section, we briefly mention the their therapy. descriptive results regarding the use of e-health ▷ 18 technical support interventions (14 for people 74 Standard deviation. 185 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 1: Sociodemographic characteristics of e-care users by pilot environment N % Celje 31 39.2 Pilot environment Krško 35 44.3 Dravograd 13 16.5 Male 23 29.1 Gender Female 56 70.9 < 65 years 3 3.8 Age ≥ 65 ≥ 80 years 26 32.9 > 80 years 50 63.3 primary school or below 36 48.0 Education vocational or secondary education 34 45.3 short-cycle higher education or higher 5 6.6 married 17 21.5 divorced 4 5.1 Marital status widowed 48 60.8 single 9 11.4 common-law partnership 1 1.3 1 53 67.9 Number of household members ≥ 2 25 32.1 Up to 500 EUR 15 21.7 Net income > 500 ≤ 750 EUR 27 39.1 > 750 27 39.1 186 Table 2: Overview of the providers and the name of the assistive technologies availabl in the pilot environments with heart failure and 4 for people with type 2 diabetes) were implemented. Introducing e-care and e-health services E-care service (Basic package) Provider Telekom Slovenije The e-care services tested by users in the pilot environments include various packages offered by Telekom Slovenije and the In life smartwatch developed by the Jožef Stefan Institute. In the pilot environments, different services were offered (see tabel 2): in all three environments, the Basic E-care E-care service (Premium package) Provider Telekom Slovenije package by Telekom Slovenije, in Celje and Krško also the Premium E-care package by Telekom Slovenije and the vital functions monitoring package, and in Dravograd the In life smartwatch developed by Jožef Stefan Institute and the telemedicine support from MKS Elektronski home vital functions monitoring sistemi d. o. o. package As can be seen from the tabel 2, Telekom Provider Telekom Slovenije Slovenije offers three different packages – E-care (Basic package), E-care (Premium package) and home vital functions monitoring. The E-care basic package provides a 24-hour availability of the assistance centre and an immediate call for help by pressing a button on the pendant or on the additional equipment Provider Telekom Slovenije protection unit. The assistance is organised by the medical staff at the assistance centre. If the assistance centre is unable to reach the user, the informal carer is contacted, or help is organised through the appropriate intervention service (ambulance, fire brigade or police). The E-care In life smartwatch Premium package additionally allows alarms to be Provider A.L.P. Peca (developer Jožef Stefan triggered automatically when the user is unable Institute) to initiate a call for help due to an emergency (e.g. fall, nausea, etc.). The package also includes five motion sensors and two magnetic sensors. The system automatically triggers certain notifications that are forwarded to the contact persons and the assistance centre through the E-care app. The Telemedicine support Provider MKS Elektronski sistemi d. o. o. assistance services in all three pilot environments are provided by Telekom Slovenije’s contractual partner, Doktor 24 d. o. o. The condition for the use of E-care is the involvement of one to three 187 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE contact persons (e.g. informal carers). If the user’s The measurement equipment used – blood needs change (e.g. the need for more support in pressure monitor, weighing scale, blood sugar independent living), the E-care service can be monitor, information and telecommunication upgraded with additional equipment: a remote infrastructure and clinical portal – came from the trigger, a waterproof fall detector, environmental Centre for Telehealth (CEZAR Centre) (Rudel, 2020). sensors and detectors in the living environment A person with a chronic disease has telemedicine (a gas detector, a smoke detector, a temperature monitoring equipment installed at home (meters detector and a water leakage detector, a magnetic with a mobile phone). The patient measures sensor that detects the opening and closing of the vital functions with the devices as instructed by refrigerator door, and the automatic switching on the specialist (the target group of patients with of the lights when there is movement) (Telekom heart failure measuring blood pressure, heart Slovenije, d. d.).75 Telekom Slovenije also enabled rate and body weight, and patients with type 2 the monitoring of vital functions in the users’ diabetes measuring blood sugar). The meters homes as part of pilot projects. Users could receive automatically transmit the data through a wireless a blood pressure monitor, a blood sugar monitor Bluetooth connection to the mobile phone which and a weighing scale. immediately forwards the data to the telemedicine The In life smartwatch76 was also used in the centre (CEZAR Centre at Slovenj Gradec General Dravograd pilot environment. A.L.P. Peca provided Hospital) where the data is processed. If the technical support to users. The smartwatch enables data exceeds personally defined thresholds, an automatic call for help in the event of a fall, this is brought to the attention of the healthcare call for help using a special button, locating the professional, a coordinator in the telemedicine user in the event of danger, measuring heart rate centre. The telemedicine coordinator contacts the and sending reminders. The user could measure patient by telephone to confirm the authenticity of their heart rate using the watch, and the watch the measured results (Rudel, 2020). also allowed remote heart rate measurement through an app. The watch allows the setting up of a so-called virtual fence which is controlled Results by the contact person using a mobile app. If the user crosses the virtual fence, the contact person In this chapter we analyse (1) the perceived receives an SMS notification including a link to the ease of use, (2) satisfaction and (3) the usefulness Google map and the coordinates where the user is of using e-services. Next, we evaluate (4) the located. For this function to work properly, a GPS perceived psychosocial effects of the use of e-care signal on the user’s side and an internet connection services in the group of users involved in the long- on the contact person’s side are required. The term care pilot projects. smartwatch is managed by the contact person For perceived usefulness, we tested agreement through a mobile application. with the statement “Overall, I found the e-care In the Dravograd pilot environment, a service easy to use” , which users rated on a telemedicine provider (MKS elektronski sistemi d. scale from 1 (do not agree at all) to 5 (strongly o. o.) was selected to support people with chronic agree). The majority of users (80.8%) agreed or diseases in self-care at home (more specifically, strongly agreed that e-care services are easy to patients with heart failure and/or type 2 diabetes). use (AM77=4.1; SD=0.978), with a slightly higher 75 More detailed information on the packages is available on Telekom Slovenije's website www.telekom.si. 76 The following presentation of the functionality is based on the information received in the focus group and the personal interview. See also https:/ www.telekom.si/en/about-us/company/press-releases/The_E_oskrba_service_for_safe_living 77 Aritmetic Mean. 78 Standard deviation. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 188 prevalence of those aged under 80 (AM=4.3; using the e-care services very positively (AM=4.3; SD=0.6) compared to those aged 80 and over SD=0.8). In reasoning, they placed the greatest (AM=4.0; SD=0.9)79, and of males compared to emphasis on a quick response to a fall and on females (AM=4.2; SD=0.8 and AM=4.0; SD=0.9, situations where the person is alone. Slightly respectively), although the differences are not higher usefulness was identified by men compared statistically significant80. No differences were to women (AM=4.5; SD=0.7 and AM=4.2; SD=0.9 found in the eligibility category for long-term care. respectively)83 and by users in the long-term care Informal carers were also involved to varying categories 3, 4 or 5 (compared to categories 1 or degrees in helping with care; as indicated in the 2)84, with no differences according to age. questionnaire, just under a third (30.7%) were In the questions related to the experience involved on a regular basis (at least several times of using the e-care services during the Covid-19 a week), while the remainder were involved less epidemic, we find that the majority of users (more frequently. A quarter did not engage in care at all, than 90%) did not change their opinion on the and a further 35.9% engaged in care once or less frequency and usefulness of using e-care services than once a month. The majority of users (76.6%) during the epidemic. However, we observe some also consider that using the service is worth the changes in the opinion and usefulness of e-care effort they put into it. services during the epidemic among informal To assess overall satisfaction, we asked users carers. Among them, 16.7% report that they have to rate on a scale from 1 (very dissatisfied) to 5 become more favourable to e-care services and (very satisfied) the statement “Considering all the 20.0% that e-care services are even more useful at experiences you have had with the e-care service up this time. to this point, how satisfied have you been with it?” . Based on the results of the survey, we conclude that satisfaction with e-care services is high. The survey Identified effects of e-care use results show that 75.6% of users are satisfied or very satisfied with their use of e-care (AM=4.0; To measure the effects of e-care on users, we SD=0.9). There is slightly higher satisfaction used the validated PIADS-10 scale, translated into among men (AM=4.2; SD=0.7) compared to women Slovenian ( Psychosocial Impact of Assistive Devices (AM=3.9; SD=0.9)81 and among people in the top Scale) (Day & Jutai, 1996; Hsieh & Lenker, 2006; three care categories (3, 4 or 5) compared to people Jutai & Day, 2002; Jutai et al., 2007; Hvalič Touzery, in the first two care categories (AM=4.2; SD=0.9 and Dolničar, Prevodnik, Škafar, & Petrovčič, 2020b)85. AM=3.9; SD=0.9 respectively)82, but the differences The scale takes into account the psychosocial are not statistically significant. No differences by effects of assistive technology use on specific age are observed. aspects of daily life (rated on a scale of -3 to 3)86. To assess usefulness, we asked users to rate We find that a significant proportion of e-care on a scale from 1 (very low) to 5 (very high) how users have a positive perception of the impact of useful they found the e-care service they have used e-care on their lives. These effects vary according or are still using. Users also rated the usefulness of to the area of daily life. Users report the greatest 79 A non-parametric Mann-Whitney U test was implemented, and the differences are statistically significant at the 10% level of significance (p<0.1). 80 A non-parametric Mann-Whitney U test was performed, and the differences are not statistically significant. 81 A non-parametric Mann-Whitney U test was performed, and the differences are not statistically significant. 82 A non-parametric Mann-Whitney U test was performed, and the differences are not statistically significant. 83 A non-parametric Mann-Whitney U test was performed, and the differences are not statistically significant. 84 A non-parametric Mann-Whitney U test was performed, and the differences are not statistically significant. 85 Cronbach alpha on the analysed data shows excellent scale reliability (α ≥ 0.9). 86 The scale is used to determine whether the use of assistive technology greatly decreased (-3), significantly decreased (-2), slightly decreased (-1), neither decreased nor increased (0), slightly increased (1), significantly increased (2) or greatly increased (3) the individual aspect of the user's daily life. 189 Figure 1: Impact of e-care for users in pilot environments (PIADS-10, change) (N=78) positive change in the areas of their independence (AM=+0.8; SD=0.9), sense of control over their life (AM=+0.8; SD=0.9) and overall quality of life CHANGE (AM=+0.7; SD=0.8), which are also among the key Sense of control dimensions listed, especially in the context of long- term care. In addition to these changes, more than 0,8 one third of e-care users report an increase in their sense of happiness, ability to take advantage of Independence opportunities offered, and ability to adapt to daily activities. The remaining positive effects (increased 0,8 productivity, self-esteem, eagerness to try new things and ability to participate) are reported by Quality of life between 20% and 30% of users. With the exception of one user, no one reported any negative effects 0,7 of use. In the interviews, users and informal carers mainly and most often point to the increased safety Happiness of users and, on the other hand, the consequent increased peace of mind of informal carers. The 0,6 concept of greater safety is not included in the PIADS-10 scale but is included in the extended Ability to take advantage of opportunities version of the PIADS-26 scale. 0,5 Discussion with key messages Ability to adapt to the activities of daily living A number of studies drawing on different 0,4 conceptual models – most often the TAM model: Self-esteem Technology Acceptance Model (Davis, Bagozzi, & Warshaw, 1989) – examine factors influencing the 0,4 (non-)acceptance of assistive technologies among older people (e.g. Peek et al., 2016; Jaschinski & Eagerness to try new things Ben Allouch, 2019). The decision on whether an individual will adopt a particular technology depends, among other things, on the individual’s 0,3 ability to judge whether the benefits of using the Ability to participate technology outweigh the risks associated with its use. Older people have a different perception 0,3 of the usefulness and barriers to the adoption of new technologies in health and social care than Productivity younger people, and it is crucial to consider their suggestions and opinions when further designing 0,3 assistive technologies. In this context, it is also important to monitor their satisfaction, in which -3 -2 -1 0 1 2 3 several aspects are relevant, including ease of EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 190 use, quality of service, adaptability to change, are key and most expected among the PIADS-10 perceived effects in daily life, sense of control, trust dimensions in the long-term care context. The in technology, perception of costs, responsiveness results of the qualitative study primarily show and support in use, and quality of care (Jaschinski psychological effects; participants report mainly & Ben Allouch, 2019; Verloo et al., 2020; Hvalič an increased sense of security, a concept not Touzery & Dolničar, 2021). measured by the PIADS-10 scale but included in the In this paper, we present results on overall extended version of the PIADS-26 scale. satisfaction, perceived usefulness, ease of use, and Testing of e-care and e-health services and effect, which we identify as key in the context of user surveys took place during the first wave of user acceptance and continued use of technology. the Covid-19 epidemic. As a result of the physical Assistive technologies were tested in the pilot distancing measures, we expected a change in projects by 16.9% of all those eligible and assessed attitudes about the usefulness of such services in their homes, which we consider to be a high among users and informal carers. The results of proportion. the pilot projects suggest that the testing period Concerns about the use of new technologies was too short or that users’ views on the usefulness among older people and long-term care users may of e-care services remained largely unchanged reflect a lower knowledge and less skill with using during the first wave of the epidemic. Informal new technologies (Tsertsidis et al., 2019; Hvalič carers became slightly more favourable to e-care Touzery & Dolničar, 2021). Nevertheless, Tsertsidis during this period. Despite this result, the potential et al. (2019) note that a number of studies have of using existing technologies to improve access identified the positive experience of older people to services while minimising the risk of human- with ease of use (Peek et al., 2016; Dupuy, Consel, to-human transmission should not be overlooked. & Sauzéon, 2016; Vaziri et al., 2016; Jaschinski, These potentials of e-care were better recognised Allouch, Peters, Cachucho, & Dijk, 2021; Tsertsidis by informal carers of older people in the national et al., 2019), which has also been confirmed in the survey, which was also conducted during the long-term care pilot projects. Indeed, the majority of Covid-19 epidemic. In this study, 41% of informal users (80.8%) agreed or strongly agreed that e-care carers found e-care services more useful than services were easy to use, but this was true slightly before the pandemic (Dolničar, et al., 2021). In more often for people younger than 80 years. this light, the use of technology has a number of Usability or usefulness is also an important advantages. One of them is overcoming at least factor that has a positive effect on the acceptance part of the experience of loneliness and staying of new technologies among long-term care in contact with relatives and service providers. recipients. The degree of perceived usefulness Despite a limited service provision, the pilot varies over time, according to existing research, environments intensively cooperated with service e.g. perceived usefulness becomes stronger with users via telephone. In the pilot environment of long-term use (e.g. Pino, Boulay, Jouen, & Rigaud, Celje, statistics of social contacts were recorded 2015; Dupuy et al., 2016). Analyses show that long- which show that in the period from 23 March 2020 term care users in the pilot projects were satisfied to 30 April 2020, they conducted more than 1,000 with the services and mostly perceived them as telephone conversations with users and relatives. useful. The positive effects of the use of e-care In overcoming loneliness, video calls came to the services are also an important result of the present fore, especially in institutional care, but data from research. The biggest positive change resulting the environments show that users at home did not from the use of e-care is observed by e-care users use them. In fact, only 9 users in total reported in the areas of their independence, sense of control using them, and only 12.5% had a desire to stay over their lives and overall quality of life, which in contact with their family during self-isolation 191 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE in this way (M12 data, N=96). This result is not proved to be a key manner of providing services surprising even for the first wave of the epidemic and a successful method of support to these and is an indicator of new/deeper inequalities; persons, taking place despite less accessible health many older people do not have a smartphone care institutions. All persons with chronic disease or tablet, nor the skills to handle this type of involved in telemedicine support had regular technology. As Rudel (2020) points out, the Covid-19 measurements during the epidemic, thus enabling epidemic has also increased the vulnerability of any necessary medical intervention without the adults with heart failure, who may experience risk of virus infection. The heart failure specialist more severe complications and require ongoing regularly reviewed the data and called each management to achieve the necessary blood patient at least once during the epidemic. For pressure, heart rate and blood oxygen saturation some, she also adjusted therapy during this time levels. Telemedicine treatment of already involved (Rudel, 2020). persons was uninterrupted even in this case and EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 192 BIBLIOGRAPHY Andersson, S., Erlingsson, C., Magnusson, L., & Hanson, E. (2017). living applications. Journal of Ambient Intelligence and Humanised Information and communication technology-mediated support for Computing, 10(2), 761–778. doi:10.1007/s12652-018-0856-6 working carers of older family members: An integrative literature review. International Journal of Care and Caring, 1(2), 247–273. Jaschinski, C., Allouch, S. B., Peters, O., Cachucho, R., & Dijk, J. A. G. M. van. doi:10.1332/239788217x14957228181753 (2021). Acceptance of Technologies for Aging in Place: A Conceptual Model. Journal of Medical Internet Research, 23(3), e22613. doi: Börsch-Supan, A. (2019). Survey of health, ageing and retirement in Europe 10.2196/22613 (SHARE) Wave 6. Release version: 7.0.0. Retrieved from http:/ www.share- slovenija.si/strani/6val_raziskave Jutai, J., & Day, H. (2002). Psychosocial impact of assistive devices scale (PIADS). Technology and Disability, 14(3), 107–111. doi:10.3233/tad-2002-Cook, E. J., Randhawa, G., Guppy, A., Sharp, C., Barton, G., Bateman, A., & 14305 Crawford-White, J. (2018). Exploring factors that impact the decision to use assistive telecare: Perspectives of family care-givers of older people Jutai, J., Day, H., Coulson, S., Demers, L., Fuhrer, M. J., Lenke,r J., & DeRuyter, in the United Kingdom. Ageing & Society, 38(9), 1912–1932. doi:10.1017/ F. (2007). D eveloping a short form of the Psychosocial Impact of Assistive s0144686x1700037x Devices Scale (PIADS). RESNA Annual Conference, Phoenix, AZ, June 15-19. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Kubitschke, L., & Cullen, K. (2010). ICT & ageing . European study on users, Management Science, 35(8), 982–1003. doi:10.1287/mnsc.35.8.982 markets and technologies. Final report. Retrieved from https:/ www.ifap. ru/library/book400.pdf Day, H., & Jutai, J. (1996). Measuring the psychosocial impact of assistive devices: The PIADS. Canadian Journal of Rehabilitation, 9(3), 159–168. Melchiorre, M. G., Papa, R., Rijken, M., van Ginneken, E., Hujala, A., & Barbabella, F. (2018). eHealth in integrated care programs for people Dolničar, V., Müller, S., & Santi, M. (2011). Designing technologies for older with multimorbidity in Europe: Insights from the ICARE4EU project. people: A user-driven research approach for the Soprano Project. V F. Health Policy, 122(1), 53–63. doi:10.1016/j.healthpol.2017.08.006 Colombo (ed.). Broadband society and generational changes: P. Lang, 5, 221–246. Nagode, M., & Dolničar, V. (2010). Assistive technology for older people and its potential for intergenerational cooperation: Critical review of the present Dolničar, V., & Nagode, M. (2010). Overcoming key constraints on assistive situation and identification of key constraints for wider uptake. Retrieved technology uptake in Slovenia. Teorija in praksa: revija za družbena from http:/ dk.fdv.uni-lj.si/db/pdfs/tip20106_nagode_dolnicar.pdf vprašanja, 47(6), 1295–1315. Retrieved from http:/ dk.fdv.uni-lj.si/db/pdfs/ tip20106_dolnicar_nagode.pdf Peek, S. T. M., Wouters, E. J. M., van Hoof, J., Luijkx, K. G., Boeije, H. R., & Vrijhoef, H. J. M. (2014). Factors influencing acceptance of technology Dolničar, V., Petrovčič, A., Šetinc, M., Košir, I., & Kavčič, M. (2017). for aging in place: A systematic review. International Journal of Medical Understanding acceptance factors for using e-care systems and Informatics, 83(4), 235–248. doi:10.1016/j.ijmedinf.2014.01.004 devices: Insights from a mixed-method intervention study in Slovenia. Lecture Notes in Computer Science, 362–377. doi:10.1007/978-3-319- Peek, S. T. M., Luijkx, K. G., Rijnaard, M. D., Nieboer, M. E., van der Voort, 58536-9_29 C. S., Aarts, … Wouters, E. J. M. (2016). Older adults' reasons for using technology while aging in place. Gerontology, 62(2), 226–237. Dolničar, V., Šetinc, M., Burnik, T., Hvalič Touzery, S., Petrovčič, A., Rudel, D., & doi:10.1159/000430949 Berzelak, J. (2018a). Assessing smart health and care in nine EU regions: ITHACA’s self-assessment online survey. 30 let izkušenj v podporo Petrovčič, A., Peek, S., & Dolničar, V. (2019). Predictors of seniors’ interest in digitalizaciji zdravstva. Journal: Slovensko društvo za medicinsko assistive applications on smartphones: Evidence from a population- informatiko, 120–126. Retrieved from https:/ www.researchgate.net/ based survey in Slovenia. International Journal of Environmental publication/336170445_Assessing_smart_health_and_care_in_nine_EU_ Research and Public Health, 16(9), 1623. doi:10.3390/ijerph16091623 regions_ITHACA%27s_self-assessment_online_survey_Ocena_stanja_ pametnega_zdravstva_in_oskrbe_v_devetih_EU_regijah_spletna_anketa_ Pino, M., Boulay, M., Jouen, F., & Rigaud, A.-S. (2015). “Are we ready for projekta_ITHACA_14_11_2018_Kong robots that care for us?” Attitudes and opinions of older adults toward socially assistive robots. Frontiers in Aging Neuroscience, 7. doi:10.3389/ Dolničar, V., Šetinc, M., Burnik, T., Hvalič Touzery, S., Petrovčič, A., Rudel, D., fnagi.2015.00141 & Berzelak, J. (2018b). Ocena stanja pametnega zdravstva in oskrbe v devetih EU regijah spletna anketa projekta ITHACA. Predstavljeno na Prevodnik, K., Hvalič Touzery, S., Dolničar, V., Laznik, J., & Petrovčič, A. (2021). Kongres MI’2018 – 30 let izkušenj v podporo digitalizaciji zdravstva. Zreče. Patients' experience with telemedicine in primary care: a focus group study. In: Care: challenges & solutions for a sustainable future: 12th - 30th Dolničar, V., Hvalič Touzery, S., Trkman, M., Berzelak, N., & Bartol, J. (2021). April 2021: conference booklet. Sheffield: The University. 82-83. Retrieved Pomoč starejšim osebam s strani družinskih članov ali sorodnikov from http:/ circle.group.shef.ac.uk/wp-content/uploads/2021/04/ in njihov odnos do novih tehnologij: Sumarni pregled frekvenčnih Conference-booklet_April-2021-2.pdf porazdelitev in opisnih statistik zbranih podatkov (raziskovalno poročilo na ARRS projektu J5-1785). Ljubljana: Fakulteta za družbene vede. Rudel, D. (2020). Telemedicinska storitev v integrirani oskrbi kroničnih bolnikov v domačem okolju (vsebinsko poročilo za projekt PPDO št. pog. Dupuy, L., Consel, C., & Sauzéon, H. (2016). Self determination-based design PPDO-03-19/JNMV-TM/02). Ljubljana: MKS elektronski sistemi d. o. o. to achieve acceptance of assisted living technologies for older adults. Computers in Human Behaviour, 65, 508–521. doi:10.1016/j.chb.2016.07.042 Sanders, C., Rogers, A., Bowen, R., Bower, P., Hirani, S., Cartwright, M., … Newman, S. P. (2012). Exploring barriers to participation and adoption of Goodwin, N. (2010). The state of telehealth and telecare in the UK: telehealth and telecare within the Whole System Demonstrator trial: a Prospects for integrated care. Journal of Integrated Care, 18(6). qualitative study. BMC Health Services Research, 12(1). doi:10.1186/1472- doi:10.5042/jic.2010.0646 6963-12-220 Hsieh, Y.-J., & Lenker, J. A. (2006). The psychosocial impact of assistive Smole-Orehek, K., Hvalič-Touzery, S., Petrovčič, A., Dolničar, V., Debevc, devices scale (PIADS): Translation and psychometric evaluation of a M., & Kožuh, I. (2019). Psychological outcomes of eCare technologies Chinese (Taiwanese) version. Disability and Rehabilitation: Assistive use for informal carers: A scoping study. Gerontechnology, 18(1), 15–28. Technology, 1(1-2), 49–57. doi:10.1080/09638280500167217 doi:10.4017/gt.2019.18.1.002.0 Hvalič Touzery, S. Lebar, L., Petrovčič, A., Smole-Orehek, K., & Dolničar, V. Telekom Slovenije (b. d.). E-oskrba. Retrieved from https:/ www.telekom.si/ (2020a). Psychological outcomes of telecare use for working family zasebni-uporabniki/ponudba/e-oskrba carers of older people. Proceedings of Smart Living Forum 2019, 26–33. Villach: Norderstedt: BoD. Retrieved from https:/ www.researchgate.net/ Tsertsidis, A., Kolkowska, E., & Hedström, K. (2019). Factors influencing publication/341372241_ Psychological_Outcomes_of_Telecare_use_for_ seniors’ acceptance of technology for ageing in place in the post- Working_Family_Carers_of_Older_People implementation stage: A literature review. International Journal of Medical Informatics, 129, 324–333. doi:10.1016/j.ijmedinf.2019.06.027 Hvalič Touzery, S., Dolničar, V., Prevodnik, K., Škafar, M., & Petrovčič, A. (2020b). The importance of patients' user experience with a telehealth Vaziri, D. D., Aal, K., Ogonowski, C., Von Rekowski, T., Kroll, M., Marston, H. R., system for their evaluation of its psychosocial impacts. In: Lattacher, … Wulf, V. (2016). Exploring user experience and technology acceptance S.L. (ed.), Krainer, D. (ed.). Proceedings of Smart Living Forum 2019, 14 for a fall prevention system: results from a randomised clinical trial November 2019, Villach, Austria. Norderstedt: BoD. 19-25. and a living lab. European Review of Aging and Physical Activity, 13(1). doi:10.1186/s11556-016-0165-z Hvalič Touzery, S., Smole-Orehek, K., & Dolničar, V. (2021). Exploring reciprocity in perceptions on telecare within the informal carer-care receiver dyad. Verloo, H., Kampel, T., Vidal, N., & Pereira, F. (2020). Perceptions About Teorija in praksa, 58(3), 840–859. doi: 10.51936/tip.58.3.840-859 Technologies That Help Community-Dwelling Older Adults Remain at Home: Qualitative Study. Journal of Medical Internet Research, 22(6). Hvalič Touzery, S., & Dolničar, V. (2021). Odnos do podpornih tehnologij med doi: 10.2196/17930 starejšimi osebami in neformalnimi oskrbovalci v Sloveniji = Attitudes towards smart technologies among older people and their informal Vermeulen, J., Neyens, J., Spreeuwenberg, van Rossum, E., Sipers, Habets, … carers in Slovenia. In: Petelin, A. (ed.). Zdravje starostnikov : 5. znanstvena de Witte. (2013). User-centred development and testing of a monitoring in strokovna konferenca z mednarodno udeležbo, [17. september 2021] system that provides feedback regarding physical functioning to elderly : zbornik povzetkov z recenzijo = Health of the elderly : 5th Scientific people. Patient Preference and Adherence, 843. doi:10.2147/ppa.s45897 and Professional International Conference, [17th September 2021]: book of abstracts. Free electronic edition. Koper: Založba Univerze na Government of the Republic of Slovenia, Ministry of Labour, Family, Social Primorskem: = University of Primorska Press, 2021. Affairs and Equal Opportunities and Institute of Macroeconomic Analysis and Development. (2017). Active Ageing Strategy. Retrieved from Jaschinski, C., & Ben Allouch, S. (2019). Listening to the ones who care: https:/ www.gov.si/assets/ministrstva/MDDSZ/Direktorat-za-starejse-in- Exploring the perceptions of informal carers towards ambient assisted deinstitucionalizacijo/strategije/Active-Ageing-Strategy-2017.pdf 193 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE HOW PILOT PROJECTS CONTRIBUTED TO THE QUALITY OF LIFE AND THE STATE OF HEALTH OF USERS Polona Dremelj Social Protection Institute of the Republic of Slovenia EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 194 HOW PILOT PROJECTS CONTRIBUTED TO THE QUALITY OF LIFE AND THE STATE OF HEALTH OF USERS KEY MESSAGES ▶ The self-assessment of the state of health of users, as measured by the EQ-5D questionnaire, increased slightly on average at the end of the implementation of the pilot projects. It is important to note that the proportion of users who reported moderate or major difficulties in walking, performing daily activities, and moderate or major feelings of pain and discomfort decreased during the project. We estimate that the pilot projects also contributed to this. ▶ The average assessment of the quality of life of users did not change significantly during the intervention. This is a subjective assessment of the quality of life of users, which was obtained on the basis of a questionnaire for measuring the quality of life of the elderly (CASP-12). Despite the questionnaire being adapted for the elderly, we find that the questions were incomprehensible and sensitive for some users, so we must be careful when interpreting the results. ▶ Despite these limitations, it is indicated that the pilot activities had a certain positive effect on users, especially in terms of their state of health or well-being. It is unreasonable to expect major changes in the self-assessment of the state of health as well as in the quality of life of users within a relatively short period of pilot activities, as the effects of such social concepts usually show only after longer periods of intervention. It would thus be sensible to observe the pilot activities for a longer period of time. ▶ From a methodological point of view, we assess that the use of the CASP-12 questionnaire for the population included in long-term care is not the most appropriate, while the EQ-5D questionnaire is recommended. 195 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Introduction or older), so we looked for tools that are adapted to the older population in terms of ease of use and Quality of life is an amorphous concept for understanding. which there is no generally accepted definition The EQ-5D questionnaire was used to or measuring instrument. There are many measure the state of health. It is a simple generic different views or definitions of quality of life, measuring instrument that is used to measure the with definitions based on objective indicators, state of health of users in five categories (mobility satisfaction of needs, subjective assessment of (walking); self-care such as washing and dressing; well-being, assessment of ability and subjective performing usual activities such as household assessment of health or health-related quality chores, family activities and leisure; feeling of of life being predominant (Brazier et al., 2014). pain, discomfort and anxiety, depression). To The latter means that quality of life is treated in measure quality of life, the CASP-12 questionnaire the context of health and disease. It is a concept developed by Hyde et al. (2003) was used. On the that includes areas related to physical, mental, basis of a theory based on the satisfaction of needs, emotional and social functioning as well as the the latter proposed a model of quality of life that social context in which people live (Ferrans, 2015). includes four dimensions or areas of life: control The concept of health-related quality of life over life, autonomy, self-realisation (realisation overlaps in many aspects with the concept of of your ideas) and pleasure. On this basis, they health and quality of life. The difference between created the CASP-19 questionnaire or scale, which quality of life and health is relatively clear, while features 19 questions, of which four are related to it is more difficult to make a distinction between the control of life, and five each to the remaining health-related quality of life, health and quality of three dimensions or areas. They defined autonomy life. Most instruments for measuring health-related as the right of an individual to renounce the quality of life actually measure self-assessment interference of others in their life, while control of the state of health. Karimi and Brazier (2016) over life means the ability to be active in the propose that separate measuring of health and environment in which the individual lives. These quality of life may perhaps be more appropriate. two dimensions are the basic conditions that must Since there is not a uniform position or be met in order for someone to freely participate clear definition of quality of life, researchers face in society. Once these are met, the individual can difficulties in any attempt to measure quality of pursue the other two dimensions - self-realisation life, and some are sceptical about the value of through activities that provide them with pleasure. such measuring due to the inconsistent definition. Following the success of the scale in the original Regardless of the non-uniform practice, it is crucial study, it was adopted for use in the English that instruments that measure various aspects of Longitudinal Study of Ageing (ELSA), and from quality of life are included in the evaluation of there a revised version (CASP-12) was included in public policy interventions that focus on improving the Survey of Health, Ageing and Retirement in the lives of individuals (Brazier et al., 2014). Europe (SHARE). When determining how pilot projects in the field of long-term care have contributed to the quality of life of users, we used two measuring Methodology instruments (EQ-5D and CASP-12) in order to cover the broadest possible range of an individual’s life. With the evaluation, we tried to determine We also took into account the fact that the users of whether the life of a user who receives long-term the pilot projects were mostly older persons (more care services in the home environment is safe and than three quarters of the users were 76 years old of high quality in various areas of their life. For this EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 196 purpose, we prepared a questionnaire for users one year of involvement in the project (M12). A that included questions from the standardised descriptive (e.g. presentation of proportions) and CASP-12 and EQ-5D questionnaires. bivariate (t-test) data analysis was performed on The CASP-12 questionnaire is a revised the collected data. In interpreting the data, we also version of the longer CASP-19 questionnaire. used some data from interviews with users. It features 12 questions or statements that respondents answer on a four-point scale: “often”, “sometimes”, “rarely”, “never”. The result is the Results sum of the answers to these questions, which ranges from 12 (minimum) to 48 (maximum). A In the article, we present data on the quality high score means a high quality of life (Mehrbrodt, of life of users and their state of health before the Gruber, & Wagner, 2019). beginning of pilot activities and after (at least) one EQ-5D is a standardised questionnaire for year of involvement in the project. By comparing measuring the state of health that was developed the results at both measurement points, we by the EuroQol Group with the aim of providing observe the effects of the intervention on the a simple, generic measuring instrument for quality of life and the state of health of users. assessing the state of health. It includes simple The questionnaire was answered by 713 questions within five categories: mobility, self- users in the first assessment (M0), and 161 users care (washing and dressing), carrying out usual answered the questionnaire after one year of activities (household chores, family, leisure), involvement in the project or after the services feeling of pain, discomfort and feeling of anxiety, were provided (M12). Of these, 133 answered depression. Users thus assessed the problems both the questionnaire in point M0 and the they face in individual categories, with five questionnaire in point M12. The latter are those response levels: 1. no problems, 2. slight problems, whose quality of life and state of health can be 3. moderate problems, 4. severe problems, 5. monitored both at the beginning and at the end of unable to perform an activity (e.g. cannot walk)/ the project, which means that we can determine extreme problems (e.g. feeling extreme anxiety or whether and to what extent the quality of their unbearable pain). Based on the answers, a total of lives and the state of health have changed during 3,125 states are defined. Each status is displayed the project or the provision of service. by a five-digit code. For example, code 11111 means that the user has no problems by individual dimensions, while code 12345 means that the user Who were the users of the has no mobility problems, has slight problems services in the pilot long-term with washing and dressing, moderate problems care projects? with performing usual activities, feels severe pain or discomfort or feels extremely anxious or We first show below the basic characteristics depressed. The answers by individual categories of the users who responded to the questionnaire (dimensions) can also be converted into an index at both points of time. There were 69 such users value (Van Reen et al., 2019). The questionnaire in the pilot environment of Celje, 42 in the was included in the evaluation in accordance pilot environment of Krško and 22 in the pilot with the recommendation of the European Centre environment of Dravograd. for Social Welfare Policy and Research (Kahlert, Women accounted for almost 60% of the Boehler, & Leichsenring, 2018). surveyed users, with their proportion being largest Users were surveyed upon entering the in Celje (more than two-thirds), and the smallest project (first assessment) (M0) and after (at least) in Dravograd (slightly more than 36%). In terms 197 Table 1: Basic characteristics of respondents (N=133) of age, those aged between 76 and 85 years prevailed, with their proportion in all pilot environments combined amounting to just under 39% - it was the largest in Celje (almost 45%) and the smallest in Dravograd (just under 29%). People aged over 85 represented GENDER87 FEMALE MALE 37% in all pilot environments, which means that more than three-quarters of the surveyed TOTAL 59.4% 38.3% users were in the two highest age groups. People younger than 65 (adults) accounted CELJE 66% 30.4% for just under 10% of respondents, with the smallest proportion of them being surveyed DRAVOGRAD 36.4% 59.1% in Celje (just under 9%) and the largest share in Krško (just under 12%). KRŠKO 59.5% 40.5% If the population of surveyed users is compared with the total population of recipients of services as part of the pilot projects (N=549), we find that women also AGE GROUPS < 65 65 - 75 76 - 85 > 85 dominated the structure of service recipients YEARS YEARS YEARS YEARS (over 58%) - they represented the largest proportion in the Celje pilot environment TOTAL 9.8% 14.4% 38.6% 37.1% (just under 66%), while their proportion in Krško and Dravograd was just under 55% CELJE 8.7% 15.9% 44.9% 30.4% and 55%, respectively. While the gender structure of surveyed users in the Dravograd DRAVOGRAD 9.5% 28.6% 28.6% 33.3% pilot environment is quite different from the total population of recipients of services, the KRŠKO 11.9% 4.8% 33.3% 50.0% structure in Celje and Krško is quite similar. The differences in the share of users in terms of the age structure between service users and respondents, observed for all pilot environments together, are not large. The differences are slightly larger by individual environments, especially in Dravograd, where the proportion of respondents aged 65 to 75 years is higher and the proportion of those aged 76 to 85 years is lower, and in Krško, where on the one hand the proportion of surveyed users in the 65-75 age group and in the 76-85 age group is lower, while on the other hand the proportion of people over 85 years of age is higher than in the total population of service users. 87 Some respondents in Celje and Dravograd did not state their gender, so the combined proportion of men and women together is not 100%. 198 Table 2: Basic characteristics of all users, recipients of services (N=549) How has the quality of life of users changed after the pilot activities? In this sub-chapter, we establish whether GENDER FEMALE MALE and to what extent the health-related quality of life of users and their state of health have changed TOTAL 58.3% 41.7% during their involvement in the pilot project. We first focus on the state of health of users, as part CELJE 65.7% 34.3% of which we present the proportion of users who have experienced problems with mobility, self- DRAVOGRAD 55.0% 45.0% care, performance of usual activities, feelings of pain, discomfort and anxiety and depression, both KRŠKO 54.5% 45.5% in the first and last measurements and possible differences between the two measurements. More than 92% of users in all pilot environments reported at least moderate walking AGE GROUPS < 65 65 - 75 76 - 85 > 85 difficulties in the first survey, with the largest YEARS YEARS YEARS YEARS proportion of these users in the Dravograd pilot environment (95.5%). Also associated with mobility TOTAL 12.0% 12.2% 40.4% 35.4% is the performance of usual activities, as evidenced by the high proportions of users with at least CELJE 12.4% 11.7% 40.6% 35.3% moderate difficulties in performing these activities. In the first survey, there were 93% of them in total DRAVOGRAD 10.4% 13.2% 38.9% 37.5% - more than 92% in Krško, slightly more than 86% in Dravograd and more than 88% in Celje. The KRŠKO 12.7% 11.8% 41.4% 34.1% proportion of users who report at least moderate problems with washing and dressing (self-care) varies considerably between pilot environments at the time of the first survey. The largest proportion was recorded in the Krško pilot environment (92.5%), and the smallest in Dravograd (over 68%). At least moderate pain was reported by more than 83% of users in all pilot environments at the time of the first measurement, and a similar proportion shows in individual pilot environments. More than half of users also reported at least a moderate feeling of anxiety and depression during the survey, the largest share of such users being in Krško (60%) and the smallest in Dravograd (over 36%). In the second survey, the proportion of users with at least moderate problems with walking decreased in all pilot environments. In total, the proportion of users who have at least moderate difficulties in performing usual activities and 199 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Figure 1: Proportion of users in terms of the level of walking difficulties (mobility), self-care (washing and dressing), performance of usual activities (household chores, family, leisure) and the level of pain/discomfort and anxiety/depression (N=130) Mobility Self-care Performance of usual activities Pain / discomfort Anxiety / depression % 0 20 40 60 80 100 % 0 20 40 60 80 100 No or slight problems No or slight problems At least moderate problems At least moderate problems (M0) (M12) (M0) (M12) EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 200 the proportion of those who experience at least was reported by half of the users in the youngest moderate pain or discomfort also decreased. and oldest age groups - just below half of those Viewed by individual environments, the aged 66 to 75 and more than 60% of those aged 76 proportion of users who report moderate problems to 85. with self-care also decreased in Dravograd and In the second measurement, the proportion Krško, while in Celje it slightly increased (from of users who reported at least moderate walking 88.2% to 94%). Meanwhile, the proportion of users problems decreased in the oldest age group in who reported at least moderate pain decreased in comparison to the first measurement (from 94% to all pilot environments. 85%), while in other age groups it either increased Compared to the first survey, the proportion slightly or remained the same. A slightly higher of users with at least moderate problems with self- proportion of users in the youngest and oldest care increased in the second, both in total and in age groups reported at least moderate difficulties Dravograd and Celje, while the proportion of users in performing usual activities in the second with greater problems with washing and dressing measurement, while the proportion of users aged in Krško significantly decreased (from over 92% between 76 and 85 slightly decreased (from 90% to just under 85%). The total proportion of users to 86%). The proportion of users aged 66 to 75 who who reported at least moderate feelings of anxiety report at least moderate difficulties in performing and depression also increased (from 54% to 71%), usual activities is similar to that in the first mostly due to an increase in the proportion in the measurement. In the last measurement, at least Celje pilot environment (from over 56% to 69%). moderate difficulties with self-care were reported In Krško and Dravograd, the proportion of users by a higher proportion of users in all age groups, with a moderate feeling of anxiety and depression except the oldest one, in which the proportion decreased. slightly decreased. In the second measurement, the Figure 2 presents the proportions of users by proportion of users who report at least moderate individual age groups who have at least moderate pain or discomfort decreased in all age groups, problems or feelings in individual observed to the greatest extent among the oldest users. categories. All younger users (under the age of 65) However, the share of users with moderate or report at least moderate walking difficulties at both severe anxiety or depression increased in the points of time. In other age groups, the proportions second measurement in all age groups except the are slightly lower at the first measurement point, oldest, increasing the most among users aged 66 with at least moderate walking problems being to 75. found in 90% of people aged 76 to 85, in 95% of In the survey, users chose a value on a value those aged 66 to 75 and in 85% of those aged 85 scale between 0 and 100, shown to them by the and over. In the first measurement, more than assessors, that corresponded to their assessment three-quarters of users in all age groups reported of health or well-being at the time of the survey. at least moderate self-care problems. The highest A value of 0 meant the worst state of health, and proportion of these users was in the oldest age a value of 100 meant the best state of health they group (slightly less than 92%). At least moderate could imagine. Shown below are the estimates in difficulties in performing usual activities are the first and last measurement. mostly reported by users aged between 66 and 75 On average, men rated their health at 43.4 (just under 95%). In the first measurement, the (M0) and 45.4 (M12), respectively, which is lower majority of users (the proportion ranges from 75% than the overall average value, while women’s in the youngest age group to 89% of the oldest) self-evaluation was higher than average and reported at least moderate or severe pain, and at amounted to 44.5 (M0) and 47.4 (M12). Both men least a moderate feeling of anxiety or depression and women rated their health as medium good, 201 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Figure 2: Proportion of users who have at least moderate problems with walking (mobility), self-care (washing and dressing), performance of usual activities (household chores, family, leisure) and at least a moderate feeling of pain/discomfort and anxiety/depression by age groups (N=130) Mobility Self-care under 65 years 65 to 75 years 76 to 85 years over 85 years Performance of usual activities Pain / discomfort under 65 years 65 to 75 years 76 to 85 years over 85 years Anxiety / depression under 65 years 65 to 75 years 76 to 85 years over 85 years % 0 20 40 60 80 100 % 0 20 40 60 80 100 At least moderate problems At least moderate problems (M0) (M12) EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 202 and it is encouraging that the values were slightly assessors, that corresponded to their assessment higher in the second assessment. The median value of health or well-being at the time of the survey. shows that half of men and half of women alike A value of 0 meant the worst state of health, and rated their health below 50 and half rated their a value of 100 meant the best state of health they health above 50. could imagine. Shown below are the estimates in In the first measurement, the worst the first and last measurement. assessment on average was made by men in Based on the answers by users to the the Celje pilot environment, while the highest questions designed to assess the problems they assessments came from men and women in have by individual dimensions of the state of the Dravograd pilot community. In the second health, it is also possible to make a joint assessment measurement, the average assessment by men in of the state of health of users. What is important in Celje increased, while the average assessment of this approach is that we can detect a change in the women’s health decreased. It is the opposite in state of health of users at different points of time. Krško, where the average health assessment by The index of the state of health of users in the men decreased and that by women significantly first measurement averaged 0.201, which means increased. The difference in the average health poor state of health (where value 0 means the assessment is statistically significant in women state of health equal to death, and 1 perfect state (t=1.735, p=0.091). In Dravograd, the average health of health). If we look at the values of individual rating by both men and women decreased slightly, dimensions of the state of health in, for example, a with the decrease for the latter being minimal. random person with an index of the state of health Figure 3 shows the self-assessment of the state of 0.201, we find that that person’s code is 45343, of health of users by individual pilot environments which means that the person has severe difficulty in terms of age groups. In two pilot environments, walking, is unable to wash and dress themselves except in Krško, the youngest users rated their on their own, has slight difficulty performing usual state of health better in the first measurement activities, severe pain and feeling of discomfort than in the last, which relates to the data that a and is moderately anxious or depressed. In this higher proportion of younger users reported at case, we can talk about a person’s poor physical least moderate problems with walking, self-care health and moderate mental health. and in normal activities in the last measurement In the last measurement, the index of compared to the first measurement. On the other the state of health of users increased slightly hand, older users, especially those over the age compared to the first, to 0.241 (the difference is of 85, rate their health better on average in the not statistically significant). The median was 0.361, second measurement than in the first. which means that half of the users had a worse, If all three pilot environments are observed and half had a better state of health than this together, users between the ages of 66 and 75, value. Compared with the first measurement, the on average, rate their health better in the second median value increased from 0.284 to 0.361. Even measurement than in the first, although the picture in individual pilot environments, we find that is slightly different if the pilot environments are the index of the state of health of users in the last observed individually. While users in the Celje measurement increased slightly compared to the pilot environment rate their health better in the first, and the median values are also higher in the second measurement than in the first, users in the last assessment. Krško and Dravograd pilot environments rate their A change in the state of health can also health worse in the second measurement. be determined on the basis of changes in the In the survey, users chose a value on a value codes resulting from the answers by individual scale between 0 and 100, shown to them by the dimensions of the health status. There are only 203 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Figure 3: Average self-assessment of users’ state of health by age groups and pilot environments (N=127) 60% 50% 40% 30% 20% 10% 0% M0 M12 M0 M12 M0 M12 M0 M12 TOTAL CELJE KRŠKO DRAVOGRAD under 65 years 65 to 75 years 76 to 85 years over 85 years Table 3: Average health self-assessment by gender of the user (N=128) 204 Figure 4: Proportion of users in terms of improvement or deterioration of the state of health, by gender and age group (N=122) four options - we can determine whether the state of health in the second measurement is better (which means that it is better in at least one dimension and not worse in any other), worse (worse in at least one dimension and not better in any other) or mixed, meaning it is better in one dimension and worse in another. The state 34.4 26.6 39.1 of health may remain the same between one measurement and another, meaning that it does not change in any of the dimensions. 26.7 20.0 53.3 Figure 4 shows the proportion of users in terms of change in the state of health, both together and by gender and age groups. We find that the situation has changed for all users - for more than 35% for the better, for slightly under 23% for the worse, and for just over 41% the condition has worsened in some categories and 50.0 25.0 25.0 improved in others. The state of health remained the same for none of the users. In terms of gender, the state of health improved in a higher proportion of women (39%) than men (just under 27%), 33.3 16.7 50.0 and when it comes to age it turned out that the proportion of users whose state of health improved decreased with age, except in the oldest age group, 31.7 26.8 41.5 where the proportion of users with better state of health in the last measurement is 40% and is higher than in the 65-75 age group (over 33%) and the 76-85 age group (up to 32%). 40.0 22.9 37.1 Presented below is the level of quality of life of users, which was calculated on the basis of twelve questions (CASP-12 questionnaire). The average quality of life of users in the first measurement was 30.1, which means a medium- high quality of life. Given the median value of 30.0, we can say that half of the users had a worse and half a better quality of life than the medium-high 35.6 22.9 41.5 quality. In the Celje pilot environment, the average quality of life of users in the first measurement was minimally lower than the estimate for all pilot 0 20 40 60 80 100 environments combined, it was similar in Krško, and minimally higher in Dravograd. In the last measurement, the quality of life of users remained at approximately the same level as in the first Better measurement, it increased minimally in all pilot Worse environments combined and in Krško (the median Mixed 205 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 4: Index of the state of health of users in the first and last assessment, combined and by individual pilot environments (N=121) N AC Me SD Min Max M0 121 0.201 0.284 0.305 –0.452 0.836 Total M12 121 0.241 0.361 0.318 –0.452 0.747 M0 63 0.190 0.247 0.320 –0.452 0.836 Celje M12 63 0.240 0.349 0.315 –0.397 0.695 M0 37 0.205 0.317 0.297 –0.452 0.625 Krško M12 37 0.241 0.336 0.317 –0.452 0.625 M0 21 0.225 0.345 0.282 –0.293 0.579 Dravograd M12 21 0.245 0.396 0.346 –0.410 0.747 Table 5: Assessment of the quality of life of users in the first and last assessment, combined and by individual pilot environments (N=90) N AC Me SD Min Max M0 90 30.1 30.0 4.42 20 41 Total M12 90 30.3 30.0 4.09 19 42 M0 46 29.8 30.0 4.18 20 39 Celje M12 46 29.7 29.0 3.95 19 42 M0 24 30.0 29.5 4.91 22 39 Krško M12 24 30.5 30.5 4.35 22 38 M0 20 31.0 30.5 4.48 25 41 Dravograd M12 20 31.2 30.5 4.12 26 39 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 206 Figure 5: Frequency distribution of users according to the quality of life assessment in the first measurement, all pilot environments combined (N=90) Number of users 12 10 8 6 4 2 2 4 5 10 5 6 7 10 6 8 5 3 6 2 3 3 2 0 1 1 1 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 Quality of life assesment M0 Figure 6: Frequency distribution of users according to the quality of life assessment in the last measurement (N=90) Number of users 12 10 8 6 4 2 2 7 12 9 11 7 8 10 4 4 3 4 2 2 2 0 1 1 1 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 Quality of life assesment M12 207 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE value is also minimally higher) and Dravograd, and a slightly higher proportion (26%) to a higher while it decreased minimally in Celje. category, while among users from the highest, fifth If the frequency distribution of users is category of care, 22% moved to lower categories in compared on the basis of the assessment of the last assessment. quality of life for all pilot environments together, A similar trend of users moving between we find that compared to the first measurement, categories of eligibility is also reflected in there were smaller shifts in the lower half of the individual pilot environments, although some distribution (values lower than the median) in the specifics are noticeable. In Krško, after one year of last measurement. In the first measurement, users involvement in the project, 9.5% of users were no were more or less evenly distributed between longer eligible for services, while in Dravograd no the values 23 and 29 (2 to 7 users) with a higher user was placed in the highest eligibility category. frequency at the value 26 (10 users), and in the last measurement, a higher density between the values 26 and 29 was observed (7 to 12 users). In Discussion with key the upper half of the distribution, the shifts were messages even smaller, although the maximum value of the quality of life assessment increased by one (from Changes in the quality of life and the state of 41 to 42). If individual users are observed, we find health of users before and after at least one year that the assessment of the quality of life of users of involvement in pilot activities were determined has changed for 78% of users, for the worse for on the basis of assessment and classification in half of them and for the better for half of them, categories of eligibility for long-term care and while the assessment is the same at both time subjective assessment by users regarding the state points for 12% of users. of health and quality of life. Changes in the state of health and quality of From the aspect of classification in eligibility life of users were also reflected in the classification categories, we find that the involvement in pilot of users in individual categories of eligibility for activities was the most positive for 3% of users, long-term care (including eligibility categories). for whom the last assessment showed that they If the proportion of users by individual no longer need services as part of pilot projects. eligibility categories in the first assessment is Other users either remained in the same eligibility compared with the assessment after one year, we category or moved to a higher or lower category. find that the proportion of users who are no longer Positive changes by individual pilot environments eligible for services (category 0) increased, the were shown in Krško, where in the last assessment proportion of users in the first eligibility category 9.5% of users were no longer eligible for long-term decreased (6% of them are no longer eligible care and thus for services as part of the pilot project, for services), more than half (51%) moved to a and in Dravograd, where all users from the highest higher category (34% up one category, 17% up two eligibility category passed into the lower ones. categories); the proportion of users in the second Assessments by users regarding the severity category also decreased (3% of them are no longer of problems they face with mobility, self-care, eligible for services, more than 29% moved to a performance of usual activities, and feelings of lower category, and just under a third moved to a pain and anxiety or depression showed positive higher category), while the proportion of users in changes in four of the five categories assessed at the last three categories increased. From the third the time of the last measurement. The proportion category, more than 11% of users moved to lower of users who reported at least moderate difficulties and 32% to higher categories, from the fourth with walking (mobility) and performing usual category 21% of users moved to lower categories, activities in the first measurement and at least 208 Figure 7: Proportion of users by individual eligibility categories in the first and last assessment (N=133) a moderate feeling of pain or discomfort and anxiety or depression decreased slightly in the second measurement. On the other hand, we notice that the proportion of users with at least moderate anxiety or depression in the second 26.3 25.6 27.1 14.3 6.8 measurement decreased only in the oldest users (aged 85 or older), while in younger users this 3.0 18.8 21.8 28.6 18.0 9.8 share slightly increased. The latter results can be partly attributed to the Covid-19 epidemic, as almost half of the users surveyed were more likely to feel anxious during the epidemic than before the epidemic, half of the users reported being more isolated, and more than 2% of respondents evaluated their health as worse compared to before the epidemic. 23.2 29.0 23.2 15.9 8.7 Pilot projects have had positive effects on the 20.3 20.3 29.0 17.4 13.0 state of health for at least a third of users, while either improvement in some areas or deterioration in other areas shows in the remaining users. Unfortunately, the state of health of almost a quarter of users deteriorated, which cannot be attributed to the possible ineffectiveness of pilot projects, but is a consequence of various factors which were not investigated in detail in the 26.2 19.0 33.3 16.7 4.8 evaluation. The data shows that the assessment of the 9.5 19.0 16.7 23.8 21.4 9.5 quality of life of users also increased during the implementation of the pilot projects, albeit minimally. To a greater extent than the quantitative data, this is evidenced by the statements of users, which show how important for the quality of life of the user is even the smallest possible intervention, such as an employee visiting 36.4 27.3 27.3 4.5 4.5 as part of a pilot project. 13.6 36.4 36.4 13.6 “It’s nice if she comes, even if you just see her, and you are already satisfied.” 0 20 40 60 80 100 “Well, I don’t know, it’s a little better when I get a little exercise and that, but I’m over the hill, and you can’t make miracles, they are trying but it doesn’t work.” “It is not much, but it’s great. It’s easier for me Category 0 Category 1 and I live better that way.” Category 2 Category 3 “So that you know, there is some progress Category 4 Category 5 for sure. For instance, if I wait for her downstairs, 209 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE which she thought was not even possible, to see me particular older ones, did not feel comfortable with there, but she did.” some questions (e.g. the question about the future). “Actually, I feel, to be honest with you, kind of On the other hand, the EQ-5D questionnaire like I’m extra protected and I don’t even know from proved to be an appropriate measuring instrument whom or from what. I feel safer, as if I have one for (self-)assessment of the health status of users in more person to trust.” pilot projects. Although measuring the quality of life of It is difficult to expect major changes in the individuals is not simple and poses significant self-assessment of the state of health as well as theoretical as well as methodological challenges, in the quality of life of users in a relatively short it has proved in recent decades to be an period of pilot activities, as such effects usually exceptionally important part of various public show only after longer periods of intervention. policy interventions aimed at improving the lives Perhaps the results would be different if objective of individuals. This is especially important for indicators were included in the measuring of interventions that focus on the quality of life of quality of life in addition to subjective indicators. the elderly, as it is more likely that the quality of We must be particularly careful when life of this demographic will be exacerbated by interpreting the obtained results regarding the events such as hospitalisation, institutionalisation, quality of life of users, as it is indicated that the illness, death of family members or friends (Borrat- most appropriate measuring instrument was not Besson, Ryser, & Gonçalves, 2015). Accordingly, as selected for the observed population. The CASP-12 mentioned earlier, any intervention, even a small questionnaire was based on a sample of people one, is exceptionally important for the quality of aged 65 to 75, and in pilot projects the average life of older individuals. age of users was 80, and more than three-quarters were older than 75. Assessors who conducted the user survey reported that the questions were difficult to understand for some users, so there were many missing values (of the 133 users who answered the questionnaire at both points, only 90 answered all the CASP-12 questions). Users often did not understand the questions, in particular users with dementia and users who had communication problems. Some users, in EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 210 BIBLIOGRAPHY Borrat-Besson, C., Ryser, V.-A., & Gonçalves, J. (2015). A n evaluation Kahlert, R., Boehler, C., & Leichsenring, K. (2018). Monitoring of the CASP-12 scale used in the Survey of Ageing and and Evaluating Integrated LTC Models. Vienna: Euro Centre Retirement in Europe (SHARE) to measure Quality of Life among Publication. people aged 50+. FORS Working Paper Series, paper 2015-4. Lausanne: FORS. Karimi, M., & Brazier, J. (2016). Health, Health-Related Quality of Life, and Quality of Life: What is the Difference? . PharmacoEconomics Brazier, J., Connell, J., Papaioannou, D., Mukuria, C., Mulhern, B., 34, p. 645–649. Peasgood, T., … Parry, G. (2014). A systematic review, psychometric analysis and qualitative assessment of Generic Preference- Mehrbrodt, T., Gruber, S., & Wagner, M. (2019). SHARE: Scales and Based Measures of Health in Mental Health Populations and Multi-Item Indicators. Retrieved from http:/ www.share-project. the estimation of mapping functions from widely used specific org/fileadmin/pdf_documentation/SHARE_Scales_and_Multi- measures. Health Technology Assessment. 18:34. Item_Indicators.pdf Ferrans, C. E. (2015). Definitions and conceptual models of quality Van Reen, M., Janssen, B., Stolk, E., Secnik Boye, K., Herdman, of life. V: Lipscomb, J., Gotay, C. C., Snyder, C. (ed.) Outcomes M.,Kennedy-Martin, M... Slaap, B. (2019). EQ-5D-5L User Guide. assessment in cancer. Cambridge, England: Cambridge Basic information on how to use the EQ-5D-5L instrument. University, p. 14–30. Retrieved from https:/ euroqol.org/publications/user-guides. Hyde, M., Wiggins, R. D., Higgs, P., & Blane, D. B. (2003). A measure of quality of life in early old age: The theory, development and properties of a needs satisfaction model (CASP-19), Aging & Mental Health, 7:3, p. 186-194. 211 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE CARE FOR THOSE WHO CARE: STUDYING THE QUALITY OF LIFE OF INFORMAL CARERS Mateja Nagode, Social Protection Institute of the Republic of Slovenia Jasmina Rosič KU Leuven, School for Mass Communication Research Maja Škafar Social Protection Institute of the Republic of Slovenia EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 212 CARE FOR THOSE WHO CARE: STUDYING THE QUALITY OF LIFE OF INFORMAL CARERS KEY MESSAGES ▶ Informal care is the backbone of long-term care, and its high incidence has also been confirmed by pilot projects. ▶ Informal care is in the domain of women, which puts them in an unequal position to men. ▶ The subjective burden on informal carers did not improve during the involvement in the pilot projects, although we can identify many relief factors (option of respite care, use of annual leave, transportation, time flexibility, etc.). ▶ Pilot projects have had a positive impact on the lives of informal carers in terms of objective relief, which further reinforces the call for organising better support for informal carers. Measures in this field are therefore necessary, as formally organised care does not sufficiently reduce the burden of informal care, and cooperation between the two types of care is too weak. ▶ Organised home care and community care are an opportunity to relieve informal carers, either from certain tasks or in the sense of support and learning about how to provide care (actionable knowledge). 213 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Introduction proportion of women (16%) is higher than the proportion of men (13%). Differences between The majority of long-term care systems in countries are also partly the result of women’s Europe are based on informal care. In literature, participation in the labour market, which is it is thus called the backbone, the core, the main generally higher in Eastern European countries. In pillar or the foundation, as it is predominant these cases, the participation of women in informal in the (co-)provision of care (Huber, Rodrigues, care is lower and there is a higher probability or Hoffmann, Gasior, & Marin, 2009; Naiditich, need for men to be involved in care (the Czech Triantafillou, Di Santo, Carretero, & Hirsch Durrett, Republic, for example) (Zigante, 2018). 2013; Verbeek-Oudijk, Woittiez, Eggink, & Putman, Research shows that the number of informal 2014; Zigante, 2018). Estimates show that around carers ranges from 10% to 25% of the total 80% of total long-term care in Europe is provided population (Spasova et al., 2018). A more recent by informal carers (Hoffmann & Rodrigues, 2010). study (Tur-Sinai, Teti, Rommel, Hlebec, & Lamura, There is no standard definition of informal 2020), based on three different surveys (EQLS, EHIS care (Zigante, 2018). What is common to the and SHARE), meanwhile, states that the proportion different definitions is that it is care provided to of informal carers in the population over the age of a person who needs support at home (usually lay 50 in 15 European countries ranges from 12.9% to and unpaid support) by their family members, 29%. At the same time, the authors point out that friends or neighbours, called informal carers. the differences in the calculated proportions can be Informal carers, i.e. family members, friends or significant compared to various surveys.88 However, neighbours, perform a lot of care work within the this is not the case for Slovenia,89 where data from system of long-term care. various surveys show that on average 16% of people When it comes to the distribution of the aged over 50 (also) perform informal care. burden of care, women are the ones who dominate The high prevalence of informal long- and take responsibility for care, which negatively term care may be influenced by the lack of affects their participation in the labour market. accessible formal services, their poorer quality Women are more likely to leave the labour market and high costs, and the tradition of family and or reduce the number of working hours more intergenerational relationships (Spasova et al., than men for this purpose (Spasova et al., 2018; 2018). Placing long-term care at home at the Colombo, Llena – Nozal, Mercier, & Tjadens, forefront and strengthening cooperation between 2011; Naiditich et al., 2013; Huber et al., 2009; informal and formal long-term care, which are Rodrigues, Schulmann, Schmidt, Kalavrezou, & characteristic trends in Europe (Spasova et al., Matsaganis, 2013). The results of an EQLS survey 2018; Nagode & Lebar, 2019), brings a reflection (2016) indicate that the gender gap varies from on the division of roles and scope of care between country to country. The largest gender gap is formal and informal carers. in Belgium, where 13% more women than men Providing care may be physically and provide informal care, similar to the Netherlands mentally exhausting. Leaving the labour market and Greece (a 10% gap). On the other hand, in puts the carer at a greater risk of poverty the Czech Republic, for example, this proportion (European Commission, Directorate-General for is equal, while in Slovakia, for example, the Economic and Financial Affairs and Economic proportion of men (22%) is even higher than Policy Committee, 2016). Carers are more exposed the proportion of women (17%). In Slovenia, the to health-related risks (Baji et al., 2019) and mental 88 For example, in Belgium, the proportion of informal carers in the population over the age of 50 is 25.3% according to the SHARE survey, 12.8% according to EHIS and 34.3% according to EQLS. 89 SHARE 15.5%, EHIS 17.6% and EQLS 14.8%. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 214 health problems (Colombo et al., 2011; Tjadens & calculated the subjective workload as an index on Colombo, 2011). However, the provision of care an interval between 0 and 88 on the basis of all can have a positive effect on the health and general 22 items measured on a scale from 0 (never) to 4 well-being of carers (if it stems from motivation, (almost always): little or no burden (0–20 points), love, affiliation, a sense of duty) (Naiditich et al., mild to moderate burden (21–40 points), moderate 2013). From the aspect of public finances, such to severe burden (41–60 points) and severe burden informal care can be understood as a cost-effective (61–88 points). way of preventing costly institutionalisation, while To measure the objective burden, the the indirect costs at the individual and state level, respondents were asked how many hours per in particular in relation to employment, health and week, on average, they spend on informal care well-being of informal carers, are being overlooked and assistance to the person included in the pilot (Rodrigues et al., 2013; Zigante, 2018). We project who listed the respondent as the key emphasised precisely the latter in the evaluation informal carer. In analysing this question, we of long-term care pilot projects, in which informal followed guidelines from relevant literature (Moya- carers were one of the important target groups. Martinez, Escribano-Sotos, & Pardo-Garcia 2014) and recalibrated those who reported 112 hours of care or more per week to 112 hours per week. Methodology We also collected data on the demographics and households of informal carers, on the provision The purpose of the evaluation was to find out of assistance and care, and on the experience and who informal carers are, how they provide care usefulness of the pilot project. We also asked them and live, and last but not least, how widespread to list the three things that make them the happiest informal care is in the pilot environments. The in caring for the person they care for and the three key goal was to find out whether the quality of life things that burden them the most. In the article, of informal carers has changed during the pilot their statements are denoted by M0. project, in particular whether their objective and The questionnaire was completed by informal subjective burden of care has decreased. The latter carers twice: first (M0) in the first eligibility serves as a basis for guidelines for the provision of assessment of the person they care for and again long-term care services that will enable a quality (M12) after a year of the inclusion of this person in life for informal carers. the pilot project (third eligibility assessment). The In order to be able to evaluate the goal, we latter was basically identical to the first one, with prepared a questionnaire for informal carers. a set of questions related to the Covid-19 epidemic Based on the recommendation of the European situation being added. The first questionnaire Centre for Social Welfare Policy and Research (M0) was answered by 395 informal carers (64.8% (Kahlert, Boehler, & Leichsenring, 2018), we used response rate91), and the second (M12) by 94 (79.7% a standardised questionnaire on the subjective response rate). 58 informal carers responded to experience of the care burden, called the Zarit both questionnaires. Univariate and bivariate Burden Interview (ZBI-22), which is among the statistical methods were used in the analysis of most widespread90 tools for measuring this burden quantitative data, and thematic qualitative analysis (Mosquera et al., 2016). We used the ZBI-22 version, was used in the analysis of open answers. which has high reliability and construct validity In addition to the questionnaire, in-depth (Herbert et al., 2000; Mosquera et al., 2016). We semi-structured interviews were also conducted 90 There are also, for example, Pearlin's Overload Scale, Screen for Caregiver Burden, Sleep Disorders Inventory, Caregiver Distress Scale (Mosquera et al., 2016), Cost of Care Index, Burden Scale for Family Caregivers (Graessel et al., 2014). 91 The response rate was calculated using data from the information system on whether the applicant receives assistance from relatives or not. Considering the type of survey (self-survey), we assess that the response rate is high. 215 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE with informal carers. Seven informal carers from assistance (also) from a relative:92 the highest each pilot environment were included in the quota incidence was in Dravograd (80.1%), similar in sample and the following inclusion criteria were Celje (79.2%) and slightly lower in Krško (66.5%). considered: eligibility category of the relative, Among the surveyed informal carers, women gender of the informal carer, their employment predominated in all three pilot environments status and distance from the relative they care for. (65.4%), with the largest proportion in Celje (68.7%) We talked to the interviewees about the experience and the lowest in Krško (60.9%). On average, the with the pilot project and individual activities, informal carers were 63.2 years old, and there methods and parts of the procedure, and about were no significant differences in the mean age the changes they have detected since they were between the environments. The youngest informal included in the pilot project. Verbatim transcripts carer in Dravograd was 26 years old, in Celje 31 of interviews were thematically analysed in a years old and in Krško 37 years old. The oldest deductive way (Boyatzis, 1998; Braun & Clarke, carer in Dravograd was 93 years old, in Celje 88 2006; Hayes, 1997) in accordance with the principle years old and in Krško 89 years old. Three-tenths of systematic coding as proposed by Saldana (29.4%) of all informal carers were over 70 years (2012). In the article, citations from interviews are of age, while no underaged informal carers were denoted by I-NF. recorded in the pilot projects. In order to obtain in-depth results, we used Retired informal carers were predominant mixed research methods, especially the concurrent (48.5%). In Dravograd, the proportion of retirees nested design (Creswell & Plano Clark, 2007), which was about ten percentage points higher (51.9%) includes an extensive (predominant) quantitative than in Krško (41.8%), while in Krško there phase with an integrated smaller qualitative part. were slightly more unemployed persons and homemakers. The Dravograd pilot environment stood out with a slightly lower proportion of full- Results time employees compared to other environments (Celje 35.6%, Krško 35.5% and Dravograd 28.6%). In this article, we first present data on The educational structure of informal carers the quality of life of informal carers before the is also statistically significantly related to the pilot implementation of pilot activities. In doing so, we environments (ᵪ2=22.897, p=0.004). In the Celje rely on the data obtained upon the entry to the pilot environment, informal carers generally had a project (M0). We then observe the effects of the higher level of education, with 37.8% having at least intervention on the quality of their life, comparing higher education. In Krško, 25.5% of them had such the situation at the start of the project (M0) with a level of education and in Dravograd it was 22.4%. the situation after a year of inclusion in the project The respondents provided informal care (M12). mostly to their parents (47.7%) or partners (33.2%), although there are differences by environments (ᵪ2=10.931, p=0.027). Dravograd stood out with Who are the informal carers, a much smaller proportion of respondents who how do they provide care and provided care to their parents (Celje 53.6%, Krško how much are they burdened? 50.9% and Dravograd 37.8%) and a much larger proportion of those who provided care to their The incidence of informal care was high in partners (Celje 27.8%, Krško 28.2% and Dravograd the pilot environments, as three quarters (75.1%) 44.1%). More than two-thirds (68.9%) of the of all applicants for eligibility assessment received informal carers surveyed lived in the same building 92 And for whom the data on receiving assistance from a relative was also recorded. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 216 as the persons they provided care to (either in a An important aspect of the quality of life of joint or separate household), which is statistically informal carers is how burdened they feel with the significantly related to the pilot environment of provision of care, and how both the objective and the informal carers (ᵪ2=21.368, p=0.06). In the subjective burdens can be measured. In terms of Krško pilot environment, significantly more objective burden, it showed that the respondents respondents lived in the same building with the provided an average of 41.2 hours of informal person they provided care to (76.7%) compared care per week upon entering the project, with half to Dravograd (67.3%) and Celje (63.9%), while in of the informal carers providing 28 hours or less Celje, for example, there was a significantly higher of care per week and half of them more (Me=28). proportion of such who live up to half an hour The difference in hours of care between the pilot away (Celje 16%, Krško 3.9% and Dravograd 5.5%). environments is statistically significant (F=6.063, In general, we find that before the beginning p=0.003). The Dravograd pilot environment stood of the pilot projects, informal carers mostly out with significantly more hours of care (AS=51, carried out instrumental activities of daily living, Me=35) than in the Krško (AS=36.4, Me=24) and particularly domestic help. The most frequent Celje pilot environments (AS=36.3, Me=24). activities were dishwashing (73.7%), cooking, We first take a look at subjective burden helping to prepare meals or delivering food (70.3%), through the feelings that were almost always or bed making, cleaning bedrooms (64.2%). This was quite often felt by at least a third of the surveyed followed by assistance in purchasing and taking informal carers and connect them with the findings medications (63.3%). Assistance with activities of from open answers and interviews. Then we daily living was less frequent, although not very present the final subjective burden. much. Informal carers most often helped with The most, eight out of ten (78.9%) of the dressing and undressing (58.5%) and slightly less surveyed informal carers, stated that the person (53.7%) with lying in bed and getting up, using whom they provide assistance to is quite often or the toilet and bathroom (46%) and maintaining almost always dependent on them. Approximately and caring for care accessories (42.3%). The two-thirds (61.7%) are afraid of what the future other tasks performed by a high proportion of holds for the person they provide care to ( “The the surveyed informal carers on a daily basis for worry that something will happen to him, that he the persons they provide care to include feeding falls and gets hurt when no one is around.” (M0), “To (36.9%), shopping, purchasing food, household watch her deteriorate.” (M0)). More than half (55.9%) goods, cleaning products (36.8%), washing and said that the person they provide care to seems to bathing (36.4%), laundry and ironing (35.1%), expect them to take care of them as if they were minor house repairs and gardening (33.8%) and the only ones they could depend on ( “He is never financial management (30.4%). Informal carers satisfied with anything, so he is not able to praise from Dravograd stood out with a higher proportion or be grateful for anything.” (M0)). In the latter, in the majority of tasks. the Dravograd pilot environment stood out with a The surveyed informal carers listed nine slightly lower proportion of respondents (45.3%) (AS=9.2, Me=9) different daily tasks on average. compared to Krško (58.5%) and Celje (63.3%). Standing out in terms of a larger number of tasks A significant proportion of the surveyed (AS=10.2) were the surveyed informal carers from informal carers felt that they did not have enough Dravograd, who on average performed one task time for themselves (39.3%). More than a third more per day than in Celje (AS=8.8) and two more experience stress due to having to reconcile work than in Krško (AS=8.4). The difference between the and family life with the demands of the person they environments is statistically significant (U=4.176, provide care to (34.8%) ( “Sometimes I can’t be with p=0.016). my family due to the obligations with my mother.” 217 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE (M0)). A quarter of the informal carers stated that whom they provide help. They mentioned concern their health suffered because of the efforts related about the health problems of the person they to the person they provide care to (25.9%) or they provided care to, disturbing behaviour or traits or felt that they would no longer be able to care for mood swings of the person they provided care to, that person for much longer (24.8%). Also, a quarter the feeling of helplessness when the effect of help of them assessed that their social life suffered as was not visible. Some lacked care competencies a result of care (24.2%) ( “I don’t have much social or were unaware of possible forms of assistance. inclusion, I live like a zombie here at my father’s Difficulties in cooperation with the formal place” (I-NF)). One-fifth of the informal carers felt assistance network were also mentioned. that because of the person they provide care to they Informal carers were made happy and did not have as much privacy as they would like motivated mainly by factors related to the person to have (21.7%). Also, a fifth of them believes that to whom care is provided. They were often they do not have enough money to take care of the motivated by this person’s satisfaction with and person they provide care to (18.5%) ( “When you gratitude for care, connection with the person are in distress, when you suddenly get an immobile and, as part of this, spending time with and having father from the hospital, with severe bedsores, you a loving and emotional relationship with the have to buy everything yourself.” (M0)). person. The visible effect of the assistance, i.e. The informal carers reported much less the contribution to changes in the daily results often that the person they provide care to exerted of care recipients, was also a motivating factor. a negative influence on relationships with other They were also made happy by the physical family members and friends (3.7%). They were health of the person to whom care is provided rarely of the opinion that they feel angry when (e.g. improvement or maintenance of the health they are with the person to whom they provide condition) and their mental health (e.g. good mood; care (1.8%). A few of them stated that they felt motivation, enthusiasm, optimism and prevention uncomfortable having friends over because of the of loneliness). An important aspect of motivation person they provide care to (1.8%). for care was the provision of care at home and Based on the reports from the informal carers not in an institution, i.e. the impact on preventing in the questionnaire on the three things related to institutionalisation. care that make them the happiest and the three On the other hand, the happiness factors in that burden them the most, we identified with a informal carers were related to a lesser extent to qualitative analysis the incidence of two major themselves and only indirectly to the well-being types of care burdens and two types of factors that of the care recipient. They have been altruistic make informal carers happy and motivate them to and helped because they enjoy it or because they provide care. value this kind of conduct. A small proportion of The first type of burden is related to self-care the informal carers helped because they had the and refers to the risks associated with their own feeling of duty to help. A handful of them were health, lack of (free) time and adjusting their time also happy about they themselves being relieved, to the person they care for, abandonment of their mentioning methods of relaxation, help from own activities, performance of activities of daily formal care and help by family members. living and instrumental activities of daily living, The presentation of the subjective burden of distance from the person they care for, negative informal care can be concluded with the index of impact of care on their finances, reduced social subjective burden. The latter shows that almost half contacts or even social exclusion. (46%) of the surveyed informal carers felt mild to In the second type of burden, the descriptions moderate burden upon entering the project, just referred to the care or concern for the person to under a quarter felt no or little burden (24.3%), EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 218 Table 1: Characteristics and burden of informal carers upon entering the project (M0) (N=395) Total Celje Dravograd Krško Number of respondents 395 153 131 111 Female gender 65.4% 68.7% 64.7% 60.9% Age, average 63.2 63.2 64.6 61.6 Age [min, median, max] [26, 63, 93] [31, 63,5 88] [26, 64, 93] [37, 60, 89] (Un)finished primary education 14.1% 9.3% 18.5% 15.5% Lower or secondary vocational education 25.1% 23.2% 26.9% 25.5% Secondary vocational or general education 31.7% 29.8% 32.3% 33.6% Short-cycle higher or higher vocational education 12.8% 12.6% 16.2% 9.1% s Higher education, university er education or higher 16.4% 25.2% 6.2% 16.4% Unemployed 6.1% 5.6% 5.3% 8.2% mal caror Employed less than full-time 3.1% 2.8% 3.8% 2.7% Employed full-time 33.3% 35.6% 28.6% 35.5% Self-employed 4.0% 2.8% 6.0% 3.6% istic of inf Retired 48.5% 50.0% 51.9% 41.8% acter Homemaker 3.8% 2.8% 1.5% 8.2% Char Other 1.2% 0.6% 3.0% 0.0% Partner themselves 33.2% 27.8% 44.1% 28.2% Child themselves 47.7% 53.6% 37.8% 50.9% I am related some other way 19.1% 18.5% 18.1% 20.9% They live in the same building 68.9% 63.9% 67.3% 76.7% They do not live in the same building, but close enough 9.8% 8.3% 10.0% 11.6% Up to a 10-minute drive apart 7.1% 6.5% 10.9% 4.7% Up to a half-hour drive apart 9.3% 16.0% 5.5% 3.9% Up to a one-hour drive or more apart 4.9% 5.3% 6.4% 3.1% Number of daily tasks, average 9.2 8.8 10.2 8.4 Weekly number of hours of care [average, median] [41.2; 28] [36.3; 24] [51; 35] [36.4; 24] Little or no burden 24.3% 18.2% 32.8% 22.6% den Mild to moderate burden 46% 4.5% 47.7% 45.1% Bur Moderate to severe burden 26.5% 31.8% 18.8% 28.4% Severe burden 3.2% 4.7% 0.8% 3.9% 219 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE more than a quarter were moderately to severely M12 63.2%). More than half of the respondents burdened (26.5%), and less than five per cent of the also listed maintenance of care accessories, such surveyed informal carers felt severely burdened as wheelchair, walker, hearing aid (M0 54.5%; M12 (3.2%). We have detected that subjective burden is 54.4%), and half of them feeding (M0 42.6%; M12 statistically significantly related to the type of pilot 50.0%). About four-tenths of carers (M0 38.6%; environment (ᵪ2=16.017, p=0.014). For example, M12 41.1%) helped with washing the whole body more informal carers felt severely burdened in Celje and bathing, and a similar proportion helped with (4.7%) compared to Krško (3.9%) and Dravograd washing and ironing clothes (M0 43.9%; M12 39.7%) (0.8%), while there were more carers with little or no and/or in managing finances (M0 46.7%; M12 burden (32.8%) in the Dravograd pilot environment 33.9%), such as paying bills. We should also mention compared to Krško (22.6%) and Celje (18.2%). minor housework or renovations and gardening (M0 39.3%; M12 29.8%), shopping, purchasing food, household goods, cleaning products, etc. (M0 44.1%; How pilot activities have M12 28.1%) and financial support, such as giving affected the lives of informal money (M0 20.0%; M12 19.6%). carers An analysis of changes in the intensity of tasks in one year shows that the proportion of the We determine below to what extent the surveyed informal carers who performed tasks assistance of formal care or involvement in the pilot every day decreased in most tasks. The proportion projects has contributed to changes in the quality related to shopping, purchasing of food, household of life of informal carers. The presented results goods, cleaning products, etc. decreased statistically are based on a sample of 58 informal carers who significantly (Z=2.288, p=0.022), by 16 percentage completed a questionnaire in the first assessment points, and the proportion related to assistance in of the eligibility of the person they provide care purchasing and taking medications decreased by to and again after one year of involvement. The 15.2 percentage points, where the decrease is not results related to this sample should be read, if by statistically significant (Z =-1.708, p=0.88). In the individual environments at all, with methodological latter, the share decreased the most in Dravograd reservation and only illustratively, as the sample of (by 35.3 percentage points). The proportions also informal carers who answered the questionnaire in decreased in relation to management of finance two measurements is small. (by 12.7 percentage points), bed making, cleaning If we initially focus on the type and frequency of bedrooms (by 11 percentage points) and of help and tasks as part of care provided by organisation of various forms of assistance, e.g. informal carers on a daily basis, even after one making a doctor appointment, contacting home year, these are still instrumental activities of daily assistance service, community nursing service and living. The most frequent were, for example, similar (by 9.9 percentage points). dishwashing (M0 82.5%; M12 80.7%), cooking, On the other hand, a smaller proportion of the helping to prepare meals or delivering food (M0 surveyed informal carers more often helped with 82.8%; M12 79.7%), putting to bed and getting out feeding (7.4 percentage points), use of toilet and of bed (M0 68.5%; M12 66.7%) and home cleaning bathroom (5.5 percentage points) and washing of and rubbish removal (M0 71.2%; M12 65.5%). This the whole body and bathing (2.5 percentage points). was followed by assistance in purchasing and Informal carers who completed the taking medications (M0 78.9%; M12 63.8%), in using questionnaire at both points of time reported toilet and bathroom (M0 58.2%; M12 63.6%), in an average of 9.5 different daily tasks in M0 and dressing and undressing (M0 65.5 M12 63.6%) and 8.2 in M12, with the difference being borderline bed making and cleaning of bedrooms (M0 74.1%; statistically significant (t=1.953; p=0.056). As for EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 220 the environments, in Dravograd, where carers In the questionnaire, we also asked informal performed the most daily tasks on average, the carers how burdened they generally feel. In the difference decreased the most, from 11.3 to 9.4. second measurement, they responded slightly A comparison of data regarding the objective more positively, as after a year of participation burden of those informal carers who responded in pilot activities they felt less burdened on to the questionnaire at both points of time shows average (3.5) than when entering pilot activities that they provided an average of 52.6 hours (3.2), although the difference is not statistically of assistance per week upon entering the pilot significant. project, with the median being lower: 42.5 hours. At the same time, we should not overlook After one year, informal carers performed an the potential impact of the Covid-19 epidemic average of 44.9 hours, and the median somewhat both on the type and frequency of care and on decreased to 32.5 hours. Although no statistically the well-being and burden of informal carers. significant change can be confirmed, the data show Approximately half of the respondents (28 out of that the number of hours of care in all three pilot 58) answered the questionnaire after one year environments decreased on average. It decreased of inclusion and precisely during the time of the the most in the Celje pilot environment (by 9.3 epidemic. However, based on control questions hours per week on average), then in Dravograd (7.8 we find that the Covid-19 epidemic did not have hours per week on average), and the least in Krško a significant impact, at least on the measurement (3.1 hours per week on average). of objective burden, which cannot be said for the A large standard deviation (SD>30) was measurement of subjective burden, as we did not recorded. The latter tells us that informal carers specifically control it. provided very different scopes of assistance, which In addition to the impact of the pilot activities is also reflected in the span of hours devoted to on the intensity of the burden, we were also care: from two hours to 112 hours per week. In both interested in how informal carers were relieved measurements, the surveyed informal carers from by the assistance as part of the project and what the Dravograd pilot environment were the most changes they have noticed since they joined burdened in terms of the number of hours of care, the project. Help received within the project which is also related to the number of different allowed some of the interviewed carers to have tasks, which was also the largest in Dravograd. a little more time for themselves and for other There is a positive correlation between the number tasks besides care. It has made it easier for those of hours of care and the number of tasks (r=0.58). who are employed to balance work and care by Data on the subjective burden of those enabling them to take less sick leave or annual informal carers who answered the questionnaire leave. They were less concerned and had a greater both at the beginning of the project and after sense of security because the care recipient was one year of involvement show that, on the one receiving professional care and was being taken hand, the proportion of those who felt little or no care of during their absence. burden increased (from 19% to 26.3%), while the Several informal carers were also physically proportion of those who felt severely burdened relieved by the assistance, which improved their also increased (from 5.2% to 10.5%). The share of well-being and alleviated possible health problems the latter increased markedly only in the Celje pilot (e.g. fewer difficulties with back pain). For those environment (by 11.5 percentage points). In general, with a reduced social network, the arrival of we can say that the subjective burden did not change employees on the project also represented stronger significantly during the project, while we again point social contacts. A small proportion of carers to the small sample of informal carers concerning mentioned that the free assistance as part of the whom we were able to monitor this change. project had significantly relieved them financially. 221 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Figure 1: Comparison of the frequency of the provision of assistance or doing chores for another person upon entering the project (M0) and after one year of the provision of services (N=58) % 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 CELJE KRŠKO DRAVOGRAD M12 M12 M12 M0 M0 M0 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 222 Informal carers (N=58) assessed as an (57), where lectures were dispersed over different important contribution of the project the municipalities (Dravograd, Kotlje, Mežica, Ravne acquisition of new knowledge about appropriate na Koroškem, Kotlje, Črna na Koroškem) in order care, as with the help of employees on the project to get as close as possible to the living spaces of they gained much information and knowledge informal carers and thus make it easier for them about care that they did not have before. The to participate. They state that 258 different people employees explained to them and showed them attended the lectures and that 25 informal carers the appropriate methods of work (e.g. on proper were trained, attending more than 80% of the relocation, sitting, caring for pressure ulcers, lectures. The lectures were attended by three to 38 etc.). Approximately two-thirds of informal carers people, with an average attendance being 12 people. (64.6%) reported that they had received advice on The majority of the participants thus attended only how to properly care for the person they provide certain lectures that were related only to those care to as part of the pilot project (61.9% in Celje, topics that interested them. 63.6% in Krško and 68% in Dravograd). A similar In Krško, all lectures were organised in the proportion of respondents stated that they had only municipality where the pilot project was received information about who they could turn to implemented. In total, 18 training sessions were for help in the local environment (e.g. associations, conducted. The events were attended by varied respite care options, self-help groups, etc.) so that numbers of participants, ranging from two to 22, they could be relieved of the burden of care of with the average attendance being five. The events the person they cared for (47.8% in Celje, 77.8% in were attended by 28 different people. Krško and 75% in Dravograd). In Celje, the events were organised only in The pilot environments paid special attention the municipality of Celje, and not in the other to the training of informal carers, which took place municipalities that were included in the pilot between September 2019 and September 2020.93 project. A total of 17 training sessions were The training covered a variety of topics, from more conducted, which were attended by two to 16 general ones concerning long-term care to very people, with the average attendance being seven specific topics relevant to the provision of informal people per one training session. care. The training providers adapted the content Self-help groups were also set up in all three of lectures in accordance with the needs detected pilot environments, and the dynamic of their work by the long-term care coordinators in the pilot varied. In Celje, the self-help group started working environments and in accordance with the interest just before the first wave of the Covid-19 epidemic. of the participants. The most desirable topics for the By the end of August 2021, they had met four times. interviewed informal carers were related to user In Dravograd and Krško, the self-help group was care (proper relocation, changing clothes) and the established as soon as at the end of 2019, and it met option to test the acquired theoretical knowledge six times until the end of the pilot projects. in practice. Resolving their dilemmas from practice The key limiting factors in the participation (either from the aspect of care or relationship with in training and lectures were the distance from the users) was crucial for them. The pilot environments place of training, transportation to organised events also organised other training for informal carers, and the lack of time of informal carers, as well as and in selecting topics they tried to follow the needs a lack of people in their social network who would perceived in the field. substitute for their care duties during their absence. In total, the largest number of training sessions Also related to the lack of time is, for some, the were organised in the Dravograd pilot environment unsuitable timing of training sessions, which took 93 The Ministry of Health has selected the Faculty of the Health Sciences of University of Ljubljana as an external contractor for this activity, which carried out the training in cooperation with partners. 223 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Table 2: Objective and subjective burden of the surveyed informal carers, comparison between M0 and M12 (N=58) Point of mea- surement Total Celje Dravograd Krško Number of daily tasks, M0 9.5 8.5 11.3 9.1 average M12 8.2 7.9 9.4 7.5 Weekly number of hours M0 [52.6; 42.5] [51.3; 40] [61.8; 50] [40.4; 40] of care [average, median] M12 [44.9; 32.5] [42; 35] [54; 40] [37.3; 32] Little or no burden 19.00% 15.40% 31.30% 12.50% Mild to moderate burden 44.80% 42.30% 50.00% 43.80% M0 Moderate to severe burden 31.00% 34.60% 18.80% 37.50% Severe burden 5.20% 7.70% 6.30% Little or no burden 26.30% 19.20% 46.70% 18.80% Mild to moderate burden 31.60% 26.90% 26.70% 43.80% M12 Moderate to severe burden 31.60% 34.60% 26.70% 31.30% Severe burden 10.50% 19.20% 6.30% EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 224 place at times when a user needs a large amount of by them. That informal care is very widespread help, while some were deterred from committing is illustrated by the fact that three-quarters of to participate by the continuity of the training. The the persons evaluated in the pilot project have at idea was that they attend all the training sessions, least one person who provides informal care and or at least most of them. Individuals also refused to assistance to them. A high incidence of informal participate due to their own obstacles and physical care is also a common feature of long-term care in problems. All the interviewed informal carers other European countries. welcomed the training, and only a minority thought The care professions in the field of long-term that they had enough experience with long-term care, including informal care, are strongly marked care and did not need training. by the gender dimension, as women predominate Let us conclude with the information that in these professions, which puts them in an unequal the vast majority of the surveyed informal carers position compared to men. The predominance of (N=58) generally assessed their involvement in women in informal care was also confirmed by the pilot projects as useful (31.4%) or very useful the pilot projects, in which women represented (58.8%). A small proportion of the surveyed as well approximately two-thirds of the informal carers. as the interviewed informal carers did not notice Gender equality is therefore a very topical issue any major changes since joining the project or did for Slovenia in this field and is strongly related to not see the inclusion as useful. These were usually both employment policy (participation in the labour those who had previously had certain forms of market) and retirement (years of active life). formal assistance (e.g. home help) or those whose Although no informal carers under the age care recipients received a minimum number of of 18 were identified in the evaluation of the hours of new services. pilot projects, these special and hidden groups of We also asked the interviewed informal informal carers should not be overlooked in policy carers what changes in their lives they expect making. A recent international survey94 (Santini et after the completion of the pilot projects. The vast al., 2020) shows that the phenomenon of underaged majority would like the project or a similar type carers is quite widespread, reaching approximately of assistance to continue. The limited duration of 8% of young people in Europe who provide the project has put them in an uncertain position. intensive care to a family member. The authors note After the completion of the pilot project, a large that long-term care providers and policy makers proportion of them had to look for other forms should aim to help underaged carers to maintain of assistance, some of them already received the intergenerational emotional bonds with older social care service of home help during the project family members (they most frequently provide or decided to have such assistance later. All other care to their grandparents), while protecting them existing forms of assistance must be paid for by from the negative consequences of inappropriate users and will place an additional financial burden responsibilities that may endanger their overall on some interviewees. health and well-being. An important aspect that we have explored as part of the evaluation was the burden of informal Discussion with key carers. We looked at this subject from the aspects of messages objective and subjective experiences. We found that the objective burden of informal carers decreased Informal carers are an important link in long- during the pilot project, by which we may confirm term care because, as it is evident from the relevant that the quality of their lives has also improved literature, a large part of care is provided precisely somewhat. The result thus shows that in the existing 94 Slovenia was also included in the survey. 225 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE organisation of long-term care, organised home one year), which is one of the limitations of our care represents too small a relief for informal care, research, i.e. the evaluation of the pilot projects. or that there is still a lot of manoeuvring space However, we have identified many relief available in which to strength cooperation between factors in terms of care. Factors that make informal the two types of care. Intensive cooperation or carers happy that they provide care or motivate integration of formal and informal care can lead to them internally are very important (Naiditich et al., positive effects in the objective relief of informal 2013). Concrete support in terms of contributing carers and the pilot activities have had this impact. (additional) hours of care for the person to whom Organised care in the homes of users is an they provide care is also important. In addition, opportunity to relieve informal cares both in an important factor in relieving the burden is terms of actual relief, with, for example, a formal shown to be the option of taking annual leave (e.g. provider performing certain tasks instead of an at least 14 days), while the person they care for is informal carer (as we have seen, the intensity provided with respite care in a care home, another of certain tasks carried out by informal carers institution or at home. We do not possess the data decreased during the project, for example, in on the number of informal carers who actually used purchase of food, assistance in taking medications, the option of respite care or daily care as part of etc.) as well as in terms of providing support to the pilot project, while we find on the basis of the informal carers by means of an expert teaching available material that there were very few of them. them how to correctly perform certain tasks (e.g. Occasional respite care of a few hours feeding, using toilet and bathroom, washing) i.e. is also exceptionally important for informal actionable knowledge (Rosenfeld, 1989; Čačinovič carers, so that they can carry out certain tasks Vogrinčič, 2002). in peace during this time or, for example, attend If the quality of life is perceived from the education or training sessions or meetings of point of view of subjective burden, in general self-help groups that they need and appreciate, we cannot say that it has improved for informal but often experience as a burden. In this regard, carers in the year of the implementation of the informal carers need support in particular in pilot activities. We have already mentioned the transportation, coordination of appointments, and potential impact of the Covid-19 epidemic. In provision of respite care and so events should be addition, the range of different aspects of the organised in their vicinity. Informal carers are burden, both from the point of view of self-care often older people who take care of their partners and from the point of view of the person for and who need transportation in order to attend whom informal carers provide care, is so wide training sessions; it would be easier for them to that some types of subjective burdens can be attend events if they were organised in their local eliminated more quickly with an intervention community. On the other hand, when it comes to of formally organised assistance (for example, informal carers who are employed, the flexible time, coordination, financial burden, etc.). Others timing of events is also important. are more difficult to eliminate, however, as they If we as a country want to overcome the depend on the personality of the informal carer strongly anchored gender inequality in the and the relationship between them and the person care professions and pursue the goal of making to whom they provide care (for example, there informal care a voluntary choice and not a are goal conflicts (Kindt, Vansteenkiste, Cano, & necessity or the only emergency exit, there is Goubert, 2017)), which should be explored to a a need to better support informal carers with greater extent in further research. At the same adequate measures on the one hand and to time, the subjective burden is the one that is more significantly strengthen organised long-term home difficult to change in the short term (for example, care services on the other. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 226 BIBLIOGRAPHY Baji, P., Golicki, D., Prevolnik-Rupel, V., Brouwer, W. B. F., Zrubka, Mosquera, I., Vergara, I., Larranaga, I., Machon, M., del Rio, M. & Z., Gulácsi L. in Péntek, M. (2019). The burden of informal Calderon, C. (2016). Measuring the impact of informal elderly caregiving & Hungary, Poland and Slovenia: results from caregiving: a systematic review of tools. National Center for national representative surveys. The European Journal of Health Biotechnology Information, U.S. National Library of Medicine. Economics, 20, 5–16. Retrieved from : https:/ www.ncbi.nlm.nih.gov/pubmed/26475138 (1. 10. 2019). Boyatzis, R. E. (1998). Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: Moya-Martinez, P., Escribano-Sotos, F. & Pardo-Garcia, I. (2014). Sage. Estimating Total Informal Care Costs in Spain. Can Formal Care Reduce it? Ageing International, 39, 403–420. Braun, V. in Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 2, 77–101. Nagode, M. & Lebar, L. (2019). Trends and challenges in long-term care in Europe. Revija za socijalnu politiku, 26, 1, 255–262. Colombo, F., Llena-Nozal, A., Mercier, J. in Tjadens, F. (2011). Help Wanted? Providing and Paying for Long – Term Care. OECD Naiditich, M., Trifantafillou, J., Di Santo, P., Carretero, S. & Hirsch Health Policy Studies: OECD Publishing. Durrett, E. (2013). User Perspectives in Long-Term Care and the Role of Informal Carers. V K. Leichsenring, J. Billings in H. Nies Creswell, J. & Plano Clark, V. (2007). Designing and Conducting (ur.), Long-Term Care in Europe. Improving Policy and Practice Mixed Methods Research. Thousand Oaks, CA: Sage. (str. 45–81). London: Palgrave Macmillan. Čačinovič Vogrinčič, G. (2002). Koncept delovnega odnosa v Rodrigues, R., Schulmann, K., Schmidt, A., Kalavrezou N. & socialnem delu. Socialno delo, 41, 2. Matsaganis, M. (2013). The indirect costs of long-term care. Research note 8/2013, European Commission. European Commission, Directorate-General for Economic and Financial Affairs and Economic Policy Committee. (2016). Rosenfeld, J. M. (1989). Emergence from Extreme Poverty, Paris: Joint Report in Health Care and Long-Term Care Systems Science et Service, Fourth World Publications. and Fiscal Sustainability and its contry reports. Brussels: European Commission. Retrieved from : https:/ op.europa.eu/ Santini, S., Socci, M., D’Amen, B., Di Rosa, M., Casu, J., Hlebec, V. en/publication-detail/-/publication/d6042a45-b535-11e6-9e3c- & Hanson, E. (2020). Positive and Negative Impacts of Caring 01aa75ed71a1/language-en among Adolescents Caring for Grandparents. Results from an Online Survey in Six European Countries and Implications for Graessel, E., Berth, H., Lichte, T. & Grau, H. (2014). Subjective Future Research, Policy and Practice. International Journal of caregiver burden: validity of the 10-item short version of the Environmental Research and Public Health, 17, 6593, 1–16. Burden Scale for Family Caregivers BSFC-s. BMC Geriatrics, 14, 23. Saldana, J. (2012). The Coding Manual for Qualitative Researchers. Hayes, N. (1997). Theory-Led Thematic Analysis: Social Identification London: Sage. in Small Companies. V Hayes N. (ed.). Doing Qualitative Analysis in Psychology. Hove, UK: Psychology Press. Spasova, S., Baeten, R., Coster, S., Ghailani, D., Peňa-Casas, R. & Vanhercke, B. (2018). Challenges in long-term care in Europe. A Hérbert, R., Bravo, G., & Préville, M. (2000). Reliability, validity, and study of national policies 2018. European Social Policy Network. reference values of the Zarit Burden Interview for assessing Brussels: European Commission. informal caregivers of community-dwelling older persons with dementia. Canadian Journal on Aging, 19, 494–507. Tjadens, F. & Colombo, F. (2011). Long-term care: valuing care providers. Eurohealth 17 (2-3), 13-17. https:/ silo.tips/download/long- Hoffmann, F. & Rodrigues, R. (2010). Informal carers: who takes care term-care-valuing-care-providers of them? Policy Brief 4/2010. Vienna: European Centre. Retrieved from : https:/ www.euro.centre.org/publications/detail/387 (1. 10. Verbeek-Oudijk, D., Woittiez, I., Eggink, E. in Putman, L. (2014). Who 2019). cares in Europe? A comparison of long-term care for the over-50s in sixteen European countries. Haag: The Netherlands Institute Huber, M., Rodrigues, R., Hoffmann, F., Gasior, K. & Marin, B. (2009). for Social Research. Facts and Figures on Long-Term Care. Europe and North America. Vienna: European Centre for Social Welfare Policy and Tur-Sinai, A., Teti, A., Rommel, A., Hlebec, V. & Lamura, G. (2020). Research. How Many Older Informal Caregivers Are There in Europe? Comparison of Estimates of Their Prevalence from Three Kahlert, R., Boehler, C. & Leichsenring, K. (2018). Monitoring European Surveys. International journal of environmental and Evaluating Integrated LTC Models. Vienna: Euro Centre research and public health, 17, 24, 9531. Publication. Zigante, V. (2018). Informal care in Europe. Exploring Formalisation, Kindt, S., Vansteenkiste, M., Cano, A. & Goubert, L. (2017). When is Availability and Quality. Luxembourg: Publications Office of the your partner willing to help you? The role of daily goal conflict European Union. Retrieved from: https:/ publications.europa.eu/ and perceived gratitude. Motivation and emotion, 41, 671–682. en/publication-detail/-/publication/96d27995-6dee-11e8-9483- 01aa75ed71a1/language-en. 227 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE ELECTRONIC MANAGEMENT OF PROCEDURES AND SERVICES AND INFORMATION SYSTEM SUITABILITY Boris Majcen Institute for Economic Research Valentina Prevolnik Rupel Institute for Economic Research EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 228 ELECTRONIC MANAGEMENT OF PROCEDURES AND SERVICES AND INFORMATION SYSTEM SUITABILITY KEY MESSAGES ▶ The activities implemented within pilot projects were recorded accordingly and the data obtained was entered in the information system. The time lag between the data acquisition and the possibility of its entry in the information system, insufficient control of a complete capture of the data required for an individual form and insufficient control of the obtained data entry in the information system by the pilot project contractors were the reasons why certain data was not captured at the end of the project. ▶ The electronic recording system developed within pilot projects is suitable in the transition towards the implementation of the systemic act on long-term care. ▶ The experience of pilot projects demonstrated that a timely construction and operation testing of an information system before the start of operations of the new long-term care system is exceptionally important. ▶ We propose that, prior to a public call for the selection of the best bidder to develop and maintain the entire information system, the Contracting Authority prepares a detailed analysis of the required databases, their content, connectivity, manner of access and minimum extract requirements for the needs of ongoing monitoring of the functioning of the long-term care system, quality control and analysis of implementing services, including the implementation of scientific research in the field of long-term care based on the already obtained experience in the development of the information system within the framework of pilot projects. ▶ We propose that the solutions already drafted as part of the pilot projects be incorporated in the public call for the selection of the best bidder to develop and maintain the entire information system. ▶ It is mandatory to establish a suitable and continuous control system for collecting and entering the data required in the system. 229 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 228 Introduction Methodology Information support for long-term care The evaluation objective was to monitor the processes connects providers of formal and course of electronic recording of the data collected informal care through digital communications about the activities implemented within the pilot tools and programme tools with data. These projects, the suitability of the developed recording systems not only include data on the users’ system for integrated provision of long-term care, medical condition but are also intended to improve and the preparation of adjustments regarding the interaction between all care recipients and to place electronic management of procedures and services the care recipient at the centre of care (Kushniruk in the field of long-term care. The following & Borycki, 2017). In long-term care, relations indicators were determined in the evaluation: and the flow of information between users and ▷ the share of all data collected during the pilot doctors, nurses or carers are very important. Each activities, which were recorded accordingly and change in the user’s condition reflected by the data entered in the electronic database, whereby a entered may lead to changes in the comprehensive 100% capture was considered the success standard. treatment process; its quality is thus closely linked In doing so, we supervised the implementation of to accurate and timely entry of suitable data, recording and entering of obtained information accessibility, and interaction and communication into the accordingly developed information system with everyone involved in the comprehensive by the assessors, long-term care coordinators and treatment (Krick et al., 2019). Digital infrastructure service providers, which was collected on the is recognised as one of twelve key components in basis of the activities implemented by pilot project the SCIROCCO Maturity Model for Integrated Care contractors and service providers; (Scirocco, 2021). ▷ the time lag from the acquisition to the entry of Within the framework of pilot project obtained information in the electronic database, assessment, the evaluation of electronic whereby information on beneficiaries was entered management/recording of procedures and services in the database immediately or the option of in the field of long-term care takes place, including using the databases created had to be provided, the evaluation of suitability of the information which were prepared by various providers when system developed through pilot projects as a implementing pilot projects. The scope and time basis for the development of a new system of of entry of acquired information in the electronic long-term care in Slovenia. Throughout the entire database at the single entry point were checked for project, the development of the programme tool this indicator; was monitored, which enabled the recording ▷ the assessment of electronic data recording of information collected through implemented within the implementation of pilot activities, activities in the electronic database. All options whereby we assessed whether the recording of data recording and monitoring which were system developed within pilot projects was subsequently developed in accordance with suitable for the transition in implementation of the environment initiatives or in compliance with the systemic act on long-term care. needs displayed were simultaneously available To obtain a better insight into the to all three pilot environments. Information on construction of the information system, we the start of the individual module application in carried out a semi-structured interview with a the system was not monitored by the software representative of the software company on 18 company Aleja Soft d.o.o., which developed the November 2020. We first forwarded the starting programme tool for all environments. points for the interview to the discussion partner, which referred to the review of chronology of EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 230 the information system construction within the ▷ monitoring of the employees’ labour calculations, framework of pilot environments, information on and possible differences in the developed information ▷ entry of the visits conducted, and work system for an individual pilot environment and implemented by providers and users. data accessing, and the assessment of suitability The system functionalities were then of the information system developed within regularly upgraded, which in practice caused pilot projects as a basis for the development of a delays in the entry of data in individual modules. new long-term care system throughout Slovenia. From the viewpoint of evaluation, irregular The interview was recorded, and the discussion development of individual modules prevented the partner also prepared written replies to the assessment of time lags from the acquisition to the questions from starting points. Based on the entry of obtained information in the electronic information required, we were able to assess the database. suitability of the developed data documenting The next major module set, “List of Users”, system. As explained below, it was impossible “List of Employees” and “Statistics” followed, to assess the time lag from the acquisition to the which the provider established on 22 February entry of obtained information in the electronic 2019. These modules were regularly upgraded database because the time lag was for the most by the provider with new options (for further part the result of an ongoing development of details, see table 1), while simultaneously creating individual modules throughout the project and not new tabs/modules as per the needs. Such a work the unsuitability of the already developed software method was the result of the fact that the public (Aleja Soft, 2019; Aleja Soft, 2019a). call (Ministrstvo za zdravje, 2018) failed to define in detail all functionalities which the information system was to provide, since the monitoring of “all” Results electronic procedures was anticipated, but it was not clear which procedures were meant. The need Table 1 displays a timeline of the for updating the information system occurred over development and possibilities of applying time, and the employees in pilot environments individual modules in the information system and the system developer resolved them during for long-term care monitoring. As evident, the the course of the project; the information system majority of requested functionalities of the was also being piloted in the project. In the interim information system was roughly developed by the period until the development of a suitable module end of August 2019. The fact that the provider was or solutions, certain data was collected manually selected only after the pilot projects had already by the employees in pilot environments and commenced further contributed to the delay in the entered in temporary Excel files. development of the suitable programme tool. The As part of the project, the software company pilot environments chose the software company developed a package of two applications, i.e. the on their own, which had already initially led to OSKRBA ONLINE web application and the OSKRBA the delay in the software development as the pilot MOBILE mobile application, a programme tool environments had already started their work. The to support the entire long-term care process.95 first application was established in January 2019 By the end of the project, the software company (the Oskrba.online application), which enabled: developed numerous modules within these two ▷ the creation of a database or a list of users with applications (Aleja Soft, 2019a). The Oskrba Online personal records and a list of employees; web application has the following modules: 95 The description is based on the material, Presentation of the programme package by Aleja Soft d.o.o., which was prepared by the software company. 231 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Application receipt, Eligibility assessment for of services on the assessment day, benefits and long-term care and personal implementation plan, services being already received by the user). Each List of Users, Work review, List of Employees, entered user is allocated a registration number Statistics, Report for the Ministry of Health , Data by means of which they are managed without for evaluation of the Social Protection Institute of their name in reports, statistics and exports. This the Republic of Slovenia and Exports. The Oskrba number is also the only piece of information Mobile mobile application has the following written on the NFC sticker through which the visit modules: Reports, Calculations, Employee accounts, is electronically recorded. Code list, Retroactive signing, Observations, Work The application enables employees of the organisation, Synchronisation. single entry point to make an eligibility assessment The applications enable: a) receipt of the regarding long-term care with the help of an application to enforce the right to long-term assessment tool which has been integrated in the care, b) preparation of an eligibility assessment application since 14 August 2019. Before this date, regarding long-term care with the help of an the assessments were made in Excel and each assessment tool, c) preparation of a personal assessment was saved individually. The application implementation plan, d) preparation of all also enables incorporation of a life story and subsequent eligibility assessments and electronic drafting of an electronic report on eligibility or recording of implementation of long-term care ineligibility of long-term care. services, e) preparation of various reports, The applicant eligible for services is statistics and evaluation studies. Since January transferred to the List of Users tab, which enables 2019, the system has been available and applicable the formation of a list of users, monitoring the in all three pilot environments and is undergoing a use of the starting scope of service hours in an continuous process of updating and adjusting. Both individual category of eligibility for long-term care, applications have five levels of application users entry and reviewing the notes or observations with arranged rights and restrictions regarding relating to a certain user. The list is equipped with access to content and application functions: a number of filters – search filter for users, buttons administrator, long-term care manager/project for displaying Active, Inactive and All users, filters coordinator, long-term care coordinator, single logically linked to the users’ status group, filter of entry point and long-term care providers. The municipality (includes permanent and temporary latter have fewest rights and may record visits and residences), selection of an observed time period tasks, notes, observations and particularities, and and further filtering of users by various dates. On may later review work which they performed. a relevant user level, it is possible to edit, complete and delete data. The Review tab gives an insight into the The OSKRBA ONLINE web current situation in the field (completed visits and application visits underway), including the review and edit of past visits and other tasks and the entering of The employees of the single entry point may new ones. Completed visits include information generate a Personal record of a user in which they on the user, provider of LTC services, date of the enter: a) user’s personal data from the application visit, start/end and duration of the visit, types on enforcing the right to long-term care, b) data of performed LTC services, manner of entering on the user’s related persons, and c) data for the visit, type of LTC service, status of the visit, statistics (marital status, monthly net income, information on notes and the user’s signature. education, number of people in the household, The recording of other tasks, which are part of the housing situation, reason for long-term care, type employees’ work obligations, is also possible. EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 232 The List of Employees is also formed within in the current month displayed by users, days the application. By registering in the database, an and individual tasks, c) Employee accounts, employee receives an identification number by which enables a review of the work done by the means of which they are featured without their employee – option for a review of calculation for name in reports, statistics and exports. In addition the last two months, d) Code lists, which include to basic data, the list also includes contact data, data several codes: codes of users (surname and name, on education, profession, workplace, level of access registration number, address and phone number, to data, daily work obligations and other data. contacts of related persons, agreed long-term care When recording visits and tasks, the application services, frequency of signing, date and assessment also enables the calculation of an employee’s category, whether the user uses e-care and date work. When entering rates of salaries, allowances, of the start of inclusion and completion), codes of benefits and sick leave, a summary table with all employees (enables making of phone calls directly employees is available with accounting data for from the mobile application without exiting it further processing by the accounting service. while the provider is making a visit), codes of Under the Statistics tab, the application services (displays a list of services of long-term supports three sets: a) Logs (monthly log of care care with information on required minimum of an individual user, total in a selected period education, environment of implementation, – total of all users, total in a selected period – all name and description of service, restrictions and providers), b) Report for the Ministry of Health exclusion between services), codes of tasks (all (semi-annual reporting from pilot environments), tasks used by a certain organisation are displayed), and c) Data for evaluation (exports of data on e) Retroactive signing, f) Observations, g) Work users, employees, visits, tasks and messages on organisation, and h) Synchronisation (enables work organisation). All three sets were updated the transmission of new data or changes from a according during the project as per the needs of remote server to and from the mobile application the pilot environment, the Contracting Authority and mobile application updates). and assessors. Discussion with key The OSKRBA MOBILE mobile messages application Experience obtained by the software company The OSKRBA MOBILE mobile application during the development of the information is intended for long-term care providers for system reveals a difference in understanding the electronic recording of completed long-term care applicability of the information system in individual services and other tasks. The application has the pilot environments. The Krško pilot environment following modules: a) Reports, which enables was thus consistently very interested in the the recording of visits and tasks through the development and suitable content of the software NFC sticker or a manual entry (user’s name and tool and, as a result, it intensively cooperated surname, date of visit, start, end and duration with the provider, while the remaining two of the service conducted, option for recording environments usually followed the development the delivery of lunch, entering of possible notes passively and accepted new functionalities of the and observations and the user’s signature, daily information system. On the one hand, this made review of conducted visits and tasks in the current the provider’s work easier because they only month), b) Calculations, which displays total followed the requirements of one environment and time of conducted visits, tasks and kilometres the development of the information system was 233 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE subsequently uniform and completely identical To ensure optimal data monitoring and for all three environments. On the other hand, the documenting, the software company carried out system may have only represented a necessity and a series of training sessions for individual pilot obligation for the passive environments, thereby environments concerning the application of the failing to encourage in those environments a software tool and separate sessions for individual consideration of the possibility of using the system users. Training sessions for five persons at the for their own needs and possible further analyses. single entry point (2 hours), two coordinators During the pilot projects, we determined (3 hours) and 14 long-term care providers (4 that some data on the applicants regarding their hours) were implemented for the Dravograd enforcing the right to long-term care was not pilot environment. Training sessions for seven entered in the information system until the end of persons at the single entry point (2 hours), two the project or certain data on individual applicants coordinators (3 hours) and 15 long-term care was missing. This points to the fact that the control providers (4 hours) were implemented for the of data entry into the information system was Celje pilot environment. Training sessions for four deficient. Deficiencies were also revealed in the persons at the single entry point of the Krško pilot inconsistent completion of the Application for environment took place by e-mail. Sessions were enforcing the right to long-term care or the form also carried out for one coordinator (3 hours) for the Eligibility assessment for long-term care or and eight long-term care providers (3 hours). inconsistent entering of data into the information Instructions for the use of both applications were system, i.e. certain important pieces of information also produced. were not entered: level of education (18.1%), income As mentioned above, a simultaneous (23.9%), number of people in the household (5.3%), development of the information system enabled marital status (6.8%), housing situation (27.2%), year the evaluation of the second indicator, i.e. a of birth (4.3%), reasons for needing long-term care time lag from the acquisition to the entering of (12.2%) and the type of care the applicant chose obtained information in the information system. (57.6%). The reason for the insufficient completion Information regarding the installation of of the application under certain variables was that the system in pilot environments and the start of the applicant was already receiving institutional application of individual modules is scarce; all care, and the resultant application was thus not three environments were enabled simultaneous considered incomplete as the desired data was access to new functionalities of the information simply not selected. system and the software company did not specify At the end of the project, we can assert that when application users in the individual pilot the activities within the implementation of the pilot environments should have actually started using projects were recorded and entered accordingly. the individual modules. Their experience reveals The reason why certain data was not captured significant differences between providers in could be attributed to the time lag regarding the individual environments regarding the entry and data obtained and the option to record it in the application of the system. They further added that information system. Reasons could also be sought the Krško pilot environment was the first to start in insufficient supervision of a complete capture using the system to the largest possible extent. In of required data in an individual application and the Dravograd pilot environment, the recording the supervision of the entry of acquired data in the of activities by means of manual entries lasted in electronic database by pilot project providers. certain cases until the end of the project.96 96 Two further added services were managed manually in the Dravograd pilot environment (music therapy and delivering food to the users involved in the project – the Municipality of Ravne in cooperation with a Lions Club). They obtained the consent of the Ministry of Health for the services agreed on and implemented separately, which were a particularity of the pilot environment, while other LTC services were entered in the information system. EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 234 Table 1: Description of milestones in the development of individual modules and additional application possibilities in the Oskrba Online application Date of es- tablishment Tab What First function- Creation of a database or a list of users with personal records is enabled Creation of a database or a list of employees is enabled January 2019 alities and launching of the Employee accounts can be monitored application Enabled entry of conducted visits and tasks; enabled preparation of Reports/ Calculations/Extracts on conducted visits and work done by providers and users 21 February 2019 Users Option for entering assessment results, assessment date, eligibility for assessment, long-term care category and marking of an e-care use The levels of a single entry point and a long-term care coordinator are activated 21 February 2019 Employees Profession of a registered nurse is added Determining care units and independence maintenance units at the level of a provider 21 February 2019 Statistics Report for the Ministry of Health may be generated 22 March 2019 Users Related persons tab is added to the personal record Employees: Expansion of rights to work at the level of a single entry point: entry of a new Levels of a single user, editing of the entry, deleting and completing of a user’s personal record 22 March 2019 entry point and is enabled a long-term care Editing of the user’s personal record is also enabled at the level of the long- coordinator term care coordinator Display of registration numbers of employees and users (data on/off) is 22 March 2019 Lists of Users and enabled – Manager, project coordinator and long-term care coordinators see Employees identification numbers of all employees, while the providers see only their own identification numbers 26 March 2019 Users Under the “Statistics Data” tag in the personal record, it is possible to tag a user who is already using home assistance services The Observations tab is added, which enables the generation of new ones 28 May 2019 Users and reviewing of observations by date levels The system for tracking more than two assessments is introduced in the personal record 7 June 2019 Users Various search and classification filters are activated on the List of Users Printing directly from the browser is enabled: from the user’s personal record, 13 June 2019 Printing the List of Users, the List of Employees, the Employee accounts, review of visits and tasks A separate Assessment tab was added in the personal record by entering: - first assessments (all three categories) - repeated assessments (all three categories) - in the “LTC category”, the programme automatically selects the highest 20 June 2019 Users category among the assessments of module 4 and module 5 as per the NBA assessment tool - under the selected “special provision”, all three assessments change into the highest All observed statistical items are added under the “Statistics Data” tag in the personal record Multi-level filters and “Quick information” with the option of displaying the 20 July 2019 Users assessor’s name, number of performed assessments, display of a warning regarding a required regular repeated assessment are added on the List of Users 25 July 2019 Review of visits Display of a total sum of services carried out by types of services and care units or independence maintenance units is enabled under the Review of visits 235 EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 235 Date of es- tablishment Tab What Several logs are added under the Logs tab: - Monthly log of user’s care in the PDF format (review of services implemented within care units or independence maintenance units) 29 July 2019 Statistics - Monthly log of user’s care in the XLSX format (detailed breakdown by services, by units) - Calculations for selected periods (total of all users) in the PDF format displays visits conducted by workdays, Sundays and holidays. In the XLSX format, visits by services and units are further displayed. Activation of the Use of hours: enables the monitoring of the use of hours 29 July 2019 Users from the fund by users Option of switching between annual (independence maintenance unit) and monthly (unit for care) use of hours Added dates: FIRST ASSESSMENT (application date, date of visit without an assessment 2 August 2019 User/assessment (reason), date of visit, date of assessment drafting, date of letter) REPEATED ASSESSMENT (date of visit, date of assessment drafting, date of letter) Users’ personal records enable: - Entry of another phone number - Selection of more than one reason for the need of long-term care 1 August 2019 Users - Selection of gender when PIN is not entered - Renaming certain fields - Added option of Care category 0 - Introduction of quick information (badges) in the header of the personal record (“User of HH”, “Eligible for LTC”, “User of e-care”, etc.) 1 August 2019 Single entry point The SEP level is enabled access to necessary Statistics and Logs 1 August 2019 Statistics Added log Calculations for selected period by providers in PDF and XLSX formats Export of data for evaluation by the IRSSV is enabled. It may be accessed 5 August 2019 Statistics by the levels manager, project coordinator, long-term care coordinator and single entry point. From the single entry point, it is possible to: 7 August 2019 Single entry point - enter one’s Tasks - generate and edit Observations All levels can enter past observations. Incorporation of the NBA assessment tool in user’s personal record under the Assessment tab: 14 August 2019 Assessment tool - all assessment modules are included - calculation of the NBA assessment of module 4 and module 5 - extract of Report on eligibility assessment - extract of Application Updates and improvements when working in the assessment tool: 19 August 2019 Assessment tool - “living circumstances” may be entered subsequently when editing assessment - Free switching between modules is enabled until the assessment completion - Subsequent entry/edit of replies with mandatory entry of a reason for 21 August 2019 Assessment tool changing replies is enabled - Preparation of a significant number of repeated assessments is supported EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 236 Date of estab- lishment Tab What Added under the Exports tab, which enables exports of data on: - Users (personal data, related persons, observations) - Employees 22 August 2019 Statistics - Visits - Tasks - Messages on Work organisation Access to Exports is enabled to levels manager, project coordinator and long- term care coordinator. The “Personal plan” tag is added in the user’s personal record with functions 22 August 2019 Users for adding, editing and deleting the plan. Preparation of the Personal plan report is enabled. - Added fields in the personal field 29 August 2019 Users - In addition to the assessment date, the List of Users also includes a category of long-term care (quick info) - Added reason for concluding “Personal assistant” Supplement to the Report for evaluation of IRSSV Document with three tabs 6 September 2019 Statistics - Data from the table template - Applicants (the “year of birth” column is added at the request of IRSSV) - Living circumstances 11 September 2019 Assessment A selection of two assessors is enabled in the assessment tool. 1 October 2019 Users In the List of Users, the levels of manager, project coordinator and long-term care coordinator can obtain the list of informal providers. Independence 1 October 2019 maintenance Access to the latest assessment permitted unit 1 October 2019 Assessment When editing and viewing the implemented assessment, it is possible to freely switch between steps. User’s personal record: a list of personal plans is a table with expandable 1 October 2019 Users rows in which short-term, long-term and other objectives are added; it is also possible to add other data which may be viewed without entering the personal plan form. In the personal record, it is also possible to edit Related persons for Inactive users. 5 November 2019 Users In the most modules, the assessment scale receives a questionnaire in the form of a survey matrix, questions are in alternating colours, the module title is bolder The Notes field for the long-term care coordinator is introduced under the 6 November 2019 Users Service tab of the personal record; these notes are entered and edited by the long-term care coordinator and are visible to the levels of manager, project coordinator and single entry point. New additions on the List of Users: - new filters classifying the range of those who were not assessed to those 18 November 2019 Users waiting, those waiting with a signature, those waiting without a signature, visits with no assessment - new column for the care category - the Address column enables filtering by permanent/temporary address 21 November 2019 Users Annex to a personal plan may be drafted 5 December 2019 Users The Records of interviews tab is enabled in the personal record. 9 December 2019 Users Functionality for collecting and returning keys is added. 237 EV E AL V U AL A U TION OF P A IL TION OF P O IL T P O RO T P JE RO C JE T C S IN THE FIELD OF L T ONG- S IN THE FIELD OF L TERM C ONG- ARE TERM C 237 Date of estab- lishment Tab What 16 December 2019 Users Numbering of contact persons by priority order is enabled in the Related persons tab in the user’s personal record. 19 December 2019 Statistics Export of personal plans with annexes is enabled 20 December 2019 Users All levels can access and review the Use of hours 20 December 2019 Statistics All users and employees, including providers within the independence maintenance unit, can access the Monthly calculations log 10 January 2020 Users Under the Assessment tab in the personal record, the proportions of long- term care modules are displayed graphically. 13 January 2020 Users WARNING and FILTER for "Six months has passed since the last repeated assessment" were added on the List of Users. 15 January 2020 Statistics Report for the Ministry of Health: new Date of reporting field is introduced. The “number of users” field in the report is linked to the date of reporting. Two fields to be completed, i.e. Telemedicine (has/does not have) and Smartwatch (has/does not have), are added under the Service tab in the 27 January 2020 Users personal record. Ticked field (“has”) appears in the form of a badge in the header of the personal record. The fields also appear among the filters of active users. 27 January 2020 Service code A location can also be determined for a service (which is observed in the Employees’ accounts). 17 February 2020 Statistics New Report for the Ministry of Health with five tabs is available. Personal record: - Deletion of a selected answer in matrix questions is enabled in the 2 March 2020 Users assessment tool - Assessment tag: it is not necessary to enter the assessment tool to review and edit “living circumstances” 17 March 2020 Documents New module which consists of generated reports, documents, logs, applications, etc. 25 March 2020 Review of visits Entry of visit cancellation for a longer period is enabled Recording of extraordinary absences is enabled 27 March 2020 Review of tasks - Covid-19 – furlough - Covid-19 – childcare 29 May 2020 Recording of visits Entry of duration for each selected service is enabled 29 June 2020 Users Attachments tab is added in personal record with the option of adding PDF documents 10 September 2020 Speeding up functioning of the application when opening the user’s personal record The Assessments sub-tab is activated with a list of all conducted First and 8 October 2020 Users Repeated assessments Different sorting and filtering is enabled Data regarding total calculations based on filtered data is added 19 October 2020 Review of tasks Recording of extraordinary absences is enabled - Covid-19 – quarantine EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 238 Due to the ongoing development of the care, it is necessary to add data collected when information system and the activities being carried carrying out the eligibility assessment with the out in individual environments, time lags between new assessment tool to the records mentioned the data being obtained and it being entered into in the act on long-term care. On the other hand, the information system were evident. The data certain records as per the selection required by already obtained was entered into the system in law are missing and will have to be added before the pilot environments (i.e. retrospectively). An transitioning to the new long-term care system: a) example of this is the assessment tool which, by internal control system (indicators of quality and means of its incorporation into the system, enables safety, adverse events and those that could have among other things an automatic calculation of occurred when implementing services and persons the eligibility category. As a result, the software responsible for work processes at contractors) company added the option of importing the and b) records on the occupancy of capacities and eligibility assessments already prepared in number of reservations. Excel so that the assessment forms drafted in From the very start of the development, individual pilot environments were gradually application and upgrading of individual modules, incorporated into the system; such an option which may represent one of the significant would not have been required when preparing sections of the new integrated long-term care information solutions before the start of project information system, the Contracting Authority, i.e. implementation. the Ministry of Health, should have been actively The information system developed within involved in the supervision of system operation the pilot projects covers databases which will be and application. Unfortunately, it was not planned of key importance in the long-term care system. that the Contracting Authority would be able to The developed software includes a web application access data. Similarly, the fate of the information accessible anywhere and fully adjusted to the work system developed within the pilot projects after the in long-term care; it is thus not an adaptation of a completion of those projects was not planned. Six general application which could also be used for months after the completion of the pilot projects, long-term care. All functionalities were developed the software company will cease maintenance specifically for the long-term care application, and of the system. It would be regrettable if a system the entire system was verified and established in developed specifically for use in the long-term care practice. system and which has also been verified in practice Due to the need for analyses that would was simply dismissed as it represents a suitable enable higher quality decision-making by foundation for the development of a long-term competent decision-makers regarding long-term care information system in Slovenia. 239 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE BIBLIOGRAPHY Kushniruk, A.W., & Borycki, E.M. (2017). Usability of Healthcare Ministrstvo za zdravje. (2018). Javni razpis »Izvedba pilotnih Information Technology: Barrier to the Exchange of Health projektov, ki bodo podpirali prehod v izvajanje sistemskega Information in the Two-Sided EHealth Market? . V Vimarlund, E. zakona o dolgotrajni oskrbi«. Retrieved from https:/ www. (ed.). E-Health and Two-Sided Markets. Academic Press London, uradni-list.si/glasilo-uradni-list-rs/vsebina/2018002400003/ 33–41. javni-razpis-za-izbor-operacij-izvedba-pilotnih-projektov-ki-bodo- podpirali-prehod-v-izvajanje-sistemskega-zakona-o-dolgotrajni- Krick, T., Huter, K., Domhoff, D., Schmidt, A., Rothgang, H. & Wolf- oskrbi-st--303-1201816-ob-198618 Ostermann, K. (2019). Digital technology and nursing care: a scoping review on acceptance, effectiveness and efficiency Aleja Soft d.o.o. (2019). Dolgotrajna oskrba – Spletno orodje Oskrba studies of informal and formal care technologies. BMC Health Online v 1.0. Navodila za uporabo. Interno gradivo Aleja soft d.o.o., Serv Res., 19:400. Ljubljana. Scirocco exchange, Maturity model. (2021). Retrieved from https:/ Aleja Soft d.o.o. (2019a). Predstavitev programskega orodja Aleje www.sciroccoexchange.com/digital-infrastructure soft. Interno gradivo Aleja soft d.o.o., Ljubljana. 241 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE AUTHOR INDEX EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 242 A Bravo, Gina 226 Aarts, Sil 192 Brazier, John 195, 210 Abele, Andrea Elisabeth 143, 159 Brouwer, Werner 226 Aeyels, Daan 96 Bruyneel, Luk 96 Allouch, Somaya Ben 182, 189, 190, 192 Burnik, Tomaž 192 Andersson, Stefan 182, 192 Büscher, Andreas 100, 115 Arlotti, Marco 99, 115 C Ausserhofer, Dietmar 143, 159 Cachucho, Ricardo 190, 192 B Calderon, Carlos 226 Baeten, Rita 32, 226 Cano, Annmarie 225, 226 Bagozzi, Richard 189, 192 Carretero, Stephanie 213, 226 Baji, Petra 213, 226 Carrington, Peter J. 151, 159 Banerjee, Jay 29, 32 Carrino, Ludovico 99, 100, 115 Barbabella, Francesco 192 Cartwright, Martin 192 Bartol, Jošt 181, 192 Casu, Giulia 226 Barton, Garry 192 Cès, Sophie 24, 32 Batagelj, Vladimir 144, 159 Chadbon, Neil 32 Bateman, Andrew 192 Ciccarelli, Nicola 23, 32 Beattie, Paul 165, 178 Cigoj-Kuzma, Nika 77 Berberoglu, Aysen 143, 159 Clarke, Victoria 53, 60, 215, 226 Berth, Hendrik 226 Coeckelberghs, Ellen 96 Berzelak, Nejc 181, 192 Colombo, Francesca 24, 32, 213, 214, 226 Billings, Jenny 226 Consel, Charles 190, 192 Blane, David 210 Cook, Erica Jane 181, 192 Boeckxstaens, Pauline 121, 139 Corselli-Nordblad, Louise 27, 32 Boehler, Carsten 40, 60, 139, 196, 210, 214, 226 Coster, Stéphanie 25, 32, 226 Boeije, Hennie 192 Coulson, Sherry 178, 192 Boeykens, Dagje 121, 139 Crawford-White, Jane 192 Bolčevič, Slavko 29, 32 Creswell, John Ward 42, 60, 215, 226 Borrat-Besson, Carmen 209, 210 Cullen, Kevin 181, 192 Börsch-Supan, Axel 181, 192 Cylus, Jonathan 24, 32 Borycki, Elisabeth 229, 239 Č Boštjančič, Eva 143, 159 Čačinovič Vogrinčič, Gabi 225, 226 Boulay, Mélodie 190, 192 Čelebič, Tanja 178 Bowen, Robert 192 Černič, Mateja 65, 77, 163, 178 Bower, Peter 192 Črnak Meglič, Andreja 32, 64, 65, 77, 178 Boyatzis, Richard Eleftherios 53, 60, 215, 226 D Brandon, Althea 120, 139 D’Amen, Barbara 226 Brandon, David 120, 139 D'Amour, Danielle 159 Brandt, Martina 23, 32 Davis, Fred D. 189, 192 Bratuž Ferk, Barbara 163, 176, 178 Day, Hy 165, 178, 188, 192 Braun, Virginia 53, 60, 215, 226 de Witte, Luc P. 192 243 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Debevc, Matjaž 192 Graessel, Elmar 214, 226 del Rio, María 226 Grau, Hannes 226 Demers, Louise 178, 192 Grebenc, Vera 77 DeRuyter, Frank 178, 192 Greer, Scott L. 32 Devi, Reena 29, 32 Gregorčič, Marta 178 Dhaini, Suzanne 143, 159 Gruber, Stefan 196, 210 Di Rosa, Mirko 226 Gulácsi, László 226 Di Santo, Patrizia 213, 226 Guppy, Andrew 192 Dolničar, Vesna 32, 181, 182, 188, 190, 192 H Domajnko, Barbara 32 Habets, Herbert 192 Domhoff, Dominik 239 Hagmaier, Tamara 143, 159 Dominkuš, Davor 27, 32, 64, 65, 77, 115, 178 Hanson, Elisabeth 182, 192, 226 Dowda, Marsha 165, 178 Harrison, Michael I. 157, 159 Dremelj, Polona 32, 77, 96, 99, 115, 139 Hayes, Nicky 53, 60, 215, 226 Drole, Janja 77 Hedström, Karin 60, 182, 192 Duffy, Joanne 29, 32 Hérbert, Réjean 214, 226 Dupuy, Lucile 190, 192 Herdman, Mike 210 Džananović Zavrl, Darinka 65, 77, 163, 178 Higgs, Paul 210 E Hinsliff-Smith, Kathyrn 29, 32 Eggink, Evelien 213, 226 Hirani, Shashivadan 192 Emilsson, Ulla Melin 143, 159 Hirsch Durrett, Elisabeth 213 Engberg, Sandra 143, 159 Hlebec, Valentina 23, 32, 213, 226 Erlingsson, Christen Lee 182, 192 Hoffmann, Frédérique 213, 226 Escribano-Sotos, Francisco 214, 226 Hsieh, Yi-Ju 165, 178, 188, 192 F Huber, Manfred 213, 226 Farkaš Lainščak, Jerneja 29, 32 Hujala, Anneli 192 Ferrans, Carol Estwing 195, 210 Huter, Kai 239 Ficko, Katarina 178 Hvalič Touzery, Simona 32, 181, 182, 188, 192 Figueras, Josep 24, 32 Hyde, Martin 195, 210 Filipovič Hrast, Maša 32 I Flaker, Vito 23, 32, 64, 65, 77, 81, 94, 96, 115, 120, 139, 163, 178 Ilinca, Stefania 163, 178 Fuhrer, Marcus J. 178, 192 Istenič, Aleš 32 G J Gasior, Katrin 213, 226 Jacović, Anita 27, 32, 64, 65, 77, 115, 178 Ghailani, Dalila 32, 226 Jakac, Sanja 32 Golicki, Dominik 226 Janssen, Bas 210 Gonçalves, Judite 209, 210 Jaschinski, Christina 182, 189, 190, 192 Goodwin, Nick 64, 77, 181, 182, 192 Jelenc Krašovec, Sabina 32 Gordon, Adam Lindsay 29, 32 Jouen, François 190, 192 Gorše, Anita 144, 159 Jutai, Jeffrey 165, 178, 188, 192 Gorup, Luka 32 K Goubert, Liesbet 225, 226 Kahlert, Rahel 40, 60, 139, 196, 210, 214, 226 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 244 Kajzer, Alenka 178 M Kalavrezou, Niki 213, 226 Machon, Mónica 226 Kampel, Thomas 182, 192 Macq, Jean 121, 139 Karanikolos, Marina 32 Magnusson, Lennart 182, 192 Karimi, Milad 195, 210 Majcen, Boris 24, 32 Kavčič, Matic 182, 192 Mali, Jana 23, 32, 77, 96, 115, 120, 139, 163, 178, 226 Kennedy-Martin, Matthew 210 Marin, Bernd 213, 226 Kennedy-Martin, Tessa 210 Marn, Stane 27, 32, 64, 77, 115, 178 Kindt, Sara 225, 226 Marston, Hannah R. 192 Kobal Straus, Klavdija 32 Matsaganis, Manos 213, 226 Kobal Tomc, Barbara 77, 96, 115, 139 McCullagh, Peter 101, 115 Kodele, Tadeja 32, 77 McKee, Martin 32 Kodner, Dennis L. 64, 77 McWinner, Yawman 143, 159 Kokalj, Andreja 178 Mehrbrodt, Tabea 196, 210 Kolkowska, Ella 60, 182, 192 Melchiorre, Maria Gabriella 182, 192 Koprivnikar Šušteršič, Mojca 77, 178 Mercier, Jérôme 24, 32, 213, 226 Košir, Igor 182, 192 Michener, Lori 165, 178 Kovač, Nadja 65, 77, 99, 115, 163, 178 Mosquera, Isabel 214, 226 Kožuh, Ines 192 Moya-Martinez, Pablo 226 Krainer, Daniela 192 Mrvar, Andrej 144, 159 Kranjc, Bogo 77 Mukuria, Clara 210 Krick, Tobias 229, 239 Mulhern, Brendan 210 Kroll, Michael 192 Müller, Sonja 182, 192 Kubitschke, Lutz 181, 192 N Kushniruk, Andre William 229, 239 Nagode, Mateja 23, 27, 32, 64, 65, 77, 94, 96, 99, 112, 115, 120, 139, Kustec Lipicer, Simona 60 163, 177, 178, 181, 182, 192, 213, 226 L Naiditich, Michel 213, 214, 225, 226 Lamura, Giovanni 213, 226 Nelson, Roger 165, 178 Larranaga, Isabel 226 Newman, Stanton P. 192 Lattacher, Lisa 192 Neyens, Jacques CL 192 Laznik, Jerneja 182, 192 Nieboer, Marianne E. 192 Lebar, Lea 64, 65, 77, 81, 96, 99, 100, 101, 113, 115, 121, 135, 136, Nies, Henk 226 139, 163, 178, 182, 192, 213, 226 Normand, Charles 24, 32 Leichsenring, Kai 40, 60, 139, 163, 178, 196, 210, 214, 226 O Lenker, James 165, 178, 188, 192 Oandasan, Ivy 143, 159 Lichte, Thomas 226 Ogonowski, Corinna 192 Limonšek, Ivanka 32 Orehek, Špela 65, 77, 163, 178 Lipar, Tina 32 Orso, Cristina 99, 100, 115 Llena-Nozal, Ana 24, 32, 213, 226 Österle, August 99, 115 Logan, Winifred W. 120, 139 P Luijkx, Katrien G. 192 Pahor, Majda 32 245 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE Pan, Peter G. 64, 65, 77 Roper, Nancy 120, 121, 139 Panella, Marzia 96 Rothgang, Heinz 239 Papa, Roberta 192, 210 Rudel, Drago 187, 191, 192 Papaioannou, Diana 210 Ryser, Valérie-Anne 209, 210 Pardo-Garcia, Isabel 214, 226 S Parma, Andrea 99, 115 Sagan, Anna 27, 32 Parry, Glenys 210 Saldana, Johnny 53, 60, 215, 226 Parsons, David 36, 60 Sanders, Caroline 182, 192 Pastor, Diane K. 157, 159 Santi, Marco 182, 192 Payne, Roy L. 143 Santini, Sara 224, 226 Peasgood, Tessa 210 Sauzéon, Hélène 190, 192 Peek, Sebastiaan T.M. 181, 182, 189, 190, 192 Schaeffer, Doris 100, 115 Peňa-Casas, Ramon 32, 226 Schmidt, Andrea 213 Péntek, Márta 226 Schmidt, Annika 229 Pereira, Filipa 182, 192 Schulmann, Katharine 213, 226 Perko, Mitja 163, 178 Schwendimann, Rene 143, 159 Peršič, Maruška Lucija 139 Scott, John 151, 159 Peternelj, Andreja 64, 77, 96, 115, 139 Secnik Boye, Kristina 210 Peters, Oscar 190, 192 Seys, Deborah 81, 96 Petrovčič, Andraž 181, 182, 188, 192 Sharp, Chloe 192 Pino, Maribel 190, 192 Simčič, Biserka 32 Pirec, Samo 178 Sipers, Walther 192 Pirnat, Tina 77 Slaap, Bernhard 210 Plano Clark, Wicky L. 41, 60, 215, 226 Smole-Orehek, Kaja 182 Poldrugovac, Mircha 32 Smolej Jež, Simona 65, 77, 96, 115, 139 Préville, Michel 226 Socci, Marco 226 Prevodnik, Katja 182, 188, 192 Söderberg, Maria 143, 159 Prevolnik-Rupel, Valentina 226 Sonola, Lara 64, 77 Putman, Lisa 213, 226 Spasova, Slavina 23, 32, 213, 226 R Spreeuwenberg, Marieke D. 192 Rafaelič, Andreja 23, 32, 64, 65, 77, 94, 96, 120, 139, 163, 178 Stolk, Elly 210 Ramovš, Jože 24, 32 Stone, Patricia 157, 159 Ranci, Costanzo 99, 115 Strandell, Helene 27, 32 Randhawa, Gurch 192 Strid, Anna-Lena 143, 159 Ratajc, Simona 120, 139, 163, 178 Š Rigaud, Anne-Sophie 190, 192 Ščavničar, Ema 120, 139 Rijken, Mieke 192 Šetinc, Mojca 182, 192 Rijnaard, Maurice D. 192 Škafar, Maja 188, 192 Rode, Nino 77, 96, 115, 139 Škerjanc, Jelka 32, 120, 139 Rodrigues, Ricardo 99, 115, 163, 178, 213, 214, 226 Štepic, Marjetka 177, 178 Rogers, Anne 192 Štromajer, Draga 32 Rommel, Alexander 213, 226 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 246 T Williams, Gemma A. 32 Teti, Andrea 213, 226 Wingenfeld, Klaus 100, 113, 115 Thiel, Veronika 64, 77 Woittiez, Isolde 213, 226 Tierney, Alison Joan 120, 139 Wolf-Ostermann, Karin 239 Tjadens, Frits 24, 32, 213, 214, 226 Wouters, Eveline 192 Trbanc, Martina 64, 77 Wulf, Volker 192 Trifantafillou, Judy 226 Z Trkman, Marina 181, 192 Zaviršek, Darja 120, 139 Tsertsidis, Antonios 52, 60, 182, 190, 192 Zigante, Valentina 213, 214, 226 Turner, Christine 165, 178 Zorn, Jelka 120, 139 Tur-Sinai, Aviad 213, 226 Zrubka, Zsombor 226 U Zúñiga, Franziska 143, 159 Udovič, Nataša 64, 77, 94, 96, 120, 139, 163, 178 Zver, Eva 27, 32, 64, 77, 115, 178 Urek, Mojca 23, 32, 139 Ž V Žakelj, Magdalena 32 van der Voort, Claire 192 Žitek, Nina 139 van Dijk, Jan A. G. M. 190, 192 Žnidarec Demšar, Simona 77 van Ginneken, Ewout 192 van Hoof, Joost 192 Van Reen, Mandy 196 van Rossum, Erik 192 Van Soest, Arthur 23, 32 Vandenbroeck, Philippe 121, 139 Vanhaecht, Kris 96 Vanhercke, Bert 32, 226 Vansteenkiste, Maarten 225, 226 VanZelm, Rosalie 96 Vaziri, Daryoush Daniel 190, 192 Verbeek-Oudijk, Debbie 213, 226 Vergara, Itziar 226 Verloo, Henk 182, 190, 192 Vermeulen, Joan 181, 192 Vidal, Nicole 182, 192 Videmšek, Petra 120, 139 Von Rekowski, Thomas 192 Vrijhoef, Hubertus 192 W Wagner, Melanie 23, 32, 196, 210 Warshaw, Paul R. 189, 192 Wasserman, Stanley 151, 159 Webb, Erin 32 Wiggins, Richard Donovan 210 247 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE SUBJECT INDEX EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 248 A concurrent triangulation plan 8, 41 absenteeism 143 consortium, consortium partner 30, 47, 50, 66, 67, 71, 72, 75 Active Ageing Strategy 28, 182 coordination of long-term care services 9, 19, 36, 55, 70, 76, 83, activities of daily living (ADL) 4, 23, 24, 85, 99, 101, 106, 107, 146, 88, 99, 119, 122-124, 167 150, 164, 166, 167, 169, 189, 216, 217, 219 Covid-19 28, 35, 39-40, 42, 44, 49-50, 52-53, 59, 74, 75, 112, 145, 158, annex to the personal plan 50, 55, 82, 89, 118, 122, 134, 137, 236, 180, 183, 184, 188, 190, 191, 208, 214, 220, 222, 225, 237 237 D annual leave 89-90, 212, 220, 225 daily care 225 appeal 29, 91, 180 democratic forum 35, 50, 57-59, 67-68, 83, 90, 101, 105, 114, 112, application for eligibility assessment 18, 19, 30, 38, 41, 42, 81, 127 82-87, 89, 90, 92-95, 101, 102, 109, 158, 166, 233 domestic help, care 24, 216 assessment of eligibility for long-term care 19, 30, 38, 44, 52, 86, duration of the assessment procedure 81, 83, 104, 112 91, 92, 97-114, 119 E assessment scale 30, 98-102, 104, 108, 109, 112, 113, 236 e-care 9, 20, 22, 29, 31, 38, 42-46, 49-52, 72, 82-83, 87, 88, 90, 170, assessor 2, 19, 37, 38, 40-42, 44, 47, 49, 50, 52-54, 57-59, 66-68, 180-190, 232, 234-235 70-71, 74, 75, 83-88, 90-94, 98, 100-102, 104-105, 108-109, 113, e-health 9, 20, 43-46, 49, 82-83, 88, 90, 180-190 114, 122, 125-127, 136, 144-147, 148, 150-151, 154, 155, 157, 200, employee satisfaction 142-147, 149, 158, 164-165, 177 202, 209, 229, 232, 234 employment 27, 38, 50, 52, 57, 62-63, 65-68, 70, 72, 73, 75, 91, 133, assistance and attendance allowance 107-109 143, 214-215, 224 assistive technologies 18, 20, 24, 29, 37, 39, 42, 44, 50-54, 90, empowerment of the user 131, 134, 168 179-191 encryption system 41 Association of Municipalities and Towns of Slovenia 58 EQ-5D 11, 42, 49, 194-196, 209 autonomy 93, 195 European Quality Framework for long-term care services 81 B evaluation training 40 burden of care 20, 41, 44, 49, 52, 182, 183, 212-214, 216-220, experimental research plan 39, 58 222-225 F burden of employee 125, 149 Faculty of Health Sciences, University of Ljubljana 222 C Faculty of Social Sciences of the University of Ljubljana 7, 18, 36 care home 30, 62, 67, 70, 72, 73, 75, 166, 225 fall detector 52, 181, 183, 187 care profession 63, 224, 225 family assistant 65, 163 care team 19, 37, 64, 66, 68-71, 75, 82, 145, 150, 164, 169 family member 22, 94, 209, 213, 217, 224 care unit 73, 89, 123, 126, 136, 149, 151, 154, 155, 158, 234-235 financial burden 224, 225 CASP-12 42, 49, 194-196, 204, 209 financial support 182, 219 category of eligibility 19, 38, 52-53, 86, 89, 92, 98, 101, 104-109, focus group 50, 53-54, 67, 183, 187 112-113, 167, 169-170, 172, 176-177, 188, 207, 215, 231, 238 food delivery 133, 216, 219, 232 centre for social work 30, 62, 67, 72, 74-75, 85, 124, 146, 155-156 formal care providers 30, 37, 38, 53, 72, 91, 113, 124, 136, 144, 166, Common European Guidelines on the Transition from 213, 225, 236 Institutional to Community-based Care 120-121 fragmentation of care 18, 19, 24, 64, 118 community nursing 46, 65, 71-73, 75, 85, 124, 133, 155-156, 163, G 219 goals in the personal plan 88, 89, 118, 120-126, 128-132, 135-137 complete network of employees 151-152 goal-oriented care 121 concurrent integrated plan 41 H M health care 25, 71, 92, 101, 121, 155, 163 master of kinesiology 63, 69, 70, 71, 164, 177 health care centre 30, 62, 67, 71-75, 113, 146, 155, 176 medical model of disability or cooperation with the user 120 Health Insurance Institute of Slovenia 67, 68, 73, 74 medical technician 73 health status 177, 202, 209 medications 73, 132, 216, 219, 225 healthcare services 28, 106, 143 Ministry of Education, Science and Sport 73 home help 65, 70-72, 75, 85, 91, 100, 124, 145, 155, 163, 176, 224 Ministry of Health 4, 5, 13, 18, 27, 28, 29, 34, 38, 50, 54, 58, 67, 72, home care 65, 132, 163, 170, 212, 225 164, 170, 222, 231, 232, 233, 234, 237, 238 household chores 133, 149, 150, 158, 159, 196, 199, 201, 221 Ministry of Labour, Family, Social Affairs and Equal implementation plan 121, 136, 155, 158, 162, 164, 167-168, 231 Opportunities 27, 28, 35, 58, 121, 182 I mixed method research 53, 57 independence maintenance team 69-71, 82, 126, 145-146, 150, module (field of life) 98, 100, 101, 104, 105, 113 164-165,169, 172-173 module in the information system 229-238 independence maintenance unit 71, 89, 90-91, 123, 125-126, monitoring (programme monitoring) 37, 38, 39, 41, 42 132, 149, 151, 154-155, 158, 235-237 N informal care 5, 24, 31, 133, 170, 212-225, 229 NBA 99-101, 105, 106, 234, 235 information and communication technologies (ICT) 27, 181, 182 NFC sticker 90, 231, 232 information system 164, 166, 214, 228-230, 232, 233, 238 nurse 229 initial service package 119, 134 nurse assistant 69, 70, 71 innovation 6, 8-9, 59, 120, 147-148, 176, 181, 182 nursing care 70, 120, 133, 163, 167, 176 Institute for Economic Research 7, 18, 36 nursing carer 63, 65, 69, 70, 71, 164 institutional care 30, 31, 38, 52, 64, 71, 75, 82, 86, 87, 101-104, 106, O 108-109, 120, 162, 176, 177, 190, 233 objective burden of informal carer 49, 214, 216, 220, 223, 224 instrumental activities of daily living (IADL) 4, 23, 24, 85, 99, 101, occupational therapist 68, 69, 70, 71, 73, 105, 108, 132, 164, 172, 177 106, 146, 150, 164, 166, 167, 169, 216, 217, 219 occupational therapy 128, 163, 172, 177 integrated approach 10, 19, 99 organisational climate 20, 47, 143, 144, 145-151, 155, 157 integrated care 5, 18, 19, 30, 37, 63, 64-76, 81, 145, 150, 229 outcome evaluation 37 integrated care team 19, 37, 64, 66-69, 71, 145, 150 P intergenerational centre 72, 133, 146 P3CEQ 42, 49 interprofessionalism 143 payment for work, salary 142, 147, 149, 158 interview 11, 41, 44, 50-53, 66, 83, 85, 87, 89, 92, 122, 124, 125, Pension and Disability Insurance Institute 58 127, 135, 137, 165, 167, 172, 182-183, 189, 196, 214, 215, 216, 135, personal assistance 65, 85, 86, 163 136, 137 personal assistant 102, 158, 236 L personal plan 20, 41, 50, 55, 82, 83, 88-89, 92, 93, 94, 118-138, 164, life story 118, 120, 121, 126-128, 231 183, 236 living circumstances 82, 86, 92, 94, 98, 113, 126,127,128, 235-237 personal planning 9, 19, 36, 53, 83, 88-89, 93, 95, 99, 118-138, 167 living conditions 94, 121, 125, 126-128, 132, 135 physiotherapist 68, 69, 70, 71, 105, 108, 164, 170, 177 local project council 10, 37, 59, 66, 72, 74, 75-76 physiotherapy 31, 128, 133, 163, 170, 177 Long-Term Care Act 4, 5, 7, 18, 19, 20, 23, 27-30, 164, 167, 169, 170 prevention 73, 131, 163, 164, 168, 217 long-term care coordinator 19, 37, 38, 46, 50-52, 54, 55, 64, 65, process evaluation 39 66, 67, 68-71, 75, 82-95, 118, 121-138, 145, 166, 167, 183, 222, process indicator 37, 83, 93, 144 229, 231, 234-236 project coordinator 47, 49, 50, 54, 83, 85, 122 EVALUATION OF PILOT PROJECTS IN THE FIELD OF LONG-TERM CARE 250 Q social worker 68-70, 105, 108-109, 123, 125, 135 qualitative approach 8, 11, 35, 36, 41, 50-58 Special Social Care and Employment Centre 72-73, 133 quality of life of informal carer 20, 37, 39, 212-225 staff turnover 48, 75, 144, 147, 151 quality of life of user 20, 38, 39, 42, 49, 100, 113, 120, 162, 174, 176, structural indicator 37, 66-68, 70 177, 182, 189, 190, 194-209 subjective burden of informal carer 12, 212, 214, 216-217, 219-220, quantitative approach 8, 11, 35, 36, 41-49, 59, 83, 101, 122, 164, 225 165, 214, 215 supervision for employees 38, 142, 144, 150, 157 R Survey of Health, Ageing and Retirement in Europe (SHARE) registered nurse 65, 68, 69, 70, 71, 105, 123, 135, 234 181, 195, 213 research plan 36, 39-40, 58 T Resolution on the National Health Care Plan 2016–2025 tailored care 5, 49, 121 "Together for a Healthy Society" 27, 164 targeted network of employees 151 Resolution on the National Social Protection Programme telemedicine centre 184, 187 2013–2020 164 telemedicine support services 183-184, 186, 191 respite care 27, 212, 222, 225 terminology 9, 18 result indicator 38, 121, 167 tool for the assessment of eligibility for long-term care 6-7, 30- revision of the plan 82, 118, 122, 133, 137 31, 36, 38, 40, 47, 49, 54, 57, 87, 98-99, 231, 234-238 rural environment 30, 62, 107, 113 training of employees 19, 23, 27, 29, 38, 40, 54, 67, 75, 98, 104, S 105, 114, 118,119, 123-124, 135, 136, 233 safety of users 5, 24, 27, 29-30, 64, 81, 95, 123, 137, 164, 168, 189, training of informal carers 27, 38, 72, 168, 222, 224, 225 195, 209 transitional service package 119, 134 schedule of the provision of service 89, 124, 126, 231 transportation of informal carers 212, 222, 225 Scirocco Exchange 5, 81, 229 triangulation plan 8, 41 scope of needs 99 U scope of services 87, 89, 118, 126, 133-137, 170 urban environment 30, 62, 107, 109, 113 self-help group 222, 225 user satisfaction 20, 29, 38, 44, 63, 81, 144, 158, 162, 164-166, 170 - semi-rural environment 30, 62, 107, 113 174, 177, 182, 187- 191 services to maintain independence 5, 9, 162, 164, 172, 176-177 W signing a personal plan 88, 89, 124, 126, 137 waiting list 9, 19, 42, 55, 70, 80, 83, 87, 90-94, 126, 163, 170 single entry point 19, 27, 38, 50, 57, 62, 64-71, 73- 75, 82, 85-87, working condition 27, 47, 142, 144, 147, 158 90, 93, 105, 113, 124, 144-147, 150, 151, 155, 158, 229, 231, 233, World Health Organisation 164 234, 235 Y SiOK (Slovenian organisational climate) 47, 144 young informal carers 215, 224 Slovenian Federation of Pensioner Associations 158 Z social care 5, 8, 9, 20, 23, 24, 27-30, 38, 63, 65, 70, 72-73, 75, 100, ZBI-22 44, 49, 214 105, 120, 123, 133-135, 142-143, 158, 163, 181, 189 social care service 30, 64, 74, 121, 125, 132, 163, 181, 224 social carer 65, 69-71, 158, 164, 166, 177 social model of cooperation with the user 119, 135 Social Protection Institute of the Republic of Slovenia 7, 18, 36, 99, 112, 163, 231, 235-236 social security 24, 81 Long-term care – a challenge and an opportunity for a better tomorrow. Evaluation of pilot projects in the field of long-term care Editors: Mateja Nagode, Klavdija Kobal Straus Technical editors: Aleš Istenič, Mojca Počič Reviewers: doc. dr. Nikolaj Lipič and dr. med Mircha Poldrugovac Language review: Translation and Interpretation Division, Secretariat-general of the Government of the Republic of Slovenia Design: Kolektiv DVA Urška Bavčar s.p. Publisher: Ministry of Health, Štefanova 5, 1000 Ljubljana Publishing year: 2022 Publication is available at: https://www.gov.si/drzavni-organi/ministrstva/ministrstvo-za-zdravje/o-ministrstvu/direktorat- za-razvoj-zdravstvenega-sistema/sektor-za-dolgotrajno-oskrbo/ Monograph was created as part of the Evaluation of Pilot Projects in the Field of Long-Term Care . The investment is co-financed by the Ministry of Health, the Republic of Slovenia and the European Union from the European Social Fund. Supported by the EU Health Programme Co-funded project SCIROCCO Exchange as part of its Knowledge Transfer Programme. Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID 101728003 ISBN 978-961-6523-80-6 (PDF) INŠ NŠTI TITU TUT RS RS ZA ZA S SOCI OCIAL ALNO O VA V RS ARSTVO All rights reserved. The reproduction of a part or the entirety in any way in any medium is prohibited without the written authorisation of the Ministry for Health. EC Disclaimer The content of this Report represents the views of the author only and is his/her sole responsibility; it cannot be considered to reflect the views of the European Commission and/or the Consumers, Health, Agriculture and Food Executive Agency or any other body of the European Union. The European Commission and the Agency do not accept any responsibility for use that may be made of the information it contains. LONG- TERM C ARE – A CHALLENGE AND AN OP LONG-TERM CARE – A CHALLENGE AND AN OPPORTUNITY FOR A BETTER TOMORROW POR TUNIT Y F OR A BET Evaluation of pilot projects in the field of long-term care TER T OMORRO W MODEL DOLGOTRAJNE OSKRBE Pilotno preizkušeni pristopi za boljšo integracijo storitev dolgotrajne oskrbe Ljubljana, March 2022