c e p s Journal | V ol.13 | N o 3 | Y ear 2023 105 Empowering Supervisors Towards Responsible Research Conduct in Supervision via an Online Course: A Pilot Study Miriam van Loon 1 and Mariëtte van den Hoven* 2 • Supervision and mentoring are highly relevant aspects of research integ - rity. Codes of Conduct, such as the ALLEA code of conduct, stipulate the relevance of training researchers how to conduct research well and about the role supervision plays in preventing unacceptable research practices. The Dutch Code of Conduct, for example, explicitly states that universi - ties are responsible for facilitating training about research integrity. We developed a course for supervisors to address their responsibility and role in training early career researchers in research integrity. This contribution describes what evidence base was used to design this course and how the course is experienced by supervisors who participated in its piloting in early 2022. A total of 147 subscribed to the course in the testing phase, and seventeen participants obtained a certificate. The main lessons from the experiences with this course and the literature are 1) to tailor supervisor courses to the small amounts of time that supervisors can schedule to take these courses and to adjust the content and assignments to their needs, 2) to make online courses very attractive, but that need to be combined with 3) a face-to-face meeting to motivate them to finish the course in time and it might help to enable shared reflection by sharing personal experiences. Keywords: supervision, RCR, training, online module 1 University medical center (Amsterdam UMC), Free University in Amsterdam, Amsterdam, the Netherlands. 2 *Corresponding Author. University medical center (Amsterdam UMC), Free University in Amsterdam, Amsterdam, the Netherlands; m.a.vandenhoven@amsterdamumc.nl. DOI: https://doi.org/10.26529/cepsj.1586 106 empowering supervisors towards responsible research conduct in supervision ... Opolnomočenje nadzornikov za odgovorno raziskovalno ravnanje pri nadzoru prek spletnega tečaja: pilotna študija Miriam van Loon in Mariëtte van den Hoven • Nadzor in mentorstvo sta zelo pomembna vidika raziskovalne integri - tete. Kodeksi ravnanja, kot je kodeks ravnanja ALLEA, določajo, kako pomembno je usposabljanje raziskovalcev za dobro izvajanje raziskav in kakšno vlogo ima nadzor pri preprečevanju nesprejemljivih raziskoval - nih praks. Nizozemski kodeks ravnanja na primer izrecno navaja, da so univerze odgovorne za omogočanje usposabljanja o raziskovalni inte - griteti. Razvili smo tečaj za nadzornike, ki obravnava njihovo odgovor - nost in vlogo pri usposabljanju raziskovalcev, ki so na začetku kariere, na področju raziskovalne integritete. V tem prispevku je opisano, ka - kšna baza dokazov je bila uporabljena za oblikovanje tega tečaja in kako tečaj doživljajo nadzorniki, ki so sodelovali pri njegovem poskusnem izvajanju v začetku leta 2022. Na tečaj se je v fazi testiranja prijavilo 147 udeležencev; sedemnajst jih je pridobilo certifikat. Glavna spoznanja, ki izhajajo iz izkušenj s tem tečajem in literature, so: 1) tečaje za nadzor - nike je treba prilagoditi majhni količini časa, ki si ga nadzorniki lahko razporedijo za udeležbo na teh tečajih, ter vsebino in naloge prilagoditi njihovim potrebam; 2) spletni tečaji naj bodo zelo privlačni, vendar jih je treba kombinirati s 3) srečanjem v živo, da bi jih motivirali za pravo - časno dokončanje tečaja in mogoče s tem tudi pomagali omogočanje skupne refleksije z izmenjavo osebnih izkušenj. Ključne besede: nadzor, odgovorno izvajanje raziskav, usposabljanje, spletni modul c e p s Journal | V ol.13 | N o 3 | Y ear 2023 107 Introduction Supervision and mentoring are considered highly relevant within the context of research integrity. In policy documents and codes of conduct, the relevance of good supervision is frequently mentioned (Lerouge & Hol, 2020; ALLEA, 2017). As supervision and mentoring are the main channels through which novice researchers learn to do research well (Bird, 2001) and learn how to behave in research practices (Fisher, 2019), this is a perfect window of oppor - tunity to place responsible conduct of research (RCR) at centre stage. As stated by Lerouge and Hol (2020) in a position paper by the League of European Re - search Universities (LERU) on research integrity: Students and supervisors have a shared responsibility to develop atti - tudes and skills to deal with issues of research integrity and to create learning situations that encourage participants to behave with integrity while maintaining a realistic understanding of the hierarchical struc - tures of academia. (p.16) To that purpose, doctoral candidates should receive training in research integrity, to gain more knowledge on related topics, but also ‘to empower re - searchers to recognise and deal with (potential) problems of research integrity and to understand its relevance’ (Lerouge & Hol, 2020, p.16). Indeed, within many institutions for higher education, training for doc - toral candidates in research integrity (RI) has been widely embraced and em - bedded (Abdi et al., 2021). RI is explicitly mentioned in the codes of conduct in the relevance of training. The Dutch Code of Conduct, for example, states that universities are responsible for facilitating training about research integrity (Netherlands Code of Conduct for Research Integrity, 2018). However, it seems only obvious to expect supervisors also to be stimulated to reflect on their re - sponsibilities towards RCR when introducing novice researchers to their re - search practices. Thus, training of supervisors has also been suggested: ‘There is not only training needed on research integrity for supervisors but also specifi - cally on how to supervise with integrity’ (Lerouge & Hol, 2020, p. 16). Haven et al. (2022) call the relationship between supervision and research integrity bidirectional: ‘poor supervision may increase the chances of the PhD candidate engaging in research misbehaviour’ (Anderson et al., 2007 in Haven et al., 2022: p.2) while supervisors can also ‘foster research integrity among their PhD can - didates’ (Haven et al., 2022, p. 2). They point out that research on misconduct cases often reveals that supervision has been inadequate in cases of misconduct of PhD candidates (Haven et al., 2022). 108 empowering supervisors towards responsible research conduct in supervision ... Good supervision can be jeopardised when supervisors are unskilled, overworked, or un-invested in research integrity (Roje et al., 2022). Also, the (inter)dependency within supervision and mentoring relations can negative - ly impact the relationship (Löfstrom & Pyhalto, 2020; Muthanna & Alduais, 2020), and studies have identified ‘abusive and exploitative supervision, bul - lying, confounded or dual relationships (Goodyear et al., 1992; Mahmud & Bretag, 2013)’ (Lofstrom & Pyahalto, 2020, p. 536). If the purpose of research supervision is to ‘help students develop criti - cal, creative thinking and research skills, and contribute to the existing body of knowledge’ (Muthanna & Alduais, 2020, p. 1), the urge to address supervisors more actively and prepare them better for their guiding role in the research and career trajectory of novice researchers needs to be embraced more widely. It would not only be beneficial for supervisors to become more knowledgeable on current high standards for responsible research practices but also for their own interpersonal skills in supervising junior researchers in order to learn how to give (and receive) feedback and address integrity topics like co-authorship more frequently with their supervisees. In an H2020 project that centres on the notion of empowerment (Theunissen & van den Hoven, 2021), the empower - ment of supervisors can also be bidirectional; supervisors can better learn to re - flect upon and handle situations in their own research practices in a responsible and integer way, and they can be trained to help empower their PhDs students. Training could, therefore, be beneficial for supervisors themselves as well as for their supervisees. Training could also enable supervisors to feel more confident and improve their academic leadership (Rathmell et al., 2019). In this contribution, we describe the design, development, and piloting of an online module for supervisors and how this can contribute to filling the gap in good supervision as part of the H2020 project INTEGRITY (project no 82456). The module mirrors online modules that were developed for doctoral students, especially three small private online courses (SPOCS) and one massive open on - line course (MOOC) that use similar course materials and teaching philosophy to stimulate empowerment towards responsible conduct of research. Methods Designing and developing an online course for supervisors In this section, we describe the main elements of the design and de - velopment of an online course for supervisors. First, the teaching philosophy will be explained based on the concept of empowerment. Second, the design c e p s Journal | V ol.13 | N o 3 | Y ear 2023 109 of the modules in the course and the process of development of the modules is described. The course consists of three thematic modules: 1) Being a good men - tor, 2) Mentoring towards RCR, and 3) Empowerment. Finally, the first testing phase of the course and the different steps that have been taken in this process are illustrated. Teaching philosophy underlying RCR education Fostering responsible conduct of research involves more than gaining knowledge on research integrity, possible types of misconduct or required standards in research practices. Mike Kalichman (2007) states clearly: […] if researchers already have and/or are learning some RCR knowl - edge and skills […] then is the most important thing to do to provide them with more? Or, is it more important to arm them with a positive disposition toward RCR, with a sense that there are things they can do in the face of concerns, and with a belief that they are part of a culture that takes RCR seriously? (p. 70)? Kalichman (2014) later asserts, ‘These attitudes are arguably more es - sential than any particular piece of knowledge or improvement in skills. In their absence, it would matter little if someone had perfect knowledge and skills…’ (p. 70). In the H2020 project INTEGRITY, the concept of empowerment has been used to develop a teaching philosophy on how to foster responsible con - duct of research. Operationalising the concept, five main aspects were defined: 1) RCR courses need to build the capacities of researchers and not only focus on knowledge or skills; 2) RCR training needs to stimulate the critical autonomy of researchers, enabling them to 3) learn to take control of integrity issues they encounter in practices, to which they will be 4) motivated to pro-actively react and 5) dare to speak up if necessary (Theunissen & van den Hoven, 2021). In line with Freire’s (1970) Pedagogy of the Oppressed , which has inspired the academic debate on empowerment, the idea of empowerment is not so much about lifting those who lack power into a situation where they will be less power - less but about seeking a system change. Misconduct in research communities is closely related to systems in which pressure to please, publish and get promoted is high, in which chances for a permanent position in academia are low, and time pressure to meet deadlines is increasing (Haven et al., 2019). Also, for many in - tegrity issues in the grey area (Theunissen & van den Hoven, 2021), the ‘pressures to perform’ and a lack of sufficient time or openness to discuss dilemmas with 110 empowering supervisors towards responsible research conduct in supervision ... colleagues can jeopardise responsible science. Many integrity questions are rec - ognised to be inherent to the research process, and more transparency and com - munication about these questions are required among research teams. For exam - ple, authorship issues often lead to disputes or dissatisfaction about the process of how authorship is decided or which order of authorship is decided upon (e.g., Lokhtina et al., 2022). Therefore, when offering training to researchers, empower - ment aims to help participants recognise, reflect, and feel able to address, decide, and act upon integrity issues in a responsible manner. The responsible solution should become the obvious and attractive way rather than turning a blind eye, feeling uncomfortable or keeping silent. This core idea of empowerment has been operationalised in training modules for high school students, undergraduate stu - dents, doctoral students, and supervisors. Literature shows that supervisors and mentors ‘can shape young re - searchers’ behaviour’, hence that their influence as role models (positive or negative) can be significant (Roje et al., 2022). Research shows that supervis - ing and mentoring relations offer excellent opportunities to address issues of responsible and good conduct, as junior researchers are introduced to research practices and will highly depend on the examples that supervisors give to them (Gray & Jordan, 2012; Clynes et al., 2019; Kalichman & Plemmons, 2018). Em - powerment in supervisors is especially interesting because it can work upwards (by empowering the supervisors themselves) and downwards (by training the supervisors to empower their PhD students). It is, therefore, important that su - pervisors are also being trained to be good supervisors. Based on our teaching philosophy of empowerment, ‘becoming more aware of and proactive towards integrity issues’ (Theunissen & van den Hoven, 2021), the aim is to enable su - pervisors to reflect on their own experiences and behaviour. In the next section, we show how the empowerment teaching philosophy is used in designing a course for supervisors. Design of the module and the process of development The course Supervision and Mentoring towards RCR has been developed based on three online modules for doctoral students that were developed and implemented earlier: 1) Responsible research through supervision, collabora - tion and working together, 2) Integrity in academic publishing: authorship and peer review, 3) Data in responsible conduct of research. One module for doc - toral students specifically focuses on supervision and collaboration with oth - ers. The topics of this module were: 1) Introduction to RCR, 2) Expectations and responsibilities in supervision and mentoring, 3) Culture, colleagues, and c e p s Journal | V ol.13 | N o 3 | Y ear 2023 111 communication, and 4) Collaboration outside the research team. To develop this module targeting supervisors and mentors, we could use elements of the doctoral course, background literature, and the experiences of doctoral stu - dents using online modules to design the module for supervisors. Content of the course An essential source for the content of the course was a document on the Office for Research Integrity (ORI) website, which mentions 11 possible challenges in mentoring: power differential, competing and conflicting roles, inability to meet research deadlines, failure to give credit, failure to ensure a supportive research environment, failure to provide sound advice, failure to monitor trainee’s conduct, failure to treat trainees fairly, failure to ensure the trainee is making progress in a timely fashion, failure to recognise problems, and observing violations of research protocol. These aspects were incorporated into an assignment to make supervisors aware of possible challenges. In addition to the sources described above, we added theory about su - pervision, mentoring and role modelling, and research culture (Haven, 2021; Fisher et al., 2009; Making the Right Moves; Embassy of Good Science, 2021) specifically relevant for the perspective of supervisors (e.g., how to deal with power differences and possibilities for power abuse (elephantinthelab, 2021)). Literature about specific topics related to research integrity was also added, pro - viding background information about plagiarism (Office of Research Integrity, 2021); a data checklist for responsible data handling (UK Data Service, 2021); literature about ghost-writing (Gotschze et al., 2009; DeTora et al., 2019) and gift authorship (Harvey, 2018). Combining important aspects from literature with the empowerment teaching philosophy, we decided that three aspects of supervision and RCR were particularly relevant for this course: 1) Reflections on being a good mentor, 2) Mentoring towards RCR, and 3) Empowerment. As explained earlier, empower - ment in training aims to help participants 1) to recognise, 2) to reflect, and 3) to feel able to address, decide, and act upon integrity issues responsibly. These aspects of empowerment have been incorporated in the course (see also Table 1: Learning goals) : 1) Recognising integrity issues is stimulated by providing back - ground information and examples of possible issues; 2) Reflection is stimulated within the different assignments, focusing on one’s own experiences; 3) The abil - ity to address, decide and act upon integrity issues responsibly follows from the first to steps; also respondents are encouraged to interact with peers about issues, lowering the threshold to speak up. Additionally, we added a third module, one 112 empowering supervisors towards responsible research conduct in supervision ... on Empowerment, making empowerment concrete and applicable to one’s re - search practice, stimulating reflection on possible solutions for integrity issues. Table 1 Learning goals Topic Learning goals 1) Being a good mentor • R eflec t on wha t the qualities of a good supervisor ar e • R eflec t on y our o wn supervising s t yle 2) Mentoring towards RCR • L earn c or e c onc epts and in f orma tion about men t oring towards RCR • Reflect on the assignment on how to connect the concepts and information to your own work as a supervisor • Interact and give and receive feedback on each other’s work and experiences as a supervisor. 3) Empowerment in Academia • L earn ho w po w er dynamics in academia ar e r ela t ed t o RCR and what is needed for empowerment with effective strate- gies • Reflect on your role in power relationships in work and improving your situation • Interact with peers in the assignment about your personal goals in empowering yourself and your PhD students. Structure of the course The course has been designed consisting of five online learning units (LU), which participants can follow at their own pace. LU 1, 2, and 3 each take about 45 minutes to finish, including the assignments. A LU is built around video scenes, with short assignments in between the clips complemented by information on integrity issues and supervision. The scenes, with actors playing a supervisor and a doctoral student, have been developed specifically to facili - tate discussion and stimulate thinking about highly relevant and recognisable integrity issues, such as authorship and publishing; communication difficulties between supervisor and doctoral student shared expectations regarding power - play and providing and receiving feedback. Personal reflection on one’s own characteristics, skills, and behaviour as a supervisor is targeted. Each learning unit, therefore, concludes with a reflec - tion assignment, inviting participants to apply learned knowledge to actual and relevant examples. Participants can also jointly discuss each other’s reflections and share experiences in the comment section of the learning environment. LU 0 ‘Introduction’ introduces the aims of the course and the learning environment. Participants are encouraged to use their own experiences in the course and to learn to reflect on things they do well and things that can be c e p s Journal | V ol.13 | N o 3 | Y ear 2023 113 improved. Assignments are designed to apply concepts to one’s daily research practice, stimulating thinking about behaviour; for example: ‘Write down one example of what you, as a supervisor, positively stimulate in PhD students to - wards RCR’ . Participants are also offered to participate under a pseudonym in order to feel safe sharing personal experiences. Figure 1 Being a good mentor LU 1, ‘Being a good mentor’ (see Figure 1, a screenshot), has been de - signed to reflect on the good qualities of mentors and on participants’ own style of supervising. The scene ‘Never waste a good talent’ is used to portray how power imbalance could negatively influence a supervisor relationship when not taking into account the needs of the doctoral student; the supervisor does not provide proper feedback, ignores the high workload and is also generally rude to the student. This ‘bad example’ is used to stimulate critical reflection on what good supervision is. In LU 2, ‘Mentoring towards RCR’, we discuss the role of instruction, modelling and research culture in mentoring PhD students towards RCR. The central scene is ‘To publish or not to publish’ , in which the supervisor and the PhD student disagree about whether there is sufficient data to publish an arti - cle, both having different opinions and interests. 114 empowering supervisors towards responsible research conduct in supervision ... Power dynamics can affect working in academia, especially in supervis - ing relations; the goal of LU 3, ‘Empowerment in academia’ , shows the teaching philosophy on empowerment of the H2020 project and challenges supervisors to reflect on this. LU 3 consists of two parts: ‘Power dynamics in academia’ and ‘Empowering your PhD students and yourself’ . Power dynamics are discussed using the scene ‘Standing on the shoulder of giants’ in which a supervisor and PhD student disagree about adding an author, which is an example of gift au - thorship. The second part of LU 3 is built around an animation showing what empowerment is and how this can be achieved. The final unit, LU 4 ‘Farewell and evaluation’ , is developed to gather ex - periences about the course from participants and to understand what could be improved. There is an evaluation form and the option to receive a certificate upon completion of the course. Testing phase The course was designed and developed in an iterative process with El - evate Health, an organisation specialised in designing online courses. Elevate provided a check on the didactical soundness of the course and its components and helped to decide which type of assignment was best suited to the differ - ent topics in each module. The design aimed to prompt (virtual) interaction, reflecting on aspects seen in the video scenes. All H2020 Integrity project part - ners were, during a hybrid project meeting in Porto, Portugal, invited to take a look at the first version of the course, test the modules, and share their experi - ences and opinions about the course. We collected feedback and made final changes. We decided to allow for continuous enrolment in the course, as most supervisors would probably want to follow the course at their own pace. After these final adaptations, the course was opened for participants. Recruitment started, including the development of a promotional video showing highlights to potential participants (https://vimeo.com/691431805/f966ec24cd). The recruitment of participants was somewhat challenging; project partners warned that recruitment of senior staff might be difficult since they often lack time and/or motivation to participate in additional courses, espe - cially if these are not obligatory. The INTEGRITY project partners were each asked to recruit participants in their own network. Additionally, we advertised the course using different departmental mailing lists of the University of Utre - cht and some (inter) national newsletters for integrity networks. On August 2, 2022, 147 participants were enrolled in the course. c e p s Journal | V ol.13 | N o 3 | Y ear 2023 115 We gathered feedback and analysed discussion board posts in order to evaluate the pilot testing of the course to understand what works and what could be improved. Consent for analysing and publishing data in the course was asked by sending an e-mail to participants of the course, informing them about our goal, and asking for permission to publish about the evaluation and discussion board input. As we both used contributions in a so-called discussion forum as well as individual evaluation responses, we pseudonymised the data for our analysis and closed the course environment for all participants so that no one could re-identify quotes. Results Experiences with the module: First impressions In this section, the first experiences with the module are described. We will first present some characteristics of participants in the pilot of the course, describing the origin of participants and their progress through the course. Then, we will give an impression of feedback on the pilot course provided by participants. Participants Participation in this course is voluntary and free of charge; in total, 147 participants are enrolled (August 5, 2022), mostly through snowball recruit - ment by project partners. An impression of the different countries’ participants registered from (not all participants listed a country), shown in 116 empowering supervisors towards responsible research conduct in supervision ... Figure 2 Country of origin Table 2 shows the activity of participants throughout the course. The number of active participants decreases as the course progresses. After comple - tion of the course, participants have the option to receive a certificate. Of the 147 participants who registered, seventeen applied for a certificate. Table 2 Participant activity LU 1 67 participants LU 2 47 participants LU 3 42 participants Obtained certificates 17 Evaluations 18 (7 participants answered to open questions) c e p s Journal | V ol.13 | N o 3 | Y ear 2023 117 Empowerment In this section, we discuss some participants’ experiences and evaluate whether participants were able to build their empowerment capacities in some way. In LU1, ‘Being a good mentor’ , participants are encouraged to reflect on the qualities of good supervisors or mentors. The discussion board in this LU shows reflection on some aspects of good supervision. In the reflection assign - ment, three fictional cases are presented describing integrity issues to reflect on. Participants also interacted with each other, discussing personal dilemmas via a discussion textbox. Two general themes were discussed by participants: coach - ing students and authorship. Participants agreed on the importance of finding a balance in coaching, not pushing students too much or too little. Tasks should be done, and students need to be given responsibility, but students should not be overburdened. Participants said expectations can be managed by having clear and explicit agreements. Authorship, another relevant RCR issue, was also discussed, considering what the right thing to do is in certain cases. The main conclusions were that authorship should be discussed openly; refraining from authorship or offering shared authorship can provide an opportunity for your PhD career progress. In LU2 ‘Mentoring towards RCR’ , participants are asked in the reflection assignment 1) to describe the research culture in their department, 2) to give an example of how they, as a supervisor, positively stimulate PhD students to - wards RCR in practice and 3) what their take-home message is regarding their supervision towards RCR. These questions were aimed to stimulate reflection and also to think of possible solutions, encouraging supervisors to take an ac - tive role in their own supervision regarding integrity issues. In the reactions, participants indicate recognising issues considering research culture. Specific examples of what could be improved were not offered, possibly because it does not feel safe enough to share this online, as explicitly mentioned by one par - ticipant. The importance of role modelling becomes clear by participants de - scribing it as important to be a good example and show what RCR looks like in their own practice; participants say that supervisors have an important role in teaching good practices. LU3 ‘Empowerment in academia’ consists of two parts, ‘Power dynam - ics’ and ‘Empowering your PhD students and yourself ’ . Participants were asked in the reflection assignment to think about 1) In what way their work is af - fected by power relations (both upwards and downwards); 2) How this may affect their own RCR and that of PhD students; 3) With this knowledge, how 118 empowering supervisors towards responsible research conduct in supervision ... they would advise the PhD students and how to empower them; 4) What is go - ing well in their own empowerment and what could be improved; 5) What is going well in empowerment of their PhD student and what could be improved; 6) Write down a personal goal regarding their own empowerment and that of their PhD students. With regards to empowering PhD students towards empowerment, par - ticipants reflected together on the discussion board, agreeing that as a supervi - sor, you can help build self-confidence by increasing their self-initiative and encouraging them to work on their own. With regards to power dynamics, it was suggested that PhD students should be told what options exist in case of a conflict; for example, they can switch supervisors or even leave institutions when necessary. Some power dynamics might be unavoidable, but a supervisor should endeavour to provide a safe and healthy group climate and maintain an open dialogue. Participants also said critical awareness of this power imbalance and having strategies to address it can be stimulated by empowering research - ers. Little has been discussed during the course about career advancement or the personal empowerment of the supervisors. Evaluation of the course Participants provided feedback on assignments and the course content on the discussion board. In LU4, we also included a questionnaire to evaluate the course, using closed and open questions. This evaluation was completed by 18 participants. c e p s Journal | V ol.13 | N o 3 | Y ear 2023 119 Table 3 Outcomes course evaluation Even though the numbers are low, on average, we see that only a minor - ity of participants did not feel stimulated at all by the content of the course. The responses are mainly to be found in the ‘to a moderate extent’ and to ‘some extent’ . The items that stand out the most concern the ‘ability to transparently discuss the roles and responsibilities I and my mentors have during my PhD project, and how these will have to shift up until my graduation. ’(rare…), ‘my knowledge about what others may reasonably expect from me and what I may reasonably expect from others in collaborating with them’ , ‘my ability to act in a responsible and accountable way when faced with dilemmas in my research project/practice’ , and ‘my ability to determine to take the necessary steps to take an issue at hand’ . Some positive aspects mentioned were concerning the outstanding im - portance of the topic and the fact that content, both videos and activities, pro - vided food for thought. The video scripts were considered valuable. The course was said to raise awareness about mentoring and supervising problems, and 120 empowering supervisors towards responsible research conduct in supervision ... raising awareness that researchers should step up ‘to foster a better climate in their research group or institution’ and: ‘what I learned about research integrity is every researcher, student or supervisor should be responsible and should have good communication’ . Also, the idea of addressing issues openly is valued, hav - ing learned that ‘RI is always honest and states everything as clearly as possible’ . Communication is thus mentioned by several participants, ‘What I learned about research integrity is… that it depends not only on the theoretical basis but also highly on your personal example and how you communicate with your students. ’ Some aspects for improvement were mentioned: the structure of the course could be clearer, fine-tuning learning aims in each module, and de - veloping downloadable summaries of the main topics for future reading. We received feedback that some assignments (e.g., fill-in-the-blank exercises) or certain feedback on interactive assignments were not appropriate for the level of supervisors. Another remark was that some topics might be too sensitive to discuss in an online discussion board, for example, discussing personal experi - ences of power differences. A participant said the course was ‘a bit short and not as interactive as it should be’ . Another participant: ‘Taking into account that most of these problems are structural and due to internalised and perpetuated behaviours, I miss suggestions or ideas on how to be proactive at the depart - ment and institutional levels’ . Finally, the videos are exaggerations of integrity issues to encourage re - flection and make people think about what the right thing to do is. This exag - geration was not appreciated by all respondents, however, making some feel uncomfortable because they deemed some scenes to be ‘highly inappropriate’ . Also, some issues were mentioned to be missing in this course: ‘Issues about race, religion, politics, harassment, and abuse’ . Discussion: What is needed with regard to training for supervisors? This course is an attempt to educate senior researchers on some impor - tant issues in integrity and supervision relationships. In this section, we dis - cuss what we can learn from our experiences in developing and evaluating this course. What do we need to take into account in the further development of training material for supervisors, tailoring it to their needs and wishes? Empowerment This course aimed to increase empowerment of supervision in RCR by training participants 1) to recognise, 2) to reflect, and 3) to feel able to address, c e p s Journal | V ol.13 | N o 3 | Y ear 2023 121 to decide and to act upon integrity issues responsibly. The first two aspects of empowerment, recognising and reflecting upon integrity issues, were present in the contributions of participants throughout the course. Participants re - flected on how to improve the empowerment of their PhD students. There was hardly any reflection, however, on supervisors’ own empowerment and career advancement. As suggested by some participants, this might be because it felt unsafe to discuss their own empowerment in an online environment. For fu - ture training, a live (online) meeting might be included to discuss personal experiences together in a safe environment guided by a teacher/facilitator. In this way, participants may feel encouraged to discuss personal experiences and learn from each other about empowerment in their own practice. Also, a more individual portfolio assignment can encourage participants to share their own experiences more. In the SPOCs we developed, such individual portfolios are already included. The third element of the empowerment philosophy: feeling able to ad - dress, to decide and to act upon integrity issues responsible; was less evident in the course contributions of participants. This probably is related to the type of assignment asking specifically to reflect. In the course evaluation, participants were asked how empowerment was improved. Some results indicated improve - ment in the ability to act responsibly in facing a dilemma and the ability to take necessary steps. The evaluation inspired us to think of further improvements for the course. Empowerment could be enhanced by providing more practical tips that participants can apply to their own research practice. Also, ideas about recog - nising harassment and abuse and tips on how to handle these issues could be included to empower participants to address harassment. Additionally, practi - cal tips to improve one’s own research culture could be added. Sensitive topics and confidentiality Topics that are part of this course can be experienced as sensitive, for ex - ample, personal experiences with power dynamics in academia or experienced personal dilemmas. Discussing sensitive topics could be difficult in an open on - line environment since there is no real-life interaction, and participants might not feel safe sharing experiences with other participants they do not know. We provide the option to participate under a pseudonym and ask to anonymise cases and experiences shared during the course in order to safeguard privacy and con - fidentiality. However, participants may still feel a barrier to being open. The safety of the online environment could be improved by providing more possibilities to share anonymously or by writing a personal, private reflection in a portfolio. 122 empowering supervisors towards responsible research conduct in supervision ... Another option to improve a safe learning environment and encourage active contribution of experiences could be to combine the online course with live meetings, enabling a dialogue about personal experiences with research in - tegrity and supervision, preferably facilitated by a trained teacher who is aware of the sensitivity. Teachers should be professionally trained in teaching these topics, and especially in this context, they should be able to ‘manage behaviour effec - tively to ensure a good and safe learning environment’ (Saqipi & Vogrinc, 2020). During this meeting, more impersonal, general examples could also be discussed during a brainstorming session to make participants feel safe to share without having to share their own examples. Live meetings, in addition to online meet - ings, can increase to ‘engage in collaborative learning, interactions, and discus - sions with diverse others’ (Dumford & Miller, 2018). Also, including elements of flipped learning, for example, by encouraging participants to prepare cases, could increase engagement and satisfaction (Gasparic, 2017). A live meeting could also stimulate further reflection between participants and encourage participants to interact with each other and come to joint solutions for certain integrity issues. Comparison of doctoral student & supervisor training Training of supervisors is important because they are in the position to be a role model for all the people they supervise. The material from previous courses used in the development of this course was aimed at doctoral students. As a result, the tone and level of the assignments and feedback did not always match the perceptions and levels of supervisors. For example, the videos used in the course are interpreted differently by supervisors and doctoral students because they experience the scenes from different perspectives. Therefore, we should further investigate what kind of material, assignments, and feedback supervisors value best and adjust material accordingly. Recruitment & progress The recruitment of supervisors was challenging since they often experi - enced high workloads. The course needs to fit into their busy schedule. An online course format that enables participants to follow the course at their own pace matches best with the template of a MOOC (massive open online course) (Guo, 2017; Filius et al., 2018). MOOCs are open course environments where continuous enrolment takes place, and limited interaction with other participants is possible. Our aim is to encourage one’s own experiences and become actively involved with course materials often better fit with the characteristics of small group on - line courses (SPOCs). A clear advantage of an open online environment is that it can help to increase the scale without adding to the workload of a teacher. c e p s Journal | V ol.13 | N o 3 | Y ear 2023 123 Currently, supervisors from over 20 different countries are enrolled in the course. However, only 17 participants of 147 reached the final stage of re - ceiving a certificate in our course, and the number of active participants was slowly diminishing throughout each consecutive learning unit. This is a well- known disadvantage of MOOCs (Filius et al., 2018). Combining the online ma - terials with live sessions and presenting them in blended ways might improve the motivation to actively participate in the course. A possible explanation for low participation in the course is the vol - untary nature of the course. There has been some discussion about whether integrity courses for supervisors should be mandatory. In a study by Ten Haven et al. (2022), participants feared that. […] making training compulsory would diminish its value. […] you could risk bringing in participants that are not able or willing to criti - cally inspect their own behaviour, and their counterproductive attitude would decrease the space for others to learn and reflect. (p.10) This contrasts with the often-heard comments in the doctoral-level courses we teach on research integrity, by which students indicate that their supervisors should also take these courses. A mandatory course would at least have more outreach and help supervisors who are considered to be ‘unwilling’ to help them reflect on their role and position as supervisors. A course could make them aware of points of improvement in their supervision, especially if the course is focused on empowerment. This course is not mandatory yet, but further inquiries on embedding it in institutional contexts will be explored. Overall, the recruitment of supervisors for integrity training and ensur - ing their completion of the training is crucial because they can influence the development of their supervisees’ integrity to a great extent. Also, because of existing power dynamics, supervisors behaving badly can have a great negative impact on the (working) lives of their supervisees. More research should be done into recruiting supervisors for integrity courses: what do they need, what do they want, and how can they help to further increase RCR? Additionally, mandatory training in responsible conduct of research, such as training in RCR and supervision, could be considered as part of researcher assessment as this training is currently already standard for early career researchers. Diversity In the evaluation of the course, some issues were mentioned to be miss - ing; ‘issues about race, religion, politics, harassment, and abuse’ . Some of these topics are present, for example, a scene depicting harassment, but it is not 124 empowering supervisors towards responsible research conduct in supervision ... explicitly mentioned. Taking into account the complexity and sensitivity of the topic, harassment and abuse could be introduced more clearly, stating the video is an exaggeration of these issues. Ideas about how to recognise harassment and abuse and tips on what to do could be included to empower participants. We did not include any information on race, religion, or cultural dif - ferences. Participants from at least 24 different countries were involved in this training. For future training, it would be interesting to investigate how this course is interpreted by respondents coming from different countries and whether we can make changes in order to include additional perspectives or topics. It is also relevant to ask for further feedback on the design, the videos used, and how the course is experienced. Especially because we focus on im - proving empowerment, we should recognise and acknowledge power differ - ences and inequality among participants (Schlossberg & Cunningham, 2016). Further research could thus increase our own awareness of possible sensitive issues, enabling us to acknowledge and address these. Training supervisors is essential for stimulating RCR in academia. In the future, we should focus on the further evaluation of outcomes of training for supervisors and focus on research asking participants what they need, want, and like in terms of training. In this way, courses can be tailored to specific needs. Acknowledgements We want to acknowledge the contribution of Eline Borsboom and Dan - ice Winkelhorst from Elevate Health in the design and development of this course and in the evaluation and improvement of the didactical principles un - derlying the course. References Abdi, S., Pizzolato, D., Nemery, B., & Dierickx, K. (2021). Educating PhD students in research integrity in Europe.  Science and engineering ethics , 27(1), 1–19. https://doi.org/10.1007/s11948-021-00290-0 ALLEA. (2017). The European Code of Conduct for Research Integrity. https://allea.org/code-of-conduct/ Bird, S. (2001). Mentors, Advisors and Supervisors: Their Role in Teaching Responsible Research Conduct. Science and Engineering Ethics, 7 (4), 455–468. https://doi.org/10.1007/s11948-001-0002-1 Burroughs Wellcome Fund & Howard Hughes Medical Institute (2006). Making the Right Moves: A Practical Guide to Scientifıc Management for Postdocs and New Faculty. Retrieved September 11, 2023, from https://www.hhmi.org/sites/default/files/Educational%20Materials/Lab%20Management/ Making%20the%20Right%20Moves/moves2.pdf c e p s Journal | V ol.13 | N o 3 | Y ear 2023 125 Clynes, M., Corbett, A., Overbaugh, J. (2019). Why We Need Good Mentoring, Nature Reviews Cancer , 19(9), 489–493. https://doi.org/10.1038/s41568-019-0173-1 DeTora, L. M., Carey, M. A., Toroser, D., & Baum, E. Z. (2019). Ghostwriting in biomedicine: a review of the published literature. Current Medical Research and Opinion , 35(9), 1643–1651. https://doi.org/10.1080/03007995.2019.1608101 Dilemma Game . Erasmus University Rotterdam. (n.d.). https://www.eur.nl/en/about-eur/policy-and-regulations/integrity/research-integrity/dilemma-game Dumford, A. D., & Miller, A. L. (2018). Online learning in higher education: exploring advantages and disadvantages for engagement. Journal of Computing in Higher Education , 30(3), 452–465. https://doi.org/10.1080/03007995.2019.1608101 Filius, R. M., Kleijn, R. A. D., Uijl, S. G., Prins, F. J., Rijen, H. V . V ., & Grobbee, D. E. (2018). Challenges concerning deep learning in SPOCs.  International Journal of Technology Enhanced Learning , 10(1–2), 111–127. https://doi.org/10.1504/IJTEL.2018.088341 Fisher, C. B., Fried, A. L., Goodman, S. J., & Germano, K. K. (2009). Measures of mentoring, department climate, and graduate student preparedness in the responsible conduct of psychological research. Ethics & Behavior , 19(3), 227–252. https://doi.org/10.1080/10508420902886726 Freire, P . (2000 ). Pedagogy of the oppressed (30 th -anniversary ed.). Continuum. Gasparič, R. P . (2017). Jonathan Bergmann and Aaron Sams, Flipped Learning: Gateway to Student Engagement, International Society for Technology in Education: Eugene, Oregon and Washington, DC, 2014; 169 pp.: ISBN 978-1-56484-344-9. Center for Educational Policy Studies Journal, 7(3), 173–176. Gøtzsche PC, Kassirer JP , Woolley KL, Wager E, Jacobs A, Gertel A, et al. (2009) ‘What should be done to tackle ghostwriting in the medical literature?’ PLoS Med, 6(2). https://doi.org/10.1371/journal.pmed.1000023 Gray, P . W ., & Jordan, S.R. (2012). Supervisors and Academic Integrity: Supervisors as Exemplars and Mentors. Journal of Academic Ethics , 10(4), 299–311. https://doi.org/10.1007/s10805-012-9155-6 Guerin, C., & Green, I. (2015) “They’re the bosses”: feedback in team supervision. Journal of Further and Higher Education , 39(3), 320–335. https://doi.org/10.1080/0309877X.2013.831039 Guo, P ., (2017) ‘MOOC and SPOC, which one is better?’ Eurasia Journal of Mathematical Sciences and Technology Education 13 (8), 5961–5967. https://doi.org/10.12973/eurasia.2017.01044a Harvey, L. A. (2018). Gift, honorary or guest authorship.  Spinal Cord , 56(2), 91–91. https://doi-org.vu-nl.idm.oclc.org/10.1038/s41393-017-0057-8 Haven, T. L., de Goede, M. E. E., Tijdink, J. K., & Oort, F. J. (2019). Personally perceived publication pressure: revising the Publication Pressure Questionnaire (PPQ) by using work stress models.  Research Integrity and Peer Review , 4(1), 1–9. https://doi.org/10.1186/s41073-019-0066-6 Haven, T., Bouter, L., Mennen, L., & Tijdink, J. (2022). Superb supervision: A pilot study on training supervisors to convey responsible research practices onto their PhD candidates.  Accountability in Research , 1-18. https://doi.org/10.1080/08989621.2022.2071153 Haven, T., (2021) Towards a responsible research climate . Vrije Universiteit Amsterdam. 126 empowering supervisors towards responsible research conduct in supervision ... Kalichman, M.W ., & Plemmons, D.K. (2018). Intervention to Promote Responsible Conduct of Research Mentoring. Science and Engineering Ethics , 24(2), 699–725. https://doi.org/10.1007/s11948-017-9929-8 Lokhtina, I., Löfström, E., Cornér, S., & Castelló, M. (2020). In pursuit of sustainable co-authorship practices in doctoral supervision: Addressing the challenges of writing, authorial identity and integrity. Innovations in Education and Teaching International , 59(1), 82–92. https://doi.org/10.1080/14703297.2020.1799839 Lerouge, I., & Hol, A. (2020). Towards a research integrity culture at universities: From recommendations to implementation.  League of European Research Universities . Löfström, E., & Pyhältö, K. (2019). What are ethics in doctoral supervision, and how do they matter? Doctoral students’ perspective. Scandinavian Journal of Educational Research , 64(4), 535–550. https://doi.org/10.1080/00313831.2019.1595711 Muthanna, A., & Alduais, A. (2020). A thematic review on Research Integrity and Research Supervision: Relationships, Crises and Critical Messages.  Journal of Academic Ethics , 19(1), 95–113. https://doi.org/10.1007/s10805-020-09368-z Netherlands Code of Conduct for Research Integrity. (2018). NWO. https://www.nwo.nl/en/netherlands-code-conduct-research-integrity Pierson, E., Vora, V ., Pells, R., Grove, J., Upton, B., Jack, P ., & Ross, J. (2019, January 17). Third of PhD students in Europe “fail to complete in six years. ” Times Higher Education (THE). https://www.timeshighereducation.com/news/third-phd-students-europe-fail-complete-six-years Rathmell, W . K., Brown, N. J., & Kilburg, R. R. (2019). Transformation to academic leadership: The role of mentorship and executive coaching. Consulting Psychology Journal: Practice and Research , 71(3). https://doi.org/10.1037/cpb0000124 Roje, R., Reyes Elizondo, A., Kaltenbrunner, W ., Buljan, I & Maru šić, A. (2022). Factors influencing the promotion and implementation of research integrity in research performing and research funding organizations: a scoping review, Accountability in Research , 1-39. https://doi.org/10.1080/08989621.2022.2073819 Saqipi, B., & Vogrinc, J. (2020). The development of teacher research as a form of developing teacher pedagogical practice. Center for Educational Policy Studies Journal , 10(3), 5-9. Scherpenberg, C. van, Bultema, L., Jahn, A., Löffler, M., Minneker, V ., & Lasser, J. (2021, March 28). Manifestations of power abuse in academia and how to prevent them . Elephant in the Lab. https://elephantinthelab.org/manifestations-of-power-abuse-in-academia-and-how-to-prevent-them Schlossberg, P ., & Cunningham, C. (2016). Diversity, instructional research, and online education. Communication Education , 65(2), 229-232. https://doi.org/10.1080/03634523.2015.1098713 The Embassy of Good Science. (2022, November 29). Main Page . https://embassy.science/wiki/Main_Page Theunissen, B., & van den Hoven, M. (2021). Taking RCR seriously in teaching scientific integrity. Bioethica Forum . https://doi.org/10.24894/bf.2021.14010 UK Data Service. (2023, January 5). Checklist. https://ukdataservice.ac.uk/learning-hub/research-data-management/plan-to-share/checklist/ c e p s Journal | V ol.13 | N o 3 | Y ear 2023 127 Biographical note Miriam v an Loon, PhD, is a post doc at the university medical center (Amsterdam UMC) at Free University in Amsterdam and works on projects both in research integrity and research ethics. She has been involved in the H2020 project INTEGRITY. Her work focuses on research on improvement of responsible conduct of research (RCR) through education and institutional changes, currently working on a review of measurements to determine the ef - fectiveness RCR of training. Mariëtte van den Hoven, PhD, is a full professor in medical phi - losophy and medical ethics at university medical center (Amsterdam UMC) at Free University in Amsterdam. Her work focuses on the fields of research ethics, research integrity, professional ethics and public health ethics. She is coordinator of the Netherlands Research Integrity Network (NRIN) and one of the two founders of NERQ, a network on educational in research quality in Europe. She has been coordinator of the H2020 project INTEGRITY on devel - oping educational tools in research integrity training.