Iztok Devetak Editor University Chemistry Teaching in the 21. Century University Chemistry Teaching in the 21. Century Editor Dr. Iztok Devetak Faculty of Education University of Ljubljana Ljubljana, Slovenia Reviews Dr. Jerneja Pavlin Dr. Miha Slapničar Published by University of Ljubljana, Faculty of Education For publisher Dr. Janez Vogrinc, Dean Technical Editors Dr. Iztok Devetak, Luka Ribič Available at (URL) https://zalozba.pef.uni- lj.si/index.php/zalozba/catalog/book/198 DOI 10.26529/9789612532970 Ljubljana, October 2022 © University of Ljubljana, Faculty of Education, 2022 ………………………………………………………………………………………………….. Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID 127261443 ISBN 978-961-253-297-0 (PDF) ………………………………………………………………………………………………….. - 2 - Contents Preface ………………………………………………………………………………………. 4 Editor and Contributors ………………………………………………………………………………………. 7 Chapter 1 Context and Inquiry-Based Chemistry Teaching and Learning for Engineering Students Gabriel Pinto and Isabel López-Hernández ………………………………………………………………………………………. 12 Chapter 2 Smartphone-Based Analytical Procedure in the Teaching Lab: A Proposal for Undergraduate Students Roberto Sáez Hernández, Agustín Pastor, Ángel Morales-Rubio, and Maria Luisa Cervera ………………………………………………………………………………………. 26 Chapter 3 Students’ Identification and Application of Models to Rationalize Organic Acid-Base Trends Sean Gao, Taylor C. Outlaw, Jason G. Liang-Lin, Alina Feng, Jennifer L. Roizen, Colton Melnick, and Charles T. Cox Jr. ………………………………………………………………………………………. 40 Chapter 4 A Whole Team Approach to Integration of Student Feedback into Continuous Assessment Activities for First-Year Students Transitioning to University Chemistry Education During the COVID-19 Pandemic Frances Heaney, Denise Rooney, Orla Fenelon, Tobias Krämer, Eithne Dempsey, Stephen Barrett, Caytlin Boylan, Kyle Doherty, Luke Marchetti, Joseph Curran, Lisa O’Regan, and Trinidad Velasco-Torrijos ………………………………………………………………………………………. 68 Chapter 5 Roadmap for Continuous Professional Development of STEM Lecturers Nataša Brouwer, Ştefania Grecea, Johanna Kärkkäinen, Iwona Maciejowska, Matti Niemalä, and Lotte Schreuders ………………………………………………………………………………………. 85 Subject Index ………………………………………………………………………………………. 112 - 3 - Preface This peer-reviewed monograph entitled “University Chemistry Teaching in the 21. Century” includes five chapters presenting different approaches to university chemistry teaching and learning. The chapters have their basis in EUROVARIETY 2021 conference presentations organised by University of Ljubljana, Faculty of Education, Slovenian Chemical Society and European Chemical Society in July 2021. The importance of effective chemistry teaching and learning at university level is addressed in this book. We often think that chemistry teaching at university level is not so important as teaching this abstract subject at lower and upper secondary level, but we all know that also students at tertiary level struggle to learn chemistry. For that reason, researching and applying new teaching and learning strategies in tertiary chemistry lecture rooms and laboratories is as important as doing this for lower educational levels. Authors come from 6 countries and presenting their views on tertiary chemistry education, from teaching chemistry to engineering students, using modern technology in analytical chemistry, using models in organic chemistry course, and how COVID-19 influences on first year university students’ chemistry learning. The book ends with the chapter dealing with continuous professional development of STEM lecturer at the university level . Gabriel Pinto and Isabel López-Hernánde in the first chapter entitled “Context and inquiry-based chemistry teaching and learning for engineering students” show some applications carried out in recent years with first-year Industrial and Chemical Engineering students who study Chemistry. The aim of these kinds of experiences is that students solve, as a team, a series of problems and cases contextualized in their day-to-day life. To do that, they must understand a given problem, search for the underlying data set, analyse different information sources (in Spanish and English) for the data search, discriminate between the contents of the subject (and others) that they must apply, carry out experiments (in some cases), proceed according to an accurate data processing, make approximations, analyse the results (whose outcome is open) and propose future inquiries and applications. In detail, the cases included in this publication, chosen among dozens of others that have been adopted and implemented, are an experimental study of the ice melting rate in various liquids, calculations, and analysis of the relationships between vehicle fuel consumption and carbon dioxide emissions, discussion about self-heating beverage containers, chemical and thermodynamic fundamentals of domestic condensing boilers, and critical analysis of pseudoscientific deceptive information. These examples, suitable for other studies and stages of education, show that students are more interested in the subject and they acquire skills in a more appropriate way than with the use of more traditional problems, which are of a closed nature regarding the baseline data and with unique findings. Therefore, there is a clear contribution to the education of more responsible citizens with better knowledge of some products and technologies that they use in their daily - 4 - lives. On the other hand, these experiences and other similar ones, have been made available for secondary teacher training courses to promote its use in pre-college educational stages. In the second chapter’s entitled “Smartphone-based analytical procedure in the teaching lab: a proposal for undergraduate students” by Roberto Sáez Hernández, Agustín Pastor, Ángel Morales-Rubio, and Maria Luisa Cervera main objective is to serve as a guide for chemistry instructors willing to implement the use of the smartphone in the undergraduate laboratory. Different samples have been studied and the laboratory protocol described, in order to ease the adaptation to the preferences of each reader. Additionally, the obtained results are shown and the theoretical concepts discussed. In this chapter, a proposal to get the students involved in the analysis process is made using the smartphone as an analytical detector. To it, a simple setup was built from locally acquired materials, and phosphate was analyzed based on color parameters extracted from the image. As an analyte, phosphate has been chosen due to its wide appearance in diverse matrices and its importance in the industry. Based on the RGB color space, phosphate can be easily analyzed in water, washing powders, eyedrops and blood matrices. Overall, with this lab practice students can use their own smartphone to carry out the analysis, and optimize the image conditions that best suit their device. Additionally, Green Analytical Chemistry principles are implemented in the approach to ensure that students can identify them. The third chapter by the group of authors Sean Gao, Taylor C. Outlaw, Jason G. Liang-Lin, Alina Feng, Jennifer L. Roizen, Colton Melnick lead by Charles T. Cox Jr. entitled “Students’ Identification and Application of Models to Rationalize Organic Acid-Base Trends” discuss that acid-base chemistry is an essential component of the undergraduate chemistry curriculum. Acid-base concepts are introduced in general chemistry and expanded on in organic chemistry, biochemistry, and other advanced chemistry courses. Through a mixed approach of surveys and think-aloud interviews, the proficiency of second-semester organic students in acid-base chemistry was measured. Students were given two questions, both requiring them to rank the acidity of three compounds and justify the ranking. The first question focused on substituted carboxylic acids, and the second question focused on substituted aromatic structures. Although most students were able to correctly rank both sets of molecules, the correctness of their justifications was structure-dependent. Students did better at justifying the acidity of the aromatic structures, but while they were more successful at ranking the acidity of the substituted carboxylic acid structures, students were largely unsuccessful at justifying the trend. Students often relied on memorization, attributing acidity to the presence of specific functional groups or substituents. Specific alternative conceptions observed in both surveys and interviews include the idea that resonance structures are always central in justifying properties for molecules that have π bonds, and that alkenes and alkynes have differing numbers of resonance structures given they have different bond orders. Finally, students had difficulty with identifying the most acidic proton and often selected sites based on content that could be memorized from lectures. Students were also asked to report their confidence in their answers on a 6-point Likert scale from 0–5, and statistically significant differences were observed between students who ranked compounds correctly versus incorrectly for both questions in the study. However, when comparing the correctness of the justifications, a statistical difference between reported confidence was only observed with the aromatic structures question. The substituted carboxylic acids question required the application of models and ideas that extended beyond memorization. - 5 - The fourth chapter with the title “A Whole Team Approach to Integration of Student Feedback into Continuous Assessment Activities for First-Year Students Transitioning to University Chemistry Education during the COVID-19 Pandemic” by Frances Heaney, Denise Rooney, Orla Fenelon, Tobias Krämer, Eithne Dempsey, Stephen Barrett, Caytlin Boylan, Kyle Doherty, Luke Marchetti, Joseph Curran, Lisa O’Regan, and Trinidad Velasco-Torrijos” reports on a team-led, discipline specific solution to the problem of gathering, analysing and responding to student feedback on teaching and learning in a timely manner and in a way that supports transition to university for the in-situ group. We demonstrate the pedagogical value of integrating student feedback into mainstream, on-going teaching and learning activities as a vehicle to increase engagement and improve representation. The cohort was a large and diverse first year chemistry class (350 students) transitioning from second to third level teaching and learning methods in an Irish University during the COVID-19 pandemic where the bulk of activities were delivered either remotely or in a blended fashion. We show that a continuous assessment framework can be piggy-backed to gather student feedback and enact informed improvements in a manner which is both immediate and noticeable. We believe our approach is an excellent fit for chemistry programmes that could readily and Iy be incorporated into other programmes in cognate subjects and could be easily adapted for second and higher year students. The last chapter by Nataša Zupancic Brouwer, Ştefania Grecea, Johanna Kärkkäinen, Iwona Maciejowska, Matti Niemalä, and Lotte Schreuders entitled “Roadmap for continuous professional development of STEM lecturers” presents the Erasmus+ project STEM-CPD@EUni. Despite the ongoing systematic integration under the Bologna Agreement, higher education systems in Europe are still different in different countries and have different focus areas in the professional development of lecturers. At many European universities, professional development is often organised from a pedagogical point of view and the lecturers are left alone to apply the acquired pedagogical knowledge in their own teaching practice. In the Erasmus+ project STEM-CPD@EUni, five European universities and the European Chemistry Thematic Network (ECTN) are collaborating to enable continuous professional development (CPD) in a local university STEM teaching practice. A new concept in CPD is introduced, the CPD ambassador. Three dimensions characterize the activities of the CPD ambassadors in their local context: (1) STEM educational competences, (2) teaching attitudes, and (3) CPD activities. To define the needs for the CPD in these dimensions, a survey was developed with 66 statements evaluated from two different perspectives: their general importance for the quality of teaching and learning and their use in teaching practice. 420 lecturers from 80 universities from 26 countries and 46 education managers from 31 universities from 11 countries in Europe completed the survey from the end of November 2020 to the end of January 2021. The results show similarities and also some differences between the European countries and indicate in which directions the CPD is needed. The survey also showed that the priority list of needs for CPD should not be blindly followed but used in an evidence-based way. It is recommended to repeat the survey after some time. Based on the results of this research, a roadmap for STEM-CPD with guidelines and recommendations was developed in the STEM-CPD@EUni project. It can be concluded that chapters in this book can influence modernising chemistry teaching at university level and for that reason they can be recommended to novice and expert tertiary education lecturers. Ljubljana, Slovenia prof. dr. Iztok Devetak - 6 - Editor and Contributors About the Editor Dr. Iztok Devetak is Professor of Chemical Education at University of Ljubljana, Faculty of Education, Slovenia. His research focuses on how students, from elementary school to university, learn chemistry at macro-, submicro- and symbolic level, how chemistry in context and active learning approaches stimulate learning, using eye-tracking technology in explaining science learning, aspects of environmental chemistry education, developing teachers’ health-managing competences etc. He has been involved in research projects in the field of science education and he was the national coordinator of PROFILES project (7th Framework Program) for 4.5 years. He co-edited a Springer monograph about active learning approaches in chemistry. He (co)authored chapters in international books (published by Springer, American Chemical Society, Routledge, Nova Science...) and published papers in respected journals (altogether about 400 different publications). He was a Fulbright scholar in 2009. He is a member of ESERA (European Science Education Research Association) and EuCheMS (European Association for Chemical and Molecular Sciences) Division of Chemical Education and he is also a Vice-chair for Eastern Europe of EuChemS DivChemEd. He is a Chair of Chemical Education Division in Slovenian Chemical Society and president of the national Subject Testing Committee for chemistry in lower secondary school. Dr. Devetak is Editor-in-Chief of SCOPUS indexed CEPS Journal, publishing papers from different educational fields and editorial board member of respected journals, such as Chemistry Education Research and Practice, Eurasian Journal of Physics and Chemistry Education. - 7 - About the Contributors Dr. Charlie Cox is an associate professor of the practice and director of undergraduate studies at Duke University. Charlie got a Ph.D. under the guidance of Melanie Cooper in 2006. Charlie worked as a lecturer for 10 years prior to joining Duke in 2020. Charlie’s research interests focus upon students’ longitudinal progression across the chemistry curriculum with a specific interest in acid base chemistry, kinetics, and thermodynamics. Dr. Gabriel Pinto received an MSc degree in chemistry in 1985 and a PhD in physical chemistry in 1990, both from the Complutense University of Madrid. He has, since 1986, been Professor (since 2010 as Full Professor) in the Technical University of Madrid, where he teaches chemistry and communication of science for engineering students, and science education for future teachers. His research has concentrated on characterization of polymers and polymer composites, on the teaching of chemistry and STEAM at different educative levels, and on history of science. He has published over 200 papers in national and international journals. He is an active member of scientific societies and he has received about 20 awards at his University and at national level, for his tasks developed in educational innovation and outreach of science. Dr. Isabel López Hernández holds a Ph.D. in English Literature from Universidad Complutense (Spain, 2005). She is an Associate Professor at the Technical University of Madrid (UPM) where she teaches English for Professional and Academic Communication. He has also taught a diverse variety of Cultural Studies, Applied Linguistics, and Literature courses. Her research specialization is the study of new teaching methodologies. She has published in journals of national and international reputation in the field of Language Learning Strategies and English for Specific Purposes. Sean Gao is a senior undergraduate chemistry major at Duke University and has been member of the Dr. Coxs’ research group since 2020. Taylor Outlaw is a third-year graduate chemistry Ph.D. student. Taylor received a Bonk Fellowship, which provides funding for a one-year research experience in a chemical education group. Alina Feng is a senior undergraduate student at Duke University. Alina joined the Coxs’ research group as one of the first members. Colton Melnick is senior undergraduate student at Duke University. Dr. Jennifer Roizen has a Ph.D. in organic chemistry from California Institute of Technology, and she completed a postdoc at Stanford University. Dr. Nataša Brouwer is a senior consultant STEM higher education working at Teaching and Learning Centre Science, University of Amsterdam. She has a PhD in chemistry. Her expertise is the professional development of academic STEM educators in teaching and learning and the use of digital technology in higher education. She was involved in many national and international education innovation projects and in the European Chemistry Thematic Network (ECTN) Working groups. She has published more than thirty publications in the field of higher STEM education. - 8 - Dr. Ştefania Grecea is Associated Professor at the Faculty of Science of the University of Amsterdam where she leads the Functional Materials research group. She is chair of the Quality Standards in Teaching Standing Committee of the European Chemistry Thematic Network. She is also teaching at both BSc and MSc levels and coordinates the MSc Chemistry tracks Science for Energy and Sustainability and Molecular Sciences. Dr. Grecea is project leader of the Erasmus+ STEM-CPD@EUni at the University of Amsterdam and together with a team of lecturers and educational developers focuses on the development of activities for the continuous development of chemistry lecturers. Dr. Johanna Kärkkäinen is university lecturer at the Research Unit of Sustainable Chemistry, University of Oulu. She has taught organic chemistry in all study levels. Her research focus on lignocellulosic biomasses, their chemical conversion and modification. She has published over 25 papers in the international scientific journals. In addition to supervising Bachelor, Master and PhD theses, she has worked as Erasmus coordinator in the field of chemistry. Dr. Iwona Maciejowska is the chairman of the Jagiellonian University Ars Docendi Council, professor at the Department of Chemistry Education JU, co-editor of the journal Chemistry Teacher International (De Gruyter, CCE IUPAC). Activities related to increasing the teaching competences of academic staff, including conducting workshops for lecturers and doctoral students, are of particular importance to her. Author and / or scientific editor of several books and monographs, including: A Guidebook of Good Practice for Pre-Service Training of Chemistry Teachers (2015). Maciejowska I., & Byers B. (ed.), and over 100 articles in the field of natural sciences and higher education. Dr. Matti Niemelä is University lecturer at the Research Unit of Sustainable Chemistry, University of Oulu. He teaches inorganic and analytical chemistry at all study levels. He is also the leader of the chemistry degree programme at the University of Oulu. His research interests include fundamentals and applications of sample pre-treatment and atomic spectrometric techniques. He has published over 40 papers in the international scientific journals. Lotte Schreuders MSc after studying chemistry and working as a lecturer within the MSc Chemistry, track Analytical Sciences (Faculty of Science), Lotte Schreuders is now working as an educational researcher at the Teaching and Learning Centre of the University of Amsterdam. The project is a collaboration between the University of Amsterdam Teaching and Learning Centre and the research group Educational Sciences (Department of Child Development and Education, University of Amsterdam) where we investigate educational innovation and professional development in higher education. Dr. Frances Heaney holds a BSc and a PhD from Queen’s University Belfast, after postdoctoral research at Trinity College Dublin she began her academic career at Galway University before taking up a post at Maynooth University where she has been teaching all levels of undergraduate students and supervising PhD and MSc students since 1999. Current interests include the synthesis of new antimicrobial agents and chemistry laboratory education. Dr. Denise Rooney obtained both her BSc and a PhD from Queen’s University Belfast. She held a postdoctoral position at the University of York in the UK and then was appointed as a lecturer at Maynooth University in 1994. Her research interests are in organometallic and materials chemistry, and she is currently investigating metal-based compounds which can act as antimicrobial agents. She is leading an Irish inter-institutional project on using educational technologies to enhance chemistry laboratory education. - 9 - Orla Fenelon holds a Hdip and a BSc from Maynooth University and a MSc from Trinity College Dublin. She spent 18 years as a chemistry technical officer in Maynooth University and was involved in teaching practical chemistry to undergraduate students. She has a keen interest in science outreach and was involved in many activities promoting chemistry to all ages. She currently works as a chemist in the State Laboratory. Dr. Tobias Krämer studied Chemistry at the Universities of Bonn and Hamburg in Germany, and after receiving a MSc he completed a Dphil at the University of Oxford. Following postdoctoral positions in Germany and Scotland, he was appointed as an Assistant Professor at Maynooth University in 2017. He is strongly involved in lecturing within the undergraduate curriculum and the supervision of PhD and project students. His research interests centre around the electronic structure of inorganic systems, encompassing transition metal and main group compounds. Besides research, Tobias has a broad interest in the pedagogy relating to learning in laboratories, research skills and best practices in lecturing. Dr. Eithne Dempsey received a PhD in Electroanalytical Chemistry from Dublin City University followed by postdoctoral research at St. Vincent’s Hospital, Dublin. She then took up a position as Lecturer in Chemistry at Technological University Dublin – Tallaght Campus where she managed the Centre for Research in Electroanalytical Technologies. She was appointed visiting Professor of Chemistry at University of the Western Cape, Capetown, South Africa in 2012 and in 2017 joined Maynooth University, currently as Associate Professor (since 2020) in Dept. Chemistry. Her research objective is to address electroanalytical challenges using bespoke (nano)materials integrated with chemo/biosensing systems suitable for onsite deployment in multiple application scenarios. Dr. Stephen Barrett holds a BSc in Chemistry and PhD from Maynooth University. After completion of his doctoral thesis Stephen took up a 1 year post as a lecturer in Maynooth University. He has extensive experience in teaching undergraduate Chemistry labs and lectures. Stephen is currently an Irish Research Council awardee and is carrying out post-doctoral research at Royal College of Surgeons Ireland in the area of bioorthogonal Chemistry and Platinum based anti-cancer therapies. His current interests are synthesis of novel and selective Platinum and copper based anti-cancer complexes, bioorthogonal Chemistry and undergraduate education. Caytlin Boylan obtained a BSc in Pharmaceutical and Biomedical Chemistry from Maynooth University in 2018 and is currently a PhD candidate in Neurochemistry. She has recently taken an assistant lecturer role in physical and analytical chemistry. Her research interests include developing electrochemical biosensors for neurochemical monitoring and she has a strong interest in chemistry teaching and education. Kyle Doherty graduated from Maynooth University with BSc in Pharmaceutical and Biomedical Chemistry in 2019. He has worked both in the State Laboratories and in Randox Laboratories. Kyle is currently completing a PhD in the field of Synthetic and Carbohydrate Chemistry, where he is preparing new glycoconjugates against opportunistic fungal pathogens. - 10 - Dr. Luke Marchetti obtained his BSc in Chemistry from Maynooth University where he continued his studies and completed his PhD in Supramolecular Organic Chemistry in 2022. Luke has recently undertaken a post-doctoral research position in Maynooth University, funded by Science Foundation Ireland and Janssen Pharmaceuticals. His research interests centre around supramolecular host-guest chemistry and its applications in medicinal chemistry. Dr. Joseph Curran was a Postdoctoral Researcher at the Centre for Teaching and Learning, Maynooth University between 2022 and 2021 where he worked on the “Enhancing Teaching and Learning through Programme and Module Evaluation” Initiative. He is particularly interested in exploring disciplinary-appropriate student feedback methods. He is also an urban historian and is currently an Irish Research Council Government of Ireland Postdoctoral Research Fellow at the Department of History, Trinity College Dublin. Dr. Lisa O’Regan is Head of the Centre for Teaching and Learning in Maynooth University and Project Manager for the ‘Enhancing Teaching and Learning through Programme and Module Evaluation’ initiative. Lisa holds a BA, Hdip Education and H.D BFIS from University College Cork and a MSC in Education and Training Management from Dublin City University. Lisa has over 15 years’ experience in Higher Education digital learning, educational development and project management and is particularly interested in the area of assessment and feedback. Dr. Trinidad Velasco-Torrijos obtained her BSc from Universidad Autonoma de Madrid, Spain, where she studied Organic Chemistry. She then completed her PhD in Supramolecular Chemistry from University of Bristol, United Kingdom. After holding postdoctoral positions in Belgium and Ireland, she took up a position as lecturer at Maynooth University in 2007. Her research interests concern the development of carbohydrate-based antiadhesion agents against infection and soft materials. Trinidad is actively involved in Teaching and Learning projects in Chemical Education seeking to enhance the learning experience in the laboratory. - 11 - Chapter 1 Doi: 10.26529/9789612532970/ch1 Context and Inquiry-Based Chemistry Teaching and Learning for Engineering Students Gabriel Pinto 1,2* and Isabel López-Hernández 1 1 Technical University of Madrid (UPM), Madrid, Spain 2 Spanish Royal Society of Chemistry (RSEQ), Madrid, Spain * Corresponding author: gabriel.pinto@upm.es Abstract The central idea of this chapter is to show some applications carried out in recent years with first-year Industrial and Chemical Engineering students who study Chemistry. The aim of these kinds of experiences is that students solve, as a team, a series of problems and cases contextualized in their day-to-day life. To do that, they must understand a given problem, search for the underlying data set, analyse different information sources (in Spanish and English) for the data search, discriminate between the contents of the subject (and others) that they must apply, carry out experiments (in some cases), proceed according to an accurate data processing, make approximations, analyse the results (whose outcome is open) and propose future inquiries and applications. In detail, the cases included in this publication, chosen among dozens of others that have been adopted and implemented, are an experimental study of the ice melting rate in various liquids, calculations, and analysis of the relationships between vehicle fuel consumption and carbon dioxide emissions, discussion about self-heating beverage containers, chemical and thermodynamic fundamentals of domestic condensing boilers, and critical analysis of pseudoscientific deceptive information. These examples, suitable for other studies and stages of education, show that students are more interested in the subject and they acquire skills in a more appropriate way than with the use of more traditional problems, which are of a closed nature regarding the baseline data and with unique findings. Therefore, there is a clear contribution to the education of more responsible citizens with better knowledge of some products and technologies that they use in their daily lives. On the other hand, these experiences and other similar ones, have been made available for secondary teacher training courses to promote its use in pre-college educational stages. Keywords: Inquiry-based learning, Contextualized Teaching, Consumer Chemistry, Chemistry for the Citizen, Competency-based learning. Introduction Students are often insufficiently interested in chemistry because they perceive science education as “irrelevant” both for themselves and for society (Dillon, 2009). This is a very worrying situation for students of the different engineering fields who usually perceive chemistry as a subject that is more remote from their interests than others, such as those related to math and physics. This work is part of a research program intended to help teachers include connections between students' daily experiences and chemical topics. The idea is that by bringing tangible examples we provide opportunities for students to apply science to familiar contexts with the hope that they will appreciate chemistry more and will be motivated to study concepts in greater detail. - 12 - We summarize here the results about contextualized open-ended and real-world problems and cases carried out with first-year engineering students, applicable to other educational stages and fields. There are only five examples, among several dozens of others, which have been developed in the last three decades and a half, which are easily accessible through the papers collected on the website https://bit.ly/33UdyYD. The goals and challenges were: - To improve students’ (and also teachers’) motivation. - To encourage critical thinking skills in students. - To promote science literacy and social responsibility for the education of students as citizens. - To facilitate the acquisition of concepts (stoichiometry, thermochemistry, combustion, fuels composition, conversion factors...). - To collaborate on the development of competencies (problem-solving, data analysis, teamwork…). - To promote science literacy, social responsibility, and the understanding of Science-Technology-Society (S-T-S) relationships. To address these goals, it has been considered appropriate to use updated educational tools (such as inquiry-based learning, active learning, and team participation). Besides, during the development of the cases, it has been observed that they are very useful to detect alternative conceptions and misconceptions in students, a fundamental aspect to bear in mind in the teaching and learning process. These aspects are summarized in Figure 1.1. Figure 1.1. Outline of the challenges, goals, and teaching tools used in the methodology described in this paper, based on the use of real-world problems and cases in the learning of chemistry. Some general characteristics of the activities suggested to the students were the following: use of short class time (usually less than 5 minutes in an hour-long class), and around 3 weeks to solve each one as homework; groups of 3 students; account for 10% of the grade; open results and the need of data mining. Also, the general rules indicated for students were, in short: the - 13 - reports must be written (including, where appropriate, the preparation of tables, drawings, and graphics); results must be indicated (with appropriate units and care taken in the significant digits); proper citation of references and sites visited; and recommendation of discussion of the activity with the teacher before the delivery of the final project. The stages and characteristics of the methodology of problem-solving followed for these cases were similar to those discussed by Rodríguez-Arteche and Martínez Aznar (2016). To solve each case, the students must understand the problem posed, search for the initial data, analyse the different sources of information (in English and Spanish) for the search of the data, select the knowledge of the subject (and others) that they must apply, experiment (in their case), proceed according to an appropriate data processing, make approximations, analyse the results (whose outcome is open within reasonable margins) and propose future inquiries and applications. Case 1: Rate of melting ice cubes in different aqueous solutions. As an example of a simple but useful experimental experience, students research at home about the variation of the rate of melting ice cubes in different aqueous solutions. This case serves to discuss physicochemical concepts (Pinto & Lahuerta, 2015) but, mainly, it is an experience that is usually done the first day of class to introduce students to the scientific method, underline the importance of critical thinking and highlight the advantages of teamwork. Thus, students are asked: “Where will it melt an ice cube before: in water or in water saturated with table salt?” After letting them think about it for a few minutes, the results are written down on the blackboard providing the opportunity to give 4 possible answers: “it melts before in water with salt”, “it melts before in water without salt”, “it is practically equal”, or “I don’t know”. As soon as the answers are gathered, the students are told to discuss the question in groups of three, asking the same questions next. The number of answers given to the 4 possible options changes, after the discussion as a group, which is a good occasion for the teacher to introduce the importance of teamwork. Also, during the time for discussion as a team, the teacher pays attention to the explanations given among the students. Typical examples are usually detected, for example, the fact that salt is added to prevent ice from roadways on very cold days because it lowers the melting point of water. Finally, students are suggested to recommend how it could be solved, as they have given different answers. Although sometimes they offer other alternatives (searching on the internet, enquiring experts on the topic, researching textbooks…), it is common, as a way of taking time to reflect, to find the option of doing it at their homes, using simple means, for example, a pair of glasses, water, table salt and two pieces of ice. Therefore, the teacher invites students to carry out experiences related to it, recommending them to study some other variable in detail too. Some of the results obtained during measurements are specified in Figure 2, in which it is observed that, opposite to what is normally considered a priori (not only by students but also by teachers too), the time of melting ice is faster with pure water. The phenomenon observed is due, mainly, to the different densities in liquid water at different temperatures and in salted water. Thus, I the case of ice in pure water, the water recently formed by the fusion descends, because it has a bigger density (at 0ºC approximately) than the liquid water at the beginning (at room temperature), forming convection currents that can be visualized easily adding some drops of food colouring (see Figure 1.2. ). On the contrary, in the case of the ice added to the salt solutions, the water of fusion remains on top, because it has - 14 - a lower density than the solution, without observing convection currents, so the ice is surrounded by colder water than in the previous case and the process is slower. Figure 1.2. Detail of the experiment: melting ice in water (left) and in saturated water of table salt (right) with some drops of food colouring. Experimental data were obtained by students from the time of melting ice (blocks of different sizes) in both liquids. Other questions the different groups of students can investigate, by their initiative and/or guided by the teacher, are: force the ice with a rod to be underwater, use of different concentrations of table salt, use of sugar instead of salt, use of constant agitation or fluid at rest, the measure of temperature to different heights of the liquid, and others. From their observations and a discussion of everybodyś results, the teacher can ask the students questions such as: “Why does one of the ice cubes move?”, “Why are water droplets produced in some cases on the outer wall of the vessel?” Surprisingly, some students suggest as an answer to this question that it is liquid water that oozes the “pores” of the glass, instead of suggesting that it is condensed water from the water vapour in the air. There are always further inquiries too, such as: “Is it possible to get a transparent (and not translucent) ice cube in the freezer at home?”, “What happens if we add an ice cube on cooking oil?”, “Why?” … They are experiences that can be studied thanks to simple practices that can be done, for example, in the kitchen at home. For instance, when you put an ice block in cooking oil, the ice stays on top, but the liquid water (of a higher density than the oil) produced by the melting of the ice sinks. Some of the results of these observations are shown in Figure 1.3. As an example, a group of students tried to work with ice blocks made up of different beverages in the freezer of their homes, and they found it weird that vodka did not freeze: that is the way they discovered, thanks to the experiment and own their own, what freezing-point depression is since this alcoholic drink has a 40 % vol. of ethanol. This was a starting point for the teacher to explain it and to comment on the example of the regular use of the automobile antifreeze. - 15 - Figure 1.3. On the left, surfaces of the experiment described in the text with the ice (on the left in pure water and on the right in water saturated with salt); it can be observed the formation of condensation water droplets, on the outside of glass cans, in the second case. On the right, ice in cooking oil at room temperature. This example is useful for motivating and also as a starting point to observe how properties of simple substances (water and sodium chloride) influence complex phenomena, such as thermohaline convection currents in oceans, caused by density gradients, as well as involved current topics, such as the circulation of microplastics at the bottom of the seas, a topic which is even discussed in the media. Case 2: Emission of CO2 by cars. The contribution of CO2 emissions to climate change is also frequently mentioned in the media. Through different questions and problems, students can discuss quantitative aspects related to the emissions, such as its relationship with vehicles fuel consumption, as it is posed in this activity. For this case, students must utilize basic chemical principles such as stoichiometry, density of liquids, and combustion reactions to calculate theoretical emission rates of CO2 which are then compared to actual consumer product information (Oliver-Hoyo & Pinto, 2008). Representing graphically the emission of CO2 versus consumption of fuel, it provides a tangible way of connecting concepts studied in chemistry classes to everyday life. Students are instructed to gather car CO2 emission and fuel consumption data from a specific source. In fact, different sources are available for the kind of information required in this exercise. For example, students can gather this data from auto supplements that appear in papers (auto advertisements), popular auto magazines, car manufacturer data information online, car sales labels, or websites, which provide the information required in this activity for a variety of carmakers and models and which is freely accessible. As the students collect this data, the link between chemistry and everyday life becomes stronger. The data are reported in units of grams per kilometre for emission of CO2 and litres/100 km for fuel consumption. Students must graph emission of CO2 (in g/km) versus fuel consumption (in L/100 km), which results in a line equation (see Figure 1.4. ) whose slope is comparable to the theoretical stoichiometric calculations for CO2 production in a combustion reaction for a particular fuel component. Since two major types of fuel are used (at least until now) in the automobile industry, gasoline and diesel, doing this exercise for the two types of fuels show differences between them in terms of CO2 emission levels and fuel consumption. Two assumptions for the theoretical calculations must be explicitly stated to the students: only the major component of the fuel is taken into account and all driving or climatic conditions are disregarded. For gasoline, octane, C8H18, is considered as the primary ingredient, and - 16 - dodecane, C12H26, for diesel. The strength of this exercise is that even after such simplification in its treatment, theoretical values are comparable to released consumer data. Figure 1.4. Example of CO2 emission representation versus fuel consumption according to a group of students. Stoichiometric relationships and the density of the fuel provide the necessary conversion factors to calculate the theoretical amount of CO2 produced. An example calculation considering the density of gasoline as 0.75kg/L (each group of students must choose the density value, either consulting a data table or measuring it in the lab, with the teacherś help) is shown below. For octane: C8H18 + 12.5 O2 → 8 CO2 + 9 H2O Octane c onsumptio (L n /100 km) kg 1 kmol octane 8 kmol CO CO e mission = · 0.75 · · 2 · 2 100 km L 114.22 kg kmol octane 44.01 kg 3 g CO · ·10 2 = 23 1 . (g CO / L ·k x m) Oc tane c onsumptio in (L n 10 0 k m) kmol CO kg 2 2 For gasoline the range falls between 21.6g CO2/L and 24.1g CO2/L (for 0.70-0.78kg/L density values) while for diesel the range is between 24.8g CO2/L and 30.7gCO2/L (for 0.80-0.99kg/L density values). When students graph the data from a specified source, regression analysis provides a slope that falls within the calculated range. Graphs by students use data from diverse car brands and models. These graphs show strong correlations to the theoretical values calculated and provide a quick visual account of the differences in emission and fuel consumption of gasoline versus diesel engines. Even though diesel engines consume less fuel, CO2 emissions reach higher levels per litre of diesel consumed. Despite this, when comparable engines are reviewed, diesel engines consume less fuel and subsequently have lower CO2 emissions per distance travelled. This activity can be extended to calculate the annual average CO2 emission per vehicle type. The magnitude of these numbers may be used to promote awareness of environmental issues in the classroom as well as an opportunity to discuss other current related topics such as the Paris Agreement and the Kyoto Protocol. - 17 - This activity may also be used to leap into discussions regarding environmental issues, hybrid engines, and the efficiency of gasoline versus diesel vehicles which may be of interest to students majoring in areas as diverse as public policy or engineering. This activity is an instructional resource that utilizes consumer product information to compare theoretical stoichiometric calculations to available car emission and fuel consumption data. Considerable simplification of an otherwise very complex chemistry problem still provides comparable theoretical and actual data that links chemistry principles to everyday life. Practice with unit conversion and graphing skills enhance this activity in a very practical way promoting skills used by professionals to perform emission measurements. Case 3: Thermochemistry of self-heating beverages. There are food commercial products that claim to heat their contents based on the dissolution process of a salt or a chemical reaction (Oliver-Hoyo, Pinto & Llorens-Molina, 2009; Pinto, Llorens-Molina & Oliver-Hoyo, 2009; Prolongo & Pinto, 2010). This is an example of inquiry-based/discovery learning, encouraged through an example of a commercialized device in Spain (https://the42degreescompany.com/) about the heating of commercially available self-heating beverages that takes place thanks to the hydration reaction of the calcium oxide. In Figure 5, it can be seen one of these kinds of drinks and the manufacturer's information specified on the label together with the different components used for its manufacture. The questions posed to the students to be solved in teams are the following: 1. Describe the container and the chemical reaction that takes place in the self-heating beverage. 2. Determine the excess and limiting reactants and calculate the mass of the product that can be formed. 3. Search (through different sources) the values of standard heat of formation, ΔfHº, at 25 ºC, for the substances involved in the reaction and present them in a table. Calculate the heat (kJ/mol) evolved. 4. Prepare a table with end temperatures (experimental, according to the manufacturer, and calculated theoretically) and compare them. 5. Identify and discuss the assumptions made. 6. Comment on the advantages and disadvantages of these containers for beverages and suggest ways to improve these cans. 7. Comment on any interesting aspect of this activity (possibility to cool beverages, instructions given by the manufacturer, additional information…). To answer the first question, the students must observe the information given by the manufacturer on the label and the website. It is the following reaction: CaO(s) + H2O(l) → Ca(OH)2(s) To solve question 2, the teacher offers, as data, the masses of the two reactants which have been previously determined by him or her in the lab; the can is opened carefully (see Figure 1.5. ) using a cutter and the substances are weighed on a scale, or the procedure can be carried out in the lab with the students (a more effective method from a pedagogical point of view). A specific case of masses of substances used and more details of the problem can be found in a previous paper (Prolongo & Pinto, 2010). The search of thermodynamic data needed in - 18 - question 3 shows the students that there are different sources of data, and not all share the same rigor. Taking into account the quantity of Ca(OH)2 that can be formed and the heat of the reaction (calculated from the data obtained in question 2), it is estimated the variation of temperature, ΔT, that the beverage would reach with the expression Q = ΔT · Σ (mi · Ci), in which Q is the heat released in the reaction, mi is the mass for each component, i.e. the different materials and substances involved (beverage, container and substances associated with the device) and Ci is the specific heat for the corresponding component (for the beverage, it can be considered as water). It is observed that the calculated final temperature theoretically is higher than the one measured with a thermometer and the one given by the manufacturer; among other reasons, it is due to the fact that, in all the thermodynamic processes, there is heat loss (ambient transfer, reaction yield …). Figure 1.5. Can, label and different components for the commercially available self-heating beverage described in the text. When students discuss this case, they consider, among the possible advantages, the benefit of drinking a warm beverage in places where it is difficult to heat it, for example, during an excursion to the mountain or when people go skiing. They take into account some disadvantages too, such as the increase in the price or the higher environmental impact compared to conventional beverages. Finally, students can go deeper into aspects related to other applications and designs of beverages, as is shown in Figure 1.6. With the new designs, the students put into practice what they have learned in other subjects during the year and they develop their creativity. Therefore, this activity can be considered an example of a STEAM (science, technology, engineering, arts and mathematics) educational case. - 19 - Figure 1.6. Self-heating beverages containers designed by students and a photo of students discussing the case. Case 4: Why should we use domestic condensing boilers? Among other examples posed to students, related to the quantitative evaluation of the reduction of CO2 emissions through several methods, such as the use of solar power (Pinto, 2009), it can be highlighted the use of domestic condensing boilers (Pinto, 2013). Here is a brief description of the questions posed to students to inquire about aspects related to this kind of boilers (they get liquid water instead of steam). The specific goals are to facilitate the learning concepts (enthalpy change, combustion, natural gas…) and at the same time, it promotes critical thinking and “consumer chemistry”, because of the discussion of aspects such as the causes of public support for the installation of condensing boilers and the use of household bills as an information source. For example, there is a known “Plan Renove” for domestic boilers in Spain that is part of the “Action Plan for Energy Saving and Efficiency” developed for promoting the use of “condensing boilers”. Taking into account that natural gas is the most used combustible in cities like Madrid, and after introducing the questions (for example, through a commercial advertisement, as the one in Figure 1.7. ) in the classroom, the type of questions posed to students to be solved as a team are the following: Figure 1.7. Commercial advertisement published in Madrid, in 2010, where citizens are invited to change a domestic conventional boiler by a condensing boiler. 1. Through suitable sources, collect in a table a typical composition of natural gas expressed as % vol. and mole fraction. - 20 - 2. Create a table with the composition of a “model” natural gas, considering only the two major hydrocarbons. 3. Consulting adequate sources, provide a table with data of standard heat of formation, ΔfHº (kJ/mol), at 25ºC, for gases selected in the previous section and of CO2(g), H2O(g) and H2O(l). 4. Calculate the standard enthalpy of combustion, ΔHºcomb (kJ/mol), of natural gas, at 25ºC, assuming that the water is obtained as gas. 5. Repeat calculation by assuming that the water is obtained as a liquid. 6. Determine the quantity of natural gas that should be used, in a condensing boiler, per each mole of natural gas that should be used in the other kind of boiler (the conventional one), to obtain the same energy. Discuss economic and social implications. 7. Discuss if the condensate water in the condensing boiler is acid or alkaline. 8. Itemize the assumptions made in your calculations. 9. Discuss any aspect of interest (additional data, sustainability, environment, need to subsidize the condensing boilers, obtaining natural gas ...). An example of a table in which it is described a typical composition of natural gas is shown in Table 1.1. Each group of students can give a different table, as the natural gas is a raw material of varying composition. Therefore, the students discover that the composition of natural products that consist of mixtures of multiple substances, such as natural gas, does not have only one composition. This seems weird to some students, who prefer to give compositions that look more exact and accurate. In question 1, it is also discovered that students do not usually have the habit of elaborating their tables, that sometimes the total composition exceeds 100% and that, frequently, they confuse the concepts of substance, chemical element, and compound. Table 1.1. Example of the composition of natural gas. Composition Substance Formula Molar % vol. fraction Methane CH4 87.0-96.0 0.870-0.960 Ethane C2H6 1.5-5.1 0.015-0.051 Propane C3H8 0.1-1.5 0.001-0.015 Isobutane C4H10 0.01-0.3 0.0001-0.003 Butane C4H10 0.01-0.3 0.0001-0.003 Isopentane C5H12 Traza-0.14 Traza-0.0014 Pentane C5H12 Traza-0.14 Traza-0.0014 Nitrogen N2 0.7-5.6 0.007-0.056 Carbon dioxide CO2 0.1-1.0 0.001-0.010 Oxygen O2 0.01-0.1 0.0001-0.001 Hydrogen H2 Trace-0.02 Trace-0.0002 - 21 - Question 2, although apparently simple, is usually a topic for discussion among students, because they find it difficult to understand that a simplification can offer an accurate result. One more time, it is expected that each group would offer a different composition for its “model” of natural gas, due to the fact that there are very different possibilities (always within certain limits). For example, one composition to simplify this kind of thermochemical calculations for the natural gas, is the one described in Table 1.2. A typical mistake in some groups of students consists in preparing a table in which the total amount of the two selected gases is not 100%. Table 1.2. Example of the composition of a “model” natural gas, considering only the two major hydrocarbons. Composition Substance Molar fraction % wt. Methane 0.850 75.1 Ethane 0.150 24.9 In question 3, similar to case 3 analysed previously, students discover that there are different ways of obtaining thermodynamic data, so they must choose the ones that are accurate, not always the same as those in other teams. Thus, the results obtained for question 4 could be as follows: CH4(g) + 2 O2(g) → CO2(g) + 2 H2O(g) ΔHº = -803kJ/mol C2H6(g) + 7/2 O2(g) → 2 CO2(g) + 3 H2O(g) ΔHº = -1429kJ/mol Taking into account the example in Table 1.2., it is obtained: ΔHºcomb = 0.85·(-803kJ/mol) + 0.15·(-1429kJ/mol) = -897kJ/mol Students solve question 4 with some difficulty, especially because of mistakes committed in the stoichiometry of the chemical reactions. When the previous calculation is repeated by assuming that the water is obtained as liquid (question 5), and, therefore, enthalpy of condensation of water at 25ºC (-44kJ/mol) is taken into account, the result is: ΔHºcomb = 0.85·(-891kJ/mol) + 0.15·(-1561kJ/mol) = -992kJ/mol In question 6, which is the key aspect in the case, using the data at the two previous sections, the calculation is: 897 kJ / mol gas conventio al n boiler = 992 kJ / mol gas conden. boiler = 90 . 0 4 mol gas conden. boiler / mol gas conventio al n boiler That is to say, the use of the condensing boiler guarantees a saving in the order of 10% of fuel (which Spain, as other countries, must import), and, also, that same percentage of CO2 emission decreases caused by the combustion. This way, students observe how aspects related to chemistry allow explaining a question of great interest that has an impact not only on the user but also in the country: facilitates the use of gas for future generations. In any case, there are not only advantages: condensing boilers are more expensive than the conventional ones and - 22 - they imply more maintenance cost, that is the reason why the State Administration offers grants to consumers to encourage the change. Besides this key issue, the case can have other applications, as suggested in question 7. The answer is that the condensate water in the condensing boiler is acid, due to the dissolution of the CO2 gas in that water, which produces the equilibrium: CO - 2(g) + H2O(aq) ⇄ HCO3 (aq) + H+(aq) In fact, the pH in the condensate water caused by condensing boilers is between 3 and 5, which causes certain technical problems in drainage. Question 8 allows students to think about the approximations in calculations, for example in the use of a model of natural gas. Finally, in question 9 they go deeper and discuss topics that arise during the handle of the case, for instance, which countries are Spain's main natural gas suppliers, what are the technical specifications of a condensing boiler, or even that they participate in the supervision of a boiler at their homes to take a closer look at the main components that make it up. As can be seen, apart from the practical development of physical and chemical knowledge and calculations, this type of problem raises classroom discussion of current issues, such as the importance of science for the achievement by 2030 of the Sustainable Development Goals (SDG), a call by United Nations for action by all countries – poor, rich and middle-income – to promote prosperity while protecting the planet. Case 5: Critical analysis of pseudoscientific deceptive information. Another case puts into practice the critical analysis of pseudoscientific deceptive information about some products. Among other examples, we have introduced students to the case of a certain salt contained in a glass ampoule that, according to the supplier, “changes” the bond angle of molecules of water and thus other properties of this liquid, with “healing effects” because it removes kidney stones. Another example is the case of a “special” bottle made with a glass “containing silica” that changes certain physicochemical properties of the water it contains, making it useful to heal certain diseases. In both cases, the students are provided with websites in which they will find the information (https://www.slackstone.com/en/ and https://www.flaska.eu/, respectively) to analyse in teams and then write a short report and discuss it in class. Surprisingly, a lot of students think that they are scientific indications, with beneficial health effects. Others consider, on the contrary, that an apparently scientific language is used, but it is not well-founded. The teacher helps in the discussion by convincing students of the fact that although it uses scientific jargon, it deals with impossible aspects (“can the bond angle of the water molecule be modified?”') or simply indifferent (“any common glass used in the manufacturing of bottles contains silica!”). Also, he/she helps them analyse how apart from the use of a pseudoscientific language, the information is based only apparently on relevant bibliographic references. This is a hot topic because of the proliferation of this kind of pseudoscientific information that, sometimes, is not banned by authorities because, despite not offering advantages, it does not have any harmful effects. It is also a topical issue of pseudoscientific information that defends the opposition to getting the Covid-19 vaccine, among other examples. - 23 - Outcomes Intending to supplement the traditional content of introductory chemistry for engineering students, with training in critical and creative thinking, we have suggested a few examples of solving open-ended problems. These examples clearly link different chemistry principles (phase changes, stoichiometry, chemical formulation, chemical thermodynamics, production of energy…) to real life and social issues. Other topics and skills were included, such as comprehension of information given in English, preparation of scientific data tables and graphs, rounding off in calculations, and searching of data. For example, students need to search for data such as atomic masses, enthalpies, composition of common products..., to solve the problems. By bringing tangible chemistry examples we provide an opportunity for students to apply chemistry to familiar products with the hope that they will be motivated to study concepts in greater detail, and will connect the relevance of chemistry to their everyday lives. First-year undergraduate engineering students appreciated the question-answer approach and were motivated and interested. They liked the activities and most of them gained an appreciation for the necessity to study chemistry as an introductory science for their specializations. Students’ responses are not always positive: some of them complain that “these topics are not part of the course syllabus” and prefer more conventional problems. But most of them express keen interest in this type of “tangible” chemistry where concrete examples of everyday life put textbook chemistry in context. Further, such cases promote training in “consumer chemistry”, and enable students to realize the relevance of chemistry outside the classroom, which is especially relevant for engineering students. By the other hand, the implications and environmental issues related to proposed chemistry studies make this science more relevant, real-life reflection, and practical to students. According to our experience, this kind of instructional tool is an effective way to help improve the students' engagement, motivation, and interest in chemistry. Some of the opinions given by students were the following: - This kind of exercise helps us better understand the world around us. - Makes chemistry a tangible experience so that it is not only solving problems on a piece of paper. - It shows chemistry is not only a set of formulas and is valuable for something. - Chemistry is very boring, any tool to make it more interesting is worth the try. - My chemistry teacher in high school said that “chemistry is everything”, and this helped me to see why. - It serves to relate concepts of chemistry with products we can easily find. - It can be observed the passion for the subject in the teacher when he/she poses practical examples In short, in this way, there is a contribution to the education of more responsible citizens and a better knowledge of some products and technologies used in their everyday lives. These experiences, and other similar ones, have been taught by us in secondary teacher training courses too, at our same University, to encourage its use in undergraduate stages. With this - 24 - kind of student (future teachers), the results are similar, and they show a great interest in applying them to their future students. Acknowledgments We are grateful for the support provided by the Technical University of Madrid (UPM) under the project “Implementing research-based learning” (IE22.0506) and by “la Caixa” Foundation under the “Science and technology within everyone’s reach” project. References Dillon, J. (2009). On Scientific Literacy and Curriculum Reform. International Journal of Environmental and Science Education, 4(3), 201-213. Oliver-Hoyo, M. T. and Pinto, G. (2008). Using the Relationship between Vehicle Fuel Consumption and CO2 Emissions to Illustrate Chemical Principles. Journal of Chemical Education, 85(2), 218-220. Oliver-Hoyo, M. T., Pinto, G. and Llorens-Molina, J. A. (2009). The Chemistry of Self-Heating Food Products: An Activity for Classroom Engagement. Journal of Chemical Education, 86(11), 1277-1280. Pinto, G. (2009). Determining the CO2 Emissions Averted by the Use of Solar Power. Journal of Chemical Education, 86(9), 1033. Pinto, G. (2013). Termoquímica de las Calderas Domésticas de Condensación: Un Caso de Aprendizaje Contextualizado por Indagación Dirigida. Educació Química, EduQ, 14, 29-38. Pinto, G. and Lahuerta, P. (2015). Velocidad de Fusión del Hielo en Distintas Disoluciones: Un Ejemplo de Aprendizaje Activo de la Ciencia. Educació Química, EduQ, 21, 54-62. Pinto, G., Oliver-Hoyo, M. T. and Llorens-Molina, J. A. (2009). Enjoy a Hot Drink, Thanks to Chemistry! Journal of Chemical Education, 86(11), 1280A-1280B. Prolongo, M. and Pinto, G. (2010). Las Bebidas Autocalentables y Autoenfriables como Recursos para un Aprendizaje Activo. Educació Química, EduQ, 7, 4-14. Rodríguez-Arteche, I. and Martínez-Aznar, M. M. (2016). Introducing Inquiry-Based Methodologies during Initial Secondary Education Teacher Training Using an Open-Ended Problem about Chemical Change. Journal of Chemical Education 93(9), 1528-1535. - 25 - Chapter 2 Doi: 10.26529/9789612532970/ch2 Smartphone-Based Analytical Procedure in the Teaching Lab: A Proposal for Undergraduate Students Roberto Sáez Hernández*, Agustín Pastor, Ángel Morales-Rubio, and Maria Luisa Cervera Department of Analytical Chemistry, Faculty of Chemistry, University of València, Valencia, Spain *Corresponding author: roberto.saez@uv.es Abstract This chapter’s main objective is to serve as a guide for chemistry instructors willing to implement the use of the smartphone in the undergraduate laboratory. Different samples have been studied and the laboratory protocol described, in order to ease the adaptation to the preferences of each reader. Additionally, the obtained results are shown and the theoretical concepts discussed. In this chapter, a proposal to get the students involved in the analysis process is made using the smartphone as an analytical detector. To it, a simple setup was built from locally acquired materials, and phosphate was analysed based on colour parameters extracted from the image. As an analyte, phosphate has been chosen due to its wide appearance in diverse matrices and its importance in the industry. Based on the RGB colour space, phosphate can be easily analysed in water, washing powders, eyedrops and blood matrices. Overall, with this lab practice students can use their own smartphone to carry out the analysis, and optimize the image conditions that best suit their device. Additionally, Green Analytical Chemistry principles are implemented in the approach to ensure that students can identify them. Keywords: Smartphone, phosphate, active learning, green analytical chemistry, digital image colorimetry. Introduction The importance of phosphorus Phosphorus is one of the most abundant elements in earth, being phosphate (PO 3-4 ) the most common specie. It is involved in many different biochemical processes (such as energy transfer, formation of DNA/RNA, pH buffer…). Also, it plays a major role in the formation of biological membranes, as phospholipids contain phosphate in their composition. It is because of that, that 700 mg of phosphorus is the recommended dietary allowance for adult population (Phosphorus – Health Professional Fact Sheet, 2021). This essential mineral is found in many different foodstuffs: milk, poultry, legumes or vegetables are a source of phosphorus. Additionally, phosphorus has found a wide variety of industrial applications, especially in the form of phosphoric acid (H3PO4). Among others, phosphoric acid is used for metal treatment, medicines, food additives or refractory industries (De Boer et al., 2019). Additionally, phosphate can form polyatomic complexes, called polyphosphates, which find use in the detergents industry as a surfactant. However, the most remarkable application is its presence in fertilizers as a nutrient for plants. Since phosphorus is a limiting factor in plant growth, applying it to the crops in fertilizers has increased the productivity to obtain more food per unit - 26 - of area (Sharpley & Menzel, 1987). Nonetheless, its excessive use in agriculture has generated a pollution problem in aquatic media: eutrophication. The application of phosphorus (and generally also nitrogen) in agricultural soils, ends up accumulating the excess of these nutrients in the aqueous environments, like lakes. Therein, an unusual growth of microorganisms is induced by the presence of sufficient N and P, distorting the natural equilibrium of the ecosystem. This process generally generates an increased rate of fish death (Smith & Schindler, 2009). Analysis of phosphorus As can be deduced from the previous section, phosphorus analysis is of great importance nowadays. Many different analytical procedures have been developed to detect and quantify phosphorus in a wide variety of samples. Due to the chemistry of the phosphorus, many of the methods target its phosphate form, since it is the most stable and common one. These methods can be classified into two major groups: chromatographic methods and colorimetric methods. Ion chromatography uses an analytical column to separate different ionic species based on electrostatic interactions (Fritz, 1987), and make use of a detector to correlate a physicochemical property (absorption, conductivity…) with the concentration of the analyte. Some analytical methods have been described to quantify phosphate using ionic chromatography using a carbonate buffer as a eluent and a conductimetric detector (Tabatabai & Dick, 1983). In this specific case, the linear range comprised up to 1.2mg L-1 of phosphorus, and the limit of detection (LOD) was 0.1mg L-1. When it comes to colorimetric procedures, a literature survey proves that many different methods and reactions have been developed. For instance, malachite green has been used to develop a colored signal which is proportional to the concentration of P in the medium, reaching the maximum sensitivity at 630nm and keeping a linear trend up to 0.620mg L-1 of P (Kallner, 1975). Similarly, quinine has also been proposed as a reagent to quantify phosphate, with a linearity up to 0.6mg L-1 of P and a LOD of 0.005mg L-1 of P (Kirkbright et al., 1972). However, the most remarkable one due to its simplicity and common use is the phosphomolybdenum blue method. This analytical procedure is based on the reaction of phosphorus (in the form of phosphate) with a Mo(VI) compound (MoO 2-4 ) in acidic media, following reaction 1. This first reaction produces a pale-yellow complex which can be related to the concentration of phosphate in the sample (Cinti et al., 2016). 51𝐻! + 7𝐻 &' "' "𝑃𝑂# + 12𝑀𝑜$𝑂%# → 7𝑃𝑀𝑜(%𝑂#) + 36𝐻%𝑂 [1] Nonetheless, this complex provides a low sensitivity to the method, and higher LODs are obtained. To solve it, a posterior reduction is often done. With it, the P-Mo complex is partially reduced to a Mo(VI) and Mo(V) complex, as can be seen in reaction 2. The resulting complex presents an intense blue color which allows to take more sensitive quantifications, lowering the LOD. 𝑃𝑀𝑜 "' *+ * "' (%𝑂#) + 2𝑒' → 𝑃𝑀𝑜()𝑀𝑜% 𝑂#) [2] - 27 - This reduction step can be done with different reagents: as stated in the APHA methods (American Public Health Association), ascorbic acid and stannous chloride are the most suitable ones (Rice et al., 2015). While the latter provides more sensitivity to the method, the former is more reproducible and robust. Either way, the phosphomolybdenum complex that is obtained can be quantified in the visible range, having a broad absorption band around 800nm. Digital image colorimetry The application of smartphones in Analytical Chemistry has been a growing tendency during the last few years, thanks to the improvement in their technical properties, and the increasing availability in the market (Capitán-Vallvey et al., 2015; Rezazadeh et al., 2019). Many different analytical problems have been addressed in the field of image treatment using smartphones or capturing devices. On the one side, regarding the inorganic analytes, iron (Mohamed & Shalaby, 2019), calcium (Peng et al., 2019), ammonium ion (Jaikang et al., 2020), lead (Seidi et al., 2014), chloride, nitrite (Sargazi & Kaykhaii, 2020) or pH (Lopez-Ruiz et al., 2014), are some examples of analytes studied with a smartphone as the analytical tool. On the other side, some examples of organic analytes determined using image treatment with smartphone devices are ascorbic acid (Aguirre et al., 2019), ethanol (Böck et al., 2018; Curbani et al., 2020; Marinho et al., 2019), and biomolecules like proteins (Gee et al., 2017). Additionally, different analytical parameters of interest have been also assessed with image treatment like the fermentation degree of cocoa (León-Roque et al., 2016). All of these are based on the capture of a specific color which is representative of the sample: either as an intrinsic characteristic, or derived from a specific reaction. Color can be defined as a mental perception to a specific part of the electromagnetic spectrum arisen from an object, either by reflection or by emission (Wu & Sun, 2013). The translation of this concept to the mathematical language is done by the color spaces, and comprises a part of colorimetry: the science devoted to quantification, analysis and decomposition of color (Wyszecki & Stiles, 2000). A color space is a way of transforming the visual experience of color into a numeric value (Kuehni, 2001). Most commonly, they consist in three different coordinates, which represent a specific quality of color, and are named tristimulus values, referencing the three specialized cones in the human eye, which capture three different ranges of wavelengths (Wu & Sun, 2013). One of the most common color spaces in electronic devices is RGB (Capitán-Vallvey et al., 2015). In it, color is decomposed into red, green and blue components. Each one of these coordinates can take a value between 0 and 255 (even though they are usually normalized from 0 to 1), and hence form a vector (R, G, B). If all of them take the value of 0, the color represented is pure black; if all of them are 255, the color is white. From the RGB coordinates, one can obtain the grayscale value. It is calculated as the average value of (R, G, B), and is converting the color signal to a graduation in black and white: if RGB has not been normalized (thus, each channel can take values between 0 and 255), grayscale will vary in that same range; if RGB has been normalized to be comprised in the (0, 1) range, so will grayscale. Equation 3 shows the formula to obtain grayscale value from RGB: 𝑅 + 𝐺 + 𝐵 𝑔𝑟𝑎𝑦𝑠𝑐𝑎𝑙𝑒 = 𝑎𝑣𝑒𝑟𝑎𝑔𝑒([𝑅, 𝐺, 𝐵]) = [3] 3 - 28 - There exist other color spaces of common use, and they can be interconverted if a reference white is used. CIE XYZ is one of the most noteworthy ones, and was developed back in 1931 by the International Commission on Illumination (CIE, from its name in French). In it, two of the three coordinates represent chroma, understood as the color itself (X and Z), and Y represents luminance. These three coordinates can be obtained by linear combinations of RGB (Hunt & Pointer, 2011; Mohamed & Shalaby, 2019). This color space provides numeric values which, unlike the case of RGB, do not depend on the device of capture, and it receives the name of uniform color space. This means that, at a given luminance, the difference between two different colors is the same on both devices (Mohamed & Shalaby, 2019). Additionally, CIE XYZ is often used as an intermediate step in the transformation from RGB to the other color spaces (Capitán-Vallvey et al., 2015). Among many others, CIE Lab is also worth mentioning. In this case, lightness is represented by L*, which ranges from 0 to 100 (black to white), and color is defined by a* and b*: the change from green to red, and from blue to yellow, respectively. Both can take values between -120 and 120. The obtention of these three parameters requires the previous obtention of CIE XYZ (Mohamed & Shalaby, 2019). A factor to be considered when applying image treatment in Analytical Chemistry is the lighting that is applied to the sample. Since the observed color will be resulting from an interaction of light with the sample, it must be carefully tuned in order to obtain valuable results. In this sense, different options arise: ambient light, flash from the device, or LED illumination can be chosen as light sources depending on the availability and suitability of each case. Furthermore, the relative position between the light source, the sample and the capture device are also capital parameters, since the result will greatly vary if non-reproducible conditions are used. Active learning Active learning can be defined as an approach that aims to get the student involved in the learning process through high order thinking tasks (Armellini et al., 2021). However, getting the student to feel involved in the practice can become a hard task if difficult and sophisticated procedures are meant to be carried out. Hence, in the chapter we propose the application of the student’s smartphones to the lab protocol, so that they feel an active part of it. Method In this section, the laboratory protocol to analyze each one of the proposed samples is described. Students must be warned that this practice involves the use of hazardous chemicals, like concentrated sulfuric acid and metals solutions, like antimony and molybdenum. Thus, the adequate safety measures must be taken (lab coat, gloves and eye protection), and the residues generated during the practice disposed in the proper way following the institution’s regulations. Reactive mixture preparation A reactive mixture must be prepared prior to the analysis of the samples. This mixture is common for any type of sample. Content: - 29 - - 2.5mL of an antimony tartrate solution 0.27% (w/v): prepared weighting 0.135g of C4H4KO7Sb·0.5H2O and diluting in 50mL of ultrapure water. - 25mL of sulfuric acid solution 2.62M: prepared by dilution of 7mL of concentrated sulfuric acid up to 50mL of total volume, with ultrapure water. - 7.5mL of Mo salt solution (4.12% (w/v), using ((NH4)6Mo7O24·4H2O): 2g of the salt are dissolved in 50mL of ultrapure water. - 15mL of ascorbic acid solution 1.79% (w/v), freshly prepared: to it, 0.88g of ascorbic acid need to be dissolved in 50mL of ultrapure water. It is important that the reagents are added in that specific order to avoid undesired side reactions. The stability of the prepared reactive mixture is 4h. Sample preparation Each type of samples needs to be prepared differently, depending on the expected concentration levels and the idiosyncrasy of the matrix. Therefore, each subsection is described independently. Different adaptations are suggested depending on the context of each case: all of the students can analyze one sample, or different ones can be selected and the results compared. Blood In this case, human blood was used. To obtain it, certified professionals were contacted to extract the blood from two different volunteers. Given the difficulty that this implies, alternatives like rabbit blood can be considered. The whole blood samples were centrifuged at 1500 g during 10 minutes to separate the serum from the cellular part of the blood. 20µL of the serum (which deposits on the top part of the tube) were combined with 320µL of the reactive mixture in an Eppendorf flask. A final volume of 2mL was obtained by adding ultrapure water. Water Rainwater and irrigation water samples were collected and filtered by a 0.22µm nylon filter, and stored at 4oC until analysis. An appropriate dilution (1, 5 or 7mL in a total volume of 10mL) was made to obtain a sample within the linear range. Before bringing to volume the volumetric flasks, 1.6mL of reactive mixture were added. Additionally, recovery studies can be carried out with the students to assess the accuracy parameter of the method. To it, we recommend analyzing a sample which is phosphate free (thus, they might need to analyze it need in advance and check if it has a concentration above the LOD of the method) and adulterating it with added phosphorus. For instance, a 1mg L-1 of phosphorus sample can be prepared using tap water. From it, a ½ dilution can be carried out to obtain a measuring solution of 0.5mg L-1. Washing powder This kind of sample, which is often in solid state, often contains phosphate in the percentage level, and so it needs a high level of dilution to be analyzed in the mg L-1 range. To it, we propose a x25000 times dilution which enables to measure samples within the 0.30 – 13.2% - 30 - (w/w) expressed as Na3PO4, comprising from the limit of quantification of the method up to the upper limit of the linear range. For instance, 0.1g of solid sample can be dissolved in 25mL of ultrapure water, and from it, 100µL transferred to a 10mL volumetric flask in which 1.6mL of reactive mixture are added. As stated above, there is also the possibility to analyze different samples which are originally phosphate free, by creating a spiked sample with the students. Solid Na3PO4 or similar can be used as a source of solid phosphorus, and mixed previously with the sample and homogenized in order to obtain reproducible results. Alternatively, students can also take part in this process by spiking the sample and learning how to obtain a spiked mixture of a solid sample to assess accuracy. This, in combination with the water analysis, would allow students to learn how to obtain both liquid and solid spiked samples, a basic skill in Analytical Chemistry. Eyedrops All of the samples that were selected declared to contain phosphate in their composition, either as an additive or as part of the active compound. Due to it, different dilutions needed to be done in order to have a measuring solution within the linear range. So, an appropriate amount of each liquid eyedrop was diluted up to 10mL with ultrapure water, containing 1.6mL of reaction mixture. For any given sample, after having added the reactive mixture, an intense blue color started to appear. A reaction time of 10-20 minutes was allowed to pass before measuring the resulting solutions. External calibration An external calibrate was prepared using a 1000mg L-1 stock solution of phosphorus (prepared from 0.44 g of KH2PO4 dissolved in 100mL of ultrapure water). From the stock solution, a working solution of 50mg L-1 of P was prepared by dilution. The calibration curve was built with the volumes shown in Table 2.1. This same working solution can be used to prepare the spiked water samples. UV-Vis analysis Measurements were carried out in a HP 8452A diode array. Samples and standards were measured at 820nm to obtain the absorbance values. However, due to the wide absorbance band of the colored complex, this maximum value might be adjusted in each case, depending on the range of the instrument. Instrumental blank was made with ultrapure water, and a reagent blank (prepared as a standard with no added phosphorus, P0 at Table 2.1.) was measured to check for any possible phosphate contamination of the reagents. Each sample/standard was measured in individual triplicates and the average value was used. Furthermore, a whole spectrum in the visible range is needed. Table 2.1. Calibration curve preparation. V working solution | Standard [P] | mg L-1 µL P0 0 0 - 31 - P1 0.2 40 P2 0.4 80 P3 0.6 120 P4 0.8 160 P5 1.0 200 P6 2.0 400 P7 3.0 600 P8 4.0 800 P9 5.0 1000 Image capture setup A Samsung Galaxy Edge S7 model SM-G93F was used to capture the images in the optimization step. This device has a 12.2MP camera sensor, and the native camera app was used in the ‘pro’ mode. More specifically, the image acquisition parameters were: ISO 50, white balance 5700K and aperture 1/1000. The smartphone was placed on a methacrylate structure made in the lab, with the main camera pointing to the 96 microwell plate containing the sample solutions. As a light source, a desktop lamp was used with the light bulb pointing up. Above the light source, a diffusive material was placed to ensure homogenic lighting of the sample. The plate was placed on top of the lamp using the diffusive material as a base, so that light could go through it and reach the smartphone. That microwell plate consisted in a 96 positions plate, with a maximum volume of 350µL, and a transparent base. The walls were made out of black material to avoid interferences of the light source . Figure 2.1. shows a scheme of the setup. Zoom was made to cover 5 x 4 wells of the microplate. The obtained image, in .jpg format, was transferred to a computer to obtain the different image parameters. Figure 2.1 shows the proposed setup. In this case, Colorlab tool (Malo & Luque, 2002) for Matlab® was used. However, different alternatives cost free can be used, being ImageJ, a free-to-use tool developed by the National Institute of Health (NIH) a recommendable option (Schneider et al., 2012). To obtain the color parameters, it is important that the selected region of the photograph, which is commonly known as Region Of Interest (ROI), contains a sufficient part of the solution to capture a representative color of the sample. Additionally, it must be avoided to include any part of the microplate within the ROI. - 32 - Figure 2.1. Proposed setup to analyze phosphate by image colorimetry. Results In this section of the chapter, the different results that are expected to be obtained by the students are addressed. Firstly, a qualitative study is proposed for them to get a grasp on the different concepts regarding image analysis and colour spaces, connecting those with chemical information. Second, the results of analysed samples are shown, both for the colorimetric and the reference method (UV-Vis spectroscopy). All in all, each instructor might select different parts of the proposed studies, or decide to take on the whole experience, depending on the specificities of each laboratory module. Qualitative study: RGB color space interpretation After all the images have been taken, colour parameters can be extracted and analysed. Additionally, some parameters of interest can be also studied, like the influence of well volume. As an example, the RGB parameters for different well volumes are plotted in Figure 2.2. Figure 2.2. RGB parameters trend for different well volumes in a 96 microwell plate. The results indicate that the maximum sensitivity is obtained using the red (R) channel and 300µL of well volume. In this sense, students are encouraged to make a reasoned interpretation of the results. Some possible questions to state the problem could be: - For a given well volume (300µL), how is it that sensitivity is decreasing in the specified order: R > G > B? - For a given color parameter (for instance, R), why is it having more and more sensitivity as volume increases? First question is expected to be answered based on the absorption spectra of the complex (which is measured during the practice, in the UV-Vis instrument) and the visible colour of the - 33 - complex (blue). In RGB colour space, as the sample gets darker and darker (in this case, as the concentration increases, the samples get a more intense blue colour), the average of the three components will tend to 0. This concept is can be easily explained in grayscale terms: as the sample gets more concentrated, its aspect becomes darker, and the grayscale ( Figure 2.3. ) is closer to 0. See Equation 3 in the Introduction section. Figure 2.3. Grayscale values for 200µL as volume sample in the 96 microwell plate. Thus, this is why we find that all three go down. However, it might be counterintuitive that R is the parameter showing more sensitivity, since the observed colour is blue. Hence, this is a great chance for them to correlate RGB colour space with absorption spectra: visible spectrum shows that the wide absorption band of the complex becomes more significant from 600 to 800nm. As can be seen in Figure 2.4. , as the absorption band gets more intense (due to higher concentration), the effect is taking place around 700-800nm, and hence it is being represented by R. Figure 2.4. RGB parameters covering the visible part of the electromagnetic spectrum. Regarding the second stated question, students are thought to correlate this concept with Lambert-Beer’s law. Since the setup is lighting the samples from the bottom part, and the detector is placed on top, the system is behaving very similarly to a UV-Vis instrument. So, the thicker the layer of solution, the higher the interaction with the analyte and the higher the sensitivity. - 34 - To this point, only RGB has been assessed since it is the easiest to obtain with common software. As this chapter focuses on the introduction of undergraduates to the application of smartphones in Analytical Chemistry, we find that it is a good starting point. However, if interested, this same procedure can be done with CIE Lab colour space, studying and extracting reasoned conclusions from the data. Quantitative study of samples Once the samples have been analysed both with the smartphone and with the ultraviolet-visible spectroscopy, a comparison between them can be made. Students are expected to plot their results in a figure like Figure 2.5. or similar. Figure 2.5. Results obtained for the reference method (UV-Vis spectroscopy) and the colorimetric analyzer (smartphone). When a expected value of concentration was known, it has been added. Blood samples are expressed in mg dL-1; Washing powders in w/w percentage of Na3PO4; Eyedrops and water in mg L-1 of phosphorus. As can be observed, results obtained with the proposed setup are comparable to those obtained with the reference method, validating the procedure. Additionally, when a sample had a known expected value, it was accordant to the experimental data. It is interesting to compare the analytical performances of both techniques. For instance, in terms of linearity. Figure 2.6. plots a comparison of both instruments. It can be observed that, while UV-Vis only keeps linearity until 1mg L-1 of phosphorus, the smartphone setup allows to keep a wider linear range. - 35 - Figure 2.6. Calibration graphs for UV-Vis spectroscopy and Smartphone device in the (0, 5)mg L-1 of phosphorus. The linear range is shown in black. Green analytical chemistry parameters discussion As a final step, Green Analytical Chemistry is assessed by the students. To it, we propose to follow the 12 different parameters of a method to be considered as green, described by Gałuszka et al. (Gałuszka et al., 2013). Namely, they can be summarized as: - Prioritize direct methods - Integrate different analytical processes and operations - Reduce the waste generation, and treat it conveniently - Reduce the energy waste - Automatize and miniaturize methods - Prioritize reagents which are obtained from renewable sources - Increase safety for operators - In-situ analysis are preferred - Avoid derivatization steps - Size and number of samples should be reduced - Multi-analyte or multi-parameter methods are preferable - Eliminate or reduce toxic reagents Hence, they should identify which are the parameters that this method has, and justify why. In this case: - Reduce the waste generation, and treat it conveniently: when compared to the reference method described in the APHA (Rice, 2015), a reduction of 160 times is obtained. - Reduce the energy waste: in this case, the energy consumption of the process is much lower, since no instrument is needed other than the smartphone. - 36 - - Automatize and miniaturize methods: this parameter is explained the same way was the first one discussed. - Increase safety for operators: since this method requires lower reagent volumes, the danger associated to their handling is reduced. Additionally, different variations to the setup can be made, as it can be a good source of discussion with the students. For instance, if the parameter ‘Multi-analyte or multi-parameter methods are preferable’ wanted to be accomplished, what adaptations should be done? Different colorimetric reactions can be carried out in the same setup and analysed in the same photograph, saving time and resources. Conclusions The implementation of smartphones in Analytical Chemistry has been a growing trend during the last few years. Thanks to their wide availability, lower cost and improved camera sensors, they have become a useful tool in the laboratory of analytical chemists. Hence, it is a topic which needs to be addressed in the chemistry undergraduates curricula in order to prepare future chemists in the communications era. In this chapter, a procedure to easily implement smartphones in the practice laboratory has been developed and applied to four different samples: washing powders, water, eyedrops and blood. The results proved to be comparable to those obtained with the UV-Vis reference method. With it, students are able to use their own devices in the lab, promoting their involvement with the practice as stated in the principles of active learning. Additionally, a part of the procedure has been designed to allow the students to identify the different principles of Green Analytical Chemistry. References Aguirre, M. Á., Long, K. D., Canals, A., and Cunningham, B. T. (2019). Point-of-use detection of ascorbic acid using a spectrometric smartphone-based system. Food Chemistry, 272, 141–147. Armellini, A., Cecilia, B., and Rodriguez, P. (2021). Active Blended Learning: Definition, Literature Review, and a Framework for Implementation. In Cases on Active Blended Learning in Higher Education, 1–22. Böck, F. C., Helfer, G. A., da Costa, A. B., Dessuy, M. B., and Ferrão, M. F. (2018). Rapid Determination of Ethanol in Sugarcane Spirit Using Partial Least Squares Regression Embedded in Smartphone. Food Analytical Methods, 11(7), 1951–1957. Capitán-Vallvey, L. F., López-Ruiz, N., Martínez-Olmos, A., Erenas, M. M., and Palma, A. J. (2015). Recent developments in computer vision-based analytical chemistry: A tutorial review. In Analytica Chimica Acta, 899, 23–56. Cinti, S., Talarico, D., Palleschi, G., Moscone, D., and Arduini, F. (2016). Novel reagentless paper-based screen-printed electrochemical sensor to detect phosphate. Analytica Chimica Acta, 919, 78–84. Curbani, L., Gelinski, J. M. L. N., and Borges, E. M. (2020). Determination of Ethanol in Beers Using a Flatbed Scanner and Automated Digital Image Analysis. Food Analytical Methods, 13(1), 249–259. De Boer, M. A., Wolzak, L., Slootweg, J. C., De Boer, M. A., Wolzak, L., and Slootweg, J. C. (2019). Phosphorus: Reserves, Production, and Applications. Phosphorus Recovery and Recycling, 75–100. - 37 - Rice, E.W., Baird R., Eaton, A.D. (2015). Standard Methods for the examination of Water and Wasterwater. American Public Health Association, American Water Works Association, Water Environment Federation. Fritz, J. S. (1987). Ion Chromatography. Analytical Chemistry, 59(4), 335A–344A. Gałuszka, A., Migaszewski, Z., and Namieśnik, J. (2013). The 12 principles of green analytical chemistry and the SIGNIFICANCE mnemonic of green analytical practices. TrAC Trends in Analytical Chemistry, 50, 78–84. Gee, C. T., Kehoe, E., Pomerantz, W. C. K., and Penn, R. L. (2017). Quantifying Protein Concentrations Using Smartphone Colorimetry: A New Method for an Established Test. Journal of Chemical Education, 94(7), 941–945. Hunt, R. W. G., and Pointer, M. R. (2011). Measuting Colour: Fourth Edition (4th edition). John Wiley and Sons. Jaikang, P., Paengnakorn, P., and Grudpan, K. (2020). Simple colorimetric ammonium assay employing well microplate with gas pervaporation and diffusion for natural indicator immobilized paper sensor via smartphone detection. Microchemical Journal, 152. Kallner, A. (1975). Determination of phosphate in serum and urine by a single step malachite-green method. Clinica Chimica Acta, 59(1), 35–39. Kirkbright, G. F., Narayanaswamy, R., and West, T. S. (1972). The spectrofluorimetric determination of orthophosphate as quinine molybdophosphate. The Analyst, 97(1152), 174–181. Kuehni, R. G. (2001). Color space and its divisions. Color Research and Application, 26(3), 209–222. León-Roque, N., Abderrahim, M., Nuñez-Alejos, L., Arribas, S. M., and Condezo-Hoyos, L. (2016). Prediction of fermentation index of cocoa beans (Theobroma cacao L.) based on color measurement and artificial neural networks. Talanta, 161, 31–39. Lopez-Ruiz, N., Curto, V. F., Erenas, M. M., Benito-Lopez, F., Diamond, D., Palma, A. J., and Capitan-Vallvey, L. F. (2014). Smartphone-based simultaneous pH and nitrite colorimetric determination for paper microfluidic devices. Analytical Chemistry, 86(19), 9554–9562. Malo, J., and Luque, M. J. (2002). COLORLAB: Color Science in Matlab. Marinho, O. R., Lima, M. J. A., Rocha, F. R. P., Reis, B. F., & Kamogawa, M. Y. (2019). A greener, fast, and cost-effective smartphone-based digital image procedure for quantification of ethanol in distilled beverages. Microchemical Journal, 147, 437–443. Mohamed, A. A., and Shalaby, A. A. (2019). Digital imaging devices as sensors for iron determination. Food Chemistry, 274, 360–367. Peng, B., Zhou, J., Xu, J., Fan, M., Ma, Y., Zhou, M., Li, T., and Zhao, S. (2019). A smartphone-based colorimetry after dispersive liquid–liquid microextraction for rapid quantification of calcium in water and food samples. Microchemical Journal, 149, 104072. Phosphorus - Health Professional Fact Sheet. (2021). National Institutes of Health. https://ods.od.nih.gov/factsheets/Phosphorus-HealthProfessional/#h2 Rezazadeh, M., Seidi, S., Lid, M., Pedersen-Bjergaard, S., and Yamini, Y. (2019). The modern role of smartphones in analytical chemistry. In TrAC - Trends in Analytical Chemistry, 118, 548–555. Elsevier B.V. Sargazi, M., and Kaykhaii, M. (2020). Application of a smartphone based spectrophotometer for rapid in-field determination of nitrite and chlorine in environmental water samples. - 38 - Spectrochimica Acta - Part A: Molecular and Biomolecular Spectroscopy, 227, 117672. Schneider, C. A., Rasband, W. S., and Eliceiri, K. W. (2012). NIH Image to ImageJ: 25 years of image analysis. Nature Methods , 9(7), 671–675. Seidi, S., Rezazadeh, M., Yamini, Y., Zamani, N., and Esmaili, S. (2014). Low voltage electrically stimulated lab-on-a-chip device followed by red-green-blue analysis: a simple and efficient design for complicated matrices. Analyst, 139(21), 5531–5537. Sharpley, A. N., and Menzel, R. G. (1987). The impact of soil and fertilizer phosphorus on the environment. Advances in Agronomy, 41, 297–324. Smith, V. H., and Schindler, D. W. (2009). Eutrophication science: where do we go from here? Trends in Ecology and Evolution, 24(4), 201–207. Tabatabai, M. A., and Dick, W. A. (1983). Simultaneous Determination of Nitrate, Chloride, Sulfate, and Phosphate in Natural Waters by Ion Chromatography. Journal of Environmental Quality, 12(2), 209–213. Wu, D., and Sun, D. W. (2013). Colour measurements by computer vision for food quality control - A review. In Trends Food Science and Technology, 29(1), 5–20. Wyszecki, G., and Stiles, W. S. (2000). Color Science. Concepts and Methods, Quantitative Data and Formulae (2nd edition). Wiley-Interscience Publications. https://www.wiley.com/en- es/Color+Science:+Concepts+and+Methods,+Quantitative+Data+and+Formulae,+2nd+ Edition-p-9780471399186 - 39 - Chapter 3 Doi: 10.26529/9789612532970/ch3 Students’ Identification and Application of Models to Rationalize Organic Acid-Base Trends Sean Gao, Taylor C. Outlaw, Jason G. Liang-Lin, Alina Feng, Jennifer L. Roizen, Colton Melnick, and Charles T. Cox Jr*. Duke University, Department of Chemistry, 124 Science Drive, Durham, NC 27708, *Corresponding Author: charlie.cox@duke.edu Abstract Acid-base chemistry is an essential component of the undergraduate chemistry curriculum. Acid-base concepts are introduced in general chemistry and expanded on in organic chemistry, biochemistry, and other advanced chemistry courses. Through a mixed approach of surveys and think-aloud interviews, the proficiency of second-semester organic students in acid-base chemistry was measured. Students were given two questions, both requiring them to rank the acidity of three compounds and justify the ranking. The first question focused on substituted carboxylic acids, and the second question focused on substituted aromatic structures. Although most students were able to correctly rank both sets of molecules, the correctness of their justifications was structure-dependent. Students did better at justifying the acidity of the aromatic structures, but while they were more successful at ranking the acidity of the substituted carboxylic acid structures, students were largely unsuccessful at justifying the trend. Students often relied on memorization, attributing acidity to the presence of specific functional groups or substituents. Specific alternative conceptions observed in both surveys and interviews include the idea that resonance structures are always central in justifying properties for molecules that have π bonds, and that alkenes and alkynes have differing numbers of resonance structures given they have different bond orders. Finally, students had difficulty with identifying the most acidic proton and often selected sites based on content that could be memorized from lectures. Students were also asked to report their confidence in their answers on a 6-point Likert scale from 0–5, and statistically significant differences were observed between students who ranked compounds correctly versus incorrectly for both questions in the study. However, when comparing the correctness of the justifications, a statistical difference between reported confidence was only observed with the aromatic structures question. The substituted carboxylic acids question required the application of models and ideas that extended beyond memorization. Keywords: Acid-Base Chemistry, Aromatic Molecules, Carboxylic Acids, Hybridization, Resonance, Inductive Effects, Alternate Conceptions, Confidence. Introduction As a central chemistry concept, acid-base chemistry is used to explain the structure and reactivity of organic, inorganic, and biological molecules, and is used to develop frameworks for qualitative and quantitative analyses. General or introductory chemistry courses introduce students to Brønsted-Lowry acid-base models and quantitative calculations involving pH, titrations, and buffers (ACS, 2015). There is an emphasis on the relationships between the magnitude of the K a, acid strength, and conjugate base strength. On the other hand, Lewis acid-base theory and rationalizing acid-base trends using molecular structure are not discussed to the same extent in most introductory chemistry courses. However, these models are central to understanding organic chemistry concepts and are either revisited or introduced at the start of the organic sequence. Lewis acid-base theory and structural models provide insight not only into organic acidity and basicity, but also readily extend to explanations of related concepts like electrophilicity and nucleophilicity, and can even explain chemo- and regioselectivity in - 40 - organic reactions. More than 85% of organic and biochemical reactions can be rationalized using acid-base concepts (Rossi, 2013; Stoyanovich, Gandhi, & Flynn, 2015). In the past two decades, chemical education research has predominantly focused on students’ understanding of acid-base chemistry and potential pedagogical reforms in introductory chemistry (Cooper, Kouyoumdjian, & Underwood, 2016; Cox, Poehlmann, Ortega, & Lopez, 2018; Mercier, 2018). More recently though, research has been extended to assess students’ understanding of acid-base chemistry in organic chemistry to identify alternative conceptions that persist from general chemistry or emerge as more sophisticated acid-base models are introduced (Bretz & McClary, 2014; Cartrette & Mayo, 2011; Duis, 2011; L. M. McClary & Bretz, 2012; L. T. McClary, V., 2011 ; Petterson et al., 2020; Schmidt-McCormack et al., 2019). The goal of this study is to assess students’ use of models to rationalize acidity at the end of the second semester of organic chemistry (OC2). More broadly, the long-term goal of our studies is to identify and develop pedagogical reforms, particularly to address alternative conceptions early to provide students with stronger conceptual frameworks for future courses. One study (Bhattacharyya, 2006) noted the progression of alternative conceptions from undergraduate cohorts to chemistry graduate students, which illustrates the importance of addressing these issues. Cartrette and Mayo (Cartrette & Mayo, 2011) outlined the relationship between students’ understanding of the Brønsted-Lowry and Lewis acid-base models. Their findings supported that students relied on declarative knowledge to rationalize answers but were unable to extend this knowledge to solve complex problems. Much of organic chemistry requires using models to predict trends, which accounts for the challenges students generally note with organic chemistry. Research supports (Anderson & Bodner, 2008; Grove & Lowery Bretz, 2012) the idea that the transition from quantitative to qualitative thinking from general to organic chemistry requires students to have a deeper understanding of fundamental concepts, which explains why organic chemistry is generally regarded as a gatekeeper course. As noted (Bhattacharyya & Bodner, 2005; Grove & Lowery Bretz, 2012), with organic chemistry, rote memory does not reflect understanding and does not ensure success or mastery of the concepts. The challenges with using rote memory, particularly with acid-base concepts in the context of organic chemistry, have been noted in several studies. For example, students tend to associate acidity with specific functional groups (Bretz & McClary, 2014), rely on heuristics that seemingly support conclusions based on associations instead of more sophisticated explanations (L. McClary & Talanquer, 2011), depend on cues as a way to identify the appropriate strategy due to a lack of conceptual grouping (Petterson et al., 2020), and struggle to relate Brønsted-Lowry and Lewis acid-base models (Schmidt-McCormack et al., 2019). The latter observations support the reliance on rote memory to navigate problems and illustrate challenges students face in applying concepts and models. Longitudinal studies have illustrated that students’ use of models in acid-base chemistry becomes more sophisticated over time (Crandell, Kouyoumdjian, Underwood, & Cooper, 2019). However, related studies measuring reported confidence indicate that students continue to overestimate their understanding of the acid-base concepts (L. M. McClary & Bretz, 2012). Our study also incorporates a confidence ranking question to assess students’ awareness of their abilities in ranking and explaining acid-base trends. Given students’ prior experience in at least two other chemistry courses, the question arises whether this experience may impact students’ self-awareness of their conceptual understanding. One of the key models used to explain acid-base trends is resonance. Recent research regarding resonance identified alternative conceptions regarding students’ understanding of resonance, which illustrates further challenges in using resonance to explain acid-base trends. Students focus on drawing resonance structures, and while they may successfully draw structures, they struggle with interpreting their meaning and applying embedded information to explain structure and reactivity (Duis, 2011; Kim, Wright, & Miller, 2019; Xue & Stains, 2020). This - 41 - study expands on the existing research by assessing students’ abilities to identify when resonance is pertinent in explaining acid-base trends. To account for students relying on heuristics to accurately rank compounds’ acidities, our study asked participants to provide a 2-3 sentence explanation to accompany their ranking. The results of this study establish which alternative conceptions students hold regarding how hybridization and resonance acidity affect acid-base chemistry, quantify the prevalence of these alternative conceptions, and inform instructional interventions to improve students’ awareness of the scope and limitations of various acid-base models. This study was guided by the three research questions below: Research Questions 1. How do second-semester organic chemistry (OC2) students apply the concepts of hybridization, inductive effects, resonance, and conjugation to rationalize acid strength? 2. What alternative conceptions do students have regarding these concepts, and which are most deeply rooted? 3. How do students gauge their own understanding of organic chemistry concepts when determining acid strength? Methods Student Participants and Data Collection These studies were performed at a large private southeastern research-intensive university. All students were informed of their rights as human research subjects, and students who did not consent to their responses being used for this study were omitted from data analysis. Only students residing in the United States and over 18 years of age were included in the study. All data were handled per the Institutional Review Board (Protocol #2021-0182); when identifiers were collected, the data were anonymized and not shown to professors until after final grades were posted. These studies followed two cohorts of OC2 students, from different instructional terms, taking the course with different professors. Due to Covid-19 restrictions, the two cohorts of students experienced the course through different instruction media. Cohort A (N = 65) from the 2020 Fall semester experienced the course completely remotely, and the survey was distributed as a for-credit assignment. Cohort B (N = 30) during the 2021 Summer semester was a hybrid course, with some students attending in-person whereas others were completely remote; the survey was distributed as an encouraged review assignment for the final exam. For both cohorts, the surveys were distributed at the same point in time during the academic term, after the class covered enolate chemistry. Aside from COVID-19 restrictions preventing control of the instruction medium, the study also did not control for students’ past experience with chemistry (Advanced Placement, International Baccalaureate, introductory college-level general chemistry). Of note is the unique introductory chemistry curriculum at the institution studied; students typically take one semester of general chemistry, then two semesters of organic chemistry, before taking the second semester of general chemistry. The chemistry courses are not separated by students’ intended major or pre-professional track. Lastly, not every section achieved high response rates (Cohort A 98%, Cohort B 50%). Along with surveys, data were also collected using think-aloud interviews (Cartrette & Mayo, 2011; Petterson et al., 2020). The questions discussed in this work were a 15-minute component of one-hour-long interviews that were conducted over Zoom, and the problems were presented by the interviewer via the screen sharing function. The interviewer filled out the student - 42 - participants’ answers to the ranking questions, then referred to the recording and transcript for their explanations. The interviews were recorded, where OC2 student participants kept their video off and microphone on; all files were password-protected and transcribed to eliminate identifiers and protect anonymity. Following the interviews, the main researcher noted general observations for each, serving as a secondary data source along with the recording and transcript. Further analysis of data is described under Qualitative Analysis. Development of Research Instruments The question involving carboxylic acids with differentially hybridized carbon chains (Q1) was designed because no existing research has investigated how organic chemistry students rationalize a compound’s acidity when both resonance effects and hybridization/inductive effects are simultaneously present. Including both concepts provides an effective way to probe whether students are merely memorizing trends, trying to apply a single model to several different situations, whether they can successfully parse through distractors, etc. On the other hand, the question containing substituted phenols (Q2) was adapted from ACID I, designed by McClary and Bretz (Bretz & McClary, 2014; McClary & Bretz, 2012) We maintained two out of the three molecules as used in ACID I ( para-methylphenol, para- nitrophenol), but modified the third molecule to meta-nitrophenol to specifically probe students’ understanding of resonance versus inductive stabilization of negative charge. Q2 was used primarily as a calibration tool because student performance on ACID I has already been thoroughly analyzed (McClary & Bretz, 2012). Each overarching question in the surveys and interviews included several sub-questions ( Figure 3.1. ): (a) a ranking of the three molecules from the most to least acidic, (b) an open-ended explanation question (no text limit, but instructions suggested that students write 2-3 sentences), (c) a confidence ranking question on a 6-point Likert scale from 0–5, adapted from Caleon and Subramaniam (Caleon & Subramaniam, 2010). The interview only investigated Q1, since Q2 has already been thoroughly investigated (McClary & Bretz, 2012). The interview question was identical to Q1, only with more opportunities to probe the students’ thought processes. Trial interviews were conducted with three chemistry majors, otherwise uninvolved with the study, who provided feedback regarding additional probing questions to ask, how to frame questions more effectively, and how to proceed with interview logistics. - 43 - Research Instruments Q1: Hybridization Acidity Q2: Aromatic Acidity a) Rank the above molecules in order of acidity, with 1 being most acidic and 3 being least acidic. Molecule A Molecule B Molecule C b) Please justify your ranking with relevant chemical reasoning. A 2-3 sentence explanation will suffice. c) On a scale from 0-5, how confident are you in your answers to questions a) and b)? 0 = not confident at all, 5 = completely confident Figure 3.1. Sets of structures used to probe students’ understanding of hybridization, inductive effects, resonance, conjugation, and their relation to acid strength. Q1 was asked in both the surveys and interviews. Q2 was asked in only the surveys. Data analysis Qualitative analysis Four researchers discussed then came to a consensus on the scoring criteria ( Table 3.1.) to use for each survey/interview question. Together, the same researchers scored each student’s explanations as correct, partially correct, or incorrect, following the scoring criteria. Furthermore, the explanations were coded both deductively and inductively; deductive codes were taken from the conclusions of the ACID I studies, whereas the inductive codes came directly from the students’ survey responses. Three researchers independently reviewed the transcripts and audio/visual data from the interviews. All three researchers noted observations of (alternative) conceptions and performed an initial coding of responses as correct, partially correct, or incorrect. The research team then reconvened to discuss their findings to reach a consensus on the final coding results. These are presented in Sankey diagram format for the 10 OC2 students interviewed. - 44 - Table 3.1. Scoring Criteria for Rankings and Explanations Question Accuracy Scoring Criteria Q1 Ranking Correct C > B > A in acidity C, prop-2-ynoic acid, p K a = 1.89 https://pubchem.ncbi.nlm.nih.gov/compound/Propiolic-acid B, prop-2-enoic acid, p K a = 4.25 https://pubchem.ncbi.nlm.nih.gov/compound/Acrylic-acid A, propanoic acid, p K a = 4.88 https://pubchem.ncbi.nlm.nih.gov/compound/Propionic-acid Incorrect Not C > B > A ranking Q1 Explanation Correct Carboxylic acid proton is deprotonated AND Increasing s-character of the carbon chains makes them more electronegative and inductively withdrawing, better stabilizing the conjugate base; trend needs to be applied across all three molecules Partial Increased s-character/electronegativity/inductive effect of the carbon chain makes molecules more acidic, but only applies this to alkynes Incorrect Wrong proton deprotonated AND/OR Only used resonance and/or conjugation to justify ranking AND/OR Incorrect chemistry (e.g., alkynes have more s-character, alkynes are electron-donating, etc.) Q2 Ranking Correct B > C > A in acidity The reported p K a values are provided as a reference (Liptak, Gross, Seybold, Feldgus, & Shields, 2002): B, p-nitrophenol, p K a = 7.91 C, m-nitrophenol, p K a = 8.13 A, p-methylphenol, p K a = 10.26 Incorrect Not B > C > A ranking Q2 Explanation Correct Para-EWG (electron-withdrawing group; nitro) stabilizes the conjugate base through resonance and inductive effects; meta-EWG stabilizes the conjugate base through inductive effects; para-EDG (electron-donating group; methyl) destabilizes the conjugate base through inductive effects OR Accurately compares p-EWG and m-EWG (resonance + induction versus induction), AND explains why EWGs stabilize and EDGs destabilize the conjugate base - 45 - Table 3.1. continued Partial Accurately states EWGs decrease and EDGs increase electron density in the conjugate base and resulting effect on acidity, BUT does not compare p-EWG and m-EWG (resonance + induction versus induction) OR Accurately compares p-EWG and m-EWG (resonance + induction versus induction), BUT does not discuss effect of EDGs on electron density and conjugate base stability OR Discusses all three substituents, BUT does not explain that m-EWG cannot resonance stabilize Incorrect Wrong proton deprotonated AND/OR Incorrect chemistry (e.g., EWGs increase electron density, nitro group is an EDG, etc.) AND/OR Does not discuss anything listed in “Partial” Quantitative analysis Data from the surveys were first tested for normality. Visual inspection of a histogram of student confidence (Likert scale) versus a normal Gaussian curve and calculations of skewness and kurtosis suggested the data were normally distributed. However, the Shapiro-Wilk test suggested otherwise (p ≈ 10-6), thus nonparametric tests were used for all statistical comparisons to be most accurate. The survey data from the two OC2 sections were compared to investigate statistically significant differences in accuracy ( Table 3.2. ) and mean confidence. Based on two-tailed Fisher’s Exact tests for accuracy, and Mann-Whitney U tests for confidence ( Table 3.2. ), no statistically significant differences were found between the two OC2 sections, thus the survey results were aggregated. The aggregated confidence data were then compared in several ways: based on ranking accuracy, explanation accuracy, and both ranking and explanation accuracy. The one-way Kruskal-Wallis test was used to make multiple group comparisons, and the Mann-Whitney U test was used to make pairwise comparisons. A p-value of 0.05 or less was considered statistically significant for all tests performed. All visualizations and statistical calculations discussed above were performed in R. Table 3.2. Statistical Comparisons and p-Values Between Survey Cohorts Question Comparison p-Value (Fisher’s Exact or Mann-Whitney U Test) Q1 Ranking Accuracy 0.14 Explanation Accuracy 0.19 Confidence 0.69 Q2 Ranking Accuracy 1.00 Explanation Accuracy 0.52 Confidence 0.50 - 46 - Results and discussion The objective of the research study was to gauge students’ abilities in acid-base chemistry at the end of the second semester of organic chemistry. The chemical structures assessed in the study are shown below. Throughout the discussion, the substituted carboxylic acid structures in Q1 will be referred to as the “hybridization structures” because these structures differ with respect to the hybridization of the alkyl, alkenyl, and alkynyl substituents. The substituted aromatic structures in Q2 will be referred to as the “aromatic structures.” These compounds are especially pertinent because a central focus in OC2 includes a discussion of the structure and reactivity of carboxylic acids and aromatic molecules. The surveys consisted of two parts per structure: (a) ranking structures based on their acidity, (b) explaining their rationale for the proposed ranking, with the explanation providing insight into students’ reasoning to identify alternative conceptions and propose interventions. For both the ranking and explanation items, students reported confidence in their responses on a scale of 0–5, with 0 being the lowest and 5 being the highest confidence. The Dunning-Kruger effect suggests that students tend to overestimate their abilities, which has been observed in the context of organic acid-base chemistry (L. M. McClary & Bretz, 2012). Therefore, the confidence data was collected to gauge students’ awareness of their understanding after three semesters of chemistry (including general chemistry and first-semester organic chemistry). Data presented will focus on students’ confidence, ranking accuracy, and correctness of their explanations. Statistical comparisons between confidence, based on student performance, will be reported to measure students’ abilities to gauge their understanding, with the acknowledgment that higher confidence does not imply greater performance. Q1: Hybridization Acidity Figure 3. 2. compares students’ reported confidence with their performance in ranking the acidity of the hybridization structures. Students who correctly ranked the acidity reported statistically higher confidence (p = 0.0053) than those who did not, which is counter to the findings by McClary and Bretz (L. M. McClary & Bretz, 2012). The difference in awareness could be attributed to the organization of the questions or to the timeline of the survey’s distribution. Figure 3.2. summarizes students’ reported confidence by their performance in explaining the trends in acidity. The explanations were coded as correct, partially correct, or incorrect as outlined in the Methods section. - 47 - Figure 3.2. A comparison of students’ (N = 95) reported confidence based on whether they correctly or incorrectly ranked the acidity of the hybridization structures. Red diamond denotes the mean. Statistical differences were not observed for students’ reported confidence based on the correctness of their explanations. Of the 66 students who correctly ranked the acidity, only 6/66 ≈ 9% provided a correct explanation and 7/66 ≈ 11% provided a partially correct explanation. Given the sizable drop in performance between the two tasks, the responses were analyzed and coded to acquire a greater perspective on students’ rationale and to identify alternative conceptions. The qualitative codes and the frequency to which they were observed are outlined in Table 3.3. Figure 3.3. A comparison of students’ (N = 95) reported confidence and the correctness of their explanations regarding the acidity of the hybridization structures. Red diamond denotes the mean. The Kruskal-Wallis test (p = 0.91) indicates that no statistical difference was observed when comparing all three groups of students. Further analysis using the Mann-Whitney test suggests no statistical differences were observed between any pairs the three groups. - 48 - Table 3.3. Deductive and Inductive Codes for Student Explanations to Q1 – Hybridization Acidity Code Descriptions Percentage of students (N = 95) Deductive Codes Mentioning conjugate base stability 35 Equating functional groups with acidity 20 Inductive Codes Stabilizing the conjugate base makes molecules more acidic 29 Alkynes have more s-character, which makes them more acidic 28 Carbon-centered proton will be deprotonated 22 Alkynes are more acidic than alkenes, which are more acidic than 15 alkanes; no justification provided Increased s-character in the carbon chain means they are more 14 electronegative / inductively electron-withdrawing, which makes alkynes more acidic than alkenes, which are more acidic than alkanes Oxygen-centered proton will be deprotonated 13 Alkynes contribute more to resonance than alkenes and alkanes 12 Alkenes contribute more to resonance than alkynes and alkanes 10 Alkynes and triple bonds are electronegative/ have an inductive 7 electron-withdrawing effect; does not explicitly discuss s-character Alkynes do not contribute as well to resonance compared to alkenes 7 due to the allene/cumulene structure Alkenes have more conjugation, which makes them more acidic than 6 alkynes and alkanes Alkynes have more conjugation, which makes them more acidic than 5 alkenes and alkanes Resonance within the carboxylate makes all three molecules more 4 acidic Alkynes have shorter / stronger bonds than alkenes than alkanes, which 4 makes them more acidic More resonance between the C=C and C=O of alkenes and alkynes 2 makes molecules B and C more acidic The qualitative codes in Table 3.3. imply students were successful in ranking the acidities of the hybridization structures despite using flawed logic. The correct reasoning should have considered the inductive effect from the neighboring alkyl, alkenyl, and alkynyl groups, with the alkynyl group being more electronegative because of the increased s-character. More s-character implies a stronger nuclear charge because of the proximity of the s-orbital to the nucleus relative to the p-orbital. An increase in the electronegativity of a neighboring substituent increases acidity through inductive effects, in which neighboring groups withdraw, or pull away electron density through σ bonds. Therefore, the alkynyl substituent “pulls away” more electron density from the carboxylate group in the conjugate base, reducing the electron density on the base. As electron density on the site decreases, conjugate base strength decreases and stability increases, which results in an increase in acid strength. Only 14% of the 95 students surveyed correctly considered both the inductive effects and s-character (hybridization). The inductive effect is normally introduced in lectures using halogen- - 49 - substituted compounds to illustrate the pull of electron density by the electronegative halogen substituents. Figure 3.4. summarizes structures commonly outlined in the textbook and on typical assessments. These structures become the benchmarks students use to rationalize inductive effects. F F OH F OH Cl OH F Br OH O O O O Figure 3.4. A summary of molecules generally used to illustrate the inductive effect. Similar structures were used to illustrate the inductive effect in the standard text (Jones & Fleming, 2010). A substantial number of students, 46%, applied a resonance or conjugation argument. Resonance is an argument used to account for the differences in the acidity of acetic acid versus methanol, and students have seen this argument in their textbooks, in lectures, and on assessments. Additionally, there are resonance structures that can be proposed with the alkenyl and alkynyl substituents ( Figure 3.5. ), but the argument would not fully account for the differences in acidity. Neither resonance structure B3 nor C3 would be considered a pertinent resonance structure; they would both be exceptionally minor resonance contributors, as they unnecessarily incorporate an incomplete octet on carbon and multiple formal charges. Furthermore, a carbocation is less stable on an sp- compared to an sp2-hybridized carbon because of the increased electronegativity of the former, which further minimizes the contribution of resonance structure C3, even relative to B3. Finally, resonance structures B3 and C3 also do not illustrate a reduction in electron density at the carboxylate; in fact, the electron density at the carboxylate is greater with both oxygens having a negative formal charge, which would support an increase in basicity and a decrease in acidity. As noted above, the argument that the alkynyl substituent could support additional resonance representations relative to the alkenyl substituent is incorrect, which was stated in 12% of responses. Students are likely using bond order to support this rationale; however, although the alkyne has a triple bond, only one of the two π bonds would contribute to resonance. For resonance and conjugation purposes, the π-systems must be within the same plane. Yet with the alkynyl substituent, only one π bond would be coplanar, while the other would not contribute to the conjugation. Figure 3.5. Possible resonance structures for the alkenyl- and alkynyl-substituted carboxylic acids. - 50 - From the resonance arguments, OC2 students articulated two alternative conceptions as parts of their rationales. The first alternative conception is that resonance can always be used to justify acidity when double or triple bonds are present. Indeed, resonance does play a role as noted, but the resonance forms should be drawn and carefully analyzed to rationalize changes in electron density. Resonance structures constitute key representational models in organic chemistry, but other models and arguments remain pertinent. The second alternative conception is that double and triple bonds yield different numbers of resonance structures. Orbital overlap occurs within a plane (along one axis), and in the triple bond, the two π bonds are orthogonal. Regarding student understanding of resonance, Xue and Stains (Xue & Stains, 2020) found that students focused more on the Lewis structure representations and faced challenges when explaining the concepts of resonance and resonance hybrids. In our study, students identified the importance of resonance, but as with the Xue and Stains’ study, students did not accurately explain the connections between resonance structural representations and the property they were rationalizing. Similar ideas were reported (Brandfonbrener, Watts, & Shultz, 2021; Duis, 2011) in which students focused more on drawing resonance structures without understanding the embedded concepts. In lectures, the acidity of the C-H bond of alkanes, alkenes, and alkynes is compared using s-character, with the alkyne C-H bond being more acidic because of the increased s-character and subsequent increase in electronegativity. The acidity of the alkyne C-H bond was noted in 43% of responses, but there was no additional elaboration for why this is significant for this system. Additionally, 22% argued the C-H bond would be deprotonated instead of the more acidic O-H bond of the carboxylic acid. Therefore, a potential alternative conception can be proposed based on these arguments in which students are considering a less acidic proton. This could stem from either only using facts from lectures or not fully analyzing all potential acidic protons in the structure. Using the acidity of the C-H bonds, students could correctly rank the structure in terms of increasing acidity, but as with resonance, this works only by coincidence and does not demonstrate a complete understanding to rationalize the acid-base properties. The following responses, ordered correct, partially correct, and incorrect, provide representative explanations from the survey. These responses reiterate the points noted above regarding the incorrect use of resonance or the incomplete use of s-character to justify their ranking. Survey Response 1 [Correct] "sp hybridized carbons are more electronegative than sp2, which is more electronegative than sp3. Carboxylic acids lose the proton bound to the oxygen when in the presence of a base, creating a carboxylate anion. Due to the polar effect, the more electronegative sp hybridized carbon can stabilize the negative charge on the oxygen of the anion better than the sp2 carbon, which can stabilize the anion better than the sp3 carbon." Assessment: The student correctly identifies the loss of the oxygen-centered proton to form the carboxylate, understands the relationship between hybridization and inductive effects, and correctly predicts relative acidity. Survey Response 2 [Partial] "sp-hybridized carbon atoms (50% s character) are more electronegative than sp2-hybridized carbon atoms (33% s character), which are more electronegative than sp3-hybridized carbon atoms (25% s character). Because of this, the negative charge of the conjugative base is best stabilized by the alkyne group, then the alkene group, then the alkane group." - 51 - Assessment: The student uses a hybridization argument and accurately connects hybridization to electronegativity. However, they do not make the final connection to inductive effects. It is also unclear if they are referring to the carbon- or oxygen-centered proton. Survey Response 3 [Incorrect] “Alkynes are the most acidic as it forms the weakest and most stable conjugate base as it does not want to receive any hydrogens. Alkynes also have more S character, making protons more easily released.” Assessment: The student focuses on the deprotonation of the alkyne based on the s-character. The student correctly recalls the role of the s-character but does not consider the acidity of the carboxylic acid. Survey Response 4 [Incorrect] “Alkynes have a lower p K a value than alkenes, which have a lower p K a value than alkanes.” Assessment: The student recalls a set of facts but does not apply concepts beyond a recall level. Survey Response 5 [Incorrect] “The conjugate bases of B and C are more stable due to there being more resonance. Of these two, the alkyne has even more resonance.” Assessment: The student rationalizes that a higher bond order results in more resonance. The Sankey diagram in Figure 3.6. illustrates students’ trajectories in successfully ranking and explaining the acidity of the hybridization structures. The diagram supports conclusions drawn above in which students were able to accurately rank the acidity using incorrect logic. Notable is the sizable drop in the accuracy of the ranking task compared to the explanation. Figure 3.6. A Sankey diagram comparing students’ ranking and explanations on the hybridization question in the survey (N = 94, one removed due to incomplete response). Note the sizable number of students who were able to successfully rank the acidity of the structures but were unsuccessful in correctly explaining the rationale. - 52 - Additional statistical analysis of reported confidence based on the ranking and explanation correctness was completed as shown in Figure 3.7. Note the statistical difference between students who incorrectly ranked the acidity versus those who correctly ranked the acidity but incorrectly explained their rationale. There was no statistical difference between students who correctly ranked and explained versus those who correctly ranked but incorrectly explained their rationale. Students who incorrectly ranked the molecules were statistically less confident in their responses than the other groups. It appears that students who knew the trends for alkane, alkene, and alkyne acidity, and knew resonance was a key factor, had a high degree of confidence in their answers, even if their rationale may have been flawed. This illustrates considerations when designing assessments: Can students correctly answer assessment questions without actually understanding the content? What are we truly assessing, and are the questions achieving that purpose? Ranking items are quite common tasks on organic chemistry assessments, particularly in the United States. This illustrates that while common, these types of assessment items have their limitations. Figure 3.7. A comparison of confidence for students (N = 94) based on their ranking and explanation, correctness formatted as [Ranking, Explanation]. Red diamond denotes the mean. The [Correct, Correct] and [Correct, Partial] groups were aggregated following the results of Figure 3.3. and Figure 3.6. ; there was no statistically significant difference in the students’ confidence, and all students who explained correctly or partially correctly also ranked correctly. The Kruskal-Wallis test (p = 0.027) indicates that a statistical difference was observed when comparing all three groups. Further analysis compared each cohort using the Mann-Whitney test. Between the specific cohorts, a statistical difference was observed between the [Correct, Incorrect] and [Incorrect, Incorrect] groups. To validate the survey findings, 10 interviews were conducted. Students’ interview responses are summarized in the Sankey diagram in Figure 3.8. Of the 10 students interviewed, 5 correctly ranked the acidity, but none of the students were able to successfully explain the ranking. Similar explanations were provided as outlined above in which resonance, the acidity of the C-H bond, conjugation, and s-character were all noted, but these concepts were applied incorrectly or only partially correctly to account for the observed trend. Example explanations from the interviews are provided below: - 53 - Interview 1 [Incorrect (B > C > A), Incorrect] Student: “Okay, so I think I'm going to put C as the most acidic, so C as 1, B as 2, and then A. So, I guess I can start talking. The reason I'm a little hesitant — I think B is definitely above A because you get more resonance there, which can kind of help, and it's also conjugated, and conjugation is happiness, from what I've learned. But with C, I'm a little scared, because if I'm thinking about, like, the hydrogen on the oxygen goes away, and you get a resonating carboxylate -- with that alkyne, if it chooses to participate in, and kind of help out — it's going to have like, I forgot the molecule, like, allene — I don't remember if there's any stability things we need to remember about allenes, but a little hesitant about that. So, I might honestly, you know, not super confident on this one, but let's put B above C, because C is a little bit weird. The reason I wanted to put C in the start is obviously, like, alkynes are the most acidic out of our little branchy things, because you have, like, the most s-character. Also, for acidity, I remember I talked with Professor X, my old professor for OC1, and he mentioned like, when you look at really niche p K a stuff, you should always think of it like, the other hydrogens. So I'm looking at C, and like, there's one other hydrogen on the other side, which would be the most acidic carbon-hydrogen, hydrogen attached to a carbon. Let's just keep B > C > A, why not.” Assessment: The student rationalizes the acidity trends using both resonance and s-character. They rationalize that the alkynyl-substituted acid will be more acidic based on the alkyne acidity. The student does expand on why they are assessing the acidity of the alkyne by assessing the acidity of all protons. This is a valid point—we always assess the acidity of all protons. The limiting factor with the response is the failure to connect the ideas and utilize the inductive effect to explain the trend. Interview 2 [Incorrect (A > B > C), Incorrect] S: “Yeah, so B and C — electrophilicity, I believe is tied to acidity. So, if you have a more electrophilic compound, it would be more acidic. And if you look at B and C, there's resonance, and if a carbocation were to form on the carbonyl, there would be resonance with the carbon-carbon double bond, and with the alkyne in C. That would just reduce electrophilicity, that partial positive charge on the carbonyl right there, and that's how it reduces the acidity. That would just make it a lot harder for that acidic hydrogen on the right (-OH) to be pulled off, if you have resonance right there. And then if you look at A, there's no resonance there, so that's just a typical carboxylic acid.” Interviewer: “Okay, are you saying C has more resonance than B because it has two double bonds, or two π bonds?” S: “See, I was debating that but I'm not sure. Yeah, I don't know. I mean, yeah, I don't know if we've ever — we might've, or maybe I just forgot it, but like, doing resonance with alkynes. Yeah, I'm not entirely sure, but yeah I'll just stick with my answer.” Assessment: The student uses a resonance argument to justify the answers. However, the student’s rationale is jumbled. The student acknowledges an increase in acidity due to resonance but does not adequately explain why. Several statements are unclear, such as, “if a carbocation were to form on the carbonyl.” The student realizes there are no resonance structures with the alkane moiety in molecule A, but they do not fully illustrate a grasp on either resonance or how resonance arguments can be used to justify acidity. Furthermore, when - 54 - probed to consider similar resonance structures in C as in B, they fall back onto recall patterns; if it was not discussed in class, it probably does not play a role. Interview 3 [Correct (C > B > A), Incorrect] S: I know the oxygen on the carboxylic acid is what's going to leave, and off the top of my head, I feel like alkynes are more acidic because there's more s-character, so then I would rank C as the most acidic, and then B as the second one, and then A is the third. I: Okay, and so you said this oxygen from the carboxylate is going to be the one leaving, is that what you said? S: Well yeah, the H. I: So, the proton attached to this oxygen. S: Yeah, it's going to form a carboxylate. I: Okay, and then you also said alkynes are more acidic than alkenes, which are more acidic than alkanes, because these carbons have more s-character, right? S: Yes. I: So, I have a question then. Your first statement is referring to this proton (-OH) being the most acidic. Your second statement is referring to this carbon-centered proton being the acidic one. S: Yes. I: Which one do you want to focus on? S: I'm guessing only one proton is going to leave? I: Uh-huh. S: I think my ranking is still going to stay the same. But for C, I think the proton on the alkyne is more likely to leave than the proton that leaves to form the carboxylate. I: Okay. So, you're saying for C, at least, this alkyne proton is going to be the one that leaves, whereas for B and A, it's going to be the carboxylate proton? S: Yes. I: Okay, we can go with that. S: Okay, yeah, I'll go with that. I: Alright, and then, how would you justify this ranking then, based on the protons that leave? S: Okay, so the first one, for C, I would justify that because of the s-character I mentioned before — the higher s-character makes it more acidic. And I think that (alkyne) one is more likely to leave than the carboxylate, just because, in my head it seems like a stronger acid. I: Okay, and so, why does more s-character make something more acidic? S: I think it's due to the positioning of the orbitals, so with that carbon (alkyne), it is sp-hybridized. And since this is a triple bond, there are two orbitals that are going to be perpendicular — two orbitals are used to form the additional bonds — and then, I feel like the one orbital that's attached to the hydrogen is like, pointing away, which makes it easier to be deprotonated. Assessment: The student accurately recalls the relationship between s-character and acidity; the student also understands the orientation of the π bonds in the alkyne. However, they argue the deprotonation will occur at carbon in the alkynyl-substituted molecule, versus the oxygen in the other two, reflecting a tendency to latch onto the idea of “alkynes being more acidic.” Interview 4 [Incorrect (B > C > A), Incorrect] S: I think it's A > B > C. Oh, hold up, but then again, resonance probably has a role in this. So, I'd say B is the least acidic. - 55 - I: B is the least, okay. So, like this, A > C > B? S: Yeah. I: Okay, and what makes you say this ranking? S: So, this is my reasoning. So, B is resonance-stabilized, which makes it more stable, which means it doesn't want to — it's going to be a weaker acid — oh wait, never mind — B is the most acidic, I change my mind. Because the conjugate base can be resonance-stabilized, making it more stable. I: Okay. Where is the resonance in the molecule? S: Because there's a double bond attached to the oxygen, and the double bond (C=O) can be moved around across the three double bonds. There's going to be a lone pair on the oxygen attached to the H, which can contribute to the resonance as well. I: Okay, so you're saying, after we lose this proton (-OH), we'll have an extra lone pair on the oxygen, and then that oxygen, plus these two double bonds (C=O and C=C) will all be in resonance. S: In resonance, yes. I: Okay. S: And then, so I think that's the most acidic. And then, I think the alkyne will be the least acidic because the triple bond is going to withdraw the electron charge — never mind, so that's more acidic than A — so it goes B > C > A, last answer. I: Okay. S: So, I think the alkyne is more acidic because the triple bond withdraws electron density from the H, making it more acidic. I: Okay, and then A has none of those things that you just talked about, right? So, it will be the least acidic. S: Yeah. I guess all can form resonance with the double-bond O, B just forms it with the other double bond too. I: Okay, so you're saying all of them have resonance at this right side of the molecule (carboxylate moiety), but B also has resonance with the left part of the molecule (carbon chain). S: Yeah. I: I have a question for you then. Does C have that same kind of resonance as B? S: Because it's a triple bond, I don't remember ever — I'm not sure — I don't remember ever learning about triple bonds contributing to resonance. I don't think so, but I could be wrong. Assessment: The student is applying a resonance argument but demonstrates uncertainty between the alkenyl- and alkynyl- substituted carboxylic acid. Again, like in Interview 2, when probed further about similar resonance within molecule C as in B, the student falls back onto memory recall strategies. However, this represents a gap in the thought process; just because a concept is discussed in class does not mean it is applicable in every scenario. In this hybridization-focused question, the tendency of some students to latch onto resonance between double bonds reflects a lack of critical thinking during the problem-solving process, where they immediately grasp at models they have seen before without considering their limitations. - 56 - Figure 3.8. A Sankey diagram comparing students’ (N = 10) ranking and explanations on Q1 in the interview. Only half of the interviewees were successful in ranking the items, and of these only one provided partially correct reasoning. To reiterate the research questions, because of the double and triple C-C bonds on the substituents, 46% used a resonance argument to explain the trends in acidity. The s-character or hybridization was explained in 43% of responses but some students (22%) argued that the C-H bond would be the more acidic site instead of the O-H bond of the carboxylic acid. Therefore, regarding the first research question: How do OC2 students apply the concepts of hybridization, inductive effects, resonance, and conjugation to rationalize acid strength? It is clear that students remember key facts from lectures or textbooks, but often they did not extend and apply these concepts to new structures. Similar findings have been reported about students’ application of resonance (Brandfonbrener et al., 2021) and with explaining acid-base trends (Bretz & McClary, 2014), but this study provides an extension by asking students to identify the most appropriate model to explain the trend. The second research question focused on the alternative conceptions students demonstrate. Three alternative conceptions were identified: the first two focus on resonance, with one being that resonance is always the “answer” regarding structures that have double or triple bonds. The second alternative conception is the idea that as bond order increases (from a double to a triple bond), the number of resonance structures also increases. Resonance is introduced in introductory chemistry and reiterated throughout organic chemistry, which may explain its prevalence as an alternative conception that influences student rationale. The third alternative conception focuses on the more acidic site. Some students used structural cues and rote memory to identify the more acidic sites rather than carefully analyzing the acidity of each proton, which is consistent with similar studies in organic chemistry (Bhattacharyya, 2006; Bretz & McClary, 2014). Finally, regarding the third subpart in every question probing confidence, students who incorrectly ranked the acidity did report statistically lower confidence in their answers, but students who correctly ranked the items using flawed logic reported the highest confidence in their answers of the three groups. This was expected given the design of the problem—in which students were expected to use incorrect models in their justifications. - 57 - Q2: Aromatic Acidity To provide a comparison with our findings concerning the hybridization structures, the second survey question focused on the acidity of substituted aromatics compounds. This is a standard question in organic chemistry, and students have certainly encountered similar problems in the textbook, during lectures, and on assessments. Because of this familiarity, we hypothesized that students would be more confident and would perform better on both the ranking and explanation questions for the aromatic structures relative to the hybridization structures. Figure 3.9. summarizes the confidence of students who correctly and incorrectly ranked the acidity of the aromatic structures. As hypothesized, students who successfully ranked the aromatic molecules did report higher confidence compared to the hybridization problem, likely because they had seen similar questions earlier in the semester. A statistically significant difference was observed between the reported confidence for students who correctly versus incorrectly ranked the acidity. Figure 3.9. A comparison of students’ (N = 95) reported confidence based on whether they correctly or incorrectly ranked the acidity of the aromatic structures. Red diamond denotes the mean. Figure 3.10. summarizes the reported confidence for students based on the correctness of their explanations to justify the ranking of the acidity of the aromatic structures. Again, the responses were coded as correct, partially correct, or incorrect as outlined in the Methods. There was a statistical difference in the confidence reported for students providing correct and partially correct explanations versus incorrect explanations. There were no statistical differences in confidence reported for students who provided correct or partially correct explanations. The responses were qualitatively coded and tabulated as shown in Table 3.4. - 58 - Figure 3.10. A comparison of students’ (N = 95) reported confidence and the correctness of their explanations regarding the acidity of the aromatic structures. Red diamond denotes the mean. The Kruskal-Wallis test (p << 0.05) indicates that a statistical difference was observed when comparing all three groups of students. Further analysis using the Mann-Whitney test to make pairwise comparisons revealed a statistical difference between the correct versus incorrect, and partially correct versus incorrect groups. Table 3.4. Deductive and Inductive Codes for Student Explanations to Q2 – Aromatic Acidity Code Descriptions Percentage of students (N = 95) Deductive Codes Mentioning conjugate base stability 32 Equating functional groups with acidity 31 Inductive Codes Para-EWGs are more acidic than meta-EWGs because 35 they participate in resonance EWGs increase acidity; no further discussion 29 Para-EWGs are more acidic than meta-EWGs / meta- 20 EWGs are less stabilizing than para-EWGs NO2 is an EWG, CH3 is an EDG; identification of 19 unique substituents Meta-EWGs are less acidic than para-EWGs because 13 they only exert inductive effects EDGs decrease acidity; no further discussion 10 EDGs increase electron density at the oxygen in the 8 conjugate base, which makes the molecules less acidic EWGs decrease electron density at the oxygen in the 8 conjugate base, which makes the molecules more acidic - 59 - Table 3.4. continued EWGs stabilize the conjugate base through inductive 7 effects Para-substituted phenols are always more acidic than 7 meta-substituted because para- has more resonance structures Activation/activation of the aryl ring determines 7 acidity; more activated rings are less acidic, more deactivated rings are more acidic Meta-EWGs are more acidic than para-EWGs because 4 they are closer to the hydroxy group, so they exert a stronger inductive effect Para-EWGs are less acidic than meta-EWGs / meta- 4 EWGs are more stabilizing than para-EWGs In comparison to the hybridization question, a larger proportion of student explanations were correct and applicable, but often insufficient detail was provided to fully explain the acidity ranking. For example, from Table 3.4., the following statements are correct and applicable, but more information is needed to fully explain the trend: 1. NO2 is an EWG, CH3 is an EDG; identification of unique substituents 2. Meta-EWGs are less acidic than para-EWGs because they only exert inductive effects 3. EWGs increase acidity; no further discussion 4. EDGs decrease acidity; no further discussion The four statements are applicable for this problem, but students needed to draw connections that were largely missing in explanations. Namely, students needed to relate both the substituent effect (i.e., electron-donating or electron-withdrawing) and its position (i.e., ortho, meta, or para) relative to the acidic proton. Student explanations were limited and focused on only one of the two items. Both resonance and inductive effects should have been emphasized to fully explain the trend. The following responses (ordered correct, partially correct, and incorrect) provide representative explanations from the survey: Survey Response 1 [Correct] “The negative charge of the phenoxide can be delocalized through resonance to the nitro in the para position, but not to the nitro group in the meta position. However, the compound with the nitro group in the meta position can still stabilize the negative charge through an inductive effect. Compound A is the least acidic because the methyl is an electron donating group that destabilizes the negative charge of the phenoxide.” Assessment: The student accurately classifies the substituents as electron-donating and electron-withdrawing and accurately describes trends using both resonance and inductive effects. Survey Response 2 [Partial] “After deprotonation, the strongly electron withdrawing NO2 group will stabilize the electron density in the conjugate base. This effect is most prominent in the para-positioning. The meta-positioning still stabilizes the electron density to an extent, and the electron-donating nature of the methyl group provides less stability to the conjugate base.” - 60 - Assessment: The student accurately rationalizes that EWGs stabilize the conjugate base, and EDGs destabilize the conjugate base. However, they do not explicitly distinguish between para-nitro and meta-nitro; “to an extent” does not mean only inductive effects versus inductive + resonance effects. Survey Response 3 [Partial] “B is the most acidic because the electron withdrawing group being in the para position allows resonance/electron delocalization to occur for c, the EWG in the meta position does not allow resonance for a, the methyl at the para position is an electron donating group that activates the ring.” Assessment: The student correctly rationalizes the role of electron-withdrawing groups versus electron-donating groups for increasing or decreasing the acidity. The student also correctly identifies the substituents as electron-withdrawing or electron-donating. The presence of resonance stabilization in the para-substituted molecule is correctly identified. Collectively, the student has all pieces except for the rationalization of the inductive effects from the meta- substituted group in C. Survey Response 4 [Incorrect] “Molecule B would be most acidic because the loss of the hydrogen could allow for resonance stabilization with the really good Electron withdrawing group in the para position to take on the charge in resonance structures well, then A next because the methyl group does a helpful inductive effect, not as much as electron withdrawing but still helpful in para position, and then the NO2 in the meta position last because the NO2 is a good EWG but isn't really that helpful when in the meta position.” Assessment: Although the student correctly notes that a para-nitro group allows for resonance stabilization of the conjugate base, the other statements are incorrect. The methyl group does not exert a “helpful inductive effect”; it destabilizes the conjugate base instead. Furthermore, the student states that the meta-position prevents substituents from exerting inductive effects. Survey Response 5 [Incorrect] “B has the most ability to delocalize charge with resonance when the OH is deprotonated negative charge can be pushed all the way around the ring onto the NO2 oxygens. C has a nitro group in the meta position, making it ineffective at stabilizing the ion that forms.” Assessment: The student accurately explains the delocalization (presumably through resonance) for the para-nitrophenol. The role of the methyl group is not discussed, and the student notes that the nitro group in C has no impact (presumably through resonance). The inductive effects of the meta group in structure C are not considered. Survey Response 6 [Incorrect] “EDG of Me will allow for the molecule to be more stable. NO2 is an EWG so when it is meta to the OH group it prevents electron stability. Stability will increase acidity since the negative charge of losing the hydrogen can be better distributed.” Assessment: The student does not understand the impact of an electron-donating versus electron-withdrawing group on increasing or decreasing acidity. The meta- substitution is said to “prevent electron stability” supporting that only resonance effects are being considered. - 61 - From the sample responses, the responses have correct ideas but fail to fully incorporate each of the key points required to explain the trend. The difference in electron-donating versus electron-withdrawing groups is emphasized to explain the reactivity of aromatic compounds. Given the focus on substituents to explain both acidity and reactivity, most students likely memorize the substituents that act as electron-donating versus electron-withdrawing groups. Therefore, while students readily draw on memorized knowledge and recognize when certain arguments are pertinent, they fall short of fully synthesizing each idea to explain trends cohesively. The primary alternative conception that emerged from this question is the idea that substituents in the meta position do not modulate acidity. Resonance models work when relating acidity for certain substituents with an ortho or para substitution, but they cannot be used to gauge the impact of substituents in the meta position. Resonance structures are drawn extensively when discussing aromatic molecules, which explains why students gravitate toward these models. Additionally, resonance structures provide tangible representations that students can draw and visualize in place of models such as inductive effects that are not as easily drawn on paper. However, inductive effects are pertinent for considering the role of electron-withdrawing or electron-donating groups in the meta position. Figure 3.11. illustrates the Sankey diagram for the aromatic acidity ranking and explanations. Additional statistical analysis of reported confidence based on the ranking and explanation correctness was completed as shown in Figure 3.12. As with the hybridization structures, students were more successful at ranking the acidity of the aromatic structures than explaining the trend, but the difference in success rates was less substantial. This question was designed to serve as a control question, and distractors were not incorporated. Students either knew how to approach the problem or did not. Students whose explanations were marked as partially correct were likely not thorough or explicit enough with their explanations to earn a correct score but still had a decent understanding of the rationale. With the hybridization structures, distractors were incorporated, such as conjugation and the C-H bonds on the alkyl, alkenyl, and alkynyl substituents to promote the wider array of models implemented to gauge alternative conceptions. For the aromatic structures, students who readily apply the models they had previously encountered were more likely to successfully rank the acidity and explain their rationale. Therefore, the reported confidence more closely aligns with their success in ranking and explaining the acidity of the aromatic structures. Figure 3.11. A Sankey diagram comparing students’ (N = 95) ranking and explanations on the aromatics question on the survey. Note that some students reversed the ranking in the first part but were able to provide a correct or partially correct explanation. The incorrect ranking could have been associated with not carefully reading the problem prompt. - 62 - Figure 3.12. A comparison of confidence for students (N = 92) based on their ranking and explanation, correctness formatted as [Ranking, Explanation]. Red diamond denotes the mean. Again, the [Correct, Correct] and [Correct, Partial] groups were aggregated following the results of Figure 3.10. and Figure 3.11.; there was no statistically significant difference in the students’ confidence, and nearly all students who explained correctly or partially correctly also ranked correctly. The Kruskal-Wallis test (p << 0.05) indicates that a statistical difference was observed when comparing all three groups. Further analysis compared each cohort using the Mann-Whitney test and found a statistical difference between the [Correct, Correct/Partial] and the other two groups. Figure 3.13. summarizes student performance on the hybridization and aromatic ranking items. The Sankey diagram illustrates students performed better with ranking the acidity of the hybridization structures relative to ranking the acidity of the aromatic structures. On the other hand, Figure 3.14. illustrates students were less successful in explaining their reasoning on the hybridization item. This was expected given the design, with the hybridization structures providing distractors to provide insight into students’ alternative conceptions. The differences between success in ranking versus explaining the items can be attributed to several factors. 1) With the hybridization structures, students could successfully rank the acidity using incorrect models, which was reiterated through the explanations. 2) With the substituted aromatic structures, students were required to incorporate several models to explain the trends based on both the effect and position of substituents, a task that necessitates a more sophisticated use of models. However, distractors were not included in the aromatic structures, which necessitated the recalibration of how models are used to explain the observed trends. Consequently, 3) success in the ranking was attributed less to luck or memory for the aromatic structures. This explains why there was a smaller gap between success in ranking and explaining the trends in acidity for the substituted aromatic molecules. - 63 - Figure 3.13. A Sankey diagram comparing students’ ranking for the hybridization and aromatic structures on the survey (N=93). Figure 3.14. A Sankey diagram comparing students’ explanations for the hybridization and aromatic structures on the survey (N=93). To reiterate the research questions for the aromatic structures: students demonstrated a greater tendency to apply and accurately explain the role of resonance in modulating acidity in the presence of a para-substituted electron-withdrawing group. To correctly explain the trend in acidity, students needed to consider both resonance and inductive effects as well as their competing electronic effects. As with the hybridization structures, students demonstrated a greater tendency to apply resonance but a weaker ability to identify and apply inductive arguments. The primary alternative conception identified from the survey responses for the aromatics question is the notion that the meta substituents do not impact acidity because of the lack of resonance delocalization. This reiterates students’ tendency to attempt to use resonance - 64 - arguments for conjugated systems, despite the existence and applicability of other models. The origin of this alternative conception may be the extensive use of resonance structures to explain aromatic reactivity and regioselectivity. Students demonstrated a greater awareness of their abilities with statistical differences in the reported confidence between correct and incorrect rankings and/or explanations. Given this problem was not designed to act as a distractor, this finding supports that students do gain insight into their abilities as they progress in chemistry. This finding is counter to McClary and Bretz’s finding (L. M. McClary & Bretz, 2012), but the timing of the survey and prior course experiences could be used to account for the differences. Conclusions and future work The findings from the analysis of the survey and interview data illustrate that students have difficulty rationalizing acid-base trends. The challenges with acid-base chemistry are widely reported in general chemistry, and these challenges persist longitudinally across the chemistry curriculum. A solid understanding of acid-base chemistry is paramount in organic chemistry because many reactions can be rationalized using the frameworks of acid-base chemistry. Students are introduced to models to rationalize these trends, but many experience challenges with identifying and applying these models to novel situations. This research illustrates that with ranking items and explanations, students implement ideas from memory, based on familiar structural features. For the alkyl, alkenyl, and alkynyl-substituted carboxylic acids, students latched onto hybridization, often recalling trends relating to s-character. However, they did not fully develop ideas from this concept. They know that carbon atoms with more s-character are more acidic, but few students made the connection to electronegativity, and even fewer students extended that to inductive stabilization of negative charge. There is a considerable gap between hybridization and inductive effects that most students, near 90%, are not bridging. Using these facts, many students were successful at comparing the acidity of structures differing in hybridization alpha to a carboxylic acid. However, when asked to explain, students did not link and synthesize these facts into a cohesive explanation. Students applied one or more of these potentially memorized facts incorrectly despite correctly ranking the acidity. Students exhibited a similar strategy with ranking the acidity of the aromatic structures in which they recalled facts, such as classifying substituents as electron-donating or electron-withdrawing, to explain the trends in acidity. Unlike ranking the acidity of the carboxylic acids, successfully ranking the acidity of the substituted aromatic molecules required using multiple pieces of information including substitution pattern and substituent effects; it was not possible to apply a single fact to be successful with the ranking. However, there were no embedded distractors that were designed to limit the efficacy of specific models for explaining the trend. As expected, there was a smaller gap in the performance on the ranking and explanation for the aromatic structures than the carboxylic acids. Future directions for the research will provide strategies for improving students’ identification, understanding, and application of models in organic molecules. The research is expanding into general chemistry and the first semester of organic chemistry, with an overarching goal of developing a longitudinal model for students’ progression with acid-base chemistry. Through these studies, we aim to identify alternative conceptions students develop in general chemistry that propagate through the curriculum, with the goal being to develop interventions aimed at promoting cognitive dissonance, such that students become aware of their alternative conceptions. Much like these surveys, which required explanations, the interventions will have students explore the scope and limitations of models. One possible intervention would be to perform our scoring criteria analysis with chemistry students, which would help them gain a better grasp on how to implement models and reiterate the importance of not solving problems using only rote memorization. From our findings, many second-semester organic chemistry students, despite a considerable background in chemistry, still appear to rely on memorized - 65 - facts and knowledge from previous examples to tackle new problems. The goal of organic chemistry is to develop a toolbox to make predictions regarding novel reactions, which requires successful synthesis of concepts and application of models, in lieu of primarily factual recollection. Acknowledgements The research was supported by Duke University and the Undergraduate Research Support Office Independent Study Grants. Special thanks to the students who participated in the surveys and interviews. Statement of contributions SG led the project and contributed to the literature review, design of the research instruments, development of frameworks for analyzing data, and collection of interview data. TCO assisted with developing frameworks for coding survey and interview data and collaborated with SG and JGL to code data. JGL assisted with coding the survey and interview data. AF assisted with the statistical analysis. JLR assisted with the design of the research instruments and aided in collecting data. CM contributed to the design of the research instruments. CTC provided mentorship for the research design, data collection, and data analysis. All authors contributed to the preparation of the manuscript. References ACS. (2015). American Chemical Society: Committee on Professional Training Retrieved from https://www.acs.org/content/dam/acsorg/about/governance/committees/training/2015-acs-guidelines-for-bachelors-degree-programs.pdf Anderson, T. L., and Bodner, G. M. (2008). What can we do about ‘Parker’? A case study of a good student who didn't ‘get’ organic chemistry. Chemistry Education Research and Practice, 9(2), 93-101. Bhattacharyya, G. (2006). Practitioner development in organic chemistry: how graduate students conceptualize organic acids. Chemistry Education Research and Practice, 7(4), 240-247. Bhattacharyya, G., and Bodner, G. M. (2005). "It Gets Me to the Product": How Students Propose Organic Mechanisms. Journal of chemical education, 82(9), 1402. Brandfonbrener, P. B., Watts, F. M., and Shultz, G. V. (2021). Organic Chemistry Students’ Written Descriptions and Explanations of Resonance and Its Influence on Reactivity. Journal of chemical education, 98(11), 3431-3441. Bretz, S., and McClary, L. (2014). Students’ Understandings of Acid Strength: How Meaningful Is Reliability When Measuring Alternative Conceptions? Journal of chemical education, 92(2), 210-219. Caleon, I. S., and Subramaniam, R. (2010). Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions. Research in Science Education, 40(3), 313-337. Cartrette, D., and Mayo, P. (2011). Students' understanding of acids/bases in organic chemistry contexts. Chem. Educ. Res. Pract., 12(1), 29-39. Cooper, M. M., Kouyoumdjian, H., and Underwood, S. M. (2016). Investigating Students’ Reasoning about Acid–Base Reactions. Journal of chemical education, 93(10), 1703-1712. Cox, C. T., Poehlmann, J. S., Ortega, C., and Lopez, J. C. (2018). Using Writing Assignments as an Intervention to Strengthen Acid–Base Skills. Journal of chemical education, 95(8), 1276-1283. - 66 - Crandell, O. M., Kouyoumdjian, H., Underwood, S. M., and Cooper, M. M. (2019). Reasoning about Reactions in Organic Chemistry: Starting It in General Chemistry. Journal of chemical education, 96(2), 213-226. Duis, J. M. (2011). Organic Chemistry Educators’ Perspectives on Fundamental Concepts and Misconceptions: An Exploratory Study. Journal of chemical education, 88(3), 346-350. Grove, N. P., and Lowery Bretz, S. (2012). A continuum of learning: from rote memorization to meaningful learning in organic chemistry. Chemistry Education Research and Practice, 13(3), 201-208. Jones, M., and Fleming, S. (2010). Organic Chemistry (E. Fahlgren, Ed. In: New York: WW Norton & Company.(Original work published 1997). Kim, T., Wright, L. K., and Miller, K. (2019). An examination of students' perceptions of the Kekulé resonance representation using a perceptual learning theory lens. Chemistry Education Research and Practice, 20(4), 659-666. Liptak, M. D., Gross, K. C., Seybold, P. G., Feldgus, S., and Shields, G. C. (2002). Absolute pKa Determinations for Substituted Phenols. Journal of the American Chemical Society, 124(22), 6421-6427. McClary, L., and Talanquer, V. (2011). College chemistry students' mental models of acids and acid strength. Journal of Research in Science Teaching, 48(4), 396-413. McClary, L. M., and Bretz, S. L. (2012). Development and Assessment of A Diagnostic Tool to Identify Organic Chemistry Students’ Alternative Conceptions Related to Acid Strength. International Journal of Science Education, 34(15), 2317-2341. McClary, L. T., V. (2011 ). College chemistry students' mental models of acids and acid strength. J ournal of Research in Science Teaching, 48(4), 396-413. Mercier, P. H. (2018). Applying Le Châtelier’s Principle To Model Strong Acid–Strong Base Titration. Journal of chemical education, 95(4), 521-527. Petterson, M. N., Watts, F. M., Snyder-White, E. P., Archer, S. R., Shultz, G. V., and Finkenstaedt-Quinn, S. A. (2020). Eliciting student thinking about acid–base reactions via app and paper–pencil based problem solving. Chemistry Education Research and Practice, 21(3), 878-892. Rossi, R. D. (2013). What Does the Acid Ionization Constant Tell You? An Organic Chemistry Student Guide. Journal of chemical education, 90(2), 183-190. Schmidt-McCormack, J. A., Judge, J. A., Spahr, K., Yang, E., Pugh, R., Karlin, A., . . . Shultz, G. V. (2019). Analysis of the role of a writing-to-learn assignment in student understanding of organic acid–base concepts. Chemistry Education Research and Practice, 20(2), 383-398. Stoyanovich, C., Gandhi, A., and Flynn, A. B. (2015). Acid–Base Learning Outcomes for Students in an Introductory Organic Chemistry Course. Journal of chemical education, 92(2), 220-229. Xue, D., and Stains, M. (2020). Exploring Students’ Understanding of Resonance and Its Relationship to Instruction. Journal of chemical education, 97(4), 894-902. - 67 - Chapter 4 Doi: 10.26529/9789612532970/ch4 A Whole Team Approach to Integration of Student Feedback into Continuous Assessment Activities for First-Year Students Transitioning to University Chemistry Education During the COVID-19 Pandemic Frances Heaney1*, Denise Rooney1, Orla Fenelon1, Tobias Krämer1, Eithne Dempsey1, Stephen Barrett1, Caytlin Boylan1, Kyle Doherty1, Luke Marchetti1, Joseph Curran2, Lisa O’Regan2, and Trinidad Velasco-Torrijos1 1Department of Chemistry, Maynooth University, Maynooth, Co. Kildare, Ireland 2Centre for Teaching and Learning, Maynooth University, Maynooth, Co. Kildare, Ireland * Corresponding author: frances.heaney@mu.ie Abstract This paper reports on a team-led, discipline specific solution to the problem of gathering, analysing and responding to student feedback on teaching and learning in a timely manner and in a way that supports transition to university for the in-situ group. We demonstrate the pedagogical value of integrating student feedback into mainstream, on-going teaching and learning activities as a vehicle to increase engagement and improve representation. The cohort was a large and diverse first year chemistry class (350 students) transitioning from second to third level teaching and learning methods in an Irish University during the COVID-19 pandemic where the bulk of activities were delivered either remotely or in a blended fashion. We show that a continuous assessment framework can be piggy-backed to gather student feedback and enact informed improvements in a manner which is both immediate and noticeable. We believe our approach is an excellent fit for chemistry programmes that could readily and successfully be incorporated into other programmes in cognate subjects and could be easily adapted for second and higher year students. Keywords: Integration of Feedback: Team Approach: Responsive Structures: Embedment of Culture: Continuous Cycle: Student Feedback: Academic Feedback. Introduction The transition from second to third level, a critical development phase for many students is accompanied by a myriad of social, financial, and academic challenges, consequently the first-year encounter has been extensively reviewed in the literature. Leong et al. identify secondary educators alongside students and tertiary educators as stakeholders in the perception of preparedness and the successful transition of chemistry students (Leong et al., 2021) whilst De Clercq et al., with a focus on biology majors, have teased out the dynamic and complex nature of the temporal adjustment process (De Clercq et al., 2018). The role of assessment practices in developing the first-year students’ confidence and sophistication as learners has been studied by Hodgson et al. (Hodgson et al., 2011) whilst Miltiadous et al. go further in associating a multitude of engagement activities, including weekly assessment tasks, with an increased likelihood of success in this formative period (Miltiadous et al., 2020). It is recognised that success, retention and development are closely linked to student satisfaction (Beltyukova & Fox, 2002) as is the probability of completing the undergraduate programme on time (Bussu et al., 2019) and the student perception of their higher education institute (de Lourdes Machado et al., 2011). In an extensive article on the use of formal instruments to measure students’ satisfaction in North American, British and Australian - 68 - settings Richardson discusses key practical issues of the why, the what, and the when of student feedback, but recognises an apparent disconnect in the significance students and teachers attach to student feedback and the seriousness with which it is taken by teachers and institutions (Richardson, 2005). Some educators recognised for excellence in teaching caution that student ratings of teaching should be used as only one component of the holistic reflection (Cain et al., 2019; Pienta, 2017). Despite this caveat across the higher education sector there is a great value placed on feedback processes with formative and summative implications for ensuring quality and continuous improvement in experience at programme, teacher and individual student level (Okogbaa, 2016). Staff-to-student academic feedback (AF), and student-to-staff course evaluation feedback (SF), formative or summative, are twin pillars of this dialogic process. The pedagogy, methods and mechanisms for provision of formal and informal AF have been extensively studied (Dawson et al., 2019; Mulliner & Tucker, 2017). Reports have focussed generally on the challenges and conditions to enable effective feedback (Henderson et al., 2019), as well as specific approaches tailored e.g., to work-based settings (O’Malley et al., 2021). In one interesting article Hwang advocates for the soliciting of weekly low-stakes assignments to gauge student learning online (Hwang, 2020). Whilst SF offers one key perspective for course evaluation as Richardson points to in his excellent review, there are significant challenges and time constraints to gathering representative feedback from large student cohorts with increasing diversity in the population; the student impetus for providing information in end-of-module surveys is weakened by the fact that their personal experience of the module/course is complete, and issues with sampling error and sample bias jeopardise the validity of data extrapolation (Richardson, 2005). One tactic to encourage the student voice and increase the response rate is to devote in-class time for survey completion (Lau, 2019), yet, where lecture attendance is not compulsory the merit of this face-to-face (f2f) approach is clearly limited by the proportion, and likely the diversity of students who turn up to class. Indeed, having identified clear differences in course assessment between no-show sub-groups and the overall class ratings, Treischl advocates for an online survey mode to capture the opinion of those not in attendance (Treischl & Wolbring, 2017). Prior to the widescale COVID-19 induced shift to online/blended teaching selected programmes had already adopted this pedagogical approach and shown that existing tools could be adapted to evaluate the effectiveness of online teaching (Ravenscroft et al., 2017), and significantly, that quality SF can be obtained (Watson et al., 2017) and is not undermined by online collection (Gakhal & Wilson, 2019). Focus group interviews, also reported to be effective in an online format, (Almendingen et al., 2021) can be a useful complement to other feedback approaches. They afford an opportunity to gain a deep insight from a few individual students, however the opinions they capture may not be statistically representative and may suffer from “groupthink”. There are challenges with student feedback literacy denoting the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies (Carless & Boud, 2018). There is also a need to empower students by showcasing the value of their opinions in the shaping of teaching and learning both for themselves and for future students (Isaeva et al., 2020). To ensure quality and enhance a sense of ownership of their learning it is important that the student information is gathered from a reasonable sample size (Holland, 2018), and that departments role model feedback. There is a need to build trust between students and staff; as partners students should be made aware that their input will be considered and acted upon as appropriate (Asghar, 2014; Bovill et al., 2015). - 69 - Research Problem Our research question was on the potential in-person or synchronous participation in weekly laboratory/workshop activities to afford a discipline specific platform to integrate student feedback (SF) into mainstream first-year chemistry activities. Integration of learning is an active topic in higher education, it implies there is an enhancement or something better, something of a higher-order about learning when it is integrated (Leadbeatter, 2021). Our research set out to explore if the integration of both SF with laboratory related activities, pre-lab talks, and AF could simultaneously deliver academic related learning outcomes and provide a platform for a holistic engagement with students about their teaching and learning experiences. Academic support, technology support, health and well-being, and a sense of community have been identified as “four pillars” supporting student success (Roddy et al., 2017). The first-year is the cornerstone of the University experience, the pedagogical transition from second level is significant however, it also provides an opportunity to embed a culture of engagement and to develop the concept of student feedback literacy. Most students perceive practical work as engaging, motivating (Smith & Alonso, 2020), and taking place in a low stakes environment. We questioned the compatibility of including ongoing and inclusive SF opportunities into main-stream academic activities associated with practical work. The (potential) benefits of structured SF, pre-, mid- and end-of-year (Holland, 2018; Sozer et al., 2019) for identifying just-in-time and specific muddy point issues, e.g. on remote learning of organic chemistry, have been reported (Ramachandran & Rodriguez, 2020). However, our question differed significantly from these published works in that it seeked to explore the value of a continuous (weekly) two-way feedback process exploiting multiple survey modes and probing holistic concerns as well as discipline related teaching and learning themes. Diligent collection of student feedback is necessary, but not sufficient to affect an improvement in the student experience. First-year chemistry programmes, far from the single lecturer led models, involve input from a number of academic and technical staff, as well as laboratory and workshop demonstrating staff; for this reason, the requirements for a responsive feedback process distil down to the need for a whole team approach, the motivation to make improvements and a degree of agility within the course structure and organisation. The team assembled for this project included the Head of Department, four academic staff, including two first-year Chemistry Coordinators, one technical staff member and four postgraduate students. Method Implementation: As part of a university wide initiative “Enhancing Teaching and Learning through Programme and Module Evaluation” our primary consideration was to discover how to synergistically gather SF whilst enhancing teaching and learning in a way that first-year chemistry students would recognise as a valuable experience in real-time. Embedding Feedback in Course Structure: Our key approach was to offer all students an opportunity to participate in a scheduled feedback meeting where a structured SF discussion would be combined with the weekly pre-lab talk and class level AF on previously submitted reports, Figure 4.1. In parallel, we introduced a weekly Chemistry drop-in centre, which students could attend on an optional basis. - 70 - Academic Student Pre-Lab Feedback Feedback Talk AF SF Weekly, Facilitator Led Student Feedback Meeting Figure 4.1. Approach to integration of student feedback (SF) with mainstream curricular activities: SF is normalised through embedment with pre-lab talks and class level academic feedback (AF) in a weekly scheduled class involving small groups of students. To best meet the needs of all stakeholders each component of the feedback meeting was carefully considered: the optimal size of the feedback groups, the approach to scheduling and meeting students during the period of COVID-19 restrictions, the affordances of our Virtual Learning Environments (VLEs, Moodle and MS Teams in our case), the most suitable personnel to lead the activities, the environment and tools to best promote engagement, the structure of the feedback discussions, how to encourage students to initiate discussions, the mechanics of recording, reviewing and analysing the opinions offered by the students, and finally the most effective means to close the feedback loop and report back to students. It is reported that initiating and maintaining instructor-student rapport is linked to motivation. Classroom engagement is facilitated by smaller class, ~20-40 students (Flanigan et al., 2021), and a sense of belonging supports student success (Roddy et al., 2017). For these reasons, feedback groups were designed to have up to 25 participants. In our department, first year lab/workshop classes are scheduled towards the end of the week, so we were constrained to scheduling feedback meetings incorporating pre-lab talks on Mondays or Tuesdays. Slots within the congested first-year timetable that were either vacant or not excessively overlapping with other first science subjects were identified. Students were invited to use a Moodle Choice activity to self-enrol in one of 12 slots that suited their individual schedule; it would have been logistically difficult, and so no effort was made to align the members of each feedback group with a particular lab group. Since COVID-19 restrictions seriously impacted on f2f gatherings during this period we relied on digital tools and the feedback groups were hosted by video conferencing on MS Teams. Since we felt students would be more likely to give honest feedback on their teaching and learning experience to someone other than their current lecturers post graduate students from the department were recruited as friendly faces to lead these meetings. We subsequently referred to these leaders as Feedback Facilitators (FF). We did not see the choice of FF as the student facing personnel as by-passing an opportunity to build up trust between staff and students, rather we ensured FFs were well briefed and carefully articulated the cyclical nature of the process from Staff-FF-Students-FF-Staff to students from the outset. - 71 - Alternative Feedback Channels: To investigate alternative mechanisms to gather feedback, we complemented the weekly structured feedback sessions which students were required to attend with a drop-in centre which students could attend on an optional basis. Academic support was integrated with feedback in the form of a short anonymous Moodle questionnaire. The drop-in centre, also via MS Teams, was staffed by one of the FF. It was hosted on a Wednesday at 3-4 pm and again at either 5-6 pm or 7-8 pm; the evening slot, not a traditional time for chemistry classes, was selected with a view to promoting inclusivity and providing flexibility for students with additional responsibilities. We considered a suggestion box to be left at the exit point of the teaching laboratory to facilitate students in posting immediate comments on any aspect of their learning in chemistry. However, this proposal did not gain any traction during this period of COVID restrictions and so our portfolio of feedback approaches was completed with the provision of two online focus group meetings to gain a deeper understanding of the student perspective. Facilitator Training, Digital Tools and Documentation: Given logistical components of online delivery, timetable constraints, and staff scheduling, we considered how to deliver a common and consistent feedback meeting approach across all 12 student groups. We were aware of the importance of the appropriate choice of digital tool(s) to achieve engagement, active learning and team teaching (Tan et al., 2020). We create a dedicated MS Team with academic and technical staff, laboratory demonstrators and FF as members to form the backbone of our information flow and this was supported by regular email communication and occasional online meetings. To empower the staff involved in the project, meetings were held to clarify the role of all members, to explain the project goals and the mechanics of the operation. From the outset it was clear that arriving at a shared understanding of key terms: academic and student feedback (AF and SF), and “closing the feedback loop” would be imperative to the success of the project. In line with their student-facing role, the facilitators worked to develop feedback literacy within their groups. Thus, we imparted an understanding of AF as information from lecturers/facilitators/demonstrators to students, and of formative class level feedback as a tool with potential to improve quality and support learning. We contrasted this with an understanding of SF as information from students to lecturers/facilitators/demonstrators, valuable in providing staff with information on what works well and what does not, and which also gives students an opportunity to reflect on their learning experience whilst promoting a relationship between the students and the department. Finally, we shared that “closing the feedback loop” meant returning to the students with a review on how their information was meaningfully considered/acted upon by departmental staff (Curran, 2021). As all facilitators were familiar with the learning outcomes of each lab and competent to deliver the pre-practical talk, the pre-lab component of the meeting did not require a shared resource. However, only one of the facilitators was also a first-year laboratory demonstrator. To provide quality AF on submitted reports, all laboratory demonstrators were asked to contribute to a summary document commenting on how lab assignments were presented, what went well and what was problematic highlighting the main knowledge gaps. These documents were accessible to all members in the above-mentioned dedicated Team folder. To focus on key elements rather than generate a diffuse collection of information the weekly SF discussions were structured by topic and by response platform. Each week, in addition to encouraging students to proactively raise topics of concern, all groups concentrated on specific themes. Thematic areas were chosen to allow an insight into both academic and non-academic student concerns. Facilitators could find the topics, identified by initiative staff, in a Teams folder laid out in the form of a templated word document: groups of questions were designed to gather information from a range of areas including technology/online learning environment, - 72 - subject content, study and home environment, approach to time management, time spent on self-study platforms, study routine, external work/caring commitments, peer support, preparedness for practical sessions, understanding of content, perception of links between lab and lecture content, usefulness of online videos, clarity of online course platforms, preferences for synchronous vs asynchronous delivery and finally an evaluation of student confidence in the subject as the year progressed. On alternate weeks facilitators invited their groups to engage, during the feedback session time, either anonymously by responding online to a Moodle questionnaire, by contributing, by voice or in chat to a discussion on MS Teams, or an anonymously to a Padlet wall. Padlet is an online noticeboard tool which allows immediate collection and real-time display of inputted information. We planned to draw qualitative information from the Padlet facilitated conversations and both qualitative and quantitative information through the Moodle survey mode. Our reliance on online over traditional, paper based questionnaires is supported by Treischl (Treischl & Wolbring, 2017) and Gakhal’s (Gakhal & Wilson, 2019) observations on the impact of survey mode on the quality of SF. Student responses to questionnaires arising from both the feedback sessions and the drop-in centre meetings were hosted on an “All First-Year” Moodle web page and were immediately accessible to all initiative staff. Facilitators populated templated word documents to summarise the live discussions and returned the completed files to the appropriate MS Team folder. Closing the Feedback Loop: To help students see the value of providing feedback it is incumbent on staff to role model good feedback practice, close the loop, and inform students of the impact their feedback has on course development. Such practices empower students to own their experience through conversations where they can know their voices are listened to. The weekly cycle of feedback meetings provides for timely opportunities to close the feedback loop; at regular intervals members of the initiative team reviewed the documents in the Teams repository summarising student information from the weekly meetings. Student opinion was categorised according to the concern or the learning experience it related to viz workshop or laboratory content, lecture, tutorial or assessment concerns, or the overall holistic student experience viz technical matters, issues with VLEs and learning support issues, workload or communication with the department. As necessary the initiative team consulted the wider staff involved in first-year teaching and items were subsequently flagged as for clarification, actionable or not feasible. The outcome of these considerations was shared with the FF who in-turn kept students abreast on how their information was being responded to. The interaction between the project partners and the feedback cycle are summarised in Figure 4.2. - 73 - Figure 4.2. Graphical representation of project partners, academic and technical staff (initiative team), demonstrators, feedback facilitators (FF) and first-year students, and the communication flow/decision making mechanisms. It indicates the importance of the whole team working together to provide a systematic approach across the weekly Student Feedback Meetings with academic feedback AF and thematic topics for student feedback SF provided to the FF by demonstrators and initiative staff respectively. It shows the circular flow of information between student partners, FF and initiative staff that allowed an efficient closing of the feedback loop through continuous analysis of, and response to student information. Results and Discussion A number of key points will help in understanding how we analysed the quantitative information from our student responses: (i) The numbers of students in first-year chemistry is approximately 350, however, the numbers responding to particular questions/discussion points are significantly smaller and can vary considerably for two main reasons. First, on alternate weeks half the groups engaged with the topics via in-class discussions and the other half via in-class online questionnaires. Second, while student attendance at the scheduled meeting was expected, participation in the questionnaires or discussions was voluntary. (ii) Some questions posed to understand the perception of a particular strand of the teaching and learning experience invited students to indicate their level of agreement with a series of statements scoring from 1 to 5 where 5 is the highest agreement and 1 the lowest agreement with the statement. Using this rubric, the maximum average score for a statement is 5/5 and the lowest is 1/5; the number and percentage of students giving each score, or a computed class average score are available. We discuss the data in terms of the average score (as X/5) and/or the percentage of students who indicated agreement over a number of categories e.g. neutral or negative (scoring 1, 2 or 3/5), or agreeing or strongly agreeing (scoring 4 or 5/5) about a statement. - 74 - (iii) Some questions invited students to tick all options that apply, in such cases we report the acceptance of the option in simple percentage terms. Impact of the initiative on development of student feedback literacy We gained two key insights into the students’ pre-university experience. Firstly, many students had had limited chances to feedback on their experience; 76% were neutral, disagreed or strongly disagreed with the statement that they “often had opportunities to give feedback to their tutors/teachers/lecturers” [2.7/5] (n=81). Secondly, the second level experience of the students also indicated varied awareness of the concepts of academic and student feedback (n=81). Against this backdrop, the students’ agreement or strong agreement with the statements that they have a clear understanding of the difference between AF and SF (79%) [4.3/5] and see the value of feedback opportunities built into the course (92%) [4.5/5] is suggestive of enhanced student feedback literacy over the course of the first-year in college (n=81), Figure 4.3. It can also be interpreted as a vote of confidence for the integrated approach. Indeed, probing students to see how they felt about having feedback opportunities built into the course elicited a response from one student that “they would be struggling a lot more without it” . Feedback Literacy: Perception of Academic and Student Feedback & Integration of Feedback 70 60 50 40 30 20 10 0 I was not aware of the concepts of AF I have a clear understanding of the It is valuable to have opportunities to & SF before I came to Maynooth difference between AF & SF provide and receive feedback built University into the course structure Strongly Disagree Disagree Neutral Agree Strongly Agree Figure 4.3. Percentage of Students indicating each level of agreement with statements relating to their prior experience, current understanding of AF and SF and their views on an integrated approach to SF (n=81). Student perception of the structure of the weekly feedback meeting There was a definite preference for the weekly meeting (64%), over a virtual suggestion box (14%), a Moodle forum (11%), a drop-in centre (9%), or a focus group (2%) as a channel for giving feedback ( Figure 4.4(a), n=81), and since only a small percentage of students disagreed, or strongly disagreed (13%) with the statement that they would attend such meetings even if attendance was not compulsory [3.6/5] (n=82) it can be surmised that the students put a high value on this activity. Most students felt the one-hour session was the appropriate length (83%), significantly a strong preference emerged for online over f2f meetings had both options been available 63 vs 37% (n=59). Attendance was good with 83% of students indicating they had attended 80% or more of the weekly feedback sessions (n=69). - 75 - Students were very strongly in favour of “in-class anonymous questionnaires/Padlets” (76%) over in-class discussion (even with the use of Padlet) (16%), or “after-class anonymous questionnaires” (8%) as inclusive methods/technologies for providing feedback ( Figure 4.4(b) n=81). (a) Preferred Communication Channel for Giving & Receiving Feedback 2% 9% 11% Scheduled Feedback Sessions Suggestion Box Moodle Forum 14% 64% Drop-in Centre Focus Group (b) What do you think are the most inclusive methods/technologies to provide feedback? 8% 16% In-class anonymous questionnaires/Padlets In-class discussions/Padlets After class anonymous questionnaires 76% Figure 4.4. Student opinion as to (a) the preferred channel and (b) preferred survey mode for giving and receiving feedback (n=81) The key attraction of Padlet was the anonymity, with one student remarking “Padlet is good as it is anonymous”. If such live polling response systems were to be rolled out more widely in future it is reassuring to note that students registered with the Maynooth Access Programme (MAP) as having a disability recorded a similar level of comfort in using Padlet [4.2/5 n=12] as those students not registered with a disability [4.8/5 n=59], Figure 4.5. - 76 - Percentage of Students Agreeing at each Level with the Statement "I found Padlet Easy to Use" 100 5 4 3 50 2 1 0 Group A Group B 1 2 3 4 5 Figure 4.5. Perception of students on the ease of use of the online polling tool Padlet: 1 = lowest and 5 = highest agreement with the statement. Group A, n=12, students registered with a disability; Group B, n=59, students not registered with a disability. Students strongly agreed the structure of the weekly feedback meeting allowed for a free and independent expression of opinion [4.2/5] with a smaller portion (26%) agreeing they were sometimes influenced by the opinions expressed by others during the session [2.8/5]. Most students felt the smaller size of the feedback sessions made them more comfortable in giving feedback [4.3/5] but it didn’t quite go far enough to give a strong sense of a community of students [3.0/5] or a connectedness to the class [2.9/5]: more students gave a neutral (~38%) than either an affirmative (~30%) or a disagreeing (~32%) responses to both these statements (n=82), Figure 4.6. Indeed, one discussion group summarised that in the coming year “lots of people looking forward to doing labs in person as well as meeting friends”. Perception of the Structure of the Weekly Feedback Meeting to Promote a Culture of Comfort and Inclusivity The feedback sessions have helped me feel part of a group/community of students I am more comfortable participating in smal er feedback sessions than in lectures/larger groups Sometimes I am influenced by others’ responses/points of view during the session I can express my own opinion freely and independently of what others may think The feedback sessions are inclusive and al ow everybody to have a voice 0 10 20 30 40 50 60 70 Strongly Disagree Disagree Neutral Agree Strongly Agree Figure 4.6. Percentage of students indicating each level of agreement with a range of statements relating to how the structure of the weekly feedback meeting facilitated inclusivity and comfort in contribution (n=82). - 77 - When asked how the structure could be made more effective some students expressed a preference for scheduling the meetings earlier rather than later in the day suggesting “not running them so late in the evenings as I feel our timetables are so full by the time it comes to extra classes in the evenings all the motivation to listen and learn and concentrate is gone”. This view was reinforced in a review of attendance numbers at the drop-in centre which showed the late afternoon, 5 pm, slot attracted only about half the number of students as the 3 pm slot. Whilst the 5 pm slot suited the timetable of ~38% of students it was the preferred slot for only 5%. For this same group the most attractive time slots were Wednesday 2-3 or 3-4 pm (17% and 16% respectively) (n=69). Student perception of the weekly meeting for receiving class level academic feedback The weekly online meeting, with an average score of [4.1/5] ranked higher as a platform for class level AF than alternative modes: f2f in the lab [3.2/5], in the form of a word document [3.1/5] or as a screencast made available on Moodle [2.5/5] (n=59). Most students [3.8/5] were neutral, agreed or strongly agreed that class level AF was usually helpful; students agreed it helped them understand the grading scheme [3.6/5], appreciate what a good report should look like [3.4/5] and to ultimately improve their grade [3.3/5] (n=59), Figure 4.7. One student reported “It is useful for getting the overall idea of how people are doing in the labs and going through issues that people had”. Perception of the Benefits of Class-Level Academic Feedback 45 40 35 30 25 20 15 10 5 0 Class-level AF is usual y It helped my understanding It helped me understand It helped my understanding It helped my grade improve helpful of the grading scheme what a good report would of the chemistry concepts look like Strongly Disagree Disagree Neutral Agree Strongly agree Figure 4.7. Percentage of students indicating their agreement at each level with statements relating to how they benefitted from class level academic feedback AF (n=59). Whilst a modest sized group of students (32%) approached the learning by making notes directly on their own returned reports, significantly more used the information gleaned from the AF to help with understanding of the concepts (64%) and to help with improving the structure of future reports (80%) ( Figure 4.8, n=69). One student indicated that they hoped to use this AF to “be made aware of errors or improvements that can be made to do better in the next lab”. - 78 - Figure 4.8. Percentage of students adopting each approach to class-level academic feedback AF (n=69). When asked what changes they would make to the AF part of the meeting one student suggested “Maybe try mentioning specific questions in the report that a lot of people lost marks on though, instead of general feedback such as structure or including units, etc”, another would like “Just if the person correcting my lab could explain how I can improve”. One student suggested “Maybe adding specific anonymous locations to ask questions that will be answered in your specific feedback group” another commented “the feedback is always helpful and I feel that its comfortable for us to ask questions”. A specific call for a review of “exam paper questions” and “organic calculations on specific rotation and optical purity” was noted in response to the question of “what would you like covered at feedback meetings”. Student perception of the reception of how their information was received and acted on: closing the feedback loop The cohort of students not agreeing/strongly disagreeing with the statement that their feedback had been heard and impacted positively on their learning experience was very small (8%). Similarly, only 15% were negative about the clarity of the communication of actions taken in response to their feedback ( Figure 4.9, n=81). One student pointed to a specific change made in response to their information “yes. the workshops changed” another remarked “Yes feedback has been taken into account both the staff and me as a student”. One student specifically told us that one of the things that worked well about the way we gathered student feedback was “letting us know what you plan on doing after the response”. A facilitator remarked that one group offered that they “liked that they could raise concerns about problems/issues they were having with the course and were happy to see their concerns being heard”. - 79 - Closing the Feedback Loop: Student Perception of the Reception of the Information Provided 45 40 35 30 25 20 15 10 5 0 Strongly Disagree Disagree Neutral Agree Strongly agree I feel my feedback has been heard and has impacted positively on my learning experience Actions taken in response to my feedback have been communicated to me clearly Figure 4.9. Percentage of students indicating each level of agreement with the statements relating to their perception of how their information was received and the impact on their learning (n=81). Selected concerns, reflections, and real-time responses/actions The students voiced a range of generally valid issues concerning their holistic, academic and structural experience. The ongoing and embedded nature of SF supported timely escalation and allowed us to reflect and respond in real time. Notwithstanding our empathy with the students not all responses were appropriate for action. For example, some concerns were dealt with through better articulation of goals and management of expectation whilst others did elicit changes in delivery/scheduling that were meaningful for the students. Selected issues are summarised in Table 4.1. Through participation in the closing the loop sessions students gained a greater understanding of our approaches/why we do things in a certain way. As an example, subsequent to addressing an early concern about structuring lab reports in a later survey 93% of students agreed/strongly agreed that they were confident in writing up a laboratory report (n=57). Table 4.1. Selected Concerns Raised by Students during Thematic Student Feedback Discussions Reflected on by Staff and Information Relayed to Students in Closing the Feedback Loop. Theme Student Concern Reflection/Action/Response: Closing the Feedback Loop Clarity and More clarity needed Selected Moodle messenger in parallel with Lecturer Communication: about commitments and Announcements as a mode of communication assignments. Find it hard to establish a Improved sign posting of activities: introduced a study routine weekly planner unclear as to what is expected - 80 - Table 4.1. continued Assignment Overwhelmed with Reviewed assignment calendar and extended Scheduling and workload, assignments, deadlines. Workload: deadlines coming Recommend study approach: work with lecture together content and associate assignment in parallel Discussed management of expectations and what is required in higher education. Pandemic More support needed to Additional resources provided: videos on lab Support for online lab classes skills/techniques, extra drop-in sessions for remote practical classes Miss the interaction with Introduced break out rooms to workshops to foster peers and learning from communication in small groups. each other Undertook to prioritise workshops as a f2f activity Report Structure Difficulty understanding Marking schemes revised, explained to students, & Marking the marking scheme included in student manual. Schemes Labs/workshop Struggling with how to Revisited in lab and the weekly meeting write up a lab report Academic Class level academic Explained how to get the most from academic Feedback AF feedback is good but feedback -this is where independent study comes in. more individual feedback Reminded students about taking responsibility for would also be good. their learning and the expectation for independent study. Provision of Solutions to workshops Explained pedagogic value of the guided-learning solutions to to be made available approach adopted. workshop problems Technology Exposure to too many Looking at capabilities of different packages for platforms was confusing chemistry teaching - a wider use of UniDoodle and but interactive software how this might help with individual/group feedback. e.g. UniDoodle, UniDoodle is a sketch-based classroom response app facilitates engagement (http://www.unidoodle.com/). Knowledge Gaps Certain concepts were Revisited content in lab and in drop-in centre not clear worked through calculations in subsequent sessions. Content Delivery A mix of synchronous Reviewed study styles and the range of learning and asynchronous resources emphasising the importance of managing delivery is too workload and ring-fencing time for independent study stressful/too time consuming. Conclusions and Implications for Future Practice We conclude that student feedback (SF) processes can be successfully integrated with mainstream continuous assessment activities in chemistry – pre-lab talks and class-level academic feedback (AF) on previously submitted lab reports. Our unique approach is effective for understanding and responding to the student experience in real-time and we believe it adaptable for any department with regular practical based activities. Key factors contributing to the success of the initiative, especially in context of the COVID-19 restrictions, included the “whole-team approach” with involvement of members from across all areas of the teaching - 81 - team, the appropriate choice of digital tools and a clear line of communication between the team members. Also critical is the choice of environment: a friendly face postgraduate student, anonymous response platforms and small group meetings allowed the collection of formal feedback in an informal setting. This study demonstrates that it is possible to integrate SF into core activities in the first-year university chemistry experience, and that the concerns raised in this transitory period can be dealt with meaningfully in real-time culminating in improvements to course organization, to aspects of teaching and learning and to better management of student expectations. We believe this experience has helped normalise feedback conversations and laid the foundation for embedment of a student feedback culture within the cohort. Limitations Considering staff time and pay for facilitators, this project was resource intensive as its scale was necessarily large. The group in focus (350 first year chemistry students) were making a particularly difficult transition to university during the COVID-19 restrictions when most teaching was remote. Readers wishing to adopt this approach in their own settings might consider working with a smaller pilot group in the first instance. Acknowledgements This project was funded by the Strategic Alignment of Teaching and Learning Enhancement Funding in Higher Education 2019 –SATLE (HEA in partnership with the National Forum for the Enhancement of Teaching and Learning in Higher Education). We acknowledge the funding and constructive discussions with the partners at Maynooth University from the Centre for Teaching and Learning (Lead), the Institutional Research Office, Maynooth Students’ Union, and the two other participating academic departments Music and Applied Social Studies; the case studies from each academic department can be accessed online: Chemistry (Heaney et al., 2021), Music (Delap et al., 2021), Applied Social Studies (Flanagan et al., 2021). Ethical considerations The Maynooth University Social Research Committee Ethics research committee approved the study, and all participants gave their informed consent. References Almendingen, K., Morseth, M. S., Gjølstad, E., Brevik, A., and Tørris, C. (2021). Student’s experiences with online teaching following COVID-19 lockdown: A mixed methods explorative study. PLOS ONE, 16(8), e0250378. Asghar, M. (2014). Staff and student experiences of dialogue days, a student engagement activity. Innovations in Education and Teaching International, 53(4), 435–444. Beltyukova, S. A., and Fox, C. M. (2002). Student satisfaction as a measure of student development: Towards a universal metric. Journal of College Student Development, 43(2), 161–172. Bovill, C., Cook-Sather, A., Felten, P., Millard, L., and Moore-Cherry, N. (2015). Addressing potential challenges in co-creating learning and teaching: overcoming resistance, navigating institutional norms and ensuring inclusivity in student–staff partnerships. Higher Education, 71(2), 195–208. Bussu, A., Detotto, C., and Serra, L. (2019). Indicators to prevent university drop-out and delayed graduation: an Italian case. Journal of Applied Research in Higher Education, 12(2), 230–249. - 82 - Cain, J., Stowe, C. D., Ali, D., and Romanelli, F. (2019). How Faculty Recognized for Teaching Excellence Interpret and Respond to Student Ratings of Teaching. American Journal of Pharmaceutical Education, 83(4), 6680–6680. Carless, D., and Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessments and Evaluation in Higher Education, 43(8), 1315–1325. Curran, J. (2021). Centre for Teaching and Learning, and Student Feedback and Teaching Evaluation Initiative Team. Maynooth University Student Feedback and Teaching Evaluation Initiative Guide: Student Feedback Matters - How and Why Student Feedback Works. M. University. Dawson, P., Henderson, M., Mahoney, P., Phillips, M., Ryan, T., Boud, D., and Molloy, E. (2019). What makes for effective feedback: staff and student perspectives. Assessment and Evaluation in Higher Education, 44(1), 25–36. De Clercq, M., Roland, N., Brunelle, M., Galand, B., and Frenay, M. (2018). The Delicate Balance to Adjustment: A Qualitative Approach of Student's Transition to the First Year at University. Psychologica. Belgica, 58(1), 67–90. de Lourdes Machado, M., Brites, R., Magalhães, A., and Sá, M. J. (2011). Satisfaction with Higher Education: critical data for student development. European Journal of Education, 46(3), 415–432. Delap, G., Murphy, E., Palmese, M., and Placanica, F. (2021). Maynooth University Student Feedback and Teaching Evaluation Initiative: Developing Student Feedback and Engagement for the Maynooth University Music Department. Flanagan, N., Bradley, C., and Kirwan, G. (2021). Maynooth University Student Feedback and Teaching Evaluation Initiative: A Culture of Feedback: Staff – Students – Peers. Flanigan, A. E., Ray, E., Titsworth, S., Hosek, A. M., and Kim, J. H. Y. (2021). Initiating and maintaining student-instructor rapport in face-to-face classes. Teaching in Higher Education, 1–20. Gakhal, S., and Wilson, C. (2019). Is students’ qualitative feedback changing, now it is online? Assessment and Evaluation in Higher Education, 44(3), 476–488. Heaney, F., Velasco-Torrijos, T., Rooney, D., Fenelon, O., Krämer, T., Dempsey, E., Barrett, S., Doherty, K., Boylan, C., and Marchetti, L. (2021). Maynooth University Student Feedback and Teaching Evaluation Initiative: Embedding a Culture of Feedback and Evaluation of Teaching and Learning In a First Year Chemistry Programme – A Whole Team Approach. Henderson, M., Phillips, M., Ryan, T., Boud, D., Dawson, P., Molloy, E., and Mahoney, P. (2019). Conditions that enable effective feedback. Higher Education Research and Development, 38(7), 1401–1416. Hodgson, P., Lam, P., and Chow, C. (2011). Assessment experience of first-year university students: dealing with the unfamiliar. Holland, E. P. (2018). Making sense of module feedback: accounting for individual behaviours in student evaluations of teaching. Assessment and Evaluation in Higher Education, 44(6), 961–972. Isaeva, R., Eisenschmidt, E., Vanari, K., and Kumpas-Lenk, K. (2020). Students’ views on dialogue: improving student engagement in the quality assurance process. Quality in Higher Education, 26(1), 80–97. Lau, V. Y. J. (2019). How to encourage student voice: obtaining effective feedback from law students in course evaluation. Legal Education Review, 29(1), 1–28. Leadbeatter, D. (2021). What is integration of learning? Teaching in Higher Education, 26(1), 1–19. - 83 - Leong, E., Mercer, A., Danczak, S. M., Kyne, S. H., and Thompson, C. D. (2021). The transition to first year chemistry: student, secondary and tertiary educator's perceptions of student preparedness. Chemical Education Research and Practice, 22(4), 923–947. Miltiadous, A., Callahan, D. L., and Schultz, M. (2020). Exploring Engagement as a Predictor of Success in the Transition to Online Learning in First Year Chemistry. Journal of Chemical Education, 97(9), 2494–2501. Mulliner, E., and Tucker, M. (2017). Feedback on feedback practice: perceptions of students and academics. Assessment and Evaluation in Higher Education, 42(2), 266–288. O’Malley, E., Scanlon, A.-M., Alpine, L., and McMahon, S. (2021). Enabling the feedback process in work-based learning: an evaluation of the 5 minute feedback form. Assessment and Evaluation in Higher Education, 46(7), 1020–1034. Okogbaa, V. (2016). Quality in higher education: the need for feedback from students. Journal of Education and Practice, 7(32), 139–143. Pienta, N. J. (2017). The Slippery Slope of Student Evaluations. Journal of Chemical Education, 94(2), 131–132. Ramachandran, R., and Rodriguez, M. C. (2020). Student Perspectives on Remote Learning in a Large Organic Chemistry Lecture Course. Journal of Chemical Education, 97(9), 2565– 2572. Ravenscroft, B., Luhanga, U., and King, B. (2017). Adapting Bangert’s online teaching effectiveness evaluation tool to a Canadian context. Innovations in Education and Teaching International, 54(4), 355–363. Richardson, J. T. E. (2005). Instruments for obtaining student feedback: a review of the literature. Assessment and Evaluation in Higher Education, 30(4), 387–415. Roddy, C., Amiet, D. L., Chung, J., Holt, C., Shaw, L., McKenzie, S., Garivaldis, F., Lodge, J. M., and Mundy, M. E. (2017). Applying Best Practice Online Learning, Teaching, and Support to Intensive Online Environments: An Integrative Review [Review]. Frontiers in Education, 2(59). Smith, K. C., & Alonso, V. (2020). Measuring student engagement in the undergraduate general chemistry laboratory. Chemical Education Research and Practice, 21(1), 399–411. Sozer, E. M., Zeybekoglu, Z., and Kaya, M. (2019). Using mid-semester course evaluation as a feedback tool for improving learning and teaching in higher education. Assessment and Evaluation in Higher Education, 44(7), 1003–1016. Tan, H. R., Chng, W. H., Chonardo, C., Ng, M. T. T., and Fung, F. M. (2020). How Chemists Achieve Active Learning Online During the COVID-19 Pandemic: Using the Community of Inquiry (CoI) Framework to Support Remote Teaching. Journal of Chemical Education, 97(9), 2512–2518. Treischl, E., and Wolbring, T. (2017). The Causal Effect of Survey Mode on Students' Evaluations of Teaching: Empirical Evidence from Three Field Experiments. Research in Higher Education, 58(8), 904–921. Watson, F. F., Castano Bishop, M., and Ferdinand-James, D. (2017). Instructional Strategies to Help Online Students Learn: Feedback from Online Students. TechTrends, 61(5), 420 – 427. - 84 - Chapter 5 Doi: 10.26529/9789612532970/ch5 Roadmap for Continuous Professional Development of STEM Lecturers Nataša Brouwer1*, Ştefania Grecea1, Johanna Kärkkäinen2, Iwona Maciejowska3, Matti Niemalä2, and Lotte Schreuders1 1University of Amsterdam, Amsterdam, The Netherlands 2University Oulu, Oulu, Finland 3Jagiellonian University, Krakow, Poland * Corresponding author: N.Brouwer-Zupancic@uva.nl Abstract Despite the ongoing systematic integration under the Bologna Agreement, higher education systems in Europe are still different in different countries and have different focus areas in the professional development of lecturers. At many European universities, professional development is often organised from a pedagogical point of view and the lecturers are left alone to apply the acquired pedagogical knowledge in their own teaching practice. In the Erasmus+ project STEM-CPD@EUni, five European universities and the European Chemistry Thematic Network (ECTN) are collaborating to enable continuous professional development (CPD) in a local university STEM teaching practice. A new concept in CPD is introduced, the CPD ambassador. Three dimensions characterize the activities of the CPD ambassadors in their local context: (1) STEM educational competences, (2) teaching attitudes, and (3) CPD activities. To define the needs for the CPD in these dimensions, a survey was developed with 66 statements evaluated from two different perspectives: their general importance for the quality of teaching and learning and their use in teaching practice. 420 lecturers from 80 universities from 26 countries and 46 education managers from 31 universities from 11 countries in Europe completed the survey from the end of November 2020 to the end of January 2021. The results show similarities and also some differences between the European countries and indicate in which directions the CPD is needed. The survey also showed that the priority list of needs for CPD should not be blindly followed but used in an evidence-based way. It is recommended to repeat the survey after some time. Based on the results of this research, a roadmap for STEM-CPD with guidelines and recommendations was developed in the STEM-CPD@EUni project. Keywords: higher education, STEM, continuous professional development (CPD), teaching competences, teaching attitudes, CPD activities Introduction Despite the ongoing systemic integration under the Bologna process (EUA, https://eua.eu) higher education systems in Europe are still very different in different countries. Many universities today recognize the need for professional development of higher education lecturers. According to the report of European University Association (te Pas, 2019) and based on results of the survey of the ECTNA Working Group “Lecturing qualifications and innovative teaching methods” (ECTN WG, 2020), some professional development is organized at many European universities. In their literature study, Stes et al. (2010) investigated the impact of institutional professional development on lecturers in higher education. In some countries, lecturers can achieve a University Teaching and Learning Certificate or a similar qualification. Yet these programs are often generic and do not focus on subject specific STEM pedagogical aspects that affect how students learn (Walsh, 2017). In general, the professional development courses are currently being developed from a pedagogical point of view and the - 85 - lecturers themselves have to apply this pedagogical knowledge in their own educational context. Teaching doesn’t get better by teaching hours alone. It is necessary that lecturers attend some professional development activities. Marsh (2007) followed student evaluations of teaching effectiveness from 195 lecturers who were continuously evaluated over 13 years and found substantial differences between individual teaching effectiveness that were also very stable over time. Educators are more likely to participate in continuous professional development activities (CPD) if they believe such programs are relevant to self-identified needs (Adu, 2017). The literature shows that teachers' beliefs about what constitutes good teaching have a strong influence on how teaching is designed and delivered in practice (Kember, 1994; Prosser, 1999, Norton, 2005). Stes et al. (2010) showed that one-off events are less effective than events that extend over time. Professional development embedded in lecturers’ daily practice (Dochy, 2011) more likely has impact on teachers’ beliefs (Rienties, 2013). Daumiller et al. (2021) studied the motivation of academics who professionalize and found a positive relationship between performance goals and learning gain. The use of information and communication technology (ICT) has revolutionized our lives and is also finding its way into higher education. The European Framework for the Digital Competence of Educators (DigCompEdu) has identified 22 competences in different areas (Redecker, 2017). To achieve a relevant use of ICT in education, the professional development of lecturers who use ICT in their teaching should focus on the development of the technological pedagogical content knowledge (TPACK) (Mishra, 2006; Rienties, 2013). The teaching context has an important role in how we teach and what we teach. In STEM professional development, lecturers develop their technological pedagogical content knowledge in the context of their courses and they use digital technology in a relevant way in specific teaching and learning activities. Bottom-up approaches where lecturers collaborate and reflect within the communities of practitioners and in this way supporting each other’s development proved to be successful (Cowan, 2006; Goodchild 2014). Lecturers in CPD are adult learners. When teaching adults, the principles of andragogy should be taken into account (Chametzky, 2014; Knowles, 1980): (a) the self-management of learning, (b) the empowerment of learners leading to increased motivation, (c) the reliance on life experiences of learners to aid with their learning, (d) the objectives of learners for taking the course, and (e) the practical, real-world solutions to problems encountered in the course. The principle of autonomous learning supports learning activities that take place in close connection with one's own workplace, in the case of lecturers their teaching practice (Dochy, 2011). Sustainable CPD activities for lecturers enable lifelong learning educators to continuously improve their teaching skills while supporting continuous improvement in the quality of the courses and curricula in which they teach. Ultimately, it enables to improve the learning process of the students and shape their skills for the future. In this article, we describe the development and results of a survey aimed at determining the CPD needs of university STEM educators in Europe. The results presented in this article reveal what knowledge and skills the participants of the survey considered important for the university STEM lecturers at the time of completing the survey. The results suggest directions for organization of continuous professional development. Nevertheless, we recommend that new measurements will be conducted every few years. CPD-Ambassador Most universities have professional development (PD) policies and organize top-down courses or a PD programme for their lecturers. In every faculty there are also lecturers who have a deeper interest in education and who explore new teaching methods in their courses or new digital tools. They innovate their teaching from intrinsic motivation, bottom-up. There are - 86 - university or national funds to support education innovation projects and there are also European funds, like Erasmus+. These bottom-up activities lead to the emergence of networks of lecturers innovators and long-term collaborations. Many lecturers innovators like explaining informally to their fellow lecturers how they use a new teaching method or help colleagues in the workplace using a new teaching tool. Unfortunately, these bottom-up efforts are usually not recognized by faculty management as important teaching duties or as part of the professional development. To the best of our knowledge, there is no known structured bottom-up continuous professionalization approach that is organized by the lecturers themselves. To drive a change, the STEM-CPD@EUni project has introduced a new concept in CPD, the CPD-ambassador (Brouwer, 2020). The role of the CPD-ambassadors is to increase the awareness of fellow lecturers on the importance of the STEM teaching competences and to promote professional development. The CPD-ambassadors organize bottom-up CPD activities for fellow lecturers that exactly match their specific needs so that they can immediately apply the knowledge and solve their educational challenges. This is important for motivation and self-regulated learning. Finding out what lecturers need is thus a fundamental step in organizing bottom-up CPD activities. The CPD-ambassadors from different universities and from different countries are united in a STEM-CPD community where they share their knowledge and exchange experiences. To become CPD-ambassadors at the European level, lecturers can participate in a Summer School for STEM-CPD (https://ectn.eu/work-groups/stem-cpd-o5/). Method To map out the needs for the professional development of lecturers in STEM in Europe, a survey was developed (STEM-CPD@EUni, https://ectn.eu/work-groups/stem-cpd/). A clustering method was used to define reliability of the survey scales. Based on the results of this survey, a Roadmap for continuous professional development of STEM lecturers was published with recommendations and guidelines for organizing meaningful CPD activities (Grecea, 2021). Survey The survey development team consisted of a core group of four persons: three STEM lecturers, two of whom also had management tasks, and one STEM education consultant, and a feedback group of 16 persons: 15 STEM lecturers, several of whom also had management tasks, and one educational expert. In order to collect feedback on the content and design of the survey, various brainstorming sessions were organized with the core development group and feedback group upon which the individual feedback was given. The final survey resulted in 66 statements, divided into three parts, each measuring a different dimension that the development team defined as relevant to the bottom-up professional development of STEM lecturers ( Figure 5.1): (1) STEM teaching competence, (2) CPD teaching and learning (CPD) attitudes and (3) CPD activities. - 87 - Figure 5.1. Dimensions that characterize a CPD-Ambassador in his/her activities to improve local teaching and learning practice The survey was anonymous but there were several demographic questions in order to be able to analyze the results. All participants gave consent to use their data for the research and publication purposes. The statements of the survey can be found in different tables. The participants of the survey were lecturers or educational managers. They evaluated each of the statements on the Likert scale 1 to 5 from two different perspectives. Perspective 1: General importance for the quality of teaching and learning in university STEM Perspective 2: Use / practice in the personal teaching practice (participants lecturers) or in the programme teaching practice (participants managers) The higher the general importance value of a statement in the survey, the greater its relevance to organizing CPD activities for it. The discrepancy in the value of the two perspectives, the general importance and the personal/program use measures the urgency to organize the CPD activities for that. The urgency is greater when greater the discrepancy between the general importance and the personal/programme use. The survey was set out first at the partner institutions. Based on the results and gained experiences at this stage, we have decided to keep the survey in the same form and we distributed it further among the lecturers and managers at other universities in Europe. Results and discussion The survey was set out between 24th November 2020 and 20th December 2020 at the partner institutions. 94 lecturers and 16 managers completed the survey in this period. Then, the same survey was distributed among lecturers and managers at other universities in Europe. 326 lecturers and 30 managers completed the survey from 21st December 2020 and 30th January 2021. To examine if the data sets could be pooled, we have executed a quadratic Levene’s test on all 66 statements in the survey and found no significant differences between the November-December and December-January sets for both the lecturers data and the managers data. Based on this outcome, we have decided to use the whole set of 420 completed surveys for the analysis - 88 - of the survey of lecturers and the whole set of 46 surveys of managers and worked it out together in order to define any difference per country. In the following paragraphs we define the participants and discuss the results per part of the survey and compare the results of the survey completed by the lecturers and the educational managers. Participants A total of 420 lecturers from 26 countries and 46 education managers from 11 countries in Europe (geographically) have completed the survey between 24th November 2020 and 30th January 2021 ( Table 5.1). Table 5.1. Countries of participants in the survey. Country No. Lecturers No. Managers Italy 249 14 Poland 35 9 Netherlands 24 6 Finland 11 1 Spain 11 1 Slovenia 8 2 Belgium 7 6 Cyprus, Hungary, Serbia, Turkey 7 0 United Kingdom 6 0 Greece 5 0 Czech Republic, Latvia 4 1 Austria, Romania, Russia 4 0 Germany 3 4 France, Ireland, Rep. of North Macedonia 3 0 Lithuania 1 1 Croatia, Portugal, Slovakia 1 0 Total 420 46 From 420 lecturers 58% were male, 40% female, 2% did not want to answer this question. The largest group (33%) was between 46-55 years old, 25% were between 36-45 years old, 23% were between 56-65 years old, 9% were older than 65 and 2 persons (0.4%) were younger than 25 years old. 53% of the participants did not have any pedagogical training yet and 26% have obtained some university teaching certificate. For 56% of the participants teaching duties were about 50% of their work tasks and for 25% teaching was a substantial part of their work tasks. 26% of the participants’ role was lecturing, 14% laboratory teaching and 12% were teaching in tutorial sessions / working sessions / seminars. 12% of the participating lecturers indicated that they are not yet effective enough and need training, 64% of the participants indicated that they are effective teachers and 46% that they are not always effective. 69% of participants said they are effective lecturers, but they still need training because there is always room for improvement. 9% said they don't have time for training. - 89 - Survey Part 1 The set of statements in Part 1 has 30 statements dealing with different educational principles, methods, pedagogical content and technology ( Table 5.2). Table 5.2. Statements survey Part 1: Teaching competences. No Statement 1 frame the course in the context of the study programme 2 define intended learning outcomes in every course they teach 3 choose an appropriate assessment method for their course 4 engage students and arouse interest for the discipline in the class 5 teach holistically by integrating social and art aspects in teaching and learning complex chemical concepts 6 cope with heterogeneous pre-knowledge of students 7 being able to bring out and correct misconceptions 8 develop critical thinking by students 9 give prompt feedback and support students during learning 10 support students in socializing (specifically e.g. during a pandemic) 11 stimulate discussion 12 design laboratory courses 13 teach about lab safety using digital tools/platform (where appropriate) 14 teach large groups of students 15 teach small groups of students (group's dynamics) 16 design interactive lectures 17 design online exams 18 design problem solving sessions 19 design active learning classes / sessions using digital technology 20 use digital tools in lab courses 21 use design thinking methods 22 use research based teaching methods 23 use project based teaching methods 24 use blended learning approach 25 use interactive online boards for teaching and learning 26 use voting in lectures to activate thinking and understanding of (e.g. chemistry) concepts 27 organize peer-assessment / peer-feedback in their courses 28 organize (online) collaborative learning 29 use advanced tools, based on artificial intelligence, in supporting students in their learning process 30 make/produce short MOOCs Figure 5.2. shows the results of the survey Part 1 completed by 420 lecturers from 26 different countries about which teaching competences lecturers need to have, evaluated from two perspectives: the general importance and personal use, i.e. how much they apply these competences in their personal practice. - 90 - Figure 5.2. Results of the survey Part 1 completed by lecturers from 26 countries regarding the general importance of teaching competences in comparison to their personal practice (evaluated on the Likert scale 1 to 5) - n= 420. In Figure 5.3 the same results are presented for the group of 46 education managers from 11 countries. Figure 5.3. Results of the survey Part 1 completed by education managers from 11 countries regarding the general importance of teaching competences in comparison to the practice of their programmes / their lecturers teams (evaluated on the Likert scale 1 to 5) - n= 46. - 91 - The results show that in all cases the general importance has a higher value than the personal or programme practice use. The education managers find all 30 competences on average important (Likert scale > 3.4). On the other hand the lecturers find the competences 29 (use advanced tools, based on artificial intelligence, in supporting students in their learning process) and 30 (make/produce short MOOCs) on average less important (Likert scale < 3.0) The results show that the General importance is larger than the Personal/programme practice use in all cases except for question 14 (teach large groups of students) in the survey lecturers, which means that discrepancy, the average general importance - average personal/program use D(G – P) is positive ( Figure 5.2 and 5.3). Table 5.3 shows the priority list of teaching competences. The four most important teaching competences in the opinion of the lecturers and the education managers are the same, q. 8, q. 4, q. 9 and q. 2 ( Table 5.3). In addition, Table 5.3 also shows the discrepancy between the general importance and the personal/programme practice use, which is larger in the group of the education managers compared to the group of lecturers in de survey questions no. 8, 4, and 9. This suggests that the experience of the personal situation of the lecturers for those questions is more positive than the experience of the education managers about their programme teams. Table 5.3. Priority list of competences based on the survey data Part 1. Lecturers Educ. Managers Gen.Imp. St. D(G-P)* Gen.Imp. St. D(G-P)* q. Dev. Dev. No Statement 4.71 0.63 0.24 4.65 0.74 0.54 8 develop critical thinking by students 4.60 0.70 0.19 4.53 0.79 0.36 4 engage students and arouse interest for the discipline in the class 4.48 0.73 0.14 4.48 0.86 0.46 9 give prompt feedback and support students during learning 4.42 0.72 0.13 4.52 0.78 0.39 1 frame the course in the context of the study programme 4.41 0.83 0.11 4.57 0.78 0.32 2 define intended learning outcomes in every course they teach 4.37 0.80 0.19 4.35 0.90 0.53 7 being able to bring out and correct misconceptions 4.36 0.79 0.16 4.50 0.81 0.37 3 choose an appropriate assessment method for their course 4.31 0.89 0.30 4.36 0.84 0.22 12 design laboratory courses 4.30 0.84 0.28 4.11 0.99 0.30 11 stimulate discussion 4.21 0.91 0.20 4.20 0.98 0.26 15 teach small groups of students (group's dynamics) 4.12 0.91 0.48 4.33 0.83 0.62 18 design problem solving sessions 4.09 0.94 0.33 4.37 0.93 0.48 22 use research based teaching methods - 92 - Table 5.3. continued 3.96 0.91 0.16 4.09 0.89 0.46 6 cope with heterogeneous pre- knowledge of students 3.93 0.98 0.45 4.07 0.90 0.53 16 design interactive lectures 3.91 1.04 0.56 3.93 1.20 0.39 13 teach about lab safety using digital tools/platform (where appropriate) 3.74 1.04 0.33 4.04 1.02 0.37 23 use project based teaching methods 3.70 1.13 0.34 3.69 1.08 0.51 20 use digital tools in lab courses 3.68 1.07 0.46 4.00 0.99 0.57 19 design active learning classes / sessions using digital technology 3.64 1.05 0.46 3.80 1.11 0.59 21 use design thinking methods 3.57 1.13 0.27 3.82 1.07 0.31 24 use blended learning approach 3.54 1.11 0.35 3.63 1.10 0.36 10 support students in socializing (specifically e.g. during a pandemic) 3.46 1.14 0.17 3.58 1.05 0.41 5 teach holistically by integrating social and art aspects in teaching and learning complex chemical concepts 3.45 1.16 0.01 3.58 1.22 0.08 17 design online exams 3.42 1.18 0.51 3.67 1.12 0.61 28 organize (online) collaborative learning 3.42 1.18 0.33 3.51 0.99 0.36 25 use interactive online boards for teaching and learning 3.31 1.13 0.50 3.72 1.17 0.58 27 organize peer-assessment / peer-feedback in their courses 3.20 1.19 -0.05 3.62 1.19 0.16 14 teach large groups of students 3.08 1.22 0.40 3.53 1.06 0.60 26 use voting in lectures to activate thinking and understanding of (e.g. chemistry) concepts 2.88 1.18 0.54 3.42 1.02 0.89 30 make/produce short MOOCs 2.81 1.30 0.70 3.41 1.26 0.95 29 use advanced tools, based on artificial intelligence, in supporting students in their learning process *(G-P) = average general importance - average personal/program use In Figure 5.4, the general importance of the five project partner countries of the STEM-CPD@EUni project (the Netherlands, Poland, Italy, Finland and Slovenia) are compared to the average general importance of all the countries that have contributed to the survey. In general, the Figure 5.4 shows many similarities and some differences between the countries. First, when we exclude the largest group (Italy) the competence “Develop critical thinking by students” (q. 8) remains the competence with the highest general importance. The figure shows that in Poland, the competence “Teaching small groups of students (group dynamics)” (q. 15) is the second most important competence, which is different from other countries. Furthermore, - 93 - in Poland the third place shows two competences having the same score (4.6, St. Dev. 0.6 and 0.7 respectively): “Engage students and arouse interest for the discipline in the class” (q. 4) and “Define intended learning outcomes in every course they teach” (q. 2). The education managers in Poland also ranked the competence “Teaching small groups of students (group dynamics)” (q. 15) second most important. and on the third place there are two competences having the same score (4.6, St. Dev. 0.6 and 0.7 respectively) “Engage students and arouse interest for the discipline in the class” (q. 4) and “Define intended learning outcomes in every course they teach” (q. 2). The education managers in Poland also put q.15 on the second place. The lecturers and the education managers in Poland assigned the competence “Teach holistically by integrating social and art aspects in teaching and learning complex chemical concepts” (q.5) a higher general importance value than average in all countries (Lecturers: 3.97, St. Dev. 0.95, all countries 3.46 St. Dev. 1.14; Managers: 4.25, St. Dev. 0.71, all countries 3.58, St. Dev. 1.05). - 94 - Figure 5.4. Results of the survey Part 1 completed by lecturers from 5 partner countries involved in the STEM-CPD@EUni project regarding the general importance of teaching competences (evaluated on the Likert scale 1 to 5) - n= 327. 95 The lecturers who completed the survey in Finland were mostly from the University Oulu and teach courses with large groups of students. They assigned the highest score for the competence q.2 (4.91, St. Dev. 0.30) and for the competence “Teach large groups of students” (q.14) a much higher general importance score (4.45, St. Dev. 0.82) than lecturers in other countries (3.20, St. Dev. 1.19). The general importance scores for the competences “Design interactive lectures” (q.16) (4.45, St. Dev. 0.82, average all countries 3.93 St. Dev. 0.98) and “Design online exams” (q.17) (4.27, St. Dev. 0.79, average all countries 3.45, St. Dev. 1.16) are on average also higher in Finland than in other countries. The results from specific countries are likely to be related to the organizational culture of their higher education institutions, traditions, and legal regulations. Indication of teaching in large student groups as a less important competence in higher education, may be related to education organisation in that country (Maciejowska, 2022). For example in Poland, courses are organized for each department separately, and therefore with a limited number of students from a few (astronomy) to approx. 200 (computer science) and there are no join lectures on basic STEM courses for really large groups of students (e.g. general chemistry for chemistry, biology, pharmacy, forestry study programmes). Another reason may be the common, traditional, also expected by students, way of lecturing without interaction with the audience. We have combined the competence statements in Part 1 in four larger education competences and sub-competences considering general pedagogical principles, constructive alignment and TPACK model (Biggs, 2011; Mishra, 2006). We have statistically evaluated these competence scales and subscales by using a statistical clustering method. P1-1 Constructive alignment (q. 1, 2, 3, 6) P1-2 Pedagogy, Interactive teaching P1-2a Competence teaching (q. 9, 10, 14, 15) P1-2b Competence design interactive teaching (q. 16, 19) P1-3 Pedagogy, Learning facilitation P1-3a Problem solving (design and teaching) (q. 18, 21, 22, 23) P1-3b Engagement and motivation, facilitation discipline specific learning (q. 4, 12, 13) P1-3c Deep learning (q. 5, 7, 8, 11) P1-3d Organize peer-feedback, collaborative learning (q. 27, 28) P1-4 Technology in facilitative teaching: P1-4a Use of digital tools for a pedagogical goal (q. 17, 25, 26, 29, 30) P1-4b Blended learning (q. 20, 24) Table 5.4 gives the values of the scales and subscales calculated from the average score values of the statements in the survey answered by both lecturers and education managers. The reliability of the scales and sub-scales is given by the Cronbach’s Alpha. The reliability Cronbach’s Alpha for the whole Part 1 (30 items) of the survey is 0.951 for General importance and 0.945 for Personal/programme use. The cluster analysis supports the sets of statements in the education competences. An exception is “Teach large groups of students” (q.14). Excluding the question q.14 from the subscale Competence Teaching (P1-2 Pedagogy- Interactive teaching) increases the value of this subscale from 3.86 to 4.07 in the lecturers group and from 3.98 to 4.10 in the managers group. The reliability Cronbach’s Alpha increases when q.14 is excluded from the subscale from 0.558 to 0.648 in general importance part and from 0.473 to 0.577 in the personal/programme part. The reason why q.14 doesn't fit well in the subscale may be the usual traditional way of teaching large groups of students without interaction. More research is needed to prove this. 96 Table 5.4. Values of the scales / subscales that measure the importance of four general education competences for the lecturers in STEM, in the opinion of the lecturers and the education managers. Scales Lecturers Lecturers Managers Managers Cronbach’s General D(G-P)a) General D(G-P)b) Alpha P1-1 G: 0.752 Constructive alignment 4.29 0.14 4.42 0.38 P: 0.718 (q. 1, 2, 3, 6 ) P1-2 Pedagogy - Interactive teaching P1-2a G: 0.558 Competence Teaching (q. 3.86 0.12 3.98 0.31 P: 0.473 9, 10, 14, 15 P1-2b G: 0.746 Competence Design 3.81 0.46 4.03 0.55 P: 0.735 interactive teaching (q.16, 19) P1-3 Pedagogy - Learning facilitation P1-3a G: 0.783 Problem solving (design 3.90 0.40 4.14 0.51 P: 0.753 and teaching) (q. 18, 21, 22, 23) P1-3b G: 0.646 Engagement and P: 0.569 motivation, facilitation 4.27 0.35 4.28 0.33 discipline specific learning (q. 4, 12, 13) P1-3c G: 0.705 Deep learningc) (q. 5, 7, 8, 4.21 0.22 4.17 0.45 P: 0.709 11) P1-3d G: 0.724 Organize peer-feedback, 3.37 0.50 3.70 0.59 P: 0.754 collaborative learning (q. 27, 28) P1-4 Technology in facilitative teaching P1-4a G: 0.820 Use of digital tools for a 3.13 0.40 3.49 0.58 P: 0.788 pedagogical goal (q. 17, 25, 26, 29, 30 ) P1-4b G: 0.572 Blended learning 3.63 0.30 3.76 0.41 P: 0.581 20, 24 a)The difference between the general importance for the lecturer and the use in the personal teaching practice b)The difference between the general importance for the education managers and the use in the programme teaching practice c)According to the definition of Biggs (2011). 97 The scales P1-1 (Constructive alignment) 4.29 (group lecturers) and 4.42 (group education managers) clearly show that both groups are aware of the importance of the Constructive alignment (Biggs, 2011) in the course design. Both subscales of P1-2 (Pedagogy - Interactive teaching), Competence teaching and Competence Design interactive teaching in the lecturers’ survey are under 4.00 (agree on Likert scale). In the group of education managers this is only slightly higher. Only the subscales P1-3b and P1-3c ( Table 5.4) of the scale P1-3 Pedagogy - Learning facilitation is higher than 4.00 in both groups. These results show that, at the moment of the survey the interactive activating teaching methods were not perceived as relevant for the higher education lecturers by both the majority of participating lecturers and education managers. This is in sharp contrast with the extensive research in this field (Freeman, 2014) that recommends omitting traditional lecturing because interactive activating teaching works better. The value of both subscales of P1-4 (Technology in facilitative teaching) are for both groups lower than 4.00. This indicates that the majority of the lecturers and education managers did not find the use of digital technology in enabling education relevant for higher education lecturers at the time of the survey, which was before the outbreak of COVID19 in Europe. It is possible that a repeat of the study now turns out differently. Survey Part 2 The survey Part 2 includes 17 statements about the attitudes considering different educational principles, pedagogies, content and technology. The list of statements used in the Survey Part 2 can be found in Table 5.5. Table 5.5. Statements survey Part 2: Teaching and learning attitudes/characteristics. No Statement 1 be reflective teachers and reflect about their courses / lectures. 2 have high expectations for the students and themselves. 3 inspire a positive attitude in their class. 4 make students feel special, included, safe and secure. 5 be interested in their students' progress. 6 use students evaluations and the feedback of students to improve courses. 7 read literature about teaching and learning in higher education. 8 discuss teaching with their colleagues. 9 observe (some) lectures / teaching sessions of colleagues and give feedback. 10 record (some) own lectures / teaching sessions on the video to reflect on. 11 organize / attend meetings of their own teaching team to discuss / reflect on the teaching methods and on the effect of those on students’ learning. 12 share experience and knowledge gained through continuous professional development (CPD) with lecturers from other institutions. 13 analyse the effect of teaching and introduce changes in an evidence based way. 14 set their own goals for professional development. 15 attend training for lecturers at the university. 16 apply for specific professional development programmes to obtain certificate(s) in teaching. (If this doesn't exist in your country, please indicate in General importance what is your personal opinion about it and choose in Personal practice not applicable) 17 participate in conferences about teaching in higher education. 98 Figure 5.5 shows the results of the survey Part 2 completed by 420 lecturers from 26 countries regarding which attitudes (characteristics) lecturers should have evaluated from two perspectives: the general importance and how much they experience a specific attitude in their personal practice. Figure 5.6 shows these results collected from 46 education managers from 11 countries and compares their two perspectives: general importance and presence in their programme practice. The results also show that in all cases the general importance is evaluated higher than the real situation in practice. This means that there is urgency for the professionalization of lecturers in the dimension of teaching and learning attitudes ( Figure 5.1). Figure 5.5. Results of the survey Part 2 completed by lecturers from 26 countries regarding the general importance of teaching and learning attitudes in comparison to their personal practice (evaluated on the Likert scale 1 to 5) - n= 420. 99 Figure 5.6. Results of the survey Part 2 completed by education managers from 11 countries regarding the general importance of teaching and learning attitudes in comparison to the practice of their programmes / their lecturers teams (evaluated on the Likert scale 1 to 5) - n= 46. Figure 5.7 shows the results for 5 countries of the project partner universities. Figure 5.7. Results of the survey Part 2 completed by lecturers from 5 partner countries involved in the STEM-CPD@EUni project regarding the general importance of teaching and learning attitudes (evaluated on the Likert scale 1 to 5) - n= 327. 100 Table 5.6 shows the priority list of attitudes (personal characteristics). The four most important teaching and learning attitudes in the opinion of the lecturers and the education managers, measured on the Likert scale 1 to 5 are q. 3., q. 1, q. 6, and q. 5 ( Table 5.6). Similarly to Part 1 the participants of the survey (lecturers or education managers) evaluated each statement from two perspectives: general importance and use in personal or programme practice. Table 5.6. Priority list of attitudes (personal characteristics) based on the survey data Part 2. Lecturers Educ. managers Gen.Imp. St. D(G-P)* Gen.Imp. St. Dev. D(G-P)* q. No Statement Dev. 4.58 0.71 0.16 4.44 0.87 0.31 3 inspire a positive attitude in their class. 4.53 0.75 0.17 4.41 0.91 0.35 1 be reflective teachers and reflect about their courses / lectures. 4.45 0.85 0.15 4.54 0.91 0.28 6 use students evaluations and the feedback of students to improve courses. 4.43 0.80 0.15 4.39 0.88 0.43 5 be interested in their students' progress. 4.18 0.86 0.31 4.17 0.95 0.48 8 discuss teaching with their colleagues. 4.12 1.00 0.60 4.15 0.94 0.78 13 analyse the effect of teaching and introduce changes in an evidence based way. 4.04 1.07 0.15 4.28 0.96 0.37 4 make students feel special, included, safe and secure. 4.01 0.93 0.06 4.13 1.04 0.29 2 have high expectations for the students and themselves. 3.87 1.04 0.37 4.11 0.97 0.80 14 set their own goals for professional development. 3.82 1.03 0.56 3.85 1.03 0.83 7 read literature about teaching and learning in higher education. 3.81 1.12 0.72 4.02 0.91 0.67 15 attend training for lecturers at the university. 3.65 1.11 0.93 3.76 1.03 0.89 12 share experience and knowledge gained through continuous professional development (CPD) with lecturers from other institutions. 3.65 1.02 0.97 3.78 1.01 0.89 9 observe (some) lectures / teaching sessions of colleagues and give feedback. 101 Table 5.6. continued 3.64 1.09 0.78 3.80 1.07 0.85 11 organize / attend meetings of their own teaching team to discuss / reflect on the teaching methods and on the effect of those on students' learning. 3.37 1.30 0.56 3.75 1.08 0.33 16 apply for specific professional development programmes to obtain certificate(s) in teaching. (If this doesn't exist in your country, please indicate in General importance what is your personal opinion about it and choose in Personal practice not applicable) 3.35 1.18 0.59 3.33 1.13 0.62 10 record (some) own lectures / teaching sessions on the video to reflect on. 3.28 1.25 0.71 3.29 1.14 0.67 17 participate in conferences about teaching in higher education. *difference between general importance G and personal use (lecturers) / programme use (managers) P We have combined the teaching and learning attitudes statements in Part 2 (Table 5.5) and assigned five larger educational attitudes that are based on general pedagogical principles about learning and motivation, teachers’ beliefs, adult learning and CPD: P2-1 Motivation and self-regulation for CPD (q. 2, 14, 15, 16) P2-2 Pastoral interest (q. 3, 4, 5) P2-3 Reflection (q. 1, 10, 11) P2-4 Evidence informed approach (q. 6, 7, 13) P2-5 Knowledge sharing (q. 8, 9, 12, 17) In Table 5.7, the scales are calculated based on the survey results of lecturers and education managers. Table 5.7. Values of the scales that measure the importance of teaching and learning attitude of the STEM lecturers, in the opinion of the lecturers and the education managers. Scales Lecturers Lecturers Managers Managers Cronbach’s General D(G-P) General D(G-P) Alpha P2-1 G: 0.771 Motivation and Self- 3.95 0.43 4.00 0.52 P: 0.752 regulation (CPD) (q. 2, 14, 15, 16) P2-2 G: 0.829 Pastoral interest (q. 3, 4, 4.35 0.15 4.37 0.37 P: 0.812 5) 102 Table 5.7. continued P2-3 G: 0.688 Reflection (q. 1, 10, 11) 3.84 0.51 3.85 0.61 P: 0.572 P2-4 G: 0.710 Evidence informed 4.13 0.43 4.18 0.63 P: 0.657 approach (q. 6, 7, 13) P2-5 G: 0.801 Knowledge sharing (q. 8, 3.69 0.73 3.75 0.73 P: 0.801 9, 12, 17) Table 5.7 shows similar values of the scales given by the lecturers and managers. The discrepancy Δ(G-P) in the score for general importance and personal / programme use of these scales is larger in the group of managers than in the group of lecturers. It can be concluded that the education managers are less positive about the situation of teaching practice in their programme than the lecturers. It is remarkable that the scales P2-3 Reflection and P2-5 Knowledge sharing in both groups are lower than 4.00 and are thus, perceived by the majority of lecturers and education managers as not important elements in continuous professional development of lecturers. Survey Part 3 Part 3 of the survey includes 19 statements dealing with the type of preferred professional development activities. The list of statements in the survey Part 3 can be found in Table 5.8. Table 5.8. Statements survey Part 3: Professional development activities. No Statements 1 reading books / journal articles on teaching and learning in HE. 2 attending presentations about teaching approaches. 3 attending webinars about teaching and learning. 4 attending hands-on workshops on specific continuous professional development (CPD) topics. 5 following online courses / MOOC about teaching and learning. 6 attending conferences on teaching and learning in HE. 7 attending a summer school on teaching and learning. 8 attending a professional development programme to get a teaching certificate in higher education* 9 attending workshops that are organized specifically for STEM lecturers. 10 attending workshops that are organized generally for lecturers from different disciplines. 11 collaborating with a peer-lecturer on a redesign of a course. 12 getting peer-feedback on own teaching practice from a colleague. 13 collaborating on a teaching innovation project. 14 getting personal coaching / support by a pedagogical expert. 15 getting mentoring from an experienced colleague. 16 getting just-in-time support on a specific teaching and learning issue. 17 giving mentoring to a junior lecturer. 18 giving workshops to other lecturers. 19 participating in a teaching and learning network or a special interest group on teaching and learning in HE. 103 *if there existed no progamme to achieve a teaching certificate in higher education in the country, the participants were requested to only indicate their opinion about the General importance and to choose not applicable in the Personal practice perspective. Figure 5.8 shows the results of the survey Part 3 completed by 420 lecturers from 26 different countries ( Table 5.1) about which professional development activities generally work well (general importance) and which activities they experience in their personal practice. Figure 5.9 shows these results collected from 46 education managers from 11 countries and compares their two perspectives: general importance and experience in their programme practice. Figure 5.8. Results of the survey Part 3 completed by lecturers from 26 countries regarding the general importance of CPD activities in comparison to their personal experience with these CPD activities (evaluated on the Likert scale 1 to 5) - n= 420. 104 Figure 5.9. Results of the survey Part 3 completed by education managers from 11 countries regarding the general importance of CPD activities in comparison to the experience in the practice of their programmes / their lecturers teams (evaluated on the Likert scale 1 to 5) - n= 46. Figure 5.10 shows the data from the survey Part 3 from the five countries of the universities that are partners in the STEM-CPD@EUni project. 105 Figure 5.10. Results of the survey Part 3 completed by lecturers from 5 partner countries involved in the STEM-CPD@EUni project regarding the general importance of professional development activities (evaluated on the Likert scale 1 to 5) - n= 327. Table 5.9 summarizes the CPD activities, that on average are recognized to work best, measured on the Likert scale 1 to 5 according to their relevance for the lecturers. Table 5.9. Priority list of professional development activities that work best according to lecturers and education managers based on survey data Part 3. Lecturers Educ. Managers Gen.Imp. St. D(G-P)* Gen.Imp St. Dev. D(G-P)* q. Statement Dev. . No 3.80 1.04 0.72 3.93 1.21 0.84 17 giving mentoring to a junior lecturer. 3.74 1.04 0.82 3.76 1.14 1.07 12 getting peer-feedback on own teaching practice from a colleague. 3.71 1.10 0.85 3.80 1.18 0.96 15 getting mentoring from an experienced colleague. 3.67 1.04 0.79 3.78 0.99 0.72 2 attending presentations about teaching approaches. 3.63 1.14 0.61 3.54 0.96 0.68 1 reading books / journal articles on teaching and learning in HE. 3.61 1.17 0.79 3.69 1.08 0.87 13 collaborating on a teaching innovation project. 106 Table 5.9. continued 3.57 1.12 0.80 3.78 1.04 0.96 11 collaborating with a peer- lecturer on a redesign of a course. 3.54 1.13 1.13 3.73 1.23 0.90 9 attending workshops that are organized specifically for STEM lecturers. 3.49 1.17 0.89 3.73 0.99 0.73 4 attending hands-on workshops on specific continuous professional development (CPD) topics. 3.48 1.09 0.80 3.52 1.01 0.72 3 attending webinars about teaching and learning. 3.36 1.17 0.80 3.68 1.25 0.82 16 getting just-in-time support on a specific teaching and learning issue. 3.30 1.22 0.83 3.54 1.13 1.00 19 participating in a teaching and learning network or a special interest group on teaching and learning in HE. 3.23 1.22 0.88 3.52 1.22 0.94 18 giving workshops to other lecturers. 3.19 1.31 0.70 3.47 1.30 0.55 8 attending a professional development programme to get a teaching certificate in higher education (if it doesn't exist in your country, please indicate in General importance what is your personal opinion about it and choose in Personal practice not applicable). 3.19 1.25 1.07 3.36 1.23 1.11 14 getting personal coaching / support by a pedagogical expert. 3.17 1.19 0.82 3.13 1.20 0.71 6 attending conferences on teaching and learning in HE. 3.15 1.19 0.96 3.11 1.02 0.82 5 following online courses / MOOC about teaching and learning. 3.10 1.20 1.06 3.20 1.25 0.87 7 attending a summer school on teaching and learning. 3.01 1.17 0.78 3.11 1.32 0.72 10 attending workshops that are organized generally for lecturers from different disciplines. *difference between general importance G and personal use (lecturers) / programme use P (managers) Table 5.9 shows that the average values of general importance of CPD activities are all lower than 4.0 (agree) on the Likert scale and this is lower than the highest obtained average values in Part 1 and Part 2 of this survey. The scattering of the answers (standard deviations) in Part 3 107 is high and it is higher than in Part 1 and 2. The discrepancies D(G-P) in scores for general importance and personal / programme practice are larger than in Parts 1 and 2. Moreover, some interesting observations on the level of the countries can be discussed. First, part of the statements in some countries were evaluated with 4.0 (agree) or higher. For example the statement “Giving mentoring to a junior lecturer” (q.17) in Poland and in Finland scores higher than 4.0 (Finland: 4.09, St. Dev. 0.83; Poland: 4.06, St. Dev. 0.81). In Finland, q.17 is not the activity with the highest score but there are two other statements in Part 3 that score in Finland higher than q.17. Q.15: “Getting mentoring from an experienced colleague” score with the score 4.27 (St. Dev. 0.79) and q.8: “Attending a professional development programme to get a teaching certificate in higher education” with the score 4.27 (St. Dev. 0.90). In the Netherlands q.8 has the highest score of all statements (3.92, St. Dev. 1.10, average all countries 3.19, St. Dev. 1.31). Finland and the Netherlands are two countries where lecturers can get the University teaching certificate. In the Netherlands there is also a national framework for University teaching qualification (de Groot, 2018). “Following online courses / MOOCs about teaching and learning” (q.5) is most appreciated in Finland, Slovenia and Poland (Figure 5.10). From the perspective of the personal practice and the programme practice and taking into account all countries in the survey only q.1 (3.01, St. Dev. 1.37) and q.17 (3.08, St. Dev. 1.36) are larger than 3.00. On the other hand, there are substantial differences between the countries. For example in Italy, in personal experience, all scores are below 2.9, in Poland only q.1 (reading books / journal articles on teaching and learning in HE) scores higher than 3.5 (3.56, St. Dev. 1.19), in Slovenia two statements, q.2 and q.3 score higher than 3.5 (3.56, St. Dev. 1.01 and 3.78, St. Dev. 1.20 respectively) while in the Netherlands these are the statements q.8 and q.11 (3.77, St. Dev. 1.45 and 3.68, St. Dev. 1.25). Last but not least in Finland 6 statements have a higher score than 3.5 among which the statement q.8 has the score 4.10 (St. Dev. 1.29). As suggested before, the differences are likely to be related to the legal regulations, traditions, and organizational culture of their higher education institutions. Polish lecturers indicated creating MOOC courses as a minor competence, probably because in Poland a legal regulation of the status of such courses in study programs is unclear (Maciejowska, 2022). We believe that this means that the frequency of professional development activities is currently not very high on average, nor may there be much variation in the types of professional development activities. Low values in personal / program experience and large discrepancies between general importance and personal/programme experience D(G-P) recommend urgent actions in the dimension of CPD activities (Figure 5.1) and suggest development of a broad range of CPD activities. In Part 3, we have combined the 19 statements ( Table 5.8) that describe professional development activities in six groups and assigned six scales of the survey. P3-1 Imparting information (trainer-centered) (q. 1, 2, 3) P3-2 Learning facilitation (person-centered) (q. 4, 5, 7, 8, 9, 10) P3-3 Collaboration (q. 11, 13) P3-4 Mentor-mentee support (q. 12, 15, 17) P3-5 (Personal/individual) expert support (q. 14, 16) P3-6 Knowledge sharing (q. 6, 18, 19) Scales P3-1 and P3-2 measure the importance of the pedagogical character of the CPD activities, namely the activities person-centered or trainer-centered. The other scales focus on the importance of the type of the interaction that takes place in activities. P3-3 includes collaboration and peer-feedback between the participants, P3-4 individual support by a more 108 experienced lecturer, P3-5 individual support by experts (not lecturers) and P3-6 activities relevant to knowledge sharing. Table 5.10 presents the values of the six scales in the dimension of the professional development activities, calculated from the results of the survey taken by 420 lecturers from 26 countries and by 46 education managers from 11 countries in the survey Part 3 of the survey. Table 5.10. Values of the scales that measure the importance of the type of professional development activities organized for the lecturers, in the opinion of the lecturers and the education managers. Scales Lecturers Lecturers Managers Managers Cronbach’s General D(G-P) General D(G-P) Alpha P3-1 G: 0.840 Imparting information 3.59 0.73 3.62 0.70 P: 0.846 (q. 1, 2, 3) P3-2 G: 0.917 Learning facilitation (q. 4, 3.25 0.92 3.39 0.77 P: 0.929 5, 7, 8, 9, 10) P3-3 G: 0.714 Collaboration (q. 11, 13) 3.59 0.80 3.73 0.91 P: 0.775 P3-4 G: 0.843 Peer-Mentor – mentee 3.75 0.80 3.83 0.96 P: 0.831 support (q. 12, 15, 17) P3-5 G: 0.725 Personal / individual 3.28 0.94 3.52 0.97 P: 0.733 Expert support (q. 14, 16) P3-6 G: 0.868 Knowledge sharing (q. 6, 3.23 0.84 3.40 0.88 P: 0.901 18, 19) The results in Table 5.10 show that the values in all scales lower than 4.00. This means that organizing professional development activities for the university lecturers, in the opinion of the lecturers and education managers who have participated in the survey was not considered as something very important and that knowledge sharing was seen as the least important activity (3.23 by lecturers and 3.40 by education managers). It is remarkable that the scale Imparting information in both groups has a higher value than learning facilitation. This is in contradiction with the general knowledge and extensive research (Freeman, 2014) about how learning works and how to make teaching effective (Biggs, 2011). Besides, the discrepancy D(G-P) between the scales that measure the general importance of activities and experiences in personal or programme practice are higher in comparison to the values obtained in the other two parts of this survey. This illustrates the high urgency for organising a broad variety of CPD that also might increase the motivation among lecturers. Conclusions This paper discusses the survey developed within the framework of the Erasmus+ project STEM-CPD@EUni which was conducted in the period from November 2020 to January 2021 with the goal to create a roadmap for continuous professional development (CPD) of STEM lecturers. The survey has 66 statements distributed in three parts and could be used in the future as an instrument to measure the professional development needs of STEM lecturers. The participants evaluate the statements in the survey from two perspectives: general importance 109 and the use in personal/programme teaching practice. We have defined three CPD dimensions ( Figure 5.1): teaching and learning competences, teaching and learning (CPD) attitudes and CPD activities. Using the results, we have defined priority lists for the teaching and learning competences, teaching and learning (CPD) attitudes and CPD activities that work best according to the opinion of 420 lecturers from 26 countries and 46 managers from 11 countries who have completed the survey at that moment. The limitation of this research is the small number of participants coming from most of the countries. We have clustered the 66 statements of the survey in the three parts according to educational principles in larger educational concepts considering competences, attitudes and CPD activities. The results show that both lecturers and education managers find the concept Constructive alignment very important and also several concepts connected to active learning. The results also show that some concepts are not yet seen as important for lecturers and their professional development such as the use of digital technology. Moreover, some conceptions regarding what is needed for professional development of lecturers refer to teacher-centered views. The CPD-ambassadors who organize the CPD activities need to do more than blindly follow the priority list defined in this survey but operate evidence based. We therefore recommend that CPD-ambassadors innovate teaching themselves to provide inspiring new examples and user cases and share their knowledge and experience in the community of CPD-ambassadors. We expect that the CPD activities that are organized at the local university will influence the needs for the CPD of lecturers. We also recommend that this survey is taken again after some time to measure the change in needs and to gain insight on the impact of CPD-ambassadors. Acknowledgement We thank Lester van der Pluijm for his advice and the calculations in the cluster analysis. The research presented in this article is co-funded by the Erasmus+ Programme of the European Union: STEM Continuous Professional Development at European Universities 2020-1-PL01-KA203-081802. References Adu, E. O., and Okeke, C. I. O. (2014). Factors affecting lecturers’ participation in Continuing Professional Development (CPD). Journal of Sociology and Social Anthropology, 5(3), 271–281. Biggs, J., and Tang, C. (2011). Teaching for Quality Learning at University, 4th edition, New York: Open University Press. Brouwer, N., Maciejowska, I., Lis, A., Machado, C., Grecea, S., Kärkkäinen, J., Niemelä, M., Kranjc, K., Podlipnik, Č., Prashar, S., Russo, V. and Tarallo, O. (2020). The Need for STEM Continuous Professional Development at European Universities. VIRT&L-COMM 2020, 21, 2279–8773. Chametzky, B. (2014). Andragogy and Engagement in Online Learning: Tenets and Solutions. Creative Education, 5(10), 813–821. Cowan, J., and Westwood, J. (2006). Collaborative and reflective professional development: A pilot. Active learning in higher education, 7(1), 63–71. Dochy, F., Gijbels, D., Segers, M., and van den Bossche, P. (Eds.). (2011). Theories of learning for the workplace. Building blocks for training and professional development programs. London: Routledge. 110 ECTN WG (2020). Quick Scan Inquiry Professional Development of University Lecturers at EU Universities 2019, website ECTN WG Lecturing qualifications and innovative teaching methods, Publications, Reports, Retrieved January 30, 2022 http://ectn.eu/work-groups/lecturing-qualifications-and-innovative-teaching-methods/publications/. Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., and Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the national academy of sciences, 111(23), 8410-8415. Goodchild, S. (2014). Theorising community of practice and community of inquiry in the context of teaching-learning mathematics at university. Research in Mathematics Education, 16(2), 177. Grecea, Ş., Brouwer, N., Niemelȁ, M. and Kȁrkkȁinen, J., Eds. (2021). Roadmap for STEM Continuous Professional Development at European Universities, Recommendations and Guideline, STEM-CPD@EUni, 2020-1-PL01-KA203-081802, Retrieved January 30, 2022 https://ectn.eu/wp-content/uploads/2021/06/Roadmap-Recommendtions-and-Guidelines-O1-April2021.pdf. de Groot, J. and Kouwenaar, R. (2018). Professionalisation of university lecturers. The UTQ and beyond, VSNU, May 2018. Retrieved January 30, 2022 http://www.vsnu.nl/files/documenten/Professionalisation%20of%20university%20lectur ers.pdf. Knowles, M. S. (1980). The modern practice of adult education: From pedagogy to andragogy (revised and updated). Englewood Cliffs, NJ: Cambridge Adult Education. Maciejowska I., Trzewik B., Zięba K., Brouwer N., Gracea S., Pirok B., Vreede J., Niemelä M. and Kärkkäinen J. (2022). Rozwój zawodowy nauczycieli akademickich na wydziałach ścisłych, przyrodniczych, technicznych i inżynieryjnych: potrzeby i oczekiwania . In A. Sajdak-Burska & I. Maciejowska (Eds.) Dydaktyka akademicka w formie zdalnej i stacjonarnej. Nowe doświadczenia. Krakow: Wydawnictwo Uniwersytetu Jagiellońskiego, accepted. Marsh, H. W. (2007). Do university teachers become more effective with experience? A multilevel growth model of students’ evaluations of teaching over 13 years. Journal of Educational Psychology, 99(4), 775-790. Mishra, P., and Koehler, M. J. (2006). Technological pedagogical content knowledge: a framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054. te Pas, S., and Zhang, T. (2019). Career paths in teaching: Thematic Peer Group Report, EUA Learning & Teaching Thematic Peer Group, European University Association, Retrieved January 30, 2022 https://eua.eu/resources/publications/808:career-paths-in-teaching-thematic-peer-group-report.html. Redecker, C. (2017). European framework for the digital competence of educators: DigCompEdu (No. JRC107466). Joint Research Centre (Seville site). Stes, A., Min-Leliveld, M., Gijbels, D., and Van Petegem, P. (2010). The impact of instructional development in higher education: the state-of-the-art of the research. Educational Research Review, 5(1), 25-49. European University Association, Issues, Bologna process, , Retrieved January 30, 2022 https://eua.eu/issues/10:bologna-process.html. 111 Subject Index A Gatekeeper, 41 Active learning, 13, 29, 37, 73, 92, 97, 114 Analytical Chemistry, 28, 29, 31, 34, 35, 37 H Advanced tools, 89 93, 97 Hazardous chemicals, 29 Holistic, 69, 70, 73, 80 B Beneficial health effects, 23 I Basic skill, 31 Inquiry-based, 12, 13, 18 Biochemistry, 40 Investigate, 15, 43, 46, 72, 85 Bronsted-Lowry, 40, 41 Instructors, 26 Blended, 68, 69, 90, 93, 96, 97 Information and communication, 86 Brainstorming, 87 J C Justifications, 40, 57 Chemical Engineering, 12 Chemical topics, 12 K Contextualized, 12, 13 Kruskal-Wallis, 46, 48, 53, 59, 63 Competencies, 13 Curricula, 37, 71, 86 L Cronbach’s Alpha, 96, 97, 102, 109 Learning concepts, 20 Lab practice, 26 D Lambert-Beer’s law, 34 Discovery learning, 18 Levene’s test, 88 Digital image colorimetry, 26, 28 Data analysis, 13, 42, 44, 66 M Module, 33, 69, 70 E Memorization, 40, 65 Educational tools, 13 Mann-Whitney, 46, 48, 53, 59, 63 Experimental, 12, 14, 15, 18, 35 Environmental, 17, 18, 19, 24 N Energy waste, 36 Normalise, 71, 82 Erasmus+, 85, 87, 109, 110 Network, 85, 87, 103, 107 F O Future teachers, 25 Outer, 15 Frameworks, 40, 41, 65, 66 Future courses, 41 P Facts, 51, 52, 57, 65, 66 Physicochemical concepts, 14 Formative, 68, 69, 72 Properties, 16, 23, 28, 40, 51 Phenomena, 16 G Pseudoscientific, 12, 23 Green analytical chemistry, 26, 36, 37 Problem-solving, 13, 14,56 112 Pre-practical, 72 TPACK, 86, 96 Q U Qualitative study, 33 Undergraduate laboratory, 26 Quantitative study, 35 V R Virtual learning, 71 Real life, 24 Video, 43, 71, 73, 81, 98, 102 Region of interest, 32 Research based, 90, 92 W Waste generation, 36 S Working sessions, 89 Science education, 12 Society 12, 13 Y Science literacy, 13 Younger, 89 Summative, 69 Z T Zoom, 32, 42 Teamwork, 13, 14 113 EUROVARIETY 2021 was organized and sponsored by: 114 From the Review »The results of the contributions presented in the monograph “University Chemistry Teaching in the 21st Century” clearly contribute to enriching the opus research already being conducted in the field of tertiary chemistry education, providing guidelines for further research, and promoting insights into the own teaching and the transfer of research results to other levels of education.« Dr. Jerneja Pavlin »The published monograph contains important, high quality and original knowledge from the authors of the articles that will influence the modernization of the way chemistry is taught at the university level. It is crucial that all this knowledge finds its way to students - future chemistry teachers and secondary and primary school chemistry teachers.« Dr. Miha Slapničar 115