5 Perfectus AC 1/2021 OBLIKOVANJE RAZISKOVALNE METODOLOGIJE UČNEGA OKOLJA: PRIMER COVID-19 Andrej Raspor https://orcid.org/0000-0002-8098-95541 Povzetek: Namen: V članku obravnavamo različne pristope pri družboslovnem raziskovanju. Namen članka je postaviti metodologijo za raziskovanje vplivov različnih dejavnikov na študente, akademsko osebje in študentske službe v času karantene, kot posledica pandemije COVID-19. Metodologija: Ta študija se opira na analizo že obstoječih sekundarnih podatkov in virov, s pomočjo katerih smo interpretirali rezultate. Primerjali smo različne metodologije, da smo lahko postavili lastno metodologijo, s katero bomo lahko izvedli raziskavo med študenti, akademskim osebjem in študentskimi službami. Ugotovitve: Ugotovili smo, da je potrebno za potrebe raziskave dopolniti anketni vprašalnik, saj mora ta vključevati tudi tista vprašanja, ki se nanašajo na dejavnike stresa, kodeks poučevanja in prihodnost študija. Omejitve: Na podlagi izdelane metodologije bo izvedena raziskava v Sloveniji. Raziskovalne ugotovitve bodo koristile tudi drugim raziskovalcem pri nadaljnjem raziskavanju. Zato pozivamo raziskovalce, da koristijo predlagano metodologijo na ravni izobraževalnih institucij in države. Praktična uporabnost: Članek ponuja neposredno raziskovalno orodje, ki bo ponujeno tudi drugim raziskovalcem v regiji, ki bi želeli raziskavo ponoviti tudi v svojem okolju. Izvirnost prispevka: Originalnost raziskave se kaže v tem, da bodo v raziskavo poleg študentov in akademskega osebja vključene tudi študentske službe. Na ta način naj bi se dobilo celovito sliko, kaj se je dogajalo v času študija na daljavo, in ustrezne napotke, kako bi se morali odzivati v prihodnosti. Ključne besede: študenti, akademsko osebje, študentske službe, epidemija COVID-19. DEVELOPING NEW RESEARCH METHODOLOGY FOR STUDYING LEARNING ENVIRONMENT: COVID-19 Abstract: Purpose: This paper discusses different approaches to social sciences research. The objective is to develop new research methodology that could be adopted to study and analyze how different factors related to Covid-19 crisis affect students, lecturers and student affairs professionals. Design/Methodology/Approach: This study is based on existing secondary data and sources which served as a tool for interpreting the results. In order to develop a completely new methodology we first had to compare other already established methods. We then used this newly developed methodology as a baseline for our research. Findings/Results and conclusions: In order to capture the intended information, we had to adapt the survey questionnaire and add new questions concerned with stress factors, code of conduct for educators and the future of education. Research limitations/implications: Adopting this new methodology we performed research targeting Slovenian university lecturers, students and student affairs professionals. Our findings can serve other researchers working in the field of education, be it at institutional or national level. Practical implications: This paper offers a valuable research tool available to other researchers in the region who show interest in conducting similar research in their own environment. Originality/value: This research is among first attempts to target not only lecturers and students but also student affairs professionals. This approach provides a wider insight into distance learning experience during Covid-19 crisis and offers valuable guidelines on how to manage a similar crisis in the future. Keywords: students, academic staff, student affairs professionals, Covid-19 pandemic. 1 CEATM d.o.o. (Central Europe Association of Tourism Management), Dolga Poljana 57, 5271 Vipava, Slovenia, andrej.raspor@ceatm.org 6 Perfectus AC 1/2021 Introduction Due to the threat of COVID-19, colleges and universities are facing decisions about how to continue teaching and learning while keeping their faculty, staff, and students safe from a public health emergency that is moving fast and is not well understood. Many institutions have opted to cancel all face-to-face classes, including labs and other learning experiences, and have mandated that faculty move their courses online to help prevent the spread of the virus that causes COVID-19 (Hodges idr., 2020). This paper studies the learning environment and all the stakeholders involved (lecturers, student affairs professionals and students), teaching methods (Face-to-Face, Distance education, Blended learning) and relevant research methods which can be adopted to enhance education. The newly designed methodology and survey questionnaire which we used in the research as research instrument will also be available to other researchers who might be interested in doing a similar research and compare the results. Literature review Learning environment Moving instruction online can enable the flexibility of teaching and learning anywhere, anytime, but the speed with which this move to online instruction is expected to happen is unprecedented and staggering (Hodges idr., 2020). The major stakeholders in education include (1) lecturers, (2) student affairs professionals and (3) students as shown in the picture below (Figure 1: Major stakeholders in education). These are also the stakeholders that we will incorporate into our research. If we look closely at the job description of Lecturer, we see that their main duty and responsibility is “To design, develop and produce learning and teaching material and deliver either across a range of modules or within a subject area. Lecturers will ensure the efficient and effective delivery of teaching programs in accordance with the University’s strategy, policy and procedures. They will contribute to activities that influence leading-edge practice, and may also undertake research activity”. The key to becoming an effective student is learning how to study smarter, not harder (Loveless, n. d.). Today’s college students are spending less time studying (Nonis & Hudson, 2006). That is even more true when it comes to online learning. For this very reason new teaching strategies and methods should be implemented to motivate students and help them to develop adaptability skills. Despite the fact that students have everything they need to develop necessary digital competences, they show little interest in applying them in the classroom to enhance learning, but rather use them in more unproductive and unrewarding ways, such is online interaction with their peers. Along with the present trend of students spending less time on academic related activities, a growing number university administrators are concerned that today’s postsecondary students are working more hours than their counterparts were years ago (Gose, 1998). Student Administration Office, Office of Study Affairs and Student Affairs, student support or student services are different names for the department or division of services and support for student success at institutions of higher education to enhance student growth and development. People who work in student affairs provide services, programs, and resources that help students to learn and grow outside the classroom. Some things that student affairs professionals do for students every day include (NASPA, n. d.): • enhance student learning; • guide academic and career decisions; • mentor students; • promote leadership skills; • offer counselling during crises. The size and organization of a student affairs division or department may vary based on the size, type, and location of an institution. Until COVID-19, employees worked in offices. Later, however, they began to work from home, or partly from home and partly at the office. 7 Perfectus AC 1/2021 Figure 1. Major stakeholders in education The interaction between the previously mentioned stakeholders is either direct or indirect. The job of student affairs professionals is to provide students with information, whereas lecturers offer support to students throughout the learning process. As a result, there is more face-to-face interaction between students and lecturers than between students and student affairs professionals who work with students indirectly. Teaching models There are different types of teaching models which can be categorized into three broad types: traditional teaching in classroom (Face- to-Face), remote teaching (Distance education) and blended or computer assisted teaching, where online learning replaces some of the face-to-face contact time in the classroom (Blended learning). Face-to-Face education is when the course is designed to be delivered face-to-face with the tutor acting as a facilitator in a highly dialogic classroom (Moorhouse, 2020). Online learning as a concept and as a keyword has consistently been a focus of education research for over two decades (Singh & Thurman, 2019). Distance education is the most renowned term used when referencing distance learning (Moore idr., 2011). E-Learning covers a wide range of processes and includes the delivery of content and instructional methods via CD-ROM, the Internet or an Intranet, audio- and videotape, satellite broadcast and interactive TV (Ellis, 2004). Online learning is described by most authors as access to learning experiences via the use of some technology (Benson, 2002; Carliner, 2004; Conrad, 2002). Blended Learning is one of the most modern and innovative type of education that combines old and new in one frame of work, it preserves traditional methods while also making use of technological invention to get students’ attention who have already become addicted to digital media in their social life (Hamad, 2017). Blended learning systems combine face-to-face instruction with computer mediated instruction (Bonk & Graham, 2012). Online learning carries a stigma of being lower quality than face-to-face learning, despite research showing otherwise. These hurried moves online by so many institutions at once could seal the perception of online learning as a weak option, when in truth nobody making the transition to online teaching under these circumstances will truly be designing to take full advantage of the affordances and possibilities of the online format (Hodges idr., 2020). Methodology Liaw and Huang suggested that four elements should be considered when developing e-learning environments: environmental characteristics, environmental satisfaction, learning activities, and learners’ characteristics (Liaw & Huang, 2006). In e-learning environments, environmental characteristics, such as synchronous or asynchronous interaction, will create a high-level communicative environment that allows learners not only to share information, but also to determine how to retrieve useful information. Additionally, environmental satisfaction will enhance learners’ perceptions of technology that might promote their participation in the learning processes. Moreover, learning activities in e-learning provide a great chance for learners and instructors to share their knowledge and experience. In essence, when users feel less self-confident toward information technology, they also show less positive feelings toward the technology (Liaw, 2008). We adopted this methodology, assessed the results (Raspor idr., 2016), and later improved it (Kleindienst & Raspor, 2020) in order to capture the intended information. We adapted the questionnaire based on our findings and test results. New questions about stress were added (Irawan et al., 2021) (Lait & Wallace, 2002) which encompass all three groups of stakeholders. Apart from that, additional question was added to the questionnaire addressing the future of education as seen by participants (Franchi, 2020). As the academics are facing radical changes in the learning environment, education institutions have no choice but to adopt new code of conduct for students (Schools, 2016) as well as for lecturers (Stahl idr., 2004). 8 Perfectus AC 1/2021 Quantitative Research In the most basic terms, quantitative research methods are concerned with collecting and analyzing data that is structured and can be represented numerically (Matthews & Ross, 2010). Quantitative research is a form of research that relies on the methods of natural sciences, which produces numerical data and hard facts. It aims at establishing cause and effect relationship between two variables by using mathematical, statistical and computational methods (Ahmad idr., 2019). The research is also known as empirical research as it can be accurately and precisely measured. The data collected by the researcher can be divided into categories or put into rank, or it can be measured in terms of units of measurement. Graphs and tables of raw data can be constructed with the help of quantitative research, making it easier for the researcher to analyze the results. Checklist of any research project (Bakkalbasi idr., 2012): 1. Project goals, objectives, and desired results 2. List of key stakeholders involved in the research 3. Project Research timeline 4. Confidentiality of Research 5. Data collection process 6. Data analysis 7. Presentation of results Quantitative Research: An Operational Description (Ahmad idr., 2019): • Purpose: explain, predict or control phenomena through focused collection and analysis of numerical data. • Approach: deductive; tries to be value-free/has objectives/is outcome-oriented. • Hypotheses: specific, testable, and stated prior to study. • Literature review: extensive; may significantly influence a particular study. • Setting: controlled to the degree possible. • Sampling: uses largest manageable random/randomized sample, to allow generalization of results to larger populations. • Measurement: standardized, numerical, "at the end" • Design and Method: Strongly structured, specified in detail in advance; involves intervention, manipulation, and control groups; descriptive, correlation, experimental. • Data Collection: via instruments, surveys, experiments, semi-structured formal interviews, tests, or questionnaires. • Data Analysis: raw data is numbers; at end of the study, usually statistical. • Data Interpretation: formulated at end of the study; stated as a degree of certainty. Survey questionnaire Questionnaires are commonly used in quantitative marketing research and social research. A questionnaire is a series of questions asked to participants to obtain statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires become a vital instrument by which statements can be made about specific groups or people or entire populations. They are a valuable method of collecting a wide range of information from a large number of individuals, often referred to as respondents or participants. Adequate questionnaire construction is critical to the success of a survey. Appropriate questions, correct ordering of questions, correct scaling, or good questionnaire format can make the survey worthwhile, as it may accurately reflect the views and opinions of the respondents. A useful method for checking a questionnaire and making sure it is accurately capturing the intended information is to pretest among a smaller subset of target respondents (Martin, 2006). Types of Survey Questions They are applied according to the purpose of the survey. There are about four different types of questionnaire designing for a survey (Goode & Hatt, 1952): 1. Contingency questions/Cascade format 2. Matrix questions 3. Closed-ended questions 4. Open-ended questions. Descriptive statistics 9 Perfectus AC 1/2021 Descriptive statistics are the numerical and graphical techniques which are used to organize, present and analyze data. The form of descriptive statistics that is used to describe a variable in a sample is dependent on the level of measurement that has been used (Fisher & Marshall, 2009). As a nominal level of measurement is the sorting of cases into one of several categories, the measure of dispersion is based on the count or frequency of cases in each category known as the frequency distribution. The measure of central tendency for nominal data is the category with the most frequent number of cases, also known as the mode. Nominal level data are often presented in tables known as ‘‘crosstabs’’ or graphically represented using line or bar graphs (Fisher & Marshall, 2009). Like for nominal data the measure of dispersion can be the frequency distribution of scores. As ordinal data are ordered in a hierarchy then all cases can be sorted from the lowest to highest score (rank-ordered distribution). The value or category of the case in the middle of a rank-ordered distribution is known as the median. Continuous data can be presented in a graphical form known as a frequency histogram (Fisher & Marshall, 2009). Correlational research The term correlation is one of the most useful and frequently used statistical concepts applied in scientific studies (Asamoah, 2014). Correlational research is a type of non-experimental research method in which a researcher measures two variables, understands and assesses the statistical relationship between them with no influence from any extraneous variable. We have two forms of variables in correlational research namely: a criterion variable (dependent variable or output variable) and predictor variable (independent variable). An Independent variable is a variable that is believed to predict the outcome. The dependent variable is the variable to be predicted. Phi correlation is used when both predictor and criterion variables are natural dichotomies (two categories). When the predictor variable is a natural (real) dichotomies (two categories) and the criterion variable is interval or continuous, the serial correlation is used. But if the dichotomies are artificial, the tetrachoric correlation is used (Asamoah, 2014). Survey questionnaire For the purpose of this research, we used survey questionnaire. The survey was created in 1KA survey platform and available online so that the targeted audience could opt when to respond to the survey. We incorporated into our research three target groups: (1) lecturers; (2) student affairs professionals and (3) students, as seen in the picture (Figure 1: Major stakeholders in education). Professional staff Your experience with remote working before COVID-19 pandemic Q2 - How would you describe your remote working experience before COVID-19 (before 1st March 2020)? Please rate the following statements on a scale from 1 to 7 (1 means "NO PRIOR EXPERIENCE"; 7 means "WELL EXPERIENCED"). Experience using the Internet Experience with e-learning (adapted to e-learning technology, e-lectures coordination and administration) Computer literacy (Office tools: Word, Excel, Power Point) Virtual presentation and speaking experience Video presentation experience Video conference experience Remote access experience Q3 - Your e-learning system (tools and software) experience before COVID-19 pandemic (prior to 1st March 2022)? Multiple answers are possible No prior experience Skype Viber Facebook Moodle Arnes Zoom Blackboard Microsoft Teams BigBlueButton MiTeam GoToMeeting Cisco WebEx Slack Other Working online during COVID-19 pandemic 10 Perfectus AC 1/2021 Q5 - Which e-learning software and tools were you using during COVID-19 pandemic? (From 15th March 2020 and from 12th October 2020 onwards)? Multiple answers are possible Skype Viber Facebook Moodle Arnes Zoom Blackboard Microsoft Teams BigBlueButton MiTeam GoToMeeting Cisco WebEx Slack Other Q6 - Rate your self-efficacy and performance using e-learning software and tools on a scale from 1 to 7 (1 means "STRONGLY DISAGREE" and 7 means "STRONGLY AGREE"). I feel confident preparing e-learning contents I feel confident operating e-learning functions, using e-learning tools and software (signing in, using discussion forums ...) I feel confident assessing peers in an e-learning environment Q7 - During COVID-19 pandemic, I was: working from University Office working from University Office most of the time both - working from University Office and working from home working from home most of the time working from home only temporarily laid off Q8 - Working conditions and information provided to the working staff: Please rate the following statements on a scale from 1 to 5 (1 means "STRONGLY DISAGREE", 5 means "STRONGLY AGREE"). I had access to technology and was provided with all the necessary support for effective remote working I had high-speed internet access My employer communicated with me regularly and kept me informed of updated development related to COVID-19 The following questions are related to the e-learning environment (e-classroom and online collaboration tools for students and professors). Q10 - Perceived e-learning satisfaction: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE"; 7 means "STRONGLY AGREE". I am satisfied with using e-learning as a learning assisted tool I am satisfied with operating e-learning functions I am satisfied with e-learning contents I am satisfied with multimedia instructions Q11 - Perceived usefulness of e-classroom: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I believe e-learning contents are informative I believe e-learning is a useful learning tool I believe e-learning contents are useful Q12 - Multimedia instruction: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I like to use voice media instructions I like to use video media instructions I like to use multimedia instructions Benefits and usefulness of acquired knowledge in e-learning environment (e-classroom and online collaboration tools for students and professors). Q14 - Application of the acquired knowledge through e-learning (multiple answers possible): Multiple answers are possible Study At work When communicating with friends In everyday life Other The following questions are concerned with working with students 11 Perfectus AC 1/2021 Q16 - Student support: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). Working with students I noticed a lack of communication Freshmen (first year students) frequently turned to Administrative Office for help and advice Students demanded more information and guidance compared to previous years When I was working from home my workload was bigger and I was putting in more hours than before COVID-19 pandemic. Q17 - The impact of remote working during COVID-19 pandemic on stress and coping behavior in university professional staff Q18 - Study and stress levels: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). My work was stressful for me before COVID-19 pandemic My work became more stressful for me after the COVID-19 outbreak Q19 - Rate the following elements during COVID-19 pandemic: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I perceived online student support as stressful. Using online tools and software was stressful for me. Distractions at home made my working hours stressful. Online work increased family conflicts (common household). I find it difficult to organize my work when working from home. Remote working induced feelings of isolation and loneliness due to the lack of social interaction and absence of face-to-face contact with my colleagues. Being confined to limited amount of space made me feel anxious and stressed. I missed social interaction with my colleagues, students and superiors/leadership. I missed in-person communication with my colleagues from Student Administration Office. What will the future of study look like? Q21 - My expectation regarding study for next year: Please rate the following statements on a scale from 1 to 5 (1 means "STRONGLY DISAGREE", 5 means "STRONGLY AGREE"). Lectures and lab-exercises should be carried out in classroom and laboratory Lectures should be held in classroom Lab-exercises should be held in laboratory More video content should be available to students which they can watch in their free time More interactive exercises should be available to students which they can do in their free time Students should do more teamwork outside the classroom and laboratory. The following part refers to personal information such as job position, work experience, career field, gender, age and status. Q23 - Employee’s position: Student Administration Office Dean’s Office Q24 - Institution where you are employed: Faculty within a University Independent Higher Education Institution Q25 - Work experience in Dean’s Office or Administrative Office: Up to 3 years More than 3 and less than 10 years 10 years or more Q26 - Study programmes available at your institution: Multiple answers are possible Humanities Engineering and Technical Sciences Medical and Health Sciences Agricultural Sciences Social Sciences Mathematics and Natural Sciences Theological Sciences Arts Other Q27 - Gender: Male Female Q28 - What is your age range? < 25 years old 26−40 years old 41−60 years old > 61 years old 12 Perfectus AC 1/2021 Professors Your experience with e-learning Q2 - How would you describe your experience before e-learning (before 1st March 2020)? Please rate the following statements on a scale from 1 to 7 (1 means "NO PRIOR EXPERIENCE"; 7 means "WELL EXPERIENCED"). Experience using the Internet Experience with e-learning (adapted to e-learning technology, e-lectures coordination and administration) Computer literacy (Office tools: Word, Excel, Power Point) Virtual presentation and speaking experience Video presentations experience Video conference experience Remote access experience Q3 - Your e-learning software experience before COVID-19 pandemic (prior to 1st March 2020)? Multiple answers are possible No prior experience Skype Viber Facebook Moodle Arnes Zoom Blackboard Microsoft Teams BigBlueButton MiTeam GoToMeeting Cisco WebEx Slack Other Q4 - Which e-learning software and tools were you using during COVID-19 pandemic? (From 15th March 2020 and from 12th October 2020 onwards)? Multiple answers are possible Skype Viber Facebook Moodle Arnes Zoom Blackboard Microsoft Teams BigBlueButton MiTeam GoToMeeting Cisco WebEx Slack Other Q5 - Rate your self-efficacy and performance using e-learning software and tools on a scale from 1 to 7 (1 means "STRONGLY DISAGREE" and 7 means "STRONGLY AGREE"). I feel confident preparing e-learning contents I feel confident operating e-learning functions, using e-learning tools and software (Signing in, using discussion forums, preparing exams, reviewing presentations ...) I feel confident assessing students in an e-learning environment. The following questions are related to the e-learning environment (e-classroom and online collaboration tools for students and professors). Q7 - Perceived e-classroom satisfaction: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE"; 7 means "STRONGLY AGREE". I am satisfied with using e-learning as a learning assisted tool I am satisfied with operating e-learning functions I am satisfied with e-learning contents I am satisfied with multimedia instructions Q8 - Perceived usefulness of e-classroom: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I believe e-learning contents are informative I believe e-learning is a useful learning tool I believe e-learning contents are useful 13 Perfectus AC 1/2021 Q9 - Behavioral intention of using e-learning: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I intend to use e-learning I intend to incorporate e-learning in my teaching process in the future I intend to use e-learning as the core/the only method of teaching Q10 - E-learning satisfaction: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE"; 7 means "STRONGLY AGREE". I am satisfied with the applications of e-learning functions I am satisfied with internet speed I am satisfied with e-learning contents I am satisfied with multimedia e-learning instructions and navigation Q11 - Interactive learning activities: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE"; 7 means "STRONGLY AGREE". I would like to share my e-learning experience with others I believe e-learning can assist teacher-student interaction I believe e-learning can assist peer-to-peer interaction Q12 - Effectiveness of e-learning: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I believe e-learning can assist learning efficiency I believe e-learning can assist learning performance I believe e-learning can assist learning motivation Q13 - Multimedia instruction: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I like to use voice media instructions I like to use video media instructions I like to use multimedia instructions. Benefits and usefulness of acquired knowledge in e-learning environment (e-classroom and online collaboration tools for students and professors). Q15 - Your skills and competencies BEFORE (left side of the scale) and AFTER (right side of the scale) presentation. Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). Before COVID-19 (before 15th March 2020) After COVID-19 (after 15th March 2020 and from 12th October 2020 onwards) I make good power point presentations (concise, comprehensive, without too much text, visually attractive, with animations). I use key elements of a good presentation (sound and clear speech, self-confidence, concise expression, no shutter or buzzwords, body language). I finish presentation in time. I encourage discussion after my presentation. I am confident presenting online. I identify possible questions from students and prepare answers. I perceive negative feedback as encouragement for improvement. I encourage every single student to participate in the discussions. I provide feedback to students with a sole intention of helping them to improve. I provide positive feedback and encouragement to my students. I can evaluate presentations of my students. Q16 - Application of the acquired knowledge through e-learning (multiple answers possible): Multiple answers are possible Study At work When communicating with friends In everyday life Other: Q17 - The impact of the distant learning during COVID-19 pandemic on stress and coping behavior Q18 - Study and stress levels: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). My work was stressful for me before COVID-19 pandemic. My stress levels increased after the COVID-19 outbreak due to e-learning. Q19 - Rate the following elements during COVID-19 pandemic: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). E-teaching was stressful for me. Using e-tools and software was stressful for me. Distractions at home made my e-teaching stressful. E-teaching increased family conflicts (common household). I find it difficult to organize my work when working from home. E-teaching induced feelings of isolation and loneliness due to the lack of social interaction and no face-to-face contact with my students. Being confined to limited amount of space made me feel anxious and stressed. 14 Perfectus AC 1/2021 I missed social interaction and in-person communication with my colleagues. I missed in-person communication with Student Administration Office. Online learning behavior code of conduct Q21 - The following questions are designed to assess how professors perceive behavioral code of conduct in e-learning environment: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). Join into the e-classroom at least 5 minutes before class. Be sure you check your equipment well prior to this time, to be sure your devices function properly (webcam, microphone, headsets). Hold e-lecture from a quiet area that has minimal background noise or movement. Start your e-lectures and exercises in time. Stick to the schedule. Set a positive example by following the code of conduct, as well as other university rules. Make sure you keep your camera on at all times. Eliminate distractions (mute your microphone when students are speaking to keep the audio channel free of clutter and available to the speaker). Turn your microphone on when you want to resume presentation/lesson. Be respectful and stay focused on the lesson. Don’t play with your computer or other devices. Encourage students to participate in discussion throughout the lesson. Don’t eat or drink during class. In general, you shouldn’t engage in activities that don’t contribute to the lesson. Come prepared. Think of the possible questions that students might pose during class and prepare answers. Q22 - Anything else you might like to add to the e-learning behavioral code list? What will the future of study look like? Q24 - My expectations regarding study for next year: Please rate the following statements on a scale from 1 to 5 (1 means "STRONGLY DISAGREE", 5 means "STRONGLY AGREE"). Lectures and lab-exercises should be held in classroom and laboratory. Lectures should be held in classroom. Lab-exercises should be held in laboratory. More video content should be available for students to watch in their free time. More interactive exercises should be available for students to do in their free time. More teamwork should be incorporated outside the classroom and laboratory. The following part refers to personal information such as education, work, gender, age and status. Q26 - Your work comprises: Lectures Lab. exercises Both - lectures and lab. exercises Q27 - Institution in which you are employed: Faculty within University Independent Higher Education Institution Q28 - At how many institutions are you currently holding lectures/lab. exercises? Only one More than one (in this case, please take this survey only once.) Q29 - Your working status: Employed Freelancer (teaching presents my main source of income) Supplementary work (work on a contract, aside from full-time employment or post-retirement employment) Q30 - Your work experience in this field: Up to 3 years More than 3 and less than 10 years 10 years or more Q31 - Your academic title: Professor Associate Professor Assistant Professor Senior Lecturer Lecturer Lector Teaching Assistant Other: Q32 - What is your field? Multiple answers are possible Humanities Engineering and Technical Sciences Medical and Health Sciences Agricultural Sciences 15 Perfectus AC 1/2021 Social Sciences Mathematics and Natural Sciences Theological Sciences Arts Other: Q33 - Gender: Male Female Q34 - What is your age range? < 25 years old 26−40 years old 41−60 years old > 61 years old Students Your experience with e-learning Q2 - How would you describe your remote learning experience before COVID-19 (before 15th March 2020)? Please rate the following statements on a scale from 1 to 7 (1 means "NO EXPERIENCE"; 7 means "WELL EXPERIENCED"). Experience using the Internet Experience with e-learning (e-classroom, e-lectures) Computer literacy (Office tools: Word, Excel, Power Point) Virtual presentation and speaking experience Video presentation experience Video conference experience Remote access experience Q3 - Your e-learning software experience before COVID-19 pandemic (prior to 15th March 2020)? Multiple answers are possible No experience Skype Viber Facebook Arnes Moodle Zoom Blackboard Microsoft Teams BigBlueButton MiTeam GoToMeeting Cisco WebEx Slack Other Q4 - Which e-learning software and tools were you using during COVID-19 pandemic? (From 15th March 2021 and from 12th October 2021 onwards)? Multiple answers are possible Skype Viber Facebook Arnes Moodle Zoom Blackboard Microsoft Teams BigBlueButton MiTeam GoToMeeting Cisco WebEx Slack Other Q5 - Rate your self-efficacy and performance using e-learning software and tools on a scale from 1 to 7 (1 means "STRONGLY DISAGREE" and 7 means "STRONGLY AGREE"). I feel confident preparing e-learning content I feel confident operating e-learning functions, using e-learning tools and software (signing in, using discussion forums, taking exams, submitting presentations ...) 16 Perfectus AC 1/2021 I feel confident assessing peers in an e-learning environment The following questions are related to e-learning environment (e-classroom and collaboration tools for students and professors). Q7 - Perceived e-classroom student satisfaction: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE"; 7 means "STRONGLY AGREE". I am satisfied with using e-learning as a learning assisted tool I am satisfied with using e-learning functions I am satisfied with e-learning contents I am satisfied with multimedia instructions Q8 - Perceived usefulness of e-classroom: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I believe e-learning contents are informative I believe e-learning is a useful learning tool I believe e-learning contents are useful Q9 - Behavioral intention of using e-learning: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I intend to use e-learning to assist my learning I intend to use e-learning contents to assist my learning I intend to use e-learning as an autonomous learning tool Q10 - The quality and effectiveness of e-learning: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I am satisfied with e-learning functions I am satisfied with Internet speed I am satisfied with e-learning contents I am satisfied with e-learning interactions (instructions) Q11 - Interactive learning activities: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I would like to share my e-learning experience I believe e-learning can assist learning performance I believe e-learning can assist learning motivation Q12 - E-learning effectiveness: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I believe e-learning can assist learning efficiency I believe e-learning can assist learning performance I believe e-learning can assist learning motivation Q13 - Multimedia instructions: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I like to use voice media instructions I like to use video media instructions I like to use multimedia instructions. Benefits and usefulness of acquired knowledge in e-learning environment (e-classroom and online collaboration tools for students and professors). Q15 - Your skills and competencies BEFORE (left side of the scale) and AFTER (right side of the scale) presentation: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). Before COVID-19 (before 15th March 2020) and after COVID-19 (after 15th March 2020 and from 12th October 2020 onwards) I make good power point presentations (concise, comprehensive, without too much text, visually attractive, with animations). I use key elements of a good presentation (sound and clear speech, self-confidence, concise expression, no shutter or buzzwords, body language). I finish presentation in time. I encourage discussion after my presentation. I am confident presenting online. I identify possible questions from the professor and colleagues and prepare answers. I perceive negative feedback as encouragement for improvement. I enroll in the discussions. I provide my feedback with a sole intention of helping others to improve. I provide positive feedback and encouragement to my colleagues. I can evaluate presentations of my study colleagues. Q16 - Application of the acquired knowledge through e-learning (multiple answers possible): Multiple answers are possible Study At work When communicating with friends In everyday life Other. 17 Perfectus AC 1/2021 The impact of the distant learning during COVID-19 pandemic on stress and coping behavior in University students Q18 - Study and stress levels during online learning: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I already perceived my study as stressful before COVID-19 pandemic. Study became more stressful for me after the COVID-19 outbreak due to e-learning. Q19 - Rate the following elements during COVID-19 pandemic: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). I perceived e-learning as stressful. Using e-learning tools and software was stressful for me. Distractions at home made my e-learning stressful E-learning increased family conflicts (common household). I find it difficult to organize my work when working from home. E-learning induced feelings of isolation and loneliness due to the lack of social interaction and no face-to-face contact with my colleagues. Being confined to limited amount of space made me feel anxious and stressed. I missed social interaction and in-person communication with my colleagues and professors. I missed in-person communication with Student Administration Office. Online learning behavior code of conduct Q21 - The following questions are designed to assess how students perceive behavioral code of conduct in e-learning environment: Please rate the following statements on a scale from 1 to 7 (1 means "STRONGLY DISAGREE", 7 means "STRONGLY AGREE"). Join into the e-classroom at least 5 minutes before class. Be sure you check your equipment well prior to this time, to be sure your devices function properly (webcam, microphone, headsets) Attend e-classroom from a quiet area that has minimal background noise or movement. Make sure you keep your camera on at all times (professor can remove students from the meeting if their videos are off) Eliminate distractions (mute your microphone to keep the audio channel free of clutter and available to the speaker). Turn your microphone on only when you want to say something or you are asked to speak up). Pay attention and stay focused on the lesson. Don’t play with your computer or other devices. Pay attention and listen so that you can respond appropriately when it’s your turn to speak up. Participate in the discussion. Be respectful. Listen to your professor and colleagues. Refrain from using strong language or making offensive comments. Don’t eat or drink during class. In general, you shouldn’t engage in activities that don’t contribute to the lesson. To be able to participate, you need to pay attention and listen. Don’t interrupt the person who is speaking. Come prepared. Think of the possible questions that professor might ask you during class and come up with answers. Q22 - Anything else you might like to add to the e-learning behavioral code list? What will the future of study look like? Q24 - My expectations regarding study for next year: Please rate the following statements on a scale from 1 to 5 (1 means "STRONGLY DISAGREE", 5 means "STRONGLY AGREE"). Lectures and lab-exercises should be held in classroom and laboratory. Lectures should be held in classroom. Lab-exercises should be held in laboratory. I would like to see more video content that I can watch when I have time. I would like to see more interactive exercises that I can do when I have time. I would like to see more teamwork outside the classroom and laboratory. The following part refers to personal information such as education, field of study, gender, age and status. Q26 - Your field of study: Multiple answers are possible Humanities Engineering and Technical Sciences Medical and Health Sciences Agricultural Sciences Social Sciences Mathematics and Natural Sciences Art, design and media Other: Q27 - Your study programme: Higher education programme Associate’s degree programme Bachelor’s degree programme Master’s degree programme Integrated Master’s degree programme (Long-cycle Master’s degree programme Doctoral degree programme Q28 - Enrolled in the academic year 2020–2021: 18 Perfectus AC 1/2021 Freshman year 2020–2021 Sophomore/Junior/Senior year 2020–2021 Graduate Q29 - Your current employment status: Student Part-time employment through Student Service (Student Employment Referral) Full-time employment through Student Service (Student Employment Referral) Unemployed Part-time employment Full-time employment (contract or permanent) Freelancer Other: Q30 - Gender: Male Female Q31 - What is your age range? < 25 years old 26–40 years old 41–60 years old > 61 years old Discussion The aim of this research was to collect a sufficient amount of data required to develop new methodology. We believe that this new methodology can serve as a baseline for future research in the field of education. If a similar crisis situation like the coronavirus epidemic should arise and repeat itself in the future, this research may serve as a handbook for other researchers who can contribute their own findings and improvements. Along with newly emerging methods of teaching and assessment increases the need and demand for development of new research tools to improve future research practices. References: 1. Ahmad, S., Wasim, S., Irfan, S., Gogoi, S., Srivastava, A., & Farheen, Z. (2019). Qualitative vs. Quantitative Research. population, 1, 2. 2. Asamoah, M. K. (2014). Re-examination of the limitations associated with correlational research. Journal of Educational Research and Reviews, 2(4), 45–52. 3. Aufderheide, P. (1993). A report on the National Leadership Conference on Media Literacy. 4. Bakkalbasi, N., Sundre, D., & Fulcher, K. (2012). Assessing assessment: a framework to evaluate assessment practices and progress for library collections and services. Proceedings of the 2012 Library Assessment Conference, Association of Research Libraries, Charlottesville, VA, 533–537. 5. Benson, A. D. (2002). Using online learning to meet workforce demand: A case study of stakeholder influence. Quarterly review of distance education, 3(4), 443–452. 6. Bonk, C. J., & Graham, C. R. (2012). The handbook of blended learning: Global perspectives, local designs. John Wiley & Sons. 7. Carliner, S. (2004). An overview of online learning. 8. Conrad, D. (2002). Deep in the hearts of learners: Insights into the nature of online community. Journal of distance education, 17(1), 1–19. 9. Đorđević, K. (2018, marec 29). Mladi u Srbiji provode tri i po sata dnevno na internetu. https://www.politika.rs/sr/clanak/401028/Drustvo/Mladi-u-Srbiji- provode-tri-i-po-sata-dnevno-na-internetu 10. Fisher, M. J., & Marshall, A. P. (2009). Understanding descriptive statistics. Australian critical care, 22(2), 93–97. 11. Franchi, T. (2020). The impact of the Covid‐19 pandemic on current anatomy education and future careers: A student’s perspective. Anatomical Sciences Education, 13(3), 312. 12. Goode, W. J., & Hatt, P. K. (1952). Methods in social research. 13. Gose, B. (1998). More freshmen than ever appear disengaged from their studies, survey finds. The Chronicle of Higher Education, 45, A37. 14. Hamad, M. M. (2017). Pros & Cons of Using Blackboard Collaborate for Blended Learning on Students’ Learning Outcomes. Higher Education Studies, 7(2), 7–16. 15. Hobbs, R. (2019). Media Literacy Foundations in The International Encyclopedia of Media Literacy. John Wiley & Sons, INc. https://doi.org/10.1002/9781118978238.iem10063 16. Hodges, C. B., Moore, S., Lockee, B. B., Trust, T., & Bond, M. A. (2020). The difference between emergency remote teaching and online learning. 17. Irawanto, D. W., Novianti, K. R., & Roz, K. (2021). Work from home: Measuring satisfaction between work–life balance and work stress during the COVID-19 pandemic in Indonesia. Economies, 9(3), 96. 18. Kleindienst, P., & Raspor, A. (2020). The effectiveness of e-learning for students acquiring online presentation competence . V M. Makarovič & B. Rončević (Ur.), Technology and social choices in the era of social transformations (str. 83–100). P. Lang . 19. Kuzmanović, D., Milošević, T., Pavlović, Z., & Popadić, D. (2019). Using the Internet and Digital Technology Among Young People in Serbia: Results of the Survey of European Children on the Internet. Belgrade Institute of Psychology. 20. Lait, J., & Wallace, J. (2002). Stress at work: A study of organizational-professional conflict and unmet expectations. Relations industrielles/Industrial relations, 57(3), 463–490. 21. Liaw, S.-S. (2008). Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51(2), 864–873. https://doi.org/10.1016/j.compedu.2007.09.005 22. Liaw, S.-S., & Huang, H.-M. (2006). Developing a collaborative e-learning system based on users’ perceptions. International Conference on Computer Supported Cooperative Work in Design, 751–759. 23. Loveless, B. (b. d.). Study Habits of Highly Effective Students. Educationcorner.com. 19 Perfectus AC 1/2021 24. Martin, E. (2006). Survey questionnaire construction. Survey methodology, 13. 25. Matthews, R., & Ross, E. (2010). Research methods: A practical guide for the social sciences. Pearson Education Ltd. 26. Media Education Center Belgrade. (2017). National Media Advisory and Information Literacy in Serbia; Preparatory Document on National Media and Information Literacy Policies and Strategies. http://www.rem.rs/uploads/files/strategija/Strategija razvoja sistema javnog informisanja u RS 2020-2025.pdf 27. Milojević, A., & Ružić, N.Vajzović, E. (2021). Regional Overview of the Development and Integration of Media and Information Literacy. Faculty of Political Sciences of the University of Sarajevo. https://rs.n1info.com/scitech 28. Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same? The Internet and higher education, 14(2), 129–135. 29. Moorhouse, B. L. (2020). Adaptations to a face-to-face initial teacher education course ‘forced’online due to the COVID-19 pandemic. Journal of Education for Teaching, 46(4), 609–611. 30. NASPA. (b. d.). Careers. 31. Nonis, S. A., & Hudson, G. I. (2006). Academic performance of college students: Influence of time spent studying and working. Journal of education for business, 81(3), 151–159. 32. Raspor, A., Gojšić, J., & Rozman, T. (2016). Self-evaluations of students’developed competences during seminar presentations in the online program. Polytechnic and design, 4(4), 377–388. 33. Schools, C. P. (2016). Student code of conduct. Chicago Public Schools. 34. Singh, V., & Thurman, A. (2019). How many ways can we define online learning? A systematic literature review of definitions of online learning (1988-2018). American Journal of Distance Education, 33(4), 289–306. 35. Stahl, B. C., Wood, C., & Howley, R. (2004). Teaching professional issues in computing through the development of a student code of conduct. Proceedings Ethicomp, 812–825. 36. UNESCO. (b. d.). The Convention on the Protection and Promotion of the Diversity of Cultural Expressions.