POLLINATORS OF SUNFLOWERS THROUGH CITIZEN SCIENCE: AN ADULT EDUCATION APPROACH Anton GRADIŠEK1*, Johanna Amalia ROBINSON2, Nevenka BOGATAJ2 and Danilo BEVK3 1Department of Condensed Matter Physics and Department of Intelligent Systems, Jožef Stefan Institute, Jamova Cesta 39, 1000, Ljubljana, Slovenia; anton.gradisek@ijs.si 2Center for research and development, Slovenian Institute for Adult Education, Ulica Ambrožiča Novljana 5, 1000 Ljubljana, Slovenia; johanna.robinson@acs.si, nevenka.bogataj@acs.si 3Department of Organisms and Ecosystems research, National Institute of Biology, Večna pot 111, 1000 Ljubljana, Slovenia; danilo.bevk@nib.si *Correspondence: anton.gradisek@ijs.si Abstract – We conducted a citizen science project where the participants observed pollinators on sunflowers. Individual or group observations resulted in over 1800 observation sessions with over 5000 pollinators counted, at various locations across Slovenia. The aim of the project was first to use the framework of citizen science as a teaching platform in adult education to educate the participants about pollinators and their importance, and second, to study the pollinators of sunflowers based on the time of the day, weather, and different geographical characteristics. We focused on studying the differences based on the level of urbanization, altitude, and between dif- ferent regions. The results are consistent with the previous studies of pollinator com- munities in Slovenia, however, we are aware of the unbalanced dataset resulting from an uneven distribution of observing sites and observation times. This is a com- mon outcome in citizen science projects but can be addressed with careful planning in future studies. KEY WORDS: Citizen Science, pollinators, pollinator communities, sunflowers, adult education 5 ACTA ENTOMOLOGICA SLOVENICA LJUBLJANA, JUNIJ 2023 Vol. 31, øt. 1: 5–22 Izvleček – ŠTUDIJA OPRAŠEVALCEV SONČNIC S POMOČJO OBČANSKE ZNANOSTI: PRISTOP Z IZOBRAŽEVANJEM ODRASLIH V projektu občanske znanosti so udeleženci opazovali opraševalce na sončnicah. Zbrali smo več kot 1800 posamičnih ali skupinskih opazovanj in več kot 5000 pre- štetih opraševalcev na različnih lokacijah po Sloveniji. Namen projekta je bil (i) upo- rabiti okvir občanske znanosti kot učno platformo v izobraževanju odraslih za izobraževanje udeležencev o opraševalcih in njihovem pomenu in (ii) preučevati opraševalce sončnic glede na čas dneva, vreme in različne geografske značilnosti. Osredotočili smo se na preučevanje razlik glede na stopnjo urbanizacije, nadmorsko višino in med različnimi regijami. Rezultati so skladni s predhodnimi študijami združb opraševalcev v Sloveniji, vendar se zavedamo neuravnoteženega nabora podatkov, ki je posledica neenakomerne porazdelitve opazovalnih mest in časa opa- zovanja. To je pogost rezultat v projektih občanske znanosti. S skrbnim načrtova- njem prihodnjih študij lahko te probleme omejimo. KLJUČNE BESEDE: Občanska znanost, opraševalci, združbe opraševalcev, sončnice, izobraževanje odraslih Introduction Pollinators provide one of the most important ecosystem services, essential for both agricultural production and ecosystem functioning (Klein et al. 2007; Ollerton et al. 2011; Smith et al. 2022). Due to environmental changes, pollinator populations are in decline (Biesmeijer et al. 2006). Climate change is an emerging local and glob- al threat to pollinators, and their habitats, homogenizing their diversity and making them less resilient to rapid environmental changes (Karlsson 2014; Vasiliev and Greenwood 2021). Sustainability of pollinator communities is essentially dependent from our under- standing not only of the specific process of pollination but of common-pool- resources in general as well as of our ability to develop their governance through practical approaches and concrete steps (Tucker et al. 2023). Ecological and biodi- versity monitoring framed into the Global Action Plan (UNESCO 2014) aims at social transformation by education for sustainable development may have multiple roles. The participation of the public in conducting scientific research of natural processes, like for example pollination, can enhance the spatio-temporal perspective of people, and increase green competences (Bianchi et al. 2022). In research, large data sets are needed in order to understand large scale patterns in nature and the par- ticipation of the public can increase the spatio-temporal coverage of data (Bonney et al., 2009; Dickinson et al., 2012; McKinley et al., 2017). These are needed to adapt to local and global environmental changes, and the needs for conservation and sus- tainable resource management. Seldom do scientists have the resources for gathering such datasets alone, hence community science, in the form of citizen science (CS) in Acta entomologica slovenica, 31 (1), 2023 6 entomology and in agricultural research is a continuing trend (Gardiner and Roy, 2022; Ebitu et al., 2021). Involving citizen scientists in data collection and analysis can raise questions on data quality, which Kosmala et al. (2016) however assure, that can be overcome as long as scientists keep their role on making sure the studies are conducted following scientific principles. Citizen science approach in connection with pollinators in the past couple of years has been adopted, for example, to study the diversity of pollen with 750 bee- keepers participating (Brodschneider et al., 2021). The projects such as the Great Sunflower Project, the Great Pollination Project, and Bee Watchers focused on CS pollinator monitoring, with the first project engaging K-12 students and the other two mostly adults (Oberhauser and LeBuhn, 2012; Domroese and Johnson, 2017). Serret et al. (2019) report on a study in France and South Korea where participants were monitoring pollinators by means of photography and species identification both by participants and experts, they highlighted the use of a social network and smartphone apps as tools that kept participants engaged and facilitated the work. Koffler et al. (2021) reviewed several CS projects on bees and highlighted the need for this type of projects to be more widely adopted in developing countries. Birkin and Goulson (2015) recruited volunteers to study pollination of fava beans (Vicia faba) that are primarily pollinated by bumble bees. The study that involved hand-pollinated, bee- pollinated, and bee-excluded plants, recruited 173 participants, 80 of which success- fully completed all parts of the protocol. Roy et al. (2016) conducted a CS study where 13,000 schoolchildren across the UK counted bumble bees visiting lavender (Lavandula spp.) and categorized them into six groups based on color patterns. More than 26,000 bumble bees were counted during the study, allowing for comparing the bumble bee abundance in different landscape types (no significant differences observed) and in regard with the distance of the focal plant from other plants, where it was found that bees were less abundant on plants that were five or more meters away from other flowers. The authors also conducted a quiz with 27 schoolchildren and adults, the results demonstrated that the precision of determination heavily var- ied between color groups. Bloom and Crowder (2020) engaged volunteers who mon- itored bees in Washington State, USA, for three years. They conducted two CS stud- ies simultaneously. In the first one, the participants were monitoring pollinators in urban gardens and characterizing them into six groups (wasp, fly, butterfly, beetle, bug, or spider), with further determination of bees. The study involved photographs. In the second study, the participants were provided nest-boxes for solitary bees and were observing the frequency of bee visits in view of time of the day and weather. The nest-boxes were then returned to the experts who incubated the bees in order to determine the species. Mason and Arathi (2019) conducted a CS project in an urban area in Fort Collins, Colorado, USA, over two years from May to September. 30 vol- unteers participated, they grouped the pollinating bees to eight groups based on mor- phology. They report a high volunteer retention rate while the quality of data collect- ed was good due to prior training and constant engagement. Lander (2019) is running A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 7 a long-term CS project aimed at monitoring solitary bees by providing participants with nest-boxes. Bila Dubaić et al. (2022) were tracking an expansion of alien bee species in Southeast Europe through citizen science. Maund et al. (2020) pointed out a shift towards projects where data is collected in isolation and submitting individual findings online, leaving little or no opportuni- ties for direct social action with other participants nor face-to face training. On the other hand, participation in scientific research through citizen science creates an opportunity for authentic learning experiences where the social aspect of social learning and community-building should not be underestimated (Dickinson et al., 2012; Deguines et al., 2020). Here, we report on our CS study, conducted in summer and autumn of 2022, where volunteers were observing the pollinators visiting sunflowers and categorizing them into eight groups: honey bees, bumble bees, solitary bees, wasps, hoverflies, butterflies, beetles, and others. The aims of the study were twofold; first, to introduce the CS approach into adult education as an education for sustainable development (ESD) and to use this concept as a teaching platform. This aim was implemented in the form of studying pollinators and their importance. The outcomes of this aspect of the study will be reported and discussed in a separate publication by the same authors (Robinson et al., n.a.). Second, from the biology perspective, addressed in this paper, the aim was to use the citizen science approach to systematically observe the polli- nators of sunflowers, both country (Slovenia)-wide and at different times of the day. In particular, we were interested to see what types of pollinators visit sunflowers, at what part of the day and in what type of weather different pollinators are active, and how the pollinator communities differ based on different geographical characteris- tics. In this view, we look into the scientific value of the pollinator data collected with the citizen science approach and compare it with other related studies. Materials and Methods The study was conducted in 2022 within the general aim of implementing climate goals into the Slovenian educational institutional frame and its adult education seg- ment which is focused on methodological integration of natural sciences approaches. Participants were invited into a participative learning process where observation of nature took place. Pollinators were recorded in August and September after prepara- tory activities described below. Approximately 150 volunteers from different parts of Slovenia took part. The guidance and supervision were carried out at all stages of observation by the authors of the paper. We started with recruitment of individual participants from the study circle net- work (more about in Bogataj 2015, see also https://sk.acs.si). Their response was bet- ter than expected – considering that activity was voluntary and substantially longer than typical study circle activities. Training of the participants was provided through web-based initial information, one-day fieldwork training, and a self-evaluative quiz. Acta entomologica slovenica, 31 (1), 2023 8 In April, May, and June three introductory lectures were organized to learn the basics of identification of different pollinators. The lectures were recorded and made avail- able to those who could not attend the meetings in person. In order to check how well the participants recognize different pollinators before they start observations, an online quiz took place in July. Participants were asked to identify 25 pollinators on photographs where pollinators were grouped into predetermined groups, same as during the actual observations. These groups were honey bees, solitary bees, bumble bees, hoverflies, wasps, butterflies, beetles, and “other”. The participants were asked to choose one of the above categories or “I don’t know”. Each participant received a project kit that included a bag of low flowering sun- flower seeds (Helianthus annuus nanus) containing 15-20 seeds, written instructions on the study protocol, a picture guide to help with the identification of pollinators, and standardized recording sheets in forms of Excel tables, either electronic or print- ed. A Facebook group was set up to facilitate the interactions with coordinators and between the participants themselves. Sunflowers were chosen for several reasons. They produce large pseudanthium, an inflorescence, which attracts various pollina- tors. Because of the size of the inflorescence, pollinators typically spend longer time on it as opposed to plants with a single flower, thus allowing the participants a longer observation, supposedly leading to easier and better determination. Furthermore, sunflowers are popular as ornamental plants and thus familiar to the participants, and are relatively easy to grow either in the garden or in sufficiently large pots. The low flowering variety with a single inflorescence was chosen to make observations easi- er. The participants planted about 700 sunflower seeds in total at the beginning of June. They tracked the growth progress, including the dates of planting, plants emer- gence, plants reaching 10 cm in height, and opening of flowers. In addition, the par- ticipants recorded the location of the plants, such as in the garden or in flower pots, floor of the balcony, etc., as well as the factors that in some cases destroyed the plants, such as hailstorm or pests. At the beginning of flowering, typically around mid-August, the participants started observing the pollinators. The protocol instructed them to observe up to five plants, with five being the desired number. To keep participants that lost their plants due to weather or other effects motivated, we permitted them to observe pollinators on other sunflowers in their surroundings, provided that the circumstances were marked separately. Exchange of surplus seedlings was also facilitated to ensure everyone who signed up could stay on board. Individual observations took place any day and time at the convenience of the participants, with recommendation to observe in the following five time slots: 8:00- 9:00, 11:00-12:00, 14:00-15:00, 17:00-18:00, and 20:00-21:00. Within a single time slot, the participants were guided to count the pollinators three times, with at least 5 min between the individual counts. Strictly speaking, the number of pollinators recorded is actually the number of pollinator visits to the sunflower, as it is possible A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 9 that the pollinator visited the flowers more than once during the observation, though it is not straightforward to determine if there were multiple visits. In addition to the pollinator numbers, the participants recorded the weather conditions and were guided to note the temperature from a weather app at the time of the counting. Recordings were taken to the tables provided by the coordinators and later digitized. In addition to individual observations, standardized group observations took place every Wednesday and Saturday in the 11:00-12:00 and/or 17:00-18:00 time slots, when all participants were asked to record the data, if possible, in order to pro- vide comparable data at the same time at multiple sites. The above protocol was developed following the recommendations from Kosmala et al. (2016) in order to comply with data quality issues, with iterative pro- ject development. Before the beginning of the observation, the protocols were tested on a small number of participants to ensure that the instructions are clear and that the proper data and metadata are being recorded. Observations concluded at the end of flowering, which typically took place in the first weeks of September, followed by the participants returning the filled forms to the coordinator. Data from the individually sent datasheets were fused and harmo- nized in a database. A meeting with the participants took place in November in order to obtain their feedback and present some preliminary results. A detailed analysis of the feedback will be reported in a separate publication (Robinson, et al., n.a.). Results and discussion In this section, we first analyze the results of the pollinator identification quiz, then we present the summary statistics of the data collected, and finally the analysis of the data based on observation time, weather, and geographical characteristics of the observation sites.  Pollination identification quiz The quiz was filled in by 40 participants. Intentionally, it was designed in a way that it included several photographs where we assumed the identification was simple and some cases that were more difficult without prior training or field experience.  Figure 1 shows the confusion matrix for the responses of the participants against the correct answers. This data representation is useful to show which categories are often mistaken for one another. The diagonal elements (bolded) indicate the shares of the cases where the participants identified the correct category. For example, honey bees were correctly identified in 65% of the cases. Categories with the largest share of correct answers were beetles, wasps, and butterflies, whereas the partici- pants had the most trouble identifying solitary bees (42%) and hoverflies. The most common mis-identifications were honey bees and solitary bees, hoverflies and soli- tary bees, honey bees and bumble bees. Looking at the out-of-diagonal elements, Acta entomologica slovenica, 31 (1), 2023 10 typically the misidentification was similar both ways, for example 21% of honey bees identified as solitary bees and 19% of honey bees identified as solitary bees. The average score was 15/25 correctly identified pollinators.  Table 1: Confusion matrix for the pollinator quiz, showing the responses of the participants against the correct categories. The diagonal elements, bolded, indicate the share of the cases where the participants identified the correct category. As the number of questions per group was somewhat low (in order to make the quiz short for the participants) and our choice of “simple” and “difficult” questions was largely subjective, it is not straightforward to evaluate the average accuracy of the participants. Nevertheless, the results of the quiz show that the users often cor- rectly identified the group in simpler cases. In field observation, the advantage is that it is possible to see the way of movement and to observe the pollinator from different angles, which makes it easier to pay attention to particular features that make the identification easier but may be obscured in the photos.  In order to improve the identification accuracy in the field study, the participants were directed to additional supporting information. They were encouraged to share photos of unknown pollinators to the Facebook group where the experts could assist with identification. In addition, prior to the opening of the sunflowers, an online meeting was organized with the participants to inform them on the species that they had more difficulty to identify in the quiz. Summary statistics of the observations In total, 93 sites signed up for the study. Due to various reasons, e.g. sunflowers not germinating, or were destroyed prior to flowering, or participants not returning datasheets, we have observations of pollinators from 36 locations. The number of sites does not directly correspond to the number of participants, as one third of par- ticipants had more than one (up to four) observation sites with sunflowers (for exam- ple in the garden and on the balcony). They were observing each site separately. On … when the correct answer was Honey bee Bumble bee Solitary bee Hoverfly Wasp Butterfly Beetle P ar ti ci p an ts a n sw er ed … Honey bee 0.65 0.12 0.21 0.14 0 0 0 Bumble bee 0.01 0.73 0.08 0.02 0 0 0 Solitary bee 0.19 0.06 0.42 0.18 0 0 0 Hoverfly 0.09 0.02 0.14 0.54 0 0.05 0 Wasp 0.06 0.01 0.05 0.07 0.98 0 0 Butterfly 0 0.01 0 0 0 0.88 0.2 Beetle 0 0.01 0 0 0 0 0.98 Other 0 0.01 0.05 0.02 0 0.07 0 Don’t know 0 0.03 0.05 0.03 0.02 0 0 A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 11 the other hand, some sites were observed by more than one participant (some sites were being observed by an organized study group). Except for a few individual occa- sions, the participants did not specifically mark down the number of flowers they were observing. Hence this information was also lacking from the datasheet.  Figure 2 shows the locations of the observation sites superimposed on the map of Slovenia showing climate types. We further clustered the location based on three cri- teria. In view of the altitude, we grouped the sites from 0 to 200 m above sea level (33% of the sites), from 200 to 400 m (53% of the sites), and above 400 m (14% of the sites). The lowest site had an altitude of 103 m and the highest of 694 m. In view of the degree of urbanization, a classification based on a common methodology pre- pared by the Commission Directorates General REGIO (Regional Statistics) and AGRI (Agriculture), Eurostat and the Joint Research Centre (JRC) in cooperation with the OECD was used, having three categories: densely-populated areas, interme- diate density areas, thinly-populated areas.  For a comparison based on geographical location, we chose two clusters with the highest number of observation sites (see Figure 1). One cluster roughly covers the sites in Central Slovenia, which covers both urban and rural areas and also includes Acta entomologica slovenica, 31 (1), 2023 12 Fig. 1. Map of Slovenia representing observation sites according to their location (height above sea level and climate type). The two groupings of observation sites are indicated with red circles: one roughly covers Central Slovenia while another one is situated in Southeast Slovenia. Clusters based on the population density are not shown on the map.  the country’s capital Ljubljana. The other cluster covers the sites in Southeast Slovenia and is predominantly rural. In total, the participants recorded 1862 observations, with 70.6% of observations being performed in thinly populated areas, 18.6% in intermediately populated areas, and 10.8% in densely populated areas. In total, at least one pollinator was spotted during an observation 71.8% of times. This share is higher for thinly populated (77.3%) and lower for densely populated areas (43.2%) while intermediately popu- lated areas, the share was 67.6%. 65% of the observations were conducted during the predefined time slots and 14% during the time slots designated for group observa- tions. Looking at the number of observations per observation site, the median value of observations was 18, whereas the maximum number of observations per site was 52. In 71% of the sites, 100 or less pollinators were observed. Four sites reported more than 400 pollinators, with the highest number recorded at one location being 614. These four sites were all sites where group observations were taking place. In total, 5219 insects and other arthropods were recorded. Figure 2 shows the pie chart diagram of the pollinator groups recorded, with honey bees being by far the strongest group at 57%, followed by bumble bees, wasps, hoverflies, and solitary bees. At least 100 pollinators were recorded in each group. 2974 honey bees; 850 bumble bees; 303 wasps; 290 hoverflies; 234 solitary bees; 161 beetles; 116 butter- flies; 291 other. In about 6% of cases, the participants chose the category “other”. Several animals in this category were not pollinators. Even though they were not asked to identify the “other”, participants had noted 50 cases of ants, 40 cases of “some kind of small flies”, 29 spiders (in several cases identified as flower crab spi- ders), 21 insects identified as heteroptera species, 6 hornets (these technically belong to wasps but they are predators and not pollinators), 5 “large shiny green insects”, 2 grasshoppers, one mosquito, and one praying mantis. In 127 cases, “other” was cho- sen without specifying the insect. In only five cases the participants marked the pol- A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 13 Fig. 2. A pie chart diagram representing the shares of individual categories of pol- linator visits observed during the study. linator as “unknown”. The dominance of the honey bees in the pollination of sun- flowers has also been shown by other research (Bartua et al., 2018; Chabert et al., 2022; Terzić et al., 2017). Figure 3 shows the pie chart diagram of weather conditions during the observa- tions. Half of the observations took place in “sunny” weather, followed by “partially sunny”, while approximately 30% took place during bad weather (cloudy, rainy, fog). The bias toward observations in favorable weather conditions is understandable as the participants were instructed to count at their convenience. Out of all bad weather observations, 5% took place during group observation hours. 60.4% of bad weather observations occurred in gardens, 20.3% in garden pots, 16.8% in fields, and 2.5% in balconies. Observing bias in regard to certain days of the week or times of the day can influ- ence some citizen science data (Courter et al., 2013). Figure 4 shows the distribution of observation times by hour of the day and by day of the week. Clearly, the partic- ipants took some liberty with the observation times and did not always strictly follow the protocol, although four of the recommended time slots are stronger represented than the other times. We see that participants observed sunflowers slightly more fre- quently on Wednesdays than other days, most likely due to the group observing hours. Fridays were the least visited days. Observations took place between 6:00 and 21:00. Over half (65%) of the observations were made at the recommended time slots. The most common observing time slot was 17:00-18:00, followed by 11:00- 12:00. During the noon observing hours (11:00-12:00) participants more often made observations during weekends, but also took the time on group observing on Wednesdays. At the 17:00-18:00 time slot participants visited sunflowers most fre- quently on Wednesdays. Mondays and Fridays were least visited after working hours (17-18h).  Acta entomologica slovenica, 31 (1), 2023 14 Fig. 3. A pie chart diagram representing the shares of different types of weather recorded during the study. Analysis of observation data The data was normalized according to the number of observations, for example, the value at the 8:00-9:00 slot for honey bees represents the ratio between the number of honey bees observed in this time slot and the number of sessions in this time slot where honey bees (at least one) were observed. This approach was used for easier comparison between different categories. The following graphs (Figs. 5-7) inform on the relative presence of pollinators in the given categories. In the analysis of each vari- able, we only used the data where the participants reported the variable, for example, when looking at the weather, we excluded the observations where the participants did not record it (but we used the data in time analysis if time was provided). As seen in Figure 5, we recorded different patterns of day activity for different pollinators. There were two different patterns of activity. The first pattern was a peak in activity in the middle of the day (honey bees), the second was a relatively steady activity throughout the whole day (all other pollinators). In particular, the high activ- ity of the honey bee in the middle of the day is in line with other research (Bevk and Prešern 2021; Vicens and Bosch 2000; Willmer et al. 1994). One should note that the studies that we compare to were conducted following the methodology used in agro- nomical and biological studies.  Pollinators have different temperature preferences (Fründ et al. 2013; Terzić et al 2017; Wratt 1968). In our study, pollinators were observed through a wide span of temperatures, from 15 to 35 °C (Figure 6), here shown with the resolution of 2 °C. A strong influence of temperature on activity was observed in honey bees and wasps, with the Pearson correlation coefficient values of 0.96 and 0.75, respectively. For bumble bees and solitary bees, the correlation was slightly negative. In this figure, we omit the data for butterflies and beetles as not enough counts per temperature were recorded per temperature interval to make the interpretation reasonable.  A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 15 Fig. 4. Bar charts showing frequencies for observing hours (left) and days of the week (right). Acta entomologica slovenica, 31 (1), 2023 16 Fig. 5. Proportional number of pollinators by category depending on the part of the day. Solid lines should be viewed as a guide to an eye. Fig. 6. Proportional number of pollinators in five categories depending on the outside temperature. Data at 15 °C and 35 °C correspond to all observations below and above the said temperature, respectively. Solid lines should be viewed as a guide to an eye. The value of Pearson correlation coefficient between the normalized num- ber of pollinators and temperature is given in the figure legend for each category. Similar results as with the influence of temperature can also be seen with the weather (Figure 7). According to previous studies, the weather has a significant influence on the activity of pollinators, which is why pollinator monitoring usually takes place in sunny and dry weather (Carvell et al., 2015). According to our data, the honey bee is most active on average in sunny weather, but activity decreases in partly cloudy weather. To some degree, this is consistent with the temperature trend observed for honey bees - sunny weather in summer is typically associated with higher temperatures. The average activity of other pollinators drops only in cloudy weather, except for hoverflies, where the activity actually increases with cloudiness. As the numbers of pollinators recorded during rain and fog were small, these weather conditions are excluded from the analysis. A one-way ANOVA was conducted to compare the effect of different weather conditions on the activity of pollinators in sunny, partially sunny, and cloudy conditions. Despite of the averages seemingly being different, there was not a statistically significant effect of the means of differ- ent weather conditions on pollinator activity at the p<.05 level for the three condi- tions [F(2, 385) = 2.7, p = 0.07], [F(2, 230) = 1.6, p = 0.2], [F(2, 118) = 0.09, p = 0.9], [F(2, 148) = 0.4, p = 0.7], [F(2, 119) = 2.1, p = 0.1], [F(2, 60) = 0.8, p = 0.4], [F(2, 63) = 0.6, p = 0.6], for honey bees, bumble bees, solitary bees, hoverflies, wasps, butterflies, and beetles respectively. Different weather conditions had unequal variances and unequal sample sizes affecting the robustness of ANOVA i.e. over 50% of observations occurred during sunny weather, while statistical power of ANOVA is based on the groups with the least observation. A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 17 Fig. 7. Proportional number of pollinators by category depending on the type of the weather – reported for sunny, partially sunny, and cloudy. Solid lines should be viewed as a guide to an eye.  In Figures 8-10, we compare the pollinator communities based on geographical characteristics: population density (Figure 8), altitude (Figure 9), and between two geographical clusters (Figure 10). This type of analysis demonstrates a shortcoming of the CS approach. While Figure 8 shows pronounced differences in the pollinator communities, the total counts for the groups are roughly in the 1:7:40 ratio, in view of the number of observation this is 45:187:1015, meaning that the thinly-populated areas are heavily overrepresented against the other two, meaning that small local variations may become more pronounced. A parallel with the more diverse pollinator communities in thinly-populated areas can be drawn with the analysis based on the altitude (Figure 9), as well as on the region (Figure 10). Among the observation sites in our study, all sites below 199 m and above 400 m were located in thinly-populated areas. Similarly, the sites in Central Slovenia were a mixture of rural and urban areas whereas the sites in Southeast Slovenia were predominantly rural. As expected, in all cases, our findings are similar to other studies conducted in central Europe (Theodorou et al., 2020), where Hymenoptera e.g. bees and bumble bees, were more abundantly present than, in contrast with Diptera and Lepidoptera species. There are two main shortcomings in the study, namely the sampling/observation bias and the validity of pollinator identification by the participants. As demonstrated in the analysis based on the population density (Figure 8), a shortcoming was iden- tified in a dataset unbalanced to the level where it did not allow for a reasonable com- Acta entomologica slovenica, 31 (1), 2023 18 Fig. 8. Stacked column chart on pollinator communities for three levels of popu- lation density. The total count of pollinators was 100 in densely-populated areas, 765 in intermediate density, and 4063 in thinly-populated areas. A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 19 Fig. 9. Stacked column chart on pollinator communities based on the altitude of the site. The total count of pollinators was 1987 for <199 m, 2245 for 200-399 m, and 696 for >400 m. Fig. 10. Stacked column chart on pollinator communities for the two clusters. The total count of pollinators was 586 for Central and 3445 for Southeastern cluster. parison. Other types of analyses that would be of scientific interest but are not pos- sible to reasonably conduct with the present dataset include locations at higher ele- vations (500 m a.s.l. and above), sites located in different parts of the country, or in different climate types. Thinly populated areas were overly represented in our study which can partially be explained by the activities of study circles, and thus partici- pants in the project, mainly in the countryside. Similar observations can be made about the pollinators in different weather conditions, with a clear bias for pleasant weather as opposed to rain. These types of shortcomings are not unexpected in citi- zen science projects as there is typically observation bias present, as well as the fact that consistent involvement of the participants is not guaranteed in advance. In future studies, to alleviate this problem, we plan to put an additional focus on recruiting larger numbers of participants in areas that we want to focus on in particular and to provide clearer instructions if weather conditions or times of the day are to be further investigated.  As the participants in the study are not experts, they can often misidentify indi- vidual pollinators. Again, this is an expected shortcoming in CS projects for the scientific perspective. However, as mentioned in introduction, the added value of our approach is essentially related to sustainability: scientifically based learning mobilized observation, reflexive learning and green competences which is a prac- tical step towards social transformation towards sustainability. A concerted collec- tive action of diverse actors was focused to both, data quality, provided by addi- tional materials and training to the participants prior to the beginning of the obser- vations; and mobilization of non-formal learning. As demonstrated by the results of the quiz, participants had the most problems in correct identification of solitary bees and hoverflies, whereas the identification of wasps, butterflies, and beetles was less problematic. Their motivation was stable and enabled continued work. According to the confusion matrix in Table 1, the participants typically mistook category A for category B at a similar rate than B for A. In a balanced dataset, the misidentification rate would cancel out, however, this is not the case here. For example, if we count 100 honey bees and 20 solitary bees, at a 20% misidentifica- tion rate, this would result in 20 honey bees identified as solitary bees and 4 the other way around, resulting in an apparent count of 84 vs. 36. In our study, we have seven categories in the confusion matrix, so a proper compensation for misidenti- fication would have to consider all of them. To properly carry this out, we would require to conduct a quiz with a larger number of participants and a larger set of questions, in order for the confusion matrix to be more reliable. This is an interest- ing research problem for future work. Observation of pollinators is in line with UNESCO proposals (2014), development of green competences (Bianchi et al., 2022) and principles of non-formal learning. We will try to keep raising interest, overcome identified shortages of the CS approach and continue monitoring of pol- linators as an example of practical training for sustainable development. Acta entomologica slovenica, 31 (1), 2023 20 Nevertheless, identifying pollinators in real-life is likely easier, as one can also observe its behavior, size, and hear its buzzing. Conclusions We conducted a citizen science project where the participants were observing the pollinators visiting sunflowers. In total, we had 86 observation sites, 1862 individual sessions, and over 5000 pollinators counted.  Based on the results of the pollinator identification quiz conducted prior to the beginning of the observations, participants had most difficulties identifying solitary bees and hoverflies, whereas the accuracy of identifying wasps, beetles, and butter- flies was very high (even if these three categories represented a smaller share of the pollinator communities observed). To improve the accuracy in other categories, we provided additional material and training to the participants. In the country-wide analysis, we investigated the activity of different pollinators based on the part of the day, weather, and the outside temperature. In the analysis based on the geographical parameters, we compared the numbers and pollinator communities based on the population density, altitude, and between two geographi- cally separated regions. We found out that the honey bees are on average more active in sunny weather and at higher outside temperatures, which is consistent with previ- ous research (Bevk and Prešern 2021), whereas the effect was less pronounced for other categories. Dataset collected with citizen scientists produced reasonable results when compared to the data from traditional biological studies. Comparison of pollinator communities based on geographical characteristics demonstrated a shortcoming of the study, with densely-populated being severely underrepresented and thinly-populated overrepresented. This disallowed a reason- able comparison between the clusters, even if the results hint at more diverse polli- nator communities in rural areas in comparison with urban ones.  Based on the feedback from the participants, which will be in detail reported in a separate publication, we have sensitized the participants to both the new approach (observation, CS) and content (environment, pollinators) which is in line with inter- national frames for sustainable development. This presents solid foundations to con- tinue the observations of pollinators in future studies. Funding This project received financing from the Slovenian Ministry of the Environment, Climate and Energy from the Climate change funds. This study was also partly sup- ported by the Slovenian Research Agency (projects P1-0255, P1-0125, P2-0209, and V1-1938). Acknowledgments The authors gratefully acknowledge the participants involved in the campaign. A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 21 References Bartual, A.M., Bocci, G., Marini, S., and Moonen, A.C. (2018). Local and Landscape Factors Affect Sunflower Pollination in a Mediterranean Agroecosystem. PLOS ONE, 13, e0203990, https://doi.org/10.1371/journal. pone.0203990. Bevk, D. and Prešern, J. (2021). Pollinators Communities Differ across Years and Crops. Plant, Soil and Environment 67, pp. 600–607. https://doi.org/10.17221/ 225/2021-PSE. Bianchi et al. (ed.) (2022). GreenComp – The European sustainability competence framework. https://www.researchgate.net/publication/357961606_Green Comp_- The_European_sustainability_competence_framework Biesmeijer, J.C., Roberts, S.P.M., Reemer, M., Ohlemüller, R., Edwards, M., Peeters, T., Schaffers, A.P., et al. (2006). Parallel Declines in Pollinators and Insect-Pollinated Plants in Britain and the Netherlands. Science 313, pp. 351– 54. https://doi.org/10.1126/science.1127863. Bila Dubaić, J., Lanner, J., Rohrbach, C., Meimberg, H., Wyatt, F., Čačija, M., Galešić, M., Ješovnik, A., Samurović, K., Plećaš, M., et al. (2022). Towards a Real-Time Tracking of an Expanding Alien Bee Species in Southeast Europe through Citizen Science and Floral Host Monitoring. Environ. Res. Commun. 4, 085001, http://doi.org/10.1088/2515-7620/ac8398 Birkin, L. and Goulson, D. (2015). Using citizen science to monitor pollination ser- vices. Ecological Entomology 40(S1), pp. 3–11. https://doi.org/10.1111/ een.12227. Bloom, E.H. and Crowder, D.W. (2020). Promoting Data Collection in Pollinator Citizen Science Projects. Citizen Science: Theory and Practice 5(1), p.3. https://doi.org/10.5334/cstp.217. Bogataj, N. (2015). Community empowerment and transformative learning through study circles. In Lifelong learning devices for sustainable local development : the study circles experience in the cross border area Italy-Slovenia, Bogataj, N., Del Gobbo, G., Eds., Education, 1, ISBN: 978-88-467-4283-4. Gorizia, Italy, pp. 47–58. https://www.edizioniets.com/priv_file_libro/2779.pdf. Bonney, R., Cooper, C. B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K.V., and Shirk, J. (2009). Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience 2009, 59(11), pp. 977– 984. https://doi.org/10.1525/bio.2009.59.11.9. Brodschneider, R., Kalcher-Sommersguter, E., Kuchling, S., Dietemann, V., Gray, A., Božič, J., Briedis, A., Carreck, N.L., Chlebo, R., Crailsheim, K., Coffey, M.F., Dahle, B., González-Porto, A.V., Filipi, J., de Graaf, D.C., Hatjina, F., Ioannidis, P., Ion, N., Jørgensen, A.S., et. al. (2021). CSI Pollen: Diversity of Honey Bee Collected Pollen Studied by Citizen Scientists. Insects 12(11), p. 987. https://doi.org/10.3390/insects12110987. Acta entomologica slovenica, 31 (1), 2023 22 Carvell, C., Isaac, N., Jitlal, M., Peyton, J., Powney, G., Roy, D.B., Vanbergen, A., O’Connor, R., Jones, C., Kunin, B., et al. (2015). Design and Testing of a National Pollinator and Pollination Monitoring Framework, Final summaryre- port to the Department for Environment, Food and Rural Affairs (Defra), Scottish Government and WelshGovernment: Project WC1101, pp 1-62. Chabert, S., Mallinger, R.E., Sénéchal, C., Fougeroux, A., Geist, O., Guillemard, V., Leylavergne, S., Malard, C., Pousse, J., and Vaissière, B.E. (2022). Importance of Maternal Resources in Pollen Limitation Studies with Pollinator Gradients: A Case Study with Sunflower. Agriculture, Ecosystems & Environment 330, 107887, https://doi.org/10.1016/j.agee.2022.107887. Courter, J.R., Johnson, R.J., Stuyck, C.M., Lang, B.A., and Kaiser, E.W. (2013). Weekend Bias in Citizen Science Data Reporting: Implications for Phenology Studies. Int J Biometeorol, 57, pp. 715–720, https://doi.org/ 10.1007/s00484-011-0457-y. Deguines, N., Princé, K., Prévot, A.-C., and Fontaine, B. (2020). Assessing the emergence of pro-biodiversity practices in citizen scientists of a backyard but- terfly survey. Science of The Total Environment 716, 136842. https://doi.org/10.1016/j.scitotenv.2020.136842. Dickinson, J.L., Shirk, J., Bonter, D., Bonney, R., Crain, R.L., Martin, J., Phillips, T., and Purcell, K. (2012). The current state of citizen science as a tool for ecological research and public engagement. Frontiers in Ecology and the Environment 10(6), pp. 291–297. https://doi.org/10.1890/110236. Domroese, M.C. and Johnson, E.A. (2017). Why watch bees? Motivations of citi- zen science volunteers in the Great Pollinator Project. Biological Conservation 208, pp. 40–47. https://doi.org/10.1016/j.biocon.2016.08.020. Ebitu, L. Avery, H. Mourad K.A., and Enyetu, J. (2021). Citizen science for sus- tainable agriculture – A systematic literature review. Land Use Policy 103, 105326. https://doi.org/10.1016/j.landusepol.2021.105326. Fründ, J., Dormann, C.F., Holzschuh, A., and Tscharntke, T. (2013). Bee Diversity Effects on Pollination Depend on Functional Complementarity and Niche Shifts. Ecology 94, pp. 2042–2054. https://www.jstor.org/ sta- ble/23597326. Gardiner, M.M. and Roy, H.E. (2022). The Role of Community Science in Entomology. Annual Review of Entomology 67(1), pp. 437–456. https://doi.org/10.1146/annurev-ento-072121-075258. Karlsson, B. (2014). Extended season for northern butterflies. International Journal of Biometeorology 58(5), pp. 691–701. https://doi.org/10.1007/s00484-013- 0649-8. Klein, A-M., Vaissière, B.E., Cane, J.H., Steffan-Dewenter, I., Cunningham, S.A., Kremen, C., and Tscharntke, T. (2007). Importance of Pollinators in Changing Landscapes for World Crops. Proceedings of the Royal Society B: Biological Sciences 274, pp. 303–13. https://doi.org/10.1098/rspb.2006.3721. A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 23 Koffler, S., Barbiéri, C., Ghilardi-Lopes, N.P., Leocadio, J.N., Albertini, B., Francoy, T. M., and Saraiva, A.M.A. (2021). Buzz for Sustainability and Conservation: The Growing Potential of Citizen Science Studies on Bees. Sustainability 13(2), p. 959. https://doi.org/10.3390/su13020959. Kosmala, M., Wiggins, A., Swanson, A., and Simmons, B. (2016). Assessing data quality in citizen science. Frontiers in Ecology and the Environment, 14(10), pp. 551–560. https://doi.org/10.1002/fee.1436. Lander, T. (2020). Network modelling, citizen science and targeted interventions to predict, monitor and reverse bee decline. PLANTS, PEOPLE, PLANET, 2(2), pp. 111–120. https://doi.org/10.1002/ppp3.10068. Mason, L. and Arathi, H.S. (2019). Assessing the efficacy of citizen scientists monitoring native bees in urban areas. Global Ecology and Conservation 17, p. e00561. https://doi.org/10.1016/j.gecco.2019.e00561. Maund, P.R., Irvine, K.N., Lawson, B., Steadman, J., Risely, K., Cunningham, A.A., Davies, Z.G. (2020). What motivates the masses: Understanding why people contribute to conservation citizen science projects. Biological Conservation 246, p. 108587. https://doi.org/10.1016/j.biocon.2020.108587. McKinley, D.C., Miller-Rushing, A.J., Ballard, H.L., Bonney, R., Brown, H., Cook-Patton, S.C., Evans, D.M., French, R.A., Parrish, J.K., Phillips, T.B., Ryan, S.F., Shanley, L.A., Shirk, J.L., Stepenuck, K.F., Weltzin, J.F., Wiggins, A., Boyle, O.D., Briggs, R.D., Chapin, S.F., et. al. (2017). Citizen science can improve conservation science, natural resource management, and environmental protection. Biological Conservation 208, pp. 15–28. https://doi.org/10.1016/j.biocon.2016.05.015. Oberhauser, K. and LeBuhn, G. (2012). Insects and plants: Engaging undergradu- ates in authentic research through citizen science. Frontiers in Ecology and the Environment, 10(6), pp. 318–320. https://doi.org/10.1890/110274. Ollerton, J., Winfree, R., and Tarrant, S. (2011). How Many Flowering Plants Are Pollinated by Animals? Oikos 120, pp. 321–26. https://doi.org/10.1111/j.1600- 0706.2010.18644.x. Robinson, J.A., Bevk, D., Gradišek, A., and Bogataj, N. In preparation. Roy, H.E., Baxter, E., Saunders, A., and Pocock, M.J.O. (2016). Focal Plant Observations as a Standardised Method for Pollinator Monitoring: Opportunities and Limitations for Mass Participation Citizen Science. PLOS ONE 11(3), p. e0150794. https://doi.org/10.1371/journal.pone.0150794. Serret, H., Deguines, N., Jang, Y., Lois, G., and Julliard, R. (2019). Data Quality and Participant Engagement in Citizen Science: Comparing Two Approaches for Monitoring Pollinators in France and South Korea. Citizen Science: Theory and Practice 4(1), p. 22. https://doi.org/10.5334/cstp.200. Smith, M.R., Mueller, N.D., Springmann, M., Sulser, T.B., Garibaldi, L.A., Gerber, J., Wiebe, K., and Myers, S.S. (2022). Pollinator Deficits, Food Consumption, and Consequences for Human Health: A Modeling Study. Acta entomologica slovenica, 31 (1), 2023 24 Environmental Health Perspectives 130, 127003, https://doi.org/10.1289/ EHP10947. Terzić, S., Miklič, V., and Čanak, P. (2017). Review of 40 Years of Research Carried out in Serbia on Sunflower Pollination. OCL 24, D608, https://doi.org/10.1051/ocl/2017049. Theodorou, P., Radzevičiūtė, R., Lentendu, G., Kahnt, B., Husemann, M., Bleidorn, C., Settele, J., Schweiger, O., Grosse, I., Wubet, T., et al. (2020). Urban Areas as Hotspots for Bees and Pollination but Not a Panacea for All Insects. Nat Commun 11, 576, https://doi.org/10.1038/s41467-020-14496-6. Tucker, C., Šmid Hribar, M., Urbanc, M., Bogataj, N., Gunya, A., Rodela, R., Sigura, M., and Piani, L. (2023). Governance of interdependent ecosystem services and common-pool resources. Land Use Policy, 127, pp. 106575. https://doi.org/10.1016/j.landusepol.2023.106575 UNESCO. (2014). UNESCO roadmap for implementing the Global Action Programme on Education for Sustainable Development. https://unesdoc.unesco.org/ark:/48223/pf0000230514 Vasiliev, D., and Greenwood, S. (2021). The role of climate change in pollinator decline across the Northern Hemisphere is underestimated. Science of The Total Environment 775, p. 145788. https://doi.org/10.1016/j.scitotenv.2021.145788. Vicens, N., and Bosch, J. (2000). Weather-Dependent Pollinator Activity in an Apple Orchard, with Special Reference to Osmia Cornuta and Apis Mellifera (Hymenoptera: Megachilidae and Apidae). Environmental Entomology 29, pp. 413–420, https://doi.org/10.1603/0046-225X-29.3.413. Willmer, P.G., Bataw, A.M., and Hughes, J.P. (1994). The Superiority of Bumblebees to Honeybees as Pollinators: Insect Visits to Raspberry Flowers. Ecological Entomology 19, pp. 271–284, https://doi.org/10.1111/j.1365- 2311.1994.tb00419.x. Wratt, E.C. (1968). The Pollinating Activities of Bumble Bees and Honeybees in Relation to Temperature, Competing Forage Plants, and Competition from Other Foragers. Journal of Apicultural Research 7, pp. 61–66, https://doi.org/10.1016/j.agee.2022.107887. A. Gradiøek, J. A. Robinson, N. Bogataj, D. Bevk: Pollinators of sunflowers through citizen science: an adult education approach 25 Acta entomologica slovenica, 31 (1), 2023 26