Questions as a basis for comparison of biology textbooks and
workbooks
Vpra{anja kot izhodi{~e za primerjavo biolo{kih u~benikov in
delovnih zvezkov
Jelka STRGAR
University of Ljubljana, BF, Department of Biology, Ve~na pot 111, SI-1001
Ljubljana, Slovenia;
E-mail: jelka.strgar@bf.uni-lj.si
Abstract. This study focused on the analysis of three biology teaching sets and
one textbook for 8
th
and 9
th
grade of elementary school. As a criterion we used the
number and structure of questions assessable by Bloom’s taxonomy of the cogni-
tive domain.
It is obvious from the differences in the number and type of questions that the
authors of the teaching materials differed in their understanding of the role of text-
books and workbooks in the teaching process. The analysed teaching sets included
between 1.1 and 1.4 questions per page; most of them found in workbooks. The
exception was one textbook with no questions for the students. The analysis of the
questions using Bloom’s scale of cognitive levels showed statistically significant
differences between teaching sets. The average cognitive level of the questions in
a teaching set was between 1.7 and 2.1.
The questions represent only one of indicators that show to what extent the
didactic aspect was considered in the teaching sets, but the results suggest biology
teaching material for elementary school is overly focused on lower cognitive levels,
mostly recognition, not providing enough knowledge of higher cognitive levels.
Key words: textbooks, evaluation, Bloom’s taxonomy of the cognitive domain.
Izvle~ek. V raziskavi smo analizirali tri u~beni{ke komplete in en u~benik za
biologijo v 8. in 9. razredu osnovne {ole. Za kriterij smo izbrali {tevilo in strukturo
vpra{anj, ki jih je mogo~e vrednotiti po Bloomovi taksonomiji kognitivnih ciljev.
Pisci u~nih gradiv o~itno razli~no pojmujejo vlogo u~benika in delovnega
zvezka v u~nem procesu, saj so se pokazale razlike tako v {tevilu kot v tipu
vpra{anj. Obravnavani u~beni{ki kompleti vsebujejo od 1.1 do 1.4 vpra{anj na
stran, ve~inoma je ve~ji dele` vpra{anj v delovnih zvezkih. Posebnost je en
u~benik, v katerem ni vpra{anj za u~ence. Analiza vpra{anj po Bloomovi lestvici
kognitivnih stopenj je pokazala statisti~no pomembne razlike med u~beni{kimi
ACTA BIOLOGICA SLOVENICA
LJUBLJANA 2005 Vol. 48, [t. 2: 21-29
Sprejeto (accepted): 2005-09-27
stevilka 2_05.qxp  14.2.2006  9:33  Page 21
Acta Biologica Slovenica, 48 (2), 2005 22
kompleti. Povpre~na vrednost kognitivne stopnje vpra{anj v u~beni{kih kompletih se giblje med 1.7
in 2.1.
Vpra{anja so samo en pokazatelj didakti~ne obdelanosti u~beni{kega kompleta, vendar rezultati
ka`ejo, da so v osnovno{olskih biolo{kih u~nih gradivih preve~ poudarjeni ni`ji kognitivni cilji,
predvsem poznavanje, premalo pa je vi{jih kognitivnih ciljev.
Klju~ne besede: u~beniki, evalvacija, Bloomova taksonomija kognitivnih ciljev
Introduction
Biology in lower grades of elementary school forms part of various subjects, but in 8th and 9th
grade it becomes a separate subject. Teaching material for new programmes is issued by various pub-
lishers, which has its advantages as well as disadvantages. A wider choice gives teachers a chance to
select a teaching set they see best suited for their students and at the same time conforms to their
teaching methods.
There are several dilemmas surrounding the use of science textbooks. Should biology teachers
use textbooks or not? Should the textbooks be the focus of a course or just supplemental material?
Who should determine textbook content and what should this content be? (KIRK & al. 2001). It turns
out that a textbook is a valuable, and in many cases central, component of biology education that
contributes to achieving biology teachers’ goals. Many science teachers, new teachers in particular,
use the assigned textbook as their content outline and story line for their courses (CHIAPPETTA & al.
1993, BYBEE 2001, GARINGER 2001). A study, conducted on a sample of Slovene elementary school
teachers (VER~KOVNIK & STRGAR 2003), showed that for 60.0 % of teachers the textbook still repre-
sents a guide for their course and not a resource for unassisted student home work. Since textbooks
have been shown to have tremendous impact on curricula, this is one variable of science instruction
that needs further investigation (LLOYD 1990).
What constitutes a good science textbook and how can we identify it? The lists used in textbook
selection are long and include such items as content accuracy, clear definition of terms, end-of-chap-
ter questions, graphics and physical characteristics, pictures and diagrams, in-text laboratory activi-
ties, etc. (CHIAPPETTA & al. 1993). Others promote the evaluation of teaching material beyond sim-
ply their contents and instead focusing on cognitive and affective factors (GARINGER 2001). The
study by LOWERY & LEONARD (1978) examined the questioning style among four widely used high
school biology textbooks in terms of types, frequency, and placement of questions in textual reading
materials. It also examined the science/learning (inquiry) processes elicited by the questions. TAMIR
(1985) and ELTINGE & ROBERTS (1993) tried to assess the degree to which science was portrayed as
a process of inquiry in high school biology textbooks, as the opposite of a portrayal of science as a
collection of facts. ANDERSON & BOTTICELLI (1990) analysed textbooks based on explicit and implic-
it content analysis techniques. The purpose of LLOYD’S (1990) analysis was to identify text-based
concepts and to describe how these concepts are elaborated in three biology textbooks. CHIAPPETTA
& al. (1993) wanted to determine if the textbook under consideration reflects the themes of scientif-
ic literacy that science educators believe are important. CONRAD (1996) investigated linguistic vari-
ation in biology textbooks and articles. NINNES (2000) evaluated the ways in which secondary sci-
ence textbooks incorporate and represent indigenous knowledge. HARRISON (2001) investigated the
ways that models are used in school science textbooks. The study reported by KESIDOU & ROSEMAN
(2002) focused on examining how well science textbooks meet the Project 2061 content and instruc-
stevilka 2_05.qxp  14.2.2006  9:33  Page 22
23
tional criteria. DIMOPOULOS & al. (2003) analysed the pedagogic function of visual images included
in school science textbooks.
According to the Slovene biology curricula for 8
th
and 9
th
grade of elementary school, students
should learn in an active way; gain cognition through their own research and investigation; reveal the
point of the given content by collecting data from different sources; compare information and
employ critical thought; learn to analyse and make connections and generalizations. The curricula
therefore prescribes attaining not only data but also knowledge of a higher cognitive level. Based on
the finding that biology textbooks and workbooks have a central role in achieving the goals of the
curricula, our intention was to evaluate to what extent they contribute to these objectives. The num-
ber and structure of the questions set in textbooks and workbooks serve as criteria. It was hypothe-
sized that:
• There would be no differences among the texts in the ratio of questions per page.
• Bloom’s taxonomy of the cognitive domain would show no significant differences among the
texts.
The results would provide guidance for teachers. This kind of analysis should also be of help to
the authors of new textbooks and workbooks.
Material and methods
The study, conducted in 2003, focused on the teaching material (textbooks and workbooks) in
biology for 8
th
and 9
th
grade of elementary school. Their authors distributed content and categories
between textbooks and workbooks at their own discretion. From the student’s point of view we can
regard the textbook and workbook as a complete whole – a teaching set. At the time of the study, two
biology teaching sets for 8
th
grade were approved, as well as one teaching set and one textbook for
9th grade.
The list of the biology textbooks and supplemental workbooks analysed is as follows:
Textbook for the 8
th
grade of elementary school (Biologija 8. U~benik za 8. razred devetletne osnovne
{ole. Novak Bernarda. Ljubljana, DZS, 2000.)
Workbook for the 8
th
grade of elementary school (Biologija 8. Delovni zvezek za 8. razred devetletne
osnovne {ole. Novak Bernarda. Ljubljana, DZS, 2000.)
Textbook for the 8
th
grade of elementary school (Biologija 8. U~benik za 8. razred devetletne osnovne
{ole. Kralj Metka, Podobnik Andrej. Ljubljana, TZS, 2001.)
Workbook for the 8
th
grade of elementary school (Biologija 8. Delovni zvezek za 8. razred devetletne
osnovne {ole. Kralj Metka. Ljubljana, TZS, 2002.)
Textbook for the 9
th
grade of elementary school (Biologija 9. U~benik za 9. razred devetletne osnovne
{ole. Sve~ko Marina. Ljubljana, DZS, 2002.)
Workbook for the 9
th
grade of elementary school (Biologija 9. Delovni zvezek za 9. razred devetletne
osnovne {ole. Sve~ko Marina. Ljubljana, DZS, 2001.)
Textbook for the 9
th
grade of elementary school (Biologija 9. U~benik za 9. razred devetletne osnovne
{ole. Kordi{ Tatjana. Ljubljana, Modrijan, 2002.
*
)
*
The supplemental workbook had not yet been published at that time.
Jelka Strgar: Questions as a basis for comparison of biology... 
stevilka 2_05.qxp  14.2.2006  9:33  Page 23
Acta Biologica Slovenica, 48 (2), 2005 24
Textbooks and workbooks were evaluated by analysing the questions for students. FISH AND
GOLDMARK (1966) stated that the kinds of questions used in instruction determine the kinds of operations
student perform (as cited in LOWERY & LEONARD 1978). The value of particular types and usage of ques-
tions in textbooks depends on certain assumptions. If one starts from the assumption based on Piagetian
research that indicates direct and active experience is more valuable to students’ learning than indirect and
passive experience, then one would judge textbooks that ask questions within the experiential realm to be
more desirable than those that ask non-experiential questions. We can also assume that asking a high pro-
portion of questions is important or agree with many researchers that all forms of teaching (in this case,
textbooks) should make greater use of »higher level« questions (LOWERY & LEONARD 1978).
As a criterion we took Bloom’s Taxonomy of the Cognitive Domain that has already been used
by CHALL & CONARD (1991) and SKIERSO (1991) to assess the processes and skills textbooks require
learners to perform (as cited in GARINGER 2001). This method was chosen following positive expe-
rience in the preparation of tests for graduation from elementary school and graduation from gram-
mar school (baccalaureate). These require an adapted three-level Bloom scale which makes the pro-
cedure more practical while retaining adequacy, but we decided on a full six-level scale nevertheless.
All the questions assessable by taxonomy of the cognitive domain were collected in grid diagrams,
distributed into cognitive levels, and statistically processed. The rating of a textbook will directly
reflect the level of skills it demands. For example, a book that uses synthesis and analysis would rate
higher than one that demands only comprehension (GARINGER 2001).
The questions were sorted by two independent evaluators. Where their assessment differed, the ques-
tion was ranked after discussion and agreement. The questions were sorted into cognitive levels with
regard to other texts in the teaching material. Evaluated out of context, many questions would be sorted
into other categories. A question that was demanding at first glance, but was provided with an answer in
the preceding text, was placed in a lower category. If the answer was given in the following text, the ques-
tion was placed higher. Similar questions could therefore be arranged into different cognitive levels.
Important factors in the measurement of the effect of a textbook on pupils are the teacher’s use of the
textbook and his method of teaching. Experience teaches us that the course of teaching often encourages
students to learn by heart. In this context MARENTI~ PO`ARNIK (1992) stressed the importance of a well
designed education and training for teachers to prepare them for proper selection and creative use of text-
books. Our analysis assumed an ideal teacher who promotes unassisted work and thinking and does not
offer definite solutions, maintaining the potential value of the questions in the textbooks and workbooks.
Results and discussion
The teaching sets examined included between 1.1 and 1.4 questions per page (Tab. 1). A detailed
analysis showed that in the set EF this ratio was the same for the workbook and the textbook, while
sets AB and CD placed most questions in the workbooks. In Slovenia, textbooks are not designed so
as to allow them to be written in, so the results are not surprising.
An exception is textbook G that can be classified as a classic textbook on the basis of its structure,
and is at the moment the only one of that type among elementary school textbooks. A textbook can
encompass a whole range of levels, and can therefore be a textbook with a strong didactic component,
or, at the other end of the spectrum, a classic textbook with the content presented systematically and in
language and style suitable for target readers, but does not include special didactic techniques, such as
a motivating introduction, activity encouraging questions, knowledge testing questions, knowledge
stevilka 2_05.qxp  14.2.2006  9:33  Page 24
reinforcement questions, etc. Given that the workbook to accompany this textbook has not yet been
published, the suitability of such design cannot be assessed. Since textbook G does not include ques-
tions for students, which were the basic element of our work, it was not included in further analysis.
The hypothesis that there would be no differences among the texts regarding the ratio of questions
per page can therefore be refuted, since the results show that the authors of teaching material took vari-
ous approaches, which leads to the conclusion that they hold different views of the role of the textbook
in the teaching process. As MARENTI~ PO`ARNIK (1992) stated, each textbook reflects the author’s view
of the learning and teaching processes, his or her undestanding of knowledge and of the pupil’s abilities.
Tab. 2 shows the percentage of questions of individual cognitive levels in each of the teaching sets. The
three sets examined include between 201 and 323 questions that we were able to analyse using Bloom’s
taxonomy of the cognitive domain. The average cognitive level of the questions in the teaching sets was
between 1.7 and 2.1. The differences between percentages of questions of individual cognitive levels
among these three teaching sets are statistically significant (α=2P<0.1, α=2P<0.001). The results show that
most questions fit into the first cognitive level (49.3 – 60.2 %). It is interesting that both 8
th
grade sets (AB
and CD) have approximately the same percentage of questions of this level. All sets include considerably
fewer questions of the second cognitive level (15.3 – 25.4 %). There are approximately the same number
of questions of the third cognitive level in sets CD and EF (15.9 – 18.4 %), while the percentage in set AB
is substantially lower (7.7 %). Set AB deviates in the fourth cognitive level as well, but this time in the other
direction, since it contains as much as 17.3 % of the questions of this level, while the other two sets include
considerably fewer (4.9 – 6.5 %). There are very few questions of the fifth and sixth level (up to 3.0 %).
25Jelka Strgar: Questions as a basis for comparison of biology...  
Table 1: Ratio of questions per page in the teaching sets examined.
*The supplemental workbook had not yet been published at that time.  
Table 2: Distribution of questions of individual cognitive levels in the teaching sets using Bloom's scale.
stevilka 2_05.qxp  14.2.2006  9:33  Page 25
From the percentages of individual cognitive levels it is possible to deduce the teaching goals
achievable with these sets. The National Examination Commission in Slovenia prepares the biology
test sheets for the final examination for elementary school using an adapted Bloom’s scale of the cog-
nitive domain. It merged the 2
nd
and 3
rd
levels, and 4
th
, 5
th
and 6
th
levels, thus creating a three-level
scale instead of a six-level scale. The Commission reached an agreement that the national biology
examinations should include 35 % of questions regarding recognition, 50 % of questions regarding
comprehension and 15 % of questions regarding higher cognitive levels (Tab. 3).
Comparison of the National Examination Commission recommendations with the analysed
teaching sets shows that the differences in the percentages of the questions of individual cognitive
levels are substantial. There are approximately 15-25 % too many questions regarding basic knowl-
edge, which are first level questions (Tab. 4). At the second-level, concerning comprehension and
application of knowledge, the teaching set CD with 41.3 % comes closest to the recommendations
of the Commission. The other two sets include only around 30 % of these questions, which is 20 %
less than the Commision advises. The results regarding questions of the highest category are surpris-
ing. Set AB with 21.7 % of these questions is the only one that meets and even exceeds the recom-
mended percentage. There are too few of these questions in the other two sets (9.5 % and 6.1 %).
Questions for pupils are only one of the indicators that show to what extent the didactic aspect
was considered in the teaching sets. Other elements, which were not taken into consideration in our
study, are also very important. But the results show that elementary school teaching material puts too
much emphasis on lower cognitive levels, especially recognition. Basic knowledge is of course a
necessary foundation that students must acquire, but the teaching process should include a wider
scope of higher levels.
Acta Biologica Slovenica, 48 (2), 2005 26
Table 3: Evaluation scale of cognitive levels for national examinations. 
stevilka 2_05.qxp  14.2.2006  9:33  Page 26
27
Similar conclusions concerning textbooks were also reached abroad. ELLIOT & NAGEL CARTER
(1987) established that natural science textbooks adequately present natural science content, but do
not encourage scientific thought. The authors of that study think that the most important message of
the current natural science textbooks is that the natural sciences are a collection of findings that must
be learned by heart. The textbooks therefore stress the content and learning of facts and principles,
but there is little application and suitable activities to help the student understand nature and connect
natural sciences with the problems of the modern world. They summarized their findings into a claim
that the textbooks’ stress is largely on the products of the natural sciences and less on the character
of the natural sciences and its processes. A review of the 10 most popular high school biology text-
books in the US also found numerous facts, but little to explain the underlying scientific importance
of the facts. The evaluators rated all 10 textbooks poor in »demonstrating use of knowledge« and
»encouraging students to reflect on their own learning« (HOFF 2000).
Conclusions
The following are the most important findings:
• The teaching sets differ in number of questions per page. They include between 1.1 and 1.4.
The workbooks contain the majority of the questions.
• The average cognitive level of the questions in teaching sets is between 1.7 and 2.1, which indi-
cates an emphasis on recognition and a lack of higher cognitive levels.
• The difference in the structure of the questions of the teaching sets, analysed by Bloom’s tax-
onomy of the cognitive domain, is statistically significant.
• The authors of teaching sets have a different perspective of the role of teaching materials.
Povzetek
Izhajajo~ iz ugotovitve, da imajo biolo{ki u~beniki in delovni zvezki osrednje mesto pri dosegan-
ju ciljev u~nega na~rta, je bil na{ namen ovrednotiti, v kolik{ni meri lahko pripomorejo k doseganju
teh ciljev. Za kriterij smo izbrali {tevilo in strukturo vpra{anj v treh u~beni{kih kompletih in enem
u~beniku za 8. in 9. razred osnovne {ole. Predpostavili smo, da med besedili ne bo razlik glede na
{tevilo vpra{anj na stran, in da ne bo statisti~no pomembnih razlik glede na Bloomovo taksonomijo
kognitivnih ciljev. Vsa vpra{anja, ki jih je mogo~e vrednotiti po taksonomiji kognitivnih ciljev, smo
Jelka Strgar: Questions as a basis for comparison of biology...  
Table 4: Distribution of questions of individual cognitive levels in the teaching sets using an adapted three-
level scale.
stevilka 2_05.qxp  14.2.2006  9:33  Page 27
Acta Biologica Slovenica, 48 (2), 2005 28
zbrali v mre`nih diagramih, jih po zahtevnosti razvrstili v kognitivne stopnje in statisti~no obdelali.
Ocena je bila neposredno odvisna od nivoja zahtevnosti, ki ga u~beni{ki komplet omogo~a razvijati.
Rezultati ka`ejo, da so imeli pisci u~nih gradiv raznolik pristop. Ugotovili smo namre~, da
obravnavani u~beni{ki kompleti vsebujejo od 1.1 do 1.4 vpra{anj na stran. Ve~inoma je ve~ji dele`
vpra{anj v delovnih zvezkih. Posebnost je u~benik G, v katerem ni vpra{anj za u~ence. Tega zato
nismo mogli vklju~iti v nadaljnjo analizo. Po Bloomovi taksonomiji kognitivnih stopenj so se
pokazale statisti~no pomembne razlike med u~beni{kimi kompleti. Rezultati ka`ejo, da je najve~
vpra{anj prve kognitivne stopnje (49,3 – 60,2 %), kar je pribli`no 15-25 % preve~ glede na
priporo~ila Komisije za vodenje nacionalnih preizkusov znanja. Pri vpra{anjih 2. in 3. stopnje, ki
zajemajo razumevanje in uporabo znanja, se priporo~ilom Komisije najbolj pribli`a u~ni komplet CD
z 41,3 % takih vpra{anj. V drugih dveh kompletih jih je okoli 20 % manj premalo. V najvi{ji kate-
goriji, torej vpra{anjih 4., 5. in 6 stopnje pa komplet AB z 21,7 % presenetljivo dosega in celo pre-
cej presega `eleni dele` teh vpra{anj. V drugih dveh kompletih je teh vpra{anj premalo (9,5 % oz.
6,1 %). Povpre~na vrednost kognitivne stopnje vpra{anj v kompletih se giblje med 1.7 in 2.1.
Vpra{anja so samo en pokazatelj didakti~ne obdelanosti u~beni{kega kompleta. Zelo pomembni
so tudi drugi elementi, ki jih v na{i raziskavi nismo zajeli. Vendar dobljeni rezultati ka`ejo, da so v
osnovno{olskih biolo{kih u~nih gradivih preve~ poudarjeni ni`ji kognitivni cilji, predvsem pozna-
vanje, premalo pa je vi{jih kognitivnih ciljev. Tudi nekatere raziskave u~benikov v tujini so pri{le do
podobnih zaklju~kov.
Literature
ANDERSON O. R. & S. Botticelli 1990: Quantitative Analysis of Content Organization in Some Biology
Texts Varying in Textual Composition. Science Education 74: 167-182.
BYBEE R. W. 2001: Unintentional Consequences of an Unacceptable Evaluation. The American Biology
Teacher 63: 2-4.
CHIAPPETTA E. L., G. H. SETHNA & D. A. FILLMAN 1993: Do Middle School Life Science Textbooks
Provide a Balance of Scientific Literacy Themes? Journal of Research in Science Teaching 30:
787-797.
CONRAD S. M. 1996: Investigating Academic Texts with Corpus-based Techniques: An Example from
Biology. Linguistic and Education 8: 299-326.
DIMOPOULOS K., V. KOULAIDIS & S. SKLAVENITI 2003: Towards an Analysis of Visual Images in School
Science Textbooks and Press Articles about Science and Technology. Research in Science
Education 33: 189-216.
ELLIOT D. L. & K. NAGEL CARTER 1987: School Science and the Pursuit of Knowledge – Deadends and
All. Science and Children 24: 9-12.
ELTINGE E. M. &. C.W. ROBERTS 1993: Linguistic Content Analysis: A Method to Measure Science as
Inquiry in Textbooks. Journal of Research in Science Teaching 30: 65-83.
GARINGER D. 2001: Textbook Evaluation. Retrieved July 14, 2005, from http://www.teflweb-
j.org/v1n1/garinger.html
HARRISON A. G. 2001. How do Teachers and Textbook Writers Model Scientific Ideas for Students?
Research in Science Education 31: 401-435.
HOFF D. J. 2000. Science Group Bemoans Quality of Biology Textbooks. Education Week 19: 13.
stevilka 2_05.qxp  14.2.2006  9:33  Page 28
KESIDOU S. & J. E. ROSEMAN 2002: How Well do Middle School Science Programs Measure up? Findings
from Project 2061’s Curriculum Review. Journal of Research in Science Teaching 39: 522-549.
KIRK M., C. E. MATTHEWS & S. KURTTS 2001: The Trouble with Textbooks. The Science Teacher 68: 42-45.
LLOYD C. V. 1990: The Elaboration of Concepts in Three Biology Textbooks: Facilitating Student
Learning. Journal of Research in Science Teaching 27: 1019-1032.
LOWERY L. F. & W. H. LEONARD 1978: A Comparison of Questioning Styles Among Four Widely Used
High School Biology Textbooks. Journal of Research in Science Teaching 15: 1-10.
MARENTI~ PO`ARNIK B. 1992. U~beniki so namenjeni u~encem. In: @eljko A. (ed.): U~beniki danes in jutri:
prispevki s sre~anja avtorjev u~benikov DZS, DZS, Ljubljana, pp. 19-26.
NINNES P. 2000: Representation of Indigenous Knowledges in Secondary School Science Textbooks in
Australia and Canada. International Journal of Science Education 22: 603-617.
VER~KOVNIK T. & J. STRGAR 2003: U~benik kot sredstvo za doseganje vzgojno-izobra`evalnih ciljev pro-
grama. Evalvacijska {tudija. Ministrstvo za {olstvo, znanost in {port, Ljubljana, 81 pp.
TAMIR P. 1985: Content Analysis Focusing on Inquiry. Journal of Curriculum Studies 17: 87-94.
29Jelka Strgar: Questions as a basis for comparison of biology...  
stevilka 2_05.qxp  14.2.2006  9:33  Page 29