110 CONTEMPORARY PEDAGOGY 5/2008
Dr Janez Vogrinc
The importance of triangulation for ensuring the quality of scientific findings of the qualitative research
Abstract: In the paper the qualitative research in which the researcher has been directly involved and has himself been examining the research phenomenon in the studied environment is presented. The aim of this qualitative study is to gather data in the form of rich content-based descriptions of people, events, and situations by using different, especially non-structural, techniques, to discover the stakeholders' views and similar, to orally analyze the gathered data, and finally to interpret the findings in the form of a concept or contextually dependent grounded theory. The purpose of the paper is further to analyze the applied criteria to assess the quality of scientific findings established with the qualitative reasearch, especially triangulation, which is a combination of different methods, techniques, data sources, researchers, theories and scientific disciplines within the same research. Triangulation is defined as a strategy to ensure the quality of scientific findings established with the qualitative research.
Key words: qualitative research, triangulation, criteria for defining quality of qualitative research, grounded theory
UDK:
Dr Janez Vogrinc, University of Ljubljana, Faculty of Education; e-mail: janez.vogrinc@guest.arnes.si CONTEMPORARY PEDAGOGY 5/2008, 110-124
111
Introduction
In the field of humanities and social humanistic sciences, two paradigms of scientific research were developed in the past, i.e. the quantitative and the qualitative one, depending on their attributes. In the paper, the expression »paradigm« is used in the sense of Kuhn's contemporary definition of the scientific paradigm. According to Kuhn, paradigms are »the series of reciprocally connected assumptions about social phenomena, providing the philosophical and notional frame for studying them« (Kuhn 1974, p. 39). Therefore, the paradigm is the sum of values, convictions, assumptions telling us which values, beliefs, convictions, assumptions, laws etc., regarding the research within the scientific discipline, are shared by the adherents of a certain scientific paradigm. In accordance to them, they form their tradition of scientific research. The criteria for assessing the quality of findings established in the research process should be in line with the paradigm a certain research form is attached to. As another form of scientific research than it is used in the quantitative research is introduced by the epistemological baese of the qualitative research, also the criteria of assessing findings deriving from the qualitative and the quantitative research should be different.
In the paper the focus will be on the qualitative research, the basic characteristics of which as well as the applied criteria for assessing the quality of scientific findings established with it will be analyzed. Special attention will be paid to triangulation. In conclusion triangulation as the criterion of assessing the quality of scientific findings established with the qualitative research will be critically examined as regards its epistemological characteristics; further, triangulation will be justified as a strategy of ensuring the quality of scientific findings established with the qualitative research, and not as one of the specific criteria of assessing the quality of findings established with the qualitative research, as well.
112 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
Basic Characteristics of Qualitative Research
The qualitative research regarding its ontological, epistemological and methodological aspect is not a consistent phenomenon; namely, it combines different kinds of research, e.g. a case study, life history, action research and the like. Bogdan and Biklen (2003, p. 2) use the term »qualitative research« as the superordinate concept, joining different research approaches with certain common characteristics as well. With the expression »qualitative research« the research is denoted consisting of the basic empirical material, collected in the research process, which is verbally described or narrated. Furthermore, the collected material is worked on and analyzed in words without numerical operations (Mesec 1998, p. 26). In other authors, (e.g. Denzin and Lincoln 2000, Creswell 1998) similar definitions of the qualitative research are found. According to Creswell, the qualitative research is the research process designed according to a clear methodological tradition of research, whereby researchers build up a complex, holistic framework by analyzing narratives and observations, conducting the research work in the habitat (Creswell 1998, p. 15). Fraenkel and Wallen (2006, p. 430) draw attention to the fact that qualitative researchers mainly focus on the examination of characteristic traits or properties of a certain activity, group, situation, materials, respectively, but they are not much interested in the frequency of appearance of this activity, group, situation, or material. "Qualitative« research is an exploratory approach emphasizing words rather than quantification at gathering and analyzing the data. It is a matter of the inductive, constructivist and interpretative exploratory approach with the following main stresses: to view the world with the eyes of the examinees, to describe and take into account the context, to emphasize the process and not only the final results, to be flexible and develop the concepts and theories as the research process outcomes (Bryman 2004, str. 266)."
To summarize, for the qualitative research it is chracteristic that data are gathered more in a verbal and visual than in a numeric form. At analyzing the gathered data statistical procedures are also not used, but predominantly the qualitative analysis, the essence of which is searching for codes in the analyzed materials (Bryman 2004, p. 392). The main part of the qualitative analysis of the material is formed by the coding process namely, i.e. interpreting the analyzed text and attributing the meaning (of key words, notions, codes) to its individual parts (Charmaz 2006, p.. 46, Bryman 2004, p. 402, Flick 1998, p. 179), respectively. Qualitative analysis of the material starts with defining the coding units, followed by the appropriate phenomena records according to our judgement and analyzing the characteristics of these phenomena, and ends with the development of the grounded theory.1. The grounded theory is read out as a nar-
1 »Utemeljena teorija« is a Slovene translation of the expression "Grounded Theory". In Slovenian professional literature the collocation is translated differently, e.g. "grounded,", "basic", "subject", "basement", "inductive" and the like. In her dissertation Mazgon (2006) translated it as "a subject developed theory". Glaser and Strauss (1967, p. 5) stated that the grounded theory was deduced from data and then illustrated by quoting examples of significant data.". Their definition implies that the grounded theory is developed in an inductive manner and justified with data.
The importance of triangulation for ensuring
113
rative about the phenomenon, which was the subject of the study. It is characteristic for the theory to be constructed from the collected data and to develop in the course of the entire research process. The grounded theory is contextually bound, i.e. it is not a general theory (the findings cannot be generalized without additional definitions), but the theory of a narrower scope, valid only in certain environments and certain conditions, respectively.
Qualitative empirical research is oriented towards examining individual cases (idiographic approach). The study is mostly conducted as a study of one case only or a smaller number of cases, therefore the techniques of data collection are adjusted to a small scale analysis, enabling the researcher to get to know the social environment. At data collection one is not limited to one source or one technique only. Apart from the data acquired by interviews and observation, usually also different documentary sources are used, such as personal documents (a birth certificate, an employment record, a passport, letters, photos...), different records produced in the process of data collecting, transcriptions of tape recordings, video shots, etc. Only the pluralism of data collection techniques and their mutual combination can provide for linking the findings of individual phenomena or aspects into a meaningful integrity. The qualitative research is carried out in line with the principles of the interpretative paradigm, i.e. the focus is on examining the subjective experiences of an individual and on recognising the importance which the individual attaches to specific events, whereby not even the subjective views of the researcher of the studied situation are neglected. The aim is integrated and detailed cognition of phenomena, preferably in natural and concrete circumstances, for the researcher is interested in the context of the pursued activities. As part of the environment, the researcher is not only able to understand what the person is conveying in a form of a rational message and standardized speech, but also the indirect implications of this speech with a specific syntax, contextual lapses, hidden meanings and speech breaks are perceived. Wishes, expectations, interests, needs and personal opinions of the people included into the research should help the researcher to better comprehend the examined phenomena. In this context, the researcher should be aware of the fact that with his or her participation and with the researched situation itself, he or she is influencing the events he or she is observing, and the discursive reality, as his/her research object.
It is important that also the criteria of assessment of scientific findings established with the qualitative research are in line with epistemological bases of the qualitative research.
Criteria to Assess the Quality of Scientific Findings Established with the Qualitative Research
The quality of the quantitative research is usually defined by the notions, such as reliability, validity, objectivity and sensitivity. If the properties of the applied measuring instruments are satisfying, the research findings should pro-
114 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
vide as detailed and consistent image as possible of the objective reality. Sagadin (1993) described the properties of measure applied in the quantitative research by means of knowledge tests. In analogy to this, the characteristics of measure apply to other data collection instruments as well. As to the validity he stated: »Test validity describes its quality to measure exactly what it is supposed to measure; a test is valid in as much as it measures exactly this.« (ibid. p. 73). »The instrument is reliable or consistent if the results or data obtained are the same when it is applied again with the same individuals." (ibid, p. 76). Sagadin defines »the test objectivity«, on the basis of different aspects, i.e. the objectivity of testing (carrying out a test is objective if its results are not influenced by the subjective factor of a testing person), the objectivity in evaluating responses (a test is objective when different assessors equally evaluate the same response) and the objectivity of interpretation (the interpretation of the test outcomes should not depend on the subjective judgement of the interpreter; the same results should be interpreted in the same manner). Apart from the aforementioned characteristics within the quantitative research the sensitivity of the instrument is often mentioned. The sensitivity of a knowledge test increases if it determines the smaller possible differences in knowledge of individuals (Tolicic, Zorman 1965). The test sensitivity is presented by the scope of dispersion of test results around the mean value.
From the eighties of the previous century onwards numerous authors (such as Luders and Reichertz 1986, Lincoln and Guba 1985, Flick 1998) posed the question whether it was possible to transfer traditional criteria established in the quantitative research into the qualitative research regarding the fact that both research paradigms deal with comprehending reality and its examination accordingly, yet in a totally different manner. Glaser and Strauss (1967) expressed their doubts about the applicability of the quantitative criteria as criteria to assess the credibility of theories based on the data obtained from the qualitative research. They proposed that criteria of assessment covering all levels of the research process (collection, analysis, interpretation and presentation of data) should be based on common characteristics of the qualitative research. Drawing on this doubt there were several attempts to establish appropriate criteria for the qualitative research substituting for the validity and reliability criteria.
The authors working on qualitative criteria can be classified into four groups regarding their different approaches. (1) Positivistic approach in qualitative research is defended by representatives in favour of applying the same criteria as in the quantitative research. They justify it by pinpointing mainly internal and external reliability and internal and external validity. Internal validity (Mason 1996, Mesec 1998, Silverman 2005) marks the level of agreeement among different researchers on what they saw and what they heard and the level of their harmonization at analyzing the collected data. External reliability (LeCompte and Goetz 1982) applies to the possibility of repeating the research. A good internal validity (Flick 1998, LeCompte and Goetz 1982, Sagadin 2001, Mesec 1998) is shown by the reserarch in which the findings are justified in the examined situation. External validity (LeCompte in Goetz 1982, Sagadin 2001,
The importance of triangulation for ensuring
115
Mesec 1998) is mainly related to the possibility of generalizing the findings from the examined situation to other situations. (2) Postpositivistic approach supports the standpoint that qualitative empirical research is a different form of the scientific research from the quantitative research, therefore different methodological approaches should be employed. The analysis of the proposed criteria shows that in the majority of cases the issue is mainly about different description of the criteria existing in the quantitative research. So, for example Moser (1977) mentions transparency demanding a systematic description of the whole research process, which is the alternative to the inner validity; harmonization within which the compliance among goals, theoretical bases and methods of research work is examined and is the alternative to reliability; and, the influence of the researcher, with which the subjective influence of the researcher on the data collection procedure and on the whole research process is examined, and which is the alternative to objectivity. Lincoln and Guba (1985) mention credibility, relating to the fact whether the researcher managed to hold different views on the examined situation and draws such a conclusion with which individuals included in the examined situation will agree and, which is the alterantive to internal validity; transferability, on the basis of which it could be estimated whether it is possible to transfer the findings of the certain research into other environments, and which is the alternative to external validity; consistency, with which by reviewing the whole research it is examined whether the procedures were correctly carried out, whether they are supopported by the gathered material, whether at repeated analysis of the collected data the researcher would come to the same conclusions, which is the alternative to reliability; and the possibility of validation relating to the fact whether also other researchers would come to the same conclusions when reviewing the research, and which is the alternative to objectivity. Steinke (1999) mentions the criterion of intersubjective reproduction enabling critical communication on empirical research among researchers and readers, and the criterion of coherence that should be ensured during the whole research process, the mentioned criteria being alternatives to internal validity. The indication criterion, against which suitability of the applied research procedures is estimated, is an alternative to reliability. The criterion of reflected subjectivity according to which one should think about the extent to which the researcher's subjectivity influenced the developed theory and the criterion of empirical grounding according to which it is estimated whether the developed theory is supported by the collected materials, are alternatives to objectivity. The limitation criterion referring to the possibility of generalizing the findings to other circumstances and the criterion of relevance at which the applicative value of the research findings is estimated, are both alternatives to external validity.
Although the criteria of assessing the quality of scientific findings established with the qualitative research were named differently by the authors supporting the postpositivistic approach within the qualitative research, the mentioned criteria are taken over from the quantitative research. Therefore also at the level of methodological theory the fact is often overlooked that quali-
116 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
tative research is based on different epistemological assumptions than quantitative research. Analytical induction, denoting the approach to the data analysis and assuming that it is possible to universally explain the research problem, i.e. to explain all the researched problems, as well as triangulation, the role and importance of which for qualitative research is analyzed in detail in this paper, are also classified into the postposivistic criteria group. (3) The postmodernist approach rejects the possibility of defining any criteria of quality assessment of the qualitative research findings at all. The supporters of this approach are convinced that the sole idea of assessing the qualitative research is contrary to the nature of the qualitative research. "If we perceive the social reality as a constantly changing reality, than there is no need to find out whether our research instruments measure accurately" (Marshall and Rossman 1989, p. 25). Scheurich (1997) and Smith (1993) are convinced that validity should be radically changed and harmonized with the basic characteristics of phenomenological research if it is to be kept in the qualitative research (4) The post-structuralist approach is to develop a wholly new set of criteria, deriving from the qualitative research (cf. Hammersley 1992, Lincoln and Denzin 1994). According to Lincoln and Denzin (1994) the quantitative research methodology should be taken into account at designing the qualitative research, including subjectivity, feelings and other factors which are neglected by the quantitative research. Lather (1993) depicted validity as a multiple, partial concept, that could never be fully captured, assuming the following four forms: the ironic validity, the paralogic / neo-pragmatic validity, the rhizomatic validity and the sensual validity. The ironic validity expounds the problem of representation, as all reality representations supposedly lack real bases, being only rhetoric. The paralogic (neo-pragmatic) validity presumes the goal of scientific research not to be the communication with realitty, but rather defining the differences and describing the contrasts. In his endeavours the researcher mainly focuses on the establishment of hete-rogeneousness, disagreements and multiple discourses. The validity of findings can also be tested by stating the extent to which the mentioned goal was achieved. The rhizomatic validity highlights the number of viewpoints included in the interpretation and enables the establishment of new, contextually bound criteria (i.e. criteria bound to individual research). The sensual validity is part of feminist discussions on objectivity, its predominant concern being the difference between the male and female aspect; the former assuming the possibility to describe the society from the viewpoint of an objective observer, whereas the latter allows »imperfection«, effort taking, and provides for accepting the views of others and for combining the partial views into the mutual integrity.
In continuation of the paper the focus will be on triangulation, one of the most frequently applied criteria of assessing the quality of scientific findings established within the qualitative research.
The importance of triangulation for ensuring
117
Definition of Triangulation
According to the conventionally accepted definition, triangulation is »the use of multiple methods in the study of the same object« (Denzin 1978, Richardson 2003, Bryman 2004). In Slovenian educational standard books on methodology, the term »method« is used for the level of studying the educational field (Sagadin 1993, p. 12). When classifying the educational research methods according to »active - manipulative« and gnoseologic criteria, the author states the descriptive method, causal - nonexperimental and causal - experimental method. For a survey, an interview and suchlike, the author employs the term »technique«. In foreign standard books, the term »method« is used: (1) in the sense of the research type (e.g. qualitative research), (2) in the sense of the level of getting to know the researched field (e.g. the descriptive method), as well as in the sense of (3) the stage of the research, e.g. collecting data (e.g. survey method, interview method) (cf. Cohen and Manion 1990).
The spade-work where triangulation employment was represented for the first time was published in 1959 by Campbell and Fiske, the experimental psychologists. They introduced the multimethod - multitrait matrix, in which they employed several quantitative techniques, by means of which they measured psychological characteristics of the studied persons. In this way, they wanted to prove that the dispersion of data is the consequence of the studied characteristics and not the consequence of the applied techniques (Tashakkori and Teddlie 1998, p. 41). In social sciences, triangulation was first used as a technique for checking the validity of the research findings (Flick 1998, Tashakkori in Teddlie 1998, Neuman 2003, Bogdan and Biklen 2003, Richardson 2003, Bryman
2004,	Stake 2005), based on the belief that we could reject or acknowledge the research hypotheses only if we had come to the same conclusions by means of different methods. Nevertheless, later, the importance of triangulation, as well as its employment, increased significantly.
Denzin (1978) extended the notion of triangulation, saying that triangulation of methods is only one form of triangulation. In his opinion there are also data sources triangulation, the investigator triangulation and the theory triangulation (about this also: Flick 1998, Tashakkori and Teddlie 1998, Neuman 2003, Janesick 1998). Janesick (1998, p. 47) added the fifth triangulation form, namely the scientific discipline triangulation. The comprehension that triangulation is not merely a technique for validating the scientific findings, but that it also provides for more thorough understanding of each researched phenomenon, was increasingly extended. »Triangulation is not a tool or a strategy of validation, but an alternative to validation. The combination of multiple methodological practices, empirical materials, perspectives, and observers in a single study is best understood, as a strategy that adds rigor, breadth, complexity, richness, and depth to any inquiry (Flick, 1998, p. 230, Denzin and Lincoln
2005,	p. 5).
We speak about data sources triangulation when researchers, observing a particular object of the research, use as many different data sources as possible.
118 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
Denzin (1978, p. 295-297) distinguishes data sources from methods of data generation. The collection data methods refer to research methods per se, whereas by triangulating data sources, researchers can employ the same method in different situations to get a more detailed insight into the observational problem (e.g. If the researcher wants to get information on the particular school climate he can interview teachers, pupils, social school workers, the headmaster and/or parents. Each of the interviewees will express his or her opinion about the researched problem. The combination of all the interviewees' answers will provide the researcher with a significantly better insight into the observational problem than he could get with only one interview).
By triangulating data sources we discern three subtypes, namely time, space and person, which are interrelated. The level of one of them demands the study of the other two. A focus on space and time as observational units implies their relationship to the observations of persons. Thus, the researcher can study the examined problem at different times of a day, a week, a month, or a year. Besides, units of observation can be different places with ongoing activities. With personal analysis, we can distinguish three levels, namely (1) the aggregate one, (2) the interactive one, and (3) the collectivity.
(1)With the aggregate analysis, we examine an individual and not a group, relationships between individuals, organizations... (2)With the interactive analysis, the observational unit is an individual as an interacting person. Thus, the focus is neither on an individual nor on a group, but on the interactions of this individual. (3)With the collectivity, the observational unit is a group, an organization, a community, or an entire society. Individuals and their interactions are examined only in as much as they reflect the characteristics of the whole group.
At the investigator triangulation multiple researchers are included into the observation of the research problem. In researches, there is mostly a research team, formed to examine the problem, each of the researchers playing a different role in the observational process. At this triangulation, multiple researchers have the same roles, performing the same tasks. Finally, their findings are compared and completed. It is more difficult to implement the triangulation of researches due to the lack of sufficient funding; at the same time it is sometimes difficult to ensure a higher number of researchers that would otherwise deal with the same field of expertise in different scientific disciplines.
Theory triangulation is employed when examining the observational problem with multiple theoretical assumptions. Theory triangulation is necessary mainly in the fields of research in case of theoretical discordance. Theory triangulation means the use of different hypotheses when planning particular stages of the research. However, different hypotheses are often employed only in the interpretation of data. The use of theoretical triangulation reduces the possibility of researchers' premature acceptance of hypotheses since every hypothesis has to be confronted with all the other assumptions, also with the contradictory ones. The triangulation procedure should begin with making an extensive list of assumptions about a certain researched problem (including the possible in-
The importance of triangulation for ensuring
119
terpretations for each of the assumptions), perceived from different theoretical aspects that are empirically tested in the further course of research. In the end the list of assumptions is devised, that were validated or not with the empirical test; thus theories are evaluated and redesigned, on the basis of which the assumptions were made. The final report is usually a combination of assumptions contradicting each other at the beginning. The theory triangulation encourages the continuity in theory and research (apart from the evidence confirming the assumptions also the counterevidence should be sought to be examined in the further course of research) (more on the topic: Denzin 1978, p. 297-301).
The review of standard books treating methodological triangulation shows that methodological triangulation is viewed as the combination of different kinds of researches (Tashakkori and Teddlie 1998, Neuman 2003, Patton 1990, Bryman 2004), as well as the combination of different data collection techniques (Denzin 1978, Flick 1998, Morse 1998, Richardson 2003).
Neuman (2003, p. 139, also Morse 1998, p. 66) distinguishes two different ways of combining methods, namely (1) the sequential method combination and (2) the parallel method combination. (1) At the sequential method combination the methods are consecutively employed (we begin with the qualitative part of the research having finished the quantitative part of it; only after we have collected data by means of an interview, a survey questionnaire is produced). (2) The parallel method combines simultaneous employment of both methods (the qualitative and the quantitative part of the research are carried out simultaneously; a survey questionnaire and an interview are employed at the same time).
To the above mentioned ways of combining methods, described by Neuman, Creswell (1995, p. 177) another two ways of method combination are added, namely (3) the equivalent combination method and (4) the dominant - less dominant combination method. (3) At the equivalent method combination both employed methods occupy equivalent positions regarding the whole research (the findings acquired by means of the qualitative research are as important as the findings acquired by means of the quantitative research). (4) The dominant - less dominant method combination implies that one employed method has a more important role in the whole research than another employed method (e.g. the findings of the qualitative research are used only as an additional interpretation for findings acquired by means of the quantitative research; the answers from the interview are included only into the interpretation of the results obtained from the survey questionnaire). Tashakkori and Teddlie (1998, p. 18) point out that one method is not necessarily in an inferior position all the time, during the whole research, while the other one is occupying a superior position. The position of a particular method depends on the stage of the research. As a result, they add the fifth way of combining methods: (5) designs with a multilevel use of approaches, whereby the superior/inferior way of the method depends on the momentary stage of the research (e.g. observing pupils, more attention is paid to the results obtained from interviews, whereas the results obtained by means of the survey questionnaires only provide for the additional dimension. Howe-
120 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
ver, in the research stage, in which we acquire data from teachers, our major information source is the survey questionnaire, whereas findings acquired by means of interviews are employed as a supplement).
The discipline triangulation means that a research problem is examined in an interdisciplinary way. Discipline triangulation is much connected to the theory triangulation represented by Denzin (1978), although a discipline has a more extensive meaning than theory. The discipline is a branch, a field of science, whereas the theory is a sum of logically connected findings, attitudes or assumptions, explaining something in a scientific way within a particular discipline. The assumption of the discipline triangulation is investigator triangulation. If investigators belonging to different disciplines (psychology, pedagogy, phylosophy, sociology, art ...) are included into the study of a research problem, the observational unit can be comprehended significantly better and in a more comprehensive manner. Each scientific discipline produces specific "glasses", through which the studied problems are observed. Different conceptual framework enables posing different questions, finding different manners of responses to them, coming to different conclusions and creating an overall understanding of the researched problem.
The Role of Triangulation in Qualitative Research
Triangulation is a strategy enabling researchers to understand the observational object significantly better and in a a more comprehensive manner. Multiple triangulation, assuming the combination of multiple triangulation forms, i.e. the triangulation of investigators, theories, data sources, methods and/or disciplines, provides for the exhaustive data interpretation. However, there is a question whether triangulation can also be used as the criterion of assessing the quality of scientific findings established with the qualitative research. If multiple methods, investigators, data sources, theories and disciplines result in the same findings, we can assume that we have achieved valid data. Triangulation can certainly not be considered as the criterion of assessing the scientific findings quality of the qualitative research only when findings confirm each other (Bloor 1997). But what if, for example, two researchers observing the same empirical unit come to different conclusions? Are their findings invalid?
Triangulation as the criterion of assessing the quality of scientific findings obtained from the qualitative research was also subjected to criticism. The majority of critics can be classified into two groups. 1) Post-modernistically oriented researchers reject triangulation as the criterion against which the quality of findings obtained from the qualitative research should be assessed, as they do not agree to the reference point of "truth". Postmodernists are changing the traditional view on the validity. L. Richardson (1997, 2003) proposed a transgressive form of validity, which she defined in form of a metaphor by means of a chrystal shape. The chrystal is a prism, reflecting and breaking the beams thus constantly projecting the changing images of reality. The image we see depends on our
The importance of triangulation for ensuring
121
view. For this reason the postmodernists do not agree to there being only one true story; there are many aspects of the same story, and they are all real (cf. Richardson 2003, p. 517, 518, Stake 2005, p. 454). On the basis of this assumption it can be inferred that all perceptions of the researchers and different sources of findings as well as all findings acquired by individual research approaches and data collection techniques are important and equal. As such it is possible to apply them as the criterion of assessing the quality of findings of indvidual researchers and the findings, respectively, that were acquired by individual approaches and techniques of data collection; however, it is reasonable to combine them mutually, thus linking the findings on individual notions or aspects in a coherent entity. 2) The second group of critics is composed of the researchers who believe in basing the triangulation on epistemological assumptions of the quantitative research which makes it contradictory to the basic characteristics of the qualitative research. Using triangualation as the criterion of assessing the quality of scientific findings obtained from qualitative research would mean comparing the results obtained with different research approaches and data collection techniques, respectively, and comparing the findings of different researchers and the data provided by different information sources. The comparisons would be drawn in order to examine the quality of findings obtained by applying one method (a technique, a researcher, a source of information) (cf. Tashakkori ans Teddlie 1998, p. 82-84). In this way, by comparing the findings, the quality of findings obtained in the quantitative research is established, which is based on the assumption, that hypotheses successfully confirmed by many tests are more valid than the ones subjected to one test only. Such a manner of research also assumes that it is possible to „measure« the same empirical unit more than once. All the aforementioned is contrary to the basic principles of the qualitative research. If a certain notion is placed within a single and specific context, it is, strictly speaking, not possible to be „measured« (observed, protocolled) (more on the topic by Kogovšek 1998). Triangulation as the criterion to assess the quality of findings is thus based on epistemological assumptions of the quantitative research.
Lincoln and Guba (1985, p. 370) point out that it is wrong to expect one researcher's observations to confirm another researcher's observations since the essential instrument of qualitative research is just the researcher himself or herself. The research approaches and views are different due to different disciplines and theoretical assumptions. Consequently, all this may also be the reason for the incoherence of the researchers' findings. The purpose of combining conclusions drawn by different researchers, in view of the qualitative research, cannot be the judgement of observations adequacy made by only one researcher, but the formation of the detailed insight into the observational problem. This is also the purpose of including more researchers into the research. Different data sources usually reflect different views of the same phenomenon. For example, if we interview more persons, each of them may tell us a slightly different story despite describing the same event. However, this cannot mean that one person's story is more exact or valid and that another person's story is wrong or invalid. Each person is involved
122 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
into the event in a different way. Therefore, his or her story is the reflection of his or her experience of the event. Wishes, expectations, interests, needs and personal opinions of the people included into the research should help the researcher involved in the qualitative research achieve more integrated knowledge of the observational phenomena. Every method and theory denote a part of reality in their own way. Thus, we cannot expect that the research results, produced on the basis of different methods and theories, will automatically produce a complete picture of the researched phenomenon. Whichever method and technique, respectively, is applied, it influences what we see. Bloor (1997) claims that the direct comparision of different methods and theories is basically problematic since the research results depend on the circumstances of their production.
Conclusion
The paper deals with the qualitative research as an independent research form, developed in opposition to the quantitative research. The criteria applied to establish the quality of scientific findings were highlighted and classified into the positivist, post-positivist, post-modern and post-structuralist group. In agreement with Sagadin (1989, p. 335) about the unfruitfulness of "paradigmatic exclusivism", we believe that when establishing the qualitative research criteria the classification of an individual criterion in a particular group is less important than its harmonisation with the basic characteristics of the qualitative research. Clearly set criteria are the very factor ensuring the scientific attribute of the certain research type and providing for the rational grounds for the action based on the research findings. Within the qualitative research we believe it is reasonable to examine the internal validity, as well, although it has been directly adopted from the quantitative research into the qualitative research. It is important that the research findings provide as precise as possible explanation of the research phenomenon, further that the researcher presents and combines different views on the researched situation into a joint conclusion, that causative relations are established, explained and justified and that all findings are supported by the collected data. In the paper special attention was paid to triangulation and its role within the qualitative research. Triangulation is defined as combinations of different methods, research approaches, techniques, sources of information, science disciplines, researchers and theories within the scope of one piece of research, in order to get as comprehensive an insight into the researched situation as possible. If we summarize the opinions of the supporters and critics of triangulation, we can conclude that triangulation contributes to building up a more true image of the researched phenomenon, but not a more objective one. Therefore it is reasonable to deploy triangulation as a strategy to ensure the quality of scientific findings obtained from the qualitative research, and not as a specific criterion of assessing the quality of findings established in the qualitative research, as it would thus be deviated from epistemological characteristics of the qualitative research.
The importance of triangulation for ensuring
123
References
Bloor, M. J. (1997). Techniques of Validation in Qualitative Research: A Critical Commentary. In: Miller, G. and Dingwall, R. (eds.), Context and Method in Qualitative Research. London: Sage, pp. 37-50. Bogdan, R. C., Biklen, K. S. (2003). Qualitative Research for Education. An Introduction
to Theory and Methods. Boston: Allyn and Bacon. Bryman, A. (2004). Social Research Methods. New York: Oxford University Press, Inc. Charmaz, K. (2006). Constructing Grounded Theory. London: Sage Publications Ltd. Cohen, L., Manion, L. (1990). Research Methods in Education. London: Routhledge. Creswell, J. W. (1995). Research design: Qualitative and Quantitative approaches.
Thousand Oaks: Sage. Creswell, J. W. (1998). Qualitative inquiry and Research Design. Thousand Oaks: Sage. Denzin, N. K. (1978). The Research act. New York: McGraw - Hill Book Company. Denzin, N. K., Lincoln Y. S. (2000): Handbook of Qualitative Research. London: Sage Publications.
Denzin, N. K., Lincoln, Y. S. (2005). The Discipline and Practise of Qualitative Research. In: Denzin, N. K., Lincoln, Y. S. (eds.), The Sage Handbook of Qualitative Research. Thousand Oaks, California: Sage Publication, pp. 1-32. Flick, U. (1998). An Introduction to Qualitative Research. Sage Publication, Inc. Fraenkel, J. R., Wallen, N. E. (2006). How to design and evaluate research in education.
New York: McGraw-Hill. Glaser, B. G., Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for
Qualitative Research. Chicago: Aldine. Hammersley, M. (1992). By what Criteria should Ethnographic Research be Judged? In: Hammersley, M. (ed.), What's wrong with Ethnography. London: Routhledge, pp. 60-75.
Janesick, V. J. (1998). The Dance of Qualitative Research Design: Metaphor, Methodolatry, and Meaning. In: Denzin, N. K. and Lincoln, Y. S., Strategies of Qualitative Inquiry.
Kogovsek, T. (1998). Kvaliteta podatkov v kvalitativnem raziskovanju. Magistrsko delo.
Ljubljana: Univerza v Ljubljani, Fakulteta za družbene vede. Kuhn, T. S. (1974). Struktura naucnih revolucija. Beograd: Nolit.
Lather, P. A. (1993). Fertile Obsession: Validity after Poststructuralism. The Sociological
Quarterly, 34, 673-693. LeCompte, M. D., Goetz, J. P. (1982). Problems of Reliability and Validity in Ethnographic
Research. Review of Educational Research, 52, 31-60. Lincoln, Y. S., Guba, E. G. (1985). Naturalistic Inquiry. Bewerly Hills: Sage. Lincoln, Y. S., Denzin, N. K. (1994). The Fifth Moment. In: Denzin, N. K., Lincoln, Y. S.
(eds.), Handbook of Qualitative Research. Thousand Oaks: Sage, pp. 575-586. Lüders, C., Reichertz, J. (1986). Wissenschaftliche Praxis ist, wenn alles funktioniert und keiner weiß warum: Bemerkungen zur Entwicklung qualitativer Sozialforschung. Sozialwissenschaftliche Literaturrundschau, 12, 10-102. Marshall, C., Rossman, G. (1989). Designing Qualitative Research. London: Sage.
124 CONTEMPORARY PEDAGOGY 5/2008
Polona Kelava
Mason, J. (1996). Qualitative Researching. London: Sage.
Mažgon, J. (2006). Vpliv paradigmatskega relativizma na metodologijo akcijskega raziskovanja: doktorska disertacija. Ljubljana: Univerza v Ljubljani, Filozofska fakulteta.
Mesec, B. (1998). Uvod v kvalitativno raziskovanje v socialnem delu. Ljubljana: Visoka šola za socialno delo.
Morse, J. M. (1998). Designing Funded Qulitative Research. In: Denzin, N. K., Lincoln, Y. S. (eds.), Strategies of Qualitative Inquiry. London, Thousand Oaks, New Delhi: Sage Publications, pp. 56-85.
Moser, H. (1977). Methoden der Aktionsforschung. Eine Einführung. München: Kösel.
Neuman, W. L. (2003). Social Research Methods. Pearson Education, Inc.
Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.
Richardson, L. (1997). Fields of play: Constructing an academic life: New Brunswick, NJ: Rutgers University Press.
Richardson, L. (2003). A Method of Inqiry. In: Denzin, N. K., Lincoln, Y. S. (eds.), Collecting and Interpreting Qualitative Materials. Thousand Oaks, California: Sage Publication, Inc, pp. 499-541.
Sagadin, J. (1989). Paradigmatska plat akcijskega in tradicionalnega empiričnega pedagoškega raziskovanja v luči objektivnosti in resničnosti znanstvenih spoznanj. Sodobna pedagogika, št. 7-8, str. 335-340.
Sagadin, J. (1993). Poglavja iz metodologije pedagoškega raziskovanja. Ljubljana: Zavod Republike Slovenije za šolstvo in šport.
Sagadin, J. (2001). Pregledno o kvalitativnem empiričnem pedagoškem raziskovanju. Sodobna pedagogika, 2, 10-25.
Scheurich, J. J. (1997). Research method in the postmodern. London: Falmer.
Silverman, D. (2005). Doing Qualitative Research. London: Sage Publications.
Smith, J. K. (1993). After the demise of empiricism: The problem of judging social and educational inquiry. Norwood, NJ: Ablex.
Stake, R. E. (2005). Qualitative Case Studies. In: Denzin, N. K., Lincoln Y. S. (eds.). The Sage Handbook of Qualitative Research. Tretja izdaja. London, Thousand Oaks, New Delhi: Sage Publications, pp. 443-466.
Steinke, I. (1999). Kriterien qualitativer Forschung: Ansätze zur Bewertung qualitativempirischer Socialforschung. Weincheim: Juventa.
Tashakkori, A., Teddlie, C. (1998). Mixed Methodology: combining qualitative and quantitative approaches. London, Thousand Oaks, New Delhi: Sage Publications.
Toličič, I., Zorman, L. (1965). Testi znanja in njihova uporaba v praksi. Ljubljana: Zveza delavskih univerz Slovenije.