Informatik 23 (1999) 461-465 461
Measuring Information Society: Some Methodological Problems
Vasja Vehovar, Matej Kovačič
Faculty of Social Sciences, Kardeljeva ploščad 5, 1000 Ljubljana, Slovenia Phone: +386 61 1805 100; Fax: +386 61 1805 102 vasja.vehovar@uni-lj.si, matej.kovacic@uni-lj.si
Keywords: information society, Internet indicators, survey research, electronic commerce Edited by: Cene Bavec and Matjaž Gams
Received: October 10, 1999 Revised: November 30, 1999 Accepted: December 20, 1999
The paper addresses methodological problem of measuring information society. Both, technical indicators and attitudinal measurements for Slovenia are discussed in this context. In particular, the results related to the interest for information society services are presented. The comparison between Slovenia and European Union — despite some methodological problems -- shows that the interest for these services is extremely high in Slovenia. Other figures also confirm that Slovenian households and businesses are generally on European average with respect to the penetration of the basic information technologies. However, certain discrepancies with other sources of data call for more efforts in performing these kind of analysis.
1 Preface
The concept of the information society has already been around for many decades. Nevertheless, its definition is relatively unclear, and the same is true for the corresponding indicators. There do exist some ad-hock measures from as early as sixties, particularly in the area of services, the information professions and the extent of the business information sector. However, it is extremely hard to establish official indicators for the phenomena in such a dynamic field. The Internet, in particular, has brought even more complications in these measurements. Even in United States, the official estimates about the scope of the electronic commerce will be available only in year 2000, after the electronic commerce transactions have already reached hundreds of billions of USD. The first official estimated will thus arrive after several years of extreme variety in the estimates from numerous consulting agencies. In addition, there are considerable discrepancies also in other measurements of information society. The paper presents an overview of most frequent divergences that arise from the interpretation of indicators of information society. The methodological misunderstanding is also an important reason that unnecessarily hinders the understanding of the position of Slovenia.
2 Quantitative measurement
Quantitative indicators of the information society usually refer to numerical figures — expressed in numbers/percentages of users, or, in the form of financial totals — which most often relate to the use
and penetration of modern technologies, especially the Internet, mobile telephony and electronic commerce. However, we have to be extremely careful when interpreting these data.
We can, for instance, classify the use of electronic payment orders of the Slovenian Agency for Payments as a form of electronic commerce. Many experts claim that this is also a specific form o Electronic Data Interchange (EDI). In this case the amount of electronic commerce in Slovenia can be measured in hundreds of billions of USD. But if we talk about the transactions where searching, ordering, billing and payment procedures are all performed in the electronic forms — without any paper recording — it is clear that the amount of electronic commerce is only a fraction of this amount, e.g. only around few millions USD. Therefore, we have to be very precise when stating such observations. It is not surprising that the estimates of leading consulting agencies on electronic commerce have varied at rates 1:10 in the past years, and at present they still vary at the rate 1:2. Often, the source of the problems is not even in the statistical methodology or in the definitions, but in a simple fact that the methodological framework is not properly reported.
Recent international efforts for standardised measurement on electronic commerce, particularly those at OECD (1999) already brought some results and we can perform certain international comparison of the Internet and electronic commerce usage among companies. The available comparisons with Singapore, Scandinavian countries and Australia shows that with respect to PC usage, Internet penetration and Web site
462 Informatica 23 (1999) 461-465
V. Vehovar et al.
penetration among companies there is, as for 1998, no significant lagging for Slovenia. However, there is a certain time lag in the adoption of electronic commerce applications. Unfortunately, the international comparisons in the area of electronic commerce are much more complicated. Recent experiments in RIS 1999 survey of companies clearly demonstrated the sensitivity tn these kinds of measurements. It has been shown that when electronic commerce was defined as any business transaction performed over the computer networks the percentage of companies claiming to use electronic commerce was 10% higher compared to the definition that was restricted only to the transactions that lead to the purchase (RIS, 1999).
A similar problem of definition is the estimate of the number of Internet users. There exist at least five categories of Internet users (Vehovar et al. 1998). This prompts to a need for an exact definition of the term "Internet user". For instance, the estimate of EITO (1999) talks about 60.000 Internet users in Slovenia in 1998, however, the definition of Internet user and description of how the estimate was obtained is not available there. In addition, this estimate differs from all other estimates for Slovenia. Even the number of users with personal e-mail accounts in 1998 is much higher than 60.000. Of course, EITO's estimate could refer to some specific group of intensive users.
When measuring information society phenomena we are also faced with divergence, which originates from the methodology of data collection. The IDC corporation, for instance, provides estimates based on distribution channels and one of the figures states that 65.000 purchases (shipments) of new personal computers were made in Slovenia in 1998, thereof 17% in households. This suggests that households buy around 10.000 personal computers yearly. Such an estimate also matches the ESIS (1999) figures stating that Slovenian households possess 100.000 personal computers (with a processor 486 or more). However, this does not match the survey estimates that Slovenian households have more than 200.000 personal computers, which is a result of practically all surveys (Statistical office, Mediana, Slovenian public opinion, RIS...). Survey estimates consistently show that the number of personal computers has surpassed 200.000 also in business use, what suggests that Slovenia is highly ranked by number of personal computers per 100 residents. There are more than 25 personal computers per 100 habitants in Slovenia. This is surprisingly high, however, as the usage of information technology is rather complex, the criticisms regarding low technology penetration in Slovenian economy may
still hold true (The World Competitiveness Yearbook 1999, IMD Lausanne).
One of the most exposed indicators of the Internet and information society is the statistics on the Internet hosts (Vehovar, 1998). This indicator shows extremely inconvenient trends for Slovenia: the growth of hosts in the last two years has almost entirely stopped while all other countries rapidly progress. However, the number of hosts is a typical example of an indicator that is more complex than a casual observer might think. For instance — all the hosts which are not included in *.si domain are excepted from Slovenian host cunts. This does not happen so often in larger countries or in countries with more liberal legislation for assigning domains. In Slovenia, non-domestic domains are very frequent, even among the most visited sites and among the largest Internet access providers: siol.net, s.net, s5.net, amis.net. It seems that the large majority of commercial dial-up modems/ hosts is registered under domains *.net. The high usage of dial-up access in Slovenia also presents a problem lor itself and contributes to a low host counts, because each host/modem serves many dial-up users. Additional problems can present the multiple IP numbers - e.g. virtual hosts - located on one computer. This is more often the case in countries such as Estonia than in Slovenia. We have to understand that the "host" does not necessarily mean a computer connected to the Internet, but only an IP number. In Slovenia, additional problems are also the computers that are connected to many large local networks with full access to the Internet but without an IP number.
The problem of host counting is getting even more complex because of the technical problems of measurement procedures, which are becoming increasingly more difficult due to fire-walls and other forms of security protections. This forced Network Wizards to change entirely the methodology and broke with the time series. The data about host numbers from RIPE (http://www.ripe.net) and Network Wizard (http://www.nw.com) thus vary considerably. The RIPE host count often shows a clear monthly a recession in the number of hosts for some countries (Italy, for instance) what is not realistic. All the above arguments may explain the situation for Slovenia, where the host counts in the last two years show less than 10% yearly growth (Picture 1, Picture 3), but all other indicators (number of registered domains, number of companies connected to the Internet, number of households with access to the Internet, number of Internet users) demonstrated more than 50% growth (Picture 2).
MEASURING INFORMATION SOCIETY: SOME .
Informatica 23 ( 1999) 461 -465 463
Number of hosts (Dec. 92 - Nov. 99) Source: RIPE
25000
20000
15000
10000
5000
<£> <£> <§> <§> <£> <£> ^f ^	/ ^ o&
number of domains - sum(dec.92 - nov.99)
(Source: ARNES)
6000 5000 4000 3000 2000 1000
0 -I
dec.92 dec.93 dec.94 dec.95 dec.96 dec.97 dec.&8 dec.99
Picture 1: Number of hosts in subdomain *.si (Source: Picture 2: Growth of registered domains *.si (Source: RIPE)	ARNES)
RIPE
¿2
e ■g
£
o 8 d o
£
o "p
2
3000
2S0Ö
d 3 d s d -m
Ü,	S.	.Ü, ,s"
S) S
-♦--Czech Italy Hungary Slovenia—Ireland«®—Estonia —»—Austria
Picture 3: Hosts per 100.000 residents, January 1995 -1
Another specific methodological problem can be observed in the areas, where growth is extremely high and the circumstances are changing very fast (in less than a year), as with mobile telephones. This example, however, has an additional dimension with the categories of mobile communications (i.e. GSM, prepaid mobile phones, etc) that are extremely structured and thus not comparable.
3 Measurement of attitudes
Above, we have observed that even the indicators that
er 1999 (Source: RIPE).
can be exactly measured on the ratio scale could be very problematic. It is thus reasonable to expect more troubles with the indicators referring to the attitudes towards the information society. However, the users' attitudes to the various aspects of the information society, such as the attitude towards security, privacy, abuse, role of the government, future and intended usage, are extremely important for understanding the context of information society.
Below, we present a typical example of attitudinal measurement. In June 1999, the RIS project and the
464 Informática 23 ( 1999) 461 -465
V. Vehovar et al.
Institute for Economic Research- (Incopernicus project) co-operated in a telephone survey on household usage of information technologies (n = 1000). Among others, the survey included questions on interest in services of the information society. The questions were based on the Eurobarometer EB 50.1 survey, which was performed among the member states of the European Union in fall 1998. The question was: "We will give you a list of services of the information society. Please tell us whether you are interested in their use or not." The June results for Slovenia were surprisingly high, so they were repeated also in September. However, the estimates were basically the same. In Table 1 we can observe the percentages of respondents interested in corresponding services in June and September for Slovenia, and the averages for EU at the end of 1998 (from EB 50.1 ).
Similar differences were noticed in other indicators, like the interest for financial management, virtual museum visiting, trip and travel planing information, consumer rights, and employment search (RIS, 1999). In addition, Slovenia has a good standing also in the area of information and communication technology penetration. In general it is on the average of the EU countries. The analysis shows that Slovenia is comparable to Scandinavia and not to Austria, which has certain historical linkages to Slovenia.
When trying to explain this specific position we first think of methodology. It is true, that the Slovenian survey was performed more than half a year after the European one. Certain methodological discrepancy could also arise from the fact that the European survey was conducted face-to-face. However, in Slovenia the telephone coverage has reached 90%, so this can not contribute to a significant overestimation. Alternatively, the overestimation could be related to the general overestimation of the information society phenomena — for instance, the overestimation of the number of Internet users, the number of PC's in households, etc. However, whenever the external control is possible - such as with mobile phones or Internet subscribers - the estimates were correct. The results were also consistent across different surveys
performed by Statistical office, RIS project, Slovenian public opinion survey, Mediana, and marketing agencies' surveys.
It is more likely that the proper interpretation of these surprising results goes into a direction of a specific climate that is very opened to the information society issues. Such a conclusion can be of the same importance for the understanding of information society processes as the standard statistical indicators (number of telephone lines, Internet users...).
4 Conclusion
Rapid technological developments bring new problems to the process of measuring social phenomena. The speed of the social change also introduces certain changes in measurement procedures. Of course, we have to be extremely precise about what these measurements actually mean.
We also have to accept the fact that the absence of more profound efforts for measuring and analysing information society phenomena prevents us from determining an exact position for Slovenia. Compared to other socio-economic statistics - like employment or national accounts that are permanently covered by large groups of experts - the research on information society indicators has not yet been instutionalized. Since the dynamics of these phenomena has become a constant, we can expect the establishment of adequate research in a very near future. The process of European unification is merely accelerating this process (i.e. ESIS, 1999). Lots of indicators on information society are also provided by global consulting agencies, which are partially compensating the lack of official statistical data.
We can thus conclude that we are in a temporary information vacuum where we do not have enough information to determine where exactly Slovenia stands in the information society developments. This is also a possible reason for the sensitivity of the data interpretation to a variety of sloppy analyses.
	Slovenia -	Slovenia -	
Information service	June'99	September '99	EU average '98
On-line medical diagnosis	54.2%	54.5%	41.9%
Contacts with politicians	18.7%	15.6%	10.9%
Education	55.2%	50.4%	33.9%
Consumer rights service	63.7%	55.6%	33.4%
Conducting public and administrative services	54.5%	58.6%	47.8%
Table 1: Interest for information society services (Source: EB 50.1, RIS)
[ 1 ] EITO ( 1999), Frankfurt: EITO.
Sources and List of References
[2] ESIS (1999), http://www.ris.org/id/esis.html.
MEASURING INFORMATION SOCIETY: SOME
Informatica 23 (1999) 461-465 465
[3]	CMEC (1999), Conference on measurement of electronic commerce, Singapore,
1999',
http://www.singstat.gov.sg/EC/echome.html.
[4]	Network Wizard (1999), http://www.nw.com.
[5]	OECD (1999), http://www.oecd.org/subject/e_commerce/.
[6]	RIPE (1999), http://www.ripe.net.
[7]	RIS (1999), http://www.ris.org.
[8]	The World Competitiveness Yearbook 1999 (1999), IMD Lausanne.
[9]	Vehovar, V., Remec, M., Kramberger, T. (1998): Statistika Interneta. Statistični dnevi 98, Radenci, Ljubljana: Statistični urad Republike Slovenije, Statistično društvo Slovenije.
[10]Vehovar,	V. et al. (1998): Internet v Sloveniji. Ljubljana: Desk.
[11]Vehovar,	V. (1999): Merjenje elektronskega poslovanja s pomočjo vzorčnih anket. Uporab, inform., 7, 2, str. 29-34.
[ 12]EuroBarometer (1999): The "Measuring Information	Society"	EuroBarometer,
http://www.ispo.cec.be/polls/.