Given the strong connection between the theme of the 2010 International Federation of Library Associations and Institutions (IFLA) conference — open access to knowledge — and the Global Impact Study’s open research approach, we were pleased to present a poster about our inventory database and web application.

The poster — Open data and open tools: The Global Impact Study inventory and web application — authored by Chris Rothschild, Araba Sey, and Alexander Tulinsky, was presented by Chris Coward. The poster highlights the data users can view through the web application, such as location, ownership, internet access fees, and types of public access venues in Bangladesh, Brazil, Chile, Lithuania, and the Philippines. The poster also explores several delicate issues surrounding open research and open data, including privacy concerns, ethical issues, data validity, and conditions of use.

While this tool is primarily intended for researchers, many practitioners at IFLA thought it would be useful for them as well: “I wish this data and database existed for all countries!” But Chris Coward cautioned that the Global Impact Study did not set out to create a comprehensive database, nor should it be used as such. Rather, it is our hope that this tool, in the spirit of open research, will be beneficial and useful to researchers and practitioners alike.

For more information about the inventory database and web application you can download a PDF of the poster.

For more information about the inventory research activity, browse inventory updates.

This study uses surveys, task analysis, and interviews with public access venue users and operators. Led by Marion Walton and Jonathan Donner, it seeks to answer the following questions:

1. What are the ways in which users combine and balance private mobile internet use and public access venue internet use in a communications repertoire?
2. How do social networks, affinity groups, and communities of practice mediate mobile and public access venue use to potential users? With what consequences?
3. Are there particular patterns of attrition (withdrawal/substitution from public access venues towards mobiles), of attraction (seeking public access venue services after mobile use), and of complementarities?
4. Which complementarities (and substitutions) are associated with higher levels of user satisfaction with public access venues and/or with various impacts for end-users?
5. Is it possible to contrast collocated use (using a mobile phone while at a public access venue) and separate use (using the phone outside of the public access venue)?
6. How do non-profit public access venue owners deal with the increasing availability of mobile internet? What practices do they describe, and what policies do they apply in their venues, specifically with teenage users?

This study will be conducted in South Africa, which boasts a vibrant mobile internet market. Trends and patterns that identified though this research may manifest in other countries in the near future, and it is our hope that we will be able to build generalizable theory as a result of this in-depth study. For more information on the Public Access, Private Phone study, you may download the full research proposal or browse project updates.

With the data from this survey, we hope to gain a greater understanding of the differences and similarities between public access ICT users and non-users, reasons for non-use, and the extent to which use of public access ICTs might give users unique advantages over non-users. We have updated our original survey research questions to include the non-user element.

Survey Research Questions

1. What is the demographic profile of public access ICT users and non-users?

2. Apart from public access ICTs, what other information and communication resources do public access ICT users and non-users have?

3. How do public access ICT users and non-users perceive their ICT skills and what are their ICT use comfort levels?

4. Why do people go to public access ICT venues?

5. What are the reasons for non-use of public access ICT venues?

6. What do people do at public access venues?

7. How accessible are public access ICT venues and services to different types of populations?

8. How do the design, services and operations of public access ICTs affect usage patterns?

9. What do public access ICT users see as the impacts of using public access ICTs?

10. Do non-users see an impact from not using public access ICTs?

11. What outcomes can be associated with public access ICT use in different domains?

12. Are the outcomes non-users experience from using non-public access information and communication resources similar to the outcomes experienced by users of public access ICTs?

13. Does public access ICT use have indirect impacts?

14.What is the value of public access ICTs to users?

15.  What is the cost of providing public access ICTs?

 The teams completed cognitive testing at the end of May.  Their reports draw attention to a number of areas where the survey questions could be improved. We can attribute the comprehension issues identified in the testing to a variety of factors including:

• Inappropriate respondent to answer the question (e.g. lacks knowledge to answer questions about a venue’s computer equipment or balance sheet).
• Use of technical jargon (e.g. log server).
• Inaccurate characterization of situation (e.g. asking for a single fee amount for internet access whereas venue has a tiered fee system).
• Inapplicability of question to respondent (e.g. does not engage in the activity a question asks about).
• Too many answer categories, making it diffcult to remember.
• Confusing wording of question.
• Complicated nature of information requested (e.g. respondent needs to estimate budget allocation to different venue expenses).
• Complex question structure (e.g. requires answers in multiple layers).
• Inaccurate translation from English to local language.

Some of these issues require revising the questionnaire, some require revising the language translations, and others require developing strategies to make the survey easier to implement (e.g. using visual aids for answer categories). We paid particular attention to those issues that occurred in more than one country, while trying to find appropriate compromises for items that were specific to individual countries. The Survey Working Group has revised the instrument and survey design where needed, taking into account recommendations supplied by the  survey implementation teams. The instruments are now moving on to field testing.

1. Cognitive testing to check for comprehension problems with the survey questions.

2. Field testing to check for potential problems in the entire survey adminstration process (e.g. structure, content, flow and length of the questionnaires) as well as  the actual data collected.

We begin with cognitive testing on a small scale, after which we make revisions to the survey instruments to address issues revealed by the cognitive tests. Next we carry out the field testing with a larger number of locations and respondents.

This strategy enables us to use the results of the cognitive testing to eliminate comprehension of survey questions as a potential major source of delays during field testing. For example, we are likely to get a more accurate estimate of the length of the surveys if interviewers don’t have to spend too much time trying to explain what specific survey questions mean.

Our survey implementation teams have been given detailed guidance in our survey testing guidelines.

1. Public access, private phone: the interplay of shared access and the mobile internet — This study will assess and describe the interplay between public PC-based internet access and private mobile phone-based access in South Africa.
2. Public access ICT use and the community information ecology — This study looks at public access ICT use in context, measuring it against the availability and use of other formal and informal community information resources. The study will take place in two to four countries.
3. Public access ICTs and changes in information ecology — This study will investigate what changes occur in information and communication practices after public access ICTs are introduced into communities in Botswana. Areas of particular interest are education and local content creation.
4. Public access ICTs, mobile phones and e-government —This study will examine the impact of government services provided through PCs at public access ICT venues in the Philippines, comparing their impact with that of similar mobile phone-based services.
5. Costs and benefits of public access ICTs — This study will examine the relationship between the costs and benefits of public access to ICTs, from service provider and user perspectives.

Full proposals for these studies will be posted shortly.

Development of the surveys has been a difficult process of trade-offs. Most people who have participated in surveys on ICT use can probably identify with the feeling that these surveys are often extremely long, some taking up to two hours to complete. And researchers will admit that once the data is collected, a large proportion of it goes unused, it is never analyzed. We’ve had to make some difficult choices about the scope of our surveys of public access ICT venue users and operators. There are several relevant issues that we would like to explore, from the demographics of public access users to the impact of usage on users’ health, education and participation in civic life. The first draft of the user survey compiled over 60 pages of questions on several of these interesting issues - a survey that could potentially take more than three hours to complete. Since our goal is to keep the surveys less than one hour long, we needed a systematic approach to decide which questions would be included in the final version. To do this we prioritized the research questions we would investigate with the surveys, developed hypotheses to go with each research question, and identified potential indicators to test each hypothesis. This process enabled us to select questions for the instruments that are directly related to our research questions, and provided a means to make the hard decisions of eliminating questions that are “interesting” but fall out of the scope we have defined for the surveys.

The research questions for the surveys are:

1. What is the demographic profile of public access ICT users?
2. Apart from public access ICTs, what other information and communication resources do public access ICT users have?
3. How do public access ICT users perceive their ICT skills and what are their ICT use comfort levels?
4. Why do people go to public access ICT venues?
5. What do people do at public access ICT venues?
6. How accessible are public access ICT venues and services to different types of populations?
7. How do the design, services, and operations of public access ICTs affect usage patterns?
8. What types of venue policies do public access ICT operators view as inhibiting their ability to provide information and services that they believe users want?
9. What do public access users see as the impacts of using public access?
10. What outcomes can be associated with public access ICT use in different domains?
11. Does public access ICT use have indirect impacts?
12. What is the value of public access ICT to users?
13. What is the cost of providing public access ICTs?

The survey instruments are in the final stages of editing before being translated into local languages and piloted. Results from the pilots will inform the final survey instruments, which will be rolled out in early 2010 in Bangladesh, Brazil, Chile, and the Philippines. We are looking forward to sharing the final instruments and additional insights about our survey development process.

Since we were trying to design an inventory data collection tool that would be usable in multiple countries, the effort required several months of collaboration amongst project members to reach agreement on issues such as what types of data to include in the inventory; definitions of telecenters, internet cafés and libraries; and how to account for unique features of public access ICT venues in different countries. For example, some people describe telecenters as  donor or publicly-funded computing venues that provide free access to computers, while others define telecenters by their mission, that is, whether they have social rather than economic goals.

It has taken about 12 months to complete this process and we will have four distinct outputs:

1. A taxonomy that provides a common language for categorizing public access venues across countries
2. A template for data collection with detailed instructions
3. A database containing inventory data from Bangladesh, Chile and Lithuania
4. A report describing our experience in developing the inventory and some preliminary descriptive analyses of the data

Following our open research data approach, the database will be accessible online after the data have been cleaned and our researchers have had the opportunity to perform some analyses of the data. First versions of the inventory guidelines and data collection template are now available to anyone interested in replicating our process. We hope this tool will be useful to others performing similar activities. The first set of data for Bangladesh, Chile, and Lithuania will be available in early 2010. Research teams in Brazil and the Philippines will begin data collection by January 2010, and their data should be accessible in mid 2010.

Download the inventory guidelines and spreadsheet.

Our next steps are to finish cleaning the data, test the database, and develop a system to enable yearly updating of the data by researchers in each country.