Given the strong connection between the theme of the 2010 International Federation of Library Associations and Institutions (IFLA) conference — open access to knowledge — and the Global Impact Study’s open research approach, we were pleased to present a poster about our inventory database and web application.

The poster — Open data and open tools: The Global Impact Study inventory and web application — authored by Chris Rothschild, Araba Sey, and Alexander Tulinsky, was presented by Chris Coward. The poster highlights the data users can view through the web application, such as location, ownership, internet access fees, and types of public access venues in Bangladesh, Brazil, Chile, Lithuania, and the Philippines. The poster also explores several delicate issues surrounding open research and open data, including privacy concerns, ethical issues, data validity, and conditions of use.

While this tool is primarily intended for researchers, many practitioners at IFLA thought it would be useful for them as well: “I wish this data and database existed for all countries!” But Chris Coward cautioned that the Global Impact Study did not set out to create a comprehensive database, nor should it be used as such. Rather, it is our hope that this tool, in the spirit of open research, will be beneficial and useful to researchers and practitioners alike.

For more information about the inventory database and web application you can download a PDF of the poster.

For more information about the inventory research activity, browse inventory updates.

This study uses surveys, task analysis, and interviews with public access venue users and operators. Led by Marion Walton and Jonathan Donner, it seeks to answer the following questions:

1. What are the ways in which users combine and balance private mobile internet use and public access venue internet use in a communications repertoire?
2. How do social networks, affinity groups, and communities of practice mediate mobile and public access venue use to potential users? With what consequences?
3. Are there particular patterns of attrition (withdrawal/substitution from public access venues towards mobiles), of attraction (seeking public access venue services after mobile use), and of complementarities?
4. Which complementarities (and substitutions) are associated with higher levels of user satisfaction with public access venues and/or with various impacts for end-users?
5. Is it possible to contrast collocated use (using a mobile phone while at a public access venue) and separate use (using the phone outside of the public access venue)?
6. How do non-profit public access venue owners deal with the increasing availability of mobile internet? What practices do they describe, and what policies do they apply in their venues, specifically with teenage users?

This study will be conducted in South Africa, which boasts a vibrant mobile internet market. Trends and patterns that identified though this research may manifest in other countries in the near future, and it is our hope that we will be able to build generalizable theory as a result of this in-depth study. For more information on the Public Access, Private Phone study, you may download the full research proposal or browse project updates.

With the data from this survey, we hope to gain a greater understanding of the differences and similarities between public access ICT users and non-users, reasons for non-use, and the extent to which use of public access ICTs might give users unique advantages over non-users. We have updated our original survey research questions to include the non-user element.

Survey Research Questions

1. What is the demographic profile of public access ICT users and non-users?

2. Apart from public access ICTs, what other information and communication resources do public access ICT users and non-users have?

3. How do public access ICT users and non-users perceive their ICT skills and what are their ICT use comfort levels?

4. Why do people go to public access ICT venues?

5. What are the reasons for non-use of public access ICT venues?

6. What do people do at public access venues?

7. How accessible are public access ICT venues and services to different types of populations?

8. How do the design, services and operations of public access ICTs affect usage patterns?

9. What do public access ICT users see as the impacts of using public access ICTs?

10. Do non-users see an impact from not using public access ICTs?

11. What outcomes can be associated with public access ICT use in different domains?

12. Are the outcomes non-users experience from using non-public access information and communication resources similar to the outcomes experienced by users of public access ICTs?

13. Does public access ICT use have indirect impacts?

14.What is the value of public access ICTs to users?

15.  What is the cost of providing public access ICTs?

In March 2010 Global Impact Study team members working on the in-depth studies, survey, and inventory activities gathered to harmonize impact indicators and discuss the project’s approach to communications knowledge sharing. We outlined our proposed approach to communications and knowledge sharing and in the discussion that followed researchers expressed the need for an internal communication space where Global Impact Study members can ask questions, request feedback, and share resources, reports, and any other materials that are not ready or intended for public consumption. TASCHA has established a Dgroup for this purpose.

For more information you can download the communications and knowledge-sharing presentation, or view photos from the workshop.

 The teams completed cognitive testing at the end of May.  Their reports draw attention to a number of areas where the survey questions could be improved. We can attribute the comprehension issues identified in the testing to a variety of factors including:

• Inappropriate respondent to answer the question (e.g. lacks knowledge to answer questions about a venue’s computer equipment or balance sheet).
• Use of technical jargon (e.g. log server).
• Inaccurate characterization of situation (e.g. asking for a single fee amount for internet access whereas venue has a tiered fee system).
• Inapplicability of question to respondent (e.g. does not engage in the activity a question asks about).
• Too many answer categories, making it diffcult to remember.
• Confusing wording of question.
• Complicated nature of information requested (e.g. respondent needs to estimate budget allocation to different venue expenses).
• Complex question structure (e.g. requires answers in multiple layers).
• Inaccurate translation from English to local language.

Some of these issues require revising the questionnaire, some require revising the language translations, and others require developing strategies to make the survey easier to implement (e.g. using visual aids for answer categories). We paid particular attention to those issues that occurred in more than one country, while trying to find appropriate compromises for items that were specific to individual countries. The Survey Working Group has revised the instrument and survey design where needed, taking into account recommendations supplied by the  survey implementation teams. The instruments are now moving on to field testing.

1. Cognitive testing to check for comprehension problems with the survey questions.

2. Field testing to check for potential problems in the entire survey adminstration process (e.g. structure, content, flow and length of the questionnaires) as well as  the actual data collected.

We begin with cognitive testing on a small scale, after which we make revisions to the survey instruments to address issues revealed by the cognitive tests. Next we carry out the field testing with a larger number of locations and respondents.

This strategy enables us to use the results of the cognitive testing to eliminate comprehension of survey questions as a potential major source of delays during field testing. For example, we are likely to get a more accurate estimate of the length of the surveys if interviewers don’t have to spend too much time trying to explain what specific survey questions mean.

Our survey implementation teams have been given detailed guidance in our survey testing guidelines.

1. Public access, private phone: the interplay of shared access and the mobile internet — This study will assess and describe the interplay between public PC-based internet access and private mobile phone-based access in South Africa.
2. Public access ICT use and the community information ecology — This study looks at public access ICT use in context, measuring it against the availability and use of other formal and informal community information resources. The study will take place in two to four countries.
3. Public access ICTs and changes in information ecology — This study will investigate what changes occur in information and communication practices after public access ICTs are introduced into communities in Botswana. Areas of particular interest are education and local content creation.
4. Public access ICTs, mobile phones and e-government —This study will examine the impact of government services provided through PCs at public access ICT venues in the Philippines, comparing their impact with that of similar mobile phone-based services.
5. Costs and benefits of public access ICTs — This study will examine the relationship between the costs and benefits of public access to ICTs, from service provider and user perspectives.

Full proposals for these studies will be posted shortly.

Users at Busy Internet, Ghana, in 2005. Photo courtesy of IDRC/telecentre.org.

The Collaborative Knowledge Sharing Study investigates the way that people share knowledge, experience, and technologies among friends and strangers while physically co-present in cybercafés. Our hypothesis is that this shared use of ICTs is in many cases preferred to individualized private use since people in public cybercafés benefit from physical interactions sharing knowledge and socializing experience with their co-present neighbors.

To examine our hypothesis we have conducted a survey of 75 computer users at a major cybercafé, Busy Internet, in Accra, Ghana. Survey participants were recruited from all computer users in the café, of majority age, during a period of four weeks across in late 2009. Our survey was designed to ascertain if these computer users connected with other people within the cybercafé — either friends, strangers, or café employees — in order to enhance their experience with communication technologies.

Survey results reveal that all respondents report interacting with café employees around technical issues. This result is, admittedly, not very surprising as simple technical issues routinely arise when using the café’s computers. More importantly, we found that more than one-third of respondents (37%) reported some deeper forms of computer sharing and collaboration with friends, family members, business associates, or even strangers while in the café.

Sharing took on different forms, for instance many respondents reported arriving together with friends or family and sitting close together while using separate machines (54%), others reported serially using a single PC one after the other (21%), still others worked simultaneously on the same machine leveraging differential levels of expertise (12%). Of those respondents reporting computer sharing, one-third reported gaining knowledge and learning from the other user as their primary reason for sharing and, surprisingly, only 18% sited purely economic reasons for sharing.

The population of respondents who reported significant or occasional computer sharing was similar in many ways to the population of users who reported never sharing. For example there was no difference in the demographics, computer fluency or experience, frequency or type of technical assistance sought, or the range of applications used between the sharing and non-sharing populations (p  > 0.05 on all questions using standard statistical measure). In contrast, these two populations were different in some interesting ways. For instance the sharing population was less concerned with privacy issues than the non-sharers, typically came to the cybercafé with more people and with a different mix of people, and had generally a better view towards collaborative group work and broader forms of interaction while in the café (p < 0.01 on all questions using standard statistical measures).

Results from our survey in Accra, Ghana, suggest that face-to-face connections among people co-located in cybercafés are an important component to the experience and enhance impact. In future work we intend to explore design innovations for the computers and café environment that will enhance the best forms of collaborative work and in-person sharing.

1. Inventory and surveys that provide a big-picture view
2. Focused studies of specific mechanisms through which public access impacts livelihoods
3. An assessment of indirect and aggregated impacts, which takes the community as the unit of analysis and looks at non-users and alternative information sources
4. A look at alternatives and complements to public access, focusing on mobile phones

Get a big-picture view — To do this, we’re combining an inventory of all of the public access venues in Bangladesh, Brazil, Chile, Lithuania, and the Philippines with representative surveys of venue operators and users. With this we will start to understand magnitude, characteristics, distribution, costs, and impacts — especially on livelihoods. And researchers around the world will finally have access to a reliable database for further investigation of the public-access phenomenon in these five countries (the public database should be available in late 2010).

Dig deeper into specific impact mechanisms — Next, we examine how public access can improve livelihoods. Using a range of methods — ethnographies, focus groups, experiments, etc. — we’re digging deeper into the specific mechanisms leading to impact through in-depth studies of particular venue features: the availability of infomediaries, patterns of shared use, rules prescribing what users can and cannot do (can you play games? chat? update your Facebook profile?). Beyond a better understanding of how change happens, these studies will help us make better policy recommendations: Is it worthwhile to provide staff who can reach out and support users? Which venue design features foster productive sharing? Should gaming be banned or encouraged?

Look at the broader community — What about people who never even walk through the door of a public access venue? How does the venue’s presence effect the community as a whole? To answer these questions we’re conducting in-depth studies on indirect and aggregated impacts. These studies focus on public access use and the community information ecology — taking the community as the unit of analysis and assessing the use of public access venues against other information resources. This way, we can understand how all community members benefit from the presence of a public access venue (whether they use it or not) and learn more about venue reach within communities.

Examine alternatives & complements — There are several alternatives to public access: free or subsidized private computers (like the One Laptop Per Child initiative), infrastructure support (such as free WiFi zones), and computer alternatives (television, radio, mobile phones). Because mobile phones have become so widespread and promising, we’ll examine their use and impact in relation to, and in comparison with, public access venues. In South Africa, we’ll explore whether mobile phones replace or complement public access, or whether the two simply co-exist. In the Philippines, we’ll look at the impact of the provision of government services through mobile phones, comparing their impact to computer-based services offered through public access venues.

Together, these four components cover a range of complementary approaches and methodologies, hypotheses about how impact occurs, national contexts, levels of analysis, and impact areas. In combination, they will provide a multi-faceted understanding of the various ways in which public access to ICTs impacts livelihoods.

Download the full description of the research design updates (excerpt from November 2009 Interim Report).

Development of the surveys has been a difficult process of trade-offs. Most people who have participated in surveys on ICT use can probably identify with the feeling that these surveys are often extremely long, some taking up to two hours to complete. And researchers will admit that once the data is collected, a large proportion of it goes unused, it is never analyzed. We’ve had to make some difficult choices about the scope of our surveys of public access ICT venue users and operators. There are several relevant issues that we would like to explore, from the demographics of public access users to the impact of usage on users’ health, education and participation in civic life. The first draft of the user survey compiled over 60 pages of questions on several of these interesting issues - a survey that could potentially take more than three hours to complete. Since our goal is to keep the surveys less than one hour long, we needed a systematic approach to decide which questions would be included in the final version. To do this we prioritized the research questions we would investigate with the surveys, developed hypotheses to go with each research question, and identified potential indicators to test each hypothesis. This process enabled us to select questions for the instruments that are directly related to our research questions, and provided a means to make the hard decisions of eliminating questions that are “interesting” but fall out of the scope we have defined for the surveys.

The research questions for the surveys are:

1. What is the demographic profile of public access ICT users?
2. Apart from public access ICTs, what other information and communication resources do public access ICT users have?
3. How do public access ICT users perceive their ICT skills and what are their ICT use comfort levels?
4. Why do people go to public access ICT venues?
5. What do people do at public access ICT venues?
6. How accessible are public access ICT venues and services to different types of populations?
7. How do the design, services, and operations of public access ICTs affect usage patterns?
8. What types of venue policies do public access ICT operators view as inhibiting their ability to provide information and services that they believe users want?
9. What do public access users see as the impacts of using public access?
10. What outcomes can be associated with public access ICT use in different domains?
11. Does public access ICT use have indirect impacts?
12. What is the value of public access ICT to users?
13. What is the cost of providing public access ICTs?

The survey instruments are in the final stages of editing before being translated into local languages and piloted. Results from the pilots will inform the final survey instruments, which will be rolled out in early 2010 in Bangladesh, Brazil, Chile, and the Philippines. We are looking forward to sharing the final instruments and additional insights about our survey development process.

Since we were trying to design an inventory data collection tool that would be usable in multiple countries, the effort required several months of collaboration amongst project members to reach agreement on issues such as what types of data to include in the inventory; definitions of telecenters, internet cafés and libraries; and how to account for unique features of public access ICT venues in different countries. For example, some people describe telecenters as  donor or publicly-funded computing venues that provide free access to computers, while others define telecenters by their mission, that is, whether they have social rather than economic goals.

It has taken about 12 months to complete this process and we will have four distinct outputs:

1. A taxonomy that provides a common language for categorizing public access venues across countries
2. A template for data collection with detailed instructions
3. A database containing inventory data from Bangladesh, Chile and Lithuania
4. A report describing our experience in developing the inventory and some preliminary descriptive analyses of the data

Following our open research data approach, the database will be accessible online after the data have been cleaned and our researchers have had the opportunity to perform some analyses of the data. First versions of the inventory guidelines and data collection template are now available to anyone interested in replicating our process. We hope this tool will be useful to others performing similar activities. The first set of data for Bangladesh, Chile, and Lithuania will be available in early 2010. Research teams in Brazil and the Philippines will begin data collection by January 2010, and their data should be accessible in mid 2010.

Download the inventory guidelines and spreadsheet.

Our next steps are to finish cleaning the data, test the database, and develop a system to enable yearly updating of the data by researchers in each country.

The program is an eighteen-month project sponsored by Canada’s International Development Research Centre (IDRC) and managed by Universitat Pompeu Fabra, Barcelona, Spain, in collaboration with scholars from Universidad de San Andrés, Argentina, and the University of the Philippines. It will award up to 12 Research Fellowships to teams of emerging scholars from developing countries in Africa and the Middle East, Asia-Pacific, and Latin America and the Caribbean. These fellowships will provide research grant funding and specialized mentoring guidance to enable Fellows to carry out a new original research study.

The Global Impact Study has identified several areas for in-depth investigation, and has started working on the first three:

• Infomediaries: public access brokers
• Collaborative knowledge sharing
• Non-instrumental use of ICT as a component of general ICT skill acquisition
• Mobiles and public access venues
• The life cycle of public access venues
• Community information ecologies
• Direct and indirect impacts
• Non-users
• Willingness to pay
• Policy and regulation
• Institutional and stakeholder influence
• The effect of venue rules and regulations on use
• The role of networks in venue ecosystems
• Local content
• Venue architecture and design
• IT skills, training, and employment
• The relationship between the costs and benefits of providing public access to ICT

The Global Impact Study team will start work on more of these areas — but we won’t be able to pursue all of them. Our goal, therefore, is to enable other researchers to take up our ideas to continue this research.

The Amy Mahan Research Fellowship Program is one way to reach this goal. We are excited to contribute to developing new researchers and furthering research on the impacts of public access ICT.

For more information on the Amy Mahan Research Fellowship Program, including application instructions, please visit the fellowship program website.

Photo of Amy by Bruce Girard, 2008.