In March 2010 Global Impact Study team members working on the in-depth studies, survey, and inventory activities gathered to harmonize impact indicators and discuss the project’s approach to communications knowledge sharing. We outlined our proposed approach to communications and knowledge sharing and in the discussion that followed researchers expressed the need for an internal communication space where Global Impact Study members can ask questions, request feedback, and share resources, reports, and any other materials that are not ready or intended for public consumption. TASCHA has established a Dgroup for this purpose.

For more information you can download the communications and knowledge-sharing presentation, or view photos from the workshop.

 The teams completed cognitive testing at the end of May.  Their reports draw attention to a number of areas where the survey questions could be improved. We can attribute the comprehension issues identified in the testing to a variety of factors including:

• Inappropriate respondent to answer the question (e.g. lacks knowledge to answer questions about a venue’s computer equipment or balance sheet).
• Use of technical jargon (e.g. log server).
• Inaccurate characterization of situation (e.g. asking for a single fee amount for internet access whereas venue has a tiered fee system).
• Inapplicability of question to respondent (e.g. does not engage in the activity a question asks about).
• Too many answer categories, making it diffcult to remember.
• Confusing wording of question.
• Complicated nature of information requested (e.g. respondent needs to estimate budget allocation to different venue expenses).
• Complex question structure (e.g. requires answers in multiple layers).
• Inaccurate translation from English to local language.

Some of these issues require revising the questionnaire, some require revising the language translations, and others require developing strategies to make the survey easier to implement (e.g. using visual aids for answer categories). We paid particular attention to those issues that occurred in more than one country, while trying to find appropriate compromises for items that were specific to individual countries. The Survey Working Group has revised the instrument and survey design where needed, taking into account recommendations supplied by the  survey implementation teams. The instruments are now moving on to field testing.

1. Cognitive testing to check for comprehension problems with the survey questions.

2. Field testing to check for potential problems in the entire survey adminstration process (e.g. structure, content, flow and length of the questionnaires) as well as  the actual data collected.

We begin with cognitive testing on a small scale, after which we make revisions to the survey instruments to address issues revealed by the cognitive tests. Next we carry out the field testing with a larger number of locations and respondents.

This strategy enables us to use the results of the cognitive testing to eliminate comprehension of survey questions as a potential major source of delays during field testing. For example, we are likely to get a more accurate estimate of the length of the surveys if interviewers don’t have to spend too much time trying to explain what specific survey questions mean.

Our survey implementation teams have been given detailed guidance in our survey testing guidelines.

1. Inventory and surveys that provide a big-picture view
2. Focused studies of specific mechanisms through which public access impacts livelihoods
3. An assessment of indirect and aggregated impacts, which takes the community as the unit of analysis and looks at non-users and alternative information sources
4. A look at alternatives and complements to public access, focusing on mobile phones

Get a big-picture view — To do this, we’re combining an inventory of all of the public access venues in Bangladesh, Brazil, Chile, Lithuania, and the Philippines with representative surveys of venue operators and users. With this we will start to understand magnitude, characteristics, distribution, costs, and impacts — especially on livelihoods. And researchers around the world will finally have access to a reliable database for further investigation of the public-access phenomenon in these five countries (the public database should be available in late 2010).

Dig deeper into specific impact mechanisms — Next, we examine how public access can improve livelihoods. Using a range of methods — ethnographies, focus groups, experiments, etc. — we’re digging deeper into the specific mechanisms leading to impact through in-depth studies of particular venue features: the availability of infomediaries, patterns of shared use, rules prescribing what users can and cannot do (can you play games? chat? update your Facebook profile?). Beyond a better understanding of how change happens, these studies will help us make better policy recommendations: Is it worthwhile to provide staff who can reach out and support users? Which venue design features foster productive sharing? Should gaming be banned or encouraged?

Look at the broader community — What about people who never even walk through the door of a public access venue? How does the venue’s presence effect the community as a whole? To answer these questions we’re conducting in-depth studies on indirect and aggregated impacts. These studies focus on public access use and the community information ecology — taking the community as the unit of analysis and assessing the use of public access venues against other information resources. This way, we can understand how all community members benefit from the presence of a public access venue (whether they use it or not) and learn more about venue reach within communities.

Examine alternatives & complements — There are several alternatives to public access: free or subsidized private computers (like the One Laptop Per Child initiative), infrastructure support (such as free WiFi zones), and computer alternatives (television, radio, mobile phones). Because mobile phones have become so widespread and promising, we’ll examine their use and impact in relation to, and in comparison with, public access venues. In South Africa, we’ll explore whether mobile phones replace or complement public access, or whether the two simply co-exist. In the Philippines, we’ll look at the impact of the provision of government services through mobile phones, comparing their impact to computer-based services offered through public access venues.

Together, these four components cover a range of complementary approaches and methodologies, hypotheses about how impact occurs, national contexts, levels of analysis, and impact areas. In combination, they will provide a multi-faceted understanding of the various ways in which public access to ICTs impacts livelihoods.

Download the full description of the research design updates (excerpt from November 2009 Interim Report).

With support from the municipality and BiblioRedes, a network 387 public libraries dedicated to improving public access and community uses of ICTs, this library/telecenter in Puerto Saavedra, Chile, provides books and information resources, a space for kids to do homework, Internet training, and help accessing online government services. (photo courtesy of telecentre.org)

What difference does public access to information and communication technologies (ICT) make in the lives of poor or marginalized people? Years of research have yet to produce concrete evidence of impact — as found in libraries, telecenters, and cybercafés. After conducting a comprehensive review of the literature in this area, we now know that there is a pressing need for systematic and comprehensive research to identify the downstream impacts, and to provide empirical evidence about the precise link (if any) between public access ICT use and impacts in areas such as health, education, and governance. The Global Impact Study is designed to address this gap.

Based on a year of exploratory fieldwork, we have created a research design that delves deeper and wider into public access impact than other studies to date. Our main research goal is to identify the observable impacts of public access and to gauge the magnitude and costs of these impacts. Our research design highlights six areas of inquiry:

1. Geographic and social reach of public access
2. Usage patterns
3. Physical design and layout of public access venues
4. Venue services and operational conditions
5. The ecology of information and communication resources within communities
6. Policy and regulatory influences

We hypothesize that there is a relationship between each of these six areas and the impacts emerging from use of public access ICT.

Previous studies have focused on one or two of these areas at a time. The Global Impact Study aims to touch on all of them to some degree. Additionally, out of the numerous areas of potential impact, this project will focus on:

• employment and income
• education
• civic engagement
• health
• democracy and government transparency
• culture and language preservation

ICT impacts are complex and diverse, so our research design combines multiple methods to examine different types and levels of uses and impacts.

First, inventories of public access facilities in four to six countries will provide a baseline count and description of existing venues against which we can assess the magnitude of the public access phenomena, reach, and distribution. Based on these inventories, we will be able to make some basic statements about the contribution of public access to the availability of ICTs.

Second, broad-based surveys of public access venues, users, and non-users in these same countries will narrow in on issues at an intermediate level, such as why people use public access ICT venues, and how operational structures influence the outcomes venues are able to engender.

Finally, a series of in-depth studies in an expanded set of countries will investigate key impact mechanisms using different methods to provide richer data than can be collected with the general inventory and survey methodologies. Questions being investigated include:

• What is the value-added of knowledge workers such as librarians at public access venues?
• Does the ability to engage in playful uses of ICTs at public access venues contribute to public access impact?
• Do public access venues facilitate valuable forms of collaborative learning among users?

We have identified several other potential areas of research, a selection of which will be pursued over the remaining four years of the Global Impact Study. These include public access model life cycles, indirect impacts, non-users, willingness to pay, local content, employment, community information ecologies, and mobile telephony.

While this study will not exhaust these research areas, our broader goal is to lay out a research agenda that provides a common framework and vocabulary, highlights key elements, and demonstrates how researchers can build on our work to contribute findings that further develop and refine our understanding of the public access ICT puzzle. To this end, we are adopting an open research approach where our research processes, tools, data, and findings will be accessible to others.

The primary output of the workshop was a shortlist of areas for in-depth studies to undertake at the beginning of phase two of the project. Workshop participants formed groups and drafted proposals for the studies. The groups are currently finalizing their proposals, with the first in-depth studies planned for early 2009.

Browse all in-depth study updates »

A community in Bangladesh gathers to discuss trusted sources of information with the Global Impact Study Bangladesh Country Research Team

This qualitative data collection tool allows a group to illustrate the networks of trust that exist in a geographic community. The tool works like a photograph of people’s personal networking experience. After developing several linkage maps, researchers can detect patterns in terms of trusted sources of information, or the information ecology around a focal point (a public access site in the context of this project).

Country Research Teams are visiting a minimum of six communities in their countries, each one with one or more of the public access venue types that are the subject of this study (libraries, telecentres, cyber cafes).

Francois Bar, Frank Tulus, and I joined the Bangladesh team on community visits on June 5, 2008.