With the data from this survey, we hope to gain a greater understanding of the differences and similarities between public access ICT users and non-users, reasons for non-use, and the extent to which use of public access ICTs might give users unique advantages over non-users. We have updated our original survey research questions to include the non-user element.

Survey Research Questions

1. What is the demographic profile of public access ICT users and non-users?

2. Apart from public access ICTs, what other information and communication resources do public access ICT users and non-users have?

3. How do public access ICT users and non-users perceive their ICT skills and what are their ICT use comfort levels?

4. Why do people go to public access ICT venues?

5. What are the reasons for non-use of public access ICT venues?

6. What do people do at public access venues?

7. How accessible are public access ICT venues and services to different types of populations?

8. How do the design, services and operations of public access ICTs affect usage patterns?

9. What do public access ICT users see as the impacts of using public access ICTs?

10. Do non-users see an impact from not using public access ICTs?

11. What outcomes can be associated with public access ICT use in different domains?

12. Are the outcomes non-users experience from using non-public access information and communication resources similar to the outcomes experienced by users of public access ICTs?

13. Does public access ICT use have indirect impacts?

14.What is the value of public access ICTs to users?

15.  What is the cost of providing public access ICTs?

 The teams completed cognitive testing at the end of May.  Their reports draw attention to a number of areas where the survey questions could be improved. We can attribute the comprehension issues identified in the testing to a variety of factors including:

• Inappropriate respondent to answer the question (e.g. lacks knowledge to answer questions about a venue’s computer equipment or balance sheet).
• Use of technical jargon (e.g. log server).
• Inaccurate characterization of situation (e.g. asking for a single fee amount for internet access whereas venue has a tiered fee system).
• Inapplicability of question to respondent (e.g. does not engage in the activity a question asks about).
• Too many answer categories, making it diffcult to remember.
• Confusing wording of question.
• Complicated nature of information requested (e.g. respondent needs to estimate budget allocation to different venue expenses).
• Complex question structure (e.g. requires answers in multiple layers).
• Inaccurate translation from English to local language.

Some of these issues require revising the questionnaire, some require revising the language translations, and others require developing strategies to make the survey easier to implement (e.g. using visual aids for answer categories). We paid particular attention to those issues that occurred in more than one country, while trying to find appropriate compromises for items that were specific to individual countries. The Survey Working Group has revised the instrument and survey design where needed, taking into account recommendations supplied by the  survey implementation teams. The instruments are now moving on to field testing.

1. Cognitive testing to check for comprehension problems with the survey questions.

2. Field testing to check for potential problems in the entire survey adminstration process (e.g. structure, content, flow and length of the questionnaires) as well as  the actual data collected.

We begin with cognitive testing on a small scale, after which we make revisions to the survey instruments to address issues revealed by the cognitive tests. Next we carry out the field testing with a larger number of locations and respondents.

This strategy enables us to use the results of the cognitive testing to eliminate comprehension of survey questions as a potential major source of delays during field testing. For example, we are likely to get a more accurate estimate of the length of the surveys if interviewers don’t have to spend too much time trying to explain what specific survey questions mean.

Our survey implementation teams have been given detailed guidance in our survey testing guidelines.

Development of the surveys has been a difficult process of trade-offs. Most people who have participated in surveys on ICT use can probably identify with the feeling that these surveys are often extremely long, some taking up to two hours to complete. And researchers will admit that once the data is collected, a large proportion of it goes unused, it is never analyzed. We’ve had to make some difficult choices about the scope of our surveys of public access ICT venue users and operators. There are several relevant issues that we would like to explore, from the demographics of public access users to the impact of usage on users’ health, education and participation in civic life. The first draft of the user survey compiled over 60 pages of questions on several of these interesting issues - a survey that could potentially take more than three hours to complete. Since our goal is to keep the surveys less than one hour long, we needed a systematic approach to decide which questions would be included in the final version. To do this we prioritized the research questions we would investigate with the surveys, developed hypotheses to go with each research question, and identified potential indicators to test each hypothesis. This process enabled us to select questions for the instruments that are directly related to our research questions, and provided a means to make the hard decisions of eliminating questions that are “interesting” but fall out of the scope we have defined for the surveys.

The research questions for the surveys are:

1. What is the demographic profile of public access ICT users?
2. Apart from public access ICTs, what other information and communication resources do public access ICT users have?
3. How do public access ICT users perceive their ICT skills and what are their ICT use comfort levels?
4. Why do people go to public access ICT venues?
5. What do people do at public access ICT venues?
6. How accessible are public access ICT venues and services to different types of populations?
7. How do the design, services, and operations of public access ICTs affect usage patterns?
8. What types of venue policies do public access ICT operators view as inhibiting their ability to provide information and services that they believe users want?
9. What do public access users see as the impacts of using public access?
10. What outcomes can be associated with public access ICT use in different domains?
11. Does public access ICT use have indirect impacts?
12. What is the value of public access ICT to users?
13. What is the cost of providing public access ICTs?

The survey instruments are in the final stages of editing before being translated into local languages and piloted. Results from the pilots will inform the final survey instruments, which will be rolled out in early 2010 in Bangladesh, Brazil, Chile, and the Philippines. We are looking forward to sharing the final instruments and additional insights about our survey development process.