People involved in community and school library projects in rural Africa know, from visiting them, that there are plenty of "libraries" (cupboards, shelves, one room affairs full of dust, closets packed full of Texas Board of Education cast-offs) that even a glance tells you have no effect on the reading population that is the target for the library. But more often the impact is intermediate, and a glance will not suffice. So how to tell whether the library is having an impact? And what if the donor demands that you have an impact assessment plan? What should you include? Here is my first cut of presenting library impact.
- Photos. If you don't have a picture of the library, with users reading books, then get one. Then plan on how you are going to get one or two more photos every couple months. I think a photo like this one speaks for itself, especially in a library that has been around for a few years and a blog site is steadily posting pictures like this. Anyone can fake statistics, but in a village setting it is pretty hard to repeatedly fake pictures like this by opening the library once a month and posing kids reading. Obviously it can be done- remember the faked moon landing?
- Testimony from "neutral" visitors. Every six months or so someone should be visiting the library to make sure that it has the appearance of functioning. If your visitor clearly finds a library obviously locked and no librarian, it is unlikely to be having an impact. The surest test is whether while visiting the library someone comes in and acts as if coming in were normal (rather than asking "What is this place?"). When I was visiting Sara library last fall, in Burkina Faso, a gendarme came in to return a book by Ahmadou Kourouma, and then grabbed another to check out. I chatted with him a bit... he checked out a novel a week... all he did as a gendarme was sit by the side of the road watching traffic and waiting for trouble... Did I need to measure how reading the books was "impacting" his life? Did I worry that maybe the librarian had paid him to "fake it"? No. Reading Ahmadou Kourouma can't be faked. Look to see whether kids walk in and grab a book. if they do, then clearly they are used to the place. Village kids are generally very shy around formal structures like libraries, and it takes a while for them to feel comfortable. Do they take off their shoes at the door? Do they lower the tone of their voices? Great signs that they are used to the place.
- Monthly librarian reports. The first obvious requirement for measuring impact. According to the librarian, how many readers does the library receive broken down by age and gender. How many books are checked out, if a lending library? How many kids attended library-sponsored activities like reading circles or quiz contests?
- Focus group with local community members. Your neutral visitor might also sit down and talk with a medium sized group- maybe the schoolteachers, maybe community leaders, maybe a women's group. Given enough time and some credibility (by having a local guide as companion), the discussion that ensues will tell a lot about the library functioning. Does someone actually have a story about a kid who studies a lot in the library and then passed an exam? If so, you are good to go!
- Survey of the local community. If you have a small budget and some local university-level manpower, then you can think of doing a user-satisfaction survey. With help from schoolteachers, you can have kids in 4th grade and up fill out questionnaires. What were the last three books they read? Do they use the library? Do they know the name of the librarian? Can they articulate how the library has helped them?
- Comparative survey. Compare reading patterns etc. between kids in the village with the library to kids in "matching" villages without libraries. This is the first cut attempt to see differences, but of course is fraught with what social scientists call selection bias- maybe you located the library in a community where kids were already reading well; or the opposite, you located the library in the most disadvantaged community. In either case, the straight comparison will be biased by the pre-existing conditions that are hard to capture and control for. Same goes for a pre-library and post-library survey or results of reading test. Lots of things change in a village over time- a new school director, different teachers, etc. might confound the effects of the library on reading.
- Randomized experimental impact assessment. If you have a million dollars, then you can establish 50 libraries in a randomly selected group of villages, and not establish villages in another 50 control villages, and then compare the differences after ten years. Yeah right.
- Randomly assess reading programs. This is more feasible, since you can allocate different kids to different reading programs sponsored by the library, and see which programs are cost-effective. One important caveat- anyone can improve reading with one on one tutoring for six months... so the reading program should be designed to be realistic in terms of costs, and not rely on presence of expensive managers or consultants. And the costs have to be measured. A program that costs $100 per kid may produce a 10% improvement in scores, and another program than costs $10 per kid produces a 2% increase in scores.... which is to be recommended?
Leave a comment