Category Archives: Lessons from Failures
By Maria Aguirre
The One Laptop per Child (OLPC) program seeks to improve learning in the poorest regions of the world by providing children with computers for use at both school and home. Since its start, the program has been implemented in 36 countries and has distributed more than 2 million laptops. In Latin America, the initiative began in the last decade, making the largest investment in Peru. However, recent research conducted by the Inter-American Development Bank (IDB) concluded that the OLPC initiative in this country failed to increase student performance in Math and Language. This post aims to discuss these results and to include analysis from other sources in order to promote a healthy discussion on this topic.
Recently a working paper by the IDB assessed the OLPC program in Peru, the country leading such initiatives in the region. With a tone of social inclusion, this initiative sought to primarily benefit rural, underprivileged communities. Since its inception, nearly a million laptops have been delivered to the students. However, the results according to this research are not encouraging. The document’s objective was to expose the lack of empirical evidence about the effects of ICT programs. The document evaluated the impacts after 15 months of implementation of the program, using a randomized control trial. The main result indicates an increase in the number of computers per student; yet, it finds no evidence of improvements in the enrollment or test scores in Math and Language.
Oscar Becerra, the person who was in charge of implementing the OLPC program from the Peruvian Ministry of Education, says in an article that “the effect is neither magic nor fast” but “it is a combination of interventions that will have long-term effects”. Becerra also points out that Peru usually suffers from the “vicious tradition” among politicians to stop their predecessors’ initiatives and start something new, regardless of the previous project’s success; yet, this was successfully prevented in the case of the OLPC program. Moreover, according to the document, the only positive result reflects a significant change in the development of cognitive skills, According to a separate blog entry posted by the IDB is a result entirely overlooked by the IDB study.
According to the World Bank blog one of the reasons why this happens -and by “this” I mean getting bad results from an initiative expected to be positive- is because of the way the program is being evaluated. This blog entry states that use of standardized tests instead of using tests developed by experts, can account for very different results. In addition, it states that “change doesn’t come unless you make real changes”; which means that change usually does not happen by a single discrete intervention. According this blog “dump hardware in schools, hope for magic to happen” is the worst practice of ICT use in education. Moreover, The Economist article follow the hypothesis that children learn much faster than teachers, and that teachers are not being prepared enough to keep it up with the technological change. Likewise, the article posted by the IDB indicates that because the OLPC program did not included specific interventions to integrate laptop use into the curriculum, an actual change in learning was not to happen.
Just as a concluding comment, I would like to say that yes, ICT use in education should be controlled, organized and promoted with support from the government and the school itself. Both teachers and students working together with this initiative that certainly DOES facilitate and promote learning. Is just a tool, but if used properly it should lead to wonderful results!
Error message, The Economist on April 2012. http://goo.gl/msMLw
Evaluating One Laptop Per Child (OLPC) in Peru, EduTech World Bank Blog on March 2012. http://goo.gl/Gd3Ha
Study: OLPC Fails Students as a Tool for Education, PC Magazine on April 2012. http://goo.gl/0khXU
One Laptop per Child program not improving math or language test scores, according to study; The Verge on April 2012. http://goo.gl/5WzPh
And the jury is back: One Laptop per Child is not enough, Inter-American Development Bank on March 2012. http://goo.gl/u6fv8
Oscar Becerra on OLPC Peru’s Long-Term Impact. EduTech Debate, on March 2012. http://goo.gl/sUKJT
Link to the IDB document: http://goo.gl/hLRsd
OLPC in Peru: http://goo.gl/fcse0
By Naoko Asano Enomoto
What do you think when you hear that the failure rate of ICT4D projects by the World Bank is about 70%? Is it higher or lower than you expected? This week, each of us, ICTs for BOP bloggers explores what we can learn from the failures of past projects. As other writers such as Anna discusses the lessons learned for a specific case, in this post, I would like to more broadly discuss how various audiences interpret the lessons learned differently. Let’s find out what the percentage I cited above means to you.
World Bank’s targeted ICT interventions were limited
The Hindu’s post, titled “World Bank’s targeted ICT interventions were limited: evaluator” reports the World Bank and it’s internal Independent Evaluation Group (IEG) released an Evaluation of World Bank Group Activities in Information and Communication Technologies, which is a review of a total $4.2 billion in World Bank committed to the ICT section during fiscal years2003 and 2010, of which about $2.9 billion was for supporting ICT sectors in “the poorest” countries. The four main domains of the World Bank ICT4D projects are: 1) ICT sector reform, 2) increased access to information infrastructure, 3) ICT skills development, 4) ICT applications.
The Hindu views it as a key finding that “limited in targeted efforts to increase access to the underserved beyond what was commercially attractive” with the result only 30 per cent of the projects targeting underserved groups succeeded and approximately 70 per cent of them failed. Through the interview with The Hindu, Laurent Besancon, Coordinator for the New ICT Strategy of the World Bank, stated that the bank’s future focus would be on policy reform to boost broadband potential in developing countries; and also on helping operators finance the “public good” aspect of infrastructure, such as setting up telecom towers in remote rural areas.
Congratulations to the World Bank!?
In relation to our initial question, The Hindi finds most of the results of the World Bank –related ICT for Development (ICT4D) projects during 2003-10, as somewhat “limited”. Many of you may support this view. For example, in his blog “ICTs for development”, Professor Heeks, who is one of the most engaging scholars in this field, stated the five broad reasons for why many ICT4D projects fail.
- Failure to involve beneficiaries and users: those who can ensure that project designs are well-matched to local realities.
- Rigidity in project delivery: following a pre-planned approach such as that mandated by methods like Structured Systems Analysis and Design Methodology, or narrow use of Log Frames.
- Failure to learn: not incorporating lessons from experiences that arises either before or during the ICT4D project.
- Ignoring local institutional capacities: not making use of good local institutions where they already exist or not strengthening those which could form a viable support base.
- Ineffective project leadership: that is unable to direct and control the ICT4D project.
(Excerpt from ICTs for Development)
However, there are other ways to interpret the same result. ICT works post provides interesting insights in this regard. It states that instead of asking why the past projects did such a bad job or calling for reduction of ICT investment, we should evaluate more about the transparency and risky practices of the World Bank. Furthermore, ICT works regards the number of 30% success rate for increasing access to the underserved as positively, compared with the “the 20% success rate of Silicon Valley start-ups who are coddled by the most business-conducive environment in the world”. Of course, it should not be easily compared with the situation of the World Bank’s ICT4D projects (relatively non-profit oriented) and Silicon Valley venture businesses (for- profit oriented). However, when we think about what is “failure ” or “Success”, especially in quantifiable terms, we have to be fully aware of evaluator’s lens as well as our own. Moreover, considering ICT4D is a relatively new field, it is most critical to share the failures with others. This transparency must replace a tendency to cover-up or hide failure, and must of course be accompanied by avoiding making mistakes (this is also very important!) in order to improve outcomes forICT4D in future projects. Some scholars and practitioners have already started an initiative to share the failures or “what does not work” in ICT4D projects. This initiative coined the term “Failfaire”, which offers online space to exchange knowledge and holds international conferences to connect people with same aspirations.
Let us know how you view the results of ICT4D projects illustrated by Anna, Maria and Fatima!
Sources and Footnotes
World Bank’s targeted ICT interventions were limited: evaluator in THE HINDU on Aug.11, 2011, http://www.thehindu.com/news/international/article2347766.ece
A Great Success: World Bank has a 70% failure rate with ICT4D projects to increase universal access in ICT works on Aug.17, 2011 http://www.ictworks.org/news/2011/08/17/great-success-world-bank-has-70-failure-rate-ict4d-projects-increase-universal-acces
Capturing Technology for Development in Independent Evaluation Group (IEG), (n.d.) http://ieg.worldbankgroup.org/content/ieg/en/home/reports/ict.html
Can a Process Approach Improve ICT4D Project Success? In ICTs for Development on Nov.29, 2011, http://ict4dblog.wordpress.com/?s=failure
By: Anna Greenstone
Alternative news sources have been following developments of the EASSy cable, damage and repairs over the last few weeks. The under-water fibre-optic cables, owned by the West Indian Ocean Cable Company (WIOCC), a conglomerate investment of 14 major telecom companies in Africa was laid in 2009. The recent damage to it has affected more than six countries in the Eastern and Southern regions of the continent. The cable connects countries along the East African coast to as far as the United Arab Emirates. In February, a ship dragging its anchor along the bottom of the ocean, off the coast of Mombasa, Kenya snagged part of the cable, causing limited internet access for users across several nations. All Things Considered, interviewed East African correspondent, Solomon Moore on 3/1/2012 sharing some of these developments with a broader audience. Moore explained that while Africa is less “wired” than other regions of the world, as new businesses sprout up, episodes like these cause an unfortunate delay and inconvenience.
AllAfrica.com, covered a story more recently, on 3/27/12 which focused on persisting internet problems in Zimbabwe. “Experts say Zimbabwe was hardest hit by the accident”, affecting internet speeds, and mobile phone users’ ability to add credit to their lines
How is Education wired in?
While much of the media following the fibre optic cable damage has focused on the burden to commerce and business, a sector becoming more and more reliant on tele-communications has been left out of the headlines. Schools around the world are increasing the use of internet and technologies for learning. While ICTs play an important role in enhancing our experience of work, school, and communication, as we become more dependent on them, if damaged, it will have more detrimental, deeply felt affect.
It is important for investors and educators alike to be strategic in finding sustainable ways to use ICTs in schools, particularly in countries with inconsistent infrastructure, or governance in place. Samsung Africa launched a pilot program last year in South Africa, with the goal of setting up solar powered computer schools. BizTechAfrica.com covered the project, and quoted a Samsung business leader in East Africa who spoke of plans to replicate the project in other places, Kenya being the next targeted country. The fact that these computer labs are powered by solar shows foresight by investors and designers, to not only consider more environmentally friendly ways of using technology in the classroom, but to be cognizant of the reality that electricity remains unreliable and inconsistent for much of this region.
While the damaged under-water EASSy cable certainly does not signify failure of ICTs in business or education, it presents a challenge to create more sustainable access to broadband, in addition to hardware and electricity. What can countries do to limit cable damage? Is stronger sea commerce regulation needed? Is the damage an expected risk for the fibre-optic cable industry? If so, what quality control, or quicker response operations could be instituted so users are not inconvenienced for such long periods of repair?
Foresight and design become more crucial as ICTs and internet play more integral roles in classrooms, businesses and homes.
BizTechAfrica 10/26/2011 “Samsung Africa launches solar-powered interned schools” http://goo.gl/03xdG
IT News Africa 2/28/2012 “Two East African undersea internet cables cut” http://goo.gl/9LxlC
AllAfrica.com 3/27/2012 “Zimbabwe Internet Problems Persist” http://goo.gl/Z3SNe
NPR 3/1/2012 “Damaged Ocean Cable Cripples Interned in East Africa: http://goo.gl/XYpDr
By Fatima Tuz Zahra
Telecenters are places where people can access computers, the Internet or other digital technologies. They are one of the major tools espoused by development workers and NGOs to provide information and education to rural and poorer areas. However for a variety of reasons Telecenters have not been sustainable without external funding. Most are short-lived enterprises that fail to forge long-term ties or participatory relationships with their local communities. This post will present the arguments made by “Telecenters are not “Sustainable”: Get Over it!” by Mike Gurstein and place it in the wider debates about ICT failures in educational development to analyze the reasons and assumptions behind those failures and why they should be used as lessons.
Talking about failures. Source: http://goo.gl/jZphl
“Telecenters are not “Sustainable”: Get Over it!”
Mike Gurenstein, Executive Director of the Centre for Community Informatics Research, Development and Training, and Editor in Chief of the Journal of Community Informatics wrote this post from notes for a talk given in an ITU sponsored workshop on Telecentre sustainability in mid-2011. In this post Gurenstein makes the simple argument that the nagging from funders – mostly government and major NGOS- about how Telecenters have to be made sustainable is ill-directed. Gurenstein says that centers funded by market mechanisms do not really add value and facilitate the education process in the community. Rather than serving a social benefit, they are more like Cybercafes that “provide computer/ Internet access to primarily young men to fulfill various fantasies via more or less violent games and other such pursuits.”
For Gurenstein, Telecenters are setup to serve and add value to disadvantaged communities by providing easy Internet access to those who lack capital to compete in the open market. Therefore it is “deeply hypocritical” of the funders to ask them to be sustainable and, more so, if run along market logic (i.e. self-sustaining profit), the centers will never serve their intended purpose. However Gurenstein also admits that individual Telecenters are also at fault. They usually have poor knowledge of the practices and long-term needs of the local community. He concludes that the Telecenters’ must be “embedded (“owned”) by local communities” to aid education.
The Debate over the Effectiveness of ICTs for Development and Education (ICT4D&E) Projects
The Gurenstein article is a good example of the debate going on between development practitioners over the effectiveness of ICT to solve problems of development. More and more development practitioners are pointing out that ICT4D&E projects need to be grown out of local needs and keep the long–term practical usefulness of projects in mind. Otherwise they risk being unsustainable and sometimes wastefully expensive. Kentaro Toyama echoes Gurenstien in his arguments: Modern technology can enhance learning in the classroom, but must be utilized only once the more exigent needs of improving teaching capacities and stronger administrations have been fulfilled.
The failures of Telecenters in so many rural areas and their fundamental lack of sustainability is simply one result of what Toyama has explained as the myth of scale. This refers to the popular argument that simply bringing the Internet to poor areas will transform them or throwing enough money at a problem will solve it. Such proponents often ignore the basic fact that technology is a multiplier of human intent and capacity, it is not a substitute. ICTs for education therefore must primarily focus on the needs of the local community. Otherwise it may do more harm than benefit the ones in need. Lack of proper guidance on how to use telecenters to learn about health, business, ICT usage and so on thus resulted in their misuse.
It is important to realize that the failures of ICT4D&E projects can often be instructive and a springboard to “spur innovation.” Recent initiatives, such as FAILfaire organized and run by MobileActive highlight failures of ICTs and assumptions behind bad projects to get insights into the reasons of failures and develop new successful ones. Additionally, websites such as “ICTs in Education: some ‘Reality Videos’” and videos such as “Top 7 Reasons Why Most ICT4D Projects Fail”, have become more common and foreground the fact that ICT is not the surefire solution to the complex problems of education or development.
Even mainstream platforms such as the New York Times have highlighted that mistakes are rarely discussed in organizations such as the World Bank. The World Bank has a 70% failure rate with their ICT4D projects on increasing universal access to education (for more on World Bank’s failure turn to Naoko’s article). This kind of prohibition on the part of large organizations often closes off avenues of learning from failure and limits innovation. One can ask if students can learn by making mistakes and from teachers’ feedback, why can’t the NGOs and international organizations spend some time learning from their failures?
Failures of telecenters have already taught us the importance of understanding the local context and custom better to sustain any educational development project. In today’s digitally connected globalized world a culture of sharing and talking about failures can help to avoid repeating past mistakes. To this end initiatives such as FAILfaire, and conversations initiated by experts such as Mike Gurenstein and Kentaro Toyama are certainly a step in the right direction.
Telecenters are not “Sustainable”: Get over it! In Gurstein’s Community Informatics on May 18, 2011: http://goo.gl/1lugv
Re-thinking Telecenters: A Community Informatics Approach in Gurstein’s Community Informatics on May 15, 2011: http://goo.gl/P562z
The Journal of Community Informatics: http://goo.gl/SEKJ8
Can Technology End Poverty? In Boston Review, November/December 2010: http://goo.gl/iF0eF
Response in Boston Review November/December 2010: http://goo.gl/bGSr5
There Are No Technology Shortcuts to Good Education. In Education Technology Debate on January 11, 2011: http://goo.gl/JjdzU
Nonprofits Review Technology Failures in the New York Times on August 16, 2010: http://goo.gl/4EFc
Top 7 Reasons Why Most ICT4D Projects fail: http://goo.gl/O6ywb
Mobile Apps for Development: Focus on Content By Users, Not Just For Users in MobileActive on March 28, 2012, http://goo.gl/nlG2V
Rip Van Winkle’s Surprise: Critical Perspectives on Mobiles in Development and Social Change on Sept. 28, 2009: http://goo.gl/rwKDu
How to Fail in Mobiles for Development: MobileActive’s Definitive Guide to Failure in MobileActive on April 14, 2010: http://goo.gl/lsZqF
ICTs in Education; Some Reality Videos in Wait…What? on March 21, 2012: http://goo.gl/LXC7P
How (not) to develop ICT literacy in students? In World Bank Blog on ICT use in Education on April 6, 2012: http://goo.gl/uRvqz
A Great Success: World Bank has a 70% failure rate with ICT4D projects to increase universal access in ICTWORKS: http://goo.gl/4oK3v