FAILFaire

(This article originally ran on MobileActive.org on April 16, and was written by Ian Thorpe)

On Wednesday evening I was lucky enough to attend the first ever “Failfaire”, organized by MobileActive.org, where several brave souls agreed to present their failed “Information Technology for Development” projects, explaining why they failed and what they learned from them.

I work on knowledge management at UNICEF, and have a strong interest in improving how we learn from our experience. This event (which was certainly not a failure!) was interesting to our work from at least two points of view:

1. The lessons learned from the projects themselves.

2. The idea for the event itself and whether this might be something we could try ourselves.

There were four presentations during the meeting:

Bradford Frost presented on Mobileimpact.org, a project to recycle old cellphones and donate them to Africa.

Matt Berg presented SMS Billboard, a project to use SMS technology to replicate the famous Liberian billboard blog in Kenyan villages, and PennyLED, a project to help power low cost lighting to the rural poor.

Chris Fabian and Erica Kochi of UNICEF presented “Our Stories,” a project to collect 5 million children’s stories.

Ian Schuler presented a project to use SMS to collate election monitoring reports in the Montenegrin independence referendum.

A few shared lessons emerged which might also seem familiar to us in UNICEF:

• Good intentions, a great idea, and great branding don’t necessarily make a good project. You need to focus on planning and execution and see whether you know how to do the project and whether it makes sense.

• Market research is needed. Before embarking on a project its important to ask yourself if there really is a need or demand for what you are doing, and whether there are already other solutions out there that might meet the need cheaper or better.

• People – not just technology – and process. Finding the right partners, listening to them and engaging them are critical success factors. A project that works well in one context might be ineffective in others if you don’t have the right partners and you don’t engage and make use of the skills and knowledge of the people you are working with.

• Make sure to pilot and test. Before scaling up a project, or before using it in a critical setting, make sure to have enough time to thoroughly test it and work out the kinks.

• Beware of “zombie” projects. If we are too attached to a “good idea” and have invested a lot of effort we are often unwilling to admit it is a failure and let go of it, and it keeps coming back from the dead, or it limps along unsuccessfully, not fully supported but still consuming valuable resources.

• Failures can lead to future successes. While a particular project might fail it can lead to new innovation and subsequent success. Look out for the learning and for the unexpected successful spin-off opportunities.

Some of these lessons might seem obvious with the benefit of hindsight – but it doesn’t stop them from recurring in development work.

As to the idea behind the event, I’m a strong believer in the value of learning from our mistakes if people would be willing to admit them and share them with others. This is challenging within a large publicly funded organization that places a lot of emphasis on delivering results and holding people accountable for them, but if we don’t do it we are at risk of continually repeating the same mistakes and in keeping alive our zombie projects because no-one wants to admit they are failing.

In terms of fostering this, I did get a few useful tips which came from the event itself:

• Make the event a safe space. Make it clear that this is about learning, not about blame. If necessary the moderator needs to reinforce this during them meeting, closing down any unhelpful discussions (which was done firmly but tactfully by Katrin during the event).
• Choose your speakers to be people who have also had notable successes. It’s easier to admit your failures when you are self-confident, and you will also have credibility because of your successes.
• Confidentiality: get people to agree on what can be shared outside the meeting and what stays in the room. I imagine that if we did something similar for UNICEF this might need to be internal only, with limited external sharing for people to feel comfortable enough with sharing.
• Don’t talk about money. The first presenter did talk about his financial outlay compared to the value of what he delivered, and it make sense in this context, but subsequent presenters were then quizzed on how much they had spent. I had the feeling that a focus on money can quickly lead to a focus how much was wasted and who is to blame which can then make people, especially in a publicly funded organization, very nervous about sharing. There are other means to look at these issues and I’d suggest that we stay away from them in our failure discussions if we want to promote honesty and learning.
• Focus not only on describing the failure but also sharing the lessons learned.
• Last but not least – ambience. Organizing the meeting with a sense of good humour, in a great location, with food and drink and socializing on the side is a great way to make people feel at ease and be willing to share and discuss frankly.

I’m now trying to plot how we can organize something similar within UNICEF. I’m sure we must have some juicy failures we can learn from and laugh about!

MobileActive hosted the inaugural FAILfaire last night, bringing together mobile technologists and NGOs to talk about failed projects in M4D and ICT4D. Presenters talked about their failed projects, answering the questions: “What was the project? What was the failure? Why did it fail? And what would you do differently next time?”

The event was filled to capacity with more than 70 people. The five presenters made us think (and laugh), and the audience asked some great questions. For those of you who couldn’t be there, here’s a quick look at the failed projects presented at the first (of what we hope will be many) FAILfaire.

Bradford Frost: MobileImpact.org? Not exactly…

Starting off the evening was Bradford Frost, who told the story of his failed non-profit venture, MobileImpact.org. The goal of his project was to bridge the gap between people trying to recycle used phones and developing countries. He felt he had a strong idea and a strong brand with the tagline “One phone. Change the World,” and that there was enough of an untapped phone recycling market (the current cell phone recycling market only captures about 25% of reusable devices) for the project to succeed.

However, the project didn’t work out as Frost had hoped. He used Facebook ads in order to target a younger, social media-savvy audience. He spent 1,000 dollars to launch an ad campaign and $5000 in a partnership with a phone recycling company. In the end, the non-profit gathered 131 phones valued at a sum total of …$252. And many of those phones were donated through word-of-mouth connections (friends and family) rather than people who saw the Facebook ads.

The low response and low quality of some of the donated phones made this a financially unsustainable project. He says that some of the failure came from not recognizing early warning signs – it was difficult for him to find partners to back the project, and it eventually folded. We also wonder that given the low prices, ubiquity, and phones-as-status symbols in many countries make recycled phones desirable at all. Brad ended his presentation joking, “I shudder when I hear people say ‘I want to start my own non-profit.’”

Matt Berg: SMS Billboard/Penny LED… #fails

Matt Berg (who we recently covered for his success with ChildCount and ChildCount+) presented two projects, SMS Billboard and Penny LED. The SMS Billboard launched in Ghana; Berg wanted to take a village billboard system (inspired by Alfred Sirless’ system of using billboards to share village news and photos) marry it with RapidSMS. This would allow villagers to have access to information more quickly than if reporters were sharing the news by mouth, expanding the scope of the billboards. However, the RapidSMS billboards didn’t catch on. Berg attributes this to being unable to recreate the level of passion and commitment to the news that Sirless and his sources had for the area, saying, “You can’t recreate passion.”

His second presentation focused on the Penny LED system. The project was designed to meet the needs for cheap, reliable lighting in developing countries. His group wanted to build converters so that people could safely use car batteries to charge their LED lights (the car batteries put out electricity in a higher voltage than the lights could support), using old copper pennies as the means of conducting electircty. They gathered copper pennies and set up a group of people to build the converters; however, the converters were expensive to produce and the likelihood for error high. But what really killed the project was the availability of other, better resources. Berg says that the project was essentially a failure in market research – cheap converters already existed, and LED flashlights produced in China were cheap and ubiquitous, making the project unnecessary.

Chris Fabian/Erica Kochi: 5 Million Stories by 2010? Nope. 400.

UNICEF Innovations’ Chris Fabian and Erica Kochi co-presented what they jokingly referred to as a “zombie project”, because despite the fact that the project couldn’t get off the ground, it kept being half-heartedly restarted over the years. “Our Stories” was designed to give children around the world the chance to tell their stories to be published online as part of a look at the global experience of childhood, with the ultimate goal of having 5 million stories posted by 2010.

Launched in 2007, Kochi and Fabian estimate the project had a .008 success rate, since it only gathered 400 stories. They say that this project was a failure of real world application, in that although the idea was good, there was no real desire for it among the community it targeted. As Kochi explained, “No one asked for this.” Other problems included using proprietary, non-open source code so that they couldn’t adjust when there were problems, a lack of ownership and commitment to the project by key stakeholders, and a long timeline that mean that resources never aligned with needs – in 2007 there was money for PR, in 2008 pro-bonoe design resources, in 2009 the software development. In 2010, they finally shelved the project.

Ian Schuler: Election Observation in Montenegro. Manual Refresh.

The final presentation of the night came from Ian Schuler, who presented his experiences with election monitoring via SMS during the elections for Montenegrin independence in 2006 (read our interview with Schuler here). The initial system was designed to let election monitors send in SMS messages from polling stations across the country to report problems and results. The first test of the system resulted in none of the volunteers receiving confirmation texts that their SMSes had been received. Because the protocol for not receiving a confirmation text was to call into the main database, all of the volunteers began calling at once, overwhelming the system. The phones were linked to a computer database that was compiling the SMSes, and the amount of data overwhelmed the system. The group solved the project by having a volunteer click through and refresh the list every 45 seconds.

This was a failure in timing – the project was started exactly two weeks before the election, not enough time to test and refine the system. Although it eventually got the results in, the group had promised they would have definitive results within 30 minutes of the closing of the polls – even by midnight, the group hadn’t released the results, although other news organizations were already reporting that the vote for independence passed. Although the group found a work-around, the project didn’t work as initially planned.

Since then and learning from the early failures in SMS election observaton, Ian’s former organization has greatly improved election monitoring with mobile technology such as in Ghana, for example. As someone in the audience noted durong Ian’s presentation: “It’s called ‘learning.’”

Final Thoughts

Thanks to everyone who came out to the event. FAILfaire is a completely open-source concept and all collateral on our FAILfaire site is licensed under Creative Commons license, including the logo, name, and content. Feel free to start your own FAILfaire in your city and your field! We think that openly sharing mistakes is a great way to prevent future failures, and we would love to see more FAILfaires.

Photo by Prabhas Pokharel

As we at MobileActive.org have been covering ICT and mobiles for development now for more than five years, we have seen our fair share of failures. For every great project that changes how a community benefits from technology to improve the lives of its people, there seem to be twice as many projects that fail, and end up wasting time, money, and maybe worst, goodwill.

Too often in our field, we talk up our successes, overhype and overestimate the value of our projects, and sweep the failures under the rug. But, if we don’t talk about what didn’t work (and, perhaps more importantly, why it didn’t work), others will keep repeating the same mistakes.

That is why we invented FailFaire, a gathering that is happening tonight in New York City and that we hope will take place in other cities around the world. FailFaire is a place where it’s ok to talk about what didn’t work to learn from for the next project using mobiles for social change and development.

Of course, there are different kinds of failure – some projects meet their basic objectives but are too expensive or unsustainable in the long run, some projects don’t fulfill their original mission but succeed in other (and sometimes surprising ways) ways, while some projects are just a flaming ball of fail. In the spirit of Hidin’s Top 10 Failure Points in Information System Implementation, we present here a sure way to fail in M4D: MobileActive.org’s Guide to Failure.

If you follow each of the headlines, you are sure to fail. If you read on below each invective, however, we point out ways to avoid utter failure. It’s not a recipe for success but may decrease the probability of failure and increasing the likelihood of helping people to have more free, healthy, prosperous, and dignified lives.

(more…)