Initiatives
Tomnod

Tomnod initiative to crowdsource detailing of satellite images of remote areas in Nigeria

Burgess COMMENTARY

Peter Burgess

Mapping Remote Settlements

29 August 2014 by Luke

In the developed world, we've come to expect detailed maps, street views and Wikipedia entries for just about any place we go. But for many parts of the planet a detailed map - or even basic knowledge of where people live - does not exist.

Take for example the states of Kano, Katsina and Jigawa in northern Nigeria where more than 20 million people are spread across 100,000km2 of the most populous country in Africa. Particularly in rural areas, maps or estimates of where exactly these populations live simply don't exist. How would you hold an election here? Where should you distribute vaccines or aid? How could the government collect taxes?

A clickable map of Nigeria exhibiting its 36 states and the federal capital territory.

How many of Nigeria's 170m people have a good map of where they live?

While we may not have maps of every corner of the planet, we can rely on amazing, high-resolution satellite images. Image scientists at DigitalGlobe develop algorithms that attempt to transform these images into maps that identify buildings, objects and people. In particular, our 'High-res Urban Globe' (codename: HUG!) uses computer vision to extract shapes from the pixels that could indicate locations of human settlements. Unfortunately, computers don't get it right all the time...

That's where Tomnod comes in!

HUG identified 130,000 locations where human settlements might be. We showed each location to the more than 8,500 people who joined our Tomnod campaign and asked them to determine if there really were any buildings. In just a few days, we collected more than half a million votes! The results were quite impressive: with at least 4 crowd votes per polygon we were able to validate all the true settlements with great reliability.

Validating human settlements on Tomnod

The following chart summarizes our crowd's performance. The horizontal axis measures the crowd's likelihood of making two types of errors: False Positives (FP, blue) and False Negatives FN, red). FP is when a user identifies a polygon as containing buildings when the consensus of the crowd is that it doesn't. Vice versa, FN is when a user identifies a polygon as not containing any buildings when the crowd consensus is that it does. Turns out about 70% of our users are very good and never confuse actual buildings with non-buildings. On the other hand, about 10% of users consistently mistake non-buildings for buildings.

Tomnod crowd false-positive and false-negative error rates.

These performance metrics are important in that they allow us to weigh user input for a given polygon in order to come up with an accurate final answer whether that polygon contains a building or not. And what is the final answer? 50% of the 130,000 polygons do have buildings in them! Since many of these 65,000 polygons contain multiple buildings, we now have a far more accurate estimate of population density in rural northern Nigeria - and it's all thanks to some smart computers and an even smarter Tomnod crowd!

Luke Barrington leads the Tomnod team at DigitalGlobe. With a PhD in crowdsourcing and machine learning, he's been inventing ways for computers, crowds and companies to solve huge data challenges for over a decade. Follow @lukeinusa on Twitter