The Data Lake

In 2015, all UN Member States agreed on a 15-year plan to meet the Sustainable Development Goals (SDGs). These were 17 targets aimed at tackling urgent global issues in four key areas: climate change, poverty, human rights and gender inequality. In order to reach these goals by 2030, the United Nations also recognised the need to use big data as a foundation for better decision-making by governments. Leaders required more data to understand the poorest and most marginalised populations, for example, to reach zero extreme poverty as laid out by the SDGs. In response to this, the UN has played a key role in facilitating the data revolution and its integration into developmental policy-making.

Sustainable Development Goals adopted by UN member states in 2015

With the massive amounts of digital information being generated each day, international authorities saw huge potential for “data exhaust” (passively generated data) to be used for public good. When used responsibly, big data allows governments to gain a deeper understanding of their people and the environment. For example, survey data for income levels in less developed countries is known for being unreliable. Thanks to the lowering costs of technology, however, mobile phone payments have become increasingly widespread in these regions, and so more information about the people’s spending patterns has become available. Transactional data, combined with satellite imagery, can be analysed using big data techniques and can reveal the areas where people are most affected by extreme poverty, allowing governments to take focused action.

PulseSatellite uses machine learning to analyse satellite images. One of its features allows it to identify rooftops (right) in hard-to-reach areas like slums.

In addition to this, better access to technology has allowed all parts of society to be represented in some way or form within datasets, which means that previously hidden issues in society can be revealed through analysis. The informal sector is an area generally not represented well in official statistics because it is difficult to identify and measure. Analysing unstructured data in the form of social media posts and local radio transcripts reveal a more accurate picture of this part of society than traditional survey methods, that potentially let hidden humanitarian crises go unnoticed.

Despite the clear benefits presented by big data, it poses several major challenges. Although data has a lot of potential to do good, if misused, it could enable breaches in human rights. Without the correct measures in place, people in positions of power could exploit technologies to cause harm to ethnic or religious groups. Although the majority of data may seem anonymous at first glance, digital footprints are unique and so combining multiple datasets could result in the re-identification of individuals, potentially putting them in harms way.

Secondly, some countries do not have the technological infrastructure to benefit from big data. These include Least Developed Countries (LDCs), Land-locked Developing Countries (LLDCs) and Small Island Developing States (SIDS). It is important to focus on developing these areas by providing them with the same access to tools that ‘data-rich’ countries have in order to prevent widening inequality from big data implementation.

An Independent Expert Advisory Group was asked by the UN Secretary-General in 2014 to make recommendations on how to tackle these big data challenges. Some of the solutions proposed included improving data literacy, implementing regulations without stunting innovation, and distributing technology equally focusing on the less developed areas first. The UN’s vision for a data revolution has already started to take shape within its own systems, as big data technologies are being integrated into UN agencies, funds and initiatives, such as Global Pulse.

Five years after the SDGs were first agreed upon, the 2020 SDG report showed that with the support of big data, progress was being made. For example, global poverty levels reduced to 8.2% in 2019 and, without factoring in the impact of COVID-19, it was projected to reduce even further to 7.4% by 2021. But as governments have been faced with the worst global crisis in years with the coronavirus pandemic, they have never been more reliant on data and the data revolution has only just been accelerated.

Proportion of people living below $1.90 a day, between 2010 and 2021, forecast before and after COVID-19 (percentage) ¹

Sources:
¹ https://unstats.un.org/sdgs/report/2020/goal-01/
https://www.undatarevolution.org/measuring-sustainable-development/
https://www.unglobalpulse.org
https://www.un.org/en/sections/issues-depth/big-data-sustainable-development/