IN THE RICH world, people worry that prying governments know too much about them. Popular culture valorises characters who go off the grid, like Jack Reacher (the hero of 25 novels by Lee Child and two films starring Tom Cruise). He drifts around America on Greyhound buses, eschewing a driving licence, credit cards and email. Why does he make himself so hard to find? “It started out as an exercise and became an addiction,” he says.

The developing world, however, is full of unwitting Jack Reachers who leave little trace in official records. Their anonymity is not an addiction but an affliction. According to the World Bank’s latest World Development Report, entitled “Data for Better Lives”, about 1bn people have no official proof of their identity. More than a quarter of the world’s children under five are not registered at birth. Half of the 29 poorest countries have not completed a census in the past ten years—Congo has not done one since 1984—and only 40% have three or more estimates of poverty that can be compared across time. Christopher Yeh of Stanford University and his colleagues have calculated that an African household will appear in a representative survey of living standards less than once every 1,000 years.

Given the shortage of conventional statistics, many people are enthusiastic about more novel forms of data, gleaned from mobile phones, social media and satellite imagery. In the early months of the covid-19 pandemic, patterns of mobile-phone use showed who could and could not afford to stay at home in a city like Jakarta, outlining the uneven impact of lockdown measures in many developing countries. That kind of data can help donors better target their aid efforts. Emily Aiken of the University of California, Berkeley, and her colleagues have tested whether a machine-learning algorithm can identify the poorest households in 80 Afghan villages based on mobile-phone data, such as the duration of their calls, their network of contacts, and how often they paid for more minutes of call-time. For the 80% of households that owned a mobile phone, the algorithm worked about as well as more traditional targeting methods, such as counting fridges, clothes irons, and other physical assets.

But, as the study’s authors are careful to note, not everyone owns a mobile phone. And algorithms that work in one place and time may not necessarily travel well or endure for long. Joshua Blumenstock of Berkeley has pointed out that international calls may be a less reliable indicator of prosperity during the Haj pilgrimage season, when many more people travel.

Satellite imagery avoids some of these problems—cameras in orbit can see how the other half lives. Countries can sharpen poverty maps by combining household surveys with clues visible from space, including building sizes, forest cover, and the intensity of night-time lights. Tanzania was able to turn a poverty map divided into 20 mainland regions (containing populations of over 2m on average) into a higher-resolution poverty map of 169 districts (containing average populations of 300,000 or so). The extra precision provided by satellite data was equivalent to increasing the sample size of a household survey by five times, according to Takaaki Masaki of the World Bank and his co-authors.

These techniques can all improve the supply of data. But supply is not the only problem. If it were, you would expect any figures a government did collect to be highly prized, widely disseminated and heavily used. But the opposite is too often the case. Data are rarely shared with outsiders. And they are poorly digested even within official circles. Daniel Rogger and Ravi Somani of the World Bank once asked over 1,800 officials in Ethiopia how many people lived in their districts. About half of them thought their districts were at least 50% bigger or smaller than they actually were, according to figures in their own databases. Education officials over- or underestimated primary-school enrolment numbers by 76%. When asked, less than 13% said that these administrative databases were their main source of information.

Big Brother is watching out for himself

Governments often lack a strong incentive to collect data, use them well, or allow others to use them better. Bureaucrats with little discretion to make decisions have scant reason to inform themselves about what the right decision would be. And in countries that lack strong safeguards to prevent data misuse, civil servants understandably hold numbers close to their chests. Moreover, statistics can be most valuable when holding governments to account. Why should governments conspire in their own embarrassment by providing the data by which they will be judged?

For this reason, the authors of the World Development Report call for something akin to a cultural shift in the handling of data. They advocate for “political champions” who recognise data’s value in improving policies and a data-literate press and public who demand better numbers to draw on. This political commitment to data would, in turn, generate incentives for civil servants to make better use of the figures at their disposal. The mindset of ministries can be as important as a government’s financial wherewithal. According to new indicators from the World Bank, a country’s statistical performance is only loosely correlated with its prosperity. Some countries, like Kyrgyzstan or Mexico, do much better than you would expect given their level of GDP per person.

In this vision, private data and official statistics are complements, not substitutes, that can be of use for either corporate or public purposes. One current example is the Gallup World Poll, carried out by the private polling firm in over 140 economies. Every three years, it adds questions on financial inclusion instigated by the World Bank. This polling showed that 1.7bn people lacked a formal bank account in 2017. That gap is both a potential business opportunity for financial firms and a problem for governments. After all, even Jack Reacher has a bank account.

This article appeared in the Finance & economics section of the print edition under the headline “Our world, not in data”