How India gamed the already flawed “ease of doing business” rankings

Prime Minister Narendra Modi wants India to get into the top 50 list of World Bank’s “Ease of Doing Business” rankings. The index, which draws extensive media coverage, ranks 190 countries according to the competitiveness of their business environment.

The DB index aims to measure, across countries, the ease of starting a business, obtaining the relevant permits, accessing essential infrastructure, and so forth. It comprises 10 indicators, each of which is based on various sub-indicators, and all of which are aggregated, according to a fixed rule, into a final score that determines a country’s ranking among 190 economies. (Mint)

Where does India stand: India’s rank jumped from 142 in 2014 to 77 in 2018. This, the Modi government says, reflects the work done in the last four years.

But there are serious issues with this index: researchers have shown that massive movements in Doing Business (DB) rankings are mostly due to changes in methodology, not reality.

A core element of the recent “credibility revolution” in empirical economics is a push for researchers to pre-commit to a methodology, before they look at their data, so that the results can’t influence their methodological choices. That’s the opposite of how Doing Business works. (CGDev)

Few points:

1. The World Bank has repeatedly changed its methodology

From the Center for Global Development (CGDev):

In January, the World Bank’s Chief Economist, Paul Romer, told the Wall Street Journal the Bank had manipulated its own competitiveness rankings to undermine Chile’s socialist government, and hinted Chile might not be alone — then he retracted the claim.

In an interview with the Journal, Romer said:

“I want to make a personal apology to Chile, and to any other country where we conveyed the wrong impression,” Mr. Romer said. The problems with the report, he said, were “my fault because we did not make things clear enough.”

Mr. Romer said the World Bank is beginning the process of correcting the past reports and republishing what the rankings would have been without the methodology changes. He said he couldn’t defend “the integrity” of the process that led to the methodology changes. (WSJ)

This CGDev blog post has more details highlighting the issues with the index.

2. India’s rise also driven by methodological changes

Indian ranking is calculated based on data collected from two cities: Delhi and Mumbai.

From Mint, on the report released in 2017:

The headline numbers of the World Bank’s ease of doing business rankings show a spectacular rise in India’s rankings in 2018. However, this rise is driven more by methodological changes, and less by domestic initiatives, as research by Justin Sandfeur and Divyanshi Wadhwa of the US-based think tank, CGDev, show.

Their research suggests that if one holds the list of countries and the metrics to judge countries constant over time, the rise in India’s rankings is a blip, rather than a spike. Even if one adopts a broader set of parameters, the findings do not change much, they show.

3. Behind the scenes: How India gamed rankings

Kaushik Basu, former chief economist of the World Bank — who was involved with the DB report from 2012 to 2016 — and a professor of economics at Cornell University, wrote an op-ed rejecting the charges of data rigging (as in Chile’s case): “I can vouch for the multiple layers of checks and balances that are in place.”

But, he said, “there certainly are ways to influence the rankings without cooking up data.”

If a country is determined to move up the ranking, it can do so by focusing on the 10 indicators that determine the final score, though this is not a national economic strategy that I would recommend. (Mint)

This is exactly what a HuffPost India investigation pointed out this week. Based on a review of meeting minutes and interviews, the story shows how the Modi government, between 2014 and 2016, “sought to lobby the World Bank into changing its methodology to reflect a better rank for India”. That failed. In the next two years (2016 to 2018), “the government prioritised minor institutional and procedural tweaks to game the ranking system.” You can read the full story here.

Why it matters: Lant Pritchett, a Harvard University economist, said in an interview “that for developing-country policy makers, focusing on rising in the Doing Business ranks could draw scarce resources away from more-substantive reforms that would help the government better administer and enforce business regulations.” (WSJ)

“The pretense that Doing Business measures the real rules, and that if we just modestly improve these Doing Business indicators, they would somehow become the reality of what the rules are and how business is really done — I think that’s a very dangerous fiction,” he said. (WSJ)

Pritchett made this claim based on a 2015 study he co-authored with Mary Hallward-Driemeier, a World Bank economist, where they found that the Doing Business report doesn’t accurately reflect the experience of companies actually doing business in developing nations.