Using ‘red flag’ indicators to identify corruption and analyse reform efforts in the procurement process

Liz David-Barrett and Mihály Fazekas

The campaign to open up contracting around the world has had a positive consequence for corruption researchers: there is an abundance of data about public procurement (government purchases of goods, works and services from an external source through a tendering process). In our GI-ACE project, we use this data to develop new proxy indicators of corruption risk, based on ‘red flags’ in the tendering process, and then use that to test how patterns of corruption differ across contexts and whether anti-corruption efforts work.

The idea behind the ‘red flags’ approach is that public procurement is supposed to be an open, competitive process that follows a clear structure. That is the way to get value for public money and to encourage economic development. Any deviation from openness and competition, by contrast, may indicate that politicians or public officials are manipulating the process in order to steer contracts to cronies or political allies. By analysing big datasets of tenders and contracts, we can spot systematic deviations, or ‘red flags,’ and thereby gather an evidence base with which to hold officeholders to account.

In the first phase of our project, in 2016-17, we analysed data about multilateral development aid spent through public procurement in developing countries. We focused on aid partly because that was where the data were of sufficiently good quality. National governments have only recently – and often reluctantly – started to collect and publish contracting data.

Aid agencies have tracked their spending for much longer. For the World Bank, for example, we were able to collect high-quality data going back almost two decades and for more than 100 countries. The data were good enough for us to identify eight red flag indicators, associated with different ways of corrupting the public procurement process:

1. Single-bidding, i.e., where only one bid is received
2. Use of non-open procedures
3. Spending on consultancy, which is more difficult to scrutinise
4. Signature period longer than 14 days, which may signal negotiations over kickbacks
5. Advertisement period less than 14 days, which can exclude bidders without advance warning
6. Share of contract awards that are published, an indicator of transparency
7. Cost over-runs: final spend higher than original budget
8. Supplier being registered in a tax haven

When one red flag is associated with another, this increases the probability that the behaviour reflects corrupt manipulations, rather than just incompetence or lack of resources.

Analysing World Bank-funded projects in 1993-2007 in our 2017 working paper, we found that having only a single bidder for a tender increased the likelihood of cost overruns. When average single bidding increases from nil to 19 percent, for example, the likelihood of a considerable cost overrun (20 percent or more) increased by 9 percentage points (from 16 to 25 percent) This was after taking into account differences in project size, country of implementation, year, and main sector.

We also tested the impact of reforms. We found that a 2003 change in World Bank rules was effective in reducing corruption risks: it lowered the share of single bidding on competitive markets by 3.8-4.3 percentage points. Yet we were cautious about how to interpret this result. While it looks like an example of anti-corruption reform success, we were concerned that it could mask strategic behaviour by corrupt elites. Perhaps they were able to circumnavigate reforms by finding new and more sophisticated ways to manipulate the system to their advantage.

In a 2018 working paper, we found that although a change in the rules was effective in the area of the procurement process that it targeted, decreasing the share of single bidding from 22 to 18 percent, there also was evidence of evasive tactics on the part of some buyers. Some actors may have switched to using non-competitive procedure types to corrupt the process: the share of these tenders increased from 7 to 10 percent. Indeed, these ‘displacement effects’ largely cancelled out the positive direct effects, and foreign companies also lost out: their market share dropped by 2 percentage points.

In our latest work, we benefit from the fact that many more national governments have started to publish procurement data. We have been able to collect this data, on national – as opposed to aid – spending in Uganda, Paraguay, Chile, Mexico, Colombia, Jamaica, India, and Indonesia. This means that we can test the impact of very specific interventions and bring in more understanding of the specific context.

We plan to analyse a range of different interventions in our efforts to collect evidence on what works in anti-corruption in procurement. For example, elsewhere we show how political elites can abuse their power to facilitate partisan favouritism in the allocation of contracts, and develop a new method for measuring this based on how companies’ success on procurement markets changes after a change in government. We hope to use this methodology to analyse changes of power and partisan favouritism in some of our new country datasets.

Although high-quality analysis of the effects of interventions requires statistical skills and a deep understanding of the data, we want to make sure that our red flags method can be used widely. To this end, we have run outreach workshops in Tanzania, Ghana, and Uganda, in collaboration with the African Maths Initiative and their open-source tool R-Instat. Our ambition is to create a tool that can easily be used by people to collect evidence for holding governments to account – whether investigative journalists, civil society groups, audit institutions, or procurement regulators.

This blog was originally published here on the Global Integrity blog.