A Straightforward Case of Fake Statistics

In the latest exposé of Rwanda’s poverty statistics, our experts reveal the methodical faking of statistical evidence. Until now the working assumption had been that this was a methodological disagreement with the figures but in the end it turns out to be a simple, straightforward (and easy to prove) case of fake statistics. The only reason it has taken so long to prove the manipulation is that our experts had not imagined the possibility that Rwandan authorities might have misreported their own results. This blogpost includes the excel files which will allow everyone, including non-experts, to check the findings. This also means that it will be impossible for the National Institute of Statistics of Rwanda and the World Bank to keep denying the evidence. Heads will have to roll.

This blogpost aims to explore the question of inflation in Rwanda, which has emerged as the last remaining issue required to resolve the disagreement about Rwanda’s poverty statistics. Using Consumer Price Index (CPI) price data, the National Institute of Statistics of Rwanda (NISR) (2016) and the World Bank (2018) claim that poverty decreased by 6 percentage points from 45% between 2010/11 and 2013/14, and then by a further 1 percentage point between 2013/14 and 2017/18 (NISR 2018). However, blogs posted on roape.net (see the series, Poverty and Development in Rwanda on the website) have shown that the price data contained in the Integrated Household Living Conditions Survey or Enquête Intégrale sur les Conditions de Vie des ménages (EICV) survey itself, as well as in the separate ESOKO dataset, indicate a much higher inflation rate over this period, resulting in a sharp increase in poverty over the same period.

This will hopefully be the last contribution from our side into this debate, as we hope that other researchers will now take over the task of checking and certifying these results. We are making all our syntax files from EICV1 to EICV5 publicly available here (EICV1, EICV2, EICV3, EICV4, EICV5) and encourage other researchers to use these and to correct any mistakes that we might have made in the estimation of poverty. There is still room for improvement in these estimates, especially for the earlier EICV surveys, where we had difficulties replicating official results due to issues with non-standardized measurement units.

In the first section, we will compare the inflation rates generated by three different price data sources, namely (1) EICV, (2) ESOKO, and (3) CPI, and estimate the poverty rates corresponding to each inflation rate. In the second section, we add non-food inflation to the assessment, in an attempt to get closer to the results obtained by NISR. Sections 3 and 4 examine the actual inflation rates used by NISR and the World Bank in their assessments, in order to ascertain the internal consistency of their results with respect to their own stated assumptions.

Food Inflation

This section looks at the inflation rates obtained using three different price data sources, namely (1) EICV, (2) ESOKO, and (3) CPI. Historically, NISR has always used the Rwandan Ministry of Agriculture and Animal Resources’ (MINAGRI) ESOKO price dataset to update the poverty line, which it regarded a more detailed and accurate reflection of the prices facing the poor, than the price data contained in the CPI data sets. In 2016, however, following the controversy that had surrounded the first EICV4 poverty profile (NISR 2015, Reyntjens 2015), NISR switched to using CPI data for updating the poverty line (NISR 2016). This switch was later endorsed by the World Bank as being both appropriate and accurate (World Bank 2018).

The different inflation rates are estimated as follows:

• EICV: The inflation rate is obtained by comparing the relative price of the food basket in each survey. From 2001 to 2011, we used the 2001 food basket (line l.1 in Table 5 below), and from 2011 to 2017, we used the new food basket estimated in 2014 (l.2).
• ESOKO: ESOKO price data are only publicly available on the MINAGRI website from 2009 onwards. Consequently, for the period 2001-2005 and from 2005 to 2011, we used the unweighted average value of the price index (January values) reported for in NISR (2012, p.12). From 2011 to 2014, we estimated the inflation in the same way as the EICV above (i.e. comparing the price of the 2014 food basket in both years) (l.3). For the period 2014 to 2017, we did the same thing. However, ESOKO prices are missing from the MINAGRI website for the period Feb 2015 to April 2017. To obtain a comparable baseline, we therefore had to exclude prices from October 2013 to April 2014 when estimating the price of the food basket in EICV4. The fact that food price data is missing for this crucial period, which was the peak of the El Nino drought, is itself a matter for concern, which should require explanation from MINAGRI/ NISR.
• CPI: The CPI food inflation is obtained from the following source: FAOSTAT =. Consumer Prices, Food Indices (2010 = 100). January prices were used for each survey year: 2001, 2006, 2011, 2014, and 2017. In this dataset, the food price index is only available from January 2004. Consequently, from January 2001 to January 2004, we used the General Price Index (i.e. food + non-food).

The results are reported in Table 1 below. They show that CPI food inflation has consistently been lower than ESOKO and EICV inflation since 2001. As explained by NISR (2012), this is due to the fact that CPI prices are heavily biased towards items consumed by wealthier urban households.

ESOKO and EICV prices, on the other hand, are similar all the way up to 2017. This suggests that EICV price data are adequate for updating the poverty line, as had been done in the ROAPE (2017) and ROAPE (2019) blogposts. The EICV inflation rate is only slightly higher than ESOKO for the period 2014-2017, when ESOKO price data are incomplete, as explained above.

Table 1: Estimated food price index (2001-2017) using EICV, ESOKO, CPI food price data (base: 100 in 2001)

Source: EICV, ESOKO, FAOSTAT

Using these three inflation rates, we calculate changes in poverty from 2001 up to 2017, starting from the official poverty line calculated by NISR in 2001. This poverty line is updated for inflation from survey to survey using the three different inflation rates, mentioned above. Since ESOKO and EICV only cover food prices, we only use the food poverty line, which was Rwf. 44,160 per adult/ year in 2001, compared to 64,000 for the total poverty line. For this reason, the absolute poverty levels are lower than those reported in our previous blogpost (ROAPE 2019), which used the total poverty line.

Despite this difference, the trends are very similar to those reported in earlier blogposts, with a 15 percentage points increase in extreme poverty between 2011 and 2017, when using the EICV and ESOKO inflation rates (see Figure 1 and Table 3).[1]

Figure 1: Extreme poverty rates estimated by updating the 2001 food poverty line using EICV/ ESOKO/ CPI inflation from 2001 to 2017.

Non-food Inflation

The results presented above, as well as those revealed in our earlier blogpost on roape.net did not include non-food inflation, since we did not have access to a reliable price data source for non-food items. The final poverty estimates should take into account both food and non-food inflation.

Since we do not have access to non-food price data, we will estimate non-food inflation indirectly by looking at the changing share of non-food consumption in total household consumption. The food poverty line is estimated in the same way as above, and then the non-food component is added to the food poverty line, based on the prevailing share of non-food consumption for 3^rd quintile households in each survey year.

If non-food inflation is lower than food inflation, the non-food share is likely to decrease, thus leading to a lower total poverty line (and thus lower poverty rate) compared to if only food-inflation had been considered. It should, however, be noted that the non-food share could decrease even if non-food inflation is as high as food inflation, by virtue of Engels law, which says that impoverished households (for instance, as a result of high food and non-food inflation) will have a smaller share of non-food consumption. The results presented here should therefore be considered as lower bound conservative estimates of changes in poverty.

The results presented in Figure 2 below yield a slightly lower final poverty rate than had been reported in our ROAPE 2019 estimate (61% vs. 64%), reflecting the fact that the non-food share of consumption decreased over this period, either as a result of lower non-food inflation and/or as a result of increased poverty/Engel’s law. However, the main conclusions of that blogpost are maintained when using the ESOKO/EICV inflation rates, namely (a) there has been a two-digit (12 percentage points) increase in poverty since 2011, and (b) total poverty is now higher than it was when NISR started to measure poverty back in 2001.

As before, the CPI inflation rate is lower and thus yields lower poverty levels in each survey. However, even with this inflation rate, we find a 10-percentage point increase in poverty between 2011 and 2017 (6). Furthermore, when CPI prices are used consistently to estimate both auto-consumption and the poverty line, the difference between the CPI poverty estimates and the ESOKO/ EICV estimates shrinks considerably (see Table 3 (10), CPI prices).

For comparison, we have also included a fourth inflation rate, here, namely the official total national CPI inflation (7). This inflation rate is not designed to reflect consumption patterns of poor people and should therefore not be used to update the poverty line, as it would have a much higher share of non-food items and be heavily biased towards items consumed by rich urban households.

The surprising finding is that even with this extremely low inflation rate, we still find a significant increase in poverty between 2011 and 2014 (+3 percentage points), and a net increase between 2011 and 2017 (+0.2 percentage points). This raises the question of how NISR (2016) and the World Bank (2018) were able to generate their much-touted decrease in poverty between EICV3 and EICV4, as we are not aware of any lower inflation rate than this one in Rwanda for this period. This question will be explored further below.

Figure 2: Poverty rates estimated by updating the 2001 food poverty line using EICV/ ESOKO/ CPI inflation from 2001 to 2017, and adding non-food component based on prevailing non-food share of consumption in each survey.

Examining NISR’s estimates

In this final section, we will use the consumption aggregates computed by NISR itself, rather than the ones that we have constructed ourselves. This is to control for the possibility that we might have made mistakes or introduced biases in the computation of the consumption aggregates that could affect the results.

As can be seen in Figure 4, the poverty rates obtained with NISR’s own consumption aggregates are almost identical for all inflation rates, to the ones reported above. In particular, they confirm that (a) there has been a double digit increase in poverty since 2011, using all price data sources (11, 12, 13), and (b) that poverty levels have returned to similar or higher levels than those observed when the NISR started to measure poverty in 2001, using EICV/ ESOKO inflation (11, 12).

The most striking finding, however, is that even when we use NISR’s own consumption aggregates, as well as NISR’s own price index, NISR’s own poverty line and even the very low untailored total national CPI inflation rate, we still get a significant increase in poverty between 2011 and 2014 (14). This is very surprising, since we have not been able to identify any assumptions of our own that could influence these results. The only transformation we have made to NISR’s own data is that we have centered the price index on its mean value in each survey, so as to remove the inter-survey inflation that NSIR had built into its price deflators.

To control for the possibility that it might be this price-index adjustment that could be driving the results, we have included two additional estimates: the first one uses no price index at all (15). The second uses the NISR price index, but instead of centering it on the yearly average value of the price index, we have centered it on the average January value of the price index (16). The rationale for centering the price index on January prices would be that NISR’s poverty lines and year-on-year inflation rates are expressed in January prices. Both alternative methods yield a higher 2011-2014 poverty increase than the original calculation, which centered the price index on yearly mean prices. This suggests that it is not our price index adjustments that are causing the increase in poverty between 2011 and 2014.

Figure 4: Poverty rates using NISR’s consumption aggregates, price index and poverty line.

To understand what might be causing this surprising result, we inspect the deflator used by NISR in its second EICV4 poverty profile (NISR, 2016). In that report, NISR re-estimated the 2011-2014 poverty trend backwards from a constant 2014 poverty line of Rwf 159,375 per adult/year. To achieve a comparable poverty rate, it had to deflate the 2011 consumption aggregates using a deflator reflecting the 2011-2014 inflation.

The mean value of this deflator was 1.00 in 2014, reflecting the fact that no deflation was required in 2014, since the poverty line was expressed in 2014 prices. In 2010/11, on the other hand, the mean value of the deflator was 0.96 (i.e. 4% lower than the 2014 deflator).[2] This implies an inflation rate of around 4.2% between 2011 and 2014. The problem is that this inflation rate does not match the inflation rate that NISR (2016, p.43) claims to have used to deflate 2011 consumption: 13.8%.[3] Had NISR deflated consumption by 13.8%, as it claimed to have done, the mean value of its EICV3 deflator should have been around 0.88, not 0.96. With a deflator of 0.88, NISR’s own consumption aggregates, its own price index, and its own poverty line of Rwf 159,375p.a./y would have yielded a 1.4 percent decrease in poverty between 2011 and 2014, not 6 percentage points as claimed by NISR.

A 13.8% total inflation rate would itself have been problematic, since NISR does not provide any price data source for this inflation rate. The official national CPI food inflation rate from January 2011 to January 2014 was 33.8% (see FAOSTAT), whereas NISR (2016, p.43) claims (without providing a source) that food prices only rose by 16.7% over this period.

The official total CPI inflation from January 2011 to January 2014 was 23%,⁷ not 13.8%. This would have implied a 4 percentage points increase in poverty between 2011 and 2014. As shown in previous sections, however, the CPI inflation rate was far too low, due to its bias towards rich urban households. The EICV and ESOKO inflation rates for this period were over 30%. After including non-food inflation, this would have meant between 6.7 and 9.3 percentage points increase in poverty,[4] using NISR’s own consumption aggregates, price indices and poverty lines (see Table 2).[5]

In summary, our findings indicate that the poverty trend reported in NISR’s second EICV4 poverty profile (NISR 2016), which was later endorsed by the World Bank as being methodologically and empirically sound (World Bank 2018), does not match its own stated inflation rate, which is itself well below the CPI inflation rate for the period, which in turn, was below the more reliable ESOKO and EICV inflation rates.

Table 2: Inspecting the deflator used by NISR between EICV3 and EICV4

	EICV3 mean deflator value	EICV4 mean deflator value	Inflation rate (2011-2014)	Poverty change
Actual deflator used by NISR*	0.96	1.00	+4.2%***	-6.9 perc. pts (46.0 to 39.1)
If NISR’s claimed inflation rate had been used	0.88**	1.00	+13.8%	-1.4 perc. pts (40.5 to 39.1)
If total CPI infl. rate Jan 2011-Jan 2014 had been used[6]	0.81**	1.00	+23%	+4.0 perc. pts (35.1 to 39.1)
If EICV infl. rate had been used	0.76**	1.00	+32.0% (food only)	+6.7 perc. pts (32.4 to 39.1)† (incl. non-food)
If ESOKO infl. rate had been used	0.73**	1.00	+36.6% (food only)	+9.3perc.pts (29.8 to 39.1)† (incl. non-food)

*obtained by dividing nominal consumption (cons1_ae) by deflated consumption (sol)

**estimate obtained by subtracting the 2011-2014 inflation rate from the average value of the price index

***Imputed from NISR price indices: Inflation= (mean 2014 deflator)/(mean 2011 deflator).

† Consumption aggregate is deflated based on food inflation only, and then non-food inflation is incorporated into the poverty line by adding the survey-specific share of non-food consumption to the original food-poverty line.

The World Bank’s Role

The question is how the World Bank (2018) could have endorsed such obviously flawed and misleading results? The World Bank was clearly aware of what inflation rate NISR used, since it noted in footnote 10, p.13 of its own paper that the NISR (2016) deflator implies an inflation rate of 4.71% for the period 2011-14.[7] It also knew that this inflation rate did not match the official total CPI inflation rate for this period, since it compared the two inflation rates in equations (6) and (6’). It must also have known that this inflation rate did not match the 13.8% inflation rate that NISR (2016, p.43) claimed to have used to deflate consumption between the two surveys.

Yet, the World Bank chose to ignore this information and to validate NISR’s results ‘as if’ they had used the official total CPI inflation. To do this, the World Bank proceeded in two steps: In the first step (section III.2.2.), it established the general validity of NISR’s chosen inflation rate by showing that if NISR had updated its original poverty line using the official total CPI inflation from 2001 to 2014, in much the same way that we have done in this paper, it would have ended up with a 2014 poverty line that was almost identical to the one calculated by NISR in 2014.[8]

In the second step (section III.2.3.), the World Bank then established the general validity of the method used by NISR (2016) to compare the 2011 and 2014 poverty rates. Since the general ‘validity’ of NISR’s inflation rate had already been ‘established’ in step 1, the World Bank seems to have decided that it could take as given whatever inflation rate that NISR used for its 2011-14 comparison, and concluded based on the generally valid method and generally valid inflation that ‘the official poverty trend, where headcount poverty declined from 46 percent to 39 percent between EICV3 and EICV4, is credible’ (World Bank  2018, p.14).

This unusual two-step validation process allowed the World Bank to avoid having to validate the specific inflation rate that NISR used for the period 2011-2014 (4.71%), which we (and they) know is nowhere near the official total CPI inflation for this period (23%). This process also meant that the World Bank was able to avoid having to look specifically at the 2011 poverty rate, which would have been around 36%, not 46% in EICV3 (see Figure 4 above and Table 4 below), using official total CPI inflation together with NISR’s own consumption aggregates, price index and poverty line. However, the World Bank could not have reached its conclusions without first calculating these intermediary steps and must therefore have been fully aware of the fact that poverty did increase even according its own preferred (but invalid) assumptions. Without fear of exaggeration, this is a shocking conclusion.

Conclusion

The results presented in this paper show that EICV and ESOKO inflation rates are almost identical at all points from 2001 to 2017, thus confirming that EICV was an adequate price data source for updating the poverty line as had been done in our previous blogposts (see all of the posts from 2017). Furthermore, the analysis shows that CPI food inflation rate is significantly lower at all points from 2001 to 2017 than ESOKO/EICV inflation, thus confirming NISR’s (2012) own assessment that CPI prices are inadequate for updating the poverty line. Finally, the blogpost shows that the discovery regarding a sharp increase in poverty since 2011 stands regardless of which method and which inflation rate (EICV, ESOKO, or CPI) is used to update the poverty line. These findings should hopefully close the Rwandan poverty debate once and for all, since they resolve the last remaining point of contention in this discussion, namely the inflation question.

The most surprising finding of this paper, however, is one that we had not checked for nor even imagined possible until now, namely that even when we use NISR’s own price index, NISR’s own consumption aggregates, NISR’s own poverty line, and even the lowest available inflation rate (i.e. the total national CPI inflation, which is not designed to measure poverty), we still find a sharp increase in poverty between 2011 and 2014, and a net increase in poverty between 2011 and 2017. This can easily be checked even by non-experts, by simply downloading the EICV3/4 datasets from NISR’s website, and using the variables already constructed by NISR.[9]

This finding provides the first direct evidence of statistical manipulation as it means that NISR reported results that corresponded to a 4.2-4.7% inflation rate between 2011 and 2014, instead of the 13.8% inflation that it claims to have used.

Most shocking of all, however, is the fact that our review of the facts clearly shows that the World Bank was aware of this discrepancy, but chose to ignore it and to work around it to ‘prove’ the validity of NISR’s results ‘as if’ NISR had used the official total CPI inflation. While, for reasons explained above, the World Bank’s paper unequivocally failed to ‘prove’ the validity of NISR’s results, our review shows that it does succeed in proving the World Bank’s complicity in manipulating and misreporting official statistics in Rwanda. We hope that those responsible for this scandal will be held accountable, as all the incriminating evidence now is publicly available and verifiable.

Featured Photograph: Jim Yong Kim, was the 12th President of the World Bank from 2012-2019.  This photograph shows  him with Rwanda’s president Paul Kagame on 27 March 2012. Kagame said on the appointment of Jim Yong Kim that year, ‘He’s … a leader who knows what it takes to address poverty.’

Notes

National Institute of Statistics of Rwanda. 2015. Rwanda Poverty Profile Report 2013/14

National Institute of Statistics of Rwanda. 2012. The evolution of poverty in Rwanda from 2000 to 2011: Results from the Household Surveys (EICV).

National Institute of Statistics of Rwanda. 2016. Poverty Trend Analysis Report 2010/11-2013/14

National Institute of Statistics of Rwanda. 2018. Rwanda Poverty Profile Report 2016/17

World Bank (2018), ‘Revisiting the Poverty Trend in Rwanda 2010/11 to 2013/14’, Freeha Fatima and Nobuo Yoshida, Policy Research Working Paper 8585.

References

[1] Some minor changes have been made to the EICV4 syntax file to improve inter-survey comparability (e.g. imputations of missing prices, etc.). These affect the 2014/17 trend, but not the final 2017 poverty rate, nor the overall 2011-2017 increase in poverty.

[2] This result is, rounded to the second decimal, identical to the one reported by the World Bank (2018): 0.955 in footnote 10, p.13.

[3] According to NSIR (2016, p.42), the inflation rate used to deflate 2011 consumption to 2014 levels was 16.7 for food (weighted at .659) and 9% for non-food (weighted at .341).

[4] Non-food inflation is estimated indirectly, as in the previous sections, by looking at the changing share of non-food consumption for 3^rd quintile households: Food poverty line in 2014 is given as 159,375x.6523. In 2011, the food share of the 3^rd quintile was .6388. The total poverty line for 2011 is thus calculated as (159,375x.6523)/.6388=162,845.

[5] These trend estimates are not exactly identical to those reported in Figure 5 above, as the method is slightly different: Firstly, consumption aggregates are deflated backwards to be compared to the 2014 poverty line, rather than forwards from the 2001 poverty line. Secondly we are not centering the price index in each survey, but using NISR’s own untransformed price index, which we are simply reducing by the required amount to match the stated inter-survey inflation rate.

[6] Source: FAOSTAT. Consumer Prices, General Indices (2010 = 100): Jan. 2011 =97.1; Jan. 2014 = 119.55.

[7] The 0.5 percentage points difference between the World Bank estimate and our own is due to rounding and to the fact that the World Bank used a population weighted average, whereas we used a simple unweighted average. The choice of assumption does not affect the conclusions in this case.

[8] Of course, this first step is beside the point, since NISR (2012) had already clearly established that the official total CPI is methodologically invalid for updating the poverty line, since it gives too much weight to non-food items and items consumed by rich urban households.

[9] For non-experts, the EICV3 & 4 files are available in excel format here and here.

Annex

Table 3: Poverty rates for different poverty lines and different price data sources (using our own consumption aggregates)

Table 4: Poverty rates for different poverty lines and different assumptions (using NISR’s consumption aggregates)

Table 5: Poverty lines (Rwf per adult/year) for different price data sources and inflation rates

Table 6: Food shares for different price data sources

Table 7: Average Kcal consumption per adult equivalent per day, using different price data sources