Official Statistics and the Challenge
of the Current Health Crisis

The last few weeks have shown that the need for statistics has not disappeared with the current health crisis – quite the contrary, in fact. They have also shown that the official statistical system needed to be able to respond quickly and to rise to new challenges and it was able to do so. The first challenge was to adapt to remote working conditions and the idea that some data collections could no longer continue. A need has also arisen for new analyses and new outputs to shed light in real time on such crucial issues as the severe economic downturn and the extent of excess mortality. Increasingly, the challenge will be to ensure maximum international comparability of statistics at a time when we have all had to adapt and innovate in response to changing and often different conditions in different countries. All three challenges will be examined in greater detail below, in that order.

Like so many other organisations, INSEE and the Ministerial Statistical Departments have had to adapt their working practices since mid-March. Since information is at the heart of what they do, much of their work can be done electronically, and continuity of service can be maintained through widespread remote working. It is regrettable, however, that many municipalities are continuing to send their vital records by post and that after reaching 60% in just a few years, the online census response rate is now almost at a standstill. The time has passed when statistics was primarily a matter of collating responses in paper format and performing a huge amount of data entry work.

However, the introduction of the lockdown has disrupted collection work in a number of areas. While business surveys have been conducted online for several years now, the same cannot be said for household surveys, where online questionnaires are still at the experimental stage. Large-scale, in-depth surveys – it should be remembered that the Labour Force Survey and the Statistics on Income and Living Conditions (SRCV) survey, for example, are far more demanding than small-scale surveys or polls – often require an initial “face-to-face” survey interview with the household at home. Of course, no surveys have been conducted in this way for almost two months now, and, when the surveys have not been suspended, telephone questionnaires have become the norm, which may cause disruptions in survey responses. In-store price collections have been completely suspended, having been replaced, where possible, by online price collections, telephone questionnaires and, above all, by the use of scanner data, which, fortunately, have been used for statistical production purposes since the beginning of the year. As far as businesses are concerned, the proportion of missing responses has increased (as expected), and it seems likely that some administrative records (such as tax returns) will be affected by disruptions.

The unprecedented times we are currently facing also raise the question of the relevance of certain statistics. What value should be placed on the criteria for unemployment as defined by the ILO (actively seeking employment and available to take up employment) in a lockdown situation? What value should be placed on price index weighting when entire swathes of consumption have decreased or disappeared altogether?

We should therefore expect to see a temporary drop in the level of accuracy of statistics and, in some cases, an increase in methodological comments and explanations to assess or qualify their significance, which is something that INSEE has already begun to do on its various short-term indicators and will continue to do on a systematic basis.

Four times a year, INSEE carries out a very comprehensive short-term forecasting exercise, its well-known economic outlook report known as the “Note de conjoncture”. At a time when the economy is understandably dependent on decisions taken for health reasons, and so long as the conditions governing the spread and treatment of the virus remain so uncertain, conducting such forecasting exercises makes little sense. The economic outlook report planned for the end of March was therefore replaced with a real-time estimate of the fall in GDP and consumption, an exercise that has been repeated every two weeks since 26 March.

Conventional methods have proven ineffective or not sufficiently responsive. For the first estimate in particular, only short-term surveys conducted before the current lockdown were available. This has meant finding innovative approaches, rapidly collecting information communicated by professional bodies and businesses, and corroborating that information with instantaneous data on electricity consumption and credit card transactions or by using the first estimates of the number of employees who have stopped working. Few statistical institutes have been able to respond in this way, meaning that the first INSEE estimate (-35%, a figure that has since been confirmed) has enjoyed success well beyond our borders. However, it should be acknowledged that the pace of work required of the teams in having to repeat these exercises is hardly sustainable in the long term.

Sadly, the production and publication of death figures have also had to be adapted to the current circumstances. Each month, INSEE publishes the number of births and deaths that occurred the previous month, and the resulting publication usually goes unnoticed. Since the end of March, and on a weekly basis, INSEE has been publishing the number of deaths recorded up to the period ending 10 days prior to publication (10 days being the time required to receive vital records from most municipalities) at the national and departmental levels, as well as providing commentary on the data by comparing them to previous years. Over time, the institute has expanded its publications with tables disaggregated by gender, age, and type of place of death (hospital, retirement home, home). This has also required a considerable amount of work of critical importance since it is only by compiling all death certificates that changes in mortality rates can be measured accurately. It is important to remember that INSEE does not receive the medical section of death certificates and therefore has no knowledge of the causes of death, making it even more regrettable that the information is so seldom sent electronically.

Using data provided by a mobile phone operator, INSEE has also produced a map showing the place of residence of French people during lockdown. We are keen, as far as possible, to work with all operators, to analyse data at a finer level than the departmental grid, and to develop cooperation on one-day travel data. The fact that we have no source other than what Google or Apple have to offer, without fully sharing the methodologies used, should not be seen as an insurmountable obstacle.

The Ministerial Statistical Departments are not to be outdone. For example, DARES publishes a weekly dashboard used to monitor trends in short-time work applications in real time. The statistical department of the Ministry of the Interior is now disseminating statistics on the filing of criminal complaints on a weekly basis to monitor crime trends during these unusual times.

INSEE and the Ministerial Statistical Departments have also launched exceptional surveys as a matter of urgency. A key example is the “Acemo Covid” survey initiated by DARES with the support of INSEE and aimed at determining the distribution of employees among people who still work at their usual place of work, people who are working remotely, people placed on short-time working arrangements, etc. The first results were published on 17 April and the plan is to conduct the survey on a monthly basis. Another important example is the launch of the EpiCOVID survey at the end of April, the result of a partnership between INSERM, DREES and INSEE (especially for sampling) aimed at investigating the prevalence of virus symptoms and living conditions during lockdown among several hundred thousand people.

It is fortunate that the official statistical system has demonstrated its ability to respond, and it is also fortunate that it has benefited from external partnerships. The crisis may also be an opportunity for long-term cooperation. After all, tracking bank card transactions and analysing the travel movements of French people using mobile phone data will remain crucial issues as the lockdown eases.

It should be remembered, however, that we can only make full sense of such original data by referring back to structural statistics on the French population compiled through population censuses or on household consumption as measured by the national accounts. Traditional statistical sources and the new data sources used since the beginning of the health crisis are not in competition, but rather complementary.

After the initial shock, followed by the full realisation of the scale of the health and economic crisis, the time has now come for analysis, comparison and observation of what is and has been more or less successful both in France and in other countries. Cross-country comparisons will be increasingly in the spotlight. Here, we face an even greater challenge.

In many areas, statistical comparability is not a given, and a national institute, however well-intentioned, cannot by itself cover all the possibilities and limitations of international comparisons.

Consider the case of COVID-19 related deaths. The initial statistics provided by the health authorities of different countries only covered hospital deaths. While this may have seemed sensible at the beginning of the crisis, it is not enough to account for the actual death toll from the virus, especially with the excess mortality seen in retirement homes having become fully apparent. Many countries, including France, have supplemented this with information provided by residential care homes. But not all countries have, and many of the figures that continue to be reported in the media are not comparable across countries. Moreover, extending the data to cover deaths in retirement homes is not sufficient either since the vital records data published by INSEE also point to an abnormally high rate of at-home deaths in recent weeks.

In other words, it is only by analysing the statistics for all deaths that we will be able to accurately measure excess mortality. However, the seasonality of deaths, the varying severity of winter influenza each year and in each country, demographic characteristics, the health status of populations in different countries and other factors need to be taken into account. Comparisons between countries are possible provided that, for most countries, vital records are provided quickly enough and national institutes are able to publish them within extremely short time frames, as illustrated by INSEE and several of its counterparts in other countries.

Of course, when it comes to epidemiological analysis, we need to go beyond overall mortality. We need to know the number of deaths caused directly by the virus and to establish the induced effects on the other causes of death, which may be positive because of a slower spread of other infections or a decrease in car accidents, or negative because people choose not to seek care or become more isolated. It is important to recognise that cause-of-death statistics – covering all places where deaths occur – take a long time to be compiled and are difficult to interpret because of the frequent presence of several causes, known as comorbidity. In France, immediate electronic transmissions of the medical section of death certificates are, as noted above, very much in the minority, and hard-copy transmissions take several weeks. It seems highly likely that the choice of the pathological cause of death and, therefore, the treatment of comorbidity vary among physicians from country to country. The channels through which this information is delivered for statistical processing purposes also varies from country to country. For example, INSEE does not have access to such data, which are processed in an INSERM department, while other statistical institutes do have access to them and process them, such as the UK’s ONS or Italy’s ISTAT.

We will therefore have to wait until the number of deaths due to coronavirus, including deaths at home, is established, but we should also expect to encounter difficulties in making international comparisons.

We are seeing a similar need arising for international comparisons of economic indicators – a need that is only expected to grow. The publication of the first national accounts estimates for the first quarter at the end of April has already highlighted a number of counterintuitive results, such as a smaller decrease in GDP in Italy (-4.7%) compared to France (-5.8%) despite the outbreak occurring earlier in Italy and a longer lockdown period during the first quarter. Does this difference reflect an actual difference in economic activity trends, or can it be explained by differences in the methods used to prepare the quarterly accounts? As things stand, it is, in truth, difficult to answer this question.

When a first-quarter estimate is produced at the end of April, very little quantitative information is available for the last month of the quarter. Something that is not usually an obstacle in normal times poses a serious problem when the last month suffers a major shock, as was the case for March in all European economies. As a result, national accountants in every country have had to innovate, to do without the usual extrapolations, to use alternative sources and to pay attention to the work of short-term analysts. At INSEE, they determined that the fortnightly short-term reports referred to above were sufficiently robust to be used with confidence, having benefited from the information provided by bank card transactions, as explained in the methodological note accompanying the publication of the accounts on 30 April.

At this stage, we have no way of knowing whether statistical institutes in other countries have carried out similar processing operations. While it is clear that the main European economies have all experienced a decrease in GDP concentrated at the end of the first quarter, there is an assumption that the first accounts estimates are more fragile than usual and that the differences between countries are subject to significant revision.

It is possible that differences in labour market measures adopted in different countries, or indeed differences in the severity of lockdown measures, make it more difficult to make cross-country comparisons of indicators estimated based on the Labour Force Survey, including the unemployment rate. The use of different price imputation methods in closed down sectors may also affect inflation differentials.

In the future, official statisticians in different countries will therefore need to be able to respond to the increasing need for comparability. It is also my view that observers everywhere will need to be aware of this difficulty and to exercise as much caution and discernment as possible when commenting on differences between countries. ■

• The consequences of the COVID-19 health crisis: Economic, demographic and societal impacts

_{Photo credits: © stock.adobe.com}