Keen Eyes for Economic Trends: The falsification uproar and evidence-based policymaking

SATO Motohiro
Faculty Fellow, RIETI

Though it is only March, "fabrication" and "falsification" already seem like candidates for this year's top buzzword.

The fabrication uproar began when Prime Minister Shinzo Abe addressed a question about the working hours of those under the "discretionary labor system" (part of Japan's workstyle reforms) at a January 29 meeting of the House of Representatives Budget Committee. Abe answered that, "We have data that they [workers in the discretionary labor system] work fewer hours on average than even general workers." The grounds for Abe's answer were found in an FY2013 general survey on working hours, etc. (below, "general survey"), which concluded that people working under the discretionary labor system average nine hours and 16 minutes of work daily, or about 20 minutes less than general workers.

The survey was conducted by labor standards inspectors, who visited 11,575 business sites throughout Japan and asked them how much overtime the average worker at those business sites worked per day, week, month, and year. Afterwards, however, it was discovered that the survey was comparing different types of results. Looking at general workers, the survey inquired about the longest period of overtime they had served, but for workers under the discretionary labor system, it merely asked how many hours they had worked.

Moreover, there were occasional irregular results. For example, the data might show the same person working less overtime in a month than in a week. Or a person might have overtime hours in monthly and weekly units, but none in daily units. Or, the data might show more than 15 hours of overtime in one day. Accordingly, the government came under severe criticism from the media and others, who accused it of data fabrication. Prime Minister Abe's subsequent retraction of the claim did nothing to make the commotion go away. Instead, the parts about the discretionary labor system were removed from the bill on workstyle reform, and the Abe administration eventually abandoned incorporation into the bill during the current Diet session.

This uproar exposed a problem of an entirely different dimension from bureaucrats proactively trying to guess what the prime minister wants to hear ("sontaku" in Japanese). The Japanese government seeks to practice evidence-based policy making (EBPM) but lacks the ability to analyze the data providing the evidence.

There have been more than a few cases like this general survey, where the very responses to surveys were laden with errors. Thus, outliers are usually eliminated from the data during analysis. Instead of assuming that the data are correct, one has to scrutinize them to ensure their correctness. There is also a problem with simply comparing differences in work hours between general workers and those under the discretionary labor system. Even among general workers, there are differences, and there are also variations based on industry sector and the scale of the business. One ought to use statistical techniques that take these into account to verify if there is a significant difference.

First of all, the percentage of workers under the discretionary labor system is very low, both in the specialists field (R&D, etc.) and the business planning field, the fields subject to the system. The aim of workstyle reform is to expand the type of work subject to the discretionary labor system. Since the fields of work would be different after reform, it is hard to make the argument that today's working hours will apply just as they are now. Moreover, the fact that the data were in error does not mean we can come to the opposite conclusion—that the discretionary labor system tends to lengthen working hours. In any case, it will be necessary to take new surveys and verify them after reform.

If the government is truly committed to practicing EBPM, bureaucrats must improve at data analysis. If not, the government will have to publicly release the raw data from its surveys at an earlier stage and call on the expertise of outside specialists and researchers to examine them, which would include screening for data errors.

>> Original text in Japanese

* Translated by RIETI.

March 31, 2018 Weekly Toyo Keizai

May 21, 2018