Toward the Implementation of Evidence-based Policymaking

Since the beginning of 2017, we have seen the publication of a series of books that provide easy-to-understand explanations of the estimation of causal relationships based on data, helping many people to recognize the importance of evidence-based policymaking. Steady public relations efforts undertaken by economists and relevant institutions have begun to bear fruit. With its importance known to legislators and administrative officers, momentum has been gathering to incorporate evidence-based policymaking into the administration process.

There will be a number of obstacles to overcome before such evidence-based approach takes root among those engaged in policymaking. Some policymakers are keen to improving the way of doing business and willing to try out something new. However, others may not be amused at all, anticipating that the introduction of a new system would increase their workload as well as feeling that their way of doing business is being criticized. In any event, evidence-based policymaking is a useful approach. Given the efforts undertaken in other countries, it is an issue that we must take seriously and will eventually have to address.

There is one report that could serve as a good reference in considering ways to have the practice of evidence-based policymaking embraced by those engaged in actual policymaking. The report, based on research commissioned by the Ministry of Economy, Trade and Industry, provides detailed explanations of how evidence-based policymaking has gradually taken root in the United Kingdom over the course of about 20 years (Note 1). According to the report, there are two types of evidence: one that provides an accurate picture of the actual state of affairs and the other that estimates policy effects in terms of causal relationships.

This classification is quite convincing. Consider the case of labor policies. Whether wages are on the rise or not is a difficult question as it involves difficult measurement problems, and yet the current wage trends as perceived by policymakers affect actual policy decisions. Suppose that we are to examine changes in wage levels since the end of 2012 to determine the impact of Abenomics. In this case, we would face a barrage of technical questions, including which price index should be used as a deflator to derive real wages and how to control for the effects of changes in the labor force composition that have occurred in the process of economic recovery. In order to calculate real wages, we need to use a price index. However, even when we limit the scope to consumer price indexes, there are multiple types and each of them has its own trend. Thus, the upward trajectory of real wages varies depending on which price index is used. Furthermore, labor participation rates for women and elderly people rose sharply in the process of the post-financial crisis recovery from 2009 onward, and that trend has been continuing to date. The resulting changes in the composition of the labor force—i.e., an increase in the proportions of women and elderly workers whose wages are relatively low—operate to depress the average wage. Usually, when we ask the question of whether wages have increased or not, we consider by how much wages for the same workers have increased, and therefore it is problematic that the composition of the labor force continues to change in calculating the average wage. As such, unless we properly address this and many other technical issues by developing new data, we cannot even tell for sure whether wages are rising or not. Failure to raise wage levels has been termed as the Achilles' heel of Abenomics, and such perception has had a significant impact on Japan's monetary and wage policies. However, the accuracy of the very perception is highly questionable.

As for the second type of evidence or the estimation of policy effects in terms of causal relationships, its importance can be easily shown by citing some specific examples. For instance, consider the effects of minimum wages on employment. It seems possible to increase income for poor households by raising minimum wages, if the implementation of such policy leads to an increase in the overall wage level without causing job losses. A typical counterargument from economists is that raising minimum wages would cause job losses, particularly among low-skilled workers, because small and medium-sized enterprises (SMEs), whose capacity to pay wages is relatively low, are unable to retain workers. In order to test this possibility, one could possibly examine whether the number of jobs has decreased in the prefectures that have raised the minimum wage. However, a mechanism is in place for automatically raising the minimum wage when the local economy is in good shape, and the average wage for workers in small enterprises rises. In other words, there is an aspect in the existing policymaking process whereby minimum wages go up only when and because the impact on employment is deemed insignificant. Meanwhile, policy proposals currently under consideration call for setting the minimum wage based on the cost of living in each prefecture or mandating a nationwide minimum wage of 1,000 yen per hour, regardless of the condition of the local labor market. Thus, in order to predict what would happen in a local labor market when those policies are implemented, we need to identify what would be the consequence of the proposed increase in the minimum wage, but independent of the condition of the local economy. It may seem impossible to carry out this task using data collected and developed over the years when the trends in local labor markets have always been a factor in making policy decisions. However, a series of econometric techniques can provide a solution, under certain assumptions, to this seemingly impossible task.

Now we understand that there are two types of evidence. Let's say that we have also read an introductory book on causal inference techniques and obtained a rough understanding. Here, suppose that a policymaker seeks to make policy decisions based on empirical evidence. One mistake that must be avoided at this point is unquestioningly believing and relying on findings from a single study. Any analysis, whether it is for understanding the actual state of affairs or causal relationships, involves making assumptions. For instance, if we are to estimate the average of changes in pay for individual workers from the change in the average pay for all workers, we need to assume that newcomers to the labor market are essentially no different from old-timers. Or, if we want to measure the impact of a minimum wage hike on employment based on prefectural data, we need to assume that each prefecture sets the minimum wage regardless of the condition of the local labor market. As changes in assumptions may result in different conclusions, it is problematic to place full trust in findings from a single study. Furthermore, the world of science is not governed by the rule of simple majority, meaning that certain findings are not necessarily correct simply because many research studies support them.

What is important is to give significant weight to reliable findings derived under acceptable assumptions and to know what sorts of issues are accepted as consensus among researchers. However, this will require highly skilled researchers—those capable of analyzing data on their own—to search and review a number of relevant research papers by investing significant time and effort. This is a literature survey, i.e., a survey of research papers written and published by other researchers. In other words, even if policymakers want to know the rough consensus on the effectiveness of a certain policy, it is extremely difficult to do this without help from researchers. Also, literature review written by a researcher for other researchers is often too technical for policymakers to understand. Probably, this is one of the problems that policymakers are bound to face in shifting to policymaking based on empirical evidence.

The good news is that at least so far as labor policies are concerned, there exists an online platform designed to inform policymakers about consensus within the academia. What I am referring to is the IZA World of Labor hosted by the Institute of Labor Economics (IZA) of Germany, a partner research institute of RIETI. Articles published on the IZA World of Labor provide a concise summary of research studies on various labor policies and their findings. Each article has an elevator pitch so that even the busiest corporate executives would be able to grasp the essence of the article in the time span of an elevator ride. As such, the site is very useful and helpful. Unfortunately, however, there are very few articles on Japanese labor policies. Although cases in other countries often provide a useful reference in evaluating Japanese policies, it is dangerous to believe that research findings in other countries are directly applicable to Japan because assumptions for analysis vary depending on the country and time period. Especially in the case of labor policies, it is necessary to ensure that differences across countries are duly taken into account because each country has its own unique labor market structure due to various historical backgrounds. In this regard, a recently published book (Note 2) would serve as a useful reference. Published in commemoration of the 70th anniversary of the Institute of Statistical Research, which has been serving as a bridge between policymakers and policy researchers, the book provides summaries of research studies and their findings to date on various labor policies, written in an easy-to-understand language by leading researchers in their respective fields. I hope that this book will find many readers among central government officials and legislators, and contribute to evidence-based policymaking in Japan in the area of labor policies.