By Japan SPOTLIGHT
PDF version [PDF:435KB]
Evidence-Based Policy Making (EBPM) has been mentioned in Japanese government official documents since 2017. Each ministry of the Japanese government has been setting up a new section to implement EBPM these past several years. But there are still issues to be resolved in acquiring high-quality evidence to enhance the effectiveness of public policy. EBPM can be considered a cost-benefit analysis of a policy’s usefulness in persuading the budget authorities to allocate money, with a quantitative analysis demonstrating objectively that its benefits exceed costs. As such, the transparency of the policy making would be enhanced.
Prof. Daiji Kawaguchi of the Graduate School of Economics at the University of Tokyo has been working on this issue at the Research Institute of Economy Trade and Industry (RIETI) as a key researcher.
(Interviewed on Jan. 26, 2022)
Introduction
JS: Could you please introduce your academic history with EBPM?
Kawaguchi: I was awarded a Ph.D. in Labor Economics from Michigan State University in 2002. In the domain of Labor Economics, there have been numerous quantitative analyses by using micro data up to now. A typical example is the analysis on the impact of the minimum wage on employment. So I have been used to such quantitative analysis to detect the impacts of an economic policy from such data. This is how I got interested in EBPM.
JS: EBPM was introduced in the Japanese government in autumn 2017. How do you assess the utilization of EBPM at this moment within the Japanese government?
Kawaguchi: I think it has been gradually established with budget allocations in the Japanese government. But there seem to be a number of issues emerging that need to be resolved, such as the shortage of available data to be used immediately for cost-benefit analysis or the shortage of experts on quantitative analysis in both the public sector and the private think-tanks.
Human Resources Necessary for EBPM
JS: Will it be necessary for Japan to develop experts on data building and quantitative analysis?
Kawaguchi: Yes, I think so. Policy assessment is included in EBPM. This is analysis for finding whether a certain policy has achieved what was originally intended or not. For example, analysis on the possible impact of a rise in the minimum wage in a region upon local employment or the possible impact of the accommodation of day-care facilities upon the percentage of working mothers in the region. In such examples, it may be easy to find out the correlation between the two variables but difficult to discover their cause and effect relations. This issue can be resolved by experimental econometric methods such as randomized controlled trials, difference in differences, methods of instrumental variables or regression discontinuity design. In applying these methods to the data on a specific policy, highly professional skills would be necessary for customizing the methods to each policy under assessment. The skilled experts on data analysis would have to select data and econometric methods and also interpret the outcome appropriately. Experts on EBPM would also have to think about a scheme for utilizing the evidence for policy making.
JS: It must also be necessary for those experts to explain to people about the outcome of EBPM in easy language. What would be needed for such communication?
Kawaguchi: When it is decided to implement a policy, it would be necessary for politicians to integrate a wide range of views including the experts’ knowledge. In this process, each player involved in policy making has a role. The administrators have to digest a variety of views and integrate them, and provide the integrated information to the Diet members responsible for policy decisions. Journalists listening to various interest groups’ views on the policies should examine them and issue their views to the public. Politicians themselves would have to meet the residents in their constituencies in person and listen to their views and integrate them into the information provided by the administrators.
I think the role of the journalists is important in communication with the public. They would have to understand the essence of the experts’ research outputs and convey it to the public. There is a section of journalism on data in The Economist or New York Times, a collection of a number of articles on data analysis. In Japan as well, the Nikkei Journal has started such a section. I believe the expansion of these trials would lead to experts’ knowledge being shared among the public. There are some press people with backgrounds in data science working as a team inside media companies to promote understanding of data analysis among ordinary people.
JS: Do you think a sort of revolving door in which experts in the academic world are sometimes involved in the real policy making in government would be necessary to develop such communication skills, while the policy makers learn about the academic skills of quantitative analysis?
Kawaguchi: Yes, that would be ideal, but in reality it is difficult for busy policy makers to spend much time learning the skills of econometrics. We can provide them with basic learning about EBPM, what it is and how it would contribute to refining policy making. Meanwhile, the government could employ postgraduates with doctorates in economics to work as data specialists for it. I do not think academic experts who have never worked on public policy could work well as government officials. The government officials have been trained as such since their young days. Each of them should have their own job – academic research for the scholars and administration work for government officials. I would say the most important thing is to respect each other’s job and collaborate for a common goal.
I think it is very important for policy makers or journalists to focus on the issues from meeting with the people to be affected by any policy that they are working on. This could help to convince ordinary people of the policy’s utility. EBPM is a process to prove whether these issues would represent well the economic development that necessitates the policy.
Areas Where EBPM Is Often Used
JS: You have been specializing in Labor Economics and EBPM has often been used in this area. In what other area is EBPM often used today?
Kawaguchi: It is used now in almost all areas. SME-supporting policies, the Education Ministry’s policy for promoting classes of small numbers of students, and the Global and Innovation Gateway for All (GIGA) initiative for promoting Information and Communications Technology (ICT) at schools are some examples. Japan’s Fair Trade Commission’s examination of a merger case would be another example, in the sense that quantitative analysis would be needed to measure a merger’s impact on the market. Quantitative analysis has been in general applied to examine the economic effects of the Bank of Japan’s monetary policy and the Ministry of Finance’s fiscal policy since a long time ago. So I believe the concept of EBPM is now being used in all these areas.
JS: We see increasingly EBPM applied to social policy areas such as raising the number of daycare facilities for working parents or education policy.
Kawaguchi: In EBPM, we must detect what would be different between the outcome with a policy and the outcome without one. In the case of macroeconomic policies such as monetary policy, it would be extremely difficult to detect what state the economy would be in without a monetary policy. However, in the case of a policy for enriching daycare facilities, we can easily compare the differences in working styles between working mothers in areas with enhanced daycare facilities and those in places without them. So in such policies, we can see who benefits and who does not simultaneously. In the case of macroeconomic policy, it is extremely difficult to compare directly the state of the economy with a macropolicy and the state without it. I believe, therefore, that EBPM can be more easily introduced in smaller units of the economy.
Another example could be EBPM for JETRO’s support program for promoting SMEs’ exports or Foreign Direct Investment. Policies which could have a chain reaction in the economy such as changes to the regulatory environment or the business environment in general would make it difficult to distinguish a policy’s impact from those secondary changes. We can of course build up knowledge in defining the details of a policy’s impact even in such cases.
JS: EBPM is allegedly widespread in the United States and Europe. How about Asian countries?
Kawaguchi: In Singapore, government officials are well paid and there are some highly competent economists working for the government. In China and South Korea as well, there are post-doctorate economists working on economic and quantitative analysis and providing policy advice close to the core of the administrations.
Next Step for EBPM
JS: Is EBPM applied to the assessment of a policy performance prior to implementation?
Kawaguchi: Yes. EPBM is a scientific approach to an expected policy performance by learning about past events and assuming the structure of the outcomes of the events unchanged, and extrapolating it into the future. It is like a clinical test of a medicine. Though a medicine is given to different patients from those in a clinical trial and thus has not been perfectly proved effective for these patients, we assume that a medicine proved to be effective in a clinical trial would be effective for all patients. Likewise, learning from policies that worked well in the past, similar policies under similar circumstances are assumed to be most likely to work well. This is the thinking behind EPBM.
JS: What do you think about the future possibilities for EBPM?
Kawaguchi: Whether EBPM can be fully developed or not will depend on whether policy makers can take advantage of the budget acquisition process as an incentive for EBPM. Today in the Japanese government, it is recommended to apply EBPM to large-scale policies requiring more than a 1 billion yen budget. To achieve EBPM effectively, the government will need well qualified external experts in data and quantitative analysis. The experts in policy assessment inside the government and those external experts must collaborate for effective EBPM. The application of only a logical model on the cause and effect of a certain policy without data would occasionally result in poor policy performance.
For example, education for classes with fewer students is supposed to raise a student’s academic performance, assuming the teacher could be more attentive to each student. But this logical model has proved to be ineffective with data application. A medical check for metabolic syndrome, though it is assumed to be effective logically, has proved to have a limited effect on restoration of health with data analysis. Therefore, the logical model is merely the preparatory stage of EBPM. Data analysis must follow it.
In addition to policies backed by budgets, EBPM may also be applied to regulatory policies without a budget. Many economic policies are related to regulations. The minimum wage is a regulation on wages and reform of working styles is related to regulations on working hours. They must have a significant impact upon the economy, even without any government expenditure. So I believe we should think about its application to these policies as well in the future.
Written by Naoyuki Haraoka, editor-in-chief of Japan SPOTLIGHT, with the assistance of TapeRewrite Corporation.
May 2022 Japan SPOTLIGHT
