Policy-Making for a Better Society

The term "Evidence-Based Policy-Making" (EBPM) has gained attention in Japan, and EBPM is becoming increasingly prominent not only on the national level but also when policies are being drawn up on the regional and local levels. Regardless, however, I am personally under the impression that the concept of "policy-making" remains poorly understood. Based on my experience at the Research Institute of Economy, Trade and Industry (RIETI), I believe that it is important to take a future-oriented perspective that aims to build a better society as a basis for policy-making. Although this paper is an "experience-based" discussion that is rooted in my personal experiences, I believe that the collective knowledge that arises from each of these experiences forms a kind of evidence. I hope, therefore, that this essay can contribute to future policy-making.

Since there are many forms of policies, it is impossible to discuss the issue of "policy" in general terms. For example, as is indicated in economics texts, government intervention is required when a "market failure" occurs. There are also policies designed to correct economic disparities. In the context of EBPM, the assumption is that policy-making will be conducted based on the prior identification of issues and the resultant clarification of objectives. Additionally, however, there is future-oriented policy-making, which will be discussed in detail below. This is differentiated from conventional policy-making, and I call it "policy vision-formation" in this column.

For example, currently, one of the fields attracting the most attention globally is automation technologies for artificial intelligence (AI) and robotics and consequently this field is now the focus of much policy debate. As these cutting-edge technologies become more widespread in daily life, we can expect that an entirely new type of society will arise. In Japan, the Cabinet Office has proposed a policy vision they call "Society 5.0" (Cabinet Office, 2016), and the Ministry of Economy, Trade and Industry has proposed "A Structural Vision for New Industries" that is based on the concept of "Connected Industries" (Ministry of Economy, Trade and Industry, 2017). Other policy visions that are enacted on a global scale include policies related to climate change and sustainable development goals (SDGs).

Why is it necessary to indicate a "policy vision"? The goal of forming a policy vision is to create an environment in which individuals with disparate interests are integrated to ignite a social movement. Policy vision-formation can also be understood as providing an incentive to all involved individuals ("constituents"). A more systematic refinement of this concept is the approach known as "Collective Impact," which is advocated as a method of solving societal problems (Kania and Kramer, 2011).

The concept of "EBPM Promotion" is considered to be a form of policy vision-formation and can be applied to the concept of Collective Impact. As Narita (2018) has argued, there is a sense that each of the various constituents involved in a given issue does not share the same policy vision for EBPM promotion. As Kawaguchi (2019) argued regarding this, efforts must not end after simply calling for the promotion of EBPM. Instead, it is important to consider how to give each of the constituents an incentive to share this vision. For EBPM promotion to be successful, it is often said that implementing randomized controlled trials (RCTs) and training specialists in statistical analysis is imperative. However, even before this, it is important to create the circumstances that will allow the constituents to share the "policy vision" of EBPM.

When formulating a policy vision, there is a need to include the active participation of people who represent a variety of perspectives. Accordingly, it is important to ensure that there is a collaborative environment shared by people with diverse value systems that will naturally therefore produce empathy for the stakeholders and recipients of the policy. The result of this empathy will be a consensus around a particular policy vision. For example, let us examine the kind of policy vision created for artificial intelligence.

As the use of automated technology in the form of AI and robotics increases, it can be expected that society will become increasingly prosperous. This has led to an interest in such technology by many of the world's governments. While there is a need for national governments to provide an environment that supports the research and development of new technologies, there is simultaneously an element of anxiety as technological advances can negatively affect employment. The presence of this kind of anxiety may mean that initiatives aimed at creating a better society may not advance efficiently. Thus, it is important to ascertain the various risks that may occur while simultaneously preparing support measures that are designed to deal with those potential problems. As natural scientists continue in their research and development activities, social scientists will have to be relied upon to investigate the kinds of effects the resulting new technologies will have on society.

With these circumstances in mind, the former President and CRO of RIETI, Masahisa Fujita, has begun a study on the effects that AI has on society and the economy. I was in charge of the "Regional Economies in the New Era of Globalization and Informatization" Project (project leader: Faculty Fellow Nobuaki Hamaguchi), which was launched in August, 2015. As the survey continued, I began to notice the potential for a gender gap to arise in the labor market as the use of AI technologies increases. I also realized the danger presented by the simplistic use of methods such as machine learning. I worried that, if past data were simplistically learned as "the correct answer," then this may lead to the reproduction of a discriminatory gap. I took on the challenge of convincingly demonstrating how this possibility may manifest in the future through the use of statistical data. Eventually, the fruits of this research were published as Kondo (2017), and Hamaguchi and Kondo (2018, 2019).

Currently available machine learning also learns human biases. Cakiskan et al. (2017) indicated that such biases are consistently reproduced. Owan (2018) argued that the use of AI in human resources data may produce statistical discrimination for the same reason. The concern about presenting machines with data previously observed in the existing labor markets as "correct answers" is becoming gradually apparent. At the OECD annual Council of Ministers meeting held on May 22, 2019, 42 nations adopted the "OECD Principles on Artificial Intelligence." These basic guidelines were included in the "G20 AI Principles" adopted at the G20 Ministerial Meeting on Trade and Digital Economy held on June 8 and 9, 2019. These efforts indicate that there is a shared global policy vision that AI development should be "human-centered."

"Policy Vision-Formation" is not simply extending the past into the future. Instead, it is the creation of concepts that currently do not exist, but whose achievement is desirable, and to create the methods by which these concepts will be realized through individual, specific policies.

I have a sense that the fact that one of the many roles played by policy think tanks is the formation of future-oriented policy visions is not yet understood. For example, in Japan, the National Institute for Research Advancement (NIRA) plays a role as a future-oriented policy think tank (Note 1). Based on my previous experience at RIETI, the role a policy think tank has in the formation of policy visions has not been sufficiently communicated to academic researchers in Japan.

One important point when forming a policy vision is how one identifies issues of high potential that have the highest possible impact from among all the possible issues. The skill that is required for this task is "data-driven hypothesis making" (DDHM). This skill entails creating a promising hypothesis based on current data about potential issues that may, but have not yet occurred. The interpretation of data is an important part of this skill. If one does not have a clear perspective, then one is at the mercy of the data itself and is liable to make errors in judgment. Thus, rather than depending upon inductive reasoning based on the data, one must have the deductive skill to make assertions based on a theoretical framework.

However, formulating a policy vision is different from simply arguing for one's vision. Instead, a policy vision needs to indicate a goal or goals that need to be accomplished in society, and consequently, a policy vision needs to be convincing and acceptable to people with a variety of value systems. Therefore, there is a need to build a consensus by presenting evidence efficiently.

Here, the term "evidence" refers to a persuasive basis for the feasibility of the policy vision; thus, it does not refer to the statistical causal evidence used in EBPM. When speaking of policy vision, it is impossible to present evidence in this narrowly-defined form. For example, would it be possible for a researcher to show a causal relationship that indicates a study funded by a research grant project (e.g., Grant-in-Aid for Scientific Research in Japan) could be successfully completed as planned in a study protocol? Although it is impossible to indicate a causal, predictive relationship in a case of this type, it is possible to present data and reasons that show the high feasibility of the study. The significance of this kind of evidence lies in its ability to play a role in consensus building rather than underpinning a causal relationship.

When one looks back on EBPM initiatives conducted recently, one sees that differentiating causal relationships and methodology have made important contributions to policy assessments performed in policy-making. Easy-to-understand explanations aimed at the general public by researchers involved in projects on a global scale, such as Ito (2018) and Yamaguchi (2019), are particularly notable.

Conversely, it has also been shown that an excessive amount of attention has been focused on the term "evidence" in EBPM. Hayashi (2019) noted the importance of policy-making (PM) in EBPM. Of course, evidence is important to policy-making, but debates over which methodology should be used to distinguish causal relationships are often biased, which, I feel, often prevents any discussion of the essence of policy-making from advancing very far.

I am deeply concerned that the decision of whether or not to engage in policy-making will be based only on the presence or absence of evidence. It is more important to review the role of evidence in EBPM from the perspective of why evidence is necessary in policy-making. As I indicated in my above discussion of policy vision-formation, the fact that evidence plays a role in consensus building is also relevant to discussions on evidence in the context of EBPM.

The reason that consensus building is important in policy-making is that it is impossible to make all types of policies using evidence of the causal type. As argued by Sekizawa (2018), in many cases, there is no appropriate evidence that can be used in policy-making, or the information required for policy-making is limited. Even so, in cases in which policy-making is required, it is important to discuss how to proceed.

Additionally, if the creation of evidence itself becomes the goal and is therefore prioritized, there is a chance that issues such as the protection of private information and ethics will be seen as of lesser importance.

From the perspective of consensus building, two concepts can be used: "closed" and "open." A view that is frequently heard when the government proposes the promotion of EBPM is, "you mean to say that until now, you haven't been making policy based on evidence?" It is not true that the government previously failed to make policy based on evidence. Instead, policies were made on a certain degree of evidence that resulted from discussions held by investigative committees attached to various government bureaus and agencies, and not only on causal relationships. Presently, many researchers claim that EBPM is designed to improve the quality of evidence.

In the absence of the need for consensus building, so-called "closed EBPM" is possible through the use of evidence that indicates causality. Specifically, this is policy-making under circumstances in which highly advanced evidence presented by specialists is available. However, evidence presented by specialists does not necessarily promote consensus building as a natural consequence. Policy-making should not unilaterally suppress divergent views simply because "there is evidence" for a particular view. "Open EBPM," therefore, is the process of forming a consensus through discussion centered on a wide variety of evidence precisely because the constituents hold a wide variety of values and interests.

The ability of each of the constituents to engage in effective discussion and debate is a crucial aspect of "open EBPM." If the evidence presented is only understandable by specialists, then it is necessary to provide an easily understandable explanation of this evidence to the general public. I have a sense that progress is gradually being made on this point. Additionally, the constituents should not simply absolve themselves of all responsibility if they cannot understand some aspects of the discussion. Instead, they need to continually educate themselves on the issues that they face. This problem also arises among people who specialize in different fields. A specialist in one field may not understand many aspects of another field. If specialists reject ideas simply because they cannot understand them, then no progress will be made. There are major differences between "criticism" and "rejection." As Kondo (2019) indicated, the ideal situation is one in which a participatory EBPM system is created that allows all constituents to engage in discussion and debate over a core set of relevant evidence.

It is necessary to understand that in policy-making, simply verifying causal relationships is not a sufficient basis for implementing the policy in question. It is necessary to think of policy as a "package." As Kondo (2015) indicated, the achievement of other policy goals may hinder a policy intended to achieve a particular objective. This brings one to the concept of "policy mix," which is a combination of multiple policies that are enacted as a package to achieve the goals. This is based on the concept of the interaction between policies (i.e., policies are not independent). However, currently, assessments resulting in the selection of the policy package with the greatest efficacy are not being conducted.

Currently, carrying out comparative examinations of the total effect of policy packages seems prohibitive, but it is important to keep this in mind as an objective. If the various constituents do not share the same policy vision, then it is impossible to make appropriate adjustments to each policy. To understand what goals need to be achieved by each policy through the process of EBPM, it is necessary to share a common view of the issue at hand.

Why are policies needed in the first place? I believe that policies are meant to create a better society. The job of a policy vision, then, is to deliver a specific image of what this "better society" would look like. The first step in a policy vision is consensus-building. To achieve this policy vision, it is necessary to have specific, individual policies. This is where EBPM is important. We should not be forming policies based only on evidence that statistically shows a causal relationship. Consensus-building requires a substantial amount of effort and the ability to have a constructive discussion based on evidence.

The above is based on my personal experience from work at RIETI, but I believe that the collective knowledge derived from each of these experiences constitutes a body of evidence. I hope that this column can contribute to future policy-making endeavors.