"Whatever Happened to Japan's Energy Deregulation?"

Over the past 50 years, three broad analytical frameworks preoccupied both policy specialists and scholars in the field of comparative political economy. These frameworks were not only applied to the industrial democracies of the West, but also found constant use in the analysis of Japan.

One framework was an "interest-based approach" where principals and agents, or bosses and underlings in plain language, took center stage. In this analysis, individuals and groups vied to maximize material self-interest. Politicians served the public only for the sake of personal reelection. Bureaucrats and judges served politicians for the sake of career preservation and advancement. Interest-groups and their coalitions lobbied the government to promote narrow self-serving agendas, and the electorate - usually depicted as a homogeneous mass dominated by the "median voter" - selected their representatives based on the size of their pocketbooks. Essentially, all actors were out for themselves. This framework was (and is) commonly known as the "rational-choice school." Mark Ramseyer and Francis Rosenbluth's book Japan's Political Marketplace was a clear example of this interest-based approach.

A second framework was an "institutional-based approach" In this analysis, the principal causative factor behind policy processes and outputs was the organizational structure of the state. Agenda-setting and decision-making were said to be based on sets of accepted statuses and roles. "Varieties of capitalism" existed. The organizational structures of the United States and the United Kingdom (liberal market economies) were said to be different fundamentally from the organizational structures of Germany and Japan (coordinated market economies). States followed pre-determined historical paths that locked themselves into social, economic and political institutions sometimes dating back centuries. Key collectives - not individuals - were the preferred units of analysis. Chalmers Johnson's seminal work "MITI and the Japanese Miracle" was a notable example of such an institutional-based analysis.

A third broad-based category was the "ideas-based approach." In recent years, ideas-based analyses have regained analytical legitimacy after decades of being sidelined in the literature. These analyses - sometimes referred to as "cognitive capture" - dealt with intellectual paradigms, popular worldviews, and norms of what is "just" and "unjust." These ideas were said to provide technical and normative roadmaps that offered decision makers and office holders a general idea of how first to identify problems, interpret alternatives, and then approach workable solutions for the benefit of all. Rather than a set of individuals simply looking out for themselves, the material self-interest of actors was subordinated to the functional role of ideas. Works such as Judith Goldstein and Robert O. Keohane's Ideas and Foreign Policy: Beliefs, Institutions and Political Change exemplified this sort of analytical framework.

For the analyst of comparative political economy, the "lost decade" presents a challenge for those wishing to understand Japanese behavior using these three frameworks. In particular, we consider energy. The Japanese case, with its emphasis on national security (nuclear power) versus economic efficiency (structural reform), serves to highlight some of the most thought-provoking issues characterizing both the agenda-setting and decision-making process and its policy outputs.

Which of these three schools best identifies and explains the puzzling situation of electricity, gas and petroleum deregulation? Why did decades of regulatory and legislative inaction in the postwar period suddenly lead to calls for reform in the early 1990s? Moreover, why did concerted attempts to restructure these provisions to industry and to private residencies during the 1990s along market competitive lines run into the sand with the new millennium? The ultimate failure of such attempts appears surprising in view of the fact that Japanese consumers pay considerably more for electricity, gas, and petroleum than consumers do in many other comparable countries.

In order to explain what happened, a comparative historical research design using an extended case study enables multiple disciplines to be combined to maximize the analysis.

Supply-side analysts have argued that the way to deal with Japan's stagnant demand is by freeing up capital locked into inefficient and closely protected businesses since the oil shock of 1973. In theory, once heavily regulated markets were deregulated, the consumer, who benefits from lower prices and greater transparency generated from a more competitive efficiency-centric environment, would finally start spending more. This was a policy prescription that applied to virtually every heavily regulated sector, ranging from natural monopolies like electricity and telecommunications, to transportation sectors like taxis and airlines.

In 1999, I began work at Dresdner Kleinwort Benson in Tokyo and was called to London to be informed of what some informally described as the "white vision," named after Dr. Anthony White.

The logic envisioned once heavily regulated natural monopolies turning unintended inefficiencies into mutually beneficial deregulated structures. New entrants would seek to capture market share by quickly selling electric power (and gas) at marginal cost. The incumbent suppliers, desperate to hold on to their customer bases, would quickly lower their prices in self-defense. The falling revenue streams of incumbent suppliers would force companies to diversify their investments into non-core businesses. They also would attempt to cut their costs in the pursuit of efficiency. Increased efficiency measures would, in theory, lead to rising earnings, higher total returns for shareholders and consequently higher share prices. The consumer would benefit from lower prices, the corporation would benefit from higher efficiency, the shareholder would benefit from higher rates of return, and the government would benefit from achieving their long-term objectives of greater transparency in the markets. These developments would be followed by a reevaluation of the sector leading to a higher beta (a measure of how the shares trade relative to the entire market). Stocks of incumbent enterprises would become more volatile because earnings would now be based on performance rather than captured revenues.

This largely academic conceptualization was popular in the 1980s. Deregulation of the energy sector spread quickly from theory into practice across the world. Starting in Augusto Pinochet's Chile on the behest of University of Chicago economists in 1982, privatization and deregulation entered the UK market under Margaret Thatcher in 1990, Oceania, the United States, and Europe in the early 1990s, and finally to Japan by 1995.

Japan's deregulation was scheduled to unfold in four stages. Phase 1 allowed incumbent electric power companies to select their preferred wholesale suppliers of electricity at competitive prices. Phase 2, implemented in March 2000, extended to retail liberalization by ending the regional monopoly status of the nine electric power companies. Large-lot power customers (29% of demand) who fell into the 2,000 kW+ contract category and whose connections were to extra-high-voltage wires of no less than 20,000 volts (e.g., industrial plants, hospitals, and hotels) could choose their preferred supplier. Phase 3, implemented in 2005, saw the retail liberalization extend to roughly 60% of the total electricity market. It included large lot and commercial users. In theory, the remaining 30-40%, residential customers, would be liberalized in Phase 4 by 2007.

The speed at which deregulation unfolded in Japan stood in contrast to the previous five decades. From the 1950s to the 1990s, regulatory provisions to change the market were virtually nonexistent. Expansive change took place with regulations introduced in 1995. The reason for this phenomenon can be found in an analysis of electricity price-related issues covered in the Asahi Shimbun. After the oil shock of 1973, there has been a certain amount of coverage on Japan's high electricity prices, but not much. By the mid-1980s, articles discussing high electricity prices, electricity deregulation (generally abroad) and deregulation in general, began to accumulate. By 1994, a huge spike in coverage occurred, peaking in 1999, and then collapsing.

Why did the politics of interest in Japan's electricity prices and deregulation increase dramatically in 1994 and then plummet in 1999?

It is a puzzle that Phase 4 of electricity deregulation in 2007 never took place, and why the general public seems not to care. The aforementioned schools of thought are useful in answering these questions. In the "institutional-based" school of thought, institutional change is more likely to occur when state actors (in this case the regulators in the Ministry of International Trade and Industry, or MITI) initiate, implement and guide the reform effort in a top-down manner toward a preconceived national goal. In the "interest-based" analysis, regulated enterprises effectively petition the regulator in a bottom-up manner to satisfy their material self-interests. Conversely, multiple actors, international or domestic, oppose the status quo to satisfy their own material self-interests, in this case, to enter the market.

In both cases, however, actors did not behave as theory would predict. Throughout the 1980s and early 1990s, MITI regulators consistently resisted the growing calls for deregulatory change by economists?refusing to accept the inefficiencies of the sector. Only after the populist rise of the Hosokawa cabinet did MITI reluctantly agree to investigate reform, changing their emphasis from "energy security" to "energy efficiency." The same situation can be said of both the incumbent electric power companies and their large-lot customers. Since the early 1970s, and despite theory's prediction that self-interested actors would soon object to unfavorable operating environments, Japanese power companies and their customers experienced both increased inefficiencies and decreased profitability without expressing concerns for over twenty years. These counter-intuitive situations suggest that both theories' predicted behavior (and their outputs) do not explain well the deregulatory process.

A third (perhaps more plausible) explanation surrounding Japan's deregulatory process involves normative and cognitive ideas, which I call the "reform idea." This theory develops in a bottom-up, cascading effect, in which one action begets another action, completely independent of the material interests of any of the actors. Deregulation happens because conventional wisdom leads actors to follow trends in the rest of the world, thereby implementing policies with certain commonly expected outcomes. Politicians like Yasuhiro Nakasone, Morihiro Hosokawa, Ryutaro Hashimoto and Junichiro Koizumi are examples of actors following the zeitgeist. Perhaps they never had any true understanding of what they were proposing or what they wanted to do.

Further examination of the industry's background offers clues to the policy making process. Japan's electric power industry is a vertically integrated, 60-year old system. At the time of inception, there were nine privately-owned regional monopolies, governed under the 1964 Electric Power Industry Utility Law (Denki jigyō hō) and regulated by MITI. The industry is not labor-intensive, but capital-intensive. Electric power companies have not made cash contributions to any political party since 1977, nor are there are any clear electricity zoku, or "policy tribes," in the Diet to defend or lobby for the industry. Whereas independent regulatory agencies supervise the electricity industry in the U.S. and the UK, Japan and Germany, on the other hand are regulated by ministries. In contrast to other countries, Japan's regulators are generally rotating career bureaucrats, mostly lawyers, rather than technocrats like engineers and former industry employees. Their effectiveness in guiding the sector is questionable.

In 1973, the oil shock had a profound effect on Japan's electricity prices. This shock resulted in a sizeable electricity price increase for industrial consumers and remained high thereafter. Why did electricity prices for industrial customers increase so sharply, why did they stay at such high levels, and why were such high prices tolerated?

Conventional wisdom was that steep oil prices caused by foreigners led to high fuel costs per kWh, and therefore high electricity prices. While such an analysis was technically correct in the 1970s, high oil prices (and therefore fuel costs per kWh) collapsed between 1980 and 1994 without a proportional decrease in the average electricity price. Other explanations were equally problematic. The Japan Communist Party (JCP) incorrectly blamed personnel costs for these persistent high electricity prices. Others incorrectly blamed high tax burdens on the Japanese power companies that were passed on to the consumer. None of these factors explain Japan's relatively high electricity prices. Nuclear power capital costs in Japan are higher than the rest of the world because the lead times that one must use to site, plan and build a nuclear power plant are between 15 and 30 years due to local politics.

It is, all said, an oddity that electricity prices in Japan have stayed so high. Deregulation was supposed to deal with this anomaly by allowing new entrants into the market. Also, Japanese utility companies stopped making political contributions in 1977, but large-lot electricity consumers are huge contributors to the Liberal Democratic Party (LDP). Despite huge donations to the LDP, industry hardly complained about high electricity prices. Part of the explanation is that the concept of natural monopoly was the dominant paradigm, allowing no room for deregulation as a popular theoretical concept. Most utility companies admitted they did nothing about the high prices because they did not think they could do anything about it.

Starting in 1995, all actors, including the JCP in the standing committee-level of the Diet, voted in favor of electricity deregulation because it had been effectively carried out abroad. Following those favorable events, the California energy crisis in 1999 and the Enron bankruptcy in 2001 caused legislators to begin questioning both the merits and the predicted outcomes of deregulation. Starting with the JCP and the Japan Socialist Party (JSP), more and more legislators were asking questions about what they were reading in the newspapers. The imagery in Japan, reflected through the media, suddenly turned against energy deregulation.

The result is that, today, real competition does not exist in the Japanese electricity market. Despite widespread market liberalization, new entrants have captured less than 2% of the contestable market. Prices are still high yet profits continue to fall. The goals of the Japanese electricity industry were clear in 1995. Greater transparency was called for; it has not yet materialized. Lower electricity prices, or else a breakup of the vertically-integrated industry were demanded; neither happened. By Japanese standards, deregulation of the electricity industry has been a failure, not reaching any of its 1995 goals. The White vision included greater value in energy-related stock prices, and thus also failed as the stocks of these firms continued to trade as semi-bonds. By international standards, for example, compared to California, it was a moderate success as electricity prices did not skyrocket, but at the same time, they did not fall significantly either. Overall, deregulation has been awash.

The refining and distribution of gas is also a downstream, highly fragmented industry. The industry is also regulated by MITI (now METI) and comprises a very small share of the labor force. Gas companies make minor cash contributions to political parties, but there are no gas "tribes" in the Diet. Since 1995, the numbers of workers and companies in the gas industry have decreased slightly and the number of contracts continues to rise. This decline is because some public firms were either dissolved by the government or privatized. Yet ironically, despite expectations, gas prices are now higher than they were in 1995. Tokyo Gas has seen a steady increase in sales with stable profit margins and a higher return on assets. Gas deregulation has seen modest success in this industry.

Nothing of material importance has happened in the petroleum industry. Theoretically, gasoline prices were supposed to come down due to increased competition from imported gasoline refined abroad. About 10% of Japan's gasoline was being imported when the market was finally opened. Since then, imports have dropped to roughly 2% of the total gasoline sold in the Japanese market. Prices are now rising again due to high oil prices and the main objective has been thwarted. Petroleum deregulation has failed largely in Japan.

Also, MITI was eager to reduce the total number of gasoline stations, along with the number of workers at gasoline stations. In 1990, there were about six workers per gasoline stand, but there are now 10 workers per gasoline station. Lack of efficiency has negatively affected the profitability of the sector. Many of these companies are "dog stocks" since management has not taken opportunities to improve profitability and sales.

Q: Regarding the political power of electric power companies, while cash contributions are decreasing, party ticket revenue is increasing. Party ticket revenues are not disclosed, but it seems as though the political power of electricity companies is increasing with more Diet members trying to stop the liberalization of the industry.

Paul J. SCALISE
I agree that most politicians now want to stop electricity deregulation and that most power companies are no longer in favor of electricity deregulation. However, I am not certain that this political shift can be attributed to money politics. The Cabinet Office's official gazette (Kanpō) reports that while some electric power companies give political donations over time, it is not clear to where these contributions are going. TEPCO management naturally insists that they have not "officially" given cash contributions to any Diet member since 1977.

Journalists continually mention that local politicians collect money from individuals affiliated with electricity companies. The problem is that there is no clear co-variation of cause and effect. There is no clear connection between someone who works for TEPCO, for example, donating money to a political party and the notion that TEPCO management is siphoning money through these individuals for some covert political gain. Asahi Shimbun and Yomiuri Shimbun have searched diligently to find these connections, but with little success so far. While it may be a possibility, a smoking gun has not been found.

I am more inclined to believe that ideas, the current notion that electricity deregulation is a conceptual mistake based on empirical outcomes worldwide, exhibit stronger explanatory power for why Japan has stopped the process. Politicians essentially agree, speaking in Diet committees about what foreign and domestic newspapers are printing. They would then question bureaucrats on these issues, and the bureaucrats would have no clear rebuttal.

Q: While I agree that cognitive capture explains politicians' motives in ending electricity deregulation, there is a reason for the falling profits of electricity companies. There are two reasons for this. One is the extremely good service provided by electricity companies. Consumers are aware of this and electricity companies use this fact in their propaganda. Second is nuclear energy, with the increase in the share of energy provided by nuclear plants working to increase the cost of electricity.

Regarding petroleum, there has been an incredible boom in self-service gas stations recently. Near Tokyo, self-service gas stations are running traditional full-service stations out of business with price differences of only about 4 yen. This signals a change in Japanese psychology since, at one time, the Japanese seemed to enjoy paying high prices for a little more service.

Paul J. SCALISE
Self-service gasoline stations were introduced to Japan in 1997. They were expected to skyrocket as Japanese consumers enjoyed a choice of supply. However, the market share of self-service stations is not supporting those expectations: the number of self-service gasoline stations in Japan relative to all types of gasoline stations is currently about 10% of the total market share. It does not seem as though self-service stations are striking a chord with Japanese consumers after a decade. It is worth keeping an open mind to see what happens in the future.

Regarding security of supply and service, I agree that steady, reliable service seems to be a large factor in justification for high electricity prices. It is also true that power companies use the fact that blackouts and brownouts in Japan are virtually nonexistent relative to what is found abroad to their advantage. By the turn of the century, CEOs from power companies, speaking at ministerial advisory council meetings, would mention the reliability of Japanese electricity services quite frequently. Stability of supply is seen as the price one pays for security. I would not label electricity companies' claims of stable service as "propaganda" because that would imply they do not believe what they preach.

We cannot judge what they are thinking, but through transcripts and personal interviews, I have noticed that security of supply as an argument was not contested until the California crisis when, in 2000, the California electricity market saw a 500% increase in prices as a result of supply not equaling demand. Increased costs were passed on to wholesale users since retail prices were capped, and many companies faced bankruptcy. This information was dissected and analyzed during both advisory council and Diet hearings. Experts brought in from abroad initially agreed that deregulation was beneficial to both the consumer and the supplier, but by 2001, about half of the economists opposed deregulation. Energy economists from Australia, New Zealand, the U.S., Canada and Europe were brought in to speak before the Diet on their views about what was happening. There was no longer a consensus, giving way to confusion and muddle.

Confusion and muddle in these academic blueprints brought the decision-making process to a screeching halt. No one knew how to operate in the electricity industry. I was present at a meeting where an Australian economist told the Diet not to deregulate. Soon after, a bandwagon effect set in and deregulation fell apart. I do not think it was a result of propaganda as people were honestly concerned.

Q: Many gas stations have both full-service and self-service facilities, making a comparison of the number of full-service and self-service gas stations difficult.

Also, many service stations have turned themselves partly into convenience stores and cafes. When counting the employees at each service station, the different services that they are now providing require an increased workforce.

Regarding political party tickets, political money has always been a reality, and while money to the LDP may or may not be diminishing, the reality is that in 1995, 1999 and 2003, deregulation did progress forward. Regardless of the political money flowing since deregulation occurred, something must have pushed it and the notion that ideas propelled the process is a very convincing one.

Is there such a huge conceptual gap between the interest-based explanation and the ideas-based explanation? The ideas that drive us cannot be divorced from our perceptions of what our interests are. For example, one of the major reasons prices have not gone down are accrued costs from nuclear power plants. This is the consequence of the Japanese notion of what Japan's energy security interests are. This has been a constant idea driving the cost structure of electricity production.

Paul J. SCALISE
This is a question of whether interests drive ideas, or ideas drive interests. Some would argue that ideas are simply ex post facto, ad hoc rationalizations to justify our material self-interests. In the 1980s, many talked about the Japanese "business mind," which was actually in many instances a means to justify self-serving protectionism against foreign import competition.

Interest-based analyses are primarily focused on material self-interests, whereas idea-based analyses can, and most likely do, focus on the big picture. Most rational-choice analyses do not consider the actions of politicians and bureaucrats to be philanthropic. In setting up the new regulatory regime, however, it seems clear that in a number of transcripts and newspapers, electricity deregulation was repeatedly touted in the 1990s as an all-encompassing answer to the general public's improved welfare - as an answer towards providing reduced consumer prices and greater corporate efficiency. It is difficult to discern whether those who adopted this interest in promoting the public good were always sincere. One method - used in my research to investigate alternative hypotheses - was to identify the material self-interests of actors. If those interests were not served in the promotion of a certain regulatory regime, ideas could be a more plausible explanation of political and economic behavior. Indeed, that is what I found in the case of electric power deregulation.

Q: Why did diversification of electricity sources stop? At one time, cogeneration was seen as a promising way to diversify electricity, but after the hike in oil prices, this turned out to be less economical. This is one of the reasons why the introduction of cogeneration has stopped, along with global warming concerns.

Paul J. SCALISE
The generation cost for nuclear power is around 7 yen per kWh. Coal is about 6 yen per kWh, liquid natural gas-based sources are about 5 yen per kWh and oil-based sources can be anywhere from 7 to 20 yen per kWh depending on the price of imported oil.

In economic terms, I agree that it is strange that utility companies always wanted the more expensive nuclear power generation when they had cheaper options available. Utility companies say that it is not their fault and that the government wants more nuclear power.

The cognitive dissonance brings us back to the issue of energy security. The public seems to find the larger incumbent power companies more secure. New entrant generation is approximately 30% to 40% cheaper than one of the incumbent suppliers, though incumbents continue to build power plants without hesitation while new entrants face demand constraints. An executive at Diamond Electric Power, a new entrant in the market, explained that this situation stems from customers being afraid that new companies will stop supplying power to them. This imagery problem limits the power plants such new entrants can build to affiliated companies' locations. These risk calculations are quite complicated and help to explain why more people do not switch over.

In India, the amount of power lost on the grid is about 65%. India has sought foreign investment to help with electricity demand and Enron was one of the companies involved. Enron eventually grew frustrated with the Indian government and left without completing construction of the plants. The Japanese electric power industry saw these developments and broadcast them over Japan, holding the Indian situation as an example of common foreign practices.