(January 12, 2005)
Given the degree of high-technology penetration in the Japanese economy and the international competitiveness of the hardware sector of its consumer electronics industries, the weakness of the Japanese packaged software industry looks puzzling. The software industry in Japan has always been fragmented among incompatible platforms provided by large systems integrators (Hitachi, Fujitsu, NEC) and dominated by customized software; therefore it has never been competitive in the global market. There is of course the videogame exception, which I do not discuss here. Not because that industry is not important - Japanese videogame publishing houses and console manufacturers should be role models for other software companies - but simply because the videogame market has a dynamic of its own, largely independent of the evolution of the rest of the software industry.
It should be stressed that a tremendous amount of software is produced and used in Japan. The problem is that most of it is produced by subsidiaries of large firms and customized for the needs of very few - oftentimes one - large customers. Despite efforts by the Ministry of the Economy, Trade and Industry (METI, formerly MITI), there are relatively few small to medium-size software companies in Japan compared to the U.S. This is because the highly fragmented, vertically integrated and specialized industrial structure precludes modularity and the emergence of popular platforms, which in turn impedes the economies of scale needed to offer sufficient innovation incentives to independent developers. It is mostly for this reason that the PC and packaged software markets are dominated by foreign, particularly U.S., software providers.
In this column I will first overview the main structural reasons for this peculiar situation: the creation and historical evolution of the Japanese computer industry (company strategies and government industrial policy), and two factors unique to the Japanese market (a strong preference for customization and a bias in favor of hardware over software). Second, I will explain why the weakness of the packaged software industry is a source of concern for the industrial sector and Japanese policy-makers given the current process of digital convergence taking place in the computer, communications and consumer electronics industries.
Historical evolution of the Japanese software industry
In the early 1960s MITI orchestrated licensing agreements that paired each major Japanese computer system developer with a U.S. counterpart. Hitachi went with RCA then IBM, NEC with Honeywell, Oki with Sperry Rand, Toshiba with GE, Mitsubishi with TRW and Fujitsu went on its own before joining IBM. The idea was to make sure Japan embarked on the computer revolution and that it competed effectively with then-almighty IBM. However, these initial pairings were one of the causes of the fragmented state of the Japanese computer software industry, leading to a Tower of Babel of incompatible systems, in which, as in the biblical story, all participants are condemned to speak different languages forever as punishment for previous sins. Since each of the major computer system suppliers had a different U.S. partner, each had a different antecedent for its operating system. In fact, even IBM-compatible producers only had the instruction set licensed from IBM in common; their operating systems were incompatible among themselves. Very rapidly, each company found it profitable to lock-in its customers by supplying highly customized software, often free of charge, which meant that clients had only one source of upgrades, support and application development. Over time, many of the former U.S. partners were forced to exit the industry due to intense global competition from IBM. However, their Japanese licensees remained and perpetuated these incompatible systems.
Next, in the United States, following a highly publicized antitrust suit, IBM was forced to unbundle its software and hardware in 1969. The IBM System/360 was the first true market platform in the computer industry in that it was the first to support third-party suppliers of software applications and hardware add-ons. It marked the beginning of the vertical disintegration and modularization of the computer industry: computer systems were no longer solely provided as fully vertically integrated products; rather users could mix and match a variety of complementary hardware and software products from independent suppliers. This led to the development of an immensely successful software industry. The new industry became prominent with the workstation and PC revolutions in the early 1980s, which brought computing power into the mainstream through smaller, cheaper, microprocessor-based machines. An important consequence was the great potential created for software/hardware platforms, which a handful of companies understood and used to dominate their respective segments: Sun Microsystems in the workstation market, Apple and Microsoft in the PC market.
By contrast, in Japan there was no catalyst for such a sweeping modularization and standardization process. No one required the large systems makers to unbundle and there were no incentives to achieve compatibility. During the next three decades, the customized strategies became entrenched. Each supplier found it profitable to lock its clients into proprietary hardware/software systems that it alone could provide, and clients set up their own software divisions to further customize the proprietary systems they received, thus increasing sunk costs and reducing the likelihood of switching to newer systems.
Japanese computer manufacturers tried to extend this strategy to the workstation and PC market, but failed due to competitive pressure from foreign (especially American) suppliers. The best known example is NEC, which until around 1992 held a virtual monopoly on the Japanese PC market with its "PC-98." Its hardware platform architecture was closed (like Apple's) and its operating system, though based on DOS, remained incompatible with the popular MS-DOS PC operating system. In the end, however, NEC's monopoly was broken by Dell, Compaq and low-cost Taiwanese PC makers (1991-92).
The prevalence of closed, proprietary strategies prevented the economies of scale necessary for the emergence of a successful independent Japanese software industry. No single computing platform became popular enough with users to provide sufficient innovation incentives for packaged application software.1
There also seems to have been a preference for customized computing systems and software on the demand-side of the market. In Japan, like everywhere else in the world, the first users of computer systems (mainframes in the beginning) were corporations. However Japanese corporations have traditionally been keen on preserving the secrecy of their processes and adhering to internal business procedures, leading to a "How can we modify the software to fit our operations?" mindset, rather than the "How can we adapt our operations in order to take advantage of this software?" reasoning that prevailed in the U.S. For this reason, Japanese companies preferred to develop long-term relationships with their hardware suppliers and to depend on those suppliers, or on vertically related2 software developers for highly customized software solutions.
Another issue was the longstanding bias in favor of hardware over software. Japanese computer companies' business strategy had always involved giving away software for free along with their hardware systems as a tool to lock in customers. Ironically, this bias was probably inherited from IBM, whose success they were seeking to emulate. IBM itself remained convinced that hardware was the most valuable part of computer systems and this led to its fateful (and, some say, strategically misguided) 1981 decision to outsource its PC operating system to Microsoft, whose subsequent rise to power signaled the beginning of the software platform era.
This development was lost on Japanese computer makers, however, for several years. And MITI, which still viewed IBM as Japan's main competitor, was at that time immersed in a highly ambitious "Fifth Generation Project," a consortium that aimed to build a new type of computer with large-scale parallel-processing capabilities, thus departing from the traditional von Neumann model. The drawback, however, was that the project focused everyone's attention on building highly specialized machines (basically mainframes), whereas the computer industry was experiencing a significant paradigm shift toward smaller, general purpose machines based on open and non-proprietary architectures (Unix workstations) or on proprietary but very popular operating system platforms (PCs), which greatly expanded the computer market. MITI and member companies of ICOT3 realized only later the potential of making a common, jointly-developed software platform available to the general public rather than concentrating on systems designed for a handful of specialized machines. This led to MITI's next initiative, The Real-time Operating-system Nucleus (TRON). The main idea of TRON was to build a pervasive and open (i.e. non-proprietary) software/hardware platform in response to the market dominance of Intel and Microsoft. TRON was supposed to be a cross-device platform: computers and all sorts of other devices everywhere would be linked by the same software, thus finally providing a popular platform for Japanese software developers. Although TRON was an excellent platform concept; it unfortunately received little support from the major industrial players, in particular NEC, which viewed it as a direct threat to its PC monopoly. More importantly, it could not break into the crucial education market4 precisely because it was incompatible with both the NEC PC-98 DOS and the IBM PC DOS standards, both of which had sizable advantages in installed bases of users and applications. Thus, TRON was too little too late: the big winners of the PC and workstation revolutions had already been defined and none of them were Japanese computer companies. Most importantly, the intended creation of an independent Japanese software industry did not materialize.
Comparative studies of the U.S. and Japanese software industries also mention several other factors that further explain the phenomenon described above. One is the relative underdevelopment of the venture capital market for technology-oriented start-up companies in Japan compared to the United States, where venture capital had widely supported the emergence of successful small and medium-size software companies. This gap, however, has been narrowed due to recent METI policies designed to improve the availability of venture capital to technology firms. Another factor is the Japanese permanent employment system, which results in low labor mobility and is quite compatible with the "closed garden" approach to technological innovation. By contrast, high labor mobility has been a crucial driving force behind the "Silicon Valley model" of technological innovation, which is based on spillovers, transfers, cumulative inventions and a high degree of modularity. The latter model seems to have been more appropriate for creating a vibrant software industry.
Future challenges and one potential solution
Although the markets for workstations and personal computers are entirely dominated by foreign software platforms and large packaged software vendors, Japanese industries that make heavy use of software, especially the automobile and consumer electronics industries, have done very well until now, which begs the obvious question: does the weakness of the packaged software industry matter or is it merely a benign idiosyncrasy?
In order to answer, one needs to distinguish between two different types of software users: non-digital industries, automobiles and steel for example, in which software is only one of many inputs into an integrated final product; and computers, communications and consumer electronics industries, where software is becoming the vital platform for building, using and transmitting data, applications, content, and so forth. The fragmented and customized nature of Japanese software fits naturally with the first type of industries, which involve close coordination and vertical relationships between several firms in complex manufacturing and assembly processes. This fit will probably persist, for example, in the automobile industry. It is unlikely that Toyota will suffer too much from the fact that the computer systems it uses in its production are entirely proprietary and not used by any other carmaker. In fact, it views its software systems as part of its unique and valuable know-how, therefore it has good reasons to prefer the customized software business model described above. And indeed, most of the software it uses is developed in-house or through its subsidiaries.
Corporate customers in the second category, in particular consumer electronics companies, have also traditionally viewed software as a secondary input in the production of highly integrated hardware products, with which they have been very successful. However, the future of these industries is contained in the process of digital convergence, which requires seamless interoperability among a variety of devices. Users of computers and electronics, both end-users and providers of applications, content and the like, will demand increasingly open and modular systems and technologies. In this context, the model relying on tight vertical relationships has come under heavy pressure and has begun to reveal its limits. Most importantly, it has become clear that competitive advantage in the digitally converged world lies in chips and software platforms and technologies, which control the all-important flows of information. Firms with experience in packaged software or modular products more generally have an advantage over firms that produce only integrated products, including customized or embedded software. This is best illustrated by Sony's current struggles and its failure to set the standard in the digital media player market. Even though Sony has historically been a pioneer in portable music devices and presently owns a media empire, it is Apple, with almost no previous experience or ownership stakes in either, which came up with the first portable digital media platform of the 21st century: the iPod - mostly thanks to a valuable digital media software technology called QuickTime. In fact, all Japanese consumer electronics companies are currently facing the specter of falling margins and commoditization of their hardware products and seem to be anxiously looking for sources of value and competitive advantage.
One such source could be the open-source software movement, with software platforms such as Linux making inroads in the markets for PCs and consumer electronics (smart-phones). Today, both METI and the consumer electronics industry seem to have understood that it makes little sense to pursue the creation of a uniquely Japanese software industry. Rather, the focus is to find a way to take advantage of the global software market and leverage Japan's strength in hardware. And open source seems to be the ideal tool for achieving just that. This is why the China-Korea-Japan alliance in promoting open-source software initiated in 2002 makes a lot of sense for Japan, at least in theory. It is not yet clear how the alliance will function, but there are already some encouraging signs. One example is the creation by Sony and Matsushita of the CE Linux Forum, a joint venture aiming to build a common open software platform for consumer electronics devices.5 The emergence of popular open software platforms might help METI attain the long elusive goal of creating a competitive Japanese independent software industry, while at the same time slowing the tendency of software platforms to commoditize the underlying hardware.
In a future column I will analyze in more detail the potential that this open source software alliance offers to the Japanese software, computer and consumer electronics industries, and compare it with related government-led initiatives around the world. I conclude here by pointing out that software is an industry in which the American experience has shown that government-academia-industry cooperation can be very effective. Many successful American software (and hardware) technologies originated in academic projects: to give just three examples, Unix was created at Berkeley; Cisco's routers and Google's search engine were born at Stanford. All were subsequently successfully transferred to industry, either through the creation of start-up companies using venture capital or through licensing agreements. Thus, it seems there are significant potential synergies between the goal of encouraging the development of the software industry and METI's current efforts to improve academia-industry linkages in all areas of science and technology. It will be interesting to see how these linkages evolve6.