Ex Post Evaluation of 2002 FIFA World Cup Korea / Japan
Senior Fellow, RIETI
Time flies like an arrow. A year has passed since the World Cup was held in Japan and South Korea. The event has three dimensions: "game" (football), "business", and "festival".
Justifiably, any business entity is subject to evaluation with its performance and results being tested all the time.
In discussing the business dimension of the World Cup, we should distinguish between two different types of business entities and stakeholders, namely, the public and private sectors.
The leading players in the private sector include TV broadcasters, an advertising agency (Dentsu Inc.), official sponsors and so on. All these entities must have internally evaluated the outcome of their respective World Cup business and have been subjected to external evaluations. Furthermore, the results of such internal and external evaluations would be reflected by their earnings reports and communicated to their shareholders, and thereby, eventually receiving the ultimate external evaluation in the form of the market's verdict.
Their public-sector counterparts, however, are not subject to evaluation by the market. Who should then be evaluating these entities? Have the "business" results of the local governments, which played a core role in the World Cup last year, been subjected to appropriate evaluation? In other words, has an appropriate "administration evaluation" been carried out? Such an evaluation would require a local government to first examine and evaluate its achievement by itself, and then, disclose the findings to the public for evaluation. This is how an orthodox democratic local administration is supposed to work. My survey over the 10 local governments that hosted the World Cup, however, found that all but one - Oita Prefecture - of these governments have not carried out a fully-fledged ex post evaluation and there was not even any sign indicating their having tried to conduct an evaluation. As I announced I would in my previous column (Japanese only), I have conducted a survey on, "Ex Post Evaluation of the World Cup by Local Host Governments," and will outline the results below. Click here for the report.
Question no. 1 of the survey was to find out whether the local governments recognize the necessity of ex post evaluation. As to the question of whether or not they have evaluated the results of the World Cup, only three - Oita Prefecture as above mentioned, plus Yokohama City and Shizuoka Prefecture - said they have conducted some kind of evaluation on their own.
Evaluation is not the purpose of the survey
To avoid misunderstanding, I would like to note that the purpose of my survey is not to substantiate or evaluate each local government's achievements made through hosting the World Cup. The survey results are not intended for use in judging success or failure of each local host government. Neither is it the intention of the survey to check or verify if, how, and to what extent each local government fulfilled its intended results. All these issues should be left to the members of each concerned local communities, to which I do not belong.
Although it may be unsolicited advice, these local communities should first map out a "regional" vision on their own. It is not necessary to try to equally fulfill all the assumed goals, or the items asked in the survey. Rather, they should be able to define their respective "regional" vision through deciding on the prioritization of these items. What I intended in my survey is not to verify achievements, but to clarify the "presence or absence and actual state of an ex post evaluation mechanism". In other words, I tried to find if and how these local governments recognize their initial intentions prior to hosting an event, incorporate such initial intentions into policy measures, and assess the outcome. Providing certain measurements as a foundation for future debates, the survey is meant to propose the creation of an evaluation mechanism that consists of a set of frameworks for clarifying "purpose", setting "goals" (target figures), and verifying "results".
Accountability in Sports Policy
Accountability is responsibility to explain one's own decisions and actions. For this responsibility to be fulfilled, the "purpose" and "goals" of concerned decisions or actions must be explained beforehand. In presenting goals, measurements for evaluating achievements against intended goals need to be clarified. In not conducting ex post verification and evaluation which is based on such preset measurements, it is theoretically impossible to prove the achievement of, or the degree of achievement with respect to intended goals. Unless such self-evaluation concerning the degree of achievement is conducted and explained, it cannot be said that accountability is fulfilled. That is to say that accountability can be fulfilled only when both "ex ante explanation" and "ex post verification and evaluation" form the complete package.
Turning to the world of sports in this regard, the notion of "regional development through international sports events" is taken so much for granted that it has become prevalent as a simple catchphrase, while the reality of such an attempt remains undiscussed or dressed in ambiguity. However, it is possible for a local government to present a general idea as to "what achievements in terms of regional development" can be expected as a result of hosting a certain sports event.
How to prioritize policy measures, which are presented by a local government in general terms, is not subject to any uniform criteria. Rather, it is a matter of strategy and must be selected by the people of each local community (strategy, as referred to here, is to discuss and determine "what not to do"). As a minimum requirement, however, it must be spelled out beforehand for what purpose and with what expectation a particular event is to be held, and consensus about the purpose must be sought through debates. At the same time, goals should be set and evaluation measurements presented. Then, once the event is over, the degrees to which the intended goals were achieved must be examined and evaluated. Without these procedures, accountability cannot be fulfilled.
Sports and Regional Development are not a priori
An evaluation of a business is usually done in the three stages of "program", "operation" and "results". Results are to be evaluated in relation to goals and purposes. To produce results, however, a program must be worked out first, and then put into operation. If regional development is the purpose of hosting the World Cup, the event's receiving much fanfare is a necessary condition of operation. In the past, however, there were many cases in which such "fanfare" was perceived as a "result". Apparently, "operation" and "results" are mixed up here. If the success of the event were the purpose of hosting the event, it would not be wrong to take the "fanfare" as a "result". What is more, this might have been the case were a sport association organizing the event. But for a local host government, the success of the event itself cannot and should not be the purpose of hosting an event.
Work often has a tendency to become autotelic, but a bit of rational thinking would show that we should avoid such ineptitude. Once trapped in this autotelic pitfall, we could easily lose sight of the original objective amid the passing gaiety, excitement and emotion (such as people might have felt when they watched Masahiko Harada breaking into tears after making a record jump and assuring the gold medal for team ski jumping in the 1998 Nagano Olympics), and fail to carry out sufficient and objective ex post evaluation. Moreover, in such a situation, there is no hoping for a long-term cost performance to be examined.
Whether or not the hosting of an event has led to sustainable regional development should be the basis of measurement in assessing the achievements of the local government that hosted the event. The fact that an event attracted a large number of volunteers, for instance, would provide an operation-stage measurement.
Thinking this way, we could conclude that a vision about "regional development", which serves as a foundation to assess achievements, is the most important. The biggest question is whether each concerned local community is able to draw up a clear vision for its own region. The hosting of an international sports event is merely a means or method to achieve the ultimate goal of regional development.
May 27, 2003
Article(s) by this author
The Economics of Professional Sports Leagues: The Bottom Line in the Kintetsu Buffaloes Merger Issue is a Lack of Management
July 27, 2004［Column］
March 16, 2004［Column］
February 10, 2004［Column］
June 12, 2003［RIETI Report］
May 27, 2003［Column］