The quality of English language teaching in Japan is disappointing when compared with other East Asian countries or with the quality of other school subjects. This column assesses the impact of an English learning programme via Skype for Japanese high school students. Although a positive impact on English communication skills could not be established, mostly likely due to the low utilisation of the programme, it did have a positive impact on student attitudes. Policymakers may wish to consider how to combine regular English lectures and online English learning programmes to improve results.
It has increasingly been recognised among researchers and policymakers that the quality of education, rather than mere length, is important for individual earning power and economic wellbeing. In Japan particularly, the quality of English education has been an issue because it has not proved effective when compared with other East Asian countries (Butler 2015), and when compared with the quality of other school subjects. For example, according to the latest OECD Program for International Student Assessment (PISA) for Grade 9 students, Japan ranked second in science, fifth in mathematics, and eighth in reading in 2015 (Note 1). In contrast, Japanese students' performance in English has been far from satisfactory. Admitting that there is no widely accepted, internationally comparable performance measure for English, Japan ranked 35th among 72 participating countries in a standardised English test conducted by EF Education First, a global language training company (Note 2), and the average Test of English as a Foreign Language (TOEFL) score among Japanese test-takers ranked 138th among 169 countries where TOEFL test centres operate (Note 3). In addition, a nationwide English test conducted in 2014 by the Japanese Ministry of Education, Science, Sports, and Culture (MEXT) revealed that a majority of Grade 12 students were ranked at the lowest level (A1) in the Common European Framework of Reference for Languages (CEFR), with their speaking performance lowest among the four skills measured. Based on these results, MEXT recognised that the quality of English education, particularly in nurturing speaking abilities, should be improved (MEXT, 2015).
On the other hand, Japan has been long portrayed as a ‘non-immigrant’ nation (Bartram 2000) where foreign residents constitute only 1.7% of the total population. Therefore, in order to lure and cultivate foreign talent, the government launched the Japan Exchange and Teaching (JET) Programme in 1987 where it dispatched English-speaking aides (Assistant English Teachers, or AETs) for Japanese English teachers in primary and middle schools (Grades 1 to 12). This programme has expanded since then-4,404 AETs were employed as of 2015, and the Ministry of Foreign Affairs (MOFA) aims to further increase the number of AETs to 6,400 by 2019 (Note 4). However, individual annual costs for an AET are approximately $53,000, including salary, coordination, transportation, and so on (Note 5), causing a heavy financial burden to be shifted onto the shoulders of the local governments. Therefore, Japan is still struggling to allocate a sufficient number of native English teachers to public schools.
In order to test an alternative method to improve students' communicative English skills, we conducted a social experiment to evaluate the impacts of a newly developed Skype English learning programme for Japanese high schoolers (Higuchi et al. 2017). In the field of economics, an education production function exists in which school inputs such as teachers, textbooks, and classroom facilities are supposed to ‘produce’ outputs including academic achievement, and the outputs should be analysed for their efficiency and cost-effectiveness of the inputs. The use of information and communication technology (ICT) as an alternative to the conventionally used school inputs has been recently examined in various areas in education (for an excellent survey, see Bulman and Fairlie 2016).
Based on the above-mentioned background, we collaborated with a Japanese public high school and introduced the Skype programme to 322 students in Grade 10. One of the most notable characteristics of this programme is its individualised and self-paced format—students communicate for 25 minutes with English-speaking Filipino interlocutors at mutually convenient times and using learning materials of their own choice. Another notable characteristic is its relatively low cost. According to the programme provider, the budget for employing one AET can provide more than 20 times as many active communication hours for each student.
In order to rigorously evaluate the impacts of the Skype programme, we adopted a crossover randomised controlled trial design. That is, we first randomly selected 161 students (half of our sample) to be given opportunities to use the programme for the five months from July to November 2015, while the remaining 161 students were allowed to use the programme during the following five months (from January to May 2016). While all of the sample students had equal opportunity to use the programme by May 2016, only half of them had availed themselves of this in December 2015, when we conducted an endline survey to collect data for the present study.
By comparing the data on students who were exposed to the programme by December 2015 and the data on those who had not yet done so, we found that the programme positively changed the attitude of the students, especially in terms of their interest in an international vocation and foreign affairs. The students' attitude was measured using one of the most established measurement scales in applied linguistics (Yashima et al. 2004). In contrast, despite the positive impacts on the students' attitude, the impacts on their English communication skills, measured using two different standardised tests, were not statistically significant.
These limited impacts on English communicative abilities were mostly likely due to the low utilisation of the Skype programme, which was introduced as an extracurricular activity. Among the 161 students who were exposed to the programme in the period between July and November 2015, only 10 students took 50 or more lessons over the five months, as recommended by the programme provider. Moreover, 31 students did not even take a single lesson. Figure 1 shows the daily changes in the number of students taking a lesson, illustrating that only a small proportion of students continued to use the programme over five months. By analysing the programme usage records and characteristics of the students, which were collected before the programme was introduced, we found that the utilisation rate was particularly low among students with a tendency to procrastinate.
These findings are the reminiscent of behavioural economists' emphasis on the problem of self-control. An emerging body of recent empirical studies illustrate how the self-control issue arises in education, where a lack of self-control, including procrastination, can result in poor test performance or low grades (for an excellent survey, see Koch et al. 2015). Similarly, our experiment suggests that the self-control problem might have hindered the utilisation of the Skype programme, and, thus, it can significantly intervene when introducing an ICT-assisted education programme.
That said, our findings imply that the Skype programme has the potential to improve English education cost effectively, since Yashima et al. (2004) found that improvement in students' attitude can eventually lead to improvements in the English communication skills of Japanese students over time. Especially, in the ‘monocultural’ and ‘monolingual’ Japanese school environment, it is very difficult for students to develop a positive attitude toward foreign languages and experiences. Identifying the causal effect of the online English learning programme on students' positive attitude may lead us to consider how to combine regular English lectures and online English learning programme in a complementary manner.
More research appears to be warranted on how to improve and maintain students' motivation, particularly those with a tendency to procrastinate, to use such personalised and self-paced programmes when given in an extracurricular manner. This experiment was one of the first social experiments conducted in a real public high school context, and it therefore provides first-step insights into how to improve the quality of English education. Furthermore, we also argue that such contextualised social experiments conducted inside the Japanese education system itself (as opposed to a lab-type experiment) are expected to make more meaningful suggestions to solve problems that might occur in the Japanese context.
Editors' note: The main research on which this column is based appeared in a Discussion Paper of the Research Institute of Economy, Trade and Industry (RIETI) of Japan.
This article first appeared on www.VoxEU.org on May 20, 2017. Reproduced with permission.