New Methodology for Difference-In-Difference Where Both No Auto-correlation Assumption And Stable Unit Treatment Value Assumption May Not Hold In Policy Impact Assessment

         
Author Name KAINOU Kazunari (Fellow, RIETI)
Creation Date/NO. November 2019 19-J-065
Download / Links

Abstract

Difference-In-Difference(DID) is frequently used methodology in Policy Impact Assessment, but necessary assumptions to be confirmed for DID or ways to fulfill and ensure them are not clearly identified nor well developed yet. Especially in case of ignoring No Auto-correlations Assumptions(NACA) or Stable Unit Treatment Value Assumption(SUTVA) related problems may cause certain bias in the estimated assessment results.

This paper shows that four majpr assumptions need to be confirmed in DID, namely Overalap, Conditional Independence, NACA and SUTVA, based on the inductive survay of academic papers in Economics, Sociology and so on, and identifies existing measures applicable for three major approaches of DID, experimental approach with rendomisation, statistical approach with matching or synthetic control group.

Based on the inductive survay above, this paper proposes new methodology applocable for statistical approaches where both NACA and SUTVA related problems may happen. Assuming that secondary effect from treated group to control group are identical and other assumptions holds, checking the significance of constant terms in the result of regression analysis of rate of before-after indicator(BAI) and difference-in-difference indicator(DIDI) with inverse DIDI for each control group samples provides solutions for both NACA and SUTVA related problems.

In order to demonstrate the practicality of the new methodology, this paper tries treatment effect evaluation of Fukushima rice price before and after the East Japan Great Earthquake and Fukusima No.1 Plant Nuclear Accident where NACA and SUTVA related problems exist. And this paper quantified possible bias caused by four major assumptions related problems of DID and concluded that SUTVA related problems potentially cause largest bias in the results in this case.