Machine Learning as Natural Experiment: Method and Deployment at Japanese Firms

Author Name NARITA Yusuke (Visiting Fellow, RIETI) / AIHARA Shunsuke (Hanjuku Kaso Inc., ZOZO Technologies, Inc.) / SAITO Yuta (Hanjuku Kaso Inc., Tokyo Institute of Technology) / MATSUTANI Megumi (ZOZO Technologies, Inc.) / YATA Kohei (Yale University)
Creation Date/NO. December 2020 20-J-045
Download / Links


From public policy to business, machine learning and other algorithms produce a growing portion of treatment decisions and recommendations. Such algorithmic decisions are natural experiments (conditionally quasi-randomly assigned instruments) since the algorithms make decisions based only on observable input variables. We use this observation to characterize the sources of causal-effect identification for a class of stochastic and deterministic algorithms. This identification result translates into consistent estimators of causal effects and the counterfactual performance of new algorithms. We apply our method to improve a large-scale fashion e-commerce platform (ZOZOTOWN). We conclude by providing public policy applications.