The purpose of this study was to verify whether the objectives of third-party evaluation services in child-care centers, “to improve the quality of welfare services provided by service providers” and “to provide information to help users select services,” were achieved, through analysis of the results of Tokyo third-party evaluations of welfare services. The results of the third-party evaluations of 3084 accredited nursery centers conducted from FY 2018 to FY 2021, which were the subjects of the analysis, showed the high implementation rate of both ”common evaluation items” and ”user surveys”, which made it difficult to capture differences among childcare centers and to verify quality improvement. In addition, using the “Early Childhood Environment Rating Scale, third edition” (ECERS-3) whose reliability and validity have been academically confirmed as a scale for quantitatively evaluating childcare quality, we found no correlation between the results of ECERS-3 at accredited nursery centers in a city, Tokyo, and the results of a third-party evaluation published for the same facilities. Furthermore, there were no correlation between the results of the third-party evaluation and the children’s developmental status, academic performance after schooling, or parents’ feelings toward their children. These findings suggest that there are issues regarding the accuracy and sensitivity of third-party evaluations as indicators of childcare quality, as well as the validity of the evaluation content.