Optimal Sample Allocation in Multilevel Experiments

Zuchao Shen

One key consideration in designing multilevel randomized trials is to ensure that designs have desired chance to detect treatment effects if they exist. Given a fixed budget and cost structures, three are sample sizes allocated across levels and treatment conditions that can maximize these chances or statistical power. However, literature investigating optimal sample allocation has largely been limited in the types of parameters it considers to be malleable (e.g., balanced and/or constrained designs). Furthermore, the lack of power analysis tools explicitly accommodating costs and budget may discourage thoughtful consideration of the cost-efficient design strategies in the planning stage. This project addresses this gap and will cover optimal sample allocations for two-, three-, and four-level (multisite) cluster-randomized trials. For each type of trial, I will derive the optimal sample allocation when sampling costs vary across treatment conditions and levels of hierarchy. I will also investigate the robustness of the derived optimal design parameters to the misspecification on initial values of cost structure and design parameters. The preliminary results for two-level cluster-randomized trials show that the proposed framework can identify sample allocation with more power than previous frameworks under the same budget, and the derived optimal design parameters are fairly robust to the misspecification on initial values of cost structure and design parameters. I am also developing an open-source R package odr (https://cran.r-project.org/package=odr) to implement proposed methods and power analyses accommodating costs and budget, which will help researchers design more cost-efficient and rigorous multilevel experiments.

About Zuchao Shen

Zuchao Shen is a doctoral candidate in the Department of Educational Studies at the University of Cincinnati. His current research centers on the development of methods that support rigorous and efficient observational, quasi-experimental and experimental designs in education and social sciences with a specific focus on multilevel structures such as those commonly seen in schooling (e.g., students nested within schools). He is also interested in methods that investigate and support large-scale complex data modeling (e.g., machine learning, missing data, and matching). His substantive research interests are in school reform, educational equity, program evaluation and policy analysis. His work has appeared in Multivariate Behavioral Research, Studies in Educational Evaluation, Evaluation Review, School Effectiveness and School Improvement, and other journals. He was a three-year recipient of a Hazel F. Gabbard Doctoral Research Associate Scholarship from University of Cincinnati. Prior to coming to University of Cincinnati, Zuchao earned his M.A. in Economics of Education and Educational Administration from Peking University and subsequently worked for four years in marketing research and consulting companies in China.