Daniel Ho

All Publications

• GOVERNING BY ASSIGNMENT UNIVERSITY OF PENNSYLVANIA LAW REVIEW Cui, I., Ho, D. E., Martin, O., O'connell, A. 2024; 173 (1): 157-242

  View details for Web of Science ID 001369298700002

• Measuring and Mitigating Racial Disparities in Tax Audits QUARTERLY JOURNAL OF ECONOMICS Elzayn, H., Smith, E., Hertz, T., Guage, C., Ramesh, A., Fisher, R., Ho, D. E., Goldin, J. 2024

  View details for DOI 10.1093/qje/qjae027

  View details for Web of Science ID 001342517800001

• Remote sensing and computer vision for marine aquaculture. Science advances Quaade, S., Vallebueno, A., Alcabes, O. D., Rodolfa, K. T., Ho, D. E. 2024; 10 (42): eadn4944

  Abstract

  Aquaculture, the cultivation of aquatic plants and animals, has grown rapidly since the 1990s, but sparse, self-reported, and aggregated production data limit the effective understanding and monitoring of the industry's trends and potential risks. Building on a manual survey of aquaculture production from remote sensing imagery, we train a computer vision model to identify marine aquaculture cages from aerial and satellite imagery and generate a spatially explicit dataset of finfish production locations in the French Mediterranean from 2000 to 2021 including 4010 cages (average cage area, 69 square meters). We demonstrate the value of our method as an easily adaptable, cost-effective approach that can improve the speed and reliability of aquaculture surveys and enables downstream analyses relevant to researchers and regulators. We illustrate its use to compute independent estimates of production and develop a flexible framework to quantify uncertainty in these estimates. Overall, our study presents an efficient, scalable, and adaptable method for monitoring aquaculture production from remote sensing imagery.

  View details for DOI 10.1126/sciadv.adn4944

  View details for PubMedID 39413181

• Considerations for governing open foundation models. Science (New York, N.Y.) Bommasani, R., Kapoor, S., Klyman, K., Longpre, S., Ramaswami, A., Zhang, D., Schaake, M., Ho, D. E., Narayanan, A., Liang, P. 2024; 386 (6718): 151-153

  Abstract

  Different policy proposals may disproportionately affect the innovation ecosystem.

  View details for DOI 10.1126/science.adp1848

  View details for PubMedID 39388576

• Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models JOURNAL OF LEGAL ANALYSIS Dahl, M., Magesh, V., Suzgun, M., Ho, D. E. 2024; 16 (1): 64-93

  View details for DOI 10.1093/jla/laae003

  View details for Web of Science ID 001254187200001

• Not (Officially) in My Backyard JOURNAL OF THE AMERICAN PLANNING ASSOCIATION Jo, N., Vallebueno, A., Ouyang, D., Ho, D. E. 2024

  View details for DOI 10.1080/01944363.2024.2345730

  View details for Web of Science ID 001238099900001

• Integrating water quality data with a Bayesian network model to improve spatial and temporal phosphorus attribution: Application to the Maumee River Basin. Journal of environmental management Wei, Z., Alam, S., Verma, M., Hilderbran, M., Wu, Y., Anderson, B., Ho, D. E., Suckale, J. 2024; 360: 121120

  Abstract

  Surface water nutrient pollution, the primary cause of eutrophication, remains a major environmental concern in Western Lake Erie despite intergovernmental efforts to regulate nutrient sources. The Maumee River Basin has been the largest nutrient contributor. The two primary nutrient sources are inorganic fertilizer and livestock manure applied to croplands, which are later carried to the streams via runoff and soil erosion. Prior studies of nutrient source attribution have focused on large watersheds or counties at annual time scales. Source attribution at finer spatiotemporal scales, which enables more effective nutrient management, remains a substantial challenge. This study aims to address this challenge by developing a generalizable Bayesian network model for phosphorus source attribution at the subwatershed scale (12-digit Hydrologic Unit Code). Since phosphorus release is uncertain, we combine excess phosphorus derived from manure and fertilizer application and crop uptake data, flow information simulated by the SWAT model, and in-stream water quality measurements using Approximate Bayesian Computation to derive a posterior that attributes phosphorus contributions to subwatersheds. Our results show significant variability in subwatershed-scale phosphorus release that is lost in coarse-scale attribution. Phosphorus contributions attributed to the subwatersheds are on average lower than the excess phosphorus estimated by the nutrient balance approach currently adopted by environmental agencies. Fertilizer contributes more soluble reactive phosphorus than manure, while manure contributes most of the unreactive phosphorus. While developed for the specific context of Maumee River Basin, our lightweight and generalizable model framework could be adapted to other regions and pollutants and could help inform targeted environmental regulation and enforcement.

  View details for DOI 10.1016/j.jenvman.2024.121120

  View details for PubMedID 38759558

• Mitigating allocative tradeoffs and harms in an environmental justice data tool NATURE MACHINE INTELLIGENCE Huynh, B. Q., Chin, E. T., Koenecke, A., Ouyang, D., Ho, D. E., Kiang, M. V., Rehkopf, D. H. 2024

  View details for DOI 10.1038/s42256-024-00793-y

  View details for Web of Science ID 001163536000001

• Limitations of Reporting Requirements under California's Livestock Antimicrobial Restriction Law. Environmental health perspectives Quaade, S., Casey, J. A., Nachman, K. E., Tartof, S. Y., Ho, D. E. 2024; 132 (2): 25001

  Abstract

  Antimicrobial use in livestock production is considered a key contributor to growing antimicrobial resistance in bacteria. In 2015, California became the first state to enact restrictions on routine antimicrobial use in livestock production via Senate Bill 27 (SB27). SB27 further required the California Department of Food and Agriculture (CDFA) to collect and disseminate data on antimicrobial use in livestock production.The goal of this report is to assess whether CDFA's data release allows us to evaluate how antimicrobial use changed after the implementation of SB27.We combine the CDFA data with feed drug concentration ranges from the Code of Federal Regulation to evaluate the spread of plausible antimicrobial use trends. We also estimate antimicrobial consumption rates using data from the National Agricultural Statistical Service (NASS) and compare these to changes in medicated feed production reported by the CDFA.We show that CDFA's reported data are insufficient to reliably estimate whether antimicrobial usage has increased or decreased, most notably because no information is provided about the mass of antimicrobials approved for use or medicated feed drug concentrations. After incorporating additional external data on feed drug concentrations, one can at best provide uninformative bounds on the effect of SB27. We find some evidence that antimicrobial use has decreased by incorporating data on national sales of antimicrobials for food-producing animals, but the weakness of this inference underlines the need for improved data collection and dissemination, especially as other states seek to implement similar policies. We provide recommendations on how to improve reporting and data collection under SB27. https://doi.org/10.1289/EHP13702.

  View details for DOI 10.1289/EHP13702

  View details for PubMedID 38415616

  View details for PubMedCentralID PMC10901107

• Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features Elzayn, H., Black, E., Vossler, P., Jo, N., Goldin, J., Ho, D. E., IEEE COMPUTER SOC IEEE COMPUTER SOC. 2024: 161-193

  View details for DOI 10.1109/SaTML59370.2024.00016

  View details for Web of Science ID 001227324000008

• Regulating AI Adaptation: An Analysis of AI Medical Device Updates Wu, K., Wu, E., Rodolfa, K., Ho, D. E., Zou, J., Pollard, T., Choi, E., Singhal, P., Hughes, M., Sizikova, E., Mortazavi, B., Chen, Wang, F., Sarker, T., McDermott, M., Ghassemi, M. JMLR-JOURNAL MACHINE LEARNING RESEARCH. 2024: 477-488

  View details for Web of Science ID 001347132700029

• Leveraging genomic sequencing data to evaluate disease surveillance strategies. iScience Anderson, B., Ouyang, D., D'Agostino, A., Bonin, B., Smith, E., Kraushaar, V., Rudman, S. L., Ho, D. E. 2023; 26 (12): 108488

  Abstract

  In the face of scarce public health resources, it is critical to understand which disease surveillance strategies are effective, yet such validation has historically been difficult. From May 1 to December 31, 2021, a cohort study was carried out in Santa Clara County, California, in which 10,131 high-quality genomic sequences from COVID-19 polymerase chain reaction tests were merged with disease surveillance data. We measured the informational value, the fraction of sequenced links surfaced that are biologically plausible according to genomic sequence data, of different disease surveillance strategies. Contact tracing appeared more effective than spatiotemporal methods at uncovering nonresidential spread settings, school reporting appeared more fruitful than workplace reporting, and passively retrieved links through survey information presented some promise. Given the rapidly dwindling cost of sequencing, the informational value metric may enable near real-time, readily available evaluation of strategies by public health authorities to fight viral diseases beyond COVID-19.

  View details for DOI 10.1016/j.isci.2023.108488

  View details for PubMedID 38089591

  View details for PubMedCentralID PMC10711492

• Implications of predicting race variables from medical images. Science (New York, N.Y.) Zou, J., Gichoya, J. W., Ho, D. E., Obermeyer, Z. 2023; 381 (6654): 149-150

  Abstract

  AI-predicted race variables pose risks and opportunities for studying health disparities.

  View details for DOI 10.1126/science.adh4260

  View details for PubMedID 37440627

• Integrating social services with disease investigation: A randomized trial of COVID-19 high-touch contact tracing. PloS one Lu, L. C., Ouyang, D., D'Agostino, A., Diaz, A., Rudman, S. L., Ho, D. E. 2023; 18 (5): e0285752

  Abstract

  COVID-19 exposed and exacerbated health disparities, and a core challenge has been how to adapt pandemic response and public health in light of these disproportionate health burdens. Responding to this challenge, the County of Santa Clara Public Health Department designed a model of "high-touch" contact tracing that integrated social services with disease investigation, providing continued support and resource linkage for clients from structurally vulnerable communities. We report results from a cluster randomized trial of 5,430 cases from February to May 2021 to assess the ability of high-touch contact tracing to aid with isolation and quarantine. Using individual-level data on resource referral and uptake outcomes, we find that the intervention, randomized assignment to the high-touch program, increased the referral rate to social services by 8.4% (95% confidence interval, 0.8%-15.9%) and the uptake rate by 4.9% (-0.2%-10.0%), with the most pronounced increases in referrals and uptake of food assistance. These findings demonstrate that social services can be effectively combined with contact tracing to better promote health equity, demonstrating a novel path for the future of public health.

  View details for DOI 10.1371/journal.pone.0285752

  View details for PubMedID 37192191

  View details for PubMedCentralID PMC10187910

• Automated vs. manual case investigation and contact tracing for pandemic surveillance: Evidence from a stepped wedge cluster randomized trial. EClinicalMedicine Raymond, C., Ouyang, D., D'Agostino, A., Rudman, S. L., Ho, D. E. 2023; 55: 101726

  Abstract

  Background: Case investigation and contact tracing (CICT) is an important tool for communicable disease control, both to proactively interrupt chains of transmission and to collect information for situational awareness. We run the first randomized trial of COVID-19 CICT to investigate the utility of manual (i.e., call-based) vs. automated (i.e., survey-based) CICT for pandemic surveillance.Methods: Between December 15, 2021 and February 5, 2022, a stepped wedge cluster randomized trial was run in which Santa Clara County ZIP Codes progressively transitioned from manual to automated CICT. Eleven individual-level data fields on demographics and disease dynamics were observed for non-response. The data contains 106,522 positive cases across 29 ZIP Codes.Findings: Automated CICT reduced overall collected information by 29 percentage points (SE=0.08, p<0.01), as well as the response rate for individual fields. However, we find no evidence of differences in information loss by race or ethnicity.Interpretations: Automated CICT can serve as a useful alternative to manual CICT, with no substantial evidence of skewing data along racial or ethnic lines, but manual CICT improves completeness of key data for monitoring epidemiologic patterns.Funding: This research was supported in part by the Stanford Office of Community Engagement and the Stanford Institute for Human-Centered Artificial Intelligence.

  View details for DOI 10.1016/j.eclinm.2022.101726

  View details for PubMedID 36386031

• Entropy Regularization for Population Estimation Chugg, B., Henderson, P., Goldin, J., Ho, D. E., Williams, B., Chen, Y., Neville, J. ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2023: 12198-12204

  View details for Web of Science ID 001243749200078

• LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models Guha, N., Nyarko, J., Ho, D. E., Re, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D. N., Zambrano, D., Talisman, D., Hoque, E., Surani, F., Fagan, F., Sarfaty, G., Dickinson, G. M., Porat, H., Hegland, J., Wu, J., Nudell, J., Niklaus, J., Nay, J., Choi, J. H., Tobia, K., Hagan, M., Ma, M., Livermore, M., Rasumov-Rahe, N., Holzenberger, N., Kolt, N., Henderson, P., Rehaag, S., Goel, S., Gao, S., Williams, S., Gandhi, S., Zur, T., Iyer, V., Li, Z., Oh, A., Neumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2023

  View details for Web of Science ID 001226352803033

• Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selection Henderson, P., Chugg, B., Anderson, B., Altenburger, K., Turk, A., Guyton, J., Goldin, J., Ho, D. E., Williams, B., Chen, Y., Neville, J. ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2023: 5087-5095

  View details for Web of Science ID 001243763400135

• Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools Black, E., Naidu, R., Ghani, R., Rodolfa, K. T., Ho, D. E., Heidari, H., ACM ASSOC COMPUTING MACHINERY. 2023

  View details for DOI 10.1145/3617694.3623259

  View details for Web of Science ID 001124266900036

• The Privacy-Bias Tradeoff: Data Minimization and Racial Disparity Assessments in US Government Gupta, A., Wu, V. Y., Webley-Brown, H., King, J., Ho, D. E., ASSOC COMPUTING MACHINERY ASSOC COMPUTING MACHINERY. 2023: 492-505

  View details for DOI 10.1145/3593013.3594015

  View details for Web of Science ID 001062819300046

• How Redundant are Redundant Encodings? Blindness in the Wild and Racial Disparity when Race is Unobserved Cheng, L., Gallegos, I. O., Ouyang, D., Goldin, J., Ho, D. E., ASSOC COMPUTING MACHINERY ASSOC COMPUTING MACHINERY. 2023: 667-686

  View details for DOI 10.1145/3593013.3594034

  View details for Web of Science ID 001062819300063

• The Bureaucratic Challenge to AI Governance: An Empirical Assessment of Implementation at US Federal Agencies Lawrence, C., Cui, I., Ho, D. E., ACM ASSOC COMPUTING MACHINERY. 2023: 606-652

  View details for DOI 10.1145/3600211.3604701

  View details for Web of Science ID 001117838100049

• Executive Control of Agency Adjudication: Capacity, Selection, and Precedential Rulemaking JOURNAL OF LAW ECONOMICS & ORGANIZATION Hausman, D. K., Ho, D. E., Krass, M. S., McDonough, A. 2022

  View details for DOI 10.1093/jleo/ewac012

  View details for Web of Science ID 000870787800001

• Advances, challenges and opportunities in creating data for trustworthy AI NATURE MACHINE INTELLIGENCE Liang, W., Tadesse, G., Ho, D., Li, F., Zaharia, M., Zhang, C., Zou, J. 2022

  View details for DOI 10.1038/s42256-022-00516-1

  View details for Web of Science ID 000842575600001

• Can transportation subsidies reduce failures to appear in criminal court? Evidence from a pilot randomized controlled trial ECONOMICS LETTERS Brough, R., Freedman, M., Ho, D. E., Phillips, D. C. 2022; 216

  View details for DOI 10.1016/j.econlet.2022.110540

  View details for Web of Science ID 000829780800006

• Science Translation During the COVID-19 Pandemic: An Academic-Public Health Partnership to Assess Capacity Limits in California. American journal of public health Maldonado, P., Peng, A., Ouyang, D., Suckale, J., Ho, D. E. 1800; 112 (2): 308-315

  Abstract

  On the basis of an extensive academic-public health partnership around COVID-19 response, we illustrate the challenge of science-policy translation by examining one of the most common nonpharmaceutical interventions: capacity limits. We study the implementation of a 20% capacity limit in retail facilities in the California Bay Area. Through a difference-in-differences analysis, we show that the intervention caused no material reduction in visits, using the same large-scale mobile device data on human movements (mobility data) originally used in the academic literature to support such limits. We show that the lack of effectiveness stems from a mismatch between the academic metric of capacity relative to peak visits and the policy metric of capacity relative to building code. The disconnect in metrics is amplified by mobility data losing predictive power after the early months of the pandemic, weakening the policy relevance of mobility-based interventions. Nonetheless, the data suggest that a better-grounded rationale for capacity limits is to reduce risk specifically during peak hours. To enhance the connection between science, policy, and public health in future times of crisis, we spell out 3 strategies: living models, coproduction, and shared metrics. (Am J Public Health. 2022;112(2):308-315. https://doi.org/10.2105/AJPH.2021.306576).

  View details for DOI 10.2105/AJPH.2021.306576

  View details for PubMedID 35080959

• Beyond Ads: Sequential Decision-Making Algorithms in Law and Public Policy Henderson, P., Chugg, B., Anderson, B., Ho, D. E., ACM ASSOC COMPUTING MACHINERY. 2022: 87-100

  View details for DOI 10.1145/3511265.3550439

  View details for Web of Science ID 001074472400009

• Detecting Environmental Violations with Satellite Imagery in Near Real Time: Land Application under the Clean Water Act Chugg, B., Rothbacher, N., Feng, A., Long, X., Ho, D. E., ACM ASSOC COMPUTING MACHINERY. 2022: 3052-3062

  View details for DOI 10.1145/3511808.3557104

  View details for Web of Science ID 001074639603001

• Mapping Industrial Poultry Operations at Scale With Deep Learning and Aerial Imagery IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING Robinson, C., Chugg, B., Anderson, B., Ferres, J., Ho, D. E. 2022; 15: 7458-7471

  View details for DOI 10.1109/JSTARS.2022.3191544

  View details for Web of Science ID 000853871300010

• Enhancing environmental enforcement with near real-time monitoring: Likelihood-based detection of structural expansion of intensive livestock farms INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION Chugg, B., Anderson, B., Eicher, S., Lee, S., Ho, D. E. 2021; 103

  View details for DOI 10.1016/j.jag.2021.102463

  View details for Web of Science ID 000696914400003

• Disparate Limbo: How Administrative Law Erased Antidiscrimination YALE LAW JOURNAL Ceballos, C., Engstrom, D., Ho, D. E. 2021; 131 (2): 370-474

  View details for Web of Science ID 000725611900001

• Disparate Limbo: How Administrative Law Erased Antidiscrimination YALE LAW JOURNAL Ceballos, C., Engstrom, D., Ho, D. E. 2021; 131 (2): 370-474

  View details for Web of Science ID 000727945500006

• A language-matching model to improve equity and efficiency of COVID-19 contact tracing. Proceedings of the National Academy of Sciences of the United States of America Lu, L., Anderson, B., Ha, R., D'Agostino, A., Rudman, S. L., Ouyang, D., Ho, D. E. 2021; 118 (43)

  Abstract

  Contact tracing is a pillar of COVID-19 response, but language access and equity have posed major obstacles. COVID-19 has disproportionately affected minority communities with many non-English-speaking members. Language discordance can increase processing times and hamper the trust building necessary for effective contact tracing. We demonstrate how matching predicted patient language with contact tracer language can enhance contact tracing. First, we show how to use machine learning to combine information from sparse laboratory reports with richer census data to predict the language of an incoming case. Second, we embed this method in the highly demanding environment of actual contact tracing with high volumes of cases in Santa Clara County, CA. Third, we evaluate this language-matching intervention in a randomized controlled trial. We show that this low-touch intervention results in 1) significant time savings, shortening the time from opening of cases to completion of the initial interview by nearly 14 h and increasing same-day completion by 12%, and 2) improved engagement, reducing the refusal to interview by 4%. These findings have important implications for reducing social disparities in COVID-19; improving equity in healthcare access; and, more broadly, leveling language differences in public services.

  View details for DOI 10.1073/pnas.2109443118

  View details for PubMedID 34686604

• Improving the Reliability of Food Safety Disclosure: Restaurant Grading in Seattle and King County, Washington JOURNAL OF ENVIRONMENTAL HEALTH Ashwood, Z. C., Elias, B., Ho, D. E. 2021; 84 (2): 30-37

  View details for Web of Science ID 000692001100005

• Evaluation of Allocation Schemes of COVID-19 Testing Resources in a Community-Based Door-to-Door Testing Program JAMA HEALTH FORUM Chugg, B., Lu, L., Ouyang, D., Anderson, B., Ha, R., D'Agostino, A., Sujeer, A., Rudman, S. L., Garcia, A., Ho, D. E. 2021; 2 (8): e212260

  Abstract

  Overcoming social barriers to COVID-19 testing is an important issue, especially given the demographic disparities in case incidence rates and testing. Delivering culturally appropriate testing resources using data-driven approaches in partnership with community-based health workers is promising, but little data are available on the design and effect of such interventions.To assess and evaluate a door-to-door COVID-19 testing initiative that allocates visits by community health workers by selecting households in areas with a high number of index cases, by using uncertainty sampling for areas where the positivity rate may be highest, and by relying on local knowledge of the health workers.This cohort study was performed from December 18, 2020, to February 18, 2021. Community health workers visited households in neighborhoods in East San Jose, California, based on index cases or uncertainty sampling while retaining discretion to use local knowledge to administer tests. The health workers, also known as promotores de salud (hereinafter referred to as promotores) spent a mean of 4 days a week conducting door-to-door COVID-19 testing during the 2-month study period. All residents of East San Jose were eligible for COVID-19 testing. The promotores were selected from the META cooperative (Mujeres Empresarias Tomando Acción [Entrepreneurial Women Taking Action]).The promotores observed self-collection of anterior nasal swab samples for SARS-CoV-2 reverse transcriptase-polymerase chain reaction tests.A determination of whether door-to-door COVID-19 testing was associated with an increase in the overall number of tests conducted, the demographic distribution of the door-to-door tests vs local testing sites, and the difference in positivity rates among the 3 door-to-door allocation strategies.A total of 785 residents underwent door-to-door testing, and 756 were included in the analysis. Among the 756 individuals undergoing testing (61.1% female; 28.2% aged 45-64 years), door-to-door COVID-19 testing reached different populations than standard public health surveillance, with 87.6% (95% CI, 85.0%-89.8%) being Latinx individuals. The closest available testing site only reached 49.0% (95% CI, 48.3%-49.8%) Latinx individuals. Uncertainty sampling provided the most effective allocation, with a 10.8% (95% CI, 6.8%-16.0%) positivity rate, followed by 6.4% (95% CI, 4.1%-9.4%) for local knowledge, and 2.6% (95% CI, 0.7%-6.6%) for index area selection. The intervention was also associated with increased overall testing capacity by 60% to 90%, depending on the testing protocol.In this cohort study of 785 participants, uncertainty sampling, which has not been used conventionally in public health, showed promising results for allocating testing resources. Community-based door-to-door interventions and leveraging of community knowledge were associated with reduced demographic disparities in testing.

  View details for DOI 10.1001/jamahealthforum.2021.2260

  View details for Web of Science ID 000837103500002

  View details for PubMedID 35977196

  View details for PubMedCentralID PMC8796878

• How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature medicine Wu, E., Wu, K., Daneshjou, R., Ouyang, D., Ho, D. E., Zou, J. 2021

  View details for DOI 10.1038/s41591-021-01312-x

  View details for PubMedID 33820998

• Mandatory Retirement and Age, Race, and Gender Diversity of University Faculties AMERICAN LAW AND ECONOMICS REVIEW Ho, D. E., Mbonu, O., McDonough, A. 2021; 23 (1): 100-136

  View details for DOI 10.1093/aler/ahab002

  View details for Web of Science ID 000727841900003

• Brevity, Speed, and Deference: An Account from the Williams Chambers YALE JOURNAL ON REGULATION Hausman, D. K., Ho, D. E., O'Connell, A. 2021; 38 (3): 745-751

  View details for Web of Science ID 000646765100003

• How US law will evaluate artificial intelligence for covid-19. BMJ (Clinical research ed.) Krass, M. n., Henderson, P. n., Mello, M. M., Studdert, D. M., Ho, D. E. 2021; 372: n234

  View details for DOI 10.1136/bmj.n234

  View details for PubMedID 33722811

• EVALUATING FACIAL RECOGNITION TECHNOLOGY: A PROTOCOL FOR PERFORMANCE ASSESSMENT IN NEW DOMAINS DENVER LAW REVIEW Ho, D. E., Black, E., Agrawala, M., Li Fei-Fei 2021; 98 (4): 753-773

  View details for Web of Science ID 000686480300001

• The Effectiveness of a Neighbor-to-Neighbor Get-Out-the-Vote Program: Evidence from the 2017 Virginia State Elections JOURNAL OF EXPERIMENTAL POLITICAL SCIENCE Handan-Nader, C., Ho, D. E., Morantz, A., Rutter, T. A. 2021; 8 (2): 145-160

  View details for DOI 10.1017/XPS.2020.11

  View details for Web of Science ID 000745035900007

• Algorithmic Accountability in the Administrative State YALE JOURNAL ON REGULATION Engstrom, D., Ho, D. E. 2020; 37 (3): 800–854

  View details for Web of Science ID 000567989300001

• Improving Scientific Judgments in Law and Government: A Field Experiment of Patent Peer Review JOURNAL OF EMPIRICAL LEGAL STUDIES Ho, D. E., Ouellette, L. 2020; 17 (2): 190–223

  View details for DOI 10.1111/jels.12249

  View details for Web of Science ID 000534645700001

• Due Process and Mass Adjudication: Crisis and Reform STANFORD LAW REVIEW Ames, D., Handan-Nader, C., Ho, D. E., Marcus, D. 2020; 72 (1): 1–78

  View details for Web of Science ID 000512208900001

• Feasible Policy Evaluation by Design: A Randomized Synthetic Stepped-Wedge Trial of Mandated Disclosure in King County. Evaluation review Handan-Nader, C. n., Ho, D. E., Elias, B. n. 2020: 193841X20930852

  Abstract

  Evidence-based policy is limited by the perception that randomized controlled trials (RCTs) are expensive and infeasible. We argue that carefully tailored research design can overcome these challenges and enable more widespread randomized evaluations of policy implementation. We demonstrate how a stepped-wedge (randomized rollout) design that adapts synthetic control methods overcame substantial practical, administrative, political, and statistical constraints to evaluating King County's new food safety rating system. The core RCT component of the evaluation came at little financial cost to the government, allowed the entire county to be treated, and resulted in no functional implementation delay. The case of restaurant sanitation grading has played a critical role in the scholarship on information disclosure, and our study provides the first evidence from a randomized trial of the causal effects of grading on health outcomes. We find that the grading system had no appreciable effects on foodborne illness, hospitalization, or food handling practices but that the system may have marginally increased public engagement by encouraging higher reporting.

  View details for DOI 10.1177/0193841X20930852

  View details for PubMedID 32527152

• Menu labeling, calories, and nutrient density: Evidence from chain restaurants. PloS one Ho, D. E., Mbonu, O. n., McDonough, A. n., Pottash, R. n. 2020; 15 (5): e0232656

  Abstract

  The Food and Drug Administration's menu labeling rule requires chain restaurants to prominently display calories, while leaving other nutritional information (e.g., fat, sodium, sugar) to the request of consumers. We use rich micronutrient data from 257 large chain brands and 24,076 menu items to examine whether calories are correlated with widely used "nutrient profile" scores that measure healthfulness based on nutrient density. We show that calories are indeed statistically significant predictors of nutrient density. However, as a substantive matter, the correlation is highly attenuated (partial R2 < 0.01). Our findings (a) suggest that the promise of calorie labeling to improve nutrient intake quality at restaurants is limited and (b) clarify the basis for transparency of nutrient composition beyond calories to promote healthy menu choices.

  View details for DOI 10.1371/journal.pone.0232656

  View details for PubMedID 32379786

• New Evidence on Information Disclosure through Restaurant Hygiene Grading AMERICAN ECONOMIC JOURNAL-ECONOMIC POLICY Ho, D. E., Ashwood, Z. C., Handan-Nader, C. 2019; 11 (4): 404–28

  View details for DOI 10.1257/pol.20180230

  View details for Web of Science ID 000493992000014

• Quality Review of Mass Adjudication: A Randomized Natural Experiment at the Board of Veterans Appeals, 2003-16 JOURNAL OF LAW ECONOMICS & ORGANIZATION Ho, D. E., Handan-Nader, C., Ames, D., Marcus, D. 2019; 35 (2): 239–88

  View details for DOI 10.1093/jleo/ewz001

  View details for Web of Science ID 000482267900001

• Deep learning to map concentrated animal feeding operations NATURE SUSTAINABILITY Handan-Nader, C., Ho, D. E. 2019; 2 (4): 298–306

  View details for DOI 10.1038/s41893-019-0246-x

  View details for Web of Science ID 000463925700016

• When Algorithms Import Private Bias into Public Enforcement: The Promise and Limitations of Statistical Debiasing Solutions Altenburger, K. M., Ho, D. E. J C B MOHR. 2019: 98–122

  View details for DOI 10.1628/jite-2019-0001

  View details for Web of Science ID 000496129300011

• Is Yelp Actually Cleaning Up the Restaurant Industry? A Re-Analysis on the Relative Usefulness of Consumer Reviews Altenburger, K. M., Ho, D. E., Assoc Comp Machinery ASSOC COMPUTING MACHINERY. 2019: 2543–50

  View details for DOI 10.1145/3308558.3313683

  View details for Web of Science ID 000483508402056

• Do Checklists Make a Difference? A Natural Experiment from Food Safety Enforcement JOURNAL OF EMPIRICAL LEGAL STUDIES Ho, D. E., Sherman, S., Wyman, P. 2018; 15 (2): 242–77

  View details for Web of Science ID 000431655600001

• Does Peer Review Work? An Experiment of Experimentalism STANFORD LAW REVIEW Ho, D. E. 2017; 69 (1): 1-119

  View details for Web of Science ID 000394413800001

• Managing Street-Level Arbitrariness: The Evidence Base for Public Sector Quality Improvement ANNUAL REVIEW OF LAW AND SOCIAL SCIENCE, VOL 13 Ho, D. E., Sherman, S., Hagan, J. 2017; 13: 251–72

  View details for DOI 10.1146/annurev-lawsocsci-110316-113608

  View details for Web of Science ID 000413327700014

• TESTING THE MARKETPLACE OF IDEAS NEW YORK UNIVERSITY LAW REVIEW Ho, D. E., Schauer, F. 2015; 90 (4): 1160-1228

  View details for Web of Science ID 000363599300006

• Randomizing ... What? A Field Experiment of Child Access Voting Laws JOURNAL OF INSTITUTIONAL AND THEORETICAL ECONOMICS-ZEITSCHRIFT FUR DIE GESAMTE STAATSWISSENSCHAFT Ho, D. E. 2015; 171 (1): 150-170

  View details for DOI 10.1628/093245615X14189721363901

  View details for Web of Science ID 000352672000016

• Does Class Size Affect the Gender Gap? A Natural Experiment in Law JOURNAL OF LEGAL STUDIES Ho, D. E., Kelman, M. G. 2014; 43 (2): 291-321

  View details for DOI 10.1086/676953

  View details for Web of Science ID 000344362400003

• Foreword: Conference Bias JOURNAL OF EMPIRICAL LEGAL STUDIES Ho, D. E. 2013; 10 (4): 603–11

  View details for DOI 10.1111/jels.12031

  View details for Web of Science ID 000325985200001

• INTRODUCTION: THE EMPIRICAL REVOLUTION IN LAW STANFORD LAW REVIEW Ho, D. E., Kramer, L. 2013; 65 (6): 1195–1202

  View details for Web of Science ID 000321534100001

• Fudging the Nudge: Information Disclosure and Restaurant Grading YALE LAW JOURNAL Ho, D. E. 2012; 122 (3): 574-688

  View details for Web of Science ID 000313382600002

• MatchIt: Nonparametric Preprocessing for Parametric Causal Inference JOURNAL OF STATISTICAL SOFTWARE Ho, D. E., Imai, K., King, G., Stuart, E. A. 2011; 42 (8)

  View details for Web of Science ID 000292097500001

• Credible Causal Inference for Empirical Legal Studies ANNUAL REVIEW OF LAW AND SOCIAL SCIENCE, VOL 7 Ho, D. E., Rubin, D. B. 2011; 7: 17-40

  View details for DOI 10.1146/annurev-lawsocsci-102510-105423

  View details for Web of Science ID 000299297100002

• How Not to Lie with Judicial Votes: Misconceptions, Measurement, and Models CALIFORNIA LAW REVIEW Ho, D. E., Quinn, K. M. 2010; 98 (3): 813-876

  View details for Web of Science ID 000283520600007

• Reconciling Punitive Damages Evidence JOURNAL OF INSTITUTIONAL AND THEORETICAL ECONOMICS-ZEITSCHRIFT FUR DIE GESAMTE STAATSWISSENSCHAFT Ho, D. E. 2010; 166 (1): 27-32

  View details for Web of Science ID 000275621300003

• DID LIBERAL JUSTICES INVENT THE STANDING DOCTRINE? AN EMPIRICAL STUDY OF THE EVOLUTION OF STANDING, 1921-2006 STANFORD LAW REVIEW Ho, D. E., Ross, E. L. 2010; 62 (3): 591-667

  View details for Web of Science ID 000276780200001

• VIEWPOINT DIVERSITY AND MEDIA CONSOLIDATION: AN EMPIRICAL STUDY STANFORD LAW REVIEW Ho, D. E., Quinn, K. M. 2009; 61 (4): 781-868

  View details for Web of Science ID 000265328000001

• Improving the Presentation and Interpretation of Online Ratings Data with Model-Based Figures AMERICAN STATISTICIAN Ho, D. E., Quinn, K. M. 2008; 62 (4): 279-288

  View details for DOI 10.1198/000313008X366145

  View details for Web of Science ID 000268503800001

• Evaluating Course Evaluations: An Empirical Analysis of a Quasi-Experiment at the Stanford Law School, 2000-2007 JOURNAL OF LEGAL EDUCATION Ho, D. E., Shapiro, T. H. 2008; 58 (3): 388-412

  View details for Web of Science ID 000264077500007

• Estimating causal effects of ballot order from a randomized natural experiment - The California alphabet lottery, 1978-2002 PUBLIC OPINION QUARTERLY Ho, D. E., Imai, K. 2008; 72 (2): 216-240

  View details for DOI 10.1093/poq/nfn018

  View details for Web of Science ID 000256523900003

• Measuring Explicit Political Positions of Media QUARTERLY JOURNAL OF POLITICAL SCIENCE Ho, D. E., Quinn, K. M. 2008; 3 (4): 353-377

  View details for DOI 10.1561/100.00008048

  View details for Web of Science ID 000262619800002

• Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference POLITICAL ANALYSIS Ho, D. E., Imai, K., King, G., Stuart, E. A. 2007; 15 (3): 199-236

  View details for DOI 10.1093/pan/mpl013

  View details for Web of Science ID 000248544000002

• Randomization inference with natural experiments: An analysis of ballot effects in the 2003 California recall election 63rd Annual Meeting of the Midwest-Political-Science-Association Ho, D. E., Imai, K. AMER STATISTICAL ASSOC. 2006: 888–900

  View details for DOI 10.1198/016214505000001258

  View details for Web of Science ID 000240158700003