Julian Nyarko

All Publications

• Introducing a New Corpus of Definitive M&A Agreements, 2000-2020 JOURNAL OF EMPIRICAL LEGAL STUDIES Adelson, P., Jennejohn, M., Nyarko, J., Talley, E. 2024

  View details for DOI 10.1111/jels.12410

  View details for Web of Science ID 001387455600001

• Risk scores, label bias, everything but the kitchen sink. Science advances Zanger-Tishler, M., Nyarko, J., Goel, S. 2024; 10 (13): eadi8411

  Abstract

  In designing risk assessment algorithms, many scholars promote a "kitchen sink" approach, reasoning that more information yields more accurate predictions. We show, however, that this rationale often fails when algorithms are trained to predict a proxy of the true outcome, for instance, predicting arrest as a proxy for criminal behavior. With this "label bias," one should exclude a feature if its correlation with the proxy and its correlation with the true outcome have opposite signs, conditional on the other model features. This criterion is often satisfied when a feature is weakly correlated with the true outcome, and, additionally, that feature and the true outcome are both direct causes of the proxy outcome. For example, criminal behavior and geography may be weakly correlated and, due to patterns of police deployment, direct causes of one's arrest record-suggesting that excluding geography in criminal risk assessment will weaken an algorithm's performance in predicting arrest but will improve its capacity to predict actual crime.

  View details for DOI 10.1126/sciadv.adi8411

  View details for PubMedID 38552013

• Designing equitable algorithms. Nature computational science Chohlas-Wood, A., Coots, M., Goel, S., Nyarko, J. 2023; 3 (7): 601-610

  Abstract

  Predictive algorithms are now commonly used to distribute society's resources and sanctions. But these algorithms can entrench and exacerbate inequities. To guard against this possibility, many have suggested that algorithms be subject to formal fairness constraints. Here we argue, however, that popular constraints-while intuitively appealing-often worsen outcomes for individuals in marginalized groups, and can even leave all groups worse off. We outline a more holistic path forward for improving the equity of algorithmically guided decisions.

  View details for DOI 10.1038/s43588-023-00485-4

  View details for PubMedID 38177749

• Designing equitable algorithms NATURE COMPUTATIONAL SCIENCE Chohlas-Wood, A., Coots, M., Goel, S., Nyarko, J. 2023; 3 (7): 601-610

  View details for DOI 10.1038/s43588-023-00485-4

  View details for Web of Science ID 001035781100010

• Racial bias as a multi-stage, multi-actor problem: An analysis of pretrial detention JOURNAL OF EMPIRICAL LEGAL STUDIES Grossman, J., Nyarko, J., Goel, S. 2023

  View details for DOI 10.1111/jels.12343

  View details for Web of Science ID 000910125100001

• Do Judges Matter? JOURNAL OF INSTITUTIONAL AND THEORETICAL ECONOMICS-ZEITSCHRIFT FUR DIE GESAMTE STAATSWISSENSCHAFT Nyarko, J. 2023; 179 (1): 247-249

  View details for DOI 10.1628/jite-2023-0021

  View details for Web of Science ID 000925424100022

• LEGALBENCH: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models Guha, N., Nyarko, J., Ho, D. E., Re, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D. N., Zambrano, D., Talisman, D., Hoque, E., Surani, F., Fagan, F., Sarfaty, G., Dickinson, G. M., Porat, H., Hegland, J., Wu, J., Nudell, J., Niklaus, J., Nay, J., Choi, J. H., Tobia, K., Hagan, M., Ma, M., Livermore, M., Rasumov-Rahe, N., Holzenberger, N., Kolt, N., Henderson, P., Rehaag, S., Goel, S., Gao, S., Williams, S., Gandhi, S., Zur, T., Iyer, V., Li, Z., Oh, A., Neumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2023

  View details for Web of Science ID 001226352803033

• Don't Use a Cannon to Kill a Fly: An Efficient Cascading Pipeline for Long Documents Li, Z., Guha, N., Nyarko, J., ACM ASSOC COMPUTING MACHINERY. 2023: 141-147

  View details for DOI 10.1145/3594536.3595142

  View details for Web of Science ID 001139079400015

• Conceptual Questions in Developing Expert-Annotated Data Ma, M., Waldon, B., Nyarko, J., ACM ASSOC COMPUTING MACHINERY. 2023: 427-431

  View details for DOI 10.1145/3594536.3595139

  View details for Web of Science ID 001139079400049

• Natural Language Processing in Legal Tech LEGAL TECH AND THE FUTURE OF CIVIL JUSTICE Frankenreiter, J., Nyarko, J., Engstrom, D. F. 2023: 70-90

  View details for Web of Science ID 001003618200004

• Contractual Evolution UNIVERSITY OF CHICAGO LAW REVIEW Jennejohn, M., Nyarko, J., Talley, E. 2022; 89 (4): 901-978

  View details for Web of Science ID 000811187200002

• Regulatory Diffusion STANFORD LAW REVIEW Nou, J., Nyarko, J. 2022; 74 (5): 897-968

  View details for Web of Science ID 000827125000001

• Police agencies on Facebook overreport on Black suspects. Proceedings of the National Academy of Sciences of the United States of America Grunwald, B., Nyarko, J., Rappaport, J. 2022; 119 (45): e2203089119

  Abstract

  A large and growing share of the American public turns to Facebook for news. On this platform, reports about crime increasingly come directly from law enforcement agencies, raising questions about content curation. We gathered all posts from almost 14,000 Facebook pages maintained by US law enforcement agencies, focusing on reporting about crime and race. We found that Facebook users are exposed to posts that overrepresent Black suspects by 25 percentage points relative to local arrest rates. This overexposure occurs across crime types and geographic regions and increases with the proportion of both Republican voters and non-Black residents. Widespread exposure to overreporting risks reinforcing racial stereotypes about crime and exacerbating punitive preferences among the polity more generally.

  View details for DOI 10.1073/pnas.2203089119

  View details for PubMedID 36322743

• A Statistical Test for Legal Interpretation: Theory and Applications JOURNAL OF LAW ECONOMICS & ORGANIZATION Nyarko, J., Sanga, S. 2021

  View details for DOI 10.1093/jleo/ewab038

  View details for Web of Science ID 000764828800001

• Stickiness and Incomplete Contracts UNIVERSITY OF CHICAGO LAW REVIEW Nyarko, J. 2021; 88 (1): 1–79

  View details for Web of Science ID 000607450500001

• Blind Justice: Algorithmically Masking Race in Charging Decisions Chohlas-Wood, A., Nudell, J., Yao, K., Lin, Z., Nyarko, J., Goel, S., ASSOC COMP MACHINERY ASSOC COMPUTING MACHINERY. 2021: 35-45

  View details for DOI 10.1145/3461702.3462524

  View details for Web of Science ID 000767973400006

• A COMPUTATIONAL ANALYSIS OF CONSTITUTIONAL POLARIZATION CORNELL LAW REVIEW Pozen, D. E., Talley, E. L., Nyarko, J. 2020; 105 (1): 1–84

  View details for Web of Science ID 000594813500001

• Conforming against Expectations: The Formalism of Nonlawyers at the World Trade Organization JOURNAL OF LEGAL STUDIES Nyarko, J., Hsiang, J. 2019; 48 (2): 341–75

  View details for DOI 10.1086/702167

  View details for Web of Science ID 000507294200004