Judith Ellen Fan

All Publications

• Evaluating convergence between two data visualization literacy assessments. Cognitive research: principles and implications Brockbank, E., Verma, A., Lloyd, H., Huey, H., Padilla, L., Fan, J. E. 2025; 10 (1): 15

  Abstract

  Data visualizations play a crucial role in communicating patterns in quantitative data, making data visualization literacy a key target of STEM education. However, it is currently unclear to what degree different assessments of data visualization literacy measure the same underlying constructs. Here, we administered two widely used graph comprehension assessments (Galesic and Garcia-Retamero in Med Dec Mak 31:444-457, 2011; Lee et al. in IEEE Trans Vis Comput Graph 235:51-560, 2016) to both a university-based convenience sample and a demographically representative sample of adult participants in the USA (N=1,113). Our analysis of individual variability in test performance suggests that overall scores are correlated between assessments and associated with the amount of prior coursework in mathematics. However, further exploration of individual error patterns suggests that these assessments probe somewhat distinct components of data visualization literacy, and we do not find evidence that these components correspond to the categories that guided the design of either test (e.g., questions that require retrieving values rather than making comparisons). Together, these findings suggest opportunities for development of more comprehensive assessments of data visualization literacy that are organized by components that better account for detailed behavioral patterns.

  View details for DOI 10.1186/s41235-025-00622-9

  View details for PubMedID 40188224

• Understanding Physical Dynamics with Counterfactual World Modeling Venkatesh, R., Chen, H., Feigelis, K., Bear, D. M., Jedoui, K., Kotar, K., Binder, F., Lee, W., Liu, S., Smith, K. A., Fan, J. E., Yamins, D. L. K. edited by Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. SPRINGER INTERNATIONAL PUBLISHING AG. 2025: 368-387

  View details for DOI 10.1007/978-3-031-72691-0_21

  View details for Web of Science ID 001353689800021

• Drawing as a means to characterize memory and cognition. Memory & cognition Bainbridge, W. A., Chamberlain, R., Wammes, J., Fan, J. E. 2024

  Abstract

  As psychological research embraces more naturalistic questions and large-scale analytic methods, drawing has emerged as an exciting tool for studying cognition. Drawing provides rich information about how we view the world, ranging from largely veridical perceptual representations to abstracted meta-cognitive representations. Drawing also requires the integration of multiple processes (e.g., vision, memory, motor learning), and experience with drawing can have an impact on such processes. As a result, drawing presents several interesting cognitive questions, while also providing a way to gain insight into a multitude of others. This Special Issue features 25 cutting-edge studies utilizing drawing to reveal discoveries transversing fields in psychology. These diverse studies investigate drawing across children, young adults, older adults, and special populations such as individuals with blindness, anterograde amnesia, apraxia, and semantic dementia. These studies detail new discoveries about the mechanisms underlying memory, attention, mathematical reasoning, and other cognitive processes. They employ a range of methods including psychophysical experiments, deep learning, and neuroimaging. Finally, many of these studies cover topics about the impact of drawing as a process on other cognitive processes, including how drawing expertise impacts other processes like visual memory or spatial abilities. Overall, this collection of studies paves the way for an exciting future of drawing as a commonplace tool used by psychologists to understand complex phenomena.

  View details for DOI 10.3758/s13421-024-01618-4

  View details for PubMedID 39192141

  View details for PubMedCentralID 9046317

• Using games to understand the mind. Nature human behaviour Allen, K., Brandle, F., Botvinick, M., Fan, J. E., Gershman, S. J., Gopnik, A., Griffiths, T. L., Hartshorne, J. K., Hauser, T. U., Ho, M. K., de Leeuw, J. R., Ma, W. J., Murayama, K., Nelson, J. D., van Opheusden, B., Pouncy, T., Rafner, J., Rahwan, I., Rutledge, R. B., Sherson, J., Simsek, O., Spiers, H., Summerfield, C., Thalmann, M., Velez, N., Watrous, A. J., Tenenbaum, J. B., Schulz, E. 2024

  Abstract

  Board, card or video games have been played by virtually every individual in the world. Games are popular because they are intuitive and fun. These distinctive qualities of games also make them ideal for studying the mind. By being intuitive, games provide a unique vantage point for understanding the inductive biases that support behaviour in more complex, ecological settings than traditional laboratory experiments. By being fun, games allow researchers to study new questions in cognition such as the meaning of 'play' and intrinsic motivation, while also supporting more extensive and diverse data collection by attracting many more participants. We describe the advantages and drawbacks of using games relative to standard laboratory-based experiments and lay out a set of recommendations on how to gain the most from using games to study cognition. We hope this Perspective will lead to a wider use of games as experimental paradigms, elevating the ecological validity, scale and robustness of research on the mind.

  View details for DOI 10.1038/s41562-024-01878-9

  View details for PubMedID 38907029

• Parallel developmental changes in children's production and recognition of line drawings of visual concepts. Nature communications Long, B., Fan, J. E., Huey, H., Chai, Z., Frank, M. C. 2024; 15 (1): 1191

  Abstract

  Childhood is marked by the rapid accumulation of knowledge and the prolific production of drawings. We conducted a systematic study of how children create and recognize line drawings of visual concepts. We recruited 2-10-year-olds to draw 48 categories via a kiosk at a children's museum, resulting in >37K drawings. We analyze changes in the category-diagnostic information in these drawings using vision algorithms and annotations of object parts. We find developmental gains in children's inclusion of category-diagnostic information that are not reducible to variation in visuomotor control or effort. Moreover, even unrecognizable drawings contain information about the animacy and size of the category children tried to draw. Using guessing games at the same kiosk, we find that children improve across childhood at recognizing each other's line drawings. This work leverages vision algorithms to characterize developmental changes in children's drawings and suggests that these changes reflect refinements in children's internal representations.

  View details for DOI 10.1038/s41467-023-44529-9

  View details for PubMedID 38331850

  View details for PubMedCentralID 2991405

• How do video content creation goals impact which concepts people prioritize when generating B-roll imagery? Huey, H., Leake, M., Aneja, D., Fisher, M., Fan, J. E., ASSOC COMPUTING MACHINERY ASSOC COMPUTING MACHINERY. 2024: 542-549

  View details for DOI 10.1145/3635636.3664252

  View details for Web of Science ID 001268585500056

• Open Vocabulary Semantic Scene Sketch Understanding Bourouis, A., Fan, J. E., Gryaditskaya, Y., IEEE COMPUTER SOC IEEE COMPUTER SOC. 2024: 4176-4186

  View details for DOI 10.1109/CVPR52733.2024.00400

  View details for Web of Science ID 001322555904054

• Communicating Design Intent Using Drawing and Text McCarthy, W. P., Matejka, J., Willis, K. D. D., Fan, J. E., Pu, Y., ASSOC COMPUTING MACHINERY ASSOC COMPUTING MACHINERY. 2024: 512-519

  View details for DOI 10.1145/3635636.3664261

  View details for Web of Science ID 001268585500051

• Consistency and Variation in Reasoning About Physical Assembly. Cognitive science McCarthy, W. P., Kirsh, D., Fan, J. E. 2023; 47 (12): e13397

  Abstract

  The ability to reason about how things were made is a pervasive aspect of how humans make sense of physical objects. Such reasoning is useful for a range of everyday tasks, from assembling a piece of furniture to making a sandwich and knitting a sweater. What enables people to reason in this way even about novel objects, and how do people draw upon prior experience with an object to continually refine their understanding of how to create it? To explore these questions, we developed a virtual task environment to investigate how people come up with step-by-step procedures for recreating block towers whose composition was not readily apparent, and analyzed how the procedures they used to build them changed across repeated attempts. Specifically, participants (N = 105) viewed 2D silhouettes of eight unique block towers in a virtual environment simulating rigid-body physics, and aimed to reconstruct each one in less than 60 s. We found that people built each tower more accurately and quickly across repeated attempts, and that this improvement reflected both group-level convergence upon a tiny fraction of all possible viable procedures, as well as error-dependent updating across successive attempts by the same individual. Taken together, our study presents a scalable approach to measuring consistency and variation in how people infer solutions to physical assemblyproblems.

  View details for DOI 10.1111/cogs.13397

  View details for PubMedID 38146204

• Creating ad hoc graphical representations of number. Cognition Holt, S., Fan, J. E., Barner, D. 2023; 242: 105665

  Abstract

  The ability to communicate about exact number is critical to many modern human practices spanning science, industry, and politics. Although some early numeral systems used 1-to-1 correspondence (e.g., 'IIII' to represent 4), most systems provide compact representations via more arbitrary conventions (e.g., '7' and 'VII'). When people are unable to rely on conventional numerals, however, what strategies do they initially use to communicate number? Across three experiments, participants used pictures to communicate about visual arrays of objects containing 1-16 items, either by producing freehand drawings or combining sets of visual tokens. We analyzed how the pictures they produced varied as a function of communicative need (Experiment 1), spatial regularities in the arrays (Experiment 2), and visual properties of tokens (Experiment 3). In Experiment 1, we found that participants often expressed number in the form of 1-to-1 representations, but sometimes also exploited the configuration of sets. In Experiment 2, this strategy of using configural cues was exaggerated when sets were especially large, and when the cues were predictably correlated with number. Finally, in Experiment 3, participants readily adopted salient numerical features of objects (e.g., four-leaf clover) and generally combined them in a cumulative-additive manner. Taken together, these findings corroborate historical evidence that humans exploit correlates of number in the external environment - such as shape, configural cues, or 1-to-1 correspondence - as the basis for innovating more abstract number representations.

  View details for DOI 10.1016/j.cognition.2023.105665

  View details for PubMedID 37992512

• Developmental changes in drawing production under different memory demands in a U.S. and Chinese sample. Developmental psychology Long, B., Wang, Y., Christie, S., Frank, M. C., Fan, J. E. 2023; 59 (10): 1784-1793

  Abstract

  Children's drawings of common object categories become dramatically more recognizable across childhood. What are the major factors that drive developmental changes in children's drawings? To what degree are children's drawings a product of their changing internal category representations versus limited by their visuomotor abilities or their ability to recall the relevant visual information? To explore these questions, we examined the degree to which developmental changes in drawing recognizability vary across different drawing tasks that vary in memory demands (i.e., drawing from verbal vs. picture cues) and with children's shape-tracing abilities across two geographical locations (San Jose, United States, and Beijing, China). We collected digital shape tracings and drawings of common object categories (e.g., cat, airplane) from 4- to 9-year-olds (N = 253). The developmental trajectory of drawing recognizability was remarkably similar when children were asked to draw from pictures versus verbal cues and across these two geographical locations. In addition, our Beijing sample produced more recognizable drawings but showed similar tracing abilities to children from San Jose. Overall, this work suggests that the developmental trajectory of children's drawings is remarkably consistent and not easily explainable by changes in visuomotor control or working memory; instead, changes in children's drawings over development may at least partly reflect changes in the internal representations of object categories. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

  View details for DOI 10.1037/dev0001600

  View details for PubMedID 37768614

• Drawing as a versatile cognitive tool NATURE REVIEWS PSYCHOLOGY Fan, J. E., Bainbridge, W. A., Chamberlain, R., Wammes, J. D. 2023; 2 (9): 556-568

  View details for DOI 10.1038/s44159-023-00212-w

  View details for Web of Science ID 001124858600008

• Drawing as a versatile cognitive tool. Nature reviews psychology Fan, J. E., Bainbridge, W. A., Chamberlain, R., Wammes, J. D. 2023; 2 (9): 556-568

  Abstract

  Drawing is a cognitive tool that makes the invisible contents of mental life visible. Humans use this tool to produce a remarkable variety of pictures, from realistic portraits to schematic diagrams. Despite this variety and the prevalence of drawn images, the psychological mechanisms that enable drawings to be so versatile have yet to be fully explored. In this Review, we synthesize contemporary work in multiple areas of psychology, computer science and neuroscience that examines the cognitive processes involved in drawing production and comprehension. This body of findings suggests that the balance of contributions from perception, memory and social inference during drawing production varies depending on the situation, resulting in some drawings that are more realistic and other drawings that are more abstract. We also consider the use of drawings as a research tool for investigating various aspects of cognition, as well as the role that drawing has in facilitating learning and communication. Taken together, information about how drawings are used in different contexts illuminates the central role of visually grounded abstractions in human thought and behaviour.

  View details for DOI 10.1038/s44159-023-00212-w

  View details for PubMedID 39239312

  View details for PubMedCentralID PMC11377027

• Socially intelligent machines that learn from humans and help humans learn. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences Gweon, H., Fan, J., Kim, B. 2023; 381 (2251): 20220048

  Abstract

  A hallmark of human intelligence is the ability to understand and influence other minds. Humans engage in inferential social learning (ISL) by using commonsense psychology to learn from others and help others learn. Recent advances in artificial intelligence (AI) are raising new questions about the feasibility of human-machine interactions that support such powerful modes of social learning. Here, we envision what it means to develop socially intelligent machines that can learn, teach, and communicate in ways that are characteristic of ISL. Rather than machines that simply predict human behaviours or recapitulate superficial aspects of human sociality (e.g. smiling, imitating), we should aim to build machines that can learn from human inputs and generate outputs for humans by proactively considering human values, intentions and beliefs. While such machines can inspire next-generation AI systems that learn more effectively from humans (as learners) and even help humans acquire new knowledge (as teachers), achieving these goals will also require scientific studies of its counterpart: how humans reason about machine minds and behaviours. We close by discussing the need for closer collaborations between the AI/ML and cognitive science communities to advance a science of both natural and artificial intelligence. This article is part of a discussion meeting issue 'Cognitive artificial intelligence'.

  View details for DOI 10.1098/rsta.2022.0048

  View details for PubMedID 37271177

• Visual resemblance and interaction history jointly constrain pictorial meaning. Nature communications Hawkins, R. D., Sano, M., Goodman, N. D., Fan, J. E. 2023; 14 (1): 2199

  Abstract

  How do drawings-ranging from detailed illustrations to schematic diagrams-reliably convey meaning? Do viewers understand drawings based on how strongly they resemble an entity (i.e., as images) or based on socially mediated conventions (i.e., as symbols)? Here we evaluate a cognitive account of pictorial meaning in which visual and social information jointly support visual communication. Pairs of participants used drawings to repeatedly communicate the identity of a target object among multiple distractor objects. We manipulated social cues across three experiments and a full replication, finding that participants developed object-specific and interaction-specific strategies for communicating more efficiently over time, beyond what task practice or a resemblance-based account alone could explain. Leveraging model-based image analyses and crowdsourced annotations, we further determined that drawings did not drift toward "arbitrariness," as predicted by a pure convention-based account, but preserved visually diagnostic features. Taken together, these findings advance psychological theories of how successful graphical conventions emerge.

  View details for DOI 10.1038/s41467-023-37737-w

  View details for PubMedID 37069160

  View details for PubMedCentralID 7060673

• Physion plus plus : Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties Tung, H., Ding, M., Chen, Z., Bear, D. M., Gan, C., Tenenbaum, J. B., Yamins, D. L. K., Fan, J., Smith, K. A. edited by Oh, A., Neumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2023

  View details for Web of Science ID 001227224008004

• SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction Mukherjee, K., Huey, H., Lu, X., Vinker, Y., Aguina-Kang, R., Shamir, A., Fan, J. E. edited by Oh, A., Neumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2023

  View details for Web of Science ID 001228825108009

• Common Object Representations for Visual Production and Recognition COGNITIVE SCIENCE Fan, J. E., Yamins, D. L. K., Turk-Browne, N. B. 2018; 42 (8): 2670-2698

  View details for DOI 10.1111/cogs.12676

  View details for Web of Science ID 000453527500009

• Improving analytical reasoning and argument understanding: a quasi-experimental field study of argument visualization. NPJ science of learning Cullen, S., Fan, J., van der Brugge, E., Elga, A. 2018; 3: 21

  Abstract

  The ability to analyze arguments is critical for higher-level reasoning, yet previous research suggests that standard university education provides only modest improvements in students' analytical-reasoning abilities. What pedagogical approaches are most effective for cultivating these skills? We investigated the effectiveness of a 12-week undergraduate seminar in which students practiced a software-based technique for visualizing the logical structures implicit in argumentative texts. Seminar students met weekly to analyze excerpts from contemporary analytic philosophy papers, completed argument visualization problem sets, and received individualized feedback on a weekly basis. We found that seminar students improved substantially more on LSAT Logical Reasoning test forms than did control students (d=0.71, 95% CI: [0.37, 1.04], p<0.001), suggesting that learning how to visualize arguments in the seminar led to large generalized improvements in students' analytical-reasoning skills. Moreover, blind scoring of final essays from seminar students and control students, drawn from a parallel lecture course, revealed large differences in favor of seminar students (d=0.87, 95% CI: [0.26, 1.48], p=0.005). Seminar students understood the arguments better, and their essays were more accurate and effectively structured. Taken together, these findings deepen our understanding of how visualizations support logical reasoning and provide a model for improving analytical-reasoning pedagogy.

  View details for PubMedID 30631482