Divergences in color perception between deep neural networks and humans.
2023; 241: 105621
Deep neural networks (DNNs) are increasingly proposed as models of human vision, bolstered by their impressive performance on image classification and object recognition tasks. Yet, the extent to which DNNs capture fundamental aspects of human vision such as color perception remains unclear. Here, we develop novel experiments for evaluating the perceptual coherence of color embeddings in DNNs, and we assess how well these algorithms predict human color similarity judgments collected via an online survey. We find that state-of-the-art DNN architectures - including convolutional neural networks and vision transformers - provide color similarity judgments that strikingly diverge from human color judgments of (i) images with controlled color properties, (ii) images generated from online searches, and (iii) real-world images from the canonical CIFAR-10 dataset. We compare DNN performance against an interpretable and cognitively plausible model of color perception based on wavelet decomposition, inspired by foundational theories in computational neuroscience. While one deep learning model - a convolutional DNN trained on a style transfer task - captures some aspects of human color perception, our wavelet algorithm provides more coherent color embeddings that better predict human color judgments compared to all DNNs we examine. These results hold when altering the high-level visual task used to train similar DNN architectures (e.g., image classification versus image segmentation), as well as when examining the color embeddings of different layers in a given DNN architecture. These findings break new ground in the effort to analyze the perceptual representations of machine learning algorithms and to improve their ability to serve as cognitively plausible models of human vision. Implications for machine learning, human perception, and embodied cognition are discussed.
View details for DOI 10.1016/j.cognition.2023.105621
View details for PubMedID 37716312
- Target Selection and Sample Characterization for the DESI LOW-Z Secondary Target Program ASTROPHYSICAL JOURNAL 2023; 954 (2)
- <monospace>ESCARGOT</monospace>: Mapping Vertical Phase Spiral Characteristics Throughout the Real and Simulated Milky Way ASTROPHYSICAL JOURNAL 2023; 955 (1)
- Six More Ultra-faint Milky Way Companions Discovered in the DECam Local Volume Exploration Survey ASTROPHYSICAL JOURNAL 2023; 953 (1)
- The Concentration-Mass relation of massive, dynamically relaxed galaxy clusters: agreement between observations and ?CDM simulations MONTHLY NOTICES OF THE ROYAL ASTRONOMICAL SOCIETY 2023; 521 (1): 790-799
- Symphony: Cosmological Zoom-in Simulation Suites over Four Decades of Host Halo Mass ASTROPHYSICAL JOURNAL 2023; 945 (2)
In Their Own Words: Sexual Assault Resistance Strategies Among Kenyan Adolescent Girls Following Participation in an Empowerment Self-Defense Program.
Violence against women
The purpose of this study was to examine, via testimonial data, resistance strategies used to thwart a sexual assault among slum-dwelling Kenyan adolescent girls (N=678) following their participation in an empowerment self-defense program (IMpower). A subset of girls from the larger trials participated. The majority (58.2%) of perpetrators were strangers; there were no differences in resistance strategies used between strangers versus known perpetrators (83.8% used verbal strategies, 33.2% used resistance strategies, 16.7% ran away, and 7.9% used distraction). Associations between resistance strategies and perpetrator tactics, number of assailants, location of the assault, and the presence of a bystander were also examined.
View details for DOI 10.1177/10778012231153360
View details for PubMedID 36710565
- From Images to Dark Matter: End-to-end Inference of Substructure from Hundreds of Strong Gravitational Lenses ASTROPHYSICAL JOURNAL 2023; 942 (2)
- Multiple phase spirals suggest multiple origins in Gaia DR3 MONTHLY NOTICES OF THE ROYAL ASTRONOMICAL SOCIETY 2022; 516 (1): L7-L11
Signal in Noise: Exploring Meaning Encoded in Random Character Sequences with Character-Aware Language Models
ASSOC COMPUTATIONAL LINGUISTICS-ACL. 2022: 7120-7134
View details for Web of Science ID 000828702307019
- Searching for Dwarf Galaxies in Gaia DR2 Phase-space Data Using Wavelet Transforms ASTROPHYSICAL JOURNAL 2021; 915 (1)