The Idiosyncrasies of Everyday Digital Lives: Using the Human Screenome Project to Study User Behavior on Smartphones.
Computers in human behavior
Most methods used to make theory-relevant observations of technology use rely on self-report or application logging data where individuals' digital experiences are purposively summarized into aggregates meant to describe how the average individual engages with broadly defined segments of content. This aggregation and averaging masks heterogeneity in how and when individuals actually engage with their technology. In this study, we use screenshots (N > 6 million) collected every five seconds that were sequenced and processed using text and image extraction tools into content-, context-, and temporally-informative "screenomes" from 132 smartphone users over several weeks to examine individuals' digital experiences. Analyses of screenomes highlight extreme between-person and within-person heterogeneity in how individuals switch among and titrate their engagement with different content. Our simple quantifications of textual and graphical content and flow throughout the day illustrate the value screenomes have for the study of individuals' smartphone use and the cognitive and psychological processes that drive use. We demonstrate how temporal, textual, graphical, and topical features of people's smartphone screens can lay the foundation for expanding the Human Screenome Project with full-scale mining that will inform researchers' knowledge of digital life.
View details for DOI 10.1016/j.chb.2020.106570
View details for PubMedID 33041494
View details for PubMedCentralID PMC7543997
Screenomics: A New Approach for Observing and Studying Individuals' Digital Lives.
Journal of adolescent research
2020; 35 (1): 16–50
This study describes when and how adolescents engage with their fast-moving and dynamic digital environment as they go about their daily lives. We illustrate a new approach - screenomics - for capturing, visualizing, and analyzing screenomes, the record of individuals' day-to-day digital experiences.Over 500,000 smartphone screenshots provided by four Latino/Hispanic youth, age 14-15 years, from low-income, racial/ethnic minority neighborhoods.Screenomes collected from smartphones for one to three months, as sequences of smartphone screenshots obtained every five seconds that the device is activated, are analyzed using computational machinery for processing images and text, machine learning algorithms, human-labeling, and qualitative inquiry.Adolescents' digital lives differ substantially across persons, days, hours, and minutes. Screenomes highlight the extent of switching among multiple applications, and how each adolescent is exposed to different content at different times for different durations - with apps, food-related content, and sentiment as illustrative examples.We propose that the screenome provides the fine granularity of data needed to study individuals' digital lives, for testing existing theories about media use, and for generation of new theory about the interplay between digital media and development.
View details for DOI 10.1177/0743558419883362
View details for PubMedID 32161431
View details for PubMedCentralID PMC7065687
- Screenomics: A New Approach for Observing and Studying Individuals' Digital Lives JOURNAL OF ADOLESCENT RESEARCH 2019
- The Accuracy of Measurements with Probability and Nonprobability Survey Samples: Replication and Extension PUBLIC OPINION QUARTERLY 2018; 82 (4): 707–44
- Text Extraction and Retrieval from Smartphone Screenshots: Building a Repository for Life in Media ASSOC COMPUTING MACHINERY. 2018: 948–55