The Idiosyncrasies of Everyday Digital Lives: Using the Human Screenome Project to Study User Behavior on Smartphones.
Computers in human behavior
Most methods used to make theory-relevant observations of technology use rely on self-report or application logging data where individuals' digital experiences are purposively summarized into aggregates meant to describe how the average individual engages with broadly defined segments of content. This aggregation and averaging masks heterogeneity in how and when individuals actually engage with their technology. In this study, we use screenshots (N > 6 million) collected every five seconds that were sequenced and processed using text and image extraction tools into content-, context-, and temporally-informative "screenomes" from 132 smartphone users over several weeks to examine individuals' digital experiences. Analyses of screenomes highlight extreme between-person and within-person heterogeneity in how individuals switch among and titrate their engagement with different content. Our simple quantifications of textual and graphical content and flow throughout the day illustrate the value screenomes have for the study of individuals' smartphone use and the cognitive and psychological processes that drive use. We demonstrate how temporal, textual, graphical, and topical features of people's smartphone screens can lay the foundation for expanding the Human Screenome Project with full-scale mining that will inform researchers' knowledge of digital life.
View details for DOI 10.1016/j.chb.2020.106570
View details for PubMedID 33041494
View details for PubMedCentralID PMC7543997
Screenomics: A New Approach for Observing and Studying Individuals' Digital Lives.
Journal of adolescent research
2020; 35 (1): 16–50
This study describes when and how adolescents engage with their fast-moving and dynamic digital environment as they go about their daily lives. We illustrate a new approach - screenomics - for capturing, visualizing, and analyzing screenomes, the record of individuals' day-to-day digital experiences.Over 500,000 smartphone screenshots provided by four Latino/Hispanic youth, age 14-15 years, from low-income, racial/ethnic minority neighborhoods.Screenomes collected from smartphones for one to three months, as sequences of smartphone screenshots obtained every five seconds that the device is activated, are analyzed using computational machinery for processing images and text, machine learning algorithms, human-labeling, and qualitative inquiry.Adolescents' digital lives differ substantially across persons, days, hours, and minutes. Screenomes highlight the extent of switching among multiple applications, and how each adolescent is exposed to different content at different times for different durations - with apps, food-related content, and sentiment as illustrative examples.We propose that the screenome provides the fine granularity of data needed to study individuals' digital lives, for testing existing theories about media use, and for generation of new theory about the interplay between digital media and development.
View details for DOI 10.1177/0743558419883362
View details for PubMedID 32161431
View details for PubMedCentralID PMC7065687
- Describing and Controlling Multivariate Dynamical Systems: A Boolean Network Method MULTIVARIATE BEHAVIORAL RESEARCH 2020; 55 (1): 163–64
Adolescents' Emotion System Dynamics: Network-Based Analysis of Physiological and Emotional Experience
2019; 55 (9): 1982–93
An individual's emotions system can be conceived of as a synchronized, coordinated, and/or emergent combination of physiology, experience, and behavioral components. Together, the interplay among these components produce emotional experiences through coordinated excitatory positive feedback (i.e., the mutual amplification of emotion concordance) and/or inhibitory negative feedback (i.e., the damping of emotion regulation) processes. Different system configurations produce differential psychophysiological reactivity profiles, and by implication, differential moment-to-moment emotional experience and long-term development. Applying dynamic systems models to second-by-second psychophysiological and experience time-series data collected from 130 adolescents (age 12.0 to 16.7 years) completing a social stress-inducing speech task, we describe the configuration of adolescents' emotion systems, and examine how differences in the dynamic outputs of those systems (psychophysiological reactivity profile) are related to individual differences in trait anxiety. We found substantial heterogeneity in the coordination patterns of these adolescents. Some individuals' emotion systems were characterized by negative feedback loops (emotion regulation processes), many by unidirectionally connected or independent components, and a few by positive feedback loops (emotion concordance). The reactivity dynamics of respiratory sinus arrhythmia were related to adolescents' level of trait anxiety. Results highlight how dynamic systems models may contribute to our understanding of interindividual and developmental differences. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
View details for DOI 10.1037/dev0000690
View details for Web of Science ID 000483067100013
View details for PubMedID 31464499
View details for PubMedCentralID PMC6716613
Screenomics: a framework to capture and analyze personal life experiences and the ways that technology shapes them
View details for DOI 10.1080/07370024.2019.1578652
- Using Screenshots to Predict Task Switching on Smartphones ASSOC COMPUTING MACHINERY. 2019
Identification of Mental States and Interpersonal Functioning in Borderline Personality Disorder
PERSONALITY DISORDERS-THEORY RESEARCH AND TREATMENT
2018; 9 (2): 172–81
Atypical identification of mental states in the self and others has been proposed to underlie interpersonal difficulties in borderline personality disorder (BPD), yet no previous empirical research has directly examined associations between these constructs. We examine 3 mental state identification measures and their associations with experience-sampling measures of interpersonal functioning in participants with BPD relative to a healthy comparison (HC) group. We also included a clinical comparison group diagnosed with avoidant personality disorder (APD) to test the specificity of this constellation of difficulties to BPD. When categorizing blended emotional expressions, the BPD group identified anger at a lower threshold than did the HC and APD groups, but no group differences emerged in the threshold for identifying happiness. These results are consistent with enhanced social threat identification and not general negativity biases in BPD. The Reading the Mind in the Eyes Test (RMET) showed no group differences in general mental state identification abilities. Alexithymia scores were higher in both BPD and APD relative to the HC group, and difficulty identifying one's own emotions was higher in BPD compared to APD and HC. Within the BPD group, lower RMET scores were associated with lower anger identification thresholds and higher alexithymia scores. Moreover, lower anger identification thresholds, lower RMET scores, and higher alexithymia scores were all associated with greater levels of interpersonal difficulties in daily life. Research linking measures of mental state identification with experience-sampling measures of interpersonal functioning can help clarify the role of mental state identification in BPD symptoms. (PsycINFO Database Record
View details for DOI 10.1037/per0000228
View details for Web of Science ID 000428433600008
View details for PubMedID 27831693
View details for PubMedCentralID PMC5425332
Socioemotional Dynamics of Emotion Regulation and Depressive Symptoms: A Person-Specific Network Approach
Socioemotional processes engaged in daily life may afford and/or constrain individuals' emotion regulation in ways that affect psychological health. Recent findings from experience sampling studies suggest that persistence of negative emotions (emotion inertia), the strength of relations among an individual's negative emotions (density of the emotion network), and cycles of negative/aggressive interpersonal transactions are related to psychological health. Using multiple bursts of intensive experience sampling data obtained from 150 persons over one year, person-specific analysis, and impulse response analysis, this study quantifies the complex and interconnected socioemotional processes that surround individuals' daily social interactions and on-going regulation of negative emotion in terms of recovery time. We also examine how this measure of regulatory inefficiency is related to interindividual differences and intraindividual change in level of depressive symptoms. Individuals with longer recovery times had higher overall level of depressive symptoms. As well, during periods where recovery time of sadness was longer than usual, individuals' depressive symptoms were also higher than usual, particularly among individuals who experienced higher overall level of stressful life events. The findings and analysis highlight the utility of a person-specific network approach to study emotion regulation, how regulatory processes change over time, and potentially how planned changes in the configuration of individuals' systems may contribute to psychological health.
View details for DOI 10.1155/2018/5094179
View details for Web of Science ID 000450215000001
View details for PubMedID 30613129
View details for PubMedCentralID PMC6319954
- Text Extraction and Retrieval from Smartphone Screenshots: Building a Repository for Life in Media ASSOC COMPUTING MACHINERY. 2018: 948–55
Impulsivity, Rejection Sensitivity, and Reactions to Stressors in Borderline Personality Disorder
COGNITIVE THERAPY AND RESEARCH
2016; 40 (4): 510–21
This research investigated baseline impulsivity, rejection sensitivity, and reactions to stressors in individuals with borderline personality disorder compared to healthy individuals and those with avoidant personality disorder. The borderline group showed greater impulsivity than the avoidant and healthy groups both in a delay-discounting task with real monetary rewards and in self-reported reactions to stressors; moreover, these findings could not be explained by co-occurring substance use disorders. Distress reactions to stressors were equally elevated in both personality disorder groups (relative to the healthy group). The borderline and avoidant groups also reported more maladaptive reactions to a stressor of an interpersonal vs. non-interpersonal nature, whereas the healthy group did not. Finally, self-reported impulsive reactions to stressors were associated with baseline impulsivity in the delay-discounting task, and greater self-reported reactivity to interpersonal than non-interpersonal stressors was associated with rejection sensitivity. This research highlights distinct vulnerabilities contributing to impulsive behavior in borderline personality disorder.
View details for DOI 10.1007/s10608-015-9752-y
View details for Web of Science ID 000380089600006
View details for PubMedID 27616800
View details for PubMedCentralID PMC5015893