Sean Follmer
Associate Professor of Mechanical Engineering and, by courtesy, of Computer Science
Bio
Sean Follmer is an Associate Professor of Mechanical Engineering and Computer Science (by courtesy) at Stanford University. His Research in Human Computer Interaction, Haptics, and Human Robot Interaction explores the design of novel tactile physical interfaces and novel robotic devices. Dr. Follmer directs the Stanford Shape Lab and is a faculty member of the Stanford HCI Group. He is a core faculty member of the Design masters program focusing on innovation and human centered design at Stanford.
The Shape lab explores how we can interact with digital information in a more physical and tangible way. Towards our goal of more human centered computing, we believe that interaction must be grounded in the physical world and leverage our innate abilities for spatial cognition and dexterous manipulation with our hands. We develop advanced technologies in robotics, mechatronics, and sensing to create interactive, dynamic physical 3D displays and haptic interfaces that allow 3D information to be touched as well as seen. We are specifically interested in using these novel interfaces to support richer remote collaboration, computer aided design, education, and interfaces for people with visual impairments. In pursuit of these goals, we use a design process grounded in iterative prototyping and human centered design and look to create new understanding about human perception and interaction through controlled studies.
Our research in Human Computer Interaction and Human Machine Interaction currently directed the following areas:
- Shape Changing and Tangible User Interfaces
- Haptic Interaction
- Accessible User Interfaces for People who Are Blind and Visually Impaired
- Shape Changing Robotics
- Design and Debugging Tools for Physical Computing and Robotic Systems
Dr. Follmer received a PhD and a Masters from the MIT Media Lab in 2015 and 2011 (respectively) for his work in human-computer interaction, and a BS in Engineering with a focus on Product Design from Stanford University. His talk featured on TED.com was named one of the best science and tech TED talks of 2015 and has been viewed more than 1.5 million times. He has received numerous awards for his research and design work such as an Alfred P. Sloan Fellowship, NSF CAREER Award, Google Faculty Research Award, 17 Best Paper Awards and nominations from premier conferences in human-computer interaction (including Five Best papers at ACM UIST, Two Best Papers at ACM CHI and an IMWUT Distinguished Paper Award), Fast Company Innovation By Design Award, Red Dot Design Award, and a Laval Virtual Award. His work has been shown at the Smithsonian Cooper Hewitt Design Museum, Ars Electronica Center, and the Milan Design Week.
Academic Appointments
-
Associate Professor, Mechanical Engineering
-
Associate Professor (By courtesy), Computer Science
-
Member, Bio-X
-
Faculty Affiliate, Institute for Human-Centered Artificial Intelligence (HAI)
-
Member, Wu Tsai Neurosciences Institute
Honors & Awards
-
Best Paper Award, ACM CHI 2023 (2023)
-
NSF CAREER Award, National Science Foundation (2022)
-
Best Paper Award, ACM CHI 2021 (2021)
-
Sloan Research Fellowship, Alfred P. Sloan Foundation (2021)
-
Best Short Paper Award, ACM VRST (2019)
-
Distinguished Paper Award, ACM IMWUT Volume 2 (2019)
-
Best Paper Award, ACM UIST 2017 (2017)
-
Google Faculty Research Award, Google (2017)
-
Best Demo Award, ACM UIST 2016 (2016)
-
Best Paper Award (x2), ACM UIST 2016 (2016)
-
Google Faculty Research Award, Google (2016)
-
Best Paper Award, ACM UIST 2013 (2013)
-
Best Paper Award, ACM UIST 2012 (2012)
Program Affiliations
-
Symbolic Systems Program
Professional Education
-
Postdoctoral Associate, MIT Media Lab (2015)
-
PhD, MIT Media Lab (2015)
-
S.M., MIT Media Lab (2011)
Current Research and Scholarly Interests
Human Computer Interaction, Haptics, Robotics, Human Centered Design
2024-25 Courses
-
Independent Studies (24)
- Advanced Reading and Research
CS 499 (Aut, Win, Spr, Sum) - Advanced Reading and Research
CS 499P (Aut, Win, Spr, Sum) - Curricular Practical Training
CS 390A (Aut, Win, Spr, Sum) - Curricular Practical Training
CS 390B (Aut, Win, Spr, Sum) - Curricular Practical Training
CS 390C (Aut, Win, Spr, Sum) - Curricular Practical Training
DESIGN 191A (Aut, Win, Spr, Sum) - Curricular Practical Training
DESIGN 391A (Aut, Win, Spr, Sum) - Engineering Problems
ME 391 (Aut, Win, Spr, Sum) - Engineering Problems and Experimental Investigation
ME 191 (Aut, Win, Spr, Sum) - Experimental Investigation of Engineering Problems
ME 392 (Aut, Win, Spr, Sum) - Graduate Independent Study and Research
DESIGN 391 (Aut, Win, Spr) - Honors Research
ME 191H (Aut, Win, Spr, Sum) - Independent Study and Research
DESIGN 191 (Aut, Win, Spr) - Master's Directed Research
ME 393 (Aut, Win, Spr, Sum) - Master's Directed Research: Writing the Report
ME 393W (Aut, Win, Spr, Sum) - Part-time Curricular Practical Training
CS 390D (Aut, Win, Spr, Sum) - Ph.D. Research Rotation
ME 398 (Aut, Win, Spr, Sum) - Ph.D. Teaching Experience
ME 491 (Aut, Win, Spr) - Practical Training
ME 199A (Win, Spr) - Practical Training
ME 299A (Aut, Win, Spr, Sum) - Practical Training
ME 299B (Aut, Win, Spr, Sum) - Senior Honors Tutorial
SYMSYS 190 (Aut, Win, Spr, Sum) - Senior Project
CS 191 (Aut, Win, Spr, Sum) - Supervised Undergraduate Research
CS 195 (Aut, Win, Spr, Sum)
- Advanced Reading and Research
-
Prior Year Courses
2023-24 Courses
- MS Design Capstone Project 2
DESIGN 361B (Win) - MS Design Capstone Project 3
DESIGN 361C (Spr) - Product Design Methods
DESIGN 141 (Win)
2022-23 Courses
- MS Design Capstone Project 1
DESIGN 361A, ME 316A (Aut) - MS Design Capstone Project 2
DESIGN 361B, ME 316B (Win) - MS Design Capstone Project 3
DESIGN 361C, ME 316C (Spr) - Product Design Methods
DESIGN 141, ME 115B (Win)
2021-22 Courses
- Human-Computer Interaction Seminar
CS 547 (Aut) - Product Design Methods
ME 115B (Win)
- MS Design Capstone Project 2
Stanford Advisees
-
Doctoral Dissertation Reader (AC)
Kai Jun Chen, Gabriel Lipkowitz, Aya Mouallem, Shuai Wu -
Orals Chair
Jackie Yang -
Doctoral Dissertation Advisor (AC)
Savannah Cofer, Dan Fan, Saehui Hwang, Wing-Sum Law, Ahad Rauf, Olivia Tomassetti, Elizabeth Vasquez -
Master's Program Advisor
Veronica Chen, Gatsby Frimpong, Francis Santiago, Laura Segura, Izma Shabbir, Mihret Tamrat, Kavya Udupa, Aayush Wadehra -
Doctoral Dissertation Co-Advisor (AC)
Eunyoung Kim, Alessandra Napoli -
Doctoral (Program)
Alessandra Napoli, Yujie Tao, Sofia Wyetzner
All Publications
-
The Accessibility of Data Visualizations on theWeb for Screen Reader Users: Practices and Experiences During COVID-19
ACM TRANSACTIONS ON ACCESSIBLE COMPUTING
2023; 16 (1)
View details for DOI 10.1145/3557899
View details for Web of Science ID 000963968700003
-
Detecting Touch and Grasp Gestures Using a Wrist-Worn Optical and Inertial Sensing Network
IEEE ROBOTICS AND AUTOMATION LETTERS
2022; 7 (4): 10842-10849
View details for DOI 10.1109/LRA.2022.3191173
View details for Web of Science ID 000844135200001
-
An All-Soft Variable Impedance Actuator Enabled by Embedded Layer Jamming
IEEE-ASME TRANSACTIONS ON MECHATRONICS
2022
View details for DOI 10.1109/TMECH.2022.3183576
View details for Web of Science ID 000824729500001
-
Robotic Presence: The Effects of Anthropomorphism and Robot State on Task Performance and Emotion
IEEE ROBOTICS AND AUTOMATION LETTERS
2022; 7 (3): 7399-7406
View details for DOI 10.1109/LRA.2022.3181726
View details for Web of Science ID 000815662100005
-
Beyond Being Real: A Sensorimotor Control Perspective on Interactions in Virtual Reality
ASSOC COMPUTING MACHINERY. 2022
View details for DOI 10.1145/3491102.3517706
View details for Web of Science ID 000922929504051
-
A Model Predictive Control Approach for Reach Redirection in Virtual Reality
ASSOC COMPUTING MACHINERY. 2022
View details for DOI 10.1145/3491102.3501907
View details for Web of Science ID 000890212501037
-
Slide-Tone and Tilt-Tone: 1-DOF Haptic Techniques for Conveying Shape Characteristics of Graphs to Blind Users
ASSOC COMPUTING MACHINERY. 2022
View details for DOI 10.1145/3491102.3517790
View details for Web of Science ID 000922929505030
-
Supporting Accessible Data Visualization Through Audio Data Narratives
ASSOC COMPUTING MACHINERY. 2022
View details for DOI 10.1145/3491102.3517678
View details for Web of Science ID 000922929504025
-
Augmenting Perceived Softness of Haptic Proxy Objects Through Transient Vibration and Visuo-Haptic Illusion in Virtual Reality
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
2021; 27 (12): 4387-4400
Abstract
In this article, we investigate the effects of active transient vibration and visuo-haptic illusion to augment the perceived softness of haptic proxy objects. We introduce a system combining active transient vibration at the fingertip with visuo-haptic illusions. In our hand-held device, a voice coil actuator transmits active transient vibrations to the index fingertip, while a force sensor measures the force applied on passive proxy objects to create visuo-haptic illusions in virtual reality. We conducted three user studies to understand both the vibrotactile effect and its combined effect with visuo-haptic illusions. A preliminary study confirmed that active transient vibrations can intuitively alter the perceived softness of a proxy object. Our first study demonstrated that those same active transient vibrations can generate different perceptions of softness depending on the material of the proxy object used. In our second study, we evaluated the combination of active transient vibration and visuo-haptic illusion, and found that both significantly influence perceived softness, with with the visuo-haptic effect being dominant. Our third study further investigated the vibrotactile effect while controlling for the visuo-haptic illusion. The combination of these two methods allows users to effectively perceive various levels of softness when interacting with haptic proxy objects.
View details for DOI 10.1109/TVCG.2020.3002245
View details for Web of Science ID 000711642800005
View details for PubMedID 32746263
-
Generating Legible and Glanceable Swarm Robot Motion through Trajectory, Collective Behavior, and Pre-attentive Processing Features
ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION
2021; 10 (3)
View details for DOI 10.1145/3442681
View details for Web of Science ID 000731456900003
-
Hybrid Actuation With Unidirectional Clutches for Handheld Haptic Controllers
IEEE ROBOTICS AND AUTOMATION LETTERS
2021; 6 (3): 4827-4834
View details for DOI 10.1109/LRA.2021.3068700
View details for Web of Science ID 000641958700005
-
COVID-19 highlights the issues facing blind and visually impaired people in accessing data on the web
W4A: Web Accessibility
2021: 1-15
View details for DOI 10.1145/3430263.3452432
-
Balloon Animal Robots: Reconfigurable Isoperimetric Inflated Soft Robots
IEEE. 2021: 6941-6947
View details for DOI 10.1109/IROS51168.2021.9635842
View details for Web of Science ID 000755125505082
-
Grasp Analysis and Manipulation Kinematics for Isoperimetric Truss Robots
IEEE. 2021: 6140-6146
View details for DOI 10.1109/ICRA48506.2021.9561332
View details for Web of Science ID 000765738804084
-
Acoustic Communication and Sensing for Inflatable Modular Soft Robots
IEEE. 2021: 11827-11833
View details for DOI 10.1109/ICRA48506.2021.9561183
View details for Web of Science ID 000771405404018
-
A Causal Feeling: How Kinesthetic Haptics Affects Causal Perception
IEEE. 2021: 347
View details for DOI 10.1109/WHC49131.2021.9517133
View details for Web of Science ID 000707066600035
-
HIGH FORCE DENSITY MULTI-STAGE ELECTROHYDRODYNAMIC JETS USING FOLDED LASER MICROFABRICATED ELECTRODES
IEEE. 2021: 54-57
View details for DOI 10.1109/TRANSDUCERS50396.2021.9495704
View details for Web of Science ID 000707041600016
-
Lightweight High Voltage Generator for Untethered Electroadhesive Perching of Micro Air Vehicles
IEEE ROBOTICS AND AUTOMATION LETTERS
2020; 5 (3): 4485–92
View details for DOI 10.1109/LRA.2020.3001520
View details for Web of Science ID 000542879400005
-
An untethered isoperimetric soft robot.
Science robotics
2020; 5 (40)
Abstract
For robots to be useful for real-world applications, they must be safe around humans, be adaptable to their environment, and operate in an untethered manner. Soft robots could potentially meet these requirements; however, existing soft robotic architectures are limited by their ability to scale to human sizes and operate at these scales without a tether to transmit power or pressurized air from an external source. Here, we report an untethered, inflated robotic truss, composed of thin-walled inflatable tubes, capable of shape change by continuously relocating its joints, while its total edge length remains constant. Specifically, a set of identical roller modules each pinch the tube to create an effective joint that separates two edges, and modules can be connected to form complex structures. Driving a roller module along a tube changes the overall shape, lengthening one edge and shortening another, while the total edge length and hence fluid volume remain constant. This isoperimetric behavior allows the robot to operate without compressing air or requiring a tether. Our concept brings together advantages from three distinct types of robots-soft, collective, and truss-based-while overcoming certain limitations of each. Our robots are robust and safe, like soft robots, but not limited by a tether; are modular, like collective robots, but not limited by complex subunits; and are shape-changing, like truss robots, but not limited by rigid linear actuators. We demonstrate two-dimensional (2D) robots capable of shape change and a human-scale 3D robot capable of punctuated rolling locomotion and manipulation, all constructed with the same modular rollers and operating without a tether.
View details for DOI 10.1126/scirobotics.aaz0492
View details for PubMedID 33022597
-
An untethered isoperimetric soft robot
SCIENCE ROBOTICS
2020; 5 (40)
View details for DOI 10.1126/scirobotics.aaz0492
View details for Web of Science ID 000520870400001
-
Foxels: Build Your Own Smart Furniture
ASSOC COMPUTING MACHINERY. 2020: 111–22
View details for DOI 10.1145/3374920.3374935
View details for Web of Science ID 000570009800011
-
User-Defined Swarm Robot Control
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Association for Computing Machinery. 2020: 13
View details for DOI 10.1145/3313831.3376814
-
Design and Analysis of High-Resolution Electrostatic Adhesive Brakes Towards Static Refreshable 2.5D Tactile Shape Display
IEEE TRANSACTIONS ON HAPTICS
2019; 12 (4): 470–82
Abstract
Tactile displays are haptic devices capable of rendering shape and texture information. Unsolved challenges in building tactile shape displays include their traditionally large form factors, low spatial resolution, and high costs. Using electrostatic adhesion to individually brake each pin and a single platform for global actuation, we developed a prototype static refreshable tactile shape display with high spatial resolution (1.7 mm pitch, 0.8 mm pin width; 4 mm pitch, 1.6 mm pin width), high resistance force (76.3 gf static-loading force per pin for 1.6 mm width) and low cost ($0.11 USD per pin for raw material). We present an analytical model of our electroadhesive brake mechanism and evaluate its maximum contact force and robustness in various conditions. To demonstrate the mechanism's potential, we built a static tactile shape display prototype with a 4×2 array of pins controlled using electroadhesive brakes. To further increase maximsum contact force allowed by our device, we develop and evaluate a global mechanical clutch which can be engaged during user interaction. A user study is carried out to compare our static tactile shape display's performance with printed 2.5D tactile graphics in a shape recognition task, and comparable shape recognition rates and response times are observed.
View details for DOI 10.1109/TOH.2019.2940219
View details for Web of Science ID 000505585900008
View details for PubMedID 31545743
-
Beyond The Force: Using Quadcopters to Appropriate Objects and the Environment for Haptics in Virtual Reality
ASSOC COMPUTING MACHINERY. 2019
View details for DOI 10.1145/3290605.3300589
View details for Web of Science ID 000474467904051
-
Investigating the Detection of Bimanual Haptic Retargeting in Virtual Reality
ASSOC COMPUTING MACHINERY. 2019
View details for DOI 10.1145/3359996.3364248
View details for Web of Science ID 000527364800012
-
shapeCAD: An Accessible 3D Modelling Workflow for the Blind and Visually-Impaired Via 2.5D Shape Displays
ASSOC COMPUTING MACHINERY. 2019: 342–54
View details for DOI 10.1145/3308561.3353782
View details for Web of Science ID 000522450600031
-
Tactile Code Skimmer: A Tool to Help Blind Programmers Feel the Structure of Code
ASSOC COMPUTING MACHINERY. 2019: 536–38
View details for DOI 10.1145/3308561.3354616
View details for Web of Science ID 000522450600055
-
Evaluating the Minimum Jerk Motion Model for Redirected Reach in Virtual Reality
ASSOC COMPUTING MACHINERY. 2019: 4–6
View details for DOI 10.1145/3332167.3357096
View details for Web of Science ID 000518192300002
-
Editing Spatial Layouts through Tactile Templates for People with Visual Impairments
ASSOC COMPUTING MACHINERY. 2019
View details for DOI 10.1145/3290605.3300436
View details for Web of Science ID 000474467902057
-
Pinpoint: A PCB Debugging Pipeline Using Interruptible Routing and Instrumentation
ASSOC COMPUTING MACHINERY. 2019
View details for DOI 10.1145/3290605.3300278
View details for Web of Science ID 000474467900048
-
SwarmHaptics: Haptic Display with Swarm Robots
ASSOC COMPUTING MACHINERY. 2019
View details for DOI 10.1145/3290605.3300918
View details for Web of Science ID 000474467908067
-
Dynamic Composite Data Physicalization Using Wheeled Micro-Robots.
IEEE transactions on visualization and computer graphics
2018
Abstract
This paper introduces dynamic composite physicalizations, a new class of physical visualizations that use collections of self-propelled objects to represent data. Dynamic composite physicalizations can be used both to give physical form to well-known interactive visualization techniques, and to explore new visualizations and interaction paradigms. We first propose a design space characterizing composite physicalizations based on previous work in the fields of Information Visualization and Human Computer Interaction. We illustrate dynamic composite physicalizations in two scenarios demonstrating potential benefits for collaboration and decision making, as well as new opportunities for physical interaction. We then describe our implementation using wheeled micro-robots capable of locating themselves and sensing user input, before discussing limitations and opportunities for future work.
View details for DOI 10.1109/TVCG.2018.2865159
View details for PubMedID 30136993
-
Electrostatic Adhesive Brakes for High Spatial Resolution Refreshable 2.5D Tactile Shape Displays
IEEE. 2018: 319–26
View details for Web of Science ID 000434958700052
-
An Accessible CAD Workflow Using Programming of 3D Models and Preview Rendering in A 2.5D Shape Display
ASSOC COMPUTING MACHINERY. 2018: 343–45
View details for DOI 10.1145/3234695.3240996
View details for Web of Science ID 000455178500034
-
Investigating Tangible Collaboration for Design Towards Augmented Physical Telepresence
DESIGN THINKING RESEARCH: MAKING DISTINCTIONS: COLLABORATION VERSUS COOPERATION
2018: 131–45
View details for DOI 10.1007/978-3-319-60967-6_7
View details for Web of Science ID 000432741300008
-
Designing Line-Based Shape-Changing Interfaces
IEEE PERVASIVE COMPUTING
2017; 16 (4): 36–46
View details for Web of Science ID 000414423400008
-
shiftIO: Reconfigurable Tactile Elements for Dynamic Affordances and Mobile Interaction
ASSOC COMPUTING MACHINERY. 2017: 5075–86
View details for DOI 10.1145/3025453.3025988
View details for Web of Science ID 000426970504081
-
Shape Displays: Spatial Interaction with Dynamic Physical Form
IEEE COMPUTER GRAPHICS AND APPLICATIONS
2015; 35 (5): 5-11
View details for Web of Science ID 000361969200002
View details for PubMedID 26416359
-
Jamming User Interfaces: Programmable Particle Stiffness and Sensing for Malleable and Shape-Changing Devices
UIST'12: PROCEEDINGS OF THE 25TH ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY
2012: 519-528
View details for Web of Science ID 000324815300054
-
TessalTable: Tile-based Creation of Patterns and Images
4th International Conference on Tangible, Embedded and Embodies Interaction
ASSOC COMPUTING MACHINERY. 2010: 203–204
View details for Web of Science ID 000285034500031
-
d.note: Revising User Interfaces Through Change Tracking, Annotations, and Alternatives
28th Annual CHI Conference on Human Factors in Computing Systems
ASSOC COMPUTING MACHINERY. 2010: 493–502
View details for Web of Science ID 000281276700055
-
Family Story Play: Reading with Young Children (and Elmo) Over a Distance
CHI2010: PROCEEDINGS OF THE 28TH ANNUAL CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, VOLS 1-4
2010: 1583-1592
View details for Web of Science ID 000281276701020