Allison Okamura
Richard W. Weiland Professor in the School of Engineering and Professor of Mechanical Engineering
Bio
Allison Okamura is the Richard W. Weiland Professor of Engineering at Stanford University in the Department of Mechanical Engineering, with a courtesy appointment in Computer Science. She is Director of Graduate Studies for Stanford Mechanical Engineering, Deputy Director of the Wu Tsai Neurosciences Institute, Science Fellow of the Hoover Institution, and Executive Committee Member of the Stanford Robotics Center. She is an IEEE Fellow and was previously editor-in-chief of the journal IEEE Robotics and Automation Letters. Her awards include the IEEE Engineering in Medicine and Biology Society Technical Achievement Award, IEEE Robotics and Automation Society Distinguished Service Award, and Duca Family University Fellow in Undergraduate Education. She received her BS degree from the University of California at Berkeley, and PhD degree from Stanford. Her interests include haptics, teleoperation, mixed reality, medical robotics, soft robotics, and rehabilitation. Outside academia, she enjoys spending time with her family, running, and playing ice hockey.
Academic Appointments
-
Professor, Mechanical Engineering
-
Member, Bio-X
-
Faculty Affiliate, Institute for Human-Centered Artificial Intelligence (HAI)
-
Member, Wu Tsai Neurosciences Institute
Honors & Awards
-
Distinguished Service Award, IEEE Robotics and Automation Society (2019)
-
Stanford Fellow, Stanford University (2018-2020)
-
Best Student Presentation Award (student author: Sean Sketch), IEEE Haptics Symposium (2018)
-
Finalist, Best Paper in Human Robot Interaction, IEEE International Conference on Robotics and Automation (2018)
-
Gilbreth Lecturer, National Academy of Engineers (2018)
-
Tau Beta Pi Teaching Honor Roll, Stanford University (2018)
-
Ten Robotics Technologies of the Year (Soft robot that navigates through growth), Science Robotics (2018)
-
ACM CHI Best Paper Award, ACM SIGCHI 2017 conference (2017)
-
Society of Scholars, Johns Hopkins University (2017)
-
Duca Family University Fellow in Undergraduate Education, Stanford University (2016)
-
Best Student Paper Award (student author: Zhan Fan Quek), IEEE Haptics Symposium (2014)
-
Robert Bosch Faculty Scholar, Stanford University (2011-2015)
-
Gabilan Fellow, Stanford University (2011)
-
IEEE Fellow, Institute of Electrical and Electronics Engineers (2010)
-
Early Career Award, IEEE Technical Committee on Haptics (2009)
-
Alumni Distinguished Scholar, Stanford University (2008)
-
Decker Faculty Scholar, Johns Hopkins University (2007-2010)
-
Award for Excellence, Outstanding Paper of the Year in Industrial Robot, Literati Club (2005)
-
Early Academic Career Award, IEEE Robotics and Automation Society (2005)
-
CAREER Award, National Science Foundation (2004-2009)
-
Diversity Recognition Award, Johns Hopkins University (2003)
Professional Education
-
BS, University of California, Berkeley, Mechanical Engineering (1994)
-
MS, Stanford University, Mechanical Engineering (1996)
-
PhD, Stanford University, Mechanical Engineering (2000)
Current Research and Scholarly Interests
My research focuses on developing the principles and tools needed to realize advanced robotic and human-machine systems capable of physical interaction. Topics of particular interest are: (1) Teleoperation: Devices, models, and control systems that allow human operators to manipulate environments that are remote in scale and/or distance. (2) Haptic systems: Devices, models, and control systems that enable compelling touch-based interaction with virtual environments, computers, and remote robots. (3) Robotic manipulation: Robots that physically manipulate their environment or their own shape, incorporating novel designs, sensors, and control systems. Application areas include surgery, simulation and training, rehabilitation, prosthetics, neuromechanics, exploration of hazardous and remote environments (e.g. space), design, and education.
Clinical Trials
-
Effects of a Compliant Arm Support on Post-stroke Upper Extremity Range of Motion
Not Recruiting
The aim of this study is to show that a wearable compliant arm support consisting of inflatable bladders with adjustable straps to connect them to the waist and arm can meaningfully increase the reachable workspace of persons with post-stroke arm weakness.
Stanford is currently not accepting patients for this trial.
-
Effects of Post-Stroke Upper Extremity Assistance
Not Recruiting
The purpose of this study is to quantify the improvement of post- stroke individuals' ability to move their arms during and after robot assisted therapy. While researchers know that robot assisted therapies improve motor performance over the course of weeks, they do not know how motor performance is affected over the course of minutes or hours. A better understanding of how robot assisted therapies affect motor performance on short time scales may help us to prescribe more effective therapy doses to maximize motor recovery after neurological injury. The study will allow us to obtain a detailed understanding of the performance of the device as described above.
Stanford is currently not accepting patients for this trial.
-
Passive Tactile Stimulation for Stroke Rehabilitation
Not Recruiting
Stroke can lead to weakness and spasticity in the arm or hand. The purpose of this study is to optimize the design of gentle vibratory stimulation delivered to the hands of individuals with chronic stroke, and explore the effect on range of movement and spasticity.
Stanford is currently not accepting patients for this trial. For more information, please contact Maarten Lansberg, MD, 650-723-6469.
2024-25 Courses
- Design and Control of Haptic Systems
ME 327 (Spr) - Dynamic Systems, Vibrations and Control
ME 161 (Win) - Perspectives of Women in Engineering
ENGR 311A (Win) -
Independent Studies (23)
- Advanced Reading and Research
CS 499 (Aut, Win, Spr, Sum) - Advanced Reading and Research
CS 499P (Aut, Win, Spr, Sum) - Curricular Practical Training
CS 390A (Aut, Win, Spr, Sum) - Curricular Practical Training
CS 390B (Aut, Win, Spr, Sum) - Curricular Practical Training
CS 390C (Aut, Win, Spr, Sum) - Directed Investigation
BIOE 392 (Aut, Win, Spr, Sum) - Directed Research and Writing in Aero/Astro
AA 190 (Aut, Win, Spr, Sum) - Directed Study
BIOE 391 (Aut, Win, Spr, Sum) - Engineering Problems
ME 391 (Aut, Win, Spr, Sum) - Engineering Problems and Experimental Investigation
ME 191 (Aut, Win, Spr, Sum) - Experimental Investigation of Engineering Problems
ME 392 (Aut, Win, Spr, Sum) - Honors Research
ME 191H (Aut, Win, Spr, Sum) - Independent Project
CS 399 (Aut, Win, Spr, Sum) - Independent Study in Aero/Astro
AA 199 (Aut, Win, Spr, Sum) - Master's Directed Research
ME 393 (Aut, Win, Spr, Sum) - Master's Directed Research: Writing the Report
ME 393W (Aut, Win, Spr, Sum) - Out-of-Department Advanced Research Laboratory in Bioengineering
BIOE 191X (Aut, Win, Spr, Sum) - Ph.D. Research Rotation
ME 398 (Aut, Win, Spr, Sum) - Ph.D. Teaching Experience
ME 491 (Aut, Win, Spr) - Practical Training
ME 199A (Win, Spr) - Practical Training
ME 299A (Aut, Win, Spr, Sum) - Practical Training
ME 299B (Aut, Win, Spr, Sum) - Writing Intensive Senior Research Project
CS 191W (Aut, Win, Spr)
- Advanced Reading and Research
-
Prior Year Courses
2023-24 Courses
- Design and Control of Haptic Systems
ME 327 (Spr)
2022-23 Courses
- Design and Control of Haptic Systems
ME 327 (Spr) - Introduction to Mechanical Engineering
ME 1 (Aut)
2021-22 Courses
- Design and Control of Haptic Systems
ME 327 (Spr) - Feedback Control Design
ENGR 105 (Win)
- Design and Control of Haptic Systems
Stanford Advisees
-
Doctoral Dissertation Reader (AC)
Dane Brouwer, Dan Fan, Shivani Guptasarma, Eunyoung Kim, Ava Lakmazaheri, Wing-Sum Law, Kirsten Seagers, Soham Sinha, Jon Stingel, Connor Yako -
Orals Chair
John Alora, Javier Yu -
Postdoctoral Faculty Sponsor
Alaa Eldin Magdy Abdelbaky Ab Abdelaal, Cosima du Pasquier -
Doctoral Dissertation Advisor (AC)
Megan Coram, Mary Kate Gale, William Heap, Sehui Jeong, Rianna Jitosho, Melissa Klein, Sreela Kodali, Godson Osele, Jasmin Palmer, Yimeng Qin, Brian Vuong, Crystal Winston -
Doctoral Dissertation Co-Advisor (AC)
Elizabeth Childs, Dan Ilyin, Carolyn Kim, Elizabeth Vasquez -
Master's Program Advisor
Sean Fowler, Jorge Garcia, Nachiketh Karthik, Haley Koo, Zachary Larson, Saimai Lau, Ting Lin, Daphne Liu, Raúl Molina Gómez, Wei Lin Puah, Savitha Srinivasan, Vakula Venkatesh, Peter Xu, Ethan Yang -
Doctoral (Program)
Katelyn King, Venny Kojouharov, Olivia Tomassetti
All Publications
-
A Comparison of Pneumatic Actuators for Soft Growing Vine Robots.
Soft robotics
2024
Abstract
Soft pneumatic actuators are used to steer soft growing "vine" robots while being flexible enough to undergo the tip eversion required for growth. In this study, we compared the performance of three types of pneumatic actuators in terms of their ability to perform eversion, quasi-static bending, dynamic motion, and force output: the pouch motor, the cylindrical pneumatic artificial muscle (cPAM), and the fabric pneumatic artificial muscle (fPAM). The pouch motor is advantageous for prototyping owing to its simple manufacturing process. The cPAM exhibits superior bending behavior and produces the highest forces, whereas the fPAM actuates fastest and everts at the lowest pressure. We evaluated a range of dimensions for each actuator type. Larger actuators can produce more significant deformations and forces, but smaller actuators inflate faster and can evert at a lower pressure. Because vine robots are lightweight, the effect of gravity on the functionality of different actuators is minimal. We developed a new analytical model that predicts the pressure-to-bending behavior of vine robot actuators. Using the actuator results, we designed and demonstrated a 4.8 m long vine robot equipped with highly maneuverable 60 × 60 mm cPAMs in a three-dimensional obstacle course. The vine robot was able to move around sharp turns, travel through a passage smaller than its diameter, and lift itself against gravity.
View details for DOI 10.1089/soro.2023.0169
View details for PubMedID 38717834
-
Stiffness Change for Reconfiguration of Inflated Beam Robots.
Soft robotics
2024
Abstract
Abstract Active control of the shape of soft robots is challenging. Despite having an infinite number of passive degrees of freedom (DOFs), soft robots typically only have a few actively controllable DOFs, limited by the number of degrees of actuation (DOAs). The complexity of actuators restricts the number of DOAs that can be incorporated into soft robots. Active shape control is further complicated by the buckling of soft robots under compressive forces; this is particularly challenging for compliant continuum robots due to their long aspect ratios. In this study, we show how variable stiffness enables shape control of soft robots by addressing these challenges. Dynamically changing the stiffness of sections along a compliant continuum robot selectively "activates" discrete joints. By changing which joints are activated, the output of a single actuator can be reconfigured to actively control many different joints, thus decoupling the number of controllable DOFs from the number of DOAs. We demonstrate embedded positive pressure layer jamming as a simple method for stiffness change in inflated beam robots, its compatibility with growing robots, and its use as an "activating" technology. We experimentally characterize the stiffness change in a growing inflated beam robot and present finite element models that serve as guides for robot design and fabrication. We fabricate a multisegment everting inflated beam robot and demonstrate how stiffness change is compatible with growth through tip eversion, enables an increase in workspace, and achieves new actuation patterns not possible without stiffening.
View details for DOI 10.1089/soro.2023.0120
View details for PubMedID 38683643
-
Reliability of Smartphone-Based Vibration Threshold Measurements
IEEE. 2024: 25-32
View details for DOI 10.1109/HAPTICS59260.2024.10520838
View details for Web of Science ID 001229108600004
-
phloSAR: a Portable, High-Flow Pressure Supply and Regulator Enabling Untethered Operation of Large Pneumatic Soft Robots
IEEE. 2024: 28-33
View details for DOI 10.1109/ROBOSOFT60065.2024.10522055
View details for Web of Science ID 001230033600142
-
Design and Evaluation of Haptic Guidance in Ultrasound-Based Needle-Insertion Procedures.
IEEE transactions on bio-medical engineering
2024; 71 (1): 26-35
Abstract
OBJECTIVE: This article presents two haptic guidance systems designed to help a clinician keep an ultrasound probe steady when completing ultrasound-assisted needle insertion tasks. These procedures demand spatial reasoning and hand-eye coordination because the clinician must align a needle with the ultrasound probe and extrapolate the needle trajectory using only a 2D ultrasound image. Past research has shown that visual guidance helps the clinician align the needle, but does not help the clinician keep the ultrasound probe steady, sometimes resulting in a failed procedure.METHODS: We created two separate haptic guidance systems to provide feedback if the user tilts the ultrasound probe away from the desired setpoint using (1) vibrotactile stimulation provided by a voice coil motor or (2) distributed tactile pressure provided by a pneumatic mechanism.RESULTS: Both systems significantly reduced probe deviation and correction time to errors during a needle insertion task. We also tested the two feedback systems in a more clinically relevant setup and showed that the perceptibility of the feedback was not affected by the addition of a sterile bag placed over the actuators and gloves worn by the user.CONCLUSION: These studies show that both types of haptic feedback are promising for helping the user keep the ultrasound probe steady during ultrasound-assisted needle insertion tasks. Survey results indicated that users preferred the pneumatic system over the vibrotactile system.SIGNIFICANCE: Haptic feedback may improve user performance in ultrasound-based needle-insertion procedures and shows promise in training for needle-insertion tasks and other medical procedures where guidance is required.
View details for DOI 10.1109/TBME.2023.3290919
View details for PubMedID 37384470
-
Haptic Relocation of Virtual Finger Forces via Pneumatic Wrist-Worn Haptic Devices
IEEE. 2024: 315-320
View details for DOI 10.1109/HAPTICS59260.2024.10520855
View details for Web of Science ID 001229108600020
-
Haptics: The Science of Touch as a Foundational Pathway to Precision Education and Assessment.
Academic medicine : journal of the Association of American Medical Colleges
2023
Abstract
Clinical touch is the cornerstone of the doctor-patient relationship and can impact patient experience and outcomes. In the current era, driven by an ever-increasing infusion of point of care technologies, physical exam skills have become undervalued. Moreover, touch and hands-on skills have been difficult to teach due to inaccurate assessments and difficulty with learning transfer through observation. In this article, the authors argue that haptics, the science of touch, provides a unique opportunity to explore new pathways to facilitate touch training. Furthermore, haptics can dramatically increase the density of touch-based assessments without increasing human rater burden-essential for realizing precision assessment. The science of haptics is reviewed, including the benefits of using haptics-informed language for objective structured clinical examinations. The authors describe how haptic devices and haptic language have and can be used to facilitate learning, communication, documentation and a much-needed reinvigoration of physical examination and touch excellence at the point of care. The synergy of haptic devices, artificial intelligence, and virtual reality environments are discussed. The authors conclude with challenges of scaling haptic technology in medical education, such as cost and translational needs, and opportunities to achieve wider adoption of this transformative approach to precision education.
View details for DOI 10.1097/ACM.0000000000005607
View details for PubMedID 38109654
-
Shared-Control Teleoperation Paradigms on a Soft-Growing Robot Manipulator
JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS
2023; 109 (2)
View details for DOI 10.1007/s10846-023-01919-x
View details for Web of Science ID 001188382800001
-
Modeling and Control of a 5-DOF Parallel Continuum Haptic Device
IEEE TRANSACTIONS ON ROBOTICS
2023
View details for DOI 10.1109/TRO.2023.3277068
View details for Web of Science ID 001006596900001
-
A Modular 3-Degrees-of-Freedom Force Sensor for Robot-Assisted Minimally Invasive Surgery Research.
Sensors (Basel, Switzerland)
2023; 23 (11)
Abstract
Effective force modulation during tissue manipulation is important for ensuring safe, robot-assisted, minimally invasive surgery (RMIS). Strict requirements for in vivo applications have led to prior sensor designs that trade off ease of manufacture and integration against force measurement accuracy along the tool axis. Due to this trade-off, there are no commercial, off-the-shelf, 3-degrees-of-freedom (3DoF) force sensors for RMIS available to researchers. This makes it challenging to develop new approaches to indirect sensing and haptic feedback for bimanual telesurgical manipulation. We present a modular 3DoF force sensor that integrates easily with an existing RMIS tool. We achieve this by relaxing biocompatibility and sterilizability requirements and by using commercial load cells and common electromechanical fabrication techniques. The sensor has a range of ±5 N axially and ±3 N laterally with errors of below 0.15 N and maximum errors below 11% of the sensing range in all directions. During telemanipulation, a pair of jaw-mounted sensors achieved average errors below 0.15 N in all directions. It achieved an average grip force error of 0.156 N. The sensor is for bimanual haptic feedback and robotic force control in delicate tissue telemanipulation. As an open-source design, the sensors can be adapted to suit other non-RMIS robotic applications.
View details for DOI 10.3390/s23115230
View details for PubMedID 37299958
-
Cognitive and Physical Activities Impair Perception of Smartphone Vibrations.
IEEE transactions on haptics
2023; PP
Abstract
Vibration feedback is common in everyday devices, from virtual reality systems to smartphones. However, cognitive and physical activities may impede our ability to sense vibrations from devices. In this study, we develop and characterize a smartphone platform to investigate how a shape-memory task (cognitive activity) and walking (physical activity) impair human perception of smartphone vibrations. We measured how Apple's Core Haptics Framework parameters can be used for haptics research, namely how hapticIntensity modulates amplitudes of 230 Hz vibrations. A 23-person user study found that physical ( ) and cognitive ( p=0.004) activity increase vibration perception thresholds. Cognitive activity also increases vibration response time ( ). This work also introduces a smartphone platform that can be used for out-of-lab vibration perception testing. Researchers can use our smartphone platform and results to design better haptic devices for diverse, unique populations.
View details for DOI 10.1109/TOH.2023.3279201
View details for PubMedID 37220041
-
Daily vibrotactile stimulation from a wearable device exhibits equal or greater spasticity relief than botulinum toxin in stroke.
Archives of physical medicine and rehabilitation
2023
Abstract
OBJECTIVE: To test the feasibility and efficacy of the VibroTactile Stimulation (VTS) Glove, a wearable device that provides vibrotactile stimulation to the impaired limb to reduce spastic hypertonia.DESIGN: Prospective two-arm intervention study - including one group of patients who use Botulinum toxin (BTX-A) for spasticity and one group of patients who do not use BTX-A.SETTING: Participants were recruited through rehabilitation and neurology clinics.PARTICIPANTS: Patients with chronic stroke (N=20; mean age=54 years, mean time since stroke=6.9 years). Patients who were previously receiving the standard of care (BTX-A injection) were eligible to participate, and started the intervention 12 weeks after their last injection.INTERVENTION: Participants were instructed to use the VTS Glove for three hours daily, at home or during everyday activities, for 8 weeks.MAIN OUTCOME MEASURES: Spasticity was assessed with the Modified Ashworth Scale and the Modified Tardieu Scale at baseline and then at 2-week intervals for 12 weeks. Primary outcomes were the difference from baseline and at week 8 (end of VTS Glove use) and week 12 (four weeks after stopping VTS Glove use). Patients who were receiving BTX-A were also assessed during the 12 weeks preceding the start of VTS Glove use to monitor the effect of BTX-A on spastic hypertonia. Range of motion and participant feedback were also studied.RESULTS: A clinically meaningful difference in spastic hypertonia was found during and after daily VTS Glove use. Modified Ashworth and Modified Tardieu scores were reduced by an average of 0.9 (p=0.0014) and 0.7 (p=0.0003), respectively, at week 8 of daily VTS Glove use, and by 1.1 (p=0.00025) and 0.9 (p=0.0001), respectively, one month after stopping VTS Glove use. For participants who used BTX-A, 6 out of 11 showed greater change in Modified Ashworth ratings during VTS Glove use (Mean=-1.8 vs. Mean=-1.6 with BTX-A) and 8 out of 11 showed their lowest level of symptoms during VTS Glove use (vs. BTX-A).CONCLUSIONS: Daily stimulation from the VTS Glove provides relief of spasticity and hypertonia. For more than half of participants who had regularly used BTX-A, the VTS Glove provided equal or greater symptom relief.
View details for DOI 10.1016/j.apmr.2023.03.031
View details for PubMedID 37149017
-
Evaluation of a Passive Wearable Device for Post-Stroke Shoulder Abduction Support.
IEEE ... International Conference on Rehabilitation Robotics : [proceedings]
2023; 2023: 1-6
Abstract
Post-stroke upper extremity function can be improved by devices that support shoulder abduction. However, many of these devices provide limited assistance in activities of daily living due to their complexity and encumbrance. We developed and evaluated a passive, lightweight (0.6 kg) wearable device consisting of an aluminum frame and elastic bands attached to a posture vest to aid in shoulder abduction. The number and thickness of bands can be adjusted to provide supportive forces to the affected arm. We measured reachable workspace area and Wolf Motor Function Test (WMFT) performance in people with a history of stroke (n = 11) with and without the wearable. The device increased workspace area in 6 participants and improved average WMFT functional and timing scores in 7 and 12 tasks, respectively, out of 16 total tasks. On average, participants increased their arm motion within 20 cm of shoulder level by 22.4% and decreased their hand's average distance from trunk by 15.2%, both improvements in the device case.
View details for DOI 10.1109/ICORR58425.2023.10304815
View details for PubMedID 37941216
-
Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms
JMLR-JOURNAL MACHINE LEARNING RESEARCH. 2023
View details for Web of Science ID 001221201501010
-
Relief of post-stroke spasticity with acute vibrotactile stimulation: controlled crossover study of muscle and skin stimulus methods.
Frontiers in human neuroscience
2023; 17: 1206027
Abstract
Background: Prior work suggests that vibratory stimulation can reduce spasticity and hypertonia. It is unknown which of three predominant approaches (stimulation of the spastic muscle, antagonist muscle, or cutaneous regions) most reduces these symptoms.Objective: Determine which vibrotactile stimulation approach is most effective at reducing spastic hypertonia among post-stroke patients.Methods: Sham-controlled crossover study with random assignment of condition order in fourteen patients with post-stroke hand spasticity. All patients were studied in four conditions over four visits: three stimulation conditions and a sham control. The primary outcome measure was the Modified Ashworth Scale, and the secondary outcome measure was the Modified Tardieu Scale measured manually and using 3D motion capture. For each condition, measures of spastic hypertonia were taken at four time points: baseline, during stimulation, after stimulation was removed, and after a gripping exercise.Results: A clinically meaningful difference in spastic hypertonia was found during and after cutaneous stimulation of the hand. Modified Ashworth and Modified Tardieu scores were reduced by a median of 1.1 (SD = 0.84, p = 0.001) and 0.75 (SD = 0.65, p = 0.003), respectively, during cutaneous stimulation, and by 1.25 (SD = 0.94, p = 0.001) and 0.71 (SD = 0.67, p = 0.003), respectively, at 15 min after cutaneous stimulation. Symptom reductions with spastic muscle stimulation and antagonist muscle stimulation were non-zero but not significant. There was no change with sham stimulation.Conclusions: Cutaneous vibrotactile stimulation of the hand provides significant reductions in spastic hypertonia, compared to muscle stimulation.Clinical trial registration: www.ClinicalTrials.gov, identifier: NCT03814889.
View details for DOI 10.3389/fnhum.2023.1206027
View details for PubMedID 37706171
-
Passive Shape Locking for Multi-Bend Growing Inflated Beam Robots
IEEE. 2023
View details for DOI 10.1109/ROBOSOFT55895.2023.10122027
View details for Web of Science ID 001008269300076
-
A Multi-Segment, Soft Growing Robot with Selective Steering
IEEE. 2023
View details for DOI 10.1109/ROBOSOFT55895.2023.10122091
View details for Web of Science ID 001008269300124
-
Exploring Human Response Times to Combinations of Audio, Haptic, and Visual Stimuli from a Mobile Device
IEEE. 2023: 121-127
View details for DOI 10.1109/WHC56415.2023.10224375
View details for Web of Science ID 001082286400018
-
Wearable Sensory Substitution for Proprioception via Deep Pressure
IEEE. 2023: 286-292
View details for DOI 10.1109/WHC56415.2023.10224435
View details for Web of Science ID 001082286400042
-
Human Perception of Wrist Flexion and Extension Torque During Upper and Lower Extremity Movement.
IEEE transactions on haptics
2022; PP
Abstract
Real-world application of haptic feedback from kinesthetic devices is implemented while the user is in motion, but human wrist torque magnitude discrimination has previously only been characterized while users are stationary. In this study, we measured wrist torque discrimination in conditions relevant to activities of daily living, using a previously developed backdrivable wrist exoskeleton capable of applying wrist flexion and extension torque. We implemented a torque comparison test using a two-alternative forced-choice paradigm while participants were both seated and walking on a treadmill, with both a stationary and a moving wrist. Like most kinesthetic haptic devices, the wrist exoskeleton output torque is commanded in an open-loop manner. Thus, the study design was informed by Monte Carlo simulations to verify that the errors in the wrist exoskeleton output torque would not significantly affect the results. Results from ten participants show that although both walking and moving wrist conditions result in higher Weber Fractions (worse perception), participants were able to detect relatively small changes in torque of 12-19% on average in all grouped conditions. The results provide insight regarding the torque magnitudes necessary to make wrist-worn kinesthetic haptic devices noticeable and meaningful to the user in various conditions relevant to activities of daily living.
View details for DOI 10.1109/TOH.2022.3219031
View details for PubMedID 36343009
-
Perceived Intensities of Normal and Shear Skin Stimuli Using a Wearable Haptic Bracelet
IEEE ROBOTICS AND AUTOMATION LETTERS
2022; 7 (3): 6099-6106
View details for DOI 10.1109/LRA.2021.3140132
View details for Web of Science ID 000785670500003
-
Data-Driven Sparse Skin Stimulation Can Convey Social Touch Information to Humans
IEEE TRANSACTIONS ON HAPTICS
2022; 15 (2): 392-404
Abstract
During social interactions, people use auditory, visual, and haptic cues to convey their thoughts, emotions, and intentions. Due to weight, energy, and other hardware constraints, it is difficult to create devices that completely capture the complexity of human touch. Here we explore whether a sparse representation of human touch is sufficient to convey social touch signals. To test this we collected a dataset of social touch interactions using a soft wearable pressure sensor array, developed an algorithm to map recorded data to an array of actuators, then applied our algorithm to create signals that drive an array of normal indentation actuators placed on the arm. Using this wearable, low-resolution, low-force device, we find that users are able to distinguish the intended social meaning, and compare performance to results based on direct human touch. As online communication becomes more prevalent, such systems to convey haptic signals could allow for improved distant socializing and empathetic remote human-human interaction.
View details for DOI 10.1109/TOH.2021.3129067
View details for Web of Science ID 000818844700016
View details for PubMedID 34793305
-
A 4-Degree-of-Freedom Parallel Origami Haptic Device for Normal, Shear, and Torsion Feedback
IEEE ROBOTICS AND AUTOMATION LETTERS
2022; 7 (2): 3310-3317
View details for DOI 10.1109/LRA.2022.3144798
View details for Web of Science ID 000753553400020
-
Predicting Hand-Object Interaction for Improved Haptic Feedback in Mixed Reality
IEEE ROBOTICS AND AUTOMATION LETTERS
2022; 7 (2): 3851-3857
View details for DOI 10.1109/LRA.2022.3148458
View details for Web of Science ID 000756831900029
-
Design of a Wearable Vibrotactile Stimulation Device for Individuals With Upper-Limb Hemiparesis and Spasticity
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING
2022; 30: 1277-1287
Abstract
Vibratory stimulation may improve post-stroke symptoms such as spasticity; however, current studies are limited by the large, clinic-based apparatus used to apply this stimulation. A wearable device could provide vibratory stimulation in a mobile form, enabling further study of this technique. An initial device, the vibrotactile stimulation (VTS) Glove, was deployed in an eight-week clinical study in which sixteen individuals with stroke used the device for several hours daily. Participants reported wearing the glove during activities such as church, social events, and dining out. However, 69% of participants struggled to extend or insert their fingers to don the device. In a follow-up study, eight individuals with stroke evaluated new VTS device prototypes in a three-round iterative design study with the aims of creating the next generation of VTS devices and understanding features that influence interaction with a wearable device by individuals with impaired upper-limb function. Interviews and interaction tasks were used to define actionable design revisions between each round of evaluation. Our analysis identified six new themes from participants regarding device designs: hand supination is challenging, separate finger attachments inhibit fit and use, fingers may be flexed or open, fabric coverage impacts comfort, a reduced concern for social comfort, and the affected hand is infrequently used. Straps that wrap around the arm and fixtures on the anterior arm were other challenging features. We discuss potential accommodations for these challenges, as well as social comfort. New VTS device designs are presented and were donned in an average time of 48 seconds.
View details for DOI 10.1109/TNSRE.2022.3174808
View details for Web of Science ID 000797424600010
View details for PubMedID 35552152
-
Characterization of Real-time Haptic Feedback from Multimodal Neural Network-based Force Estimates during Teleoperation
IEEE. 2022: 1471-1478
View details for DOI 10.1109/IROS47612.2022.9981662
View details for Web of Science ID 000908368201062
-
Perception of Mechanical Properties via Wrist Haptics: Effects of Feedback Congruence
IEEE. 2022: 620-627
View details for DOI 10.1109/IROS47612.2022.9982079
View details for Web of Science ID 000908368200060
-
Haptic Feedback Relocation from the Fingertips to the Wrist for Two-Finger Manipulation in Virtual Reality
IEEE. 2022: 628-633
View details for DOI 10.1109/IROS47612.2022.9981392
View details for Web of Science ID 000908368200061
-
A Large-Area Wearable Soft Haptic Device Using Stacked Pneumatic Pouch Actuation
IEEE. 2022: 591-598
View details for DOI 10.1109/IROS47612.2022.9981919
View details for Web of Science ID 000908368200057
-
Deep Learning Classification of Touch Gestures Using Distributed Normal and Shear Force
IEEE. 2022: 3659-3665
View details for DOI 10.1109/IROS47612.2022.9981457
View details for Web of Science ID 000908368202109
-
FingerPrint: A 3-D Printed Soft Monolithic 4-Degree-of-Freedom Fingertip Haptic Device with Embedded Actuation
IEEE. 2022: 938-944
View details for DOI 10.1109/ROBOSOFT54090.2022.9762107
View details for Web of Science ID 000945389700094
-
A Lightweight, High-Extension, Planar 3-Degree-of-Freedom Manipulator Using Pinched Bistable Tapes
IEEE. 2022
View details for DOI 10.1109/ICRA.46639.2022.9811976
View details for Web of Science ID 000941265700108
-
Between-Tactor Display Using Dynamic Tactile Stimuli
SPRINGER INTERNATIONAL PUBLISHING AG. 2022: 379-381
View details for Web of Science ID 000886345900051
-
Feasibility of Smartphone Vibrations as a Sensory Diagnostic Tool
SPRINGER INTERNATIONAL PUBLISHING AG. 2022: 337-339
View details for Web of Science ID 000886345900039
-
Wearable Haptic Device for Individuals with Congenital Absence of Proprioception
SPRINGER INTERNATIONAL PUBLISHING AG. 2022: 433-435
View details for Web of Science ID 000886345900066
-
Geometric Solutions for General Actuator Routing on Inflated-Beam Soft Growing Robots
IEEE TRANSACTIONS ON ROBOTICS
2021
View details for DOI 10.1109/TRO.2021.3115230
View details for Web of Science ID 000733460200001
-
Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex
IEEE TRANSACTIONS ON HAPTICS
2021; 14 (4): 762-775
Abstract
Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.
View details for DOI 10.1109/TOH.2021.3072615
View details for Web of Science ID 000731146900007
View details for PubMedID 33844633
-
Teaching With Hapkit Enabling Online Haptics Courses With Hands-On Laboratories
IEEE ROBOTICS & AUTOMATION MAGAZINE
2021; 28 (3): 79-91
View details for DOI 10.1109/MRA.2020.3024395
View details for Web of Science ID 000695640900006
-
Body-Mounted Vibrotactile Stimuli: Simultaneous Display of Taps on the Fingertips and Forearm
IEEE TRANSACTIONS ON HAPTICS
2021; 14 (2): 432-444
Abstract
In this article, we present a body-mounted tactile display to deliver haptic feedback to the forearm and user-initiated haptic feedback to the fingertips. The display mounts two vibrotactile actuators on the forearm, leaving the user's hands free for manipulation tasks when the hands are not interacting with the tactile display, while also exploiting the tactile sensitivity of the fingertips when needed. We test the effectiveness of the display using paired vibrotactile taps sensed through the forearm and the fingertips, either separately or simultaneously. We measure the ability of participants to identify the vibrotactile taps. The results show that mounting the device on the forearm, so that the participant touches the forearm-mounted device with their fingertips receiving feedback to both locations simultaneously, decreases performance relative to mounting on the fingertips unless large amplitudes are used. We also test the accuracy with which participants identified different numbers of vibration taps (4, 8, 16, and 25 signals). The results show that as the number of signals changes, participant accuracy is not different when stimulating the fingertips alone compared to stimulating the fingertips and forearm together. We conclude with an example of a portable and wearable vibration display, and discuss future use cases of such a display.
View details for DOI 10.1109/TOH.2020.3042955
View details for Web of Science ID 000665616100026
View details for PubMedID 33290228
-
Affective Ratings of Vibrotactile Signals in Older Adults With and Without History of Stroke
IEEE. 2021: 457-462
View details for DOI 10.1109/WHC49131.2021.9517216
View details for Web of Science ID 000707066600060
-
A Dynamics Simulator or Soft Growing Robots
IEEE. 2021: 11775-11781
View details for DOI 10.1109/ICRA48506.2021.9561420
View details for Web of Science ID 000771405404012
-
Macro-Mini Actuation of Pneumatic Pouches for Soft Wearable Haptic Displays
IEEE. 2021: 14499-14505
View details for DOI 10.1109/ICRA48506.2021.9560786
View details for Web of Science ID 000771405405125
-
Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State
IEEE. 2021: 12335-12341
View details for DOI 10.1109/ICRA48506.2021.9560945
View details for Web of Science ID 000771405404054
-
Augmented Haptic Guidance for Needle Insertion with a 2-DoF Wrist-Worn Haptic Device
IEEE. 2021: 872
View details for DOI 10.1109/WHC49131.2021.9517164
View details for Web of Science ID 000707066600104
-
Embedded Laser-Cut Constraints for Elastomeric Soft Actuators
IEEE. 2021: 863
View details for DOI 10.1109/WHC49131.2021.9517206
View details for Web of Science ID 000707066600095
-
Human Perception of Wrist Torque Magnitude During Upper and Lower Extremity Movement
IEEE. 2021: 870
View details for DOI 10.1109/WHC49131.2021.9517214
View details for Web of Science ID 000707066600102
-
Augmented Needle Decompression Task with a Wrist-Worn Haptic Device
IEEE. 2021: 873
View details for DOI 10.1109/WHC49131.2021.9517260
View details for Web of Science ID 000707066600105
-
Design, Modeling, Control, and Application of Everting Vine Robots.
Frontiers in robotics and AI
2020; 7: 548266
Abstract
In nature, tip-localized growth allows navigation in tightly confined environments and creation of structures. Recently, this form of movement has been artificially realized through pressure-driven eversion of flexible, thin-walled tubes. Here we review recent work on robots that "grow" via pressure-driven eversion, referred to as "everting vine robots," due to a movement pattern that is similar to that of natural vines. We break this work into four categories. First, we examine the design of everting vine robots, highlighting tradeoffs in material selection, actuation methods, and placement of sensors and tools. These tradeoffs have led to application-specific implementations. Second, we describe the state of and need for modeling everting vine robots. Quasi-static models of growth and retraction and kinematic and force-balance models of steering and environment interaction have been developed that use simplifying assumptions and limit the involved degrees of freedom. Third, we report on everting vine robot control and planning techniques that have been developed to move the robot tip to a target, using a variety of modalities to provide reference inputs to the robot. Fourth, we highlight the benefits and challenges of using this paradigm of movement for various applications. Everting vine robot applications to date include deploying and reconfiguring structures, navigating confined spaces, and applying forces on the environment. We conclude by identifying gaps in the state of the art and discussing opportunities for future research to advance everting vine robots and their usefulness in the field.
View details for DOI 10.3389/frobt.2020.548266
View details for PubMedID 33501315
View details for PubMedCentralID PMC7805729
-
Continuous Closed-Loop 4-Degree-of-Freedom Holdable Haptic Guidance
IEEE ROBOTICS AND AUTOMATION LETTERS
2020; 5 (4): 6853–60
View details for DOI 10.1109/LRA.2020.3020581
View details for Web of Science ID 000569783100001
-
Efficient and Trustworthy Social Navigation via Explicit and Implicit Robot-Human Communication
IEEE TRANSACTIONS ON ROBOTICS
2020; 36 (3): 692–707
View details for DOI 10.1109/TRO.2020.2964824
View details for Web of Science ID 000543027200008
-
Model-Based Design of a Soft 3-D Haptic Shape Display
IEEE TRANSACTIONS ON ROBOTICS
2020; 36 (3): 613–28
View details for DOI 10.1109/TRO.2020.2980114
View details for Web of Science ID 000543027200003
-
3D Electromagnetic Reconfiguration Enabled by Soft Continuum Robots
IEEE ROBOTICS AND AUTOMATION LETTERS
2020; 5 (2): 1704–11
View details for DOI 10.1109/LRA.2020.2969922
View details for Web of Science ID 000526520500005
-
Retraction of Soft Growing Robots Without Buckling
IEEE ROBOTICS AND AUTOMATION LETTERS
2020; 5 (2): 2115–22
View details for DOI 10.1109/LRA.2020.2970629
View details for Web of Science ID 000526572000003
-
Robust navigation of a soft growing robot by exploiting contact with the environment
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2020
View details for DOI 10.1177/0278364920903774
View details for Web of Science ID 000523938000001
-
An untethered isoperimetric soft robot.
Science robotics
2020; 5 (40)
Abstract
For robots to be useful for real-world applications, they must be safe around humans, be adaptable to their environment, and operate in an untethered manner. Soft robots could potentially meet these requirements; however, existing soft robotic architectures are limited by their ability to scale to human sizes and operate at these scales without a tether to transmit power or pressurized air from an external source. Here, we report an untethered, inflated robotic truss, composed of thin-walled inflatable tubes, capable of shape change by continuously relocating its joints, while its total edge length remains constant. Specifically, a set of identical roller modules each pinch the tube to create an effective joint that separates two edges, and modules can be connected to form complex structures. Driving a roller module along a tube changes the overall shape, lengthening one edge and shortening another, while the total edge length and hence fluid volume remain constant. This isoperimetric behavior allows the robot to operate without compressing air or requiring a tether. Our concept brings together advantages from three distinct types of robots-soft, collective, and truss-based-while overcoming certain limitations of each. Our robots are robust and safe, like soft robots, but not limited by a tether; are modular, like collective robots, but not limited by complex subunits; and are shape-changing, like truss robots, but not limited by rigid linear actuators. We demonstrate two-dimensional (2D) robots capable of shape change and a human-scale 3D robot capable of punctuated rolling locomotion and manipulation, all constructed with the same modular rollers and operating without a tether.
View details for DOI 10.1126/scirobotics.aaz0492
View details for PubMedID 33022597
-
An untethered isoperimetric soft robot
SCIENCE ROBOTICS
2020; 5 (40)
View details for DOI 10.1126/scirobotics.aaz0492
View details for Web of Science ID 000520870400001
-
Investigating Social Haptic Illusions for Tactile Stroking (SHIFTS)
IEEE. 2020: 629–36
View details for DOI 10.1109/HAPTICS45997.2020.ras.HAP20.35.f631355d
View details for Web of Science ID 000578523300012
-
Teleoperation of an ankle-foot prosthesis with a wrist exoskeleton.
IEEE transactions on bio-medical engineering
2020; PP
Abstract
We aimed to develop a system for people with amputation that non-invasively restores missing control and sensory information for an ankle-foot prosthesis.In our approach, a wrist exoskeleton allows people with amputation to control and receive feedback from their prosthetic ankle via teleoperation. We implemented two control schemes: position control with haptic feedback of ankle torque at the wrist; and torque control that allows the user to modify a baseline torque profile by moving their wrist against a virtual spring. We measured tracking error and frequency response for the ankle-foot prosthesis and the wrist exoskeleton. To demonstrate feasibility and evaluate system performance, we conducted an experiment in which one participant with a transtibial amputation tracked desired wrist trajectories during walking, while we measured wrist and ankle response.Benchtop testing demonstrated that for relevant walking frequencies, system error was below human perceptual error. During the walking experiment, the participant was able to voluntarily follow different wrist trajectories with an average RMS error of 1.55° after training. The ankle was also able to track desired trajectories below human perceptual error for both position control (RMSE = 0.8°) and torque control (RMSE = 8.4%).We present a system that allows a user with amputation to control an ankle-foot prosthesis and receive feedback about its state using a wrist exoskeleton, with accuracy comparable to biological neuromotor control.This bilateral teleoperation system enables novel prosthesis control and feedback strategies that could improve prosthesis control and aid motor learning.
View details for DOI 10.1109/TBME.2020.3046357
View details for PubMedID 33347402
-
Human Interface for Teleoperated Object Manipulation with a Soft Growing Robot
IEEE. 2020: 726-732
View details for Web of Science ID 000712319500083
-
Dynamically Reconfigurable Discrete Distributed Stiffness for Inflated Beam Robots
IEEE. 2020: 9050-9056
View details for Web of Science ID 000712319505140
-
Evaluation of Non-collocated Force Feedback Driven by Signal-independent Noise
IEEE. 2020: 3686-3692
View details for Web of Science ID 000712319502084
-
A Tip Mount for Transporting Sensors and Tools using Soft Growing Robots
IEEE. 2020: 8781-8788
View details for DOI 10.1109/IROS45743.2020.9340950
View details for Web of Science ID 000724145802131
-
AFREEs: Active Fiber Reinforced Elastomeric Enclosures
IEEE. 2020: 305–11
View details for Web of Science ID 000610491800036
-
Understanding Continuous and Pleasant Linear Sensations on the Forearm From a Sequential Discrete Lateral Skin-Slip Haptic Device
IEEE TRANSACTIONS ON HAPTICS
2019; 12 (4): 414–27
Abstract
A continuous stroking sensation on the skin can convey messages or emotion cues. We seek to induce this sensation using a combination of illusory motion and lateral stroking via a haptic device. Our system provides discrete lateral skin-slip on the forearm with rotating tactors, which independently provide lateral skin-slip in a timed sequence. We vary the sensation by changing the angular velocity and delay between adjacent tactors, such that the apparent speed of the perceived stroke ranges from 2.5 to 48.2 cm/s. We investigated which actuation parameters create the most pleasant and continuous sensations through a user study with 16 participants. On average, the sensations were rated by participants as both continuous and pleasant. The most continuous and pleasant sensations were created by apparent speeds of 7.7 and 5.1 cm/s, respectively. We also investigated the effect of spacing between contact points on the pleasantness and continuity of the stroking sensation, and found that the users experience a pleasant and continuous linear sensation even when the space between contact points is relatively large (40 mm). Understanding how sequential discrete lateral skin-slip creates continuous linear sensations can influence the design and control of future wearable haptic devices.
View details for DOI 10.1109/TOH.2019.2941190
View details for Web of Science ID 000505585900003
View details for PubMedID 31536015
-
Soft Haptic Device to Render the Sensation of Flying Like a Drone
IEEE ROBOTICS AND AUTOMATION LETTERS
2019; 4 (3): 2524–31
View details for DOI 10.1109/LRA.2019.2907432
View details for Web of Science ID 000464913800005
-
Resonant Frequency Skin Stretch for Wearable Haptics.
IEEE transactions on haptics
2019
Abstract
Resonant frequency skin stretch uses cyclic lateral skin stretches matching the skin's resonant frequency to create highly noticeable stimuli, signifying a new approach for wearable haptic stimulation. Three experiments were performed to explore biomechanical and perceptual aspects of resonant frequency skin stretch. In the first experiment, effective skin resonant frequencies were quantified at the forearm, shank, and foot. In the second experiment, perceived haptic stimuli were characterized for skin stretch actuations across a spectrum of frequencies. In the third experiment, haptic classification ability was determined as subjects differentiated haptic stimulation cues while sitting, walking, and jogging. Results showed that subjects perceived stimulations at, above, and below the skin's resonant frequency differently: stimulations lower than the skin resonant frequency felt like distinct impacts, stimulations at the skin resonant frequency felt like cyclic skin stretches, and stimulations higher than the skin resonant frequency felt like standard vibrations. Subjects successfully classified stimulations while sitting, walking, and jogging, and classification accuracy decreased with increasing speed, especially for stimulations at the shank. This work could facilitate more widespread use of wearable skin stretch. Potential applications include gaming, medical simulation, and surgical augmentation, and for training to reduce injury risk or improve sports performance.
View details for DOI 10.1109/TOH.2019.2917072
View details for PubMedID 31095499
-
Stiffness Control of Deformable Robots Using Finite Element Modeling
IEEE ROBOTICS AND AUTOMATION LETTERS
2019; 4 (2): 469–76
View details for DOI 10.1109/LRA.2019.2890897
View details for Web of Science ID 000456652300026
-
Design and Analysis of Pneumatic 2-DoF Soft Haptic Devices for Shear Display
IEEE ROBOTICS AND AUTOMATION LETTERS
2019; 4 (2): 1365–71
View details for DOI 10.1109/LRA.2019.2895890
View details for Web of Science ID 000459538100025
-
Evaluation of Skin Deformation Tactile Feedback for Teleoperated Surgical Tasks
IEEE TRANSACTIONS ON HAPTICS
2019; 12 (2): 102–13
Abstract
During interaction with objects using a tool, we experience force and tactile feedback. One form of tactile feedback is local fingerpad skin deformation. In this paper, we provide haptic feedback to users of a teleoperation system through a skin deformation tactile feedback device. The device is able to provide tangential and normal skin deformation in a coupled manner, and is designed so that users can grasp it with a precision grip using multiple fingerpads. By applying skin deformation feedback on multiple fingerpads, the device is able to provide multi-degree-of-freedom interaction force direction and magnitude information to the user. To evaluate the effectiveness of this approach for the performance of teleoperated manipulation tasks, we performed a study in which 20 participants used a teleoperation system to perform one of two manipulation tasks (peg transfer and tube connection) using force feedback, skin deformation feedback, and the combination of both feedback. Results showed that participants are able to use all feedback to improve task performance compared to the case without haptic feedback, although the degree of improvement depended on the nature of the task. The feedback also improved situation awareness, felt consistent with prior experience, and did not affect concentration on the task, as reported by participants.
View details for DOI 10.1109/TOH.2018.2873398
View details for Web of Science ID 000472569100001
View details for PubMedID 30281480
-
Effects of Different Hand-Grounding Locations on Haptic Performance With a Wearable Kinesthetic Haptic Device
IEEE ROBOTICS AND AUTOMATION LETTERS
2019; 4 (2): 351–58
View details for DOI 10.1109/LRA.2018.2890198
View details for Web of Science ID 000456652300011
-
Evolution and Analysis of Hapkit: An Open-Source Haptic Device for Educational Applications.
IEEE transactions on haptics
2019
Abstract
We present the design, evolution and analysis of "Hapkit", a low-cost, open-source kinesthetic haptic device for use in educational applications. Hapkit was developed in 2013 based on the design of the Stanford Haptic Paddle, with the goal of decreasing cost and increasing accessibility for educational applications, including online teaching, K-12 school use, and college dynamic systems and control courses. In order to develop Hapkit for these purposes, we tested a variety of transmission, actuation, and structural materials. Hapkit 3.0, the latest version, uses a capstan drive, inexpensive DC motor, and 3-D printed structural materials. A frequency-domain system identification method was used to characterize Hapkit dynamics across the various designs. This method was validated using a first principles parameter measurement and a transient response analysis. This characterization shows that Hapkit 3.0 has lower damping and Coulomb friction than previous designs. We also performed a user study demonstrating that Hapkit 3.0 improves discrimination of virtual stiffness compared to previous designs. The design evolution of Hapkit resulted in a low-cost, high-performance device appropriate for open-source dissemination and educational applications.
View details for DOI 10.1109/TOH.2019.2948609
View details for PubMedID 31634847
-
Perception of a Wearable Haptic Feedback Device to Render the Sensation of Flight
IEEE. 2019: 61–66
View details for Web of Science ID 000538798200011
-
3-DoF Wearable, Pneumatic Haptic Device to Deliver Normal, Shear, Vibration, and Torsion Feedback
IEEE. 2019: 97–102
View details for Web of Science ID 000538798200017
-
Holdable Haptic Device for 4-DOF Motion Guidance
IEEE. 2019: 109–14
View details for Web of Science ID 000538798200019
-
Vine Robots: Design, Teleoperation, and Deployment for Navigation and Exploration
IEEE Robotics & Automation Magazine
2019
View details for DOI 10.1109/MRA.2019.2947538
-
Facilitating Human-Mobile Robot Communication via Haptic Feedback and Gesture Teleoperation
ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION
2018; 7 (3)
View details for DOI 10.1145/3243503
View details for Web of Science ID 000457673200003
-
A Soft, Steerable Continuum Robot That Grows via Tip Extension
SOFT ROBOTICS
2019; 6 (1): 95–108
View details for DOI 10.1089/soro.2018.0034
View details for Web of Science ID 000447723100001
-
Toward the Design of Personalized Continuum Surgical Robots
ANNALS OF BIOMEDICAL ENGINEERING
2018; 46 (10): 1522–33
View details for DOI 10.1007/s10439-018-2062-2
View details for Web of Science ID 000445180100008
-
Toward the Design of Personalized Continuum Surgical Robots.
Annals of biomedical engineering
2018
Abstract
Robot-assisted minimally invasive surgical systems enable procedures with reduced pain, recovery time, and scarring compared to traditional surgery. While these improvements benefit a large number of patients, safe access to diseased sites is not always possible for specialized patient groups, including pediatric patients, due to their anatomical differences. We propose a patient-specific design paradigm that leverages the surgeon's expertise to design and fabricate robots based on preoperative medical images. The components of the patient-specific robot design process are a virtual reality design interface enabling the surgeon to design patient-specific tools, 3-D printing of these tools with a biodegradable polyester, and an actuation and control system for deployment. The designed robot is a concentric tube robot, a type of continuum robot constructed from precurved, elastic, nesting tubes. We demonstrate the overall patient-specific design workflow, from preoperative images to physical implementation, for an example clinical scenario: nonlinear renal access to a pediatric kidney. We also measure the system's behavior as it is deployed through real and artificial tissue. System integration and successful benchtop experiments in ex vivo liver and in a phantom patient model demonstrate the feasibility of using a patient-specific design workflow to plan, fabricate, and deploy personalized, flexible continuum robots.
View details for PubMedID 29855755
-
Haptic Dimensions of Human-Robot Interaction
ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION
2018; 7 (1)
View details for DOI 10.1145/3209768
View details for Web of Science ID 000457700500007
-
A Tip-Extending Soft Robot Enables Reconfigurable and Deployable Antennas
IEEE ROBOTICS AND AUTOMATION LETTERS
2018; 3 (2): 949–56
View details for DOI 10.1109/LRA.2018.2793303
View details for Web of Science ID 000424646100014
-
Comparing Proprioceptive Acuity in the Arm between Joint Space and Task Space
IEEE. 2018: 125–32
View details for Web of Science ID 000434958700021
-
Obstacle-Aided Navigation of a Soft Growing Robot
IEEE COMPUTER SOC. 2018: 4165–72
View details for Web of Science ID 000446394503026
-
HapWRAP: Soft Growing Wearable Haptic Device
IEEE COMPUTER SOC. 2018: 5466–72
View details for Web of Science ID 000446394504019
-
Scaling Inertial Forces to Alter Weight Perception in Virtual Reality
IEEE COMPUTER SOC. 2018: 484–89
View details for Web of Science ID 000446394500051
-
Effects of Latency and Refresh Rate on Force Perception via Sensory Substitution by Force-Controlled Skin Deformation Feedback
IEEE COMPUTER SOC. 2018: 506–11
View details for Web of Science ID 000446394500052
-
APAM: Antagonistic Pneumatic Artificial Muscle
IEEE COMPUTER SOC. 2018: 1539–46
View details for Web of Science ID 000446394501027
-
Avoiding Human-Robot Collisions using Haptic Communication
IEEE COMPUTER SOC. 2018: 5828–34
View details for Web of Science ID 000446394504057
-
Magnified Force Sensory Substitution for Telemanipulation via Force-Controlled Skin Deformation
IEEE COMPUTER SOC. 2018: 4142–48
View details for Web of Science ID 000446394503025
-
Robotic Assistance-as-Needed for Enhanced Visuomotor Learning in Surgical Robotics Training: An Experimental Study
IEEE COMPUTER SOC. 2018: 6631–36
View details for Web of Science ID 000446394505012
-
Development and Evaluation of an Intuitive Flexible Interface for Teleoperating Soft Growing Robots
IEEE. 2018: 4995–5002
View details for Web of Science ID 000458872704086
-
Gaussian Process Dynamic Programming for Optimizing Ungrounded Haptic Guidance
IEEE. 2018: 8758–64
View details for Web of Science ID 000458872707119
-
Haptics: The Present and Future of Artificial Touch Sensation
ANNUAL REVIEW OF CONTROL, ROBOTICS, AND AUTONOMOUS SYSTEMS, VOL 1
2018; 1: 385–409
View details for DOI 10.1146/annurev-control-060117-105043
View details for Web of Science ID 000467687700016
-
A Social Haptic Device to Create Continuous Lateral Motion using Sequential Normal Indentation
IEEE. 2018: 32–39
View details for Web of Science ID 000434958700006
-
A soft robot that navigates its environment through growth.
Science robotics
2017; 2 (8)
Abstract
Across kingdoms and length scales, certain cells and organisms navigate their environments not through locomotion but through growth. This pattern of movement is found in fungal hyphae, developing neurons, and trailing plants, and is characterized by extension from the tip of the body, length change of hundreds of percent, and active control of growth direction. This results in the abilities to move through tightly constrained environments and form useful three-dimensional structures from the body. We report a class of soft pneumatic robot that is capable of a basic form of this behavior, growing substantially in length from the tip while actively controlling direction using onboard sensing of environmental stimuli; further, the peak rate of lengthening is comparable to rates of animal and robot locomotion. This is enabled by two principles: Pressurization of an inverted thin-walled vessel allows rapid and substantial lengthening of the tip of the robot body, and controlled asymmetric lengthening of the tip allows directional control. Further, we demonstrate the abilities to lengthen through constrained environments by exploiting passive deformations and form three-dimensional structures by lengthening the body of the robot along a path. Our study helps lay the foundation for engineered systems that grow to navigate the environment.
View details for DOI 10.1126/scirobotics.aan3028
View details for PubMedID 33157883
-
Highly Articulated Robotic Needle Achieves Distributed Ablation of Liver Tissue.
IEEE robotics and automation letters
2017; 2 (3): 1367-1374
Abstract
Robotic needle steering will improve percutaneous radio-frequency ablation (RFA) in the liver by performing distributed ablations without requiring multiple punctures of the liver capsule, thus enabling the treatment of large or multifocal tumors. However, state-of-the-art asymmetric-tip robotic needle steering systems do not yet achieve clinically relevant curvature. This work presents the design and development of a highly articulated needle that enables distributed RFA in liver tissue under ultrasound (US) image guidance. Our new needle design attains the target curvature required for liver procedures while meeting important clinical requirements, such as the use of fixed diameter needle introducers, presence of a free needle working channel, robustness for repeated insertions, and conductivity for the delivery of RF current for tissue ablation. The new needle tip includes two important design features: A tendon-actuated Nitinol asymmetric flexure joint, which allows for an active amplification of the needle steering force, and a steel back-bevel tip profile, which decreases the risk of needle jamming. The needle's resulting curvature was evaluated in both phantom and ex vivo liver tissues using segmented US images. The average radius of minimum curvature in ex vivo liver tissue was found to be 33.6 mm, the smallest reported to date. Furthermore, RFA in ex vivo porcine liver tissue tests were performed to demonstrate that distributedablation with a single puncture of the liver capsule is possible via robotic needle steering.
View details for DOI 10.1109/LRA.2017.2668467
View details for PubMedID 28664186
View details for PubMedCentralID PMC5484158
-
Design of a Compact Actuation and Control System for Flexible Medical Robots.
IEEE robotics and automation letters
2017; 2 (3): 1579-1585
Abstract
Flexible medical robots can improve surgical procedures by decreasing invasiveness and increasing accessibility within the body. Using preoperative images, these robots can be designed to optimize a procedure for a particular patient. To minimize invasiveness and maximize biocompatibility, the actuation units of flexible medical robots should be placed fully outside the patient's body. In this letter, we present a novel, compact, lightweight, modular actuation, and control system for driving a class of these flexible robots, known as concentric tube robots. A key feature of the design is the use of three-dimensional printed waffle gears to enable compact control of two degrees of freedom within each module. We measure the precision and accuracy of a single actuation module and demonstrate the ability of an integrated set of three actuation modules to control six degrees of freedom. The integrated system drives a three-tube concentric tube robot to reach a final tip position that is on average less than 2 mm from a given target. In addition, we show a handheld manifestation of the device and present its potential applications.
View details for DOI 10.1109/LRA.2017.2676240
View details for PubMedID 28664187
View details for PubMedCentralID PMC5484156
-
Series Pneumatic Artificial Muscles (sPAMs) and Application to a Soft Continuum Robot.
IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation
2017; 2017: 5503–10
Abstract
We describe a new series pneumatic artificial muscle (sPAM) and its application as an actuator for a soft continuum robot. The robot consists of three sPAMs arranged radially round a tubular pneumatic backbone. Analogous to tendons, the sPAMs exert a tension force on the robot's pneumatic backbone, causing bending that is approximately constant curvature. Unlike a traditional tendon driven continuum robot, the robot is entirely soft and contains no hard components, making it safer for human interaction. Models of both the sPAM and soft continuum robot kinematics are presented and experimentally verified. We found a mean position accuracy of 5.5 cm for predicting the end-effector position of a 42 cm long robot with the kinematic model. Finally, closed-loop control is demonstrated using an eye-in-hand visual servo control law which provides a simple interface for operation by a human. The soft continuum robot with closed-loop control was found to have a step-response rise time and settling time of less than two seconds.
View details for PubMedID 29379672
-
Three-dimensional skin deformation as force substitution: Wearable device design and performance during haptic exploration of virtual environments.
IEEE transactions on haptics
2017
Abstract
Virtual reality systems would benefit from a compelling force sensory substitute when workspace or stability limitations prevent the use of kinesthetic force feedback systems. We present a wearable fingertip haptic device with the ability to make and break contact in addition to rendering both shear and normal skin deformation to the fingerpad. A delta mechanism with novel bias spring and tether actuator relocation method enables the use of high-end motors and encoders, allowing precise device control: 10 Hz bandwidth and 0.255 mm RMS tracking error were achieved during testing. In the first of two experiments, participants determined the orientation of a stiff region in a surrounding compliant virtual surface with an average angular error of 7.6, similar to that found in previous studies using traditional force feedback. In the second experiment, we evaluated participants' ability to interpret differences in friction. The Just Noticeable Difference (JND) of surface friction coefficient discrimination using our skin deformation device was 0.20, corresponding with a reference friction coefficient of 0.5. While higher than that found using kinesthetic feedback, this demonstrates that users can perceive differences in surface friction without world-grounded kinesthetic forces. These experiments show that three DoF skin deformation enables both stiffness and friction discrimination capability in the absence of kinesthetic force feedback.
View details for DOI 10.1109/TOH.2017.2672969
View details for PubMedID 28237933
-
Deformable Model-Based Methods for Shape Control of a Haptic Jamming Surface.
IEEE transactions on visualization and computer graphics
2017; 23 (2): 1029-1041
Abstract
Haptic Jamming, the approach of simultaneously controlling mechanical properties and surface deformation of a tactile display via particle jamming and pneumatics, shows promise as a tangible, shape-changing human-computer interface. Previous research introduced device design and described the force-displacement interactions for individual jamming cells. The work in this article analyzes the shape output capabilities of a multi-cell array. A spring-mass deformable body simulation combines models of the three actuation inputs of a Haptic Jamming surface: node pinning, chamber pressurization, and cell jamming. Surface measurements of a 12-cell prototype from a depth camera fit the mass and stiffness parameters to the device during pressurization tests and validate the accuracy of the model for various actuation sequences. The simulator is used to develop an algorithm that generates a sequence of actuation inputs for a Haptic Jamming array of any size in order to match a desired surface output shape. Data extracted from topographical maps and three-dimensional solid object models are used to evaluate the shape-matching algorithm and assess the utility of increasing array size and resolution. Results show that a discrete Laplace operator applied to the input is a suitable predictor of the correlation coefficient between the desired shape and the device output.
View details for DOI 10.1109/TVCG.2016.2525788
View details for PubMedID 26863666
-
Perception of force and stiffness in the presence of low-frequency haptic noise.
PloS one
2017; 12 (6)
Abstract
This work lays the foundation for future research on quantitative modeling of human stiffness perception. Our goal was to develop a method by which a human's ability to perceive suprathreshold haptic force stimuli and haptic stiffness stimuli can be affected by adding haptic noise.Five human participants performed a same-different task with a one-degree-of-freedom force-feedback device. Participants used the right index finger to actively interact with variations of force (∼5 and ∼8 N) and stiffness (∼290 N/m) stimuli that included one of four scaled amounts of haptically rendered noise (None, Low, Medium, High). The haptic noise was zero-mean Gaussian white noise that was low-pass filtered with a 2 Hz cut-off frequency; the resulting low-frequency signal was added to the force rendered while the participant interacted with the force and stiffness stimuli.We found that the precision with which participants could identify the magnitude of both the force and stiffness stimuli was affected by the magnitude of the low-frequency haptically rendered noise added to the haptic stimulus, as well as the magnitude of the haptic stimulus itself. The Weber fraction strongly correlated with the standard deviation of the low-frequency haptic noise with a Pearson product-moment correlation coefficient of ρ > 0.83. The mean standard deviation of the low-frequency haptic noise in the haptic stimuli ranged from 0.184 N to 1.111 N across the four haptically rendered noise levels, and the corresponding mean Weber fractions spanned between 0.042 and 0.101.The human ability to perceive both suprathreshold haptic force and stiffness stimuli degrades in the presence of added low-frequency haptic noise. Future work can use the reported methods to investigate how force perception and stiffness perception may relate, with possible applications in haptic watermarking and in the assessment of the functionality of peripheral pathways in individuals with haptic impairments.
View details for DOI 10.1371/journal.pone.0178605
View details for PubMedID 28575068
-
Haptic Orientation Guidance Using Two Parallel Double-Gimbal Control Moment Gyroscopes.
IEEE transactions on haptics
2017
Abstract
This paper presents a system of two double-gimbal control moment gyroscopes (CMGs) for providing ungrounded kinesthetic haptic feedback. By spinning a second flywheel opposite the first, and rotating them through opposite trajectories, undesired gyroscopic effects can be eliminated, isolating a single torque axis. This produces a moment pulse proportional to the flywheel spin speed and rotation speed. Rotating the CMG gimbals quickly in one direction, then resetting them more slowly generates repeated torque pulses indicating a clear direction cue. We present the mathematical model for moments produced by this system and verify that the performance of our device matches this model. Using these asymmetric moment pulses, we provide haptic cues to participants in two studies. In the first study, users simply identify the direction of torque cues. In the second study, we use the torque pulses to guide users to target orientations. Performance in both studies shows that this system has the potential to provide useful guidance for applications where ungrounded haptic feedback is desired.
View details for DOI 10.1109/TOH.2017.2713380
View details for PubMedID 28600261
-
Open Source, Modular, Customizable, 3-D Printed Kinesthetic Haptic Devices
IEEE. 2017: 142–47
View details for Web of Science ID 000426705900025
-
Design of Patient-Specific Concentric Tube Robots Using Path Planning from 3-D Ultrasound
IEEE. 2017: 165–68
Abstract
Percutaneous techniques and robot-assisted surgical systems have enabled minimally invasive procedures that offer reduced scarring, recovery time, and complications compared to traditional open surgeries. Despite these improvements, access to diseased sites using the standard, straight needle-based percutaneous techniques is still limited for certain procedures due to intervening tissues. These limitations can be further exacerbated in specific patient groups, particularly pediatric patients, whose anatomy does not fit the traditional tools and systems. We therefore propose a patient-specific paradigm to design and fabricate dexterous, robotic tools based on the patient's preoperative images. In this paper, we present the main steps of our proposed paradigm - image-based path planning, robot design, and fabrication - along with an example case that focuses on a class of dexterous, snake-like tools called concentric tube robots. We demonstrate planning a safe path using a patient's preoperative ultrasound images. We then determine the concentric tube robot parameters needed to achieve this path, and finally, we use 3-D printing to fabricate the patient-specific robot.
View details for Web of Science ID 000427085300041
View details for PubMedID 29059836
-
Simulating the impact of sensorimotor deficits on reaching performance
IEEE. 2017: 31–37
Abstract
The healthy human nervous system accurately and robustly controls movements despite nonlinear dynamics, noise, and delays. After a stroke, motor ability frequently becomes impaired. To provide insight into the relative impact of specific sensorimotor deficits on motor performance, we modeled neural control of reaching with the human upper limb as a near-optimally feedback-controlled two-degree-of-freedom system with biologically based parameters. We added three sensorimotor impairments commonly associated with post-stroke hemiparesis - abnormal joint coupling, increased noise on internally modeled dynamics, and muscular weakness - and examined the impact on reaching performance. We found that abnormal joint coupling unknown to the system's internal model caused systematic perturbations to trajectories, longer reach durations, and target overshoot. Increasing internal model noise and muscular weakness had little impact on motor performance unless model noise was increased by several orders of magnitude. Many reaches performed by our perturbed models replicate features commonly observed in reaches by hemiparetic stroke survivors. The sensitivity to unmodeled abnormal joint coupling agrees with experimental findings that abnormal coupling (possibly related to internal model errors) is the main cause of post-stroke motor impairment.
View details for Web of Science ID 000426850800006
View details for PubMedID 28813789
-
Fingertip Tactile Devices for Virtual Object Manipulation and Exploration
ASSOC COMPUTING MACHINERY. 2017: 3115–19
View details for DOI 10.1145/3025453.3025744
View details for Web of Science ID 000426970503006
-
WAVES: A Wearable Asymmetric Vibration Excitation System for Presenting Three-Dimensional Translation and Rotation Cues
ASSOC COMPUTING MACHINERY. 2017: 4972–82
View details for DOI 10.1145/3025453.3025741
View details for Web of Science ID 000426970504071
-
Design of a Soft Catheter for Low-Force and Constrained Surgery
IEEE. 2017: 174–80
View details for Web of Science ID 000426978200025
-
Analysis of Effective Impedance Transmitted to the Operator in Position-Exchange Bilateral Teleoperation
IEEE. 2017: 328–33
View details for Web of Science ID 000426705900056
-
Propagation of Joint Space Quantization Error to Operational Space Coordinates and Their Derivatives
IEEE. 2017: 2054–61
View details for Web of Science ID 000426978202048
-
Training in Divergent and Convergent Force Fields During 6-DOF Teleoperation with a Robot-Assisted Surgical System
IEEE. 2017: 195–200
View details for Web of Science ID 000426705900034
-
Design of 3-D Printed Concentric Tube Robots
IEEE TRANSACTIONS ON ROBOTICS
2016; 32 (6): 1419-1430
View details for DOI 10.1109/TRO.2016.2602368
View details for Web of Science ID 000389849700008
-
Design of 3-D Printed Concentric Tube Robots.
IEEE transactions on robotics : a publication of the IEEE Robotics and Automation Society
2016; 32 (6): 1419-1430
Abstract
Concentric tube surgical robots are minimally invasive devices with the advantages of snake-like reconfigurability, long and thin form factor, and placement of actuation outside the patient's body. These robots can also be designed and manufactured to acquire targets in specific patients for treating specific diseases in a manner that minimizes invasiveness. We propose that concentric tube robots can be manufactured using 3-D printing technology on a patient- and procedure-specific basis. In this paper, we define the design requirements and manufacturing constraints for 3-D printed concentric tube robots and experimentally demonstrate the capabilities of these robots. While numerous 3-D printing technologies and materials can be used to create such robots, one successful example uses selective laser sintering to make an outer tube with a polyether block amide and uses stereolithography to make an inner tube with a polypropylene-like material. This enables a tube pair with precurvatures of 0.0775 and 0.0455 mm-1, which can withstand strains of 20% and 5.5% for the outer and inner tubes, respectively.
View details for DOI 10.1109/TRO.2016.2602368
View details for PubMedID 28713227
View details for PubMedCentralID PMC5505693
-
Stability and quantization-error analysis of haptic rendering of virtual stiffness and damping
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2016; 35 (9): 1103-1120
View details for DOI 10.1177/0278364915596234
View details for Web of Science ID 000380936500005
-
Methods for Improving the Curvature of Steerable Needles in Biological Tissue
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2016; 63 (6): 1167-1177
Abstract
Robotic needle steering systems have the potential to improve percutaneous interventions such as radiofrequency ablation of liver tumors, but steering techniques described to date have not achieved sufficiently small radius of curvature in biological tissue to be relevant to this application. In this study, the impact of tip geometry on steerable needle curvature was examined.Finite-element simulations and experiments with bent-tip needles in ex vivo liver tissue were performed. Motivated by the results of this analysis, a new articulated-tip steerable needle was designed, in which a distal section is actively switched by a robotic system between a straight tip (resulting in a straight path) and a bent tip (resulting in a curved path).Selection of tip length and angle can greatly improve curvature, with radius of curvature below 5 cm in liver tissue possible through judicious selection of these parameters. An articulated-tip mechanism allows the tip length and angle to be increased, while the straight configuration allows the needle tip to still pass through an introducer sheath and rotate inside the body.Validation testing in liver tissue shows that the new articulated-tip steerable needle achieves smaller radius of curvature compared to bent-tip needles described in previous work.Steerable needles with optimized tip parameters, which can generate tight curves in liver tissue, increase the clinical relevance of needle steering to percutaneous interventions.
View details for DOI 10.1109/TBME.2015.2484262
View details for Web of Science ID 000377045500009
View details for PubMedID 26441438
View details for PubMedCentralID PMC4862936
-
Surgeon Design Interface for Patient-Specific Concentric Tube Robots.
Proceedings of the ... IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics
2016; 2016: 41-48
Abstract
Concentric tube robots have potential for use in a wide variety of surgical procedures due to their small size, dexterity, and ability to move in highly curved paths. Unlike most existing clinical robots, the design of these robots can be developed and manufactured on a patient- and procedure-specific basis. The design of concentric tube robots typically requires significant computation and optimization, and it remains unclear how the surgeon should be involved. We propose to use a virtual reality-based design environment for surgeons to easily and intuitively visualize and design a set of concentric tube robots for a specific patient and procedure. In this paper, we describe a novel patient-specific design process in the context of the virtual reality interface. We also show a resulting concentric tube robot design, created by a pediatric urologist to access a kidney stone in a pediatric patient.
View details for DOI 10.1109/BIOROB.2016.7523596
View details for PubMedID 28656124
View details for PubMedCentralID PMC5483336
-
A Framework for Multilateral Manipulation in Surgical Tasks
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
2016; 13 (1): 68-77
View details for DOI 10.1109/TASE.2015.2499195
View details for Web of Science ID 000374443300009
-
Motor learning affects car-to-driver handover in automated vehicles.
Science robotics
2016; 1 (1)
Abstract
Vehicles in the foreseeable future will be required to transition between autonomous driving (without human involvement) and full human control. During this transition period, the human, who has not been actively engaged in the driving process, must resume the motor control necessary to steer the car. The in-car study presented here demonstrates that when human drivers are presented with a steering behavior that is different from the last time they were in control, specifically the ratio of hand wheel angle to road wheel angle (emulating a change in vehicle speed), they undergo a significant period of adaptation before they return to their previous steering behavior. However, drivers do not require an adaptation period to return to previous driving behavior after changes in steering torque. These findings have implications for the design of vehicles that transition from automated to manual driving and for understanding of human motor control in real-world tasks.
View details for DOI 10.1126/scirobotics.aah5682
View details for PubMedID 33157857
-
Tactor-Induced Skin Stretch as a Sensory Substitution Method in Teleoperated Palpation
IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
2015; 45 (6): 714-726
View details for DOI 10.1109/THMS.2015.2463090
View details for Web of Science ID 000366893700006
-
Rendered and Characterized Closed-Loop Accuracy of Impedance-Type Haptic Displays
IEEE TRANSACTIONS ON HAPTICS
2015; 8 (4): 434-446
Abstract
Impedance-type kinesthetic haptic displays aim to render arbitrary desired dynamics to a human operator using force feedback. To render realistic virtual environments, the difference between desired and rendered dynamics must be small. In this paper, we analyze the closed-loop dynamics of haptic displays for three common virtual environments: a spring, a damper, and a spring-damper, including the effects of time delay and low-pass filtering. Using a linear model, we identify important parameters for the rendered dynamics in terms of effective impedances, a conceptual tool that decomposes the displays closed-loop impedance into components with physical analogs. Our results establish bandwidth limits for rendering effective stiffness and damping. The effective stiffness bandwidth is limited by the virtual stiffness and device mass, and the effective damping bandwidth is limited by the cut-off frequency of the low-pass filter which filters the device velocity estimate. We show that a general system impedance can be characterized by a mass, damper, and spring optimally by the solution to a convex optimization problem, and we present a quantitative metric, the Average Distortion Error (ADE), to describe the fidelity of this model. Time delay has no significant effect on characterized stiffness, and reduces characterized damping by the product of virtual stiffness and total time delay. Reducing the low-pass filter cut-off frequency reduces the characterized damping. Experimental data gathered with a Phantom Premium 1.5 validates the theoretical analysis. We also conducted human user experiments to investigate the effects of time delay and low-pass filtering on perceived stiffness and damping. Similar to the characterized dynamics results, we observed no significant effect of time delay on perceived stiffness, and increasing time delay resulted in reduced perceived damping. Lower filter cut-off frequencies resulted in lower perceived damping. This work informs haptic display design by presenting how closed-loop behavior changes with key parameters.
View details for DOI 10.1109/TOH.2015.2457438
View details for Web of Science ID 000369611500008
View details for PubMedID 26208363
-
M-Width: Stability, noise characterization, and accuracy of rendering virtual mass
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2015; 34 (6): 781-798
View details for DOI 10.1177/0278364914559294
View details for Web of Science ID 000353981500004
-
Remote Electromagnetic Vibration of Steerable Needles for Imaging in Power Doppler Ultrasound.
IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation
2015; 2015: 2244-2249
Abstract
Robotic needle steering systems for minimally invasive medical procedures require complementary medical imaging systems to track the needles in real time. Ultrasound is a promising imaging modality because it offers relatively low-cost, real-time imaging of the needle. Previous methods applied vibration to the base of the needle using a voice coil actuator, in order to make the needle visible in power Doppler ultrasound. We propose a new method for needle tip vibration, using electromagnetic actuation of small permanent magnets placed inside the needle to improve needle tip visibility in power Doppler imaging. Robotic needle insertion experiments using artificial tissue and ex vivo porcine liver showed that the electromagnetic tip vibration method can generate a stronger Doppler response compared to the previous base vibration method, resulting in better imaging at greater needle depth in tissue. It also eliminates previous issues with vibration damping along the shaft of the needle.
View details for DOI 10.1109/ICRA.2015.7139496
View details for PubMedID 26413379
View details for PubMedCentralID PMC4578321
-
Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback
IEEE TRANSACTIONS ON HAPTICS
2015; 8 (2): 209-221
Abstract
During tool-mediated interaction with everyday objects, we experience kinesthetic forces and tactile sensations in the form of vibration and skin deformation at the fingerpad. Fingerpad skin deformation is caused by forces applied tangentially and normally to the fingerpad skin, resulting in tangential and normal skin displacement. We designed a device to convey 3-degree-of-freedom (DoF) force information to the user via skin deformation, and conducted two experiments to determine the devices effectiveness for force-feedback substitution and augmentation. For sensory substitution, participants used 1-DoF and 3-DoF skin deformation feedback to locate a feature in a 3-DoF virtual environment. Participants showed improved precision and shorter completion time when using 3-DoF compared to 1-DoF skin deformation feedback. For sensory augmentation, participants traced a path in space from an initial to a target location, while under guidance from force and/or skin deformation feedback. When force feedback was augmented with skin deformation, participants reduced their path-following error over the cases when force or skin deformation feedback are used separately. We conclude that 3-DoF skin deformation feedback is effective in substituting or augmenting force feedback. Such substitution or augmentation could be used when force feedback is unattainable or attenuated due to device limitations or system instability.
View details for DOI 10.1109/TOH.2015.2398448
View details for Web of Science ID 000356521100009
View details for PubMedID 25647582
-
Methods to Segment Hard Inclusions in Soft Tissue During Autonomous Robotic Palpation
IEEE TRANSACTIONS ON ROBOTICS
2015; 31 (2): 344-354
View details for DOI 10.1109/TRO.2015.2402531
View details for Web of Science ID 000352361300010
-
Artificial Tactile Sensing of Position and Slip Speed by Exploiting Geometrical Features
IEEE-ASME TRANSACTIONS ON MECHATRONICS
2015; 20 (1): 263-274
View details for DOI 10.1109/TMECH.2014.2321680
View details for Web of Science ID 000352060700026
-
Uncontrolled Manifold Analysis of Arm Joint Angle Variability During Robotic Teleoperation and Freehand Movement of Surgeons and Novices
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2014; 61 (12): 2869-2881
View details for DOI 10.1109/TBME.2014.2332359
View details for Web of Science ID 000345501000006
-
Testing models of cerebellar ataxia via dynamic simulation.
Robotica
2014; 32 (8): 1383-1397
Abstract
Patients with damage to the cerebellum make reaching movements that are uncoordinated or "ataxic." One prevailing hypothesis is that the cerebellum functions as an internal model for planning movements, and that damage to the cerebellum results in movements that do not properly account for arm dynamics. An exoskeleton robot was used to record multi-joint reaching movements. Subsequently, joint-torque trajectories were calculated and a gradient descent algorithm found optimal, patient-specific perturbations to actual limb dynamics predicted to reduce directional reaching errors by an average of 41%, elucidating a promising form of robotic intervention and adding support to the internal model hypothesis.
View details for DOI 10.1017/s0263574714002306
View details for PubMedID 37090787
View details for PubMedCentralID PMC10117632
-
3-D ultrasound-guided robotic needle steering in biological tissue.
IEEE transactions on bio-medical engineering
2014; 61 (12): 2899-2910
Abstract
Robotic needle steering systems have the potential to greatly improve medical interventions, but they require new methods for medical image guidance. Three-dimensional (3-D) ultrasound is a widely available, low-cost imaging modality that may be used to provide real-time feedback to needle steering robots. Unfortunately, the poor visibility of steerable needles in standard grayscale ultrasound makes automatic segmentation of the needles impractical. A new imaging approach is proposed, in which high-frequency vibration of a steerable needle makes it visible in ultrasound Doppler images. Experiments demonstrate that segmentation from this Doppler data is accurate to within 1-2 mm. An image-guided control algorithm that incorporates the segmentation data as feedback is also described. In experimental tests in ex vivo bovine liver tissue, a robotic needle steering system implementing this control scheme was able to consistently steer a needle tip to a simulated target with an average error of 1.57 mm. Implementation of 3-D ultrasound-guided needle steering in biological tissue represents a significant step toward the clinical application of robotic needle steering.
View details for DOI 10.1109/TBME.2014.2334309
View details for PubMedID 25014948
-
Uncontrolled manifold analysis of arm joint angle variability during robotic teleoperation and freehand movement of surgeons and novices.
IEEE transactions on bio-medical engineering
2014; 61 (12): 2869-2881
Abstract
Teleoperated robot-assisted surgery (RAS) is used to perform a wide variety of minimally invasive procedures. However, current understanding of the effect of robotic manipulation on the motor coordination of surgeons is limited. Recent studies in human motor control suggest that we optimize hand movement stability and task performance while minimizing control effort and improving robustness to unpredicted disturbances. To achieve this, the variability of joint angles and muscle activations is structured to reduce task-relevant variability and increase task-irrelevant variability. In this study, we determine whether teleoperation of a da Vinci Si surgical system in a nonclinical task of simple planar movements changes this structure of variability in experienced surgeons and novices. To answer this question, we employ the UnControlled manifold analysis that partitions users' joint angle variability into task-irrelevant and task-relevant manifolds. We show that experienced surgeons coordinate their joint angles to stabilize hand movements more than novices, and that the effect of teleoperation depends on experience--experts increase teleoperated stabilization relative to freehand whereas novices decrease it. We suggest that examining users' exploitation of the task-irrelevant manifold for stabilization of hand movements may be applied to: (1) evaluation and optimization of teleoperator design and control parameters, and (2) skill assessment and optimization of training in RAS.
View details for DOI 10.1109/TBME.2014.2332359
View details for PubMedID 24967980
-
Testing models of cerebellar ataxia via dynamic simulation
ROBOTICA
2014; 32 (8): 1383-1397
View details for DOI 10.1017/S0263574714002306
View details for Web of Science ID 000346921000013
-
3-D Ultrasound-Guided Robotic Needle Steering in Biological Tissue
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2014; 61 (12): 2899-2910
View details for DOI 10.1109/TBME.2014.2334309
View details for Web of Science ID 000345501000009
-
Augmentation Of Stiffness Perception With a 1-Degree-of-Freedom Skin Stretch Device
IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
2014; 44 (6): 731-742
View details for DOI 10.1109/THMS.2014.2348865
View details for Web of Science ID 000346225700004
-
Torsional Dynamics of Steerable Needles: Modeling and Fluoroscopic Guidance
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2014; 61 (11): 2707-2717
Abstract
Needle insertions underlie a diversity of medical interventions. Steerable needles provide a means by which to enhance existing needle-based interventions and facilitate new ones. Tip-steerable needles follow a curved path and can be steered by twisting the needle base during insertion, but this twisting excites torsional dynamics that introduce a discrepancy between the base and tip twist angles. Here, we model the torsional dynamics of a flexible rod-such as a tip-steerable needle-during subsurface insertion and develop a new controller based on the model. The torsional model incorporates time-varying mode shapes to capture the changing boundary conditions inherent during insertion. Numerical simulations and physical experiments using two distinct setups-stereo camera feedback in semitransparent artificial tissue and feedback control with real-time X-ray imaging in optically opaque artificial tissue-demonstrate the need to account for torsional dynamics in control of the needle tip.
View details for DOI 10.1109/TBME.2014.2326161
View details for Web of Science ID 000344082000006
-
Torsional dynamics of steerable needles: modeling and fluoroscopic guidance.
IEEE transactions on bio-medical engineering
2014; 61 (11): 2707-2717
Abstract
Needle insertions underlie a diversity of medical interventions. Steerable needles provide a means by which to enhance existing needle-based interventions and facilitate new ones. Tip-steerable needles follow a curved path and can be steered by twisting the needle base during insertion, but this twisting excites torsional dynamics that introduce a discrepancy between the base and tip twist angles. Here, we model the torsional dynamics of a flexible rod-such as a tip-steerable needle-during subsurface insertion and develop a new controller based on the model. The torsional model incorporates time-varying mode shapes to capture the changing boundary conditions inherent during insertion. Numerical simulations and physical experiments using two distinct setups-stereo camera feedback in semitransparent artificial tissue and feedback control with real-time X-ray imaging in optically opaque artificial tissue-demonstrate the need to account for torsional dynamics in control of the needle tip.
View details for DOI 10.1109/TBME.2014.2326161
View details for PubMedID 24860026
-
Neural coding of passive lump detection in compliant artificial tissue.
Journal of neurophysiology
2014; 112 (5): 1131-1141
Abstract
Here, we investigate the neural mechanisms of detecting lumps embedded in artificial compliant tissues. We performed a combined psychophysical study of humans performing a passive lump detection task with a neurophysiological study in nonhuman primates (Macaca mulatta) where we recorded the responses of peripheral mechanoreceptive afferents to lumps embedded at various depths in intermediates (rubbers) of varying compliance. The psychophysical results reveal that human lump detection is greatly degraded by both lump depth and decreased compliance of the intermediate. The neurophysiology results reveal that only the slowly adapting type 1 (SA1) afferents provide a clear spatial representation of lumps at all depths and that the representation is affected by lump size, depth, and compliance of the intermediate. The rapidly adapting afferents are considerably less sensitive to the lump. We defined eight neural response measures that we hypothesized could explain the psychophysical behavior, including peak firing rate, spatial spread of neural activity, and additional parameters derived from these measures. We find that peak firing rate encodes the depth of the lump, and the neural spatial spread of the SA1 response encodes for lump size but not lump shape. We also find that the perception of lump size may be affected by the compliance of the intermediate. The results show that lump detection is based on a spatial population code of the SA1 afferents, which is distorted by the depth of the lump and compliance of the tissue.
View details for DOI 10.1152/jn.00032.2013
View details for PubMedID 24805077
View details for PubMedCentralID PMC4122732
-
Effects of robotic manipulators on movements of novices and surgeons
SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES
2014; 28 (7): 2145-2158
Abstract
Robot-assisted surgery is widely adopted for many procedures but has not realized its full potential to date. Based on human motor control theories, the authors hypothesized that the dynamics of the master manipulators impose challenges on the motor system of the user and may impair performance and slow down learning. Although studies have shown that robotic outcomes are correlated with the case experience of the surgeon, the relative contribution of cognitive versus motor skill is unknown. This study quantified the effects of da Vinci Si master manipulator dynamics on movements of novice users and experienced surgeons and suggests possible implications for training and robot design.In the reported study, six experienced robotic surgeons and ten novice nonmedical users performed movements under two conditions: teleoperation of a da Vinci Si Surgical system and freehand. A linear mixed model was applied to nine kinematic metrics (including endpoint error, movement time, peak speed, initial jerk, and deviation from a straight line) to assess the effects of teleoperation and expertise. To assess learning effects, t tests between the first and last movements of each type were used.All the users moved slower during teleoperation than during freehand movements (F 1,9343 = 345; p < 0.001). The experienced surgeons had smaller errors than the novices (F 1,14 = 36.8; p < 0.001). The straightness of movements depended on their direction (F 7,9343 = 117; p < 0.001). Learning effects were observed in all conditions. Novice users first learned the task and then the dynamics of the manipulator.The findings showed differences between the novices and the experienced surgeons for extremely simple point-to-point movements. The study demonstrated that manipulator dynamics affect user movements, suggesting that these dynamics could be improved in future robot designs. The authors showed the partial adaptation of novice users to the dynamics. Future studies are needed to evaluate whether it will be beneficial to include early training sessions dedicated to learning the dynamics of the manipulator.
View details for DOI 10.1007/s00464-014-3446-5
View details for Web of Science ID 000338278800017
-
Predicting and correcting ataxia using a model of cerebellar function
BRAIN
2014; 137: 1931-1944
View details for DOI 10.1093/brain/awu115
View details for Web of Science ID 000338646800020
-
Predicting and correcting ataxia using a model of cerebellar function.
Brain
2014; 137: 1931-1944
Abstract
Cerebellar damage results in uncoordinated, variable and dysmetric movements known as ataxia. Here we show that we can reliably model single-joint reaching trajectories of patients (n = 10), reproduce patient-like deficits in the behaviour of controls (n = 11), and apply patient-specific compensations that improve reaching accuracy (P < 0.02). Our approach was motivated by the theory that the cerebellum is essential for updating and/or storing an internal dynamic model that relates motor commands to changes in body state (e.g. arm position and velocity). We hypothesized that cerebellar damage causes a mismatch between the brain's modelled dynamics and the actual body dynamics, resulting in ataxia. We used both behavioural and computational approaches to demonstrate that specific cerebellar patient deficits result from biased internal models. Our results strongly support the idea that an intact cerebellum is critical for maintaining accurate internal models of dynamics. Importantly, we demonstrate how subject-specific compensation can improve movement in cerebellar patients, who are notoriously unresponsive to treatment.
View details for DOI 10.1093/brain/awu115
View details for PubMedID 24812203
View details for PubMedCentralID PMC4065021
-
Task-dependent impedance and implications for upper-limb prosthesis control
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2014; 33 (6): 827-846
View details for DOI 10.1177/0278364913517728
View details for Web of Science ID 000337560700001
-
Grip Force Control during Virtual ObjectInteraction: Effect of Force Feedback,Accuracy Demands, and Training.
IEEE transactions on haptics
2014; 7 (1): 37-47
Abstract
When grasping and manipulating objects, people are able to efficiently modulate their grip force according to the experienced load force. Effective grip force control involves providing enough grip force to prevent the object from slipping, while avoiding excessive force to avoid damage and fatigue. During indirect object manipulation via teleoperation systems or in virtual environments, users often receive limited somatosensory feedback about objects with which they interact. This study examines the effects of force feedback, accuracy demands, and training on grip force control during object interaction in a virtual environment. The task required subjects to grasp and move a virtual object while tracking a target. When force feedback was not provided, subjects failed to couple grip and load force, a capability fundamental to direct object interaction. Subjects also exerted larger grip force without force feedback and when accuracy demands of the tracking task were high. In addition, the presence or absence of force feedback during training affected subsequent performance, even when the feedback condition was switched. Subjects' grip force control remained reminiscent of their employed grip during the initial training. These results motivate the use of force feedback during telemanipulation and highlight the effect of force feedback during training.
View details for DOI 10.1109/TOH.2013.60
View details for PubMedID 24845744
-
Haptic feedback enhances rhythmic motor control by reducing variability, not improving convergence rate.
Journal of neurophysiology
2014; 111 (6): 1286-1299
Abstract
Stability and performance during rhythmic motor behaviors such as locomotion are critical for survival across taxa: falling down would bode well for neither cheetah nor gazelle. Little is known about how haptic feedback, particularly during discrete events such as the heel-strike event during walking, enhances rhythmic behavior. To determine the effect of haptic cues on rhythmic motor performance, we investigated a virtual paddle juggling behavior, analogous to bouncing a table tennis ball on a paddle. Here, we show that a force impulse to the hand at the moment of ball-paddle collision categorically improves performance over visual feedback alone, not by regulating the rate of convergence to steady state (e.g., via higher gain feedback or modifying the steady-state hand motion), but rather by reducing cycle-to-cycle variability. This suggests that the timing and state cues afforded by haptic feedback decrease the nervous system's uncertainty of the state of the ball to enable more accurate control but that the feedback gain itself is unaltered. This decrease in variability leads to a substantial increase in the mean first passage time, a measure of the long-term metastability of a stochastic dynamical system. Rhythmic tasks such as locomotion and juggling involve intermittent contact with the environment (i.e., hybrid transitions), and the timing of such transitions is generally easy to sense via haptic feedback. This timing information may improve metastability, equating to less frequent falls or other failures depending on the task.
View details for DOI 10.1152/jn.00140.2013
View details for PubMedID 24371296
-
Effects of robotic manipulators on movements of novices and surgeons.
Surgical endoscopy
2014
Abstract
Robot-assisted surgery is widely adopted for many procedures but has not realized its full potential to date. Based on human motor control theories, the authors hypothesized that the dynamics of the master manipulators impose challenges on the motor system of the user and may impair performance and slow down learning. Although studies have shown that robotic outcomes are correlated with the case experience of the surgeon, the relative contribution of cognitive versus motor skill is unknown. This study quantified the effects of da Vinci Si master manipulator dynamics on movements of novice users and experienced surgeons and suggests possible implications for training and robot design.In the reported study, six experienced robotic surgeons and ten novice nonmedical users performed movements under two conditions: teleoperation of a da Vinci Si Surgical system and freehand. A linear mixed model was applied to nine kinematic metrics (including endpoint error, movement time, peak speed, initial jerk, and deviation from a straight line) to assess the effects of teleoperation and expertise. To assess learning effects, t tests between the first and last movements of each type were used.All the users moved slower during teleoperation than during freehand movements (F 1,9343 = 345; p < 0.001). The experienced surgeons had smaller errors than the novices (F 1,14 = 36.8; p < 0.001). The straightness of movements depended on their direction (F 7,9343 = 117; p < 0.001). Learning effects were observed in all conditions. Novice users first learned the task and then the dynamics of the manipulator.The findings showed differences between the novices and the experienced surgeons for extremely simple point-to-point movements. The study demonstrated that manipulator dynamics affect user movements, suggesting that these dynamics could be improved in future robot designs. The authors showed the partial adaptation of novice users to the dynamics. Future studies are needed to evaluate whether it will be beneficial to include early training sessions dedicated to learning the dynamics of the manipulator.
View details for DOI 10.1007/s00464-014-3446-5
View details for PubMedID 24519031
-
Mapping stiffness perception in the brain with an fMRI-compatible particle-jamming haptic interface.
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference
2014; 2014: 2051-2056
Abstract
We demonstrate reliable neural responses to changes in haptic stiffness perception using a functional magnetic resonance imaging (fMRI) compatible particle-jamming haptic interface. Our haptic interface consists of a silicone tactile surface whose stiffness we can control by modulating air-pressure in a sub-surface pouch of coarsely ground particles. The particles jam together as the pressure decreases, which stiffens the surface. During fMRI acquisition, subjects performed a constant probing task, which involved continuous contact between the index fingertip and the interface and rhythmic increases and decreases in fingertip force (1.6 Hz) to probe stiffness. Without notifying subjects, we randomly switched the interface's stiffness (switch time, 300-500 ms) from soft (200 N/m) to hard (1400 N/m). Our experiment design's constant motor activity and cutaneous tactile sensation helped disassociate neural activation for both from stiffness perception, which helped localized it to a narrow region in somatosensory cortex near the supra-marginal gyrus. Testing different models of neural activation, we found that assuming indepedent stiffness-change responses at both soft-hard and hard-soft transitions provides the best explanation for observed fMRI measurements (three subjects; nine four-minute scan runs each). Furthermore, we found that neural activation related to stiffness-change and absolute stiffness can be localized to adjacent but disparate anatomical locations. We also show that classical finger-tapping experiments activate a swath of cortex and are not suitable for localizing stiffness perception. Our results demonstrate that decorrelating motor and sensory neural activation is essential for characterizing somatosensory cortex, and establish particle-jamming haptics as an attractive low-cost method for fMRI experiments.
View details for DOI 10.1109/EMBC.2014.6944019
View details for PubMedID 25570387
-
Real-Time 3D Curved Needle Segmentation Using Combined B-Mode and Power Doppler Ultrasound
17th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
SPRINGER-VERLAG BERLIN. 2014: 381–388
Abstract
This paper presents a real-time segmentation method for curved needles in biological tissue based on analysis of B-mode and power Doppler images from a tracked 2D ultrasound transducer. Mechanical vibration induced by an external voice coil results in a Doppler response along the needle shaft, which is centered around the needle section in the ultrasound image. First, B-mode image analysis is performed within regions of interest indicated by the Doppler response to create a segmentation of the needle section in the ultrasound image. Next, each needle section is decomposed into a sequence of points and transformed into a global coordinate system using the tracked transducer pose. Finally, the 3D shape is reconstructed from these points. The results of this method differ from manual segmentation by 0.71 ± 0.55 mm in needle tip location and 0.38 ± 0.27 mm along the needle shaft. This method is also fast, taking 5-10 ms to run on a standard PC, and is particularly advantageous in robotic needle steering, which involves thin, curved needles with poor echogenicity.
View details for Web of Science ID 000347686400048
-
Real-time 3D curved needle segmentation using combined B-mode and power Doppler ultrasound.
Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention
2014; 17: 381-388
Abstract
This paper presents a real-time segmentation method for curved needles in biological tissue based on analysis of B-mode and power Doppler images from a tracked 2D ultrasound transducer. Mechanical vibration induced by an external voice coil results in a Doppler response along the needle shaft, which is centered around the needle section in the ultrasound image. First, B-mode image analysis is performed within regions of interest indicated by the Doppler response to create a segmentation of the needle section in the ultrasound image. Next, each needle section is decomposed into a sequence of points and transformed into a global coordinate system using the tracked transducer pose. Finally, the 3D shape is reconstructed from these points. The results of this method differ from manual segmentation by 0.71 ± 0.55 mm in needle tip location and 0.38 ± 0.27 mm along the needle shaft. This method is also fast, taking 5-10 ms to run on a standard PC, and is particularly advantageous in robotic needle steering, which involves thin, curved needles with poor echogenicity.
View details for PubMedID 25485402
-
Integration of a Particle Jamming Tactile Display with a Cable-Driven Parallel Robot
9th International Conference of the EuroHaptics on Neuroscience, Devices, Modeling, and Applications
SPRINGER-VERLAG BERLIN. 2014: 258–265
View details for DOI 10.1007/978-3-662-44196-1_32
View details for Web of Science ID 000345644600032
-
Recursive Estimation of Needle Pose for Control of 3D-Ultrasound-Guided Robotic Needle Steering
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
IEEE. 2014: 4303–4308
View details for Web of Science ID 000349834604062
-
Cerebellar ataxia impairs modulation of arm stiffness during postural maintenance
JOURNAL OF NEUROPHYSIOLOGY
2013; 110 (7): 1611-1620
Abstract
Impedance control enables humans to effectively interact with their environment during postural and movement tasks, adjusting the mechanical behavior of their limbs to account for instability. Previous work has shown that people are able to selectively modulate the end-point stiffness of their arms, adjusting for varying directions of environmental disturbances. Behavioral studies also suggest that separate controllers are used for impedance modulation versus joint torque coordination. Here we tested whether people with cerebellar damage have deficits in impedance control. It is known that these individuals have poor motor coordination, which has typically been attributed to deficits in joint torque control. Subjects performed a static postural maintenance task with two different types of directional force perturbations. On average, patients with cerebellar ataxia modified stiffness differentially for the two perturbation conditions, although significantly less than age-matched control subjects. Thus cerebellar damage may impair the ability to modulate arm impedance. Surprisingly, the patients' intact ability to generally alter their limb stiffness during the postural task (albeit less than age-matched control subjects) improved their movement performance in a subsequent tracing task. The transfer of stiffness control from the static to the movement task may be a strategy that can be used by patients to compensate for their motor deficits.
View details for DOI 10.1152/jn.00294.2013
View details for Web of Science ID 000325346300015
View details for PubMedID 23843434
-
Predictive modeling by the cerebellum improves proprioception.
journal of neuroscience
2013; 33 (36): 14301-14306
Abstract
Because sensation is delayed, real-time movement control requires not just sensing, but also predicting limb position, a function hypothesized for the cerebellum. Such cerebellar predictions could contribute to perception of limb position (i.e., proprioception), particularly when a person actively moves the limb. Here we show that human cerebellar patients have proprioceptive deficits compared with controls during active movement, but not when the arm is moved passively. Furthermore, when healthy subjects move in a force field with unpredictable dynamics, they have active proprioceptive deficits similar to cerebellar patients. Therefore, muscle activity alone is likely insufficient to enhance proprioception and predictability (i.e., an internal model of the body and environment) is important for active movement to benefit proprioception. We conclude that cerebellar patients have an active proprioceptive deficit consistent with disrupted movement prediction rather than an inability to generally enhance peripheral proprioceptive signals during action and suggest that active proprioceptive deficits should be considered a fundamental cerebellar impairment of clinical importance.
View details for DOI 10.1523/JNEUROSCI.0784-13.2013
View details for PubMedID 24005283
View details for PubMedCentralID PMC3761044
-
Cerebellar motor learning: are environment dynamics more important than error size?
JOURNAL OF NEUROPHYSIOLOGY
2013; 110 (2): 322-333
Abstract
Cerebellar damage impairs the control of complex dynamics during reaching movements. It also impairs learning of predictable dynamic perturbations through an error-based process. Prior work suggests that there are distinct neural mechanisms involved in error-based learning that depend on the size of error experienced. This is based, in part, on the observation that people with cerebellar degeneration may have an intact ability to learn from small errors. Here we studied the relative effect of specific dynamic perturbations and error size on motor learning of a reaching movement in patients with cerebellar damage. We also studied generalization of learning within different coordinate systems (hand vs. joint space). Contrary to our expectation, we found that error size did not alter cerebellar patients' ability to learn the force field. Instead, the direction of the force field affected patients' ability to learn, regardless of whether the force perturbations were introduced gradually (small error) or abruptly (large error). Patients performed best in fields that helped them compensate for movement dynamics associated with reaching. However, they showed much more limited generalization patterns than control subjects, indicating that patients rely on a different learning mechanism. We suggest that patients typically use a compensatory strategy to counteract movement dynamics. They may learn to relax this compensatory strategy when the external perturbation is favorable to counteracting their movement dynamics, and improve reaching performance. Altogether, these findings show that dynamics affect learning in cerebellar patients more than error size.
View details for DOI 10.1152/jn.00745.2012
View details for Web of Science ID 000321843800005
View details for PubMedID 23596337
View details for PubMedCentralID PMC3727069
-
Novel algorithm for real-time onset detection of surface electromyography in step-tracking wrist movements.
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference
2013; 2013: 2056-2059
Abstract
We present a novel algorithm for real-time detection of the onset of surface electromyography signal in step-tracking wrist movements. The method identifies abrupt increase of the quasi-tension signal calculated from sEMG resulting from the step-by-step recruitment of activated motor units. We assessed the performance of our proposed algorithm using both simulated and real sEMG signals, and compared with two existing detection methods. Evaluation with simulated sEMG showed that the detection accuracy of our method is robust to different signal-to-noise ratios, and that it outperforms the existing methods in terms of bias when the noise is large (low SNR). Evaluation with real sEMG analysis also indicated better detection performance compared to existing methods.
View details for DOI 10.1109/EMBC.2013.6609936
View details for PubMedID 24110123
-
The effect of a robot-assisted surgical system on the kinematics of user movements.
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference
2013; 2013: 6257-6260
Abstract
Teleoperated robot-assisted surgery (RAS) offers many advantages over traditional minimally invasive surgery. However, RAS has not yet realized its full potential, and it is not clear how to optimally train surgeons to use these systems. We hypothesize that the dynamics of the master manipulator impact the ability of users to make desired movements with the robot. We compared freehand and teleoperated movements of novices and experienced surgeons. To isolate the effects of dynamics from procedural knowledge, we chose simple movements rather than surgical tasks. We found statistically significant effects of teleoperation and user expertise in several aspects of motion, including target acquisition error, movement speed, and movement smoothness. Such quantitative assessment of human motor performance in RAS can impact the design of surgical robots, their control, and surgeon training methods, and eventually, improve patient outcomes.
View details for DOI 10.1109/EMBC.2013.6610983
View details for PubMedID 24111170
-
Adaptation to visuomotor rotation in isometric reaching is similar to movement adaptation.
IEEE ... International Conference on Rehabilitation Robotics : [proceedings]
2013; 2013: 1-6
Abstract
Isometric reaching, in which the arm remains stationary and the user controls a virtual cursor via force input, is a motor task that has not been thoroughly compared to real reaching. In this study, we ask if isometric adaptation to a kinematic perturbation is similar to adaptation in movement, and if the type of isometric mapping (position or velocity control) influences learning. Healthy subjects made real and virtual reaches with the arm in plane. In some trials, the cursor was rotated counter clockwise by 45° to perturb the kinematic mapping. To assess adaptation, the angular error of cursor movement at 150 ms from movement onset was measured for each reach; error was averaged across subjects and a two-state learning mode was fit to error data. For movement and isometric groups, average angular error peaked at perturbation onset, reduced over 200 reaches, and reversed direction when the perturbation was removed. We show that subjects are able to adapt to a visuomotor rotation in both position- and velocity-based cursor control, and that the time course of adaptation resembles that of movement adaptation. Training of virtual reaching using force/torque input could be particularly applicable for stroke patients with significant movement deficits, who could benefit from intensive treatments using simple, cost-effective devices.
View details for DOI 10.1109/ICORR.2013.6650431
View details for PubMedID 24187249
-
Effect of age on stiffness modulation during postural maintenance of the arm.
IEEE ... International Conference on Rehabilitation Robotics : [proceedings]
2013; 2013: 1-6
Abstract
The ability to modify the mechanical impedance of our limbs allows us to perform a variety of motor control tasks while interacting with the environment in a stable manner. Prior work has shown that young, healthy people are capable of modulating arm stiffness via selective muscle co-contraction to account for external disturbances in various directions. Increased age detrimentally affects control of movement and stability, although the neural mechanisms underlying these deficits are not entirely understood. In this study, younger and older subjects performed a static postural maintenance task with two types of directional force perturbations. Older individuals showed significantly less stiffness modification between the two perturbation conditions compared to the younger individuals, indicating less optimal modulation of arm impedance. This impairment should be considered during motor control evaluation in older populations, whether it be activities of daily living or skill assessment.
View details for DOI 10.1109/ICORR.2013.6650395
View details for PubMedID 24187214
-
Characterization and Psychophysical Studies of an Air-Jet Lump Display
IEEE TRANSACTIONS ON HAPTICS
2013; 6 (2): 156-166
Abstract
Development of tactile displays to enhance palpation of lumps during robot-assisted minimally invasive surgery is challenging due to size and weight constraints, motivating a pneumatic actuation strategy. This work describes the quantitative and psychophysical assessment of an air-jet tactile display that creates a lump percept by directing pressurized air through an aperture onto the finger. The air pressure and aperture size are meant to control the hardness and size, respectively, of the perceived lump. Jet impingement pressure and flow rate were measured by capacitive tactile sensors and mass flow meters at varying aperture sizes and pressures. The air-jet pressure profile width evolves as jet theory predicts and is largely independent of supply pressure (and therefore jet exit velocity). The method of constant stimuli was used to determine the just noticeable differences (JNDs) for the air pressure and aperture size. Qualitative results indicate that subjects perceive the stimulus as a "lump-like" shape. Pressure JNDs ranged from 19.6-24.4 kPag and aperture size JNDs ranged from 0.50-0.66 mm. No significant correlation exists between the supply pressure and changes in perceived lump size. However, pressure JNDs show significant (p < 0.001) inverse correlation with aperture size, with improved discrimination at larger apertures, where a greater finger pad area is stimulated.
View details for DOI 10.1109/ToH.2012.71
View details for Web of Science ID 000319877500003
-
Characterization and psychophysical studies of an air-jet lump display.
IEEE transactions on haptics
2013; 6 (2): 156-166
Abstract
Development of tactile displays to enhance palpation of lumps during robot-assisted minimally invasive surgery is challenging due to size and weight constraints, motivating a pneumatic actuation strategy. This work describes the quantitative and psychophysical assessment of an air-jet tactile display that creates a lump percept by directing pressurized air through an aperture onto the finger. The air pressure and aperture size are meant to control the hardness and size, respectively, of the perceived lump. Jet impingement pressure and flow rate were measured by capacitive tactile sensors and mass flow meters at varying aperture sizes and pressures. The air-jet pressure profile width evolves as jet theory predicts and is largely independent of supply pressure (and therefore jet exit velocity). The method of constant stimuli was used to determine the just noticeable differences (JNDs) for the air pressure and aperture size. Qualitative results indicate that subjects perceive the stimulus as a "lump-like" shape. Pressure JNDs ranged from 19.6-24.4 kPag and aperture size JNDs ranged from 0.50-0.66 mm. No significant correlation exists between the supply pressure and changes in perceived lump size. However, pressure JNDs show significant (p < 0.001) inverse correlation with aperture size, with improved discrimination at larger apertures, where a greater finger pad area is stimulated.
View details for DOI 10.1109/TOH.2012.71
View details for PubMedID 24808300
-
Does a basic deficit in force control underlie cerebellar ataxia?
JOURNAL OF NEUROPHYSIOLOGY
2013; 109 (4): 1107-1116
Abstract
Because damage to the cerebellum results in characteristic movement incoordination known as "ataxia," it has been hypothesized that it is involved in estimation of limb dynamics that occur during movement. However, cerebellar function may extend beyond movement to force control in general, with or without movement. Here we tested whether the cerebellum is involved in controlling force separate from estimating limb dynamics and whether ataxia could result from a deficit in force control. We studied patients with cerebellar ataxia controlling their arm force isometrically; in this condition arm dynamics are absent and there is no need for (or effect from an impairment in) estimations of limb dynamics. Subjects were required to control their force magnitude, direction, or both. Cerebellar patients were able to match force magnitude or direction similarly to control subjects. Furthermore, when controlling force magnitude, they intuitively chose directions (not specified) that required minimal effort at the joint level--this ability was also similar to control subjects. In contrast, cerebellar patients performed significantly worse than control subjects when asked to match both force magnitude and direction. This was surprising, since they did not exhibit significant impairment in doing either in isolation. These results show that cerebellum-dependent computations are not limited to estimations of body dynamics needed for active movement. Deficits occur even in isometric conditions, but apparently only when multiple degrees of freedom must be controlled simultaneously. Thus a fundamental cerebellar operation may be combining/coordinating degrees of freedom across many kinds of movements and behaviors.
View details for DOI 10.1152/jn.00152.2012
View details for Web of Science ID 000315141100021
View details for PubMedID 23175807
View details for PubMedCentralID PMC3569121
-
Coaxial Needle Insertion Assistant With Enhanced Force Feedback
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2013; 60 (2): 379-389
Abstract
Many medical procedures involving needle insertion into soft tissues, such as anesthesia, biopsy, brachytherapy, and placement of electrodes, are performed without image guidance. In such procedures, haptic detection of changing tissue properties at different depths during needle insertion is important for needle localization and detection of subsurface structures. However, changes in tissue mechanical properties deep inside the tissue are difficult for human operators to sense, because the relatively large friction force between the needle shaft and the surrounding tissue masks the smaller tip forces. A novel robotic coaxial needle insertion assistant, which enhances operator force perception, is presented. This one-degree-of-freedom cable-driven robot provides to the operator a scaled version of the force applied by the needle tip to the tissue, using a novel design and sensors that separate the needle tip force from the shaft friction force. The ability of human operators to use the robot to detect membranes embedded in artificial soft tissue was tested under the conditions of 1) tip force and shaft force feedback, and 2) tip force only feedback. The ratio of successful to unsuccessful membrane detections was significantly higher (up to 50%) when only the needle tip force was provided to the user.
View details for DOI 10.1109/TBME.2012.2227316
View details for Web of Science ID 000316809800013
View details for PubMedID 23193302
-
Kinematic analysis of motor performance in robot-assisted surgery: a preliminary study.
Studies in health technology and informatics
2013; 184: 302-308
Abstract
The inherent dynamics of the master manipulator of a teleoperated robot-assisted surgery (RAS) system can affect the movements of a human operator, in comparison with free-space movements. To measure the effects of these dynamics on operators with differing levels of surgical expertise, a da Vinci Si system was instrumented with a custom surgeon grip fixture and magnetic pose trackers. We compared users' performance of canonical motor control movements during teleoperation with the manipulator and freehand cursor control, and found significant differences in several aspects of motion, including target acquisition error, movement speed, and acceleration. In addition, there was preliminary evidence for differences between experts and novices. These findings could impact robot design, control, and training methods for RAS.
View details for PubMedID 23400175
-
Autonomous Robotic Palpation: Machine Learning Techniques to Identify Hard Inclusions in Soft Tissues
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2013: 4384–4389
View details for Web of Science ID 000337617304060
-
A HAPTIC SYSTEM FOR EDUCATIONAL GAMES: DESIGN AND APPLICATION-SPECIFIC KINEMATIC OPTIMIZATION
ASME Dynamic Systems and Control Conference (DSCC)
AMER SOC MECHANICAL ENGINEERS. 2013
View details for DOI 10.1115/DSCC2013-4077
View details for Web of Science ID 000337043200054
-
MODEL-MEDIATED TELEOPERATION WITH PREDICTIVE MODELS AND RELATIVE TRACKING
ASME Dynamic Systems and Control Conference (DSCC)
AMER SOC MECHANICAL ENGINEERS. 2013
View details for DOI 10.1115/DSCC2013-3842
View details for Web of Science ID 000337043200051
-
ROBOT-GUIDED SHEATHS (RoGS) FOR PERCUTANEOUS ACCESS TO THE PEDIATRIC KIDNEY: PATIENT-SPECIFIC DESIGN AND PRELIMINARY RESULTS
ASME Dynamic Systems and Control Conference
AMER SOC MECHANICAL ENGINEERS. 2013
View details for DOI 10.1115/DSCC2013-3917
View details for Web of Science ID 000335377800043
-
A Framework for Analysis of Surgeon Arm Posture Variability in Robot-Assisted Surgery
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2013: 245–251
View details for Web of Science ID 000337617300036
-
Tissue Fixation by Suction Increases the Accuracy of Robotic Needle Insertion
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2013: 1694–1699
View details for Web of Science ID 000337617301104
-
Sensory Substitution via Cutaneous Skin Stretch Feedback
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2013: 2341–2346
View details for Web of Science ID 000337617302052
- Sensorimotor Performance in Robot Assisted Surgery. 2013
- Arm Stiffness in Patients with Cerebellar Ataxia: Modulation during Postural Maintenance and its Subsequent Effects on Movement. 2013
- Sensory Substitution via Cutaneous Skin Stretch Feedback. 2013
- A Haptic System for Educational Games: Design and Application-Specific Kinematic Optimization. 2013
- A Framework for Analysis of Surgeon Arm Posture Variability in Robot-Assisted Surgery. 2013
-
Haptic Jamming: A Deformable Geometry, Variable Stiffness Tactile Display using Pneumatics and Particle Jamming
IEEE World Haptics Conference (WHC)
IEEE. 2013: 25–30
View details for Web of Science ID 000325187400005
-
Sensory Augmentation of Stiffness using Fingerpad Skin Stretch
IEEE World Haptics Conference (WHC)
IEEE. 2013: 467–472
View details for Web of Science ID 000325187400079
-
Cartesian and Joint Space Teleoperation for Nonholonomic Steerable Needles
IEEE World Haptics Conference (WHC)
IEEE. 2013: 395–400
View details for Web of Science ID 000325187400067
- Novel Algorithm for Real-Time Onset Detection of Surface Electromyography in Step-Tracking Wrist Movements. 2013
- Cerebellar Ataxia Impairs Modulation of Arm Stiffness during Postural Maintenance. Journal of Neurophysiology 2013; 7 (110): 1611-1620
- A Haptic Display for Medical Simulation Using Particle Jamming. Medicine Meets Virtual Reality (Studies in Health Technology and Information). 2013
- 3D Segmentation of Curved Needles using Doppler Ultrasound and Vibration. Information Processing in Computer-Assisted Interventions 2013: 61-70
- Kinematic Analysis of Motor Performance in Robot-Assisted Surgery: Implications for Robot Design and Surgical Training. 2013
- Robot-Guided Sheaths (RoGS) for Percutaneous Access to the Pedatric Kidney: Patient-Specific Design and Preliminary Results. 2013
-
Effect of Age on Stiffness Modulation during Postural Maintenance of the Arm.
2013
View details for DOI 10.1109/ICORR.2014.6650395
- Autonomous Robotic Palpation: Machine Learning Techniques to Identify Hard Inclusions in Soft Tissues. 2013
- Adaptation to Visuomotor Rotation in Isometric Virtual Reaching Parallels Learning in Movement. 2013
- 3D Segmentation of Curved Needles using Doppler Ultrasound and Vibration. Information Processing in Computer-Assisted Interventions 2013: 61-70
- Transfer of isometric motor learning depends on the mapping of force input to cursor movement. 2013
- Tissue Fixation by Suction Increases the Accuracy of Robotic Needle Insertion. 2013
- Model-Mediated Teleoperation with Predictive Models and Relative Tracking. 2013
-
Grip Force Control During Virtual Object Interaction: Effect of Force Feedback, Accuracy Demands, and Training.
IEEE Transactions on Haptics
2013
View details for DOI 10.1109/TOH.2014.60
-
Perception of Springs With Visual and Proprioceptive Motion Cues: Implications for Prosthetics
IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
2013; 43 (1): 102-114
View details for DOI 10.1109/TSMCA.2012.2221038
View details for Web of Science ID 000317644800010
-
Behavior of Tip-Steerable Needles in Ex Vivo and In Vivo Tissue
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2012; 59 (10): 2705-2715
Abstract
Robotic needle steering is a promising technique to improve the effectiveness of needle-based clinical procedures, such as biopsies and ablation, by computer-controlled, curved insertions of needles within solid organs. In this paper, we explore the capabilities, challenges, and clinical relevance of asymmetric-tip needle steering through experiments in ex vivo and in vivo tissue. We evaluate the repeatability of needle insertion in inhomogeneous biological tissue and compare ex vivo and in vivo needle curvature and insertion forces. Steerable needles curved more in kidney than in liver and prostate, likely due to differences in tissue properties. Pre-bent needles produced higher insertion forces in liver and more curvature in vivo than ex vivo. When compared to straight stainless steel needles, steerable needles did not cause a measurable increase in tissue damage and did not exert more force during insertion. The minimum radius of curvature achieved by prebent needles was 5.23 cm in ex vivo tissue, and 10.4 cm in in vivo tissue. The curvatures achieved by bevel tip needles were negligible for in vivo tissue. The minimum radius of curvature for bevel tip needles in ex vivo tissue was 16.4 cm; however, about half of the bevel tip needles had negligible curvatures. We also demonstrate a potential clinical application of needle steering by targeting and ablating overlapping regions of cadaveric canine liver.
View details for DOI 10.1109/TBME.2012.2204749
View details for Web of Science ID 000308989000003
View details for PubMedID 22711767
View details for PubMedCentralID PMC3448818
-
Active force perception depends on cerebellar function
JOURNAL OF NEUROPHYSIOLOGY
2012; 107 (6): 1612-1620
Abstract
Damage to the cerebellum causes characteristic movement abnormalities but is thought to have minimal impact on somatosensory perception. Traditional clinical assessments of patients with cerebellar lesions reveal no perceptual deficits despite the fact that the cerebellum receives substantial somatosensory information. Given that abnormalities have been reported in predicting the visual consequences of movement, we suspect that the cerebellum broadly participates in perception when motor output is required (i.e., active perception). Thus we hypothesize that cerebellar integrity is essential for somatosensory perception that requires motor activity, but not passive somatosensory perception. We compared the perceptual acuity of human cerebellar patients to that of healthy control subjects in several different somatosensory perception tasks with minimal visual information. We found that patients were worse at active force and stiffness discrimination but similar to control subjects with regard to passive cutaneous force detection, passive proprioceptive detection, and passive proprioceptive discrimination. Furthermore, the severity of movement symptoms as assessed by a clinical exam was positively correlated with impairment of active force perception. Notably, within the context of these perceptual tasks, control subjects and cerebellar patients displayed similar movement characteristics, and hence differing movement strategies are unlikely to underlie the differences in perception. Our results are consistent with the hypothesis that the cerebellum is vital to sensory prediction of self-generated movement and suggest a general role for the cerebellum in multiple forms of active perception.
View details for DOI 10.1152/jn.00983.2011
View details for Web of Science ID 000302140300008
View details for PubMedID 22190620
-
Augmented reality and haptic interfaces for robot-assisted surgery
INTERNATIONAL JOURNAL OF MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERY
2012; 8 (1): 45-56
Abstract
Current teleoperated robot-assisted minimally invasive surgical systems do not take full advantage of the potential performance enhancements offered by various forms of haptic feedback to the surgeon. Direct and graphical haptic feedback systems can be integrated with vision and robot control systems in order to provide haptic feedback to improve safety and tissue mechanical property identification.An interoperable interface for teleoperated robot-assisted minimally invasive surgery was developed to provide haptic feedback and augmented visual feedback using three-dimensional (3D) graphical overlays. The software framework consists of control and command software, robot plug-ins, image processing plug-ins and 3D surface reconstructions.The feasibility of the interface was demonstrated in two tasks performed with artificial tissue: palpation to detect hard lumps and surface tracing, using vision-based forbidden-region virtual fixtures to prevent the patient-side manipulator from entering unwanted regions of the workspace.The interoperable interface enables fast development and successful implementation of effective haptic feedback methods in teleoperation.
View details for DOI 10.1002/rcs.421
View details for Web of Science ID 000301120100005
View details for PubMedID 22069247
-
User comprehension of task performance with varying impedance in a virtual prosthetic arm: A pilot study
4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) / Symposium on Surgical Robotics
IEEE. 2012: 500–507
View details for Web of Science ID 000313014600085
- Haptic Footstep Display. 2012
- Development of the KineSys MedSim: A Novel Hands-Free Haptic Robot for Medical Simulation. Medicine Meets Virtual Reality (Studies in Health Technology and Information). 2012
- Haptic feedback enhances rhythmic motor control performance. 2012
- Predicting and correcting human ataxia using a model of cerebellar function. 2012
- HAPI Bands: A Haptic Augmented Posture Interface. 2012
- Design and Control of an Air-Jet Lump Display. 2012
- Conveying the Configuration of a Virtual Human Hand Using Vibrotactile Feedback. 2012
- M-Width: Stability and Accuracy of Haptic Rendering of Virtual Mass. In Robotics: Science and Systems. 2012
- Discrimination of Springs with Vision, Proprioception, and Artificial Skin Stretch Cues. 2012
- Augmented Reality and Haptic Interfaces for Robot-Assisted Surgery. International Journal of Medical Robotics and Computer Assisted Surgery 2012; 1 (8): 45-46
-
Characterization of Robotic Needle Insertion and Rotation in Artificial and Ex Vivo Tissues
4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) / Symposium on Surgical Robotics
IEEE. 2012: 62–68
View details for Web of Science ID 000313014600011
-
Wearable Haptic Device for Cutaneous Force and Slip Speed Display
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2012: 1038–1043
View details for Web of Science ID 000309406701007
-
Robot-Assisted Needle Steering
IEEE ROBOTICS & AUTOMATION MAGAZINE
2011; 18 (4): 35-46
Abstract
Needle insertion is a critical aspect of many medical treatments, diagnostic methods, and scientific studies, and is considered to be one of the simplest and most minimally invasive medical procedures. Robot-assisted needle steering has the potential to improve the effectiveness of existing medical procedures and enable new ones by allowing increased accuracy through more dexterous control of the needle tip path and acquisition of targets not accessible by straight-line trajectories. In this article, we describe a robot-assisted needle steering system that uses three integrated controllers: a motion planner concerned with guiding the needle around obstacles to a target in a desired plane, a planar controller that maintains the needle in the desired plane, and a torsion compensator that controls the needle tip orientation about the axis of the needle shaft. Experimental results from steering an asymmetric-tip needle in artificial tissue demonstrate the effectiveness of the system and its sensitivity to various environmental and control parameters. In addition, we show an example of needle steering in ex vivo biological tissue to accomplish a clinically relevant task, and highlight challenges of practical needle steering implementation.
View details for DOI 10.1109/MRA.2011.942997
View details for Web of Science ID 000297994600010
View details for PubMedCentralID PMC3460644
-
Haptics in Medicine and Clinical Skill Acquisition
IEEE TRANSACTIONS ON HAPTICS
2011; 4 (3): 153-154
View details for Web of Science ID 000293455500001
-
Experimental evaluation of a coaxial needle insertion assistant with enhanced force feedback
33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS)
IEEE. 2011: 3447–3450
Abstract
During needle insertion in soft tissue, detection of change in tissue properties is important both for diagnosis to detect pathological tissue and for prevention to avoid puncture of important structures. The presence of a membrane located deep inside the tissue results in a relatively small force variation at the needle tip that can be masked by relatively large friction force between the needle shaft and the surrounding tissue. Also, user perception of force can be limited due to the overall small force amplitude in some applications (e.g. brain surgery). A novel robotic coaxial needle insertion assistant was developed to enhance operator force perception. The coaxial needle separates the cutting force at the needle tip from shear friction on the needle shaft. The assistant is force controlled (admittance control), providing the operator with force feedback that is a scaled version of the force applied by the needle tip to the tissue. The effectiveness of the assistant in enhancing the detection of different tissue types was tested experimentally. Users were asked to blindly insert a needle into artificial tissues with membranes at various depths under two force feedback conditions: (1) shaft and tip force together, and (2) only tip force. The ratio of successful to unsuccessful membrane detection was significantly higher when only the needle tip force is displayed to the user. The system proved to be compliant with the clinical applications requirements.
View details for Web of Science ID 000298810002292
View details for PubMedID 22255081
- Robotic Needle Steering: Design, Modeling, Planning, and Image Guidance. In Surgical Robotics - Systems, Applications, and Visions. edited by Rosen, J., Hannaford, B., Satava, R. Springer. 2011: 557–582
-
Task-Dependent Impedance Improves User Performance with a Virtual Prosthetic Arm
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2011: 2235–2242
View details for Web of Science ID 000324383401071
-
Characterization of an Air Jet Haptic Lump Display
33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS)
IEEE. 2011: 3467–3470
Abstract
During manual palpation, clinicians rely on distributed tactile information to identify and localize hard lumps embedded in soft tissue. The development of tactile feedback systems to enhance palpation using robot-assisted minimally invasive surgery (RMIS) systems is challenging due to size and weight constraints, motivating a pneumatic actuation strategy. Recently, an air jet approach has been proposed for generating a lump percept. We use this technique to direct a thin stream of air through an aperture directly on the finger pad, which indents the skin in a hemispherical manner, producing a compelling lump percept. We hypothesize that the perceived parameters of the lump (e.g. size and stiffness) can be controlled by jointly adjusting air pressure and the aperture size through which air escapes. In this work, we investigate how these control variables interact to affect perceived pressure on the finger pad. First, we used a capacitive tactile sensor array to measure the effect of aperture size on output pressure, and found that peak output pressure increases with aperture size. Second, we performed a psychophysical experiment for each aperture size to determine the just noticeable difference (JND) of air pressure on the finger pad. Subject-averaged pressure JND values ranged from 19.4-24.7 kPa, with no statistical differences observed between aperture sizes. The aperture-pressure relationship and the pressure JND values will be fundamental for future display control.
View details for Web of Science ID 000298810002297
View details for PubMedID 22255086
-
Assessing the Quality of Force Feedback in Soft Tissue Simulation
33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS)
IEEE. 2011: 3451–3454
Abstract
Many types of deformable models have been proposed for simulation of soft tissue in surgical simulators, but their realism in comparison to actual tissue is rarely assessed. In this paper, a nonlinear mass-spring model is used for realtime simulation of deformable soft tissues and providing force feedback to a human operator. Force-deformation curves of real soft tissue samples were obtained experimentally, and the model was tuned accordingly. To test the realism of the model, we conducted two human-user experiments involving palpation with a rigid probe. First, in a discrimination test, users identified the correct category of real and virtual tissue better than chance, and tended to identify the tissues as real more often than virtual. Second, users identified real and virtual tissues by name, after training on only real tissues. The sorting accuracy was the same for both real and virtual tissues. These results indicate that, despite model limitations, the simulation could convey the feel of touching real tissues. This evaluation approach could be used to compare and validate various soft-tissue simulators.
View details for Web of Science ID 000298810002293
View details for PubMedID 22255082
-
Gradual anisometric-isometric transition for human-machine interfaces
33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS)
IEEE. 2011: 4507–4510
Abstract
Human-machine interfaces (HMIs) are widely used in biomedical applications, from teleoperated surgical systems to rehabilitation devices. This paper investigates a method of control that allows an HMI to transition from anisometric to isometric mode, shifting the control input from position to force as the user's movement is gradually reduced. Two different approaches for achieving this transition are discussed: one is based on the natural system dynamics, whereas the other involves selecting and controlling dynamics. The two approaches were implemented on a custom haptic device in a targeting task. Anisometric to isometric transitioning can potentially be used for training purposes, enabling transfer of what was learned in one mode to the other, as well as novel studies of the human sensorimotor system.
View details for Web of Science ID 000298810003208
View details for PubMedID 22255340
- Robot-Assisted Steerable Needles: Prototype Development and Feasibility Studies in an Animal Model. 2011
- Do Cerebellar Patients Generalize after Abrupt or Gradual Motor Learning? 2011
- Cerebellar predictions underlie the improvement of proprioceptive precision in active versus passive movements. 2011
- Task-Dependent Impedance Improves User Performance with a Virtual Prosthetic Arm. 2011
- Characterization of an Air Jet Haptic Lump Display. 2011
- Cutaneous Force Display via Shaped Contacts. 2011
- Force Feedback and Sensory Substitution for Robot-Assisted Surgery. Surgical Robotics - Systems, Applications, and Visions 2011: 419-448
-
Coaxial Needle Insertion Assistant for Epidural Puncture
IEEE/RSJ International Conference on Intelligent Robots and Systems
IEEE. 2011
View details for Web of Science ID 000297477502141
-
Design and Evaluation of a Multi-Modal Haptic Skin Stimulation Apparatus
33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS)
IEEE. 2011: 3455–3458
Abstract
Human grasping and manipulation are facilitated by cutaneous mechanoreceptors that provide information about contact location, pressure, and events such as making and breaking contact. A challenge in designing haptic feedback devices for the wearer of a prosthetic hand is simultaneous display of multiple types of haptic information. We present the preliminary design and evaluation of an apparatus for relaying multi-modal haptic information. The apparatus moves a set of contact points tangentially over the skin at a controlled speed, with controlled normal force. We apply this stimulus to an artificial skin instrumented with an embedded accelerometer, and characterize the resulting signals. Vibration frequency increases with applied normal force and tangential speed, whereas vibration amplitude increases with normal force and depends on skin properties. The results indicate that different forces and speeds can, under some conditions, be discriminated using vibration signals alone. Accurate identification of speeds is provided by series of vibration events that depend on the spatial distribution of contact points. This study motivates future work to perform human perception studies and create a wearable haptic display for prosthetics based on this concept.
View details for Web of Science ID 000298810002294
View details for PubMedID 22255083
-
Mechanics of Flexible Needles Robotically Steered through Soft Tissue
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2010; 29 (13): 1640-1660
View details for DOI 10.1177/0278364910369714
View details for Web of Science ID 000283847100004
-
Identifying the Role of Proprioception in Upper-Limb Prosthesis Control: Studies on Targeted Motion
ACM TRANSACTIONS ON APPLIED PERCEPTION
2010; 7 (3)
View details for DOI 10.1145/1773965.1773966
View details for Web of Science ID 000279361800001
-
Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine.
IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation
2010; 2010: 1733-1738
Abstract
Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.
View details for PubMedID 24748993
View details for PubMedCentralID PMC3988879
-
Evaluation of Robotic Needle Steering in ex vivo Tissue.
IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation
2010; 2010: 2068-2073
Abstract
Insertion velocity, tip asymmetry, and shaft diameter may influence steerable needle insertion paths in soft tissue. In this paper we examine the effects of these variables on needle paths in ex vivo goat liver, and demonstrate practical applications of robotic needle steering for ablation, biopsy, and brachytherapy. All experiments were performed using a new portable needle steering robot that steers asymmetric-tip needles under fluoroscopic imaging. For bevel-tip needles, we found that larger diameter needles resulted in less curvature, i.e. less steerability, confirming previous experiments in artificial tissue. The needles steered with radii of curvature ranging from 3:4 cm (for the most steerable pre-bent needle) to 2:97m (for the least steerable bevel needle). Pre-bend angle significantly affected needle curvature, but bevel angle did not. We hypothesize that biological tissue characteristics such as inhomogeneity and viscoelasticity significantly increase path variability. These results underscore the need for closed-loop image guidance for needle steering in biological tissues with complex internal structure.
View details for DOI 10.1109/ROBOT.2010.5509873
View details for PubMedID 21339851
View details for PubMedCentralID PMC3040792
-
Evaluation of Robotic Needle Steering in ex vivo Tissue
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2010: 2068–2073
View details for Web of Science ID 000284150004147
-
Estimation of Model Parameters for Steerable Needles.
IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation
2010: 3703-3708
Abstract
Flexible needles with bevel tips are being developed as useful tools for minimally invasive surgery and percutaneous therapy. When such a needle is inserted into soft tissue, it bends due to the asymmetric geometry of the bevel tip. This insertion with bending is not completely repeatable. We characterize the deviations in needle tip pose (position and orientation) by performing repeated needle insertions into artificial tissue. The base of the needle is pushed at a constant speed without rotating, and the covariance of the distribution of the needle tip pose is computed from experimental data. We develop the closed-form equations to describe how the covariance varies with different model parameters. We estimate the model parameters by matching the closed-form covariance and the experimentally obtained covariance. In this work, we use a needle model modified from a previously developed model with two noise parameters. The modified needle model uses three noise parameters to better capture the stochastic behavior of the needle insertion. The modified needle model provides an improvement of the covariance error from 26.1% to 6.55%.
View details for DOI 10.1109/ROBOT.2010.5509380
View details for PubMedID 21643451
View details for PubMedCentralID PMC3107577
-
Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2010: 1733–1738
View details for Web of Science ID 000284150002005
-
Estimation of Model Parameters for Steerable Needles
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2010: 3703–3708
View details for Web of Science ID 000284150001111
- Human vs. Robotic Tactile Sensing: Detecting Lumps in Soft Tissue. 2010
- Does a Basic Deficit in Force Control Underlie Cerebellar Ataxia? 2010
- Design of a Haptic Simulator for Osteosynthesis Screw Insertion. 2010
- A Proposed Method for Correcting Coordination Deficits: Models and Simulation. 2010
-
Defining Performance Tradeoffs for Multi-Degree-of-Freedom Bilateral Teleoperators with LQG Control
49th IEEE Conference on Decision and Control (CDC)
IEEE. 2010: 3542–3547
View details for Web of Science ID 000295049104016
-
Modelling of non-linear elastic tissues for surgical simulation
COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING
2010; 13 (6): 811-818
Abstract
Realistic modelling of the interaction between surgical instruments and human organs has been recognised as a key requirement in the development of high-fidelity surgical simulators. Primarily due to computational considerations, most of the past real-time surgical simulation research has assumed linear elastic behaviour for modelling tissues, even though human soft tissues generally possess non-linear properties. For a non-linear model, the well-known Poynting effect developed during shearing of the tissue results in normal forces not seen in a linear elastic model. Using constitutive equations of non-linear tissue models together with experiments, we show that the Poynting effect results in differences in force magnitude larger than the absolute human perception threshold for force discrimination in some tissues (e.g. myocardial tissues) but not in others (e.g. brain tissue simulants).
View details for DOI 10.1080/10255840903505121
View details for Web of Science ID 000285145800019
View details for PubMedID 20503126
- The Cerebellum Contributes to Constant and Dynamic Load Perception but not Position Perception. 2010
- Defining Performance Tradeoffs for Multi-Degree-of-Freedom Bilateral Teleoperators with LQG Control. 2010
- Medical and Healthcare Robotics: Achievements and Opportunities IEEE Robotics and Automation Magazine 2010; 3 (17): 26-37
- Neural coding of lump detection in soft tissue. 2010
- Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine. 2010
- Modeling of Nonlinear Elastic Tissues for Surgical Simulation. Computer Methods in Biomechanics and Biomedical Engineering 2010; 6 (13): 811-818
-
Haptics for Robot-Assisted Minimally Invasive Surgery
13th International Symposium on Robotics Research (ISSR)
SPRINGER-VERLAG BERLIN. 2010: 361–372
View details for Web of Science ID 000289498800030
- Haptic Feedback: Technology and Medical Applications. Handbook of Touch. 2010
-
Modeling and Control of Needles With Torsional Friction
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2009; 56 (12): 2905-2916
Abstract
A flexible needle can be accurately steered by robotically controlling the bevel tip orientation as the needle is inserted into tissue. Friction between the long, flexible needle shaft and the tissue can cause a significant discrepancy between the orientation of the needle tip and the orientation of the base where the needle angle is controlled. Our experiments show that several common phantom tissues used in needle steering experiments impart substantial friction forces to the needle shaft, resulting in a lag of more than 45 ( degrees ) for a 10 cm insertion depth in some phantoms; clinical studies report torques large enough to cause similar errors during needle insertions. Such angle discrepancies will result in poor performance or failure of path planners and image-guided controllers, since the needles used in percutaneous procedures are too small for state-of-the-art imaging to accurately measure the tip angle. To compensate for the angle discrepancy, we develop an estimator using a mechanics-based model of the rotational dynamics of a needle being inserted into tissue. Compared to controllers that assume a rigid needle in a frictionless environment, our estimator-based controller improves the tip angle convergence time by nearly 50% and reduces the path deviation of the needle by 70%.
View details for DOI 10.1109/TBME.2009.2029240
View details for Web of Science ID 000272042400016
View details for PubMedID 19695979
View details for PubMedCentralID PMC2859043
-
Feedback Control for Steering Needles Through 3D Deformable Tissue Using Helical Paths.
Robotics science and systems : online proceedings
2009; V: 37-?
Abstract
Bevel-tip steerable needles are a promising new technology for improving accuracy and accessibility in minimally invasive medical procedures. As yet, 3D needle steering has not been demonstrated in the presence of tissue deformation and uncertainty, despite the application of progressively more sophisticated planning algorithms. This paper presents a feedback controller that steers a needle along 3D helical paths, and varies the helix radius to correct for perturbations. It achieves high accuracy for targets sufficiently far from the needle insertion point; this is counterintuitive because the system is highly under-actuated and not locally controllable. The controller uses a model predictive control framework that chooses a needle twist rate such that the predicted helical trajectory minimizes the distance to the target. Fast branch and bound techniques enable execution at kilohertz rates on a 2GHz PC. We evaluate the controller under a variety of simulated perturbations, including imaging noise, needle deflections, and curvature estimation errors. We also test the controller in a 3D finite element simulator that incorporates deformation in the tissue as well as the needle. In deformable tissue examples, the controller reduced targeting error by up to 88% compared to open-loop execution.
View details for PubMedID 21179401
-
Controlling a Robotically Steered Needle in the Presence of Torsional Friction.
IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation
2009: 3476-3481
Abstract
A flexible needle can be accurately steered by robotically controlling the orientation of the bevel tip as the needle is inserted into tissue. Here, we demonstrate the significant effect of friction between the long, flexible needle shaft and the tissue, which can cause a significant discrepancy between the orientation of the needle tip and the orientation of the base where the needle is controlled. Our experiments show that several common phantom tissues used in needle steering experiments impart substantial frictional forces to the needle shaft, resulting in a lag of over 45° for a 10 cm insertion depth in some phantoms; clinical studies have reported torques large enough to could cause similar errors during needle insertions. Such angle discrepancies will result in poor performance or failure of path planners and image-guided controllers, since the needles used in percutaneous procedures are too small for state-of-the-art imaging to accurately measure the tip angle. To compensate for the angle discrepancy, we develop a model for the rotational dynamics of a needle being continuously inserted into tissue and show how a PD controller is sufficient to compensate for the rotational dynamics.
View details for PubMedID 21461175
-
The importance of organ geometry and boundary constraints for planning of medical interventions
MEDICAL ENGINEERING & PHYSICS
2009; 31 (2): 195-206
Abstract
Realistic modeling of medical interventions involving tool-tissue interactions has been considered to be a key requirement in the development of high-fidelity simulators and planners. Organ geometry, soft-tissue constitutive laws, and boundary conditions imposed by the connective tissues surrounding the organ are some of the factors that govern the accuracy of medical intervention planning. In this study it is demonstrated that, for needle path planning, the organ geometry and boundary constraints surrounding the organ are the most important factors influencing the deformation. As an example, the procedure of needle insertion into the prostate (e.g. for biopsy or brachytherapy) is considered. Image segmentation is used to extract the anatomical details from magnetic resonance images, while object-oriented finite element analysis (OOF) software is used to generate finite element (FE) meshes from the segmented images. Two-dimensional FE simulations that account for complex anatomical details along with relative motion between the prostate and its surrounding structure using cohesive zone models are compared with traditional simulation models having simple organ geometry and boundary constraints. Nodal displacements for these simpler models were observed to be up to 14 times larger than those obtained from the anatomically accurate models.
View details for DOI 10.1016/j.medengphy.2008.08.002
View details for Web of Science ID 000263872200005
View details for PubMedID 18815068
-
Stiffness Discrimination with Visual and Proprioceptive Cues
3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE. 2009: 121–126
View details for Web of Science ID 000266714300022
-
Observations and Models for Needle-Tissue Interactions
IEEE International Conference on Robotics and Automation
IEEE. 2009: 3423–3428
View details for Web of Science ID 000276080401237
-
Quantifying Perception of Nonlinear Elastic Tissue Models using Multidimensional Scaling
3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE. 2009: 570–575
View details for Web of Science ID 000266714300093
-
Force & Torque Feedback vs Force Only Feedback
3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE. 2009: 406–410
View details for Web of Science ID 000266714300066
-
Haptic feedback in robot-assisted minimally invasive surgery
CURRENT OPINION IN UROLOGY
2009; 19 (1): 102-107
Abstract
Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon and minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal. It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor. This review explains the technical challenges of creating haptic feedback for robot-assisted surgery and provides recent results that evaluate the effectiveness of haptic feedback in mock surgical tasks.Haptic feedback systems for RMIS are still under development and evaluation. Most provide only force feedback, with limited fidelity. The major challenge at this time is sensing forces applied to the patient. A few tactile feedback systems for RMIS have been created, but their practicality for clinical implementation needs to be shown. It is particularly difficult to sense and display spatially distributed tactile information. The cost-benefit ratio for haptic feedback in RMIS has not been established.The designs of existing commercial RMIS systems are not conducive for force feedback, and creative solutions are needed to create compelling tactile feedback systems. Surgeons, engineers, and neuroscientists should work together to develop effective solutions for haptic feedback in RMIS.
View details for DOI 10.1097/MOU.0b013e32831a478c
View details for Web of Science ID 000262215600020
View details for PubMedID 19057225
View details for PubMedCentralID PMC2701448
- Feedback Control for Steering Needles through 3D Deformable Tissue using Helical Paths. Robotics: Science and Systems. 2009
-
Observations of Needle-Tissue Interactions
Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society
IEEE. 2009: 262–265
Abstract
Needles with asymmetric bevel tips naturally bend when they are inserted into soft tissue. In this study, we present an analytical model for the loads developed at the bevel tip during needle-tissue interaction. The model calculates the loads based on the geometry of the bevel edge and gel material properties. The modeled transverse force developed at the tip is compared to forces measured experimentally. The analytical model explains the trends observed in the experiments. In addition to macroscopic studies, we also present microscopic observations of needle-tissue interactions. These results contribute to a mechanics-based model of robotic needle steering, extending previous work on kinematic models.
View details for Web of Science ID 000280543600070
View details for PubMedID 19963709
-
Design Considerations and Human-Machine Performance of Moving Virtual Fixtures
IEEE International Conference on Robotics and Automation
IEEE. 2009: 2958–2963
View details for Web of Science ID 000276080401164
-
Haptics as an Aid to Copying for People with Williams Syndrome
3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE. 2009: 356–361
View details for Web of Science ID 000266714300061
- Stiffness Discrimination with Visual and Proprioceptive Cues. 2009
- Can Cerebellar Dysmetria be Explained by an Incorrect Internal Model of Limb Dynamics? 2009
- Modeling and Control of Needles with Torsional Friction. IEEE Transactions on Biomedical Engineering 2009; 12 (56): 2905-2916
-
Controlling a Robotically Steered Needle in the Presence of Torsional Friction
IEEE International Conference on Robotics and Automation
IEEE. 2009: 3598–3603
View details for Web of Science ID 000276080401265
-
Environment Discrimination with Vibration Feedback to the Foot, Arm, and Fingertip
11th IEEE International Conference on Rehabilitation Robotics
IEEE. 2009: 398–403
View details for Web of Science ID 000277086500056
-
Tissue Property Estimation and Graphical Display for Teleoperated Robot-Assisted Surgery
IEEE International Conference on Robotics and Automation
IEEE. 2009: 3117–3123
View details for Web of Science ID 000276080401190
-
Effects of Haptic and Graphical Force Feedback on Teleoperated Palpation
IEEE International Conference on Robotics and Automation
IEEE. 2009: 3315–3320
View details for Web of Science ID 000276080401221
- Quantification and Reproduction of Human Hand Skin Stretch and its Effects on Proprioception. 2009
-
Characterization of Pre-Curved Needles for Steering in Tissue
Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society
IEEE. 2009: 1200–1203
Abstract
Needles with tip asymmetry deflect upon insertion into soft tissue, an effect that can be used to steer needles within the body. This paper presents a phenomenological characterization of the steering behavior of pre-curved needles, which have tip asymmetry due to curvature of the needle near the tip. We describe needle construction methods and a needle shaft triangulation algorithm to compute the shape of the needle based on images. Experimental results show that pre-curved needles possess greater dexterity than bevel-tipped needles and achieve radii of curvature similar to pre-bent needles. For long pre-curve arc lengths, the radius of curvature of the needle was found to approach the radius of curvature of the pre-curve. Pre-curved needles were found to display behaviors not seen with bevel-tipped needles, such as the insertion velocity influencing the path of the tip within the tissue and the ability to plastically deform the needle during steering.
View details for Web of Science ID 000280543601032
View details for PubMedID 19963994
-
Surgical and Interventional Robotics: Part III Surgical Assistance Systems
IEEE ROBOTICS & AUTOMATION MAGAZINE
2008; 15 (4): 84-93
View details for DOI 10.1109/MRA.2008.930401
View details for Web of Science ID 000261616400012
-
Integrated Planning and Image-Guided Control for Planar Needle Steering.
Proceedings of the ... IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics
2008; 2008: 819-824
Abstract
Flexible, tip-steerable needles promise to enhance physicians' abilities to accurately reach targets and maneuver inside the human body while minimizing patient trauma. Here, we present a functional needle steering system that integrates two components: (1) a patient-specific 2D pre- and intra-operative planner that finds an achievable route to a target within a planar slice of tissue (Stochastic Motion Roadmap), and (2) a low-level image-guided feedback controller that keeps the needle tip within that slice. The planner generates a sequence of circular arcs that can be realized by interleaving pure insertions with 180° rotations of the needle shaft. This preplanned sequence is updated in realtime at regular intervals. Concurrently, the low-level image-based controller servos the needle to remain close to the desired plane between plan updates. Both planner and controller are predicated on a previously developed kinematic nonholonomic model of bevel-tip needle steering. We use slighly different needles here that have a small bend near the tip, so we extend the model to account for discontinuities of the tip position caused by 180° rotations. Further, during large rotations of the needle base, we maintain the desired tip angle by compensating for torsional compliance in the needle shaft, neglected in previous needle steering work. By integrating planning, control, and torsion compensation, we demonstrate both accurate targeting and obstacle avoidance.
View details for PubMedID 20640197
-
Modeling of tool-tissue interactions for computer-based surgical simulation: A literature review
PRESENCE-TELEOPERATORS AND VIRTUAL ENVIRONMENTS
2008; 17 (5): 463-491
View details for Web of Science ID 000259865300003
-
Modeling of Tool-Tissue Interactions for Computer-Based Surgical Simulation: A Literature Review.
Presence (Cambridge, Mass.)
2008; 17 (5): 463
Abstract
Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in robot-assisted surgery for pre- and intra-operative planning. Accurate modeling of the interaction between surgical instruments and organs has been recognized as a key requirement in the development of high-fidelity surgical simulators. Researchers have attempted to model tool-tissue interactions in a wide variety of ways, which can be broadly classified as (1) linear elasticity-based, (2) nonlinear (hyperelastic) elasticity-based finite element (FE) methods, and (3) other techniques that not based on FE methods or continuum mechanics. Realistic modeling of organ deformation requires populating the model with real tissue data (which are difficult to acquire in vivo) and simulating organ response in real time (which is computationally expensive). Further, it is challenging to account for connective tissue supporting the organ, friction, and topological changes resulting from tool-tissue interactions during invasive surgical procedures. Overcoming such obstacles will not only help us to model tool-tissue interactions in real time, but also enable realistic force feedback to the user during surgical simulation. This review paper classifies the existing research on tool-tissue interactions for surgical simulators specifically based on the modeling techniques employed and the kind of surgical operation being simulated, in order to inform and motivate future research on improved tool-tissue interaction models.
View details for PubMedID 20119508
View details for PubMedCentralID PMC2813063
-
Surgical and interventional robotics: Part II - Surgical CAD-CAM systems
IEEE ROBOTICS & AUTOMATION MAGAZINE
2008; 15 (3): 94-102
View details for DOI 10.1109/MRA.2008.927971
View details for Web of Science ID 000259300600014
-
Surgical and interventional robotics - Core concepts, technology, and design
IEEE ROBOTICS & AUTOMATION MAGAZINE
2008; 15 (2): 122-130
View details for DOI 10.1109/MRA.2008.926390
View details for Web of Science ID 000256801100017
-
Modeling the forces of cutting with scissors
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2008; 55 (3): 848-856
Abstract
Modeling forces applied to scissors during cutting of biological materials is useful for surgical simulation. Previous approaches to haptic display of scissor cutting are based on recording and replaying measured data. This paper presents an analytical model based on the concepts of contact mechanics and fracture mechanics to calculate forces applied to scissors during cutting of a slab of material. The model considers the process of cutting as a sequence of deformation and fracture phases. During deformation phases, forces applied to the scissors are calculated from a torque-angle response model synthesized from measurement data multiplied by a ratio that depends on the position of the cutting crack edge and the curve of the blades. Using the principle of conservation of energy, the forces of fracture are related to the fracture toughness of the material and the geometry of the blades of the scissors. The forces applied to scissors generally include high-frequency fluctuations. We show that the analytical model accurately predicts the average applied force. The cutting model is computationally efficient, so it can be used for real-time computations such as haptic rendering. Experimental results from cutting samples of paper, plastic, cloth, and chicken skin confirm the model, and the model is rendered in a haptic virtual environment.
View details for DOI 10.1109/TBME.2007.908069
View details for Web of Science ID 000253733800001
View details for PubMedID 18334376
View details for PubMedCentralID PMC2709828
-
Integrated Planning and Image-Guided Control for Planar Needle Steering
2nd Biennial IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2008)
IEEE. 2008: 907–912
View details for Web of Science ID 000270540400151
-
The touch thimble: Providing fingertip contact feedback during point-force haptic interaction
16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2008: 239–246
View details for Web of Science ID 000254949300034
-
A Simulator to Explore the Role of Haptic Feedback in Cataract Surgery Training
16th Conference on Medicine Meets Virtual Reality
I O S PRESS. 2008: 106–111
Abstract
Phacoemulsification cataract surgery, a minimally invasive technique to remove a cloudy lens from the eye, is one of the most commonly performed surgical procedures in the western world. Conventional training for this procedure involves didactic lectures and practice on pig and human cadaver eyes, none of which allow trainees to form an accurate predictive model of human tissue behavior during surgery. A virtual environment simulator for capsulorrhexis, one of the first steps in cataract surgery, has been developed that allows a trainee to use surgical instruments to excise a circle of tissue on the anterior side of the lens capsule through tearing. The simulator invokes a deformable mass-spring-damper mesh model of the tissue that can be grasped and torn via shearing. A novel algorithm for mesh division and maintenance enables realistic tearing behavior. The trainee controls tool motion using a 3-degree-of-freedom haptic device, and haptic feedback is provided from the virtual tissue. Although the haptic feedback in a real capsulorrhexis procedure is below the human threshold of haptic sensing, this simulator enables an experiment to determine the effectiveness of "haptic training wheels" -- the idea of haptic training for a task without haptic feedback.
View details for Web of Science ID 000272668400023
View details for PubMedID 18391267
- Measurement-Based Modeling for Haptic Display. In Haptic Rendering: Foundations, Algorithms, and Applications. edited by Lin, M., C., Otaduy, M. AK Peters.. 2008: 1
-
Control Methods for Guidance Virtual Fixtures in Compliant Human-Machine Interfaces
IEEE/RSJ International Conference on Intelligent Robots and Systems
IEEE. 2008: 1166–1172
View details for Web of Science ID 000259998200183
- Chapter 30: Haptics. In Springer Handbook of Robotics Springer. 2008: 719–739
-
Force-feedback surgical teleoperator: Controller design and palpation experiments
16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2008: 465–471
View details for Web of Science ID 000254949300070
- A Simulator to Explore the Role of Haptic Feedback in Cataract Surgery Training. Medicine Meets Virtual Reality. 2008
-
Needle-Tissue Interaction Forces for Bevel-Tip Steerable Needles
2nd Biennial IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2008)
IEEE. 2008: 224–231
View details for Web of Science ID 000270540400038
-
Effects of visual force feedback on robot-assisted surgical task performance
JOURNAL OF THORACIC AND CARDIOVASCULAR SURGERY
2008; 135 (1): 196-202
Abstract
Direct haptic (force or tactile) feedback is negligible in current surgical robotic systems. The relevance of haptic feedback in robot-assisted performances of surgical tasks is controversial. We studied the effects of visual force feedback, a haptic feedback surrogate, on tying surgical knots with fine sutures similar to those used in cardiovascular surgery.By using a modified da Vinci robotic system (Intuitive Surgical, Inc, Sunnyvale, Calif) equipped with force-sensing instrument tips and real-time visual force feedback overlays in the console image, 10 surgeons each tied 10 knots with and 10 knots without visual force feedback. Four surgeons had significant prior da Vinci experience, and the remaining 6 surgeons did not. Performance parameters, including suture breakage and secure knots, peak and standard deviation of applied forces, and completion times using 5-0 silk sutures, were recorded. Chi-square and Student t test analyses determined the differences between groups.Among surgeon subjects with robotic experience, no differences in measured performance parameters were found between robot-assisted knot ties executed with and without visual force feedback. Among surgeons without robotic experience, however, visual force feedback was associated with lower suture breakage rates, peak applied forces, and standard deviations of applied forces. Visual force feedback did not impart differences in knot completion times or loose knots for either surgeon group.Visual force feedback resulted in reduced suture breakage, lower forces, and decreased force inconsistencies among novice robotic surgeons, although elapsed time and knot quality were unaffected. In contrast, visual force feedback did not affect these metrics among surgeons experienced with the da Vinci system. These results suggest that visual force feedback primarily benefits novice robot-assisted surgeons, with diminishing benefits among experienced surgeons.
View details for DOI 10.1016/j.jtcvs.2007.08.043
View details for Web of Science ID 000252139600029
View details for PubMedID 18179942
View details for PubMedCentralID PMC2674617
-
Techniques for Environment Parameter Estimation During Telemanipulation
2nd Biennial IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2008)
IEEE. 2008: 217–223
View details for Web of Science ID 000270540400037
-
Haptic simulation of elbow joint spasticity
16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2008: 475–476
View details for Web of Science ID 000254949300072
-
Effects of proprioceptive motion feedback on sighted and non-sighted control of a virtual hand prosthesis
16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2008: 141–142
View details for Web of Science ID 000254949300021
-
Modeling realistic tool-tissue interactions with haptic feedback: A learning-based method
16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2008: 209–215
View details for Web of Science ID 000254949300030
- Compensating for Torsion Windup in Steerable Needles. 2008
- The Touch Thimble: Providing Fingertip Contact Feedback During Point-Force Haptic Interaction. 2008
- Telemanipulators with Sensor/Actuator Asymmetries Fail the Robustness Criterion. 2008
- Multi-Estimator Technique for Environment Parameter Estimation During Telemanipulation. 2008
-
Physically Valid Surgical Simulators: Linear Versus Nonlinear Tissue Models
16th Conference on Medicine Meets Virtual Reality
I O S PRESS. 2008: 293–295
Abstract
Realistic modeling of the interaction between surgical instruments and organs has been recognized as a key requirement in the development of high-fidelity surgical simulators. For a nonlinear model, the well-known Poynting effect developed during shearing of the tissue results in normal forces not seen in a linear elastic model. It is demonstrated that the difference in force magnitude for myocardial tissue is larger than the just noticeable difference for contact force discrimination thresholds published in the psychophysics literature. This work also proposes the validation of simulators by careful examination of relevant simulator design parameters that relate to final simulator behaviors affecting clinical outcomes.
View details for Web of Science ID 000272668400063
View details for PubMedID 18391307
- Control Methods for Guidance Virtual Fixtures in Compliant Human-Machine Interfaces. 2008
-
Friction compensation for enhancing transparency of a teleoperator with compliant transmission
IEEE TRANSACTIONS ON ROBOTICS
2007; 23 (6): 1240-1246
View details for DOI 10.1109/TRO.2007.909825
View details for Web of Science ID 000251944100013
-
Friction Compensation for Enhancing Transparency of a Teleoperator with Compliant Transmission.
IEEE transactions on robotics : a publication of the IEEE Robotics and Automation Society
2007; 23 (6): 1240-1246
Abstract
This article presents a model-based compensator for canceling friction in the tendon-driven joints of a haptic-feedback teleoperator. Unlike position-tracking systems, a teleoperator involves an unknown environment force that prevents the use of tracking position error as a feedback to the compensator. Thus, we use a model-based feedforward friction compensator to cancel the friction forces. We provide conditions for selecting compensator parameters to ensure passivity of the teleoperator and demonstrate performance experimentally.
View details for DOI 10.1109/TRO.2007.909825
View details for PubMedID 20514151
View details for PubMedCentralID PMC2877600
-
Pseudo-admittance bilateral telemanipulation with guidance virtual fixtures
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2007; 26 (8): 865-884
View details for DOI 10.1177/0278364907080425
View details for Web of Science ID 000248543500006
-
Human performance in a knob-turning task
2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2007: 96–101
View details for Web of Science ID 000246105200017
-
Teleoperation of steerable needles
IEEE International Conference on Robotics and Automation
IEEE. 2007: 934–939
View details for Web of Science ID 000250915300149
-
Force feedback is noticeably different for linear versus nonlinear elastic tissue models
2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2007: 519–524
View details for Web of Science ID 000246105200086
-
Dynamic guidance with pseudoadmittance virtual fixtures
IEEE International Conference on Robotics and Automation
IEEE. 2007: 1761–1767
View details for Web of Science ID 000250915301122
-
Virtual fixture control for compliant human-machine interfaces
IEEE International Conference on Robotics and Automation
IEEE. 2007: 4018–4024
View details for Web of Science ID 000250915304004
-
Effects of translational and gripping force feedback are decoupled in a 4-degree-of-freedom telemanipulator
2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2007: 286–291
View details for Web of Science ID 000246105200048
-
Enhancing transparency of a posit ion-exchange teleoperator
2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2007: 470–475
View details for Web of Science ID 000246105200078
-
Evaluation of human performance with kinematic and haptic errors
2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2007: 78–83
View details for Web of Science ID 000246105200014
-
Haptic virtual fixtures for robot-assisted manipulation
12th International Symposium on Robotics Research (ISRR)
SPRINGER-VERLAG BERLIN. 2007: 49–64
View details for Web of Science ID 000245915500004
- Design of a Pill-Sized 12-legged Endoscopic Capsule Robot. 2007
- Force Sensing in Robot-Assisted Surgery: Which Degrees of Freedom are Most Important? Medicine Meets Virtual Reality (MMVR) 15. 2007
- Educational Haptics. 2007
- Electrolytic Silicone Bourdon Tube Microactuator for Reconfigurable Surgical Robots. 2007
- Friction Compensation for Enhancing Transparency of a Teleoperator with Compliant Transmission. IEEE Transactions on Robotics 2007; 6 (23): 1240-1246
-
Quantifying the value of visual and haptic position feedback during force-based motion control
2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2007: 561–562
View details for Web of Science ID 000246105200096
-
Effects of visual and proprioceptive motion feedback on human control of targeted movement
10th IEEE International Conference on Rehabilitation Robotics
IEEE. 2007: 513–524
View details for Web of Science ID 000255389600077
-
Nonholonomic modeling of needle steering
9th International Symposium on Experimental Robotics (ISER)
SAGE PUBLICATIONS LTD. 2006: 509–25
View details for DOI 10.1177/0278364906065388
View details for Web of Science ID 000238146600006
-
Stable forbidden-region virtual fixtures for bilateral telemanipulation
JOURNAL OF DYNAMIC SYSTEMS MEASUREMENT AND CONTROL-TRANSACTIONS OF THE ASME
2006; 128 (1): 53-64
View details for DOI 10.1115/1.2168163
View details for Web of Science ID 000236439100007
-
Nonholonomic modeling of needle steering
9th International Symposium on Experimental Robotics (ISER)
SPRINGER-VERLAG BERLIN. 2006: 35–44
View details for Web of Science ID 000236887500004
-
Dynamic augmented reality for sensory substitution in robot-assisted surgical systems.
Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference
2006; 1: 567-570
Abstract
Teleoperated robot-assisted surgical systems provide surgeons with improved precision, dexterity, and visualization over traditional minimally invasive surgery. The addition of haptic (force and/or tactile) feedback has been proposed as a way to further enhance the performance of these systems. However, due to limitations in sensing and control technologies, implementing direct haptic feedback to the surgeon's hands remains impractical for clinical application. A new, intuitive augmented reality system for presentation of force information through sensory substitution has been developed and evaluated. The augmented reality system consists of force-sensing robotic instruments, a kinematic tool tracker, and a graphic display that overlays a visual representation of force levels on top of the moving instrument tips. The system is integrated with the da Vinci Surgical System (Intuitive Surgical, Inc.) and tested by several users in a phantom knot tying task. The augmented reality system decreases the number of broken sutures, decreases the number of loose knots, and results in more consistent application of forces.
View details for PubMedID 17945986
-
Effect of hand dynamics on virtual fixtures for compliant human-machine interfaces*
14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2006: 109–115
View details for Web of Science ID 000237225000019
-
Portability and applicability of virtual fixtures across medical and manufacturing tasks
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2006: 225–230
View details for Web of Science ID 000240886900037
- Object Capture With a Camera-Mobile Robot System: An Introductory Robotics Project. IEEE Robotics and Automation Magazine 2006; 1 (13): 85-88
-
Pseudo-admittance bilateral telemanipulation with guidance virtual fixtures
14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2006: 169–175
View details for Web of Science ID 000237225000027
-
Sensor/Actuator asymmetries in telemanipulators: Implications of partial force feedback
14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2006: 309–314
View details for Web of Science ID 000237225000047
-
Toward active cannulas: Miniature snake-like surgical robots
IEEE/RSJ International Conference on Intelligent Robots and Systems
IEEE. 2006: 2857–2863
View details for Web of Science ID 000245452403001
-
Dynamic augmented reality for sensory substitution in robot-assisted surgical systems
28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society
IEEE. 2006: 3090–3093
View details for Web of Science ID 000247284703142
-
Friction compensation for a force-feedback telerobotic system
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2006: 3268–3273
View details for Web of Science ID 000240886907017
-
Friction compensation for a force-feedback teleoperator with compliant transmission
45th IEEE Conference on Decision and Control
IEEE. 2006: 4508–4513
View details for Web of Science ID 000252251602110
- Effects of Haptic Feedback on Exploration. 2006
-
Environment parameter estimation during bilateral telemanipulation
14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2006: 301–307
View details for Web of Science ID 000237225000046
-
Effects of position quantization and sampling rate on virtual-wall passivity
ASME International Mechanical Engineering Congress
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC. 2005: 952–64
View details for DOI 10.1109/TRO.2005.851377
View details for Web of Science ID 000232118800015
-
Human-machine collaborative systems for microsurgical applications
11th International Symposium on Robotics Research
SAGE PUBLICATIONS LTD. 2005: 731–41
View details for DOI 10.1177/0278364905057059
View details for Web of Science ID 000232274300005
-
Transrectal fiducial carrier for radiographic image registration in prostate Brachytherapy
47th Annual Meeting of the American-Association-of-Physicists-in-Medicine
AMER ASSOC PHYSICISTS MEDICINE AMER INST PHYSICS. 2005: 2108–
View details for Web of Science ID 000229908601356
-
Effect of sensory substitution on suture-manipulation forces for robotic surgical systems
JOURNAL OF THORACIC AND CARDIOVASCULAR SURGERY
2005; 129 (1): 151-158
Abstract
Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators.Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback.The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces.Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.
View details for DOI 10.1016/j.jtcvs.2004.05.029
View details for Web of Science ID 000226216600021
View details for PubMedID 15632837
-
A velocity-dependent model for needle insertion in soft tissue.
Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention
2005; 8: 624-632
Abstract
Models that predict the soft tissue deformation caused by needle insertion could improve the accuracy of procedures such as brachytherapy and needle biopsy. Prior work on needle insertion modeling has focused on static deformation; the experiments presented here show that dynamic effects such as relaxation are important. An experimental setup is described for recording and measuring the deformation that occurs with needle insertion into a soft tissue phantom. Analysis of the collected data demonstrates the time- and velocity-dependent nature of the deformation. Deformation during insertion is shown to be well represented using a velocity-dependent force function with a linear elastic finite element model. The model's accuracy is limited to the period during needle motion, indicating that a viscoelastic tissue model may be required to capture tissue relaxation after the needle stops.
View details for PubMedID 16686012
-
Human-machine collaborative systems for microsurgical applications
11th International Symposium on Robotics Research
SPRINGER-VERLAG BERLIN. 2005: 162–171
View details for Web of Science ID 000228862600014
-
Planning for steerable bevel-tip needle insertion through 2D soft tissue with obstacles
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2005: 1640–1645
View details for Web of Science ID 000235460101070
-
Automatic detection and segmentation of robot-assisted surgical motions.
Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention
2005; 8: 802-810
Abstract
Robotic surgical systems such as Intuitive Surgical's da Vinci system provide a rich source of motion and video data from surgical procedures. In principle, this data can be used to evaluate surgical skill, provide surgical training feedback, or document essential aspects of a procedure. If processed online, the data can be used to provide context-specific information or motion enhancements to the surgeon. However, in every case, the key step is to relate recorded motion data to a model of the procedure being performed. This paper examines our progress at developing techniques for "parsing" raw motion data from a surgical task into a labelled sequence of surgical gestures. Our current techniques have achieved >90% fully automated recognition rates on 15 datasets.
View details for PubMedID 16685920
-
A fracture mechanics approach to haptic synthesis of tissue cutting with scissors
1st Joint Eurohaptics Conference/Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2005: 356–362
View details for Web of Science ID 000228426200050
-
Steering flexible needles under Markov motion uncertainty
IEEE/RSJ International Conference on Intelligent Robots and Systems
IEEE. 2005: 120–125
View details for Web of Science ID 000235632100020
-
Design considerations for robotic needle steering
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2005: 3588–3594
View details for Web of Science ID 000235460103006
-
Effects of velocity on human force control
1st Joint Eurohaptics Conference/Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2005: 73–79
View details for Web of Science ID 000228426200010
-
A velocity-dependent model for needle insertion in soft tissue
8th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2005: 624–632
View details for Web of Science ID 000233356900077
-
Diffusion-based motion planning for a nonholonomic flexible needle model
IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2005: 4600–4605
View details for Web of Science ID 000235460103169
-
Effects of gripping and translational forces on teleoperation
Workshop on Multi-Point Interaction with Real and Virtual Objects
WORLD SCIENTIFIC PUBL CO PTE LTD. 2005: 231–241
View details for Web of Science ID 000231731400016
-
The snaptic paddle: A modular haptic device
1st Joint Eurohaptics Conference/Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2005: 537–538
View details for Web of Science ID 000228426200093
-
Automatic detection and segmentation of robot-assisted surgical motions
8th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2005: 802–810
View details for Web of Science ID 000233337000099
-
Vision-assisted control for manipulation using virtual fixtures
IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION
2004; 20 (6): 953-966
View details for DOI 10.1109/TRO.2004.829483
View details for Web of Science ID 000225484300003
-
Force modeling for needle insertion into soft tissue
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2004; 51 (10): 1707-1716
Abstract
The modeling of forces during needle insertion into soft tissue is important for accurate surgical simulation, preoperative planning, and intelligent robotic assistance for percutaneous therapies. We present a force model for needle insertion and experimental procedures for acquiring data from ex vivo tissue to populate that model. Data were collected from bovine livers using a one-degree-of-freedom robot equipped with a load cell and needle attachment. computed tomography imaging was used to segment the needle insertion process into phases identifying different relative velocities between the needle and tissue. The data were measured and modeled in three parts: 1) capsule stiffness, a nonlinear spring model; 2) friction, a modified Karnopp model; and 3) cutting, a constant for a given tissue. In addition, we characterized the effects of needle diameter and tip type on insertion force using a silicone rubber phantom. In comparison to triangular and diamond tips, a bevel tip causes more needle bending and is more easily affected by tissue density variations. Forces for larger diameter needles are higher due to increased cutting and friction forces.
View details for DOI 10.1109/TBME.2004.831542
View details for Web of Science ID 000224001900002
View details for PubMedID 15490818
-
Speed-accuracy characteristics of human-machine cooperative manipulation using virtual fixtures with variable admittance
HUMAN FACTORS
2004; 46 (3): 518-532
Abstract
This work explores the effect of virtual fixture admittance on the performance, defined by error and time, of task execution with a human-machine cooperative system. A desired path is obtained using computer vision, and virtual fixtures for assistance in planar path following were implemented on an admittance-controlled robot. The admittance controller uses a velocity gain, so that the speed of the robot is proportional to the force applied by the operator. The level of virtual fixture guidance is determined by the admittance ratio, which is the ratio of the admittance gain of the force components orthogonal to the path to the gain of the force components parallel to the path. In Experiment 1, we found a linear relationship between admittance ratio and performance. In Experiment 2, we examined the effect of admittance ratio on the performance of three tasks: path following, off-path targeting, and obstacle avoidance. An algorithm was developed to select an appropriate admittance ratio based on the nature of the task. Automatic admittance ratio tuning is recommended for next-generation virtual fixtures. Actual or potential applications of this research include surgery, assembly, and manipulation at the macro and micro scales.
View details for Web of Science ID 000224978900011
View details for PubMedID 15573549
-
Application of haptic feedback to robotic surgery
JOURNAL OF LAPAROENDOSCOPIC & ADVANCED SURGICAL TECHNIQUES-PART A
2004; 14 (3): 191-195
Abstract
Robotic surgical systems have greatly contributed to the advancement of minimally invasive endoscopic surgery. However, current robotic systems do not provide tactile or haptic feedback to the operating surgeon. Under certain circumstances, particularly with the manipulation of delicate tissues and suture materials, this may prove to be a significant irritation. We hypothesize that haptic feedback, in the form of sensory substitution, facilitates the performance of surgical knot tying. This preliminary study describes evidence that visual sensory substitution permits the surgeon to apply more consistent, precise, and greater tensions to fine suture materials without breakage during robot-assisted knot tying.
View details for Web of Science ID 000222276000015
View details for PubMedID 15245675
-
Teleoperation with sensor/actuator asymmetry: Task performance with partial force feedback
12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2004)
IEEE COMPUTER SOC. 2004: 121–127
View details for Web of Science ID 000189492300016
-
Haptic rendering of tissue cutting with scissors
12th Conference on Medicine Meets Virtual Reality
I O S PRESS. 2004: 407–409
Abstract
Current surgical simulations that involve cutting with scissors lack display of forces "between the fingers" of the operator. The objective of this work is to develop a fast, realistic haptic rendering technique for scissor cutting that can be easily integrated into soft tissue models commonly used in surgical simulators. A virtual environment was created for combined graphic and haptic display of cutting. Two 2-dimensional deformable mass-spring-damper models were developed: The first is a large mesh that is used to calculate translational forces. The second is a smaller, one-element-thick mesh that is used to calculate cutting (angular) forces. This technique was implemented on the haptic scissors, a haptic interface that allows motion and force feedback in translation and cutting.
View details for Web of Science ID 000222326200090
View details for PubMedID 15544316
-
The effect of visual and haptic feedback on computer-assisted needle insertion.
Computer aided surgery
2004; 9 (6): 243-249
Abstract
We present a study evaluating the effects of visual and haptic feedback on human performance in a needle insertion task.A one-degree-of-freedom needle insertion simulator with a three-layer tissue model (skin, fat and muscle) was used in perceptual experiments. The objective of the 14 subjects was to detect the puncture of each tissue layer using varying haptic and visual cues. Performance was measured by overshoot error-the distance traveled by the virtual needle after puncture.Without force feedback, real-time visual feedback reduced error by at least 87% in comparison to static image overlay. Force feedback, in comparison to no force feedback, reduced puncture overshoot by at least 52% when visual feedback was absent or limited to static image overlay. A combination of force and visual feedback improved performance, especially for tissues with low stiffness, by at least 43% with visual display of the needle position, and by at least 67% with visual display of layer deflection.Real-time image overlay significantly enhances controlled puncture during needle insertion. Force feedback may not be necessary except in circumstances where visual feedback is limited.
View details for PubMedID 16112974
-
Performance analysis of steady-hand teleoperation versus cooperative manipulation
12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2004)
IEEE COMPUTER SOC. 2004: 316–322
View details for Web of Science ID 000189492300044
-
Vision-based assistance for ophthalmic micro-surgery
7th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2004)
SPRINGER-VERLAG BERLIN. 2004: 49–57
View details for Web of Science ID 000224322400007
-
Effect of sensory substitution on suture manipulation forces for surgical teleoperation
12th Conference on Medicine Meets Virtual Reality
I O S PRESS. 2004: 157–163
Abstract
Bilateral telemanipulation, which applies haptic feedback to the operator, is not yet available in most commercial robot-assisted surgical systems. We have shown in previous work that the lack of haptic (force or tactile) feedback is detrimental in applications requiring fine suture manipulation. In this paper, we study the effect of substituting direct haptic feedback with visual and auditory cues. Using the da Vinci robot from Intuitive Surgical, we observed the difference between applied forces during a knot tying procedure for four different sensory feedback substitution scenarios: no feedback, auditory feedback, visual feedback, and a combination of auditory and visual feedback. Our results indicate that visual feedback, which provides continuous force information, would improve robot-assisted performance during complex surgical tasks such as knot tying with fine sutures. Discrete auditory feedback gives additional useful support to the surgeon.
View details for Web of Science ID 000222326200037
View details for PubMedID 15544263
-
Methods for haptic feedback in teleoperated robot-assisted surgery
INDUSTRIAL ROBOT-AN INTERNATIONAL JOURNAL
2004; 31 (6): 499-508
View details for DOI 10.1108/01439910410566362
View details for Web of Science ID 000224825400008
-
Virtual remote center of motion control for needle placement robots.
Computer aided surgery
2004; 9 (5): 175-183
Abstract
We present an algorithm that enables percutaneous needle-placement procedures to be performed with unencoded, unregistered, minimally calibrated robots while removing the constraint of placing the needle tip on a mechanically enforced Remote Center of Motion (RCM).The algorithm requires only online tracking of the surgical tool and a five-degree-of-freedom (5-DOF) robot comprising three prismatic DOF and two rotational DOF. An incremental adaptive motion control cycle guides the needle to the insertion point and also orients it to align with the target-entry-point line. The robot executes RCM motion without having a physically constrained fulcrum point.The proof-of-concept prototype system achieved 0.78 mm translation accuracy and 1.4 degrees rotational accuracy (this is within the tracker accuracy) within 17 iterative steps (0.5-1 s).This research enables robotic assistant systems for image-guided percutaneous procedures to be prototyped/constructed more quickly and less expensively than has been previously possible. Since the clinical utility of such systems is clear and has been demonstrated in the literature, our work may help promote widespread clinical adoption of this technology by lowering system cost and complexity.
View details for PubMedID 16192059
-
A modular 2-DOF force-sensing instrument for laparoscopic surgery
6th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2003: 279–286
View details for Web of Science ID 000188592600035
-
Recognition of operator motions for real-time assistance using virtual fixtures
11th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2003: 125–131
View details for Web of Science ID 000182249800017
-
The Haptic Scissors: Cutting in virtual environments
20th IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2003: 828–833
View details for Web of Science ID 000187419900133
-
Virtual fixture architectures for telemanipulation
20th IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2003: 2798–2805
View details for Web of Science ID 000187419900449
-
Uniting haptic exploration and display
10th International Symposium on Robotics Research (ISRR 2001)
SPRINGER-VERLAG BERLIN. 2003: 225–238
View details for Web of Science ID 000183013700015
-
Robotic needle insertion: Effects of friction and needle geometry
20th IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2003: 1774–1780
View details for Web of Science ID 000187419900285
-
Steady-hand teleoperation with virtual fixtures
12th IEEE International Workshop on Robot and Human Interactive Communication
IEEE. 2003: 145–151
View details for Web of Science ID 000186827500025
-
Methods for intelligent localization and mapping during haptic exploration
IEEE International Conference on Systems, Man and Cybernetics (SMC 03)
IEEE. 2003: 3438–3445
View details for Web of Science ID 000186578600561
-
Virtual remote center of motion control for needle placement robots
6th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2003: 157–164
View details for Web of Science ID 000188592600020
-
Activation cues and force scaling methods for virtual fixtures
11th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems
IEEE COMPUTER SOC. 2003: 404–409
View details for Web of Science ID 000182249800053
-
Analysis of virtual fixture contact stability for telemanipulation
IEEE/RSJ International Conference on Intelligent Robots and Systems
IEEE. 2003: 2699–2706
View details for Web of Science ID 000187883300437
-
Spatial motion constraints: Theory and demonstrations for robot guidance using virtual fixtures
20th IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2003: 1954–1959
View details for Web of Science ID 000187419900313
-
Measurement, analysis, and display of haptic signals during surgical cutting
PRESENCE-TELEOPERATORS AND VIRTUAL ENVIRONMENTS
2002; 11 (6): 626-651
View details for Web of Science ID 000180224100006
-
The effect of visual and haptic feedback on manual and teleoperated needle insertion
5th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2002: 147–154
View details for Web of Science ID 000189412100019
-
On the display of haptic recordings for cutting biological tissues
10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2002)
IEEE COMPUTER SOC. 2002: 80–87
View details for Web of Science ID 000175458900011
-
Vision assisted control for manipulation using virtual fixtures: Experiments at macro and micro scales
19th IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2002: 3354–3361
View details for Web of Science ID 000178573200530
-
Effect of virtual fixture compliance on human-machine cooperative manipulation
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2002)
IEEE. 2002: 1089–1095
View details for Web of Science ID 000179289100177
-
Measurement of the tip and friction force acting on a needle during penetration
5th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2002: 216–223
View details for Web of Science ID 000189412100027
-
Modeling of needle insertion forces for robot-assisted percutaneous therapy
19th IEEE International Conference on Robotics and Automation (ICRA)
IEEE. 2002: 2085–2091
View details for Web of Science ID 000178573200328
-
Building a task language for segmentation and recognition of user input to cooperative manipulation systems
10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2002)
IEEE COMPUTER SOC. 2002: 225–230
View details for Web of Science ID 000175458900029
-
Analysis of suture manipulation forces for teleoperation with force feedback
5th International Conference on Medical Image Computing and Computer-Assisted Intervention
SPRINGER-VERLAG BERLIN. 2002: 155–162
View details for Web of Science ID 000189412100020
-
Feature detection for haptic exploration with robotic fingers
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
2001; 20 (12): 925-938
View details for Web of Science ID 000174018800001
-
Reality-based models for vibration feedback in virtual environments
IEEE-ASME TRANSACTIONS ON MECHATRONICS
2001; 6 (3): 245-252
View details for Web of Science ID 000171123900005
-
Feature-guided exploration with a robotic finger
IEEE International Conference on Robotics and Automation
IEEE. 2001: 589–596
View details for Web of Science ID 000172615800095
-
Vision assisted control for manipulation using virtual fixtures
IEEE Conference on Intelligent Robots and Systems (IROS 2001)
IEEE. 2001: 1171–1176
View details for Web of Science ID 000176593900187
-
Haptic surface exploration
6th International Symposium on Experimental Robotics
SPRINGER-VERLAG LONDON LTD. 2000: 423–432
View details for Web of Science ID 000086068500041
-
Haptic exploration of fine surface features
International Conference on Robotics and Automation (ICRA '99)
IEEE. 1999: 2930–2936
View details for Web of Science ID 000081625700464
-
Vibration feedback models for virtual environments
IEEE International Conference on Robotics and Automation
IEEE. 1998: 674–679
View details for Web of Science ID 000074368900107
-
Haptic exploration of objects with rolling and sliding
1997 IEEE International Conference on Robotics and Automation (ICRA97) - Teaming to Make an Impact
IEEE. 1997: 2485–2490
View details for Web of Science ID A1997BJ42X00394