Allison Okamura

All Publications

• A Comparison of Pneumatic Actuators for Soft Growing Vine Robots. Soft robotics Kübler, A. M., du Pasquier, C., Low, A., Djambazi, B., Aymon, N., Förster, J., Agharese, N., Siegwart, R., Okamura, A. M. 2024

  Abstract

  Soft pneumatic actuators are used to steer soft growing "vine" robots while being flexible enough to undergo the tip eversion required for growth. In this study, we compared the performance of three types of pneumatic actuators in terms of their ability to perform eversion, quasi-static bending, dynamic motion, and force output: the pouch motor, the cylindrical pneumatic artificial muscle (cPAM), and the fabric pneumatic artificial muscle (fPAM). The pouch motor is advantageous for prototyping owing to its simple manufacturing process. The cPAM exhibits superior bending behavior and produces the highest forces, whereas the fPAM actuates fastest and everts at the lowest pressure. We evaluated a range of dimensions for each actuator type. Larger actuators can produce more significant deformations and forces, but smaller actuators inflate faster and can evert at a lower pressure. Because vine robots are lightweight, the effect of gravity on the functionality of different actuators is minimal. We developed a new analytical model that predicts the pressure-to-bending behavior of vine robot actuators. Using the actuator results, we designed and demonstrated a 4.8 m long vine robot equipped with highly maneuverable 60 × 60 mm cPAMs in a three-dimensional obstacle course. The vine robot was able to move around sharp turns, travel through a passage smaller than its diameter, and lift itself against gravity.

  View details for DOI 10.1089/soro.2023.0169

  View details for PubMedID 38717834

• Stiffness Change for Reconfiguration of Inflated Beam Robots. Soft robotics Do, B. H., Wu, S., Zhao, R. R., Okamura, A. M. 2024

  Abstract

  Abstract Active control of the shape of soft robots is challenging. Despite having an infinite number of passive degrees of freedom (DOFs), soft robots typically only have a few actively controllable DOFs, limited by the number of degrees of actuation (DOAs). The complexity of actuators restricts the number of DOAs that can be incorporated into soft robots. Active shape control is further complicated by the buckling of soft robots under compressive forces; this is particularly challenging for compliant continuum robots due to their long aspect ratios. In this study, we show how variable stiffness enables shape control of soft robots by addressing these challenges. Dynamically changing the stiffness of sections along a compliant continuum robot selectively "activates" discrete joints. By changing which joints are activated, the output of a single actuator can be reconfigured to actively control many different joints, thus decoupling the number of controllable DOFs from the number of DOAs. We demonstrate embedded positive pressure layer jamming as a simple method for stiffness change in inflated beam robots, its compatibility with growing robots, and its use as an "activating" technology. We experimentally characterize the stiffness change in a growing inflated beam robot and present finite element models that serve as guides for robot design and fabrication. We fabricate a multisegment everting inflated beam robot and demonstrate how stiffness change is compatible with growth through tip eversion, enables an increase in workspace, and achieves new actuation patterns not possible without stiffening.

  View details for DOI 10.1089/soro.2023.0120

  View details for PubMedID 38683643

• phloSAR: a Portable, High-Flow Pressure Supply and Regulator Enabling Untethered Operation of Large Pneumatic Soft Robots Ahlquist, M., Jitosho, R., Bao, J., Okamura, A. M., IEEE IEEE. 2024: 28-33

  View details for DOI 10.1109/ROBOSOFT60065.2024.10522055

  View details for Web of Science ID 001230033600142

• Design and Evaluation of Haptic Guidance in Ultrasound-Based Needle-Insertion Procedures. IEEE transactions on bio-medical engineering Raitor, M., Nunez, C. M., Stolka, P. J., Okamura, A. M., Culbertson, H. 2024; 71 (1): 26-35

  Abstract

  OBJECTIVE: This article presents two haptic guidance systems designed to help a clinician keep an ultrasound probe steady when completing ultrasound-assisted needle insertion tasks. These procedures demand spatial reasoning and hand-eye coordination because the clinician must align a needle with the ultrasound probe and extrapolate the needle trajectory using only a 2D ultrasound image. Past research has shown that visual guidance helps the clinician align the needle, but does not help the clinician keep the ultrasound probe steady, sometimes resulting in a failed procedure.METHODS: We created two separate haptic guidance systems to provide feedback if the user tilts the ultrasound probe away from the desired setpoint using (1) vibrotactile stimulation provided by a voice coil motor or (2) distributed tactile pressure provided by a pneumatic mechanism.RESULTS: Both systems significantly reduced probe deviation and correction time to errors during a needle insertion task. We also tested the two feedback systems in a more clinically relevant setup and showed that the perceptibility of the feedback was not affected by the addition of a sterile bag placed over the actuators and gloves worn by the user.CONCLUSION: These studies show that both types of haptic feedback are promising for helping the user keep the ultrasound probe steady during ultrasound-assisted needle insertion tasks. Survey results indicated that users preferred the pneumatic system over the vibrotactile system.SIGNIFICANCE: Haptic feedback may improve user performance in ultrasound-based needle-insertion procedures and shows promise in training for needle-insertion tasks and other medical procedures where guidance is required.

  View details for DOI 10.1109/TBME.2023.3290919

  View details for PubMedID 37384470

• Haptic Relocation of Virtual Finger Forces via Pneumatic Wrist-Worn Haptic Devices Palmer, J. E., Vuong, B. B., Zhakypov, Z., Qin, Y., Tilton, L., Okamura, A. M., Kajimoto, H., Birznieks, Bianchi, M. IEEE. 2024: 315-320

  View details for DOI 10.1109/HAPTICS59260.2024.10520855

  View details for Web of Science ID 001229108600020

• Reliability of Smartphone-Based Vibration Threshold Measurements Adenekan, R. G., Yoshida, K. T., Benyoucef, A., Reyes, A., Adenekan, A. E., Okamura, A. M., Nunez, C. M., Kajimoto, H., Birznieks, Bianchi, M. IEEE. 2024: 25-32

  View details for DOI 10.1109/HAPTICS59260.2024.10520838

  View details for Web of Science ID 001229108600004

• Haptics: The Science of Touch as a Foundational Pathway to Precision Education and Assessment. Academic medicine : journal of the Association of American Medical Colleges Perrone, K., Abdelaal, A. E., Pugh, C., Okamura, A. 2023

  Abstract

  Clinical touch is the cornerstone of the doctor-patient relationship and can impact patient experience and outcomes. In the current era, driven by an ever-increasing infusion of point of care technologies, physical exam skills have become undervalued. Moreover, touch and hands-on skills have been difficult to teach due to inaccurate assessments and difficulty with learning transfer through observation. In this article, the authors argue that haptics, the science of touch, provides a unique opportunity to explore new pathways to facilitate touch training. Furthermore, haptics can dramatically increase the density of touch-based assessments without increasing human rater burden-essential for realizing precision assessment. The science of haptics is reviewed, including the benefits of using haptics-informed language for objective structured clinical examinations. The authors describe how haptic devices and haptic language have and can be used to facilitate learning, communication, documentation and a much-needed reinvigoration of physical examination and touch excellence at the point of care. The synergy of haptic devices, artificial intelligence, and virtual reality environments are discussed. The authors conclude with challenges of scaling haptic technology in medical education, such as cost and translational needs, and opportunities to achieve wider adoption of this transformative approach to precision education.

  View details for DOI 10.1097/ACM.0000000000005607

  View details for PubMedID 38109654

• Shared-Control Teleoperation Paradigms on a Soft-Growing Robot Manipulator JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS Stroppa, F., Selvaggio, M., Agharese, N., Luo, M., Blumenschein, L. H., Hawkes, E. W., Okamura, A. M. 2023; 109 (2)

  View details for DOI 10.1007/s10846-023-01919-x

  View details for Web of Science ID 001188382800001

• Modeling and Control of a 5-DOF Parallel Continuum Haptic Device IEEE TRANSACTIONS ON ROBOTICS Koehler, M., Bieze, T., Kruszewski, A., Okamura, A. M., Duriez, C. 2023

  View details for DOI 10.1109/TRO.2023.3277068

  View details for Web of Science ID 001006596900001

• A Modular 3-Degrees-of-Freedom Force Sensor for Robot-Assisted Minimally Invasive Surgery Research. Sensors (Basel, Switzerland) Chua, Z., Okamura, A. M. 2023; 23 (11)

  Abstract

  Effective force modulation during tissue manipulation is important for ensuring safe, robot-assisted, minimally invasive surgery (RMIS). Strict requirements for in vivo applications have led to prior sensor designs that trade off ease of manufacture and integration against force measurement accuracy along the tool axis. Due to this trade-off, there are no commercial, off-the-shelf, 3-degrees-of-freedom (3DoF) force sensors for RMIS available to researchers. This makes it challenging to develop new approaches to indirect sensing and haptic feedback for bimanual telesurgical manipulation. We present a modular 3DoF force sensor that integrates easily with an existing RMIS tool. We achieve this by relaxing biocompatibility and sterilizability requirements and by using commercial load cells and common electromechanical fabrication techniques. The sensor has a range of ±5 N axially and ±3 N laterally with errors of below 0.15 N and maximum errors below 11% of the sensing range in all directions. During telemanipulation, a pair of jaw-mounted sensors achieved average errors below 0.15 N in all directions. It achieved an average grip force error of 0.156 N. The sensor is for bimanual haptic feedback and robotic force control in delicate tissue telemanipulation. As an open-source design, the sensors can be adapted to suit other non-RMIS robotic applications.

  View details for DOI 10.3390/s23115230

  View details for PubMedID 37299958

• Cognitive and Physical Activities Impair Perception of Smartphone Vibrations. IEEE transactions on haptics Yoshida, K. T., Kiernan, J. X., Adenekan, R. A., Trinh, S. H., Lowber, A. J., Okamura, A. M., Nunez, C. M. 2023; PP

  Abstract

  Vibration feedback is common in everyday devices, from virtual reality systems to smartphones. However, cognitive and physical activities may impede our ability to sense vibrations from devices. In this study, we develop and characterize a smartphone platform to investigate how a shape-memory task (cognitive activity) and walking (physical activity) impair human perception of smartphone vibrations. We measured how Apple's Core Haptics Framework parameters can be used for haptics research, namely how hapticIntensity modulates amplitudes of 230 Hz vibrations. A 23-person user study found that physical ( ) and cognitive ( p=0.004) activity increase vibration perception thresholds. Cognitive activity also increases vibration response time ( ). This work also introduces a smartphone platform that can be used for out-of-lab vibration perception testing. Researchers can use our smartphone platform and results to design better haptic devices for diverse, unique populations.

  View details for DOI 10.1109/TOH.2023.3279201

  View details for PubMedID 37220041

• Daily vibrotactile stimulation from a wearable device exhibits equal or greater spasticity relief than botulinum toxin in stroke. Archives of physical medicine and rehabilitation Seim, C., Chen, B., Han, C., Vacek, D., Lowber, A., Lansberg, M., Okamura, A. M. 2023

  Abstract

  OBJECTIVE: To test the feasibility and efficacy of the VibroTactile Stimulation (VTS) Glove, a wearable device that provides vibrotactile stimulation to the impaired limb to reduce spastic hypertonia.DESIGN: Prospective two-arm intervention study - including one group of patients who use Botulinum toxin (BTX-A) for spasticity and one group of patients who do not use BTX-A.SETTING: Participants were recruited through rehabilitation and neurology clinics.PARTICIPANTS: Patients with chronic stroke (N=20; mean age=54 years, mean time since stroke=6.9 years). Patients who were previously receiving the standard of care (BTX-A injection) were eligible to participate, and started the intervention 12 weeks after their last injection.INTERVENTION: Participants were instructed to use the VTS Glove for three hours daily, at home or during everyday activities, for 8 weeks.MAIN OUTCOME MEASURES: Spasticity was assessed with the Modified Ashworth Scale and the Modified Tardieu Scale at baseline and then at 2-week intervals for 12 weeks. Primary outcomes were the difference from baseline and at week 8 (end of VTS Glove use) and week 12 (four weeks after stopping VTS Glove use). Patients who were receiving BTX-A were also assessed during the 12 weeks preceding the start of VTS Glove use to monitor the effect of BTX-A on spastic hypertonia. Range of motion and participant feedback were also studied.RESULTS: A clinically meaningful difference in spastic hypertonia was found during and after daily VTS Glove use. Modified Ashworth and Modified Tardieu scores were reduced by an average of 0.9 (p=0.0014) and 0.7 (p=0.0003), respectively, at week 8 of daily VTS Glove use, and by 1.1 (p=0.00025) and 0.9 (p=0.0001), respectively, one month after stopping VTS Glove use. For participants who used BTX-A, 6 out of 11 showed greater change in Modified Ashworth ratings during VTS Glove use (Mean=-1.8 vs. Mean=-1.6 with BTX-A) and 8 out of 11 showed their lowest level of symptoms during VTS Glove use (vs. BTX-A).CONCLUSIONS: Daily stimulation from the VTS Glove provides relief of spasticity and hypertonia. For more than half of participants who had regularly used BTX-A, the VTS Glove provided equal or greater symptom relief.

  View details for DOI 10.1016/j.apmr.2023.03.031

  View details for PubMedID 37149017

• Evaluation of a Passive Wearable Device for Post-Stroke Shoulder Abduction Support. IEEE ... International Conference on Rehabilitation Robotics : [proceedings] Vasquez, E. D., Simpson, C. S., Zhou, G., Lansberg, M., Okamura, A. M. 2023; 2023: 1-6

  Abstract

  Post-stroke upper extremity function can be improved by devices that support shoulder abduction. However, many of these devices provide limited assistance in activities of daily living due to their complexity and encumbrance. We developed and evaluated a passive, lightweight (0.6 kg) wearable device consisting of an aluminum frame and elastic bands attached to a posture vest to aid in shoulder abduction. The number and thickness of bands can be adjusted to provide supportive forces to the affected arm. We measured reachable workspace area and Wolf Motor Function Test (WMFT) performance in people with a history of stroke (n = 11) with and without the wearable. The device increased workspace area in 6 participants and improved average WMFT functional and timing scores in 7 and 12 tasks, respectively, out of 16 total tasks. On average, participants increased their arm motion within 20 cm of shoulder level by 22.4% and decreased their hand's average distance from trunk by 15.2%, both improvements in the device case.

  View details for DOI 10.1109/ICORR58425.2023.10304815

  View details for PubMedID 37941216

• Reinforcement Learning Enables Real-Time Planning and Control of Agile Maneuvers for Soft Robot Arms Jitosho, R., Lum, T., Okamura, A. M., Liu, C., Tan, J., Toussaint, M., Darvish, K. JMLR-JOURNAL MACHINE LEARNING RESEARCH. 2023

  View details for Web of Science ID 001221201501010

• Relief of post-stroke spasticity with acute vibrotactile stimulation: controlled crossover study of muscle and skin stimulus methods. Frontiers in human neuroscience Seim, C., Chen, B., Han, C., Vacek, D., Wu, L. S., Lansberg, M., Okamura, A. 2023; 17: 1206027

  Abstract

  Background: Prior work suggests that vibratory stimulation can reduce spasticity and hypertonia. It is unknown which of three predominant approaches (stimulation of the spastic muscle, antagonist muscle, or cutaneous regions) most reduces these symptoms.Objective: Determine which vibrotactile stimulation approach is most effective at reducing spastic hypertonia among post-stroke patients.Methods: Sham-controlled crossover study with random assignment of condition order in fourteen patients with post-stroke hand spasticity. All patients were studied in four conditions over four visits: three stimulation conditions and a sham control. The primary outcome measure was the Modified Ashworth Scale, and the secondary outcome measure was the Modified Tardieu Scale measured manually and using 3D motion capture. For each condition, measures of spastic hypertonia were taken at four time points: baseline, during stimulation, after stimulation was removed, and after a gripping exercise.Results: A clinically meaningful difference in spastic hypertonia was found during and after cutaneous stimulation of the hand. Modified Ashworth and Modified Tardieu scores were reduced by a median of 1.1 (SD = 0.84, p = 0.001) and 0.75 (SD = 0.65, p = 0.003), respectively, during cutaneous stimulation, and by 1.25 (SD = 0.94, p = 0.001) and 0.71 (SD = 0.67, p = 0.003), respectively, at 15 min after cutaneous stimulation. Symptom reductions with spastic muscle stimulation and antagonist muscle stimulation were non-zero but not significant. There was no change with sham stimulation.Conclusions: Cutaneous vibrotactile stimulation of the hand provides significant reductions in spastic hypertonia, compared to muscle stimulation.Clinical trial registration: www.ClinicalTrials.gov, identifier: NCT03814889.

  View details for DOI 10.3389/fnhum.2023.1206027

  View details for PubMedID 37706171

• Passive Shape Locking for Multi-Bend Growing Inflated Beam Robots Jitosho, R., Simon-Trench, S., Okamura, A. M., Do, B. H., IEEE IEEE. 2023

  View details for DOI 10.1109/ROBOSOFT55895.2023.10122027

  View details for Web of Science ID 001008269300076

• A Multi-Segment, Soft Growing Robot with Selective Steering Kubler, A. M., Rivera, S., Raphael, F. B., Forster, J., Siegwart, R., Okamura, A. M., IEEE IEEE. 2023

  View details for DOI 10.1109/ROBOSOFT55895.2023.10122091

  View details for Web of Science ID 001008269300124

• Exploring Human Response Times to Combinations of Audio, Haptic, and Visual Stimuli from a Mobile Device Yoshida, K. T., Kiernan, J. X., Okamura, A. M., Nunez, C. M., Bello, F., Choi, S., Kuchenbecker, K. J., MacLean, K. IEEE. 2023: 121-127

  View details for DOI 10.1109/WHC56415.2023.10224375

  View details for Web of Science ID 001082286400018

• Wearable Sensory Substitution for Proprioception via Deep Pressure Kodali, S., Vuong, B. B., Bulea, T. C., Chesler, A. T., Bonnemann, C. G., Okamura, A. M., Bello, F., Choi, S., Kuchenbecker, K. J., MacLean, K. IEEE. 2023: 286-292

  View details for DOI 10.1109/WHC56415.2023.10224435

  View details for Web of Science ID 001082286400042

• Human Perception of Wrist Flexion and Extension Torque During Upper and Lower Extremity Movement. IEEE transactions on haptics Welker, C. G., Collins, S. H., Okamura, A. M. 2022; PP

  Abstract

  Real-world application of haptic feedback from kinesthetic devices is implemented while the user is in motion, but human wrist torque magnitude discrimination has previously only been characterized while users are stationary. In this study, we measured wrist torque discrimination in conditions relevant to activities of daily living, using a previously developed backdrivable wrist exoskeleton capable of applying wrist flexion and extension torque. We implemented a torque comparison test using a two-alternative forced-choice paradigm while participants were both seated and walking on a treadmill, with both a stationary and a moving wrist. Like most kinesthetic haptic devices, the wrist exoskeleton output torque is commanded in an open-loop manner. Thus, the study design was informed by Monte Carlo simulations to verify that the errors in the wrist exoskeleton output torque would not significantly affect the results. Results from ten participants show that although both walking and moving wrist conditions result in higher Weber Fractions (worse perception), participants were able to detect relatively small changes in torque of 12-19% on average in all grouped conditions. The results provide insight regarding the torque magnitudes necessary to make wrist-worn kinesthetic haptic devices noticeable and meaningful to the user in various conditions relevant to activities of daily living.

  View details for DOI 10.1109/TOH.2022.3219031

  View details for PubMedID 36343009

• Perceived Intensities of Normal and Shear Skin Stimuli Using a Wearable Haptic Bracelet IEEE ROBOTICS AND AUTOMATION LETTERS Sarac, M., Huh, T., Choi, H., Cutkosky, M. R., Di Luca, M., Okamura, A. M. 2022; 7 (3): 6099-6106

  View details for DOI 10.1109/LRA.2021.3140132

  View details for Web of Science ID 000785670500003

• Data-Driven Sparse Skin Stimulation Can Convey Social Touch Information to Humans IEEE TRANSACTIONS ON HAPTICS Salvato, M., R. Williams, S., M. Nunez, C., Zhu, X., Israr, A., Lau, F., Klumb, K., Abnousi, F., M. Okamura, A., Culbertson, H. 2022; 15 (2): 392-404

  Abstract

  During social interactions, people use auditory, visual, and haptic cues to convey their thoughts, emotions, and intentions. Due to weight, energy, and other hardware constraints, it is difficult to create devices that completely capture the complexity of human touch. Here we explore whether a sparse representation of human touch is sufficient to convey social touch signals. To test this we collected a dataset of social touch interactions using a soft wearable pressure sensor array, developed an algorithm to map recorded data to an array of actuators, then applied our algorithm to create signals that drive an array of normal indentation actuators placed on the arm. Using this wearable, low-resolution, low-force device, we find that users are able to distinguish the intended social meaning, and compare performance to results based on direct human touch. As online communication becomes more prevalent, such systems to convey haptic signals could allow for improved distant socializing and empathetic remote human-human interaction.

  View details for DOI 10.1109/TOH.2021.3129067

  View details for Web of Science ID 000818844700016

  View details for PubMedID 34793305

• A 4-Degree-of-Freedom Parallel Origami Haptic Device for Normal, Shear, and Torsion Feedback IEEE ROBOTICS AND AUTOMATION LETTERS Williams, S. R., Suchoski, J. M., Chua, Z., Okamura, A. M. 2022; 7 (2): 3310-3317

  View details for DOI 10.1109/LRA.2022.3144798

  View details for Web of Science ID 000753553400020

• Predicting Hand-Object Interaction for Improved Haptic Feedback in Mixed Reality IEEE ROBOTICS AND AUTOMATION LETTERS Salvato, M., Heravi, N., Okamura, A. M., Bohg, J. 2022; 7 (2): 3851-3857

  View details for DOI 10.1109/LRA.2022.3148458

  View details for Web of Science ID 000756831900029

• Design of a Wearable Vibrotactile Stimulation Device for Individuals With Upper-Limb Hemiparesis and Spasticity IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING Seim, C. E., Ritter, B., Starner, T. E., Flavin, K., Lansberg, M. G., Okamura, A. M. 2022; 30: 1277-1287

  Abstract

  Vibratory stimulation may improve post-stroke symptoms such as spasticity; however, current studies are limited by the large, clinic-based apparatus used to apply this stimulation. A wearable device could provide vibratory stimulation in a mobile form, enabling further study of this technique. An initial device, the vibrotactile stimulation (VTS) Glove, was deployed in an eight-week clinical study in which sixteen individuals with stroke used the device for several hours daily. Participants reported wearing the glove during activities such as church, social events, and dining out. However, 69% of participants struggled to extend or insert their fingers to don the device. In a follow-up study, eight individuals with stroke evaluated new VTS device prototypes in a three-round iterative design study with the aims of creating the next generation of VTS devices and understanding features that influence interaction with a wearable device by individuals with impaired upper-limb function. Interviews and interaction tasks were used to define actionable design revisions between each round of evaluation. Our analysis identified six new themes from participants regarding device designs: hand supination is challenging, separate finger attachments inhibit fit and use, fingers may be flexed or open, fabric coverage impacts comfort, a reduced concern for social comfort, and the affected hand is infrequently used. Straps that wrap around the arm and fixtures on the anterior arm were other challenging features. We discuss potential accommodations for these challenges, as well as social comfort. New VTS device designs are presented and were donned in an average time of 48 seconds.

  View details for DOI 10.1109/TNSRE.2022.3174808

  View details for Web of Science ID 000797424600010

  View details for PubMedID 35552152

• Characterization of Real-time Haptic Feedback from Multimodal Neural Network-based Force Estimates during Teleoperation Chua, Z., Okamura, A. M., IEEE IEEE. 2022: 1471-1478

  View details for DOI 10.1109/IROS47612.2022.9981662

  View details for Web of Science ID 000908368201062

• Perception of Mechanical Properties via Wrist Haptics: Effects of Feedback Congruence Sarac, M., Di Luca, M., Okamura, A. M., IEEE IEEE. 2022: 620-627

  View details for DOI 10.1109/IROS47612.2022.9982079

  View details for Web of Science ID 000908368200060

• Haptic Feedback Relocation from the Fingertips to the Wrist for Two-Finger Manipulation in Virtual Reality Palmer, J. E., Sarac, M., Garza, A. A., Okamura, A. M., IEEE IEEE. 2022: 628-633

  View details for DOI 10.1109/IROS47612.2022.9981392

  View details for Web of Science ID 000908368200061

• A Large-Area Wearable Soft Haptic Device Using Stacked Pneumatic Pouch Actuation Nunez, C. M., Do, B. H., Low, A. K., Blumenschein, L. H., Yamane, K., Okamura, A. M., IEEE IEEE. 2022: 591-598

  View details for DOI 10.1109/IROS47612.2022.9981919

  View details for Web of Science ID 000908368200057

• Deep Learning Classification of Touch Gestures Using Distributed Normal and Shear Force Choi, H., Brouwer, D., Lin, M. A., Yoshida, K. T., Rognon, C., Stephens-Fripp, B., Okamura, A. M., Cutkosky, M. R., IEEE IEEE. 2022: 3659-3665

  View details for DOI 10.1109/IROS47612.2022.9981457

  View details for Web of Science ID 000908368202109

• FingerPrint: A 3-D Printed Soft Monolithic 4-Degree-of-Freedom Fingertip Haptic Device with Embedded Actuation Zhakypov, Z., Okamura, A. M., IEEE IEEE. 2022: 938-944

  View details for DOI 10.1109/ROBOSOFT54090.2022.9762107

  View details for Web of Science ID 000945389700094

• A Lightweight, High-Extension, Planar 3-Degree-of-Freedom Manipulator Using Pinched Bistable Tapes Osele, O., Okamura, A. M., Do, B. H., IEEE IEEE. 2022

  View details for DOI 10.1109/ICRA.46639.2022.9811976

  View details for Web of Science ID 000941265700108

• Between-Tactor Display Using Dynamic Tactile Stimuli Eguchi, R., Vacek, D., Godzinski, C., Curry, S., Evans, M., Okamura, A. M., Seifi, H., Kappers, A. M., Schneider, O., Drewing, K., Pacchierotti, C., Abbasimoshaei, A., Huisman, G., Kern, T. A. SPRINGER INTERNATIONAL PUBLISHING AG. 2022: 379-381

  View details for Web of Science ID 000886345900051

• Feasibility of Smartphone Vibrations as a Sensory Diagnostic Tool Adenekan, R. G., Lowber, A. J., Huerta, B. N., Okamura, A. M., Yoshida, K. T., Nunez, C. M., Seifi, H., Kappers, A. M., Schneider, O., Drewing, K., Pacchierotti, C., Abbasimoshaei, A., Huisman, G., Kern, T. A. SPRINGER INTERNATIONAL PUBLISHING AG. 2022: 337-339

  View details for Web of Science ID 000886345900039

• Wearable Haptic Device for Individuals with Congenital Absence of Proprioception Kodali, S., Okamura, A. M., Bulea, T. C., Chesler, A. T., Bonnemann, C. G., Seifi, H., Kappers, A. M., Schneider, O., Drewing, K., Pacchierotti, C., Abbasimoshaei, A., Huisman, G., Kern, T. A. SPRINGER INTERNATIONAL PUBLISHING AG. 2022: 433-435

  View details for Web of Science ID 000886345900066

• Geometric Solutions for General Actuator Routing on Inflated-Beam Soft Growing Robots IEEE TRANSACTIONS ON ROBOTICS Blumenschein, L. H., Koehler, M., Usevitch, N. S., Hawkes, E., Rucker, D., Okamura, A. M. 2021

  View details for DOI 10.1109/TRO.2021.3115230

  View details for Web of Science ID 000733460200001

• Effects of Peripheral Haptic Feedback on Intracortical Brain-Computer Interface Control and Associated Sensory Responses in Motor Cortex IEEE TRANSACTIONS ON HAPTICS Deo, D. R., Rezaii, P., Hochberg, L. R., Okamura, A. M., Shenoy, K., Henderson, J. M. 2021; 14 (4): 762-775

  Abstract

  Intracortical brain-computer interfaces (iBCIs) provide people with paralysis a means to control devices with signals decoded from brain activity. Despite recent impressive advances, these devices still cannot approach able-bodied levels of control. To achieve naturalistic control and improved performance of neural prostheses, iBCIs will likely need to include proprioceptive feedback. With the goal of providing proprioceptive feedback via mechanical haptic stimulation, we aim to understand how haptic stimulation affects motor cortical neurons and ultimately, iBCI control. We provided skin shear haptic stimulation as a substitute for proprioception to the back of the neck of a person with tetraplegia. The neck location was determined via assessment of touch sensitivity using a monofilament test kit. The participant was able to correctly report skin shear at the back of the neck in 8 unique directions with 65% accuracy. We found motor cortical units that exhibited sensory responses to shear stimuli, some of which were strongly tuned to the stimuli and well modeled by cosine-shaped functions. In this article, we also demonstrated online iBCI cursor control with continuous skin-shear feedback driven by decoded command signals. Cursor control performance increased slightly but significantly when the participant was given haptic feedback, compared to the purely visual feedback condition.

  View details for DOI 10.1109/TOH.2021.3072615

  View details for Web of Science ID 000731146900007

  View details for PubMedID 33844633

• Teaching With Hapkit Enabling Online Haptics Courses With Hands-On Laboratories IEEE ROBOTICS & AUTOMATION MAGAZINE Morimoto, T. K., Martinez, M., Davis, R. L., Blikstein, P., Okamura, A. M. 2021; 28 (3): 79-91

  View details for DOI 10.1109/MRA.2020.3024395

  View details for Web of Science ID 000695640900006

• Body-Mounted Vibrotactile Stimuli: Simultaneous Display of Taps on the Fingertips and Forearm IEEE TRANSACTIONS ON HAPTICS Williams, S. R., Okamura, A. M. 2021; 14 (2): 432-444

  Abstract

  In this article, we present a body-mounted tactile display to deliver haptic feedback to the forearm and user-initiated haptic feedback to the fingertips. The display mounts two vibrotactile actuators on the forearm, leaving the user's hands free for manipulation tasks when the hands are not interacting with the tactile display, while also exploiting the tactile sensitivity of the fingertips when needed. We test the effectiveness of the display using paired vibrotactile taps sensed through the forearm and the fingertips, either separately or simultaneously. We measure the ability of participants to identify the vibrotactile taps. The results show that mounting the device on the forearm, so that the participant touches the forearm-mounted device with their fingertips receiving feedback to both locations simultaneously, decreases performance relative to mounting on the fingertips unless large amplitudes are used. We also test the accuracy with which participants identified different numbers of vibration taps (4, 8, 16, and 25 signals). The results show that as the number of signals changes, participant accuracy is not different when stimulating the fingertips alone compared to stimulating the fingertips and forearm together. We conclude with an example of a portable and wearable vibration display, and discuss future use cases of such a display.

  View details for DOI 10.1109/TOH.2020.3042955

  View details for Web of Science ID 000665616100026

  View details for PubMedID 33290228

• Affective Ratings of Vibrotactile Signals in Older Adults With and Without History of Stroke Seim, C. E., Ritter, B., Flavin, K. E., Lansberg, M. G., Okamura, A. M., IEEE IEEE. 2021: 457-462

  View details for DOI 10.1109/WHC49131.2021.9517216

  View details for Web of Science ID 000707066600060

• A Dynamics Simulator or Soft Growing Robots Jitosho, R., Agharese, N., Okamura, A., Manchester, Z., IEEE IEEE. 2021: 11775-11781

  View details for DOI 10.1109/ICRA48506.2021.9561420

  View details for Web of Science ID 000771405404012

• Macro-Mini Actuation of Pneumatic Pouches for Soft Wearable Haptic Displays Do, B. H., Okamura, A. M., Yamane, K., Blumenschein, L. H., IEEE IEEE. 2021: 14499-14505

  View details for DOI 10.1109/ICRA48506.2021.9560786

  View details for Web of Science ID 000771405405125

• Toward Force Estimation in Robot-Assisted Surgery using Deep Learning with Vision and Robot State Chua, Z., Jarc, A. M., Okamura, A. M., IEEE IEEE. 2021: 12335-12341

  View details for DOI 10.1109/ICRA48506.2021.9560945

  View details for Web of Science ID 000771405404054

• Augmented Haptic Guidance for Needle Insertion with a 2-DoF Wrist-Worn Haptic Device Sarac, M., Loke, D., Evans, M., Chong, O., Saunders, J., Makled, B., Okamura, A. M., Hallett, K., IEEE IEEE. 2021: 872

  View details for DOI 10.1109/WHC49131.2021.9517164

  View details for Web of Science ID 000707066600104

• Embedded Laser-Cut Constraints for Elastomeric Soft Actuators Winston, C. E., Yoshida, K. T., Williams, S. R., Okamura, A. M., IEEE IEEE. 2021: 863

  View details for DOI 10.1109/WHC49131.2021.9517206

  View details for Web of Science ID 000707066600095

• Human Perception of Wrist Torque Magnitude During Upper and Lower Extremity Movement Welker, C., Collins, S. H., Okamura, A. M., IEEE IEEE. 2021: 870

  View details for DOI 10.1109/WHC49131.2021.9517214

  View details for Web of Science ID 000707066600102

• Augmented Needle Decompression Task with a Wrist-Worn Haptic Device Sarac, M., Hallett, K., Saunders, J., Makled, B., Okamura, A. M., IEEE IEEE. 2021: 873

  View details for DOI 10.1109/WHC49131.2021.9517260

  View details for Web of Science ID 000707066600105

• Design, Modeling, Control, and Application of Everting Vine Robots. Frontiers in robotics and AI Blumenschein, L. H., Coad, M. M., Haggerty, D. A., Okamura, A. M., Hawkes, E. W. 2020; 7: 548266

  Abstract

  In nature, tip-localized growth allows navigation in tightly confined environments and creation of structures. Recently, this form of movement has been artificially realized through pressure-driven eversion of flexible, thin-walled tubes. Here we review recent work on robots that "grow" via pressure-driven eversion, referred to as "everting vine robots," due to a movement pattern that is similar to that of natural vines. We break this work into four categories. First, we examine the design of everting vine robots, highlighting tradeoffs in material selection, actuation methods, and placement of sensors and tools. These tradeoffs have led to application-specific implementations. Second, we describe the state of and need for modeling everting vine robots. Quasi-static models of growth and retraction and kinematic and force-balance models of steering and environment interaction have been developed that use simplifying assumptions and limit the involved degrees of freedom. Third, we report on everting vine robot control and planning techniques that have been developed to move the robot tip to a target, using a variety of modalities to provide reference inputs to the robot. Fourth, we highlight the benefits and challenges of using this paradigm of movement for various applications. Everting vine robot applications to date include deploying and reconfiguring structures, navigating confined spaces, and applying forces on the environment. We conclude by identifying gaps in the state of the art and discussing opportunities for future research to advance everting vine robots and their usefulness in the field.

  View details for DOI 10.3389/frobt.2020.548266

  View details for PubMedID 33501315

  View details for PubMedCentralID PMC7805729

• Continuous Closed-Loop 4-Degree-of-Freedom Holdable Haptic Guidance IEEE ROBOTICS AND AUTOMATION LETTERS Walker, J. M., Okamura, A. M. 2020; 5 (4): 6853–60

  View details for DOI 10.1109/LRA.2020.3020581

  View details for Web of Science ID 000569783100001

• Efficient and Trustworthy Social Navigation via Explicit and Implicit Robot-Human Communication IEEE TRANSACTIONS ON ROBOTICS Che, Y., Okamura, A. M., Sadigh, D. 2020; 36 (3): 692–707

  View details for DOI 10.1109/TRO.2020.2964824

  View details for Web of Science ID 000543027200008

• Model-Based Design of a Soft 3-D Haptic Shape Display IEEE TRANSACTIONS ON ROBOTICS Koehler, M., Usevitch, N. S., Okamura, A. M. 2020; 36 (3): 613–28

  View details for DOI 10.1109/TRO.2020.2980114

  View details for Web of Science ID 000543027200003

• 3D Electromagnetic Reconfiguration Enabled by Soft Continuum Robots IEEE ROBOTICS AND AUTOMATION LETTERS Gan, L. T., Blumenschein, L. H., Huang, Z., Okamura, A. M., Hawkes, E. W., Fan, J. A. 2020; 5 (2): 1704–11

  View details for DOI 10.1109/LRA.2020.2969922

  View details for Web of Science ID 000526520500005

• Retraction of Soft Growing Robots Without Buckling IEEE ROBOTICS AND AUTOMATION LETTERS Coad, M. M., Thomasson, R. P., Blumenschein, L. H., Usevitch, N. S., Hawkes, E. W., Okamura, A. M. 2020; 5 (2): 2115–22

  View details for DOI 10.1109/LRA.2020.2970629

  View details for Web of Science ID 000526572000003

• Robust navigation of a soft growing robot by exploiting contact with the environment INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Greer, J. D., Blumenschein, L. H., Alterovitz, R., Hawkes, E. W., Okamura, A. M. 2020

  View details for DOI 10.1177/0278364920903774

  View details for Web of Science ID 000523938000001

• An untethered isoperimetric soft robot. Science robotics Usevitch, N. S., Hammond, Z. M., Schwager, M., Okamura, A. M., Hawkes, E. W., Follmer, S. 2020; 5 (40)

  Abstract

  For robots to be useful for real-world applications, they must be safe around humans, be adaptable to their environment, and operate in an untethered manner. Soft robots could potentially meet these requirements; however, existing soft robotic architectures are limited by their ability to scale to human sizes and operate at these scales without a tether to transmit power or pressurized air from an external source. Here, we report an untethered, inflated robotic truss, composed of thin-walled inflatable tubes, capable of shape change by continuously relocating its joints, while its total edge length remains constant. Specifically, a set of identical roller modules each pinch the tube to create an effective joint that separates two edges, and modules can be connected to form complex structures. Driving a roller module along a tube changes the overall shape, lengthening one edge and shortening another, while the total edge length and hence fluid volume remain constant. This isoperimetric behavior allows the robot to operate without compressing air or requiring a tether. Our concept brings together advantages from three distinct types of robots-soft, collective, and truss-based-while overcoming certain limitations of each. Our robots are robust and safe, like soft robots, but not limited by a tether; are modular, like collective robots, but not limited by complex subunits; and are shape-changing, like truss robots, but not limited by rigid linear actuators. We demonstrate two-dimensional (2D) robots capable of shape change and a human-scale 3D robot capable of punctuated rolling locomotion and manipulation, all constructed with the same modular rollers and operating without a tether.

  View details for DOI 10.1126/scirobotics.aaz0492

  View details for PubMedID 33022597

• An untethered isoperimetric soft robot SCIENCE ROBOTICS Usevitch, N. S., Hammond, Z. M., Schwager, M., Okamura, A. M., Hawkes, E. W., Follmer, S. 2020; 5 (40)

  View details for DOI 10.1126/scirobotics.aaz0492

  View details for Web of Science ID 000520870400001

• Investigating Social Haptic Illusions for Tactile Stroking (SHIFTS) Nunez, C. M., Huerta, B. N., Okamura, A. M., Culbertson, H., IEEE IEEE. 2020: 629–36

  View details for DOI 10.1109/HAPTICS45997.2020.ras.HAP20.35.f631355d

  View details for Web of Science ID 000578523300012

• Teleoperation of an ankle-foot prosthesis with a wrist exoskeleton. IEEE transactions on bio-medical engineering Welker, C. G., Chiu, V. L., Voloshina, A. n., Collins, S. n., Okamura, A. M. 2020; PP

  Abstract

  We aimed to develop a system for people with amputation that non-invasively restores missing control and sensory information for an ankle-foot prosthesis.In our approach, a wrist exoskeleton allows people with amputation to control and receive feedback from their prosthetic ankle via teleoperation. We implemented two control schemes: position control with haptic feedback of ankle torque at the wrist; and torque control that allows the user to modify a baseline torque profile by moving their wrist against a virtual spring. We measured tracking error and frequency response for the ankle-foot prosthesis and the wrist exoskeleton. To demonstrate feasibility and evaluate system performance, we conducted an experiment in which one participant with a transtibial amputation tracked desired wrist trajectories during walking, while we measured wrist and ankle response.Benchtop testing demonstrated that for relevant walking frequencies, system error was below human perceptual error. During the walking experiment, the participant was able to voluntarily follow different wrist trajectories with an average RMS error of 1.55° after training. The ankle was also able to track desired trajectories below human perceptual error for both position control (RMSE = 0.8°) and torque control (RMSE = 8.4%).We present a system that allows a user with amputation to control an ankle-foot prosthesis and receive feedback about its state using a wrist exoskeleton, with accuracy comparable to biological neuromotor control.This bilateral teleoperation system enables novel prosthesis control and feedback strategies that could improve prosthesis control and aid motor learning.

  View details for DOI 10.1109/TBME.2020.3046357

  View details for PubMedID 33347402

• Human Interface for Teleoperated Object Manipulation with a Soft Growing Robot Stroppa, F., Luo, M., Yoshida, K., Coad, M. M., Blumenschein, L. H., Okamura, A. M., IEEE IEEE. 2020: 726-732

  View details for Web of Science ID 000712319500083

• Dynamically Reconfigurable Discrete Distributed Stiffness for Inflated Beam Robots Do, B. H., Banashek, V., Okamura, A. M., IEEE IEEE. 2020: 9050-9056

  View details for Web of Science ID 000712319505140

• Evaluation of Non-collocated Force Feedback Driven by Signal-independent Noise Chua, Z., Okamura, A. M., Deo, D. R., IEEE IEEE. 2020: 3686-3692

  View details for Web of Science ID 000712319502084

• A Tip Mount for Transporting Sensors and Tools using Soft Growing Robots Jeong, S., Coad, M. M., Blumenschein, L. H., Luo, M., Mehmood, U., Kim, J., Okamura, A. M., Ryu, J., IEEE IEEE. 2020: 8781-8788

  View details for DOI 10.1109/IROS45743.2020.9340950

  View details for Web of Science ID 000724145802131

• AFREEs: Active Fiber Reinforced Elastomeric Enclosures Yoshida, K. T., Ren, X., Blumenschein, L. H., Okamura, A. M., Luo, M., IEEE IEEE. 2020: 305–11

  View details for Web of Science ID 000610491800036

• Understanding Continuous and Pleasant Linear Sensations on the Forearm From a Sequential Discrete Lateral Skin-Slip Haptic Device IEEE TRANSACTIONS ON HAPTICS Nunez, C. M., Williams, S. R., Okamura, A. M., Culbertson, H. 2019; 12 (4): 414–27

  Abstract

  A continuous stroking sensation on the skin can convey messages or emotion cues. We seek to induce this sensation using a combination of illusory motion and lateral stroking via a haptic device. Our system provides discrete lateral skin-slip on the forearm with rotating tactors, which independently provide lateral skin-slip in a timed sequence. We vary the sensation by changing the angular velocity and delay between adjacent tactors, such that the apparent speed of the perceived stroke ranges from 2.5 to 48.2 cm/s. We investigated which actuation parameters create the most pleasant and continuous sensations through a user study with 16 participants. On average, the sensations were rated by participants as both continuous and pleasant. The most continuous and pleasant sensations were created by apparent speeds of 7.7 and 5.1 cm/s, respectively. We also investigated the effect of spacing between contact points on the pleasantness and continuity of the stroking sensation, and found that the users experience a pleasant and continuous linear sensation even when the space between contact points is relatively large (40 mm). Understanding how sequential discrete lateral skin-slip creates continuous linear sensations can influence the design and control of future wearable haptic devices.

  View details for DOI 10.1109/TOH.2019.2941190

  View details for Web of Science ID 000505585900003

  View details for PubMedID 31536015

• Soft Haptic Device to Render the Sensation of Flying Like a Drone IEEE ROBOTICS AND AUTOMATION LETTERS Rognone, C., Koehler, M., Duriez, C., Floreano, D., Okamura, A. M. 2019; 4 (3): 2524–31

  View details for DOI 10.1109/LRA.2019.2907432

  View details for Web of Science ID 000464913800005

• Resonant Frequency Skin Stretch for Wearable Haptics. IEEE transactions on haptics Shull, P. B., Tan, T., Culbertson, H. M., Zhu, X., Okamura, A. 2019

  Abstract

  Resonant frequency skin stretch uses cyclic lateral skin stretches matching the skin's resonant frequency to create highly noticeable stimuli, signifying a new approach for wearable haptic stimulation. Three experiments were performed to explore biomechanical and perceptual aspects of resonant frequency skin stretch. In the first experiment, effective skin resonant frequencies were quantified at the forearm, shank, and foot. In the second experiment, perceived haptic stimuli were characterized for skin stretch actuations across a spectrum of frequencies. In the third experiment, haptic classification ability was determined as subjects differentiated haptic stimulation cues while sitting, walking, and jogging. Results showed that subjects perceived stimulations at, above, and below the skin's resonant frequency differently: stimulations lower than the skin resonant frequency felt like distinct impacts, stimulations at the skin resonant frequency felt like cyclic skin stretches, and stimulations higher than the skin resonant frequency felt like standard vibrations. Subjects successfully classified stimulations while sitting, walking, and jogging, and classification accuracy decreased with increasing speed, especially for stimulations at the shank. This work could facilitate more widespread use of wearable skin stretch. Potential applications include gaming, medical simulation, and surgical augmentation, and for training to reduce injury risk or improve sports performance.

  View details for DOI 10.1109/TOH.2019.2917072

  View details for PubMedID 31095499

• Stiffness Control of Deformable Robots Using Finite Element Modeling IEEE ROBOTICS AND AUTOMATION LETTERS Koehler, M., Okamura, A. M., Duriez, C. 2019; 4 (2): 469–76

  View details for DOI 10.1109/LRA.2019.2890897

  View details for Web of Science ID 000456652300026

• Design and Analysis of Pneumatic 2-DoF Soft Haptic Devices for Shear Display IEEE ROBOTICS AND AUTOMATION LETTERS Kanjanapas, S., Nunez, C. M., Williams, S. R., Okamura, A. M., Luo, M. 2019; 4 (2): 1365–71

  View details for DOI 10.1109/LRA.2019.2895890

  View details for Web of Science ID 000459538100025

• Evaluation of Skin Deformation Tactile Feedback for Teleoperated Surgical Tasks IEEE TRANSACTIONS ON HAPTICS Quek, Z., Provancher, W. R., Okamura, A. M. 2019; 12 (2): 102–13

  Abstract

  During interaction with objects using a tool, we experience force and tactile feedback. One form of tactile feedback is local fingerpad skin deformation. In this paper, we provide haptic feedback to users of a teleoperation system through a skin deformation tactile feedback device. The device is able to provide tangential and normal skin deformation in a coupled manner, and is designed so that users can grasp it with a precision grip using multiple fingerpads. By applying skin deformation feedback on multiple fingerpads, the device is able to provide multi-degree-of-freedom interaction force direction and magnitude information to the user. To evaluate the effectiveness of this approach for the performance of teleoperated manipulation tasks, we performed a study in which 20 participants used a teleoperation system to perform one of two manipulation tasks (peg transfer and tube connection) using force feedback, skin deformation feedback, and the combination of both feedback. Results showed that participants are able to use all feedback to improve task performance compared to the case without haptic feedback, although the degree of improvement depended on the nature of the task. The feedback also improved situation awareness, felt consistent with prior experience, and did not affect concentration on the task, as reported by participants.

  View details for DOI 10.1109/TOH.2018.2873398

  View details for Web of Science ID 000472569100001

  View details for PubMedID 30281480

• Effects of Different Hand-Grounding Locations on Haptic Performance With a Wearable Kinesthetic Haptic Device IEEE ROBOTICS AND AUTOMATION LETTERS Nisar, S., Martinez, M., Endo, T., Matsuno, F., Okamura, A. M. 2019; 4 (2): 351–58

  View details for DOI 10.1109/LRA.2018.2890198

  View details for Web of Science ID 000456652300011

• Evolution and Analysis of Hapkit: An Open-Source Haptic Device for Educational Applications. IEEE transactions on haptics Orta Martinez, M. n., Nunez, C. M., Liao, T. n., Morimoto, T. K., Okamura, A. n. 2019

  Abstract

  We present the design, evolution and analysis of "Hapkit", a low-cost, open-source kinesthetic haptic device for use in educational applications. Hapkit was developed in 2013 based on the design of the Stanford Haptic Paddle, with the goal of decreasing cost and increasing accessibility for educational applications, including online teaching, K-12 school use, and college dynamic systems and control courses. In order to develop Hapkit for these purposes, we tested a variety of transmission, actuation, and structural materials. Hapkit 3.0, the latest version, uses a capstan drive, inexpensive DC motor, and 3-D printed structural materials. A frequency-domain system identification method was used to characterize Hapkit dynamics across the various designs. This method was validated using a first principles parameter measurement and a transient response analysis. This characterization shows that Hapkit 3.0 has lower damping and Coulomb friction than previous designs. We also performed a user study demonstrating that Hapkit 3.0 improves discrimination of virtual stiffness compared to previous designs. The design evolution of Hapkit resulted in a low-cost, high-performance device appropriate for open-source dissemination and educational applications.

  View details for DOI 10.1109/TOH.2019.2948609

  View details for PubMedID 31634847

• Perception of a Wearable Haptic Feedback Device to Render the Sensation of Flight Rognon, C., Koehler, M., Floreano, D., Okamura, A. M., IEEE IEEE. 2019: 61–66

  View details for Web of Science ID 000538798200011

• 3-DoF Wearable, Pneumatic Haptic Device to Deliver Normal, Shear, Vibration, and Torsion Feedback Yoshida, K. T., Nunez, C. M., Williams, S. R., Okamura, A. M., Luo, M., IEEE IEEE. 2019: 97–102

  View details for Web of Science ID 000538798200017

• Holdable Haptic Device for 4-DOF Motion Guidance Walker, J. M., Zemiti, N., Poignet, P., Okamura, A. M., IEEE IEEE. 2019: 109–14

  View details for Web of Science ID 000538798200019

• Vine Robots: Design, Teleoperation, and Deployment for Navigation and Exploration IEEE Robotics & Automation Magazine Coad, M., Blumenschein, L., Cutler, S., Reyna Zepeda, J., Naclerio, N., ElHussieny, H., Mehmood, U., Ryu, J., Hawkes, E., Okamura, A. 2019

  View details for DOI 10.1109/MRA.2019.2947538

• Facilitating Human-Mobile Robot Communication via Haptic Feedback and Gesture Teleoperation ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION Che, Y., Culbertson, H., Tang, C., Aich, S., Okamura, A. M. 2018; 7 (3)

  View details for DOI 10.1145/3243503

  View details for Web of Science ID 000457673200003

• A Soft, Steerable Continuum Robot That Grows via Tip Extension SOFT ROBOTICS Greer, J. D., Morimoto, T. K., Okamura, A. M., Hawkes, E. W. 2019; 6 (1): 95–108

  View details for DOI 10.1089/soro.2018.0034

  View details for Web of Science ID 000447723100001

• Toward the Design of Personalized Continuum Surgical Robots ANNALS OF BIOMEDICAL ENGINEERING Morimoto, T. K., Greer, J. D., Hawkes, E. W., Hsieh, M. H., Okamura, A. M. 2018; 46 (10): 1522–33

  View details for DOI 10.1007/s10439-018-2062-2

  View details for Web of Science ID 000445180100008

• Toward the Design of Personalized Continuum Surgical Robots. Annals of biomedical engineering Morimoto, T. K., Greer, J. D., Hawkes, E. W., Hsieh, M. H., Okamura, A. M. 2018

  Abstract

  Robot-assisted minimally invasive surgical systems enable procedures with reduced pain, recovery time, and scarring compared to traditional surgery. While these improvements benefit a large number of patients, safe access to diseased sites is not always possible for specialized patient groups, including pediatric patients, due to their anatomical differences. We propose a patient-specific design paradigm that leverages the surgeon's expertise to design and fabricate robots based on preoperative medical images. The components of the patient-specific robot design process are a virtual reality design interface enabling the surgeon to design patient-specific tools, 3-D printing of these tools with a biodegradable polyester, and an actuation and control system for deployment. The designed robot is a concentric tube robot, a type of continuum robot constructed from precurved, elastic, nesting tubes. We demonstrate the overall patient-specific design workflow, from preoperative images to physical implementation, for an example clinical scenario: nonlinear renal access to a pediatric kidney. We also measure the system's behavior as it is deployed through real and artificial tissue. System integration and successful benchtop experiments in ex vivo liver and in a phantom patient model demonstrate the feasibility of using a patient-specific design workflow to plan, fabricate, and deploy personalized, flexible continuum robots.

  View details for PubMedID 29855755

• Haptic Dimensions of Human-Robot Interaction ACM TRANSACTIONS ON HUMAN-ROBOT INTERACTION Okamura, A. M. 2018; 7 (1)

  View details for DOI 10.1145/3209768

  View details for Web of Science ID 000457700500007

• A Tip-Extending Soft Robot Enables Reconfigurable and Deployable Antennas IEEE ROBOTICS AND AUTOMATION LETTERS Blumenschein, L. H., Gan, L. T., Fan, J. A., Okamura, A. M., Hawkes, E. W. 2018; 3 (2): 949–56

  View details for DOI 10.1109/LRA.2018.2793303

  View details for Web of Science ID 000424646100014

• Comparing Proprioceptive Acuity in the Arm between Joint Space and Task Space Sketch, S. M., Bastian, A. J., Okamura, A. M., Kuchenbecker, K. J., Gerling, G. J., Visell, Y. IEEE. 2018: 125–32

  View details for Web of Science ID 000434958700021

• Obstacle-Aided Navigation of a Soft Growing Robot Greer, J. D., Blumenschein, L. H., Okamura, A. M., Hawkes, E. W., IEEE IEEE COMPUTER SOC. 2018: 4165–72

  View details for Web of Science ID 000446394503026

• HapWRAP: Soft Growing Wearable Haptic Device Agharese, N., Cloyd, T., Blumenschein, L. H., Raitor, M., Hawkes, E. W., Culbertson, H., Okamura, A. M., IEEE IEEE COMPUTER SOC. 2018: 5466–72

  View details for Web of Science ID 000446394504019

• Scaling Inertial Forces to Alter Weight Perception in Virtual Reality Suchoski, J. M., Martinez, S., Okamura, A. M., IEEE IEEE COMPUTER SOC. 2018: 484–89

  View details for Web of Science ID 000446394500051

• Effects of Latency and Refresh Rate on Force Perception via Sensory Substitution by Force-Controlled Skin Deformation Feedback Zook, Z. A., Okamura, A. M., Kamikawa, Y., IEEE IEEE COMPUTER SOC. 2018: 506–11

  View details for Web of Science ID 000446394500052

• APAM: Antagonistic Pneumatic Artificial Muscle Usevitch, N. S., Okamura, A. M., Hawkes, E. W., IEEE IEEE COMPUTER SOC. 2018: 1539–46

  View details for Web of Science ID 000446394501027

• Avoiding Human-Robot Collisions using Haptic Communication Che, Y., Sun, C. T., Okamura, A. M., IEEE IEEE COMPUTER SOC. 2018: 5828–34

  View details for Web of Science ID 000446394504057

• Magnified Force Sensory Substitution for Telemanipulation via Force-Controlled Skin Deformation Kamikawa, Y., Enayati, N., Okamura, A. M., IEEE IEEE COMPUTER SOC. 2018: 4142–48

  View details for Web of Science ID 000446394503025

• Development and Evaluation of an Intuitive Flexible Interface for Teleoperating Soft Growing Robots El-Hussieny, H., Mehmood, U., Mehdi, Z., Jeong, S., Usman, M., Hawkes, E. W., Okamura, A. M., Ryu, J., Kosecka, J., Maciejewski, A. A., Okamura, A., Bicchi, A., Stachniss, C., Song, D. Z., Lee, D. H., Chaumette, F., Ding, H., Li, J. S., Wen, J., Roberts, J., Masamune, K., Chong, N. Y., Amato, N., Tsagwarakis, N., Rocco, P., Asfour, T., Chung, W. K., Yasuyoshi, Y., Sun, Y., Maciekeski, T., Althoefer, K., AndradeCetto, J., Chung, W. K., Demircan, E., Dias, J., Fraisse, P., Gross, R., Harada, H., Hasegawa, Y., Hayashibe, M., Kiguchi, K., Kim, K., Kroeger, T., Li, Y., Ma, S., Mochiyama, H., Monje, C. A., Rekleitis, Roberts, R., Stulp, F., Tsai, C. H., Zollo, L. IEEE. 2018: 4995–5002

  View details for Web of Science ID 000458872704086

• Gaussian Process Dynamic Programming for Optimizing Ungrounded Haptic Guidance Walker, J. M., Okamura, A. M., Kochenderfer, M. J., Kosecka, J., Maciejewski, A. A., Okamura, A., Bicchi, A., Stachniss, C., Song, D. Z., Lee, D. H., Chaumette, F., Ding, H., Li, J. S., Wen, J., Roberts, J., Masamune, K., Chong, N. Y., Amato, N., Tsagwarakis, N., Rocco, P., Asfour, T., Chung, W. K., Yasuyoshi, Y., Sun, Y., Maciekeski, T., Althoefer, K., AndradeCetto, J., Chung, W. K., Demircan, E., Dias, J., Fraisse, P., Gross, R., Harada, H., Hasegawa, Y., Hayashibe, M., Kiguchi, K., Kim, K., Kroeger, T., Li, Y., Ma, S., Mochiyama, H., Monje, C. A., Rekleitis, Roberts, R., Stulp, F., Tsai, C. H., Zollo, L. IEEE. 2018: 8758–64

  View details for Web of Science ID 000458872707119

• Haptics: The Present and Future of Artificial Touch Sensation ANNUAL REVIEW OF CONTROL, ROBOTICS, AND AUTONOMOUS SYSTEMS, VOL 1 Culbertson, H., Schorr, S. B., Okamura, A. M., Leonard, N. E. 2018; 1: 385–409

  View details for DOI 10.1146/annurev-control-060117-105043

  View details for Web of Science ID 000467687700016

• A Social Haptic Device to Create Continuous Lateral Motion using Sequential Normal Indentation Culbertson, H., Nunez, C. M., Israr, A., Lau, F., Abnousi, F., Okamura, A. M., Kuchenbecker, K. J., Gerling, G. J., Visell, Y. IEEE. 2018: 32–39

  View details for Web of Science ID 000434958700006

• Robotic Assistance-as-Needed for Enhanced Visuomotor Learning in Surgical Robotics Training: An Experimental Study Enayati, N., Okamura, A. M., Mariani, A., Pellegrini, E., Coad, M. M., Ferrigno, G., De Momi, E., IEEE IEEE COMPUTER SOC. 2018: 6631–36

  View details for Web of Science ID 000446394505012

• A soft robot that navigates its environment through growth. Science robotics Hawkes, E. W., Blumenschein, L. H., Greer, J. D., Okamura, A. M. 2017; 2 (8)

  Abstract

  Across kingdoms and length scales, certain cells and organisms navigate their environments not through locomotion but through growth. This pattern of movement is found in fungal hyphae, developing neurons, and trailing plants, and is characterized by extension from the tip of the body, length change of hundreds of percent, and active control of growth direction. This results in the abilities to move through tightly constrained environments and form useful three-dimensional structures from the body. We report a class of soft pneumatic robot that is capable of a basic form of this behavior, growing substantially in length from the tip while actively controlling direction using onboard sensing of environmental stimuli; further, the peak rate of lengthening is comparable to rates of animal and robot locomotion. This is enabled by two principles: Pressurization of an inverted thin-walled vessel allows rapid and substantial lengthening of the tip of the robot body, and controlled asymmetric lengthening of the tip allows directional control. Further, we demonstrate the abilities to lengthen through constrained environments by exploiting passive deformations and form three-dimensional structures by lengthening the body of the robot along a path. Our study helps lay the foundation for engineered systems that grow to navigate the environment.

  View details for DOI 10.1126/scirobotics.aan3028

  View details for PubMedID 33157883

• Design of a Compact Actuation and Control System for Flexible Medical Robots. IEEE robotics and automation letters Morimoto, T. K., Hawkes, E. W., Okamura, A. M. 2017; 2 (3): 1579-1585

  Abstract

  Flexible medical robots can improve surgical procedures by decreasing invasiveness and increasing accessibility within the body. Using preoperative images, these robots can be designed to optimize a procedure for a particular patient. To minimize invasiveness and maximize biocompatibility, the actuation units of flexible medical robots should be placed fully outside the patient's body. In this letter, we present a novel, compact, lightweight, modular actuation, and control system for driving a class of these flexible robots, known as concentric tube robots. A key feature of the design is the use of three-dimensional printed waffle gears to enable compact control of two degrees of freedom within each module. We measure the precision and accuracy of a single actuation module and demonstrate the ability of an integrated set of three actuation modules to control six degrees of freedom. The integrated system drives a three-tube concentric tube robot to reach a final tip position that is on average less than 2 mm from a given target. In addition, we show a handheld manifestation of the device and present its potential applications.

  View details for DOI 10.1109/LRA.2017.2676240

  View details for PubMedID 28664187

  View details for PubMedCentralID PMC5484156

• Highly Articulated Robotic Needle Achieves Distributed Ablation of Liver Tissue. IEEE robotics and automation letters Gerboni, G., Greer, J. D., Laeseke, P. F., Hwang, G. L., Okamura, A. M. 2017; 2 (3): 1367-1374

  Abstract

  Robotic needle steering will improve percutaneous radio-frequency ablation (RFA) in the liver by performing distributed ablations without requiring multiple punctures of the liver capsule, thus enabling the treatment of large or multifocal tumors. However, state-of-the-art asymmetric-tip robotic needle steering systems do not yet achieve clinically relevant curvature. This work presents the design and development of a highly articulated needle that enables distributed RFA in liver tissue under ultrasound (US) image guidance. Our new needle design attains the target curvature required for liver procedures while meeting important clinical requirements, such as the use of fixed diameter needle introducers, presence of a free needle working channel, robustness for repeated insertions, and conductivity for the delivery of RF current for tissue ablation. The new needle tip includes two important design features: A tendon-actuated Nitinol asymmetric flexure joint, which allows for an active amplification of the needle steering force, and a steel back-bevel tip profile, which decreases the risk of needle jamming. The needle's resulting curvature was evaluated in both phantom and ex vivo liver tissues using segmented US images. The average radius of minimum curvature in ex vivo liver tissue was found to be 33.6 mm, the smallest reported to date. Furthermore, RFA in ex vivo porcine liver tissue tests were performed to demonstrate that distributedablation with a single puncture of the liver capsule is possible via robotic needle steering.

  View details for DOI 10.1109/LRA.2017.2668467

  View details for PubMedID 28664186

  View details for PubMedCentralID PMC5484158

• Series Pneumatic Artificial Muscles (sPAMs) and Application to a Soft Continuum Robot. IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation Greer, J. D., Morimoto, T. K., Okamura, A. M., Hawkes, E. W. 2017; 2017: 5503–10

  Abstract

  We describe a new series pneumatic artificial muscle (sPAM) and its application as an actuator for a soft continuum robot. The robot consists of three sPAMs arranged radially round a tubular pneumatic backbone. Analogous to tendons, the sPAMs exert a tension force on the robot's pneumatic backbone, causing bending that is approximately constant curvature. Unlike a traditional tendon driven continuum robot, the robot is entirely soft and contains no hard components, making it safer for human interaction. Models of both the sPAM and soft continuum robot kinematics are presented and experimentally verified. We found a mean position accuracy of 5.5 cm for predicting the end-effector position of a 42 cm long robot with the kinematic model. Finally, closed-loop control is demonstrated using an eye-in-hand visual servo control law which provides a simple interface for operation by a human. The soft continuum robot with closed-loop control was found to have a step-response rise time and settling time of less than two seconds.

  View details for PubMedID 29379672

• Three-dimensional skin deformation as force substitution: Wearable device design and performance during haptic exploration of virtual environments. IEEE transactions on haptics Schorr, S. B., Okamura, A. 2017

  Abstract

  Virtual reality systems would benefit from a compelling force sensory substitute when workspace or stability limitations prevent the use of kinesthetic force feedback systems. We present a wearable fingertip haptic device with the ability to make and break contact in addition to rendering both shear and normal skin deformation to the fingerpad. A delta mechanism with novel bias spring and tether actuator relocation method enables the use of high-end motors and encoders, allowing precise device control: 10 Hz bandwidth and 0.255 mm RMS tracking error were achieved during testing. In the first of two experiments, participants determined the orientation of a stiff region in a surrounding compliant virtual surface with an average angular error of 7.6, similar to that found in previous studies using traditional force feedback. In the second experiment, we evaluated participants' ability to interpret differences in friction. The Just Noticeable Difference (JND) of surface friction coefficient discrimination using our skin deformation device was 0.20, corresponding with a reference friction coefficient of 0.5. While higher than that found using kinesthetic feedback, this demonstrates that users can perceive differences in surface friction without world-grounded kinesthetic forces. These experiments show that three DoF skin deformation enables both stiffness and friction discrimination capability in the absence of kinesthetic force feedback.

  View details for DOI 10.1109/TOH.2017.2672969

  View details for PubMedID 28237933

• Deformable Model-Based Methods for Shape Control of a Haptic Jamming Surface. IEEE transactions on visualization and computer graphics Stanley, A. A., Okamura, A. M. 2017; 23 (2): 1029-1041

  Abstract

  Haptic Jamming, the approach of simultaneously controlling mechanical properties and surface deformation of a tactile display via particle jamming and pneumatics, shows promise as a tangible, shape-changing human-computer interface. Previous research introduced device design and described the force-displacement interactions for individual jamming cells. The work in this article analyzes the shape output capabilities of a multi-cell array. A spring-mass deformable body simulation combines models of the three actuation inputs of a Haptic Jamming surface: node pinning, chamber pressurization, and cell jamming. Surface measurements of a 12-cell prototype from a depth camera fit the mass and stiffness parameters to the device during pressurization tests and validate the accuracy of the model for various actuation sequences. The simulator is used to develop an algorithm that generates a sequence of actuation inputs for a Haptic Jamming array of any size in order to match a desired surface output shape. Data extracted from topographical maps and three-dimensional solid object models are used to evaluate the shape-matching algorithm and assess the utility of increasing array size and resolution. Results show that a discrete Laplace operator applied to the input is a suitable predictor of the correlation coefficient between the desired shape and the device output.

  View details for DOI 10.1109/TVCG.2016.2525788

  View details for PubMedID 26863666

• Perception of force and stiffness in the presence of low-frequency haptic noise. PloS one Gurari, N., Okamura, A. M., Kuchenbecker, K. J. 2017; 12 (6)

  Abstract

  This work lays the foundation for future research on quantitative modeling of human stiffness perception. Our goal was to develop a method by which a human's ability to perceive suprathreshold haptic force stimuli and haptic stiffness stimuli can be affected by adding haptic noise.Five human participants performed a same-different task with a one-degree-of-freedom force-feedback device. Participants used the right index finger to actively interact with variations of force (∼5 and ∼8 N) and stiffness (∼290 N/m) stimuli that included one of four scaled amounts of haptically rendered noise (None, Low, Medium, High). The haptic noise was zero-mean Gaussian white noise that was low-pass filtered with a 2 Hz cut-off frequency; the resulting low-frequency signal was added to the force rendered while the participant interacted with the force and stiffness stimuli.We found that the precision with which participants could identify the magnitude of both the force and stiffness stimuli was affected by the magnitude of the low-frequency haptically rendered noise added to the haptic stimulus, as well as the magnitude of the haptic stimulus itself. The Weber fraction strongly correlated with the standard deviation of the low-frequency haptic noise with a Pearson product-moment correlation coefficient of ρ > 0.83. The mean standard deviation of the low-frequency haptic noise in the haptic stimuli ranged from 0.184 N to 1.111 N across the four haptically rendered noise levels, and the corresponding mean Weber fractions spanned between 0.042 and 0.101.The human ability to perceive both suprathreshold haptic force and stiffness stimuli degrades in the presence of added low-frequency haptic noise. Future work can use the reported methods to investigate how force perception and stiffness perception may relate, with possible applications in haptic watermarking and in the assessment of the functionality of peripheral pathways in individuals with haptic impairments.

  View details for DOI 10.1371/journal.pone.0178605

  View details for PubMedID 28575068

• Haptic Orientation Guidance Using Two Parallel Double-Gimbal Control Moment Gyroscopes. IEEE transactions on haptics Walker, J. n., Culbertson, H. n., Raitor, M. n., Okamura, A. n. 2017

  Abstract

  This paper presents a system of two double-gimbal control moment gyroscopes (CMGs) for providing ungrounded kinesthetic haptic feedback. By spinning a second flywheel opposite the first, and rotating them through opposite trajectories, undesired gyroscopic effects can be eliminated, isolating a single torque axis. This produces a moment pulse proportional to the flywheel spin speed and rotation speed. Rotating the CMG gimbals quickly in one direction, then resetting them more slowly generates repeated torque pulses indicating a clear direction cue. We present the mathematical model for moments produced by this system and verify that the performance of our device matches this model. Using these asymmetric moment pulses, we provide haptic cues to participants in two studies. In the first study, users simply identify the direction of torque cues. In the second study, we use the torque pulses to guide users to target orientations. Performance in both studies shows that this system has the potential to provide useful guidance for applications where ungrounded haptic feedback is desired.

  View details for DOI 10.1109/TOH.2017.2713380

  View details for PubMedID 28600261

• Design of Patient-Specific Concentric Tube Robots Using Path Planning from 3-D Ultrasound Morimoto, T. K., Cerrolaza, J. J., Hsieh, M. H., Cleary, K., Okamura, A. M., Linguraru, M., IEEE IEEE. 2017: 165–68

  Abstract

  Percutaneous techniques and robot-assisted surgical systems have enabled minimally invasive procedures that offer reduced scarring, recovery time, and complications compared to traditional open surgeries. Despite these improvements, access to diseased sites using the standard, straight needle-based percutaneous techniques is still limited for certain procedures due to intervening tissues. These limitations can be further exacerbated in specific patient groups, particularly pediatric patients, whose anatomy does not fit the traditional tools and systems. We therefore propose a patient-specific paradigm to design and fabricate dexterous, robotic tools based on the patient's preoperative images. In this paper, we present the main steps of our proposed paradigm - image-based path planning, robot design, and fabrication - along with an example case that focuses on a class of dexterous, snake-like tools called concentric tube robots. We demonstrate planning a safe path using a patient's preoperative ultrasound images. We then determine the concentric tube robot parameters needed to achieve this path, and finally, we use 3-D printing to fabricate the patient-specific robot.

  View details for Web of Science ID 000427085300041

  View details for PubMedID 29059836

• Simulating the impact of sensorimotor deficits on reaching performance Sketch, S. M., Simpson, C. S., Crevecoeur, F., Okamura, A. M., Amirabdollahian, F., Burdet, E., Masia, L. IEEE. 2017: 31–37

  Abstract

  The healthy human nervous system accurately and robustly controls movements despite nonlinear dynamics, noise, and delays. After a stroke, motor ability frequently becomes impaired. To provide insight into the relative impact of specific sensorimotor deficits on motor performance, we modeled neural control of reaching with the human upper limb as a near-optimally feedback-controlled two-degree-of-freedom system with biologically based parameters. We added three sensorimotor impairments commonly associated with post-stroke hemiparesis - abnormal joint coupling, increased noise on internally modeled dynamics, and muscular weakness - and examined the impact on reaching performance. We found that abnormal joint coupling unknown to the system's internal model caused systematic perturbations to trajectories, longer reach durations, and target overshoot. Increasing internal model noise and muscular weakness had little impact on motor performance unless model noise was increased by several orders of magnitude. Many reaches performed by our perturbed models replicate features commonly observed in reaches by hemiparetic stroke survivors. The sensitivity to unmodeled abnormal joint coupling agrees with experimental findings that abnormal coupling (possibly related to internal model errors) is the main cause of post-stroke motor impairment.

  View details for Web of Science ID 000426850800006

  View details for PubMedID 28813789

• Fingertip Tactile Devices for Virtual Object Manipulation and Exploration Schorr, S. B., Okamura, A. M., ACM ASSOC COMPUTING MACHINERY. 2017: 3115–19

  View details for DOI 10.1145/3025453.3025744

  View details for Web of Science ID 000426970503006

• WAVES: A Wearable Asymmetric Vibration Excitation System for Presenting Three-Dimensional Translation and Rotation Cues Culbertson, H., Walker, J. M., Raitor, M., Okamura, A. M., ACM ASSOC COMPUTING MACHINERY. 2017: 4972–82

  View details for DOI 10.1145/3025453.3025741

  View details for Web of Science ID 000426970504071

• Design of a Soft Catheter for Low-Force and Constrained Surgery Slade, P., Gruebele, A., Hammond, Z., Raitor, M., Okamura, A. M., Hawkes, E. W., Bicchi, A., Okamura, A. IEEE. 2017: 174–80

  View details for Web of Science ID 000426978200025

• Analysis of Effective Impedance Transmitted to the Operator in Position-Exchange Bilateral Teleoperation Colonnese, N., Okamura, A. M., Gerling, G., Otaduy, M. A., Ryu, J. H. IEEE. 2017: 328–33

  View details for Web of Science ID 000426705900056

• Propagation of Joint Space Quantization Error to Operational Space Coordinates and Their Derivatives Colonnese, N., Okamura, A. M., Bicchi, A., Okamura, A. IEEE. 2017: 2054–61

  View details for Web of Science ID 000426978202048

• Training in Divergent and Convergent Force Fields During 6-DOF Teleoperation with a Robot-Assisted Surgical System Coad, M. M., Okamura, A. M., Wren, S., Mintz, Y., Lendvay, T. S., Jarc, A. M., Nisky, I., Gerling, G., Otaduy, M. A., Ryu, J. H. IEEE. 2017: 195–200

  View details for Web of Science ID 000426705900034

• Open Source, Modular, Customizable, 3-D Printed Kinesthetic Haptic Devices Martinez, M., Campion, J., Gholami, T., Rittikaidachar, M. K., Barron, A. C., Okamura, A. M., Gerling, G., Otaduy, M. A., Ryu, J. H. IEEE. 2017: 142–47

  View details for Web of Science ID 000426705900025

• Design of 3-D Printed Concentric Tube Robots IEEE TRANSACTIONS ON ROBOTICS Morimoto, T. K., Okamura, A. M. 2016; 32 (6): 1419-1430

  View details for DOI 10.1109/TRO.2016.2602368

  View details for Web of Science ID 000389849700008

• Design of 3-D Printed Concentric Tube Robots. IEEE transactions on robotics : a publication of the IEEE Robotics and Automation Society Morimoto, T. K., Okamura, A. M. 2016; 32 (6): 1419-1430

  Abstract

  Concentric tube surgical robots are minimally invasive devices with the advantages of snake-like reconfigurability, long and thin form factor, and placement of actuation outside the patient's body. These robots can also be designed and manufactured to acquire targets in specific patients for treating specific diseases in a manner that minimizes invasiveness. We propose that concentric tube robots can be manufactured using 3-D printing technology on a patient- and procedure-specific basis. In this paper, we define the design requirements and manufacturing constraints for 3-D printed concentric tube robots and experimentally demonstrate the capabilities of these robots. While numerous 3-D printing technologies and materials can be used to create such robots, one successful example uses selective laser sintering to make an outer tube with a polyether block amide and uses stereolithography to make an inner tube with a polypropylene-like material. This enables a tube pair with precurvatures of 0.0775 and 0.0455 mm-1, which can withstand strains of 20% and 5.5% for the outer and inner tubes, respectively.

  View details for DOI 10.1109/TRO.2016.2602368

  View details for PubMedID 28713227

  View details for PubMedCentralID PMC5505693

• Stability and quantization-error analysis of haptic rendering of virtual stiffness and damping INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Colonnese, N., Okamura, A. 2016; 35 (9): 1103-1120

  View details for DOI 10.1177/0278364915596234

  View details for Web of Science ID 000380936500005

• Methods for Improving the Curvature of Steerable Needles in Biological Tissue IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Adebar, T. K., Greer, J. D., Laeseke, P. F., Hwang, G. L., Okamura, A. M. 2016; 63 (6): 1167-1177

  Abstract

  Robotic needle steering systems have the potential to improve percutaneous interventions such as radiofrequency ablation of liver tumors, but steering techniques described to date have not achieved sufficiently small radius of curvature in biological tissue to be relevant to this application. In this study, the impact of tip geometry on steerable needle curvature was examined.Finite-element simulations and experiments with bent-tip needles in ex vivo liver tissue were performed. Motivated by the results of this analysis, a new articulated-tip steerable needle was designed, in which a distal section is actively switched by a robotic system between a straight tip (resulting in a straight path) and a bent tip (resulting in a curved path).Selection of tip length and angle can greatly improve curvature, with radius of curvature below 5 cm in liver tissue possible through judicious selection of these parameters. An articulated-tip mechanism allows the tip length and angle to be increased, while the straight configuration allows the needle tip to still pass through an introducer sheath and rotate inside the body.Validation testing in liver tissue shows that the new articulated-tip steerable needle achieves smaller radius of curvature compared to bent-tip needles described in previous work.Steerable needles with optimized tip parameters, which can generate tight curves in liver tissue, increase the clinical relevance of needle steering to percutaneous interventions.

  View details for DOI 10.1109/TBME.2015.2484262

  View details for Web of Science ID 000377045500009

  View details for PubMedID 26441438

  View details for PubMedCentralID PMC4862936

• Surgeon Design Interface for Patient-Specific Concentric Tube Robots. Proceedings of the ... IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics Morimoto, T. K., Greer, J. D., Hsieh, M. H., Okamura, A. M. 2016; 2016: 41-48

  Abstract

  Concentric tube robots have potential for use in a wide variety of surgical procedures due to their small size, dexterity, and ability to move in highly curved paths. Unlike most existing clinical robots, the design of these robots can be developed and manufactured on a patient- and procedure-specific basis. The design of concentric tube robots typically requires significant computation and optimization, and it remains unclear how the surgeon should be involved. We propose to use a virtual reality-based design environment for surgeons to easily and intuitively visualize and design a set of concentric tube robots for a specific patient and procedure. In this paper, we describe a novel patient-specific design process in the context of the virtual reality interface. We also show a resulting concentric tube robot design, created by a pediatric urologist to access a kidney stone in a pediatric patient.

  View details for DOI 10.1109/BIOROB.2016.7523596

  View details for PubMedID 28656124

  View details for PubMedCentralID PMC5483336

• A Framework for Multilateral Manipulation in Surgical Tasks IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING Nichols, K. A., Okamura, A. M. 2016; 13 (1): 68-77

  View details for DOI 10.1109/TASE.2015.2499195

  View details for Web of Science ID 000374443300009

• Motor learning affects car-to-driver handover in automated vehicles. Science robotics Russell, H. E., Harbott, L. K., Nisky, I. n., Pan, S. n., Okamura, A. M., Gerdes, J. C. 2016; 1 (1)

  Abstract

  Vehicles in the foreseeable future will be required to transition between autonomous driving (without human involvement) and full human control. During this transition period, the human, who has not been actively engaged in the driving process, must resume the motor control necessary to steer the car. The in-car study presented here demonstrates that when human drivers are presented with a steering behavior that is different from the last time they were in control, specifically the ratio of hand wheel angle to road wheel angle (emulating a change in vehicle speed), they undergo a significant period of adaptation before they return to their previous steering behavior. However, drivers do not require an adaptation period to return to previous driving behavior after changes in steering torque. These findings have implications for the design of vehicles that transition from automated to manual driving and for understanding of human motor control in real-world tasks.

  View details for DOI 10.1126/scirobotics.aah5682

  View details for PubMedID 33157857

• Tactor-Induced Skin Stretch as a Sensory Substitution Method in Teleoperated Palpation IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS Schorr, S. B., Quek, Z. F., Nisky, I., Provancher, W. R., Okamura, A. M. 2015; 45 (6): 714-726

  View details for DOI 10.1109/THMS.2015.2463090

  View details for Web of Science ID 000366893700006

• Rendered and Characterized Closed-Loop Accuracy of Impedance-Type Haptic Displays IEEE TRANSACTIONS ON HAPTICS Colonnese, N., Siu, A. F., Abbott, C. M., Okamura, A. M. 2015; 8 (4): 434-446

  Abstract

  Impedance-type kinesthetic haptic displays aim to render arbitrary desired dynamics to a human operator using force feedback. To render realistic virtual environments, the difference between desired and rendered dynamics must be small. In this paper, we analyze the closed-loop dynamics of haptic displays for three common virtual environments: a spring, a damper, and a spring-damper, including the effects of time delay and low-pass filtering. Using a linear model, we identify important parameters for the rendered dynamics in terms of effective impedances, a conceptual tool that decomposes the displays closed-loop impedance into components with physical analogs. Our results establish bandwidth limits for rendering effective stiffness and damping. The effective stiffness bandwidth is limited by the virtual stiffness and device mass, and the effective damping bandwidth is limited by the cut-off frequency of the low-pass filter which filters the device velocity estimate. We show that a general system impedance can be characterized by a mass, damper, and spring optimally by the solution to a convex optimization problem, and we present a quantitative metric, the Average Distortion Error (ADE), to describe the fidelity of this model. Time delay has no significant effect on characterized stiffness, and reduces characterized damping by the product of virtual stiffness and total time delay. Reducing the low-pass filter cut-off frequency reduces the characterized damping. Experimental data gathered with a Phantom Premium 1.5 validates the theoretical analysis. We also conducted human user experiments to investigate the effects of time delay and low-pass filtering on perceived stiffness and damping. Similar to the characterized dynamics results, we observed no significant effect of time delay on perceived stiffness, and increasing time delay resulted in reduced perceived damping. Lower filter cut-off frequencies resulted in lower perceived damping. This work informs haptic display design by presenting how closed-loop behavior changes with key parameters.

  View details for DOI 10.1109/TOH.2015.2457438

  View details for Web of Science ID 000369611500008

  View details for PubMedID 26208363

• M-Width: Stability, noise characterization, and accuracy of rendering virtual mass INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Colonnese, N., Okamura, A. M. 2015; 34 (6): 781-798

  View details for DOI 10.1177/0278364914559294

  View details for Web of Science ID 000353981500004

• Remote Electromagnetic Vibration of Steerable Needles for Imaging in Power Doppler Ultrasound. IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation Cabreros, S. S., Jimenez, N. M., Greer, J. D., Adebar, T. K., Okamura, A. M. 2015; 2015: 2244-2249

  Abstract

  Robotic needle steering systems for minimally invasive medical procedures require complementary medical imaging systems to track the needles in real time. Ultrasound is a promising imaging modality because it offers relatively low-cost, real-time imaging of the needle. Previous methods applied vibration to the base of the needle using a voice coil actuator, in order to make the needle visible in power Doppler ultrasound. We propose a new method for needle tip vibration, using electromagnetic actuation of small permanent magnets placed inside the needle to improve needle tip visibility in power Doppler imaging. Robotic needle insertion experiments using artificial tissue and ex vivo porcine liver showed that the electromagnetic tip vibration method can generate a stronger Doppler response compared to the previous base vibration method, resulting in better imaging at greater needle depth in tissue. It also eliminates previous issues with vibration damping along the shaft of the needle.

  View details for DOI 10.1109/ICRA.2015.7139496

  View details for PubMedID 26413379

  View details for PubMedCentralID PMC4578321

• Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback IEEE TRANSACTIONS ON HAPTICS Quek, Z. F., Schorr, S. B., Nisky, I., Provancher, W. R., Okamura, A. M. 2015; 8 (2): 209-221

  Abstract

  During tool-mediated interaction with everyday objects, we experience kinesthetic forces and tactile sensations in the form of vibration and skin deformation at the fingerpad. Fingerpad skin deformation is caused by forces applied tangentially and normally to the fingerpad skin, resulting in tangential and normal skin displacement. We designed a device to convey 3-degree-of-freedom (DoF) force information to the user via skin deformation, and conducted two experiments to determine the devices effectiveness for force-feedback substitution and augmentation. For sensory substitution, participants used 1-DoF and 3-DoF skin deformation feedback to locate a feature in a 3-DoF virtual environment. Participants showed improved precision and shorter completion time when using 3-DoF compared to 1-DoF skin deformation feedback. For sensory augmentation, participants traced a path in space from an initial to a target location, while under guidance from force and/or skin deformation feedback. When force feedback was augmented with skin deformation, participants reduced their path-following error over the cases when force or skin deformation feedback are used separately. We conclude that 3-DoF skin deformation feedback is effective in substituting or augmenting force feedback. Such substitution or augmentation could be used when force feedback is unattainable or attenuated due to device limitations or system instability.

  View details for DOI 10.1109/TOH.2015.2398448

  View details for Web of Science ID 000356521100009

  View details for PubMedID 25647582

• Methods to Segment Hard Inclusions in Soft Tissue During Autonomous Robotic Palpation IEEE TRANSACTIONS ON ROBOTICS Nichols, K. A., Okamura, A. M. 2015; 31 (2): 344-354

  View details for DOI 10.1109/TRO.2015.2402531

  View details for Web of Science ID 000352361300010

• Artificial Tactile Sensing of Position and Slip Speed by Exploiting Geometrical Features IEEE-ASME TRANSACTIONS ON MECHATRONICS Damian, D. D., Newton, T. H., Pfeifer, R., Okamura, A. M. 2015; 20 (1): 263-274

  View details for DOI 10.1109/TMECH.2014.2321680

  View details for Web of Science ID 000352060700026

• Uncontrolled Manifold Analysis of Arm Joint Angle Variability During Robotic Teleoperation and Freehand Movement of Surgeons and Novices IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Nisky, I., Hsieh, M. H., Okamura, A. M. 2014; 61 (12): 2869-2881

  View details for DOI 10.1109/TBME.2014.2332359

  View details for Web of Science ID 000345501000006

• Testing models of cerebellar ataxia via dynamic simulation. Robotica Grow, D., Bastian, A. J., Okamura, A. M. 2014; 32 (8): 1383-1397

  Abstract

  Patients with damage to the cerebellum make reaching movements that are uncoordinated or "ataxic." One prevailing hypothesis is that the cerebellum functions as an internal model for planning movements, and that damage to the cerebellum results in movements that do not properly account for arm dynamics. An exoskeleton robot was used to record multi-joint reaching movements. Subsequently, joint-torque trajectories were calculated and a gradient descent algorithm found optimal, patient-specific perturbations to actual limb dynamics predicted to reduce directional reaching errors by an average of 41%, elucidating a promising form of robotic intervention and adding support to the internal model hypothesis.

  View details for DOI 10.1017/s0263574714002306

  View details for PubMedID 37090787

  View details for PubMedCentralID PMC10117632

• 3-D ultrasound-guided robotic needle steering in biological tissue. IEEE transactions on bio-medical engineering Adebar, T. K., Fletcher, A. E., Okamura, A. M. 2014; 61 (12): 2899-2910

  Abstract

  Robotic needle steering systems have the potential to greatly improve medical interventions, but they require new methods for medical image guidance. Three-dimensional (3-D) ultrasound is a widely available, low-cost imaging modality that may be used to provide real-time feedback to needle steering robots. Unfortunately, the poor visibility of steerable needles in standard grayscale ultrasound makes automatic segmentation of the needles impractical. A new imaging approach is proposed, in which high-frequency vibration of a steerable needle makes it visible in ultrasound Doppler images. Experiments demonstrate that segmentation from this Doppler data is accurate to within 1-2 mm. An image-guided control algorithm that incorporates the segmentation data as feedback is also described. In experimental tests in ex vivo bovine liver tissue, a robotic needle steering system implementing this control scheme was able to consistently steer a needle tip to a simulated target with an average error of 1.57 mm. Implementation of 3-D ultrasound-guided needle steering in biological tissue represents a significant step toward the clinical application of robotic needle steering.

  View details for DOI 10.1109/TBME.2014.2334309

  View details for PubMedID 25014948

• Uncontrolled manifold analysis of arm joint angle variability during robotic teleoperation and freehand movement of surgeons and novices. IEEE transactions on bio-medical engineering Nisky, I., Hsieh, M. H., Okamura, A. M. 2014; 61 (12): 2869-2881

  Abstract

  Teleoperated robot-assisted surgery (RAS) is used to perform a wide variety of minimally invasive procedures. However, current understanding of the effect of robotic manipulation on the motor coordination of surgeons is limited. Recent studies in human motor control suggest that we optimize hand movement stability and task performance while minimizing control effort and improving robustness to unpredicted disturbances. To achieve this, the variability of joint angles and muscle activations is structured to reduce task-relevant variability and increase task-irrelevant variability. In this study, we determine whether teleoperation of a da Vinci Si surgical system in a nonclinical task of simple planar movements changes this structure of variability in experienced surgeons and novices. To answer this question, we employ the UnControlled manifold analysis that partitions users' joint angle variability into task-irrelevant and task-relevant manifolds. We show that experienced surgeons coordinate their joint angles to stabilize hand movements more than novices, and that the effect of teleoperation depends on experience--experts increase teleoperated stabilization relative to freehand whereas novices decrease it. We suggest that examining users' exploitation of the task-irrelevant manifold for stabilization of hand movements may be applied to: (1) evaluation and optimization of teleoperator design and control parameters, and (2) skill assessment and optimization of training in RAS.

  View details for DOI 10.1109/TBME.2014.2332359

  View details for PubMedID 24967980

• Testing models of cerebellar ataxia via dynamic simulation ROBOTICA Grow, D., Bastian, A. J., Okamura, A. M. 2014; 32 (8): 1383-1397

  View details for DOI 10.1017/S0263574714002306

  View details for Web of Science ID 000346921000013

• 3-D Ultrasound-Guided Robotic Needle Steering in Biological Tissue IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Adebar, T. K., Fletcher, A. E., Okamura, A. M. 2014; 61 (12): 2899-2910

  View details for DOI 10.1109/TBME.2014.2334309

  View details for Web of Science ID 000345501000009

• Augmentation Of Stiffness Perception With a 1-Degree-of-Freedom Skin Stretch Device IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS Quek, Z. F., Schorr, S. B., Nisky, I., Okamura, A. M., Provancher, W. R. 2014; 44 (6): 731-742

  View details for DOI 10.1109/THMS.2014.2348865

  View details for Web of Science ID 000346225700004

• Torsional Dynamics of Steerable Needles: Modeling and Fluoroscopic Guidance IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Swensen, J. P., Lin, M., Okamura, A. M., Cowan, N. J. 2014; 61 (11): 2707-2717

  Abstract

  Needle insertions underlie a diversity of medical interventions. Steerable needles provide a means by which to enhance existing needle-based interventions and facilitate new ones. Tip-steerable needles follow a curved path and can be steered by twisting the needle base during insertion, but this twisting excites torsional dynamics that introduce a discrepancy between the base and tip twist angles. Here, we model the torsional dynamics of a flexible rod-such as a tip-steerable needle-during subsurface insertion and develop a new controller based on the model. The torsional model incorporates time-varying mode shapes to capture the changing boundary conditions inherent during insertion. Numerical simulations and physical experiments using two distinct setups-stereo camera feedback in semitransparent artificial tissue and feedback control with real-time X-ray imaging in optically opaque artificial tissue-demonstrate the need to account for torsional dynamics in control of the needle tip.

  View details for DOI 10.1109/TBME.2014.2326161

  View details for Web of Science ID 000344082000006

• Torsional dynamics of steerable needles: modeling and fluoroscopic guidance. IEEE transactions on bio-medical engineering Swensen, J. P., Lin, M., Okamura, A. M., Cowan, N. J. 2014; 61 (11): 2707-2717

  Abstract

  Needle insertions underlie a diversity of medical interventions. Steerable needles provide a means by which to enhance existing needle-based interventions and facilitate new ones. Tip-steerable needles follow a curved path and can be steered by twisting the needle base during insertion, but this twisting excites torsional dynamics that introduce a discrepancy between the base and tip twist angles. Here, we model the torsional dynamics of a flexible rod-such as a tip-steerable needle-during subsurface insertion and develop a new controller based on the model. The torsional model incorporates time-varying mode shapes to capture the changing boundary conditions inherent during insertion. Numerical simulations and physical experiments using two distinct setups-stereo camera feedback in semitransparent artificial tissue and feedback control with real-time X-ray imaging in optically opaque artificial tissue-demonstrate the need to account for torsional dynamics in control of the needle tip.

  View details for DOI 10.1109/TBME.2014.2326161

  View details for PubMedID 24860026

• Neural coding of passive lump detection in compliant artificial tissue. Journal of neurophysiology Gwilliam, J. C., Yoshioka, T., Okamura, A. M., Hsiao, S. S. 2014; 112 (5): 1131-1141

  Abstract

  Here, we investigate the neural mechanisms of detecting lumps embedded in artificial compliant tissues. We performed a combined psychophysical study of humans performing a passive lump detection task with a neurophysiological study in nonhuman primates (Macaca mulatta) where we recorded the responses of peripheral mechanoreceptive afferents to lumps embedded at various depths in intermediates (rubbers) of varying compliance. The psychophysical results reveal that human lump detection is greatly degraded by both lump depth and decreased compliance of the intermediate. The neurophysiology results reveal that only the slowly adapting type 1 (SA1) afferents provide a clear spatial representation of lumps at all depths and that the representation is affected by lump size, depth, and compliance of the intermediate. The rapidly adapting afferents are considerably less sensitive to the lump. We defined eight neural response measures that we hypothesized could explain the psychophysical behavior, including peak firing rate, spatial spread of neural activity, and additional parameters derived from these measures. We find that peak firing rate encodes the depth of the lump, and the neural spatial spread of the SA1 response encodes for lump size but not lump shape. We also find that the perception of lump size may be affected by the compliance of the intermediate. The results show that lump detection is based on a spatial population code of the SA1 afferents, which is distorted by the depth of the lump and compliance of the tissue.

  View details for DOI 10.1152/jn.00032.2013

  View details for PubMedID 24805077

  View details for PubMedCentralID PMC4122732

• Effects of robotic manipulators on movements of novices and surgeons SURGICAL ENDOSCOPY AND OTHER INTERVENTIONAL TECHNIQUES Nisky, I., Okamura, A. M., Hsieh, M. H. 2014; 28 (7): 2145-2158

  Abstract

  Robot-assisted surgery is widely adopted for many procedures but has not realized its full potential to date. Based on human motor control theories, the authors hypothesized that the dynamics of the master manipulators impose challenges on the motor system of the user and may impair performance and slow down learning. Although studies have shown that robotic outcomes are correlated with the case experience of the surgeon, the relative contribution of cognitive versus motor skill is unknown. This study quantified the effects of da Vinci Si master manipulator dynamics on movements of novice users and experienced surgeons and suggests possible implications for training and robot design.In the reported study, six experienced robotic surgeons and ten novice nonmedical users performed movements under two conditions: teleoperation of a da Vinci Si Surgical system and freehand. A linear mixed model was applied to nine kinematic metrics (including endpoint error, movement time, peak speed, initial jerk, and deviation from a straight line) to assess the effects of teleoperation and expertise. To assess learning effects, t tests between the first and last movements of each type were used.All the users moved slower during teleoperation than during freehand movements (F 1,9343 = 345; p < 0.001). The experienced surgeons had smaller errors than the novices (F 1,14 = 36.8; p < 0.001). The straightness of movements depended on their direction (F 7,9343 = 117; p < 0.001). Learning effects were observed in all conditions. Novice users first learned the task and then the dynamics of the manipulator.The findings showed differences between the novices and the experienced surgeons for extremely simple point-to-point movements. The study demonstrated that manipulator dynamics affect user movements, suggesting that these dynamics could be improved in future robot designs. The authors showed the partial adaptation of novice users to the dynamics. Future studies are needed to evaluate whether it will be beneficial to include early training sessions dedicated to learning the dynamics of the manipulator.

  View details for DOI 10.1007/s00464-014-3446-5

  View details for Web of Science ID 000338278800017

• Predicting and correcting ataxia using a model of cerebellar function BRAIN Bhanpuri, N. H., Okamura, A. M., Bastian, A. J. 2014; 137: 1931-1944

  View details for DOI 10.1093/brain/awu115

  View details for Web of Science ID 000338646800020

• Predicting and correcting ataxia using a model of cerebellar function. Brain Bhanpuri, N. H., Okamura, A. M., Bastian, A. J. 2014; 137: 1931-1944

  Abstract

  Cerebellar damage results in uncoordinated, variable and dysmetric movements known as ataxia. Here we show that we can reliably model single-joint reaching trajectories of patients (n = 10), reproduce patient-like deficits in the behaviour of controls (n = 11), and apply patient-specific compensations that improve reaching accuracy (P < 0.02). Our approach was motivated by the theory that the cerebellum is essential for updating and/or storing an internal dynamic model that relates motor commands to changes in body state (e.g. arm position and velocity). We hypothesized that cerebellar damage causes a mismatch between the brain's modelled dynamics and the actual body dynamics, resulting in ataxia. We used both behavioural and computational approaches to demonstrate that specific cerebellar patient deficits result from biased internal models. Our results strongly support the idea that an intact cerebellum is critical for maintaining accurate internal models of dynamics. Importantly, we demonstrate how subject-specific compensation can improve movement in cerebellar patients, who are notoriously unresponsive to treatment.

  View details for DOI 10.1093/brain/awu115

  View details for PubMedID 24812203

  View details for PubMedCentralID PMC4065021

• Task-dependent impedance and implications for upper-limb prosthesis control INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Blank, A. A., Okamura, A. M., Whitcomb, L. L. 2014; 33 (6): 827-846

  View details for DOI 10.1177/0278364913517728

  View details for Web of Science ID 000337560700001

• Grip Force Control during Virtual ObjectInteraction: Effect of Force Feedback,Accuracy Demands, and Training. IEEE transactions on haptics Gibo, T. L., Bastian, A. J., Okamura, A. M. 2014; 7 (1): 37-47

  Abstract

  When grasping and manipulating objects, people are able to efficiently modulate their grip force according to the experienced load force. Effective grip force control involves providing enough grip force to prevent the object from slipping, while avoiding excessive force to avoid damage and fatigue. During indirect object manipulation via teleoperation systems or in virtual environments, users often receive limited somatosensory feedback about objects with which they interact. This study examines the effects of force feedback, accuracy demands, and training on grip force control during object interaction in a virtual environment. The task required subjects to grasp and move a virtual object while tracking a target. When force feedback was not provided, subjects failed to couple grip and load force, a capability fundamental to direct object interaction. Subjects also exerted larger grip force without force feedback and when accuracy demands of the tracking task were high. In addition, the presence or absence of force feedback during training affected subsequent performance, even when the feedback condition was switched. Subjects' grip force control remained reminiscent of their employed grip during the initial training. These results motivate the use of force feedback during telemanipulation and highlight the effect of force feedback during training.

  View details for DOI 10.1109/TOH.2013.60

  View details for PubMedID 24845744

• Haptic feedback enhances rhythmic motor control by reducing variability, not improving convergence rate. Journal of neurophysiology Ankarali, M. M., Tutkun Sen, H., De, A., Okamura, A. M., Cowan, N. J. 2014; 111 (6): 1286-1299

  Abstract

  Stability and performance during rhythmic motor behaviors such as locomotion are critical for survival across taxa: falling down would bode well for neither cheetah nor gazelle. Little is known about how haptic feedback, particularly during discrete events such as the heel-strike event during walking, enhances rhythmic behavior. To determine the effect of haptic cues on rhythmic motor performance, we investigated a virtual paddle juggling behavior, analogous to bouncing a table tennis ball on a paddle. Here, we show that a force impulse to the hand at the moment of ball-paddle collision categorically improves performance over visual feedback alone, not by regulating the rate of convergence to steady state (e.g., via higher gain feedback or modifying the steady-state hand motion), but rather by reducing cycle-to-cycle variability. This suggests that the timing and state cues afforded by haptic feedback decrease the nervous system's uncertainty of the state of the ball to enable more accurate control but that the feedback gain itself is unaltered. This decrease in variability leads to a substantial increase in the mean first passage time, a measure of the long-term metastability of a stochastic dynamical system. Rhythmic tasks such as locomotion and juggling involve intermittent contact with the environment (i.e., hybrid transitions), and the timing of such transitions is generally easy to sense via haptic feedback. This timing information may improve metastability, equating to less frequent falls or other failures depending on the task.

  View details for DOI 10.1152/jn.00140.2013

  View details for PubMedID 24371296

• Effects of robotic manipulators on movements of novices and surgeons. Surgical endoscopy Nisky, I., Okamura, A. M., Hsieh, M. H. 2014

  Abstract

  Robot-assisted surgery is widely adopted for many procedures but has not realized its full potential to date. Based on human motor control theories, the authors hypothesized that the dynamics of the master manipulators impose challenges on the motor system of the user and may impair performance and slow down learning. Although studies have shown that robotic outcomes are correlated with the case experience of the surgeon, the relative contribution of cognitive versus motor skill is unknown. This study quantified the effects of da Vinci Si master manipulator dynamics on movements of novice users and experienced surgeons and suggests possible implications for training and robot design.In the reported study, six experienced robotic surgeons and ten novice nonmedical users performed movements under two conditions: teleoperation of a da Vinci Si Surgical system and freehand. A linear mixed model was applied to nine kinematic metrics (including endpoint error, movement time, peak speed, initial jerk, and deviation from a straight line) to assess the effects of teleoperation and expertise. To assess learning effects, t tests between the first and last movements of each type were used.All the users moved slower during teleoperation than during freehand movements (F 1,9343 = 345; p < 0.001). The experienced surgeons had smaller errors than the novices (F 1,14 = 36.8; p < 0.001). The straightness of movements depended on their direction (F 7,9343 = 117; p < 0.001). Learning effects were observed in all conditions. Novice users first learned the task and then the dynamics of the manipulator.The findings showed differences between the novices and the experienced surgeons for extremely simple point-to-point movements. The study demonstrated that manipulator dynamics affect user movements, suggesting that these dynamics could be improved in future robot designs. The authors showed the partial adaptation of novice users to the dynamics. Future studies are needed to evaluate whether it will be beneficial to include early training sessions dedicated to learning the dynamics of the manipulator.

  View details for DOI 10.1007/s00464-014-3446-5

  View details for PubMedID 24519031

• Mapping stiffness perception in the brain with an fMRI-compatible particle-jamming haptic interface. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference Menon, S., Stanley, A. A., Zhu, J., Okamura, A. M., Khatib, O. 2014; 2014: 2051-2056

  Abstract

  We demonstrate reliable neural responses to changes in haptic stiffness perception using a functional magnetic resonance imaging (fMRI) compatible particle-jamming haptic interface. Our haptic interface consists of a silicone tactile surface whose stiffness we can control by modulating air-pressure in a sub-surface pouch of coarsely ground particles. The particles jam together as the pressure decreases, which stiffens the surface. During fMRI acquisition, subjects performed a constant probing task, which involved continuous contact between the index fingertip and the interface and rhythmic increases and decreases in fingertip force (1.6 Hz) to probe stiffness. Without notifying subjects, we randomly switched the interface's stiffness (switch time, 300-500 ms) from soft (200 N/m) to hard (1400 N/m). Our experiment design's constant motor activity and cutaneous tactile sensation helped disassociate neural activation for both from stiffness perception, which helped localized it to a narrow region in somatosensory cortex near the supra-marginal gyrus. Testing different models of neural activation, we found that assuming indepedent stiffness-change responses at both soft-hard and hard-soft transitions provides the best explanation for observed fMRI measurements (three subjects; nine four-minute scan runs each). Furthermore, we found that neural activation related to stiffness-change and absolute stiffness can be localized to adjacent but disparate anatomical locations. We also show that classical finger-tapping experiments activate a swath of cortex and are not suitable for localizing stiffness perception. Our results demonstrate that decorrelating motor and sensory neural activation is essential for characterizing somatosensory cortex, and establish particle-jamming haptics as an attractive low-cost method for fMRI experiments.

  View details for DOI 10.1109/EMBC.2014.6944019

  View details for PubMedID 25570387

• Real-Time 3D Curved Needle Segmentation Using Combined B-Mode and Power Doppler Ultrasound 17th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) Greer, J. D., Adebar, T. K., Hwang, G. L., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2014: 381–388

  Abstract

  This paper presents a real-time segmentation method for curved needles in biological tissue based on analysis of B-mode and power Doppler images from a tracked 2D ultrasound transducer. Mechanical vibration induced by an external voice coil results in a Doppler response along the needle shaft, which is centered around the needle section in the ultrasound image. First, B-mode image analysis is performed within regions of interest indicated by the Doppler response to create a segmentation of the needle section in the ultrasound image. Next, each needle section is decomposed into a sequence of points and transformed into a global coordinate system using the tracked transducer pose. Finally, the 3D shape is reconstructed from these points. The results of this method differ from manual segmentation by 0.71 ± 0.55 mm in needle tip location and 0.38 ± 0.27 mm along the needle shaft. This method is also fast, taking 5-10 ms to run on a standard PC, and is particularly advantageous in robotic needle steering, which involves thin, curved needles with poor echogenicity.

  View details for Web of Science ID 000347686400048

• Real-time 3D curved needle segmentation using combined B-mode and power Doppler ultrasound. Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention Greer, J. D., Adebar, T. K., Hwang, G. L., Okamura, A. M. 2014; 17: 381-388

  Abstract

  This paper presents a real-time segmentation method for curved needles in biological tissue based on analysis of B-mode and power Doppler images from a tracked 2D ultrasound transducer. Mechanical vibration induced by an external voice coil results in a Doppler response along the needle shaft, which is centered around the needle section in the ultrasound image. First, B-mode image analysis is performed within regions of interest indicated by the Doppler response to create a segmentation of the needle section in the ultrasound image. Next, each needle section is decomposed into a sequence of points and transformed into a global coordinate system using the tracked transducer pose. Finally, the 3D shape is reconstructed from these points. The results of this method differ from manual segmentation by 0.71 ± 0.55 mm in needle tip location and 0.38 ± 0.27 mm along the needle shaft. This method is also fast, taking 5-10 ms to run on a standard PC, and is particularly advantageous in robotic needle steering, which involves thin, curved needles with poor echogenicity.

  View details for PubMedID 25485402

• Integration of a Particle Jamming Tactile Display with a Cable-Driven Parallel Robot 9th International Conference of the EuroHaptics on Neuroscience, Devices, Modeling, and Applications Stanley, A. A., Mayhew, D., Irwin, R., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2014: 258–265

  View details for DOI 10.1007/978-3-662-44196-1_32

  View details for Web of Science ID 000345644600032

• Recursive Estimation of Needle Pose for Control of 3D-Ultrasound-Guided Robotic Needle Steering IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Adebar, T. K., Okamura, A. M. IEEE. 2014: 4303–4308

  View details for Web of Science ID 000349834604062

• Cerebellar ataxia impairs modulation of arm stiffness during postural maintenance JOURNAL OF NEUROPHYSIOLOGY Gibo, T. L., Bastian, A. J., Okamura, A. M. 2013; 110 (7): 1611-1620

  Abstract

  Impedance control enables humans to effectively interact with their environment during postural and movement tasks, adjusting the mechanical behavior of their limbs to account for instability. Previous work has shown that people are able to selectively modulate the end-point stiffness of their arms, adjusting for varying directions of environmental disturbances. Behavioral studies also suggest that separate controllers are used for impedance modulation versus joint torque coordination. Here we tested whether people with cerebellar damage have deficits in impedance control. It is known that these individuals have poor motor coordination, which has typically been attributed to deficits in joint torque control. Subjects performed a static postural maintenance task with two different types of directional force perturbations. On average, patients with cerebellar ataxia modified stiffness differentially for the two perturbation conditions, although significantly less than age-matched control subjects. Thus cerebellar damage may impair the ability to modulate arm impedance. Surprisingly, the patients' intact ability to generally alter their limb stiffness during the postural task (albeit less than age-matched control subjects) improved their movement performance in a subsequent tracing task. The transfer of stiffness control from the static to the movement task may be a strategy that can be used by patients to compensate for their motor deficits.

  View details for DOI 10.1152/jn.00294.2013

  View details for Web of Science ID 000325346300015

  View details for PubMedID 23843434

• Predictive modeling by the cerebellum improves proprioception. journal of neuroscience Bhanpuri, N. H., Okamura, A. M., Bastian, A. J. 2013; 33 (36): 14301-14306

  Abstract

  Because sensation is delayed, real-time movement control requires not just sensing, but also predicting limb position, a function hypothesized for the cerebellum. Such cerebellar predictions could contribute to perception of limb position (i.e., proprioception), particularly when a person actively moves the limb. Here we show that human cerebellar patients have proprioceptive deficits compared with controls during active movement, but not when the arm is moved passively. Furthermore, when healthy subjects move in a force field with unpredictable dynamics, they have active proprioceptive deficits similar to cerebellar patients. Therefore, muscle activity alone is likely insufficient to enhance proprioception and predictability (i.e., an internal model of the body and environment) is important for active movement to benefit proprioception. We conclude that cerebellar patients have an active proprioceptive deficit consistent with disrupted movement prediction rather than an inability to generally enhance peripheral proprioceptive signals during action and suggest that active proprioceptive deficits should be considered a fundamental cerebellar impairment of clinical importance.

  View details for DOI 10.1523/JNEUROSCI.0784-13.2013

  View details for PubMedID 24005283

  View details for PubMedCentralID PMC3761044

• Cerebellar motor learning: are environment dynamics more important than error size? JOURNAL OF NEUROPHYSIOLOGY Gibo, T. L., Criscimagna-Hemminger, S. E., Okamura, A. M., Bastian, A. J. 2013; 110 (2): 322-333

  Abstract

  Cerebellar damage impairs the control of complex dynamics during reaching movements. It also impairs learning of predictable dynamic perturbations through an error-based process. Prior work suggests that there are distinct neural mechanisms involved in error-based learning that depend on the size of error experienced. This is based, in part, on the observation that people with cerebellar degeneration may have an intact ability to learn from small errors. Here we studied the relative effect of specific dynamic perturbations and error size on motor learning of a reaching movement in patients with cerebellar damage. We also studied generalization of learning within different coordinate systems (hand vs. joint space). Contrary to our expectation, we found that error size did not alter cerebellar patients' ability to learn the force field. Instead, the direction of the force field affected patients' ability to learn, regardless of whether the force perturbations were introduced gradually (small error) or abruptly (large error). Patients performed best in fields that helped them compensate for movement dynamics associated with reaching. However, they showed much more limited generalization patterns than control subjects, indicating that patients rely on a different learning mechanism. We suggest that patients typically use a compensatory strategy to counteract movement dynamics. They may learn to relax this compensatory strategy when the external perturbation is favorable to counteracting their movement dynamics, and improve reaching performance. Altogether, these findings show that dynamics affect learning in cerebellar patients more than error size.

  View details for DOI 10.1152/jn.00745.2012

  View details for Web of Science ID 000321843800005

  View details for PubMedID 23596337

  View details for PubMedCentralID PMC3727069

• Novel algorithm for real-time onset detection of surface electromyography in step-tracking wrist movements. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Kuroda, Y., Nisky, I., Uranishi, Y., Imura, M., Okamura, A. M., Oshiro, O. 2013; 2013: 2056-2059

  Abstract

  We present a novel algorithm for real-time detection of the onset of surface electromyography signal in step-tracking wrist movements. The method identifies abrupt increase of the quasi-tension signal calculated from sEMG resulting from the step-by-step recruitment of activated motor units. We assessed the performance of our proposed algorithm using both simulated and real sEMG signals, and compared with two existing detection methods. Evaluation with simulated sEMG showed that the detection accuracy of our method is robust to different signal-to-noise ratios, and that it outperforms the existing methods in terms of bias when the noise is large (low SNR). Evaluation with real sEMG analysis also indicated better detection performance compared to existing methods.

  View details for DOI 10.1109/EMBC.2013.6609936

  View details for PubMedID 24110123

• The effect of a robot-assisted surgical system on the kinematics of user movements. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Nisky, I., Hsieh, M. H., Okamura, A. M. 2013; 2013: 6257-6260

  Abstract

  Teleoperated robot-assisted surgery (RAS) offers many advantages over traditional minimally invasive surgery. However, RAS has not yet realized its full potential, and it is not clear how to optimally train surgeons to use these systems. We hypothesize that the dynamics of the master manipulator impact the ability of users to make desired movements with the robot. We compared freehand and teleoperated movements of novices and experienced surgeons. To isolate the effects of dynamics from procedural knowledge, we chose simple movements rather than surgical tasks. We found statistically significant effects of teleoperation and user expertise in several aspects of motion, including target acquisition error, movement speed, and movement smoothness. Such quantitative assessment of human motor performance in RAS can impact the design of surgical robots, their control, and surgeon training methods, and eventually, improve patient outcomes.

  View details for DOI 10.1109/EMBC.2013.6610983

  View details for PubMedID 24111170

• Adaptation to visuomotor rotation in isometric reaching is similar to movement adaptation. IEEE ... International Conference on Rehabilitation Robotics : [proceedings] Rotella, M. F., Koehler, M., Nisky, I., Bastian, A. J., Okamura, A. M. 2013; 2013: 1-6

  Abstract

  Isometric reaching, in which the arm remains stationary and the user controls a virtual cursor via force input, is a motor task that has not been thoroughly compared to real reaching. In this study, we ask if isometric adaptation to a kinematic perturbation is similar to adaptation in movement, and if the type of isometric mapping (position or velocity control) influences learning. Healthy subjects made real and virtual reaches with the arm in plane. In some trials, the cursor was rotated counter clockwise by 45° to perturb the kinematic mapping. To assess adaptation, the angular error of cursor movement at 150 ms from movement onset was measured for each reach; error was averaged across subjects and a two-state learning mode was fit to error data. For movement and isometric groups, average angular error peaked at perturbation onset, reduced over 200 reaches, and reversed direction when the perturbation was removed. We show that subjects are able to adapt to a visuomotor rotation in both position- and velocity-based cursor control, and that the time course of adaptation resembles that of movement adaptation. Training of virtual reaching using force/torque input could be particularly applicable for stroke patients with significant movement deficits, who could benefit from intensive treatments using simple, cost-effective devices.

  View details for DOI 10.1109/ICORR.2013.6650431

  View details for PubMedID 24187249

• Effect of age on stiffness modulation during postural maintenance of the arm. IEEE ... International Conference on Rehabilitation Robotics : [proceedings] Gibo, T. L., Bastian, A. J., Okamura, A. M. 2013; 2013: 1-6

  Abstract

  The ability to modify the mechanical impedance of our limbs allows us to perform a variety of motor control tasks while interacting with the environment in a stable manner. Prior work has shown that young, healthy people are capable of modulating arm stiffness via selective muscle co-contraction to account for external disturbances in various directions. Increased age detrimentally affects control of movement and stability, although the neural mechanisms underlying these deficits are not entirely understood. In this study, younger and older subjects performed a static postural maintenance task with two types of directional force perturbations. Older individuals showed significantly less stiffness modification between the two perturbation conditions compared to the younger individuals, indicating less optimal modulation of arm impedance. This impairment should be considered during motor control evaluation in older populations, whether it be activities of daily living or skill assessment.

  View details for DOI 10.1109/ICORR.2013.6650395

  View details for PubMedID 24187214

• Characterization and Psychophysical Studies of an Air-Jet Lump Display IEEE TRANSACTIONS ON HAPTICS Gwilliam, J. C., Bianchi, M., Su, L. K., Okamura, A. M. 2013; 6 (2): 156-166

  Abstract

  Development of tactile displays to enhance palpation of lumps during robot-assisted minimally invasive surgery is challenging due to size and weight constraints, motivating a pneumatic actuation strategy. This work describes the quantitative and psychophysical assessment of an air-jet tactile display that creates a lump percept by directing pressurized air through an aperture onto the finger. The air pressure and aperture size are meant to control the hardness and size, respectively, of the perceived lump. Jet impingement pressure and flow rate were measured by capacitive tactile sensors and mass flow meters at varying aperture sizes and pressures. The air-jet pressure profile width evolves as jet theory predicts and is largely independent of supply pressure (and therefore jet exit velocity). The method of constant stimuli was used to determine the just noticeable differences (JNDs) for the air pressure and aperture size. Qualitative results indicate that subjects perceive the stimulus as a "lump-like" shape. Pressure JNDs ranged from 19.6-24.4 kPag and aperture size JNDs ranged from 0.50-0.66 mm. No significant correlation exists between the supply pressure and changes in perceived lump size. However, pressure JNDs show significant (p < 0.001) inverse correlation with aperture size, with improved discrimination at larger apertures, where a greater finger pad area is stimulated.

  View details for DOI 10.1109/ToH.2012.71

  View details for Web of Science ID 000319877500003

• Characterization and psychophysical studies of an air-jet lump display. IEEE transactions on haptics Gwilliam, J. C., Bianchi, M., Su, L. K., Okamura, A. M. 2013; 6 (2): 156-166

  Abstract

  Development of tactile displays to enhance palpation of lumps during robot-assisted minimally invasive surgery is challenging due to size and weight constraints, motivating a pneumatic actuation strategy. This work describes the quantitative and psychophysical assessment of an air-jet tactile display that creates a lump percept by directing pressurized air through an aperture onto the finger. The air pressure and aperture size are meant to control the hardness and size, respectively, of the perceived lump. Jet impingement pressure and flow rate were measured by capacitive tactile sensors and mass flow meters at varying aperture sizes and pressures. The air-jet pressure profile width evolves as jet theory predicts and is largely independent of supply pressure (and therefore jet exit velocity). The method of constant stimuli was used to determine the just noticeable differences (JNDs) for the air pressure and aperture size. Qualitative results indicate that subjects perceive the stimulus as a "lump-like" shape. Pressure JNDs ranged from 19.6-24.4 kPag and aperture size JNDs ranged from 0.50-0.66 mm. No significant correlation exists between the supply pressure and changes in perceived lump size. However, pressure JNDs show significant (p < 0.001) inverse correlation with aperture size, with improved discrimination at larger apertures, where a greater finger pad area is stimulated.

  View details for DOI 10.1109/TOH.2012.71

  View details for PubMedID 24808300

• Does a basic deficit in force control underlie cerebellar ataxia? JOURNAL OF NEUROPHYSIOLOGY Charles, S. K., Okamura, A. M., Bastian, A. J. 2013; 109 (4): 1107-1116

  Abstract

  Because damage to the cerebellum results in characteristic movement incoordination known as "ataxia," it has been hypothesized that it is involved in estimation of limb dynamics that occur during movement. However, cerebellar function may extend beyond movement to force control in general, with or without movement. Here we tested whether the cerebellum is involved in controlling force separate from estimating limb dynamics and whether ataxia could result from a deficit in force control. We studied patients with cerebellar ataxia controlling their arm force isometrically; in this condition arm dynamics are absent and there is no need for (or effect from an impairment in) estimations of limb dynamics. Subjects were required to control their force magnitude, direction, or both. Cerebellar patients were able to match force magnitude or direction similarly to control subjects. Furthermore, when controlling force magnitude, they intuitively chose directions (not specified) that required minimal effort at the joint level--this ability was also similar to control subjects. In contrast, cerebellar patients performed significantly worse than control subjects when asked to match both force magnitude and direction. This was surprising, since they did not exhibit significant impairment in doing either in isolation. These results show that cerebellum-dependent computations are not limited to estimations of body dynamics needed for active movement. Deficits occur even in isometric conditions, but apparently only when multiple degrees of freedom must be controlled simultaneously. Thus a fundamental cerebellar operation may be combining/coordinating degrees of freedom across many kinds of movements and behaviors.

  View details for DOI 10.1152/jn.00152.2012

  View details for Web of Science ID 000315141100021

  View details for PubMedID 23175807

  View details for PubMedCentralID PMC3569121

• Coaxial Needle Insertion Assistant With Enhanced Force Feedback IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING De Lorenzo, D., Koseki, Y., De Momi, E., Chinzei, K., Okamura, A. M. 2013; 60 (2): 379-389

  Abstract

  Many medical procedures involving needle insertion into soft tissues, such as anesthesia, biopsy, brachytherapy, and placement of electrodes, are performed without image guidance. In such procedures, haptic detection of changing tissue properties at different depths during needle insertion is important for needle localization and detection of subsurface structures. However, changes in tissue mechanical properties deep inside the tissue are difficult for human operators to sense, because the relatively large friction force between the needle shaft and the surrounding tissue masks the smaller tip forces. A novel robotic coaxial needle insertion assistant, which enhances operator force perception, is presented. This one-degree-of-freedom cable-driven robot provides to the operator a scaled version of the force applied by the needle tip to the tissue, using a novel design and sensors that separate the needle tip force from the shaft friction force. The ability of human operators to use the robot to detect membranes embedded in artificial soft tissue was tested under the conditions of 1) tip force and shaft force feedback, and 2) tip force only feedback. The ratio of successful to unsuccessful membrane detections was significantly higher (up to 50%) when only the needle tip force was provided to the user.

  View details for DOI 10.1109/TBME.2012.2227316

  View details for Web of Science ID 000316809800013

  View details for PubMedID 23193302

• Kinematic analysis of motor performance in robot-assisted surgery: a preliminary study. Studies in health technology and informatics Nisky, I., Patil, S., Hsieh, M. H., Okamura, A. M. 2013; 184: 302-308

  Abstract

  The inherent dynamics of the master manipulator of a teleoperated robot-assisted surgery (RAS) system can affect the movements of a human operator, in comparison with free-space movements. To measure the effects of these dynamics on operators with differing levels of surgical expertise, a da Vinci Si system was instrumented with a custom surgeon grip fixture and magnetic pose trackers. We compared users' performance of canonical motor control movements during teleoperation with the manipulator and freehand cursor control, and found significant differences in several aspects of motion, including target acquisition error, movement speed, and acceleration. In addition, there was preliminary evidence for differences between experts and novices. These findings could impact robot design, control, and training methods for RAS.

  View details for PubMedID 23400175

• Autonomous Robotic Palpation: Machine Learning Techniques to Identify Hard Inclusions in Soft Tissues IEEE International Conference on Robotics and Automation (ICRA) Nichols, K. A., Okamura, A. M. IEEE. 2013: 4384–4389

  View details for Web of Science ID 000337617304060

• A HAPTIC SYSTEM FOR EDUCATIONAL GAMES: DESIGN AND APPLICATION-SPECIFIC KINEMATIC OPTIMIZATION ASME Dynamic Systems and Control Conference (DSCC) Kessler, J. A., Lovelace, R. C., Okamura, A. M. AMER SOC MECHANICAL ENGINEERS. 2013

  View details for DOI 10.1115/DSCC2013-4077

  View details for Web of Science ID 000337043200054

• MODEL-MEDIATED TELEOPERATION WITH PREDICTIVE MODELS AND RELATIVE TRACKING ASME Dynamic Systems and Control Conference (DSCC) Winck, R. C., Okamura, A. M. AMER SOC MECHANICAL ENGINEERS. 2013

  View details for DOI 10.1115/DSCC2013-3842

  View details for Web of Science ID 000337043200051

• ROBOT-GUIDED SHEATHS (RoGS) FOR PERCUTANEOUS ACCESS TO THE PEDIATRIC KIDNEY: PATIENT-SPECIFIC DESIGN AND PRELIMINARY RESULTS ASME Dynamic Systems and Control Conference Morimoto, T. K., Hsieh, M. H., Okamura, A. M. AMER SOC MECHANICAL ENGINEERS. 2013

  View details for DOI 10.1115/DSCC2013-3917

  View details for Web of Science ID 000335377800043

• A Framework for Analysis of Surgeon Arm Posture Variability in Robot-Assisted Surgery IEEE International Conference on Robotics and Automation (ICRA) Nisky, I., Hsieh, M. H., Okamura, A. M. IEEE. 2013: 245–251

  View details for Web of Science ID 000337617300036

• Tissue Fixation by Suction Increases the Accuracy of Robotic Needle Insertion IEEE International Conference on Robotics and Automation (ICRA) Wedlick, T. R., Lin, D. J., Okamura, A. M. IEEE. 2013: 1694–1699

  View details for Web of Science ID 000337617301104

• Sensory Substitution via Cutaneous Skin Stretch Feedback IEEE International Conference on Robotics and Automation (ICRA) Schorr, S. B., Quek, Z. F., Romano, R. Y., Nisky, I., Provancher, W. R., Okamura, A. M. IEEE. 2013: 2341–2346

  View details for Web of Science ID 000337617302052

• Sensorimotor Performance in Robot Assisted Surgery. Nisky, I., Hsieh, M., H., Okamura, A., M. 2013
• Arm Stiffness in Patients with Cerebellar Ataxia: Modulation during Postural Maintenance and its Subsequent Effects on Movement. Gibo, T., L., Bastian, A., J., Okamura, A., M. 2013
• Sensory Substitution via Cutaneous Skin Stretch Feedback. Schorr, S., B., Quek, Z., F., Romano, R., Y., Nisky, I., Provancher, W., R., Okamura, A., M. 2013
• A Haptic System for Educational Games: Design and Application-Specific Kinematic Optimization. Kessler, J., A., Lovelace, R., C., Okamura, A., M. 2013
• A Framework for Analysis of Surgeon Arm Posture Variability in Robot-Assisted Surgery. Nisky, I., Hsieh, M., H., Okamura, A., M. 2013
• Haptic Jamming: A Deformable Geometry, Variable Stiffness Tactile Display using Pneumatics and Particle Jamming IEEE World Haptics Conference (WHC) Stanley, A. A., Gwilliam, J. C., Okamura, A. M. IEEE. 2013: 25–30

  View details for Web of Science ID 000325187400005

• Sensory Augmentation of Stiffness using Fingerpad Skin Stretch IEEE World Haptics Conference (WHC) Quek, Z. F., Schorr, S. B., Nisky, I., Okamura, A. M., Provancher, W. R. IEEE. 2013: 467–472

  View details for Web of Science ID 000325187400079

• Cartesian and Joint Space Teleoperation for Nonholonomic Steerable Needles IEEE World Haptics Conference (WHC) Majewicz, A., Okamura, A. M. IEEE. 2013: 395–400

  View details for Web of Science ID 000325187400067

• Novel Algorithm for Real-Time Onset Detection of Surface Electromyography in Step-Tracking Wrist Movements. Kuroda, Y., Nisky, I., Uranishi, Y., Imura, M., Okamura, A., M., Oshiro, O. 2013
• Cerebellar Ataxia Impairs Modulation of Arm Stiffness during Postural Maintenance. Journal of Neurophysiology Gibo, T., L., Bastian, A., J., Okamura, A., M. 2013; 7 (110): 1611-1620
• A Haptic Display for Medical Simulation Using Particle Jamming. Medicine Meets Virtual Reality (Studies in Health Technology and Information). Stanley, A., Gwilliam, J., C., Okamura, A., M. 2013
• 3D Segmentation of Curved Needles using Doppler Ultrasound and Vibration. Information Processing in Computer-Assisted Interventions Adebar, T., K., Okamura, A., M. 2013: 61-70
• Kinematic Analysis of Motor Performance in Robot-Assisted Surgery: Implications for Robot Design and Surgical Training. Nisky, I., Patil, S., Hsieh, M., H., Okamura, A., M. 2013
• Robot-Guided Sheaths (RoGS) for Percutaneous Access to the Pedatric Kidney: Patient-Specific Design and Preliminary Results. Morimoto, T., K., Hsieh, M., H., Okamura, A., M. 2013
• Effect of Age on Stiffness Modulation during Postural Maintenance of the Arm. Gibo, T., L., Bastian, A., J., Okamura, A., M. 2013

  View details for DOI 10.1109/ICORR.2014.6650395

• Autonomous Robotic Palpation: Machine Learning Techniques to Identify Hard Inclusions in Soft Tissues. Nichols, K., A., Okamura, A., M. 2013
• Adaptation to Visuomotor Rotation in Isometric Virtual Reaching Parallels Learning in Movement. Rotella, M., F., Koehler, M., Nisky, I., Bastian, A., J., Okamura, A., M. 2013
• 3D Segmentation of Curved Needles using Doppler Ultrasound and Vibration. Information Processing in Computer-Assisted Interventions Adebar, T., K., Okamura, A., M. 2013: 61-70
• Transfer of isometric motor learning depends on the mapping of force input to cursor movement. Rotella, M., F., Koehler, M., Nisky, I., Bastian, A., J., Okamura, A., M. 2013
• Tissue Fixation by Suction Increases the Accuracy of Robotic Needle Insertion. Wedlick, T., R., Lin, D., J., Okamura, A., M. 2013
• Model-Mediated Teleoperation with Predictive Models and Relative Tracking. Wedlick, R., C., Okamura, A., M. 2013
• Grip Force Control During Virtual Object Interaction: Effect of Force Feedback, Accuracy Demands, and Training. IEEE Transactions on Haptics Gibo, T., L., Bastian, A., J., Okamura, A., M. 2013

  View details for DOI 10.1109/TOH.2014.60

• Perception of Springs With Visual and Proprioceptive Motion Cues: Implications for Prosthetics IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS Gurari, N., Kuchenbecker, K. J., Okamura, A. M. 2013; 43 (1): 102-114

  View details for DOI 10.1109/TSMCA.2012.2221038

  View details for Web of Science ID 000317644800010

• Behavior of Tip-Steerable Needles in Ex Vivo and In Vivo Tissue IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Majewicz, A., Marra, S. P., van Vledder, M. G., Lin, M., Choti, M. A., Song, D. Y., Okamura, A. M. 2012; 59 (10): 2705-2715

  Abstract

  Robotic needle steering is a promising technique to improve the effectiveness of needle-based clinical procedures, such as biopsies and ablation, by computer-controlled, curved insertions of needles within solid organs. In this paper, we explore the capabilities, challenges, and clinical relevance of asymmetric-tip needle steering through experiments in ex vivo and in vivo tissue. We evaluate the repeatability of needle insertion in inhomogeneous biological tissue and compare ex vivo and in vivo needle curvature and insertion forces. Steerable needles curved more in kidney than in liver and prostate, likely due to differences in tissue properties. Pre-bent needles produced higher insertion forces in liver and more curvature in vivo than ex vivo. When compared to straight stainless steel needles, steerable needles did not cause a measurable increase in tissue damage and did not exert more force during insertion. The minimum radius of curvature achieved by prebent needles was 5.23 cm in ex vivo tissue, and 10.4 cm in in vivo tissue. The curvatures achieved by bevel tip needles were negligible for in vivo tissue. The minimum radius of curvature for bevel tip needles in ex vivo tissue was 16.4 cm; however, about half of the bevel tip needles had negligible curvatures. We also demonstrate a potential clinical application of needle steering by targeting and ablating overlapping regions of cadaveric canine liver.

  View details for DOI 10.1109/TBME.2012.2204749

  View details for Web of Science ID 000308989000003

  View details for PubMedID 22711767

  View details for PubMedCentralID PMC3448818

• Active force perception depends on cerebellar function JOURNAL OF NEUROPHYSIOLOGY Bhanpuri, N. H., Okamura, A. M., Bastian, A. J. 2012; 107 (6): 1612-1620

  Abstract

  Damage to the cerebellum causes characteristic movement abnormalities but is thought to have minimal impact on somatosensory perception. Traditional clinical assessments of patients with cerebellar lesions reveal no perceptual deficits despite the fact that the cerebellum receives substantial somatosensory information. Given that abnormalities have been reported in predicting the visual consequences of movement, we suspect that the cerebellum broadly participates in perception when motor output is required (i.e., active perception). Thus we hypothesize that cerebellar integrity is essential for somatosensory perception that requires motor activity, but not passive somatosensory perception. We compared the perceptual acuity of human cerebellar patients to that of healthy control subjects in several different somatosensory perception tasks with minimal visual information. We found that patients were worse at active force and stiffness discrimination but similar to control subjects with regard to passive cutaneous force detection, passive proprioceptive detection, and passive proprioceptive discrimination. Furthermore, the severity of movement symptoms as assessed by a clinical exam was positively correlated with impairment of active force perception. Notably, within the context of these perceptual tasks, control subjects and cerebellar patients displayed similar movement characteristics, and hence differing movement strategies are unlikely to underlie the differences in perception. Our results are consistent with the hypothesis that the cerebellum is vital to sensory prediction of self-generated movement and suggest a general role for the cerebellum in multiple forms of active perception.

  View details for DOI 10.1152/jn.00983.2011

  View details for Web of Science ID 000302140300008

  View details for PubMedID 22190620

• Augmented reality and haptic interfaces for robot-assisted surgery INTERNATIONAL JOURNAL OF MEDICAL ROBOTICS AND COMPUTER ASSISTED SURGERY Yamamoto, T., Abolhassani, N., Jung, S., Okamura, A. M., Judkins, T. N. 2012; 8 (1): 45-56

  Abstract

  Current teleoperated robot-assisted minimally invasive surgical systems do not take full advantage of the potential performance enhancements offered by various forms of haptic feedback to the surgeon. Direct and graphical haptic feedback systems can be integrated with vision and robot control systems in order to provide haptic feedback to improve safety and tissue mechanical property identification.An interoperable interface for teleoperated robot-assisted minimally invasive surgery was developed to provide haptic feedback and augmented visual feedback using three-dimensional (3D) graphical overlays. The software framework consists of control and command software, robot plug-ins, image processing plug-ins and 3D surface reconstructions.The feasibility of the interface was demonstrated in two tasks performed with artificial tissue: palpation to detect hard lumps and surface tracing, using vision-based forbidden-region virtual fixtures to prevent the patient-side manipulator from entering unwanted regions of the workspace.The interoperable interface enables fast development and successful implementation of effective haptic feedback methods in teleoperation.

  View details for DOI 10.1002/rcs.421

  View details for Web of Science ID 000301120100005

  View details for PubMedID 22069247

• M-Width: Stability and Accuracy of Haptic Rendering of Virtual Mass. In Robotics: Science and Systems. Colonnese, N., Okamura, A., M. 2012
• Haptic Footstep Display. Jayakumar, R., P., Mishra, S., K., Dannenhoffer, J., F., Okamura, A., M. 2012
• Development of the KineSys MedSim: A Novel Hands-Free Haptic Robot for Medical Simulation. Medicine Meets Virtual Reality (Studies in Health Technology and Information). Judkins, T., N., Stevenson, M., Mayhew, D., Okamura, A., M. 2012
• Haptic feedback enhances rhythmic motor control performance. Ankarali, M., Mert, Sen, H., Tutkun, Okamura, Allison, M., Cowan, Noah, J. 2012
• Predicting and correcting human ataxia using a model of cerebellar function. Bhanpuri, N., Okamura, A., M., Bastian, A., J. 2012
• HAPI Bands: A Haptic Augmented Posture Interface. Rotella, M., F., Guerin, K., He, X., Okamura, A., M. 2012
• Design and Control of an Air-Jet Lump Display. Gwilliam, J., C., Degirmenci, A., Bianchi, M., Okamura, A., M. 2012
• Conveying the Configuration of a Virtual Human Hand Using Vibrotactile Feedback. Cheng, A., Nichols, K., A., Weeks, H., M., Gurari, N., Okamura, A., M. 2012
• Discrimination of Springs with Vision, Proprioception, and Artificial Skin Stretch Cues. Gurari, N., Wheeler, J., Shelton, A., Okamura, A., M. 2012
• Augmented Reality and Haptic Interfaces for Robot-Assisted Surgery. International Journal of Medical Robotics and Computer Assisted Surgery Yamamoto, T., Abolhassani, N., Jung, S., Okamura, A., M., Judkins, T., N. 2012; 1 (8): 45-46
• Characterization of Robotic Needle Insertion and Rotation in Artificial and Ex Vivo Tissues 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) / Symposium on Surgical Robotics Wedlick, T. R., Okamura, A. M. IEEE. 2012: 62–68

  View details for Web of Science ID 000313014600011

• Wearable Haptic Device for Cutaneous Force and Slip Speed Display IEEE International Conference on Robotics and Automation (ICRA) Damian, D. D., Ludersdorfer, M., Kim, Y., Arieta, A. H., Pfeifer, R., Okamura, A. M. IEEE. 2012: 1038–1043

  View details for Web of Science ID 000309406701007

• User comprehension of task performance with varying impedance in a virtual prosthetic arm: A pilot study 4th IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) / Symposium on Surgical Robotics Blank, A., Okamura, A. M., Whitcomb, L. L. IEEE. 2012: 500–507

  View details for Web of Science ID 000313014600085

• Robot-Assisted Needle Steering IEEE ROBOTICS & AUTOMATION MAGAZINE Reed, K. B., Majewicz, A., Kallem, V., Alterovitz, R., Goldberg, K., Cowan, N. J., Okamura, A. M. 2011; 18 (4): 35-46

  Abstract

  Needle insertion is a critical aspect of many medical treatments, diagnostic methods, and scientific studies, and is considered to be one of the simplest and most minimally invasive medical procedures. Robot-assisted needle steering has the potential to improve the effectiveness of existing medical procedures and enable new ones by allowing increased accuracy through more dexterous control of the needle tip path and acquisition of targets not accessible by straight-line trajectories. In this article, we describe a robot-assisted needle steering system that uses three integrated controllers: a motion planner concerned with guiding the needle around obstacles to a target in a desired plane, a planar controller that maintains the needle in the desired plane, and a torsion compensator that controls the needle tip orientation about the axis of the needle shaft. Experimental results from steering an asymmetric-tip needle in artificial tissue demonstrate the effectiveness of the system and its sensitivity to various environmental and control parameters. In addition, we show an example of needle steering in ex vivo biological tissue to accomplish a clinically relevant task, and highlight challenges of practical needle steering implementation.

  View details for DOI 10.1109/MRA.2011.942997

  View details for Web of Science ID 000297994600010

  View details for PubMedCentralID PMC3460644

• Haptics in Medicine and Clinical Skill Acquisition IEEE TRANSACTIONS ON HAPTICS Okamura, A. M., Basdogan, C., Baillie, S., Harwin, W. S. 2011; 4 (3): 153-154

  View details for Web of Science ID 000293455500001

• Experimental evaluation of a coaxial needle insertion assistant with enhanced force feedback 33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) De Lorenzo, D., Koseki, Y., De Momi, E., Chinzei, K., Okamura, A. M. IEEE. 2011: 3447–3450

  Abstract

  During needle insertion in soft tissue, detection of change in tissue properties is important both for diagnosis to detect pathological tissue and for prevention to avoid puncture of important structures. The presence of a membrane located deep inside the tissue results in a relatively small force variation at the needle tip that can be masked by relatively large friction force between the needle shaft and the surrounding tissue. Also, user perception of force can be limited due to the overall small force amplitude in some applications (e.g. brain surgery). A novel robotic coaxial needle insertion assistant was developed to enhance operator force perception. The coaxial needle separates the cutting force at the needle tip from shear friction on the needle shaft. The assistant is force controlled (admittance control), providing the operator with force feedback that is a scaled version of the force applied by the needle tip to the tissue. The effectiveness of the assistant in enhancing the detection of different tissue types was tested experimentally. Users were asked to blindly insert a needle into artificial tissues with membranes at various depths under two force feedback conditions: (1) shaft and tip force together, and (2) only tip force. The ratio of successful to unsuccessful membrane detection was significantly higher when only the needle tip force is displayed to the user. The system proved to be compliant with the clinical applications requirements.

  View details for Web of Science ID 000298810002292

  View details for PubMedID 22255081

• Robotic Needle Steering: Design, Modeling, Planning, and Image Guidance. In Surgical Robotics - Systems, Applications, and Visions. Cowan, N., J., Goldberg, K., Chirikjian, G., S., Fichtinger, G., Alterovitz, R., Reed, K., B., Okamura, A. M. edited by Rosen, J., Hannaford, B., Satava, R. Springer. 2011: 557–582
• Task-Dependent Impedance Improves User Performance with a Virtual Prosthetic Arm IEEE International Conference on Robotics and Automation (ICRA) Blank, A., Okamura, A. M., Whitcomb, L. L. IEEE. 2011: 2235–2242

  View details for Web of Science ID 000324383401071

• Characterization of an Air Jet Haptic Lump Display 33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Bianchi, M., Gwilliam, J. C., Degirmenci, A., Okamura, A. M. IEEE. 2011: 3467–3470

  Abstract

  During manual palpation, clinicians rely on distributed tactile information to identify and localize hard lumps embedded in soft tissue. The development of tactile feedback systems to enhance palpation using robot-assisted minimally invasive surgery (RMIS) systems is challenging due to size and weight constraints, motivating a pneumatic actuation strategy. Recently, an air jet approach has been proposed for generating a lump percept. We use this technique to direct a thin stream of air through an aperture directly on the finger pad, which indents the skin in a hemispherical manner, producing a compelling lump percept. We hypothesize that the perceived parameters of the lump (e.g. size and stiffness) can be controlled by jointly adjusting air pressure and the aperture size through which air escapes. In this work, we investigate how these control variables interact to affect perceived pressure on the finger pad. First, we used a capacitive tactile sensor array to measure the effect of aperture size on output pressure, and found that peak output pressure increases with aperture size. Second, we performed a psychophysical experiment for each aperture size to determine the just noticeable difference (JND) of air pressure on the finger pad. Subject-averaged pressure JND values ranged from 19.4-24.7 kPa, with no statistical differences observed between aperture sizes. The aperture-pressure relationship and the pressure JND values will be fundamental for future display control.

  View details for Web of Science ID 000298810002297

  View details for PubMedID 22255086

• Assessing the Quality of Force Feedback in Soft Tissue Simulation 33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Basafa, E., Sefati, S., Okamura, A. M. IEEE. 2011: 3451–3454

  Abstract

  Many types of deformable models have been proposed for simulation of soft tissue in surgical simulators, but their realism in comparison to actual tissue is rarely assessed. In this paper, a nonlinear mass-spring model is used for realtime simulation of deformable soft tissues and providing force feedback to a human operator. Force-deformation curves of real soft tissue samples were obtained experimentally, and the model was tuned accordingly. To test the realism of the model, we conducted two human-user experiments involving palpation with a rigid probe. First, in a discrimination test, users identified the correct category of real and virtual tissue better than chance, and tended to identify the tissues as real more often than virtual. Second, users identified real and virtual tissues by name, after training on only real tissues. The sorting accuracy was the same for both real and virtual tissues. These results indicate that, despite model limitations, the simulation could convey the feel of touching real tissues. This evaluation approach could be used to compare and validate various soft-tissue simulators.

  View details for Web of Science ID 000298810002293

  View details for PubMedID 22255082

• Gradual anisometric-isometric transition for human-machine interfaces 33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Gibo, T. L., Rotella, M. F., Bastian, A. J., Okamura, A. M. IEEE. 2011: 4507–4510

  Abstract

  Human-machine interfaces (HMIs) are widely used in biomedical applications, from teleoperated surgical systems to rehabilitation devices. This paper investigates a method of control that allows an HMI to transition from anisometric to isometric mode, shifting the control input from position to force as the user's movement is gradually reduced. Two different approaches for achieving this transition are discussed: one is based on the natural system dynamics, whereas the other involves selecting and controlling dynamics. The two approaches were implemented on a custom haptic device in a targeting task. Anisometric to isometric transitioning can potentially be used for training purposes, enabling transfer of what was learned in one mode to the other, as well as novel studies of the human sensorimotor system.

  View details for Web of Science ID 000298810003208

  View details for PubMedID 22255340

• Robot-Assisted Steerable Needles: Prototype Development and Feasibility Studies in an Animal Model. Vledder, Mark, G. Van, Majewicz, A., Marra, Steven, P., Song, Danny, Y., Okamura, Allison, M., Choti, M. A. 2011
• Do Cerebellar Patients Generalize after Abrupt or Gradual Motor Learning? Gibo, T., L., Okamura, A., M., Bastian, A., J. 2011
• Cerebellar predictions underlie the improvement of proprioceptive precision in active versus passive movements. Bhanpuri, N., Okamura, A., M., Bastian, A., J. 2011
• Task-Dependent Impedance Improves User Performance with a Virtual Prosthetic Arm. Blank, A., Okamura, A., M., Whitcomb, L., L. 2011
• Characterization of an Air Jet Haptic Lump Display. Bianchi, M., Gwilliam, J., C., Degirmenci, A., Okamura, A., M. 2011
• Cutaneous Force Display via Shaped Contacts. Ludersdorfer, M., Damian, D., D., Arieta, A., Hernandez, Pfeifer, R., Okamura, A., M. 2011
• Force Feedback and Sensory Substitution for Robot-Assisted Surgery. Surgical Robotics - Systems, Applications, and Visions Okamura, A., M., Verner, L., N., Yamamoto, T., Gwilliam, J., C., Griffiths, P., G. 2011: 419-448
• Coaxial Needle Insertion Assistant for Epidural Puncture IEEE/RSJ International Conference on Intelligent Robots and Systems Koseki, Y., De Lorenzo, D., Chinzei, K., Okamura, A. M. IEEE. 2011

  View details for Web of Science ID 000297477502141

• Design and Evaluation of a Multi-Modal Haptic Skin Stimulation Apparatus 33rd Annual International Conference of the IEEE Engineering-in-Medicine-and-Biology-Society (EMBS) Damian, D. D., Arieta, A. H., Okamura, A. M. IEEE. 2011: 3455–3458

  Abstract

  Human grasping and manipulation are facilitated by cutaneous mechanoreceptors that provide information about contact location, pressure, and events such as making and breaking contact. A challenge in designing haptic feedback devices for the wearer of a prosthetic hand is simultaneous display of multiple types of haptic information. We present the preliminary design and evaluation of an apparatus for relaying multi-modal haptic information. The apparatus moves a set of contact points tangentially over the skin at a controlled speed, with controlled normal force. We apply this stimulus to an artificial skin instrumented with an embedded accelerometer, and characterize the resulting signals. Vibration frequency increases with applied normal force and tangential speed, whereas vibration amplitude increases with normal force and depends on skin properties. The results indicate that different forces and speeds can, under some conditions, be discriminated using vibration signals alone. Accurate identification of speeds is provided by series of vibration events that depend on the spatial distribution of contact points. This study motivates future work to perform human perception studies and create a wearable haptic display for prosthetics based on this concept.

  View details for Web of Science ID 000298810002294

  View details for PubMedID 22255083

• Mechanics of Flexible Needles Robotically Steered through Soft Tissue INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Misra, S., REED, K. B., Schafer, B. W., Ramesh, K. T., Okamura, A. M. 2010; 29 (13): 1640-1660

  View details for DOI 10.1177/0278364910369714

  View details for Web of Science ID 000283847100004

• Identifying the Role of Proprioception in Upper-Limb Prosthesis Control: Studies on Targeted Motion ACM TRANSACTIONS ON APPLIED PERCEPTION Blank, A., Okamura, A. M., Kuchenbecker, K. J. 2010; 7 (3)

  View details for DOI 10.1145/1773965.1773966

  View details for Web of Science ID 000279361800001

• Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine. IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation King, H. H., Hannaford, B., Kwok, K. W., Yang, G. Z., Griffiths, P., Okamura, A., Farkhatdinov, I., Ryu, J. H., Sankaranarayanan, G., Arikatla, V., Tadano, K., Kawashima, K., Peer, A., Schauß, T., Buss, M., Miller, L., Glozman, D., Rosen, J., Low, T. 2010; 2010: 1733-1738

  Abstract

  Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.

  View details for PubMedID 24748993

  View details for PubMedCentralID PMC3988879

• Evaluation of Robotic Needle Steering in ex vivo Tissue. IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation Majewicz, A., Wedlick, T. R., Reed, K. B., Okamura, A. M. 2010; 2010: 2068-2073

  Abstract

  Insertion velocity, tip asymmetry, and shaft diameter may influence steerable needle insertion paths in soft tissue. In this paper we examine the effects of these variables on needle paths in ex vivo goat liver, and demonstrate practical applications of robotic needle steering for ablation, biopsy, and brachytherapy. All experiments were performed using a new portable needle steering robot that steers asymmetric-tip needles under fluoroscopic imaging. For bevel-tip needles, we found that larger diameter needles resulted in less curvature, i.e. less steerability, confirming previous experiments in artificial tissue. The needles steered with radii of curvature ranging from 3:4 cm (for the most steerable pre-bent needle) to 2:97m (for the least steerable bevel needle). Pre-bend angle significantly affected needle curvature, but bevel angle did not. We hypothesize that biological tissue characteristics such as inhomogeneity and viscoelasticity significantly increase path variability. These results underscore the need for closed-loop image guidance for needle steering in biological tissues with complex internal structure.

  View details for DOI 10.1109/ROBOT.2010.5509873

  View details for PubMedID 21339851

  View details for PubMedCentralID PMC3040792

• Evaluation of Robotic Needle Steering in ex vivo Tissue IEEE International Conference on Robotics and Automation (ICRA) Majewicz, A., Wedlick, T. R., Reed, K. B., Okamura, A. M. IEEE. 2010: 2068–2073

  View details for Web of Science ID 000284150004147

• Estimation of Model Parameters for Steerable Needles. IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation Park, W., Reed, K. B., Okamura, A. M., Chirikjian, G. S. 2010: 3703-3708

  Abstract

  Flexible needles with bevel tips are being developed as useful tools for minimally invasive surgery and percutaneous therapy. When such a needle is inserted into soft tissue, it bends due to the asymmetric geometry of the bevel tip. This insertion with bending is not completely repeatable. We characterize the deviations in needle tip pose (position and orientation) by performing repeated needle insertions into artificial tissue. The base of the needle is pushed at a constant speed without rotating, and the covariance of the distribution of the needle tip pose is computed from experimental data. We develop the closed-form equations to describe how the covariance varies with different model parameters. We estimate the model parameters by matching the closed-form covariance and the experimentally obtained covariance. In this work, we use a needle model modified from a previously developed model with two noise parameters. The modified needle model uses three noise parameters to better capture the stochastic behavior of the needle insertion. The modified needle model provides an improvement of the covariance error from 26.1% to 6.55%.

  View details for DOI 10.1109/ROBOT.2010.5509380

  View details for PubMedID 21643451

  View details for PubMedCentralID PMC3107577

• Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine IEEE International Conference on Robotics and Automation (ICRA) King, H. H., Hannaford, B., Kwok, K., Yang, G., Griffiths, P., Okamura, A., Farkhatdinov, I., Ryu, J., Sankaranarayanan, G., Arikatla, V., Tadano, K., Kawashima, K., Peer, A., Schauss, T., Buss, M., Miller, L., Glozman, D., Rosen, J., Low, T. IEEE. 2010: 1733–1738

  View details for Web of Science ID 000284150002005

• Estimation of Model Parameters for Steerable Needles IEEE International Conference on Robotics and Automation (ICRA) Park, W., Reed, K. B., Okamura, A. M., Chirikjian, G. S. IEEE. 2010: 3703–3708

  View details for Web of Science ID 000284150001111

• Human vs. Robotic Tactile Sensing: Detecting Lumps in Soft Tissue. Gwilliam, J., Pezzementi, Z., Jantho, E., Okamura, A., M., Hsiao, S., S. 2010
• Does a Basic Deficit in Force Control Underlie Cerebellar Ataxia? Charles, S., K., Okamura, A., M., Bastian, A., J. 2010
• Design of a Haptic Simulator for Osteosynthesis Screw Insertion. Majewicz, A., Glasser, J., Bauer, R., Belkoff, S., Mears, S., Okamura, A., M. 2010
• A Proposed Method for Correcting Coordination Deficits: Models and Simulation. Grow, D., I., Bhanpuri, N., H., Charles, S., K., Bastian, A., J., Okamura, A., M. 2010
• Defining Performance Tradeoffs for Multi-Degree-of-Freedom Bilateral Teleoperators with LQG Control 49th IEEE Conference on Decision and Control (CDC) Griffiths, P. G., Okamura, A. M. IEEE. 2010: 3542–3547

  View details for Web of Science ID 000295049104016

• Modelling of non-linear elastic tissues for surgical simulation COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING Misra, S., Ramesh, K. T., Okamura, A. M. 2010; 13 (6): 811-818

  Abstract

  Realistic modelling of the interaction between surgical instruments and human organs has been recognised as a key requirement in the development of high-fidelity surgical simulators. Primarily due to computational considerations, most of the past real-time surgical simulation research has assumed linear elastic behaviour for modelling tissues, even though human soft tissues generally possess non-linear properties. For a non-linear model, the well-known Poynting effect developed during shearing of the tissue results in normal forces not seen in a linear elastic model. Using constitutive equations of non-linear tissue models together with experiments, we show that the Poynting effect results in differences in force magnitude larger than the absolute human perception threshold for force discrimination in some tissues (e.g. myocardial tissues) but not in others (e.g. brain tissue simulants).

  View details for DOI 10.1080/10255840903505121

  View details for Web of Science ID 000285145800019

  View details for PubMedID 20503126

• The Cerebellum Contributes to Constant and Dynamic Load Perception but not Position Perception. Bhanpuri, N., H., Okamura, A., M., Bastian, A., J. 2010
• Defining Performance Tradeoffs for Multi-Degree-of-Freedom Bilateral Teleoperators with LQG Control. Griffiths, P., G., Okamura, A., M. 2010
• Medical and Healthcare Robotics: Achievements and Opportunities IEEE Robotics and Automation Magazine Okamura, A., M., Mataric, M., J., Christensen, H., I. 2010; 3 (17): 26-37
• Neural coding of lump detection in soft tissue. Gwilliam, J., C., Yoshioka, T., Okamura, A., M., Hsiao, S., S. 2010
• Plugfest 2009: Global Interoperability in Telerobotics and Telemedicine. King, H., H., Hannaford, B., Kwok, K., W., Yang, G., Z., Griffiths, P., G., Okamura, A., M. 2010
• Modeling of Nonlinear Elastic Tissues for Surgical Simulation. Computer Methods in Biomechanics and Biomedical Engineering Misra, S., Ramesh, K., T., Okamura, A., M. 2010; 6 (13): 811-818
• Haptics for Robot-Assisted Minimally Invasive Surgery 13th International Symposium on Robotics Research (ISSR) Okamura, A. M., Verner, L. N., Reiley, C. E., Mahvash, M. SPRINGER-VERLAG BERLIN. 2010: 361–372

  View details for Web of Science ID 000289498800030

• Haptic Feedback: Technology and Medical Applications. Handbook of Touch. Okamura, A., M. 2010
• Modeling and Control of Needles With Torsional Friction IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Reed, K. B., Okamura, A. M., Cowan, N. J. 2009; 56 (12): 2905-2916

  Abstract

  A flexible needle can be accurately steered by robotically controlling the bevel tip orientation as the needle is inserted into tissue. Friction between the long, flexible needle shaft and the tissue can cause a significant discrepancy between the orientation of the needle tip and the orientation of the base where the needle angle is controlled. Our experiments show that several common phantom tissues used in needle steering experiments impart substantial friction forces to the needle shaft, resulting in a lag of more than 45 ( degrees ) for a 10 cm insertion depth in some phantoms; clinical studies report torques large enough to cause similar errors during needle insertions. Such angle discrepancies will result in poor performance or failure of path planners and image-guided controllers, since the needles used in percutaneous procedures are too small for state-of-the-art imaging to accurately measure the tip angle. To compensate for the angle discrepancy, we develop an estimator using a mechanics-based model of the rotational dynamics of a needle being inserted into tissue. Compared to controllers that assume a rigid needle in a frictionless environment, our estimator-based controller improves the tip angle convergence time by nearly 50% and reduces the path deviation of the needle by 70%.

  View details for DOI 10.1109/TBME.2009.2029240

  View details for Web of Science ID 000272042400016

  View details for PubMedID 19695979

  View details for PubMedCentralID PMC2859043

• Feedback Control for Steering Needles Through 3D Deformable Tissue Using Helical Paths. Robotics science and systems : online proceedings Hauser, K., Alterovitz, R., Chentanez, N., Okamura, A., Goldberg, K. 2009; V: 37-?

  Abstract

  Bevel-tip steerable needles are a promising new technology for improving accuracy and accessibility in minimally invasive medical procedures. As yet, 3D needle steering has not been demonstrated in the presence of tissue deformation and uncertainty, despite the application of progressively more sophisticated planning algorithms. This paper presents a feedback controller that steers a needle along 3D helical paths, and varies the helix radius to correct for perturbations. It achieves high accuracy for targets sufficiently far from the needle insertion point; this is counterintuitive because the system is highly under-actuated and not locally controllable. The controller uses a model predictive control framework that chooses a needle twist rate such that the predicted helical trajectory minimizes the distance to the target. Fast branch and bound techniques enable execution at kilohertz rates on a 2GHz PC. We evaluate the controller under a variety of simulated perturbations, including imaging noise, needle deflections, and curvature estimation errors. We also test the controller in a 3D finite element simulator that incorporates deformation in the tissue as well as the needle. In deformable tissue examples, the controller reduced targeting error by up to 88% compared to open-loop execution.

  View details for PubMedID 21179401

• Controlling a Robotically Steered Needle in the Presence of Torsional Friction. IEEE International Conference on Robotics and Automation : ICRA : [proceedings]. IEEE International Conference on Robotics and Automation Reed, K. B., Okamura, A. M., Cowan, N. J. 2009: 3476-3481

  Abstract

  A flexible needle can be accurately steered by robotically controlling the orientation of the bevel tip as the needle is inserted into tissue. Here, we demonstrate the significant effect of friction between the long, flexible needle shaft and the tissue, which can cause a significant discrepancy between the orientation of the needle tip and the orientation of the base where the needle is controlled. Our experiments show that several common phantom tissues used in needle steering experiments impart substantial frictional forces to the needle shaft, resulting in a lag of over 45° for a 10 cm insertion depth in some phantoms; clinical studies have reported torques large enough to could cause similar errors during needle insertions. Such angle discrepancies will result in poor performance or failure of path planners and image-guided controllers, since the needles used in percutaneous procedures are too small for state-of-the-art imaging to accurately measure the tip angle. To compensate for the angle discrepancy, we develop a model for the rotational dynamics of a needle being continuously inserted into tissue and show how a PD controller is sufficient to compensate for the rotational dynamics.

  View details for PubMedID 21461175

• The importance of organ geometry and boundary constraints for planning of medical interventions MEDICAL ENGINEERING & PHYSICS Misra, S., Macura, K. J., Ramesh, K. T., Okamura, A. M. 2009; 31 (2): 195-206

  Abstract

  Realistic modeling of medical interventions involving tool-tissue interactions has been considered to be a key requirement in the development of high-fidelity simulators and planners. Organ geometry, soft-tissue constitutive laws, and boundary conditions imposed by the connective tissues surrounding the organ are some of the factors that govern the accuracy of medical intervention planning. In this study it is demonstrated that, for needle path planning, the organ geometry and boundary constraints surrounding the organ are the most important factors influencing the deformation. As an example, the procedure of needle insertion into the prostate (e.g. for biopsy or brachytherapy) is considered. Image segmentation is used to extract the anatomical details from magnetic resonance images, while object-oriented finite element analysis (OOF) software is used to generate finite element (FE) meshes from the segmented images. Two-dimensional FE simulations that account for complex anatomical details along with relative motion between the prostate and its surrounding structure using cohesive zone models are compared with traditional simulation models having simple organ geometry and boundary constraints. Nodal displacements for these simpler models were observed to be up to 14 times larger than those obtained from the anatomically accurate models.

  View details for DOI 10.1016/j.medengphy.2008.08.002

  View details for Web of Science ID 000263872200005

  View details for PubMedID 18815068

• Stiffness Discrimination with Visual and Proprioceptive Cues 3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Gurari, N., Kuchenbecker, K. J., Okamura, A. M. IEEE. 2009: 121–126

  View details for Web of Science ID 000266714300022

• Observations and Models for Needle-Tissue Interactions IEEE International Conference on Robotics and Automation Misra, S., Reed, K. B., Schafer, B. W., Ramesh, K. T., Okamura, A. M. IEEE. 2009: 3423–3428

  View details for Web of Science ID 000276080401237

• Quantifying Perception of Nonlinear Elastic Tissue Models using Multidimensional Scaling 3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Misra, S., Fuernstahl, P., Ramesh, K. T., Okamura, A. M., Harders, M. IEEE. 2009: 570–575

  View details for Web of Science ID 000266714300093

• Force & Torque Feedback vs Force Only Feedback 3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Verner, L. N., Okamura, A. M. IEEE. 2009: 406–410

  View details for Web of Science ID 000266714300066

• Haptic feedback in robot-assisted minimally invasive surgery CURRENT OPINION IN UROLOGY Okamura, A. M. 2009; 19 (1): 102-107

  Abstract

  Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon and minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal. It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor. This review explains the technical challenges of creating haptic feedback for robot-assisted surgery and provides recent results that evaluate the effectiveness of haptic feedback in mock surgical tasks.Haptic feedback systems for RMIS are still under development and evaluation. Most provide only force feedback, with limited fidelity. The major challenge at this time is sensing forces applied to the patient. A few tactile feedback systems for RMIS have been created, but their practicality for clinical implementation needs to be shown. It is particularly difficult to sense and display spatially distributed tactile information. The cost-benefit ratio for haptic feedback in RMIS has not been established.The designs of existing commercial RMIS systems are not conducive for force feedback, and creative solutions are needed to create compelling tactile feedback systems. Surgeons, engineers, and neuroscientists should work together to develop effective solutions for haptic feedback in RMIS.

  View details for DOI 10.1097/MOU.0b013e32831a478c

  View details for Web of Science ID 000262215600020

  View details for PubMedID 19057225

  View details for PubMedCentralID PMC2701448

• Feedback Control for Steering Needles through 3D Deformable Tissue using Helical Paths. Robotics: Science and Systems. Hauser, K., Alterovitz, R., Chentanez, N., Okamura, A., Goldberg, K. 2009
• Observations of Needle-Tissue Interactions Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Misra, S., Reed, K. B., Ramesh, K. T., Okamura, A. M. IEEE. 2009: 262–265

  Abstract

  Needles with asymmetric bevel tips naturally bend when they are inserted into soft tissue. In this study, we present an analytical model for the loads developed at the bevel tip during needle-tissue interaction. The model calculates the loads based on the geometry of the bevel edge and gel material properties. The modeled transverse force developed at the tip is compared to forces measured experimentally. The analytical model explains the trends observed in the experiments. In addition to macroscopic studies, we also present microscopic observations of needle-tissue interactions. These results contribute to a mechanics-based model of robotic needle steering, extending previous work on kinematic models.

  View details for Web of Science ID 000280543600070

  View details for PubMedID 19963709

• Design Considerations and Human-Machine Performance of Moving Virtual Fixtures IEEE International Conference on Robotics and Automation Gibo, T. L., Verner, L. N., Yuh, D. D., Okamura, A. M. IEEE. 2009: 2958–2963

  View details for Web of Science ID 000276080401164

• Haptics as an Aid to Copying for People with Williams Syndrome 3rd Joint EuroHaptics Conference Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Lee, J., Okamura, A. M., Landau, B. IEEE. 2009: 356–361

  View details for Web of Science ID 000266714300061

• Stiffness Discrimination with Visual and Proprioceptive Cues. Gurari, N., Kuchenbecker, K., J., Okamura, A., M. 2009
• Can Cerebellar Dysmetria be Explained by an Incorrect Internal Model of Limb Dynamics? Bhanpuri, N., H., Grow, D., I., Okamura, A., M., Bastian, A., J. 2009
• Modeling and Control of Needles with Torsional Friction. IEEE Transactions on Biomedical Engineering Reed, K., B., Okamura, A., M., Cowan, N., J. 2009; 12 (56): 2905-2916
• Controlling a Robotically Steered Needle in the Presence of Torsional Friction IEEE International Conference on Robotics and Automation Reed, K. B., Okamura, A. M., Cowan, N. J. IEEE. 2009: 3598–3603

  View details for Web of Science ID 000276080401265

• Environment Discrimination with Vibration Feedback to the Foot, Arm, and Fingertip 11th IEEE International Conference on Rehabilitation Robotics Gurari, N., Smith, K., Madhav, M., Okamura, A. M. IEEE. 2009: 398–403

  View details for Web of Science ID 000277086500056

• Tissue Property Estimation and Graphical Display for Teleoperated Robot-Assisted Surgery IEEE International Conference on Robotics and Automation Yamamoto, T., Vagvolgyi, B., Balaji, K., Whitcomb, L. L., Okamura, A. M. IEEE. 2009: 3117–3123

  View details for Web of Science ID 000276080401190

• Effects of Haptic and Graphical Force Feedback on Teleoperated Palpation IEEE International Conference on Robotics and Automation Gwilliam, J. C., Mahvash, M., Vagvolgyi, B., Vacharat, A., Yuh, D. D., Okamura, A. M. IEEE. 2009: 3315–3320

  View details for Web of Science ID 000276080401221

• Quantification and Reproduction of Human Hand Skin Stretch and its Effects on Proprioception. Greenwald, E., Pompe, J., Hsiao, S., Okamura, A. 2009
• Characterization of Pre-Curved Needles for Steering in Tissue Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Wedlick, T. R., Okamura, A. M. IEEE. 2009: 1200–1203

  Abstract

  Needles with tip asymmetry deflect upon insertion into soft tissue, an effect that can be used to steer needles within the body. This paper presents a phenomenological characterization of the steering behavior of pre-curved needles, which have tip asymmetry due to curvature of the needle near the tip. We describe needle construction methods and a needle shaft triangulation algorithm to compute the shape of the needle based on images. Experimental results show that pre-curved needles possess greater dexterity than bevel-tipped needles and achieve radii of curvature similar to pre-bent needles. For long pre-curve arc lengths, the radius of curvature of the needle was found to approach the radius of curvature of the pre-curve. Pre-curved needles were found to display behaviors not seen with bevel-tipped needles, such as the insertion velocity influencing the path of the tip within the tissue and the ability to plastically deform the needle during steering.

  View details for Web of Science ID 000280543601032

  View details for PubMedID 19963994

• Surgical and Interventional Robotics: Part III Surgical Assistance Systems IEEE ROBOTICS & AUTOMATION MAGAZINE Hager, G. D., Okamura, A. M., Kazanzides, P., Whitcomb, L. L., Fichtinger, G., Taylor, R. H. 2008; 15 (4): 84-93

  View details for DOI 10.1109/MRA.2008.930401

  View details for Web of Science ID 000261616400012

• Integrated Planning and Image-Guided Control for Planar Needle Steering. Proceedings of the ... IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics Reed, K. B., Kallem, V., Alterovitz, R., Goldberg, K., Okamura, A. M., Cowan, N. J. 2008; 2008: 819-824

  Abstract

  Flexible, tip-steerable needles promise to enhance physicians' abilities to accurately reach targets and maneuver inside the human body while minimizing patient trauma. Here, we present a functional needle steering system that integrates two components: (1) a patient-specific 2D pre- and intra-operative planner that finds an achievable route to a target within a planar slice of tissue (Stochastic Motion Roadmap), and (2) a low-level image-guided feedback controller that keeps the needle tip within that slice. The planner generates a sequence of circular arcs that can be realized by interleaving pure insertions with 180° rotations of the needle shaft. This preplanned sequence is updated in realtime at regular intervals. Concurrently, the low-level image-based controller servos the needle to remain close to the desired plane between plan updates. Both planner and controller are predicated on a previously developed kinematic nonholonomic model of bevel-tip needle steering. We use slighly different needles here that have a small bend near the tip, so we extend the model to account for discontinuities of the tip position caused by 180° rotations. Further, during large rotations of the needle base, we maintain the desired tip angle by compensating for torsional compliance in the needle shaft, neglected in previous needle steering work. By integrating planning, control, and torsion compensation, we demonstrate both accurate targeting and obstacle avoidance.

  View details for PubMedID 20640197

• Modeling of tool-tissue interactions for computer-based surgical simulation: A literature review PRESENCE-TELEOPERATORS AND VIRTUAL ENVIRONMENTS Misra, S., Ramesh, K. T., Okamura, A. M. 2008; 17 (5): 463-491

  View details for Web of Science ID 000259865300003

• Modeling of Tool-Tissue Interactions for Computer-Based Surgical Simulation: A Literature Review. Presence (Cambridge, Mass.) Misra, S., Ramesh, K. T., Okamura, A. M. 2008; 17 (5): 463

  Abstract

  Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in robot-assisted surgery for pre- and intra-operative planning. Accurate modeling of the interaction between surgical instruments and organs has been recognized as a key requirement in the development of high-fidelity surgical simulators. Researchers have attempted to model tool-tissue interactions in a wide variety of ways, which can be broadly classified as (1) linear elasticity-based, (2) nonlinear (hyperelastic) elasticity-based finite element (FE) methods, and (3) other techniques that not based on FE methods or continuum mechanics. Realistic modeling of organ deformation requires populating the model with real tissue data (which are difficult to acquire in vivo) and simulating organ response in real time (which is computationally expensive). Further, it is challenging to account for connective tissue supporting the organ, friction, and topological changes resulting from tool-tissue interactions during invasive surgical procedures. Overcoming such obstacles will not only help us to model tool-tissue interactions in real time, but also enable realistic force feedback to the user during surgical simulation. This review paper classifies the existing research on tool-tissue interactions for surgical simulators specifically based on the modeling techniques employed and the kind of surgical operation being simulated, in order to inform and motivate future research on improved tool-tissue interaction models.

  View details for PubMedID 20119508

  View details for PubMedCentralID PMC2813063

• Surgical and interventional robotics: Part II - Surgical CAD-CAM systems IEEE ROBOTICS & AUTOMATION MAGAZINE Fichtinger, G., Kazanzides, P., Okamura, A. M., Hager, G. D., Whitcomb, L. L., Taylor, R. H. 2008; 15 (3): 94-102

  View details for DOI 10.1109/MRA.2008.927971

  View details for Web of Science ID 000259300600014

• Surgical and interventional robotics - Core concepts, technology, and design IEEE ROBOTICS & AUTOMATION MAGAZINE Kazanzides, P., Fichtinger, G., Hager, G. D., Okamura, A. M., Whitcomb, L. L., Taylor, R. H. 2008; 15 (2): 122-130

  View details for DOI 10.1109/MRA.2008.926390

  View details for Web of Science ID 000256801100017

• Modeling the forces of cutting with scissors IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Mahvash, M., Voo, L. M., Kim, D., Jeung, K., Wainer, J., Okamura, A. M. 2008; 55 (3): 848-856

  Abstract

  Modeling forces applied to scissors during cutting of biological materials is useful for surgical simulation. Previous approaches to haptic display of scissor cutting are based on recording and replaying measured data. This paper presents an analytical model based on the concepts of contact mechanics and fracture mechanics to calculate forces applied to scissors during cutting of a slab of material. The model considers the process of cutting as a sequence of deformation and fracture phases. During deformation phases, forces applied to the scissors are calculated from a torque-angle response model synthesized from measurement data multiplied by a ratio that depends on the position of the cutting crack edge and the curve of the blades. Using the principle of conservation of energy, the forces of fracture are related to the fracture toughness of the material and the geometry of the blades of the scissors. The forces applied to scissors generally include high-frequency fluctuations. We show that the analytical model accurately predicts the average applied force. The cutting model is computationally efficient, so it can be used for real-time computations such as haptic rendering. Experimental results from cutting samples of paper, plastic, cloth, and chicken skin confirm the model, and the model is rendered in a haptic virtual environment.

  View details for DOI 10.1109/TBME.2007.908069

  View details for Web of Science ID 000253733800001

  View details for PubMedID 18334376

  View details for PubMedCentralID PMC2709828

• Techniques for Environment Parameter Estimation During Telemanipulation 2nd Biennial IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2008) Yamamoto, T., Bernhardt, M., Peer, A., Buss, M., Okamura, A. M. IEEE. 2008: 217–223

  View details for Web of Science ID 000270540400037

• The touch thimble: Providing fingertip contact feedback during point-force haptic interaction 16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Kuchenbecker, K. J., Ferguson, D., Kutzer, M., Moses, M., Okamura, A. M. IEEE COMPUTER SOC. 2008: 239–246

  View details for Web of Science ID 000254949300034

• A Simulator to Explore the Role of Haptic Feedback in Cataract Surgery Training 16th Conference on Medicine Meets Virtual Reality Doyle, L., Gauthier, N., Ramanathan, S., Okamura, A. I O S PRESS. 2008: 106–111

  Abstract

  Phacoemulsification cataract surgery, a minimally invasive technique to remove a cloudy lens from the eye, is one of the most commonly performed surgical procedures in the western world. Conventional training for this procedure involves didactic lectures and practice on pig and human cadaver eyes, none of which allow trainees to form an accurate predictive model of human tissue behavior during surgery. A virtual environment simulator for capsulorrhexis, one of the first steps in cataract surgery, has been developed that allows a trainee to use surgical instruments to excise a circle of tissue on the anterior side of the lens capsule through tearing. The simulator invokes a deformable mass-spring-damper mesh model of the tissue that can be grasped and torn via shearing. A novel algorithm for mesh division and maintenance enables realistic tearing behavior. The trainee controls tool motion using a 3-degree-of-freedom haptic device, and haptic feedback is provided from the virtual tissue. Although the haptic feedback in a real capsulorrhexis procedure is below the human threshold of haptic sensing, this simulator enables an experiment to determine the effectiveness of "haptic training wheels" -- the idea of haptic training for a task without haptic feedback.

  View details for Web of Science ID 000272668400023

  View details for PubMedID 18391267

• Measurement-Based Modeling for Haptic Display. In Haptic Rendering: Foundations, Algorithms, and Applications. Okamura, A., M., Kuchenbecker, K., J., Mahvash, M. edited by Lin, M., C., Otaduy, M. AK Peters.. 2008: 1
• Control Methods for Guidance Virtual Fixtures in Compliant Human-Machine Interfaces IEEE/RSJ International Conference on Intelligent Robots and Systems Marayong, P., Hager, G. D., Okamura, A. M. IEEE. 2008: 1166–1172

  View details for Web of Science ID 000259998200183

• Chapter 30: Haptics. In Springer Handbook of Robotics Hannaford, B., Okamura, A., M. Springer. 2008: 719–739
• Force-feedback surgical teleoperator: Controller design and palpation experiments 16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Mahvash, M., Gwilliam, J., Agarwal, R., Vagvolgyi, B., Su, L., Yuh, D. D., Okamura, A. M. IEEE COMPUTER SOC. 2008: 465–471

  View details for Web of Science ID 000254949300070

• A Simulator to Explore the Role of Haptic Feedback in Cataract Surgery Training. Medicine Meets Virtual Reality. Doyle, L., Gauthier, N., Ramanathan, S., Okamura, A., M. 2008
• Needle-Tissue Interaction Forces for Bevel-Tip Steerable Needles 2nd Biennial IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2008) Misra, S., Reed, K. B., Douglas, A. S., Ramesh, K. T., Okamura, A. M. IEEE. 2008: 224–231

  View details for Web of Science ID 000270540400038

• Effects of visual force feedback on robot-assisted surgical task performance JOURNAL OF THORACIC AND CARDIOVASCULAR SURGERY Reiley, C. E., Akinbiyi, T., Burschka, D., Chang, D. C., Okamura, A. M., Yuh, D. D. 2008; 135 (1): 196-202

  Abstract

  Direct haptic (force or tactile) feedback is negligible in current surgical robotic systems. The relevance of haptic feedback in robot-assisted performances of surgical tasks is controversial. We studied the effects of visual force feedback, a haptic feedback surrogate, on tying surgical knots with fine sutures similar to those used in cardiovascular surgery.By using a modified da Vinci robotic system (Intuitive Surgical, Inc, Sunnyvale, Calif) equipped with force-sensing instrument tips and real-time visual force feedback overlays in the console image, 10 surgeons each tied 10 knots with and 10 knots without visual force feedback. Four surgeons had significant prior da Vinci experience, and the remaining 6 surgeons did not. Performance parameters, including suture breakage and secure knots, peak and standard deviation of applied forces, and completion times using 5-0 silk sutures, were recorded. Chi-square and Student t test analyses determined the differences between groups.Among surgeon subjects with robotic experience, no differences in measured performance parameters were found between robot-assisted knot ties executed with and without visual force feedback. Among surgeons without robotic experience, however, visual force feedback was associated with lower suture breakage rates, peak applied forces, and standard deviations of applied forces. Visual force feedback did not impart differences in knot completion times or loose knots for either surgeon group.Visual force feedback resulted in reduced suture breakage, lower forces, and decreased force inconsistencies among novice robotic surgeons, although elapsed time and knot quality were unaffected. In contrast, visual force feedback did not affect these metrics among surgeons experienced with the da Vinci system. These results suggest that visual force feedback primarily benefits novice robot-assisted surgeons, with diminishing benefits among experienced surgeons.

  View details for DOI 10.1016/j.jtcvs.2007.08.043

  View details for Web of Science ID 000252139600029

  View details for PubMedID 18179942

  View details for PubMedCentralID PMC2674617

• Haptic simulation of elbow joint spasticity 16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Grow, D. I., Wu, M. (., Locastro, M. J., Arora, S. K., Bastian, A. J., Okamura, A. M. IEEE COMPUTER SOC. 2008: 475–476

  View details for Web of Science ID 000254949300072

• Effects of proprioceptive motion feedback on sighted and non-sighted control of a virtual hand prosthesis 16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Blank, A., Okamura, A. M., Kuchenbecker, K. J. IEEE COMPUTER SOC. 2008: 141–142

  View details for Web of Science ID 000254949300021

• Modeling realistic tool-tissue interactions with haptic feedback: A learning-based method 16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Pezzernenti, Z., Ursu, D., Misra, S., Okamura, A. M. IEEE COMPUTER SOC. 2008: 209–215

  View details for Web of Science ID 000254949300030

• Compensating for Torsion Windup in Steerable Needles. Okamura, A., M., Reed, K., B. 2008
• The Touch Thimble: Providing Fingertip Contact Feedback During Point-Force Haptic Interaction. Kuchenbecker, K., J., Ferguson, D., Kutzer, M., Moses, M., Okamura, A., M. 2008
• Telemanipulators with Sensor/Actuator Asymmetries Fail the Robustness Criterion. Verner, L., N., Okamura, A., M. 2008
• Multi-Estimator Technique for Environment Parameter Estimation During Telemanipulation. Yamamoto, T., Bernhardt, M., Peer, A., Buss, M., Okamura, A., M. 2008
• Physically Valid Surgical Simulators: Linear Versus Nonlinear Tissue Models 16th Conference on Medicine Meets Virtual Reality Misra, S., Ramesh, K. T., Okamura, A. M. I O S PRESS. 2008: 293–295

  Abstract

  Realistic modeling of the interaction between surgical instruments and organs has been recognized as a key requirement in the development of high-fidelity surgical simulators. For a nonlinear model, the well-known Poynting effect developed during shearing of the tissue results in normal forces not seen in a linear elastic model. It is demonstrated that the difference in force magnitude for myocardial tissue is larger than the just noticeable difference for contact force discrimination thresholds published in the psychophysics literature. This work also proposes the validation of simulators by careful examination of relevant simulator design parameters that relate to final simulator behaviors affecting clinical outcomes.

  View details for Web of Science ID 000272668400063

  View details for PubMedID 18391307

• Control Methods for Guidance Virtual Fixtures in Compliant Human-Machine Interfaces. Marayong, P., Hager, G., D., Okamura, A., M. 2008
• Integrated Planning and Image-Guided Control for Planar Needle Steering 2nd Biennial IEEE RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2008) Reed, K. B., Kallem, V., Alterovitz, R., Goldberg, K., Okamura, A. M., Cowan, N. J. IEEE. 2008: 907–912

  View details for Web of Science ID 000270540400151

• Friction compensation for enhancing transparency of a teleoperator with compliant transmission IEEE TRANSACTIONS ON ROBOTICS Mahvash, M., Okamura, A. 2007; 23 (6): 1240-1246

  View details for DOI 10.1109/TRO.2007.909825

  View details for Web of Science ID 000251944100013

• Friction Compensation for Enhancing Transparency of a Teleoperator with Compliant Transmission. IEEE transactions on robotics : a publication of the IEEE Robotics and Automation Society Mahvash, M., Okamura, A. 2007; 23 (6): 1240-1246

  Abstract

  This article presents a model-based compensator for canceling friction in the tendon-driven joints of a haptic-feedback teleoperator. Unlike position-tracking systems, a teleoperator involves an unknown environment force that prevents the use of tracking position error as a feedback to the compensator. Thus, we use a model-based feedforward friction compensator to cancel the friction forces. We provide conditions for selecting compensator parameters to ensure passivity of the teleoperator and demonstrate performance experimentally.

  View details for DOI 10.1109/TRO.2007.909825

  View details for PubMedID 20514151

  View details for PubMedCentralID PMC2877600

• Pseudo-admittance bilateral telemanipulation with guidance virtual fixtures INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Abbott, J. J., Okamura, A. M. 2007; 26 (8): 865-884

  View details for DOI 10.1177/0278364907080425

  View details for Web of Science ID 000248543500006

• Quantifying the value of visual and haptic position feedback during force-based motion control 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Kuchenbecker, K. J., Gurari, N., Okamura, A. M. IEEE COMPUTER SOC. 2007: 561–562

  View details for Web of Science ID 000246105200096

• Teleoperation of steerable needles IEEE International Conference on Robotics and Automation Romano, J. M., Webster, R. J., Okamura, A. M. IEEE. 2007: 934–939

  View details for Web of Science ID 000250915300149

• Force feedback is noticeably different for linear versus nonlinear elastic tissue models 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Misra, S., Okamura, A. M., Ramesh, K. T. IEEE COMPUTER SOC. 2007: 519–524

  View details for Web of Science ID 000246105200086

• Virtual fixture control for compliant human-machine interfaces IEEE International Conference on Robotics and Automation Marayong, P., Na, H. S., Okamura, A. M. IEEE. 2007: 4018–4024

  View details for Web of Science ID 000250915304004

• Effects of translational and gripping force feedback are decoupled in a 4-degree-of-freedom telemanipulator 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Verner, L. N., Okamura, A. M. IEEE COMPUTER SOC. 2007: 286–291

  View details for Web of Science ID 000246105200048

• Enhancing transparency of a posit ion-exchange teleoperator 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Mahvash, M., Okamura, A. M. IEEE COMPUTER SOC. 2007: 470–475

  View details for Web of Science ID 000246105200078

• Dynamic guidance with pseudoadmittance virtual fixtures IEEE International Conference on Robotics and Automation Pezzementi, Z., Okamura, A. M., Hager, G. D. IEEE. 2007: 1761–1767

  View details for Web of Science ID 000250915301122

• Evaluation of human performance with kinematic and haptic errors 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Yamamoto, T., Okamura, A. M. IEEE COMPUTER SOC. 2007: 78–83

  View details for Web of Science ID 000246105200014

• Haptic virtual fixtures for robot-assisted manipulation 12th International Symposium on Robotics Research (ISRR) Abbott, J. J., Marayong, P., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2007: 49–64

  View details for Web of Science ID 000245915500004

• Design of a Pill-Sized 12-legged Endoscopic Capsule Robot. Okamura, A., M., Quirini, M., Webster III, R., J., Menciassi, A., Dario, P. 2007
• Force Sensing in Robot-Assisted Surgery: Which Degrees of Freedom are Most Important? Medicine Meets Virtual Reality (MMVR) 15. Saha, S., Mahvash, M., Verner, L., N., Okamura, A., M. 2007
• Educational Haptics. Grow, D., I., Verner, L., N., Okamura, A., M. 2007
• Electrolytic Silicone Bourdon Tube Microactuator for Reconfigurable Surgical Robots. Okamura, A., M., Pak, N., Ng, Webster III, R., J., Menciassi, A., Dario, P. 2007
• Friction Compensation for Enhancing Transparency of a Teleoperator with Compliant Transmission. IEEE Transactions on Robotics Mahvash, M., Okamura, A., M. 2007; 6 (23): 1240-1246
• Effects of visual and proprioceptive motion feedback on human control of targeted movement 10th IEEE International Conference on Rehabilitation Robotics Kuchenbecker, K. J., Gurari, N., Okamura, A. M. IEEE. 2007: 513–524

  View details for Web of Science ID 000255389600077

• Human performance in a knob-turning task 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Gurari, N., Okamura, A. M. IEEE COMPUTER SOC. 2007: 96–101

  View details for Web of Science ID 000246105200017

• Nonholonomic modeling of needle steering 9th International Symposium on Experimental Robotics (ISER) Webster, R. J., Kim, J. S., Cowan, N. J., Chirikjian, G. S., Okamura, A. M. SAGE PUBLICATIONS LTD. 2006: 509–25

  View details for DOI 10.1177/0278364906065388

  View details for Web of Science ID 000238146600006

• Stable forbidden-region virtual fixtures for bilateral telemanipulation JOURNAL OF DYNAMIC SYSTEMS MEASUREMENT AND CONTROL-TRANSACTIONS OF THE ASME Abbott, J. J., Okamura, A. M. 2006; 128 (1): 53-64

  View details for DOI 10.1115/1.2168163

  View details for Web of Science ID 000236439100007

• Nonholonomic modeling of needle steering 9th International Symposium on Experimental Robotics (ISER) Iii, R. J., Cowan, N. J., Chirikjian, G., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2006: 35–44

  View details for Web of Science ID 000236887500004

• Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference Akinbiyi, T., Reiley, C. E., Saha, S., Burschka, D., Hasser, C. J., Yuh, D. D., Okamura, A. M. 2006; 1: 567-570

  Abstract

  Teleoperated robot-assisted surgical systems provide surgeons with improved precision, dexterity, and visualization over traditional minimally invasive surgery. The addition of haptic (force and/or tactile) feedback has been proposed as a way to further enhance the performance of these systems. However, due to limitations in sensing and control technologies, implementing direct haptic feedback to the surgeon's hands remains impractical for clinical application. A new, intuitive augmented reality system for presentation of force information through sensory substitution has been developed and evaluated. The augmented reality system consists of force-sensing robotic instruments, a kinematic tool tracker, and a graphic display that overlays a visual representation of force levels on top of the moving instrument tips. The system is integrated with the da Vinci Surgical System (Intuitive Surgical, Inc.) and tested by several users in a phantom knot tying task. The augmented reality system decreases the number of broken sutures, decreases the number of loose knots, and results in more consistent application of forces.

  View details for PubMedID 17945986

• Effect of hand dynamics on virtual fixtures for compliant human-machine interfaces* 14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Marayong, P., Hager, G. D., Okamura, A. M. IEEE COMPUTER SOC. 2006: 109–115

  View details for Web of Science ID 000237225000019

• Portability and applicability of virtual fixtures across medical and manufacturing tasks IEEE International Conference on Robotics and Automation (ICRA) Lin, H. C., Mills, K., Kazanzides, P., Hager, G. D., Marayong, P., Okamura, A. M., Karam, R. IEEE. 2006: 225–230

  View details for Web of Science ID 000240886900037

• Object Capture With a Camera-Mobile Robot System: An Introductory Robotics Project. IEEE Robotics and Automation Magazine Okamura, A., M., Webster III, R., J. 2006; 1 (13): 85-88
• Pseudo-admittance bilateral telemanipulation with guidance virtual fixtures 14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Abbott, J. J., Okamura, A. M. IEEE COMPUTER SOC. 2006: 169–175

  View details for Web of Science ID 000237225000027

• Sensor/Actuator asymmetries in telemanipulators: Implications of partial force feedback 14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Verner, L. N., Okamura, A. M. IEEE COMPUTER SOC. 2006: 309–314

  View details for Web of Science ID 000237225000047

• Toward active cannulas: Miniature snake-like surgical robots IEEE/RSJ International Conference on Intelligent Robots and Systems Webster, R. J., Okamura, A. M., Cowan, N. J. IEEE. 2006: 2857–2863

  View details for Web of Science ID 000245452403001

• Dynamic augmented reality for sensory substitution in robot-assisted surgical systems 28th Annual International Conference of the IEEE-Engineering-in-Medicine-and-Biology-Society Akinbiyi, T., Reiley, C. E., Saha, S., Burschka, D., Hasser, C. J., Yuh, D. D., Okamura, A. M. IEEE. 2006: 3090–3093

  View details for Web of Science ID 000247284703142

• Friction compensation for a force-feedback telerobotic system IEEE International Conference on Robotics and Automation (ICRA) Mahvash, M., Okamura, A. M. IEEE. 2006: 3268–3273

  View details for Web of Science ID 000240886907017

• Friction compensation for a force-feedback teleoperator with compliant transmission 45th IEEE Conference on Decision and Control Mahvash, M., Okamura, A. M. IEEE. 2006: 4508–4513

  View details for Web of Science ID 000252251602110

• Effects of Haptic Feedback on Exploration. Wu, M., Okamura, A., M. 2006
• Environment parameter estimation during bilateral telemanipulation 14th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems Misra, S., Okamura, A. M. IEEE COMPUTER SOC. 2006: 301–307

  View details for Web of Science ID 000237225000046

• Effects of position quantization and sampling rate on virtual-wall passivity ASME International Mechanical Engineering Congress Abbott, J. J., Okamura, A. M. IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC. 2005: 952–64

  View details for DOI 10.1109/TRO.2005.851377

  View details for Web of Science ID 000232118800015

• Human-machine collaborative systems for microsurgical applications 11th International Symposium on Robotics Research Kragic, D., Marayong, P., Li, M., Okamura, A. M., Hager, G. A. SAGE PUBLICATIONS LTD. 2005: 731–41

  View details for DOI 10.1177/0278364905057059

  View details for Web of Science ID 000232274300005

• Transrectal fiducial carrier for radiographic image registration in prostate Brachytherapy 47th Annual Meeting of the American-Association-of-Physicists-in-Medicine Kemper, J., Burkholder, A., Jain, A., Mustufa, T., Wyrobek, K., Burdette, C., Song, D., Okamura, A., Fichtinger, G. AMER ASSOC PHYSICISTS MEDICINE AMER INST PHYSICS. 2005: 2108–

  View details for Web of Science ID 000229908601356

• Effect of sensory substitution on suture-manipulation forces for robotic surgical systems JOURNAL OF THORACIC AND CARDIOVASCULAR SURGERY Kitagawa, M., Dokko, D., Okamura, A. M., Yuh, D. D. 2005; 129 (1): 151-158

  Abstract

  Direct haptic (force or tactile) feedback is not yet available in commercial robotic surgical systems. Previous work by our group and others suggests that haptic feedback might significantly enhance the execution of surgical tasks requiring fine suture manipulation, specifically those encountered in cardiothoracic surgery. We studied the effects of substituting direct haptic feedback with visual and auditory cues to provide the operating surgeon with a representation of the forces he or she is applying with robotic telemanipulators.Using the robotic da Vinci surgical system (Intuitive Surgical, Inc, Sunnyvale, Calif), we compared applied forces during a standardized surgical knot-tying task under 4 different sensory-substitution scenarios: no feedback, auditory feedback, visual feedback, and combined auditory-visual feedback.The forces applied with these sensory-substitution modes more closely approximate suture tensions achieved under ideal haptic conditions (ie, hand ties) than forces applied without such sensory feedback. The consistency of applied forces during robot-assisted suture tying aided by visual feedback or combined auditory-visual feedback sensory substitution is superior to that achieved with hand ties. Robot-assisted ties aided with auditory feedback revealed levels of consistency that were generally equivalent or superior to those attained with hand ties. Visual feedback and auditory feedback improve the consistency of robotically applied forces.Sensory substitution, in the form of visual feedback, auditory feedback, or both, confers quantifiable advantages in applied force accuracy and consistency during the performance of a simple surgical task.

  View details for DOI 10.1016/j.jtcvs.2004.05.029

  View details for Web of Science ID 000226216600021

  View details for PubMedID 15632837

• A velocity-dependent model for needle insertion in soft tissue. Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention Crouch, J. R., Schneider, C. M., Wainer, J., Okamura, A. M. 2005; 8: 624-632

  Abstract

  Models that predict the soft tissue deformation caused by needle insertion could improve the accuracy of procedures such as brachytherapy and needle biopsy. Prior work on needle insertion modeling has focused on static deformation; the experiments presented here show that dynamic effects such as relaxation are important. An experimental setup is described for recording and measuring the deformation that occurs with needle insertion into a soft tissue phantom. Analysis of the collected data demonstrates the time- and velocity-dependent nature of the deformation. Deformation during insertion is shown to be well represented using a velocity-dependent force function with a linear elastic finite element model. The model's accuracy is limited to the period during needle motion, indicating that a viscoelastic tissue model may be required to capture tissue relaxation after the needle stops.

  View details for PubMedID 16686012

• Human-machine collaborative systems for microsurgical applications 11th International Symposium on Robotics Research Kragic, D., Marayong, P., Li, M., Okamura, A. M., Hager, G. D. SPRINGER-VERLAG BERLIN. 2005: 162–171

  View details for Web of Science ID 000228862600014

• Planning for steerable bevel-tip needle insertion through 2D soft tissue with obstacles IEEE International Conference on Robotics and Automation (ICRA) Alterovitz, R., Goldberg, K., Okamura, A. IEEE. 2005: 1640–1645

  View details for Web of Science ID 000235460101070

• Automatic detection and segmentation of robot-assisted surgical motions. Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention Lin, H. C., Shafran, I., Murphy, T. E., Okamura, A. M., Yuh, D. D., Hager, G. D. 2005; 8: 802-810

  Abstract

  Robotic surgical systems such as Intuitive Surgical's da Vinci system provide a rich source of motion and video data from surgical procedures. In principle, this data can be used to evaluate surgical skill, provide surgical training feedback, or document essential aspects of a procedure. If processed online, the data can be used to provide context-specific information or motion enhancements to the surgeon. However, in every case, the key step is to relate recorded motion data to a model of the procedure being performed. This paper examines our progress at developing techniques for "parsing" raw motion data from a surgical task into a labelled sequence of surgical gestures. Our current techniques have achieved >90% fully automated recognition rates on 15 datasets.

  View details for PubMedID 16685920

• A fracture mechanics approach to haptic synthesis of tissue cutting with scissors 1st Joint Eurohaptics Conference/Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Mahvash, M., Okamura, A. M. IEEE COMPUTER SOC. 2005: 356–362

  View details for Web of Science ID 000228426200050

• Steering flexible needles under Markov motion uncertainty IEEE/RSJ International Conference on Intelligent Robots and Systems Alterovitz, R., Lim, A., Goldberg, K., Chirikjian, G. S., Okamura, A. M. IEEE. 2005: 120–125

  View details for Web of Science ID 000235632100020

• Design considerations for robotic needle steering IEEE International Conference on Robotics and Automation (ICRA) Webster, R. J., Memisevic, J., Okamura, A. M. IEEE. 2005: 3588–3594

  View details for Web of Science ID 000235460103006

• Effects of velocity on human force control 1st Joint Eurohaptics Conference/Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Wu, M., Abbott, J. J., Okamura, A. M. IEEE COMPUTER SOC. 2005: 73–79

  View details for Web of Science ID 000228426200010

• A velocity-dependent model for needle insertion in soft tissue 8th International Conference on Medical Image Computing and Computer-Assisted Intervention Crouch, J. R., Schneider, C. M., Wainer, J., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2005: 624–632

  View details for Web of Science ID 000233356900077

• Diffusion-based motion planning for a nonholonomic flexible needle model IEEE International Conference on Robotics and Automation (ICRA) Park, W., Kim, J. S., Zhou, Y., Cowan, N. J., Okamura, A. M., Chirikjian, G. S. IEEE. 2005: 4600–4605

  View details for Web of Science ID 000235460103169

• Effects of gripping and translational forces on teleoperation Workshop on Multi-Point Interaction with Real and Virtual Objects Verner, L. N., Jeung, K. A., Okamura, A. M. WORLD SCIENTIFIC PUBL CO PTE LTD. 2005: 231–241

  View details for Web of Science ID 000231731400016

• The snaptic paddle: A modular haptic device 1st Joint Eurohaptics Conference/Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Wong, C. E., Okamura, A. M. IEEE COMPUTER SOC. 2005: 537–538

  View details for Web of Science ID 000228426200093

• Automatic detection and segmentation of robot-assisted surgical motions 8th International Conference on Medical Image Computing and Computer-Assisted Intervention Lin, H. C., Shafran, I., Murphy, T. E., Okamura, A. M., Yuh, D. D., Hager, G. D. SPRINGER-VERLAG BERLIN. 2005: 802–810

  View details for Web of Science ID 000233337000099

• Vision-assisted control for manipulation using virtual fixtures IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION Bettini, A., Marayong, P., Lang, S., Okamura, A. M., Hager, G. D. 2004; 20 (6): 953-966

  View details for DOI 10.1109/TRO.2004.829483

  View details for Web of Science ID 000225484300003

• Force modeling for needle insertion into soft tissue IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Okamura, A. M., Simone, C., O'Leary, M. D. 2004; 51 (10): 1707-1716

  Abstract

  The modeling of forces during needle insertion into soft tissue is important for accurate surgical simulation, preoperative planning, and intelligent robotic assistance for percutaneous therapies. We present a force model for needle insertion and experimental procedures for acquiring data from ex vivo tissue to populate that model. Data were collected from bovine livers using a one-degree-of-freedom robot equipped with a load cell and needle attachment. computed tomography imaging was used to segment the needle insertion process into phases identifying different relative velocities between the needle and tissue. The data were measured and modeled in three parts: 1) capsule stiffness, a nonlinear spring model; 2) friction, a modified Karnopp model; and 3) cutting, a constant for a given tissue. In addition, we characterized the effects of needle diameter and tip type on insertion force using a silicone rubber phantom. In comparison to triangular and diamond tips, a bevel tip causes more needle bending and is more easily affected by tissue density variations. Forces for larger diameter needles are higher due to increased cutting and friction forces.

  View details for DOI 10.1109/TBME.2004.831542

  View details for Web of Science ID 000224001900002

  View details for PubMedID 15490818

• Speed-accuracy characteristics of human-machine cooperative manipulation using virtual fixtures with variable admittance HUMAN FACTORS Marayong, P., Okamura, A. M. 2004; 46 (3): 518-532

  Abstract

  This work explores the effect of virtual fixture admittance on the performance, defined by error and time, of task execution with a human-machine cooperative system. A desired path is obtained using computer vision, and virtual fixtures for assistance in planar path following were implemented on an admittance-controlled robot. The admittance controller uses a velocity gain, so that the speed of the robot is proportional to the force applied by the operator. The level of virtual fixture guidance is determined by the admittance ratio, which is the ratio of the admittance gain of the force components orthogonal to the path to the gain of the force components parallel to the path. In Experiment 1, we found a linear relationship between admittance ratio and performance. In Experiment 2, we examined the effect of admittance ratio on the performance of three tasks: path following, off-path targeting, and obstacle avoidance. An algorithm was developed to select an appropriate admittance ratio based on the nature of the task. Automatic admittance ratio tuning is recommended for next-generation virtual fixtures. Actual or potential applications of this research include surgery, assembly, and manipulation at the macro and micro scales.

  View details for Web of Science ID 000224978900011

  View details for PubMedID 15573549

• Application of haptic feedback to robotic surgery JOURNAL OF LAPAROENDOSCOPIC & ADVANCED SURGICAL TECHNIQUES-PART A Bethea, B. T., Okamura, A. M., Kitagawa, M., Fitton, T. P., Cattaneo, S. M., Gott, V. L., Baumgartner, W. A., Yuh, D. D. 2004; 14 (3): 191-195

  Abstract

  Robotic surgical systems have greatly contributed to the advancement of minimally invasive endoscopic surgery. However, current robotic systems do not provide tactile or haptic feedback to the operating surgeon. Under certain circumstances, particularly with the manipulation of delicate tissues and suture materials, this may prove to be a significant irritation. We hypothesize that haptic feedback, in the form of sensory substitution, facilitates the performance of surgical knot tying. This preliminary study describes evidence that visual sensory substitution permits the surgeon to apply more consistent, precise, and greater tensions to fine suture materials without breakage during robot-assisted knot tying.

  View details for Web of Science ID 000222276000015

  View details for PubMedID 15245675

• Teleoperation with sensor/actuator asymmetry: Task performance with partial force feedback 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2004) Semere, W., Kitagawa, M., Okamura, A. M. IEEE COMPUTER SOC. 2004: 121–127

  View details for Web of Science ID 000189492300016

• Haptic rendering of tissue cutting with scissors 12th Conference on Medicine Meets Virtual Reality Weiss, D. J., Okamura, A. M. I O S PRESS. 2004: 407–409

  Abstract

  Current surgical simulations that involve cutting with scissors lack display of forces "between the fingers" of the operator. The objective of this work is to develop a fast, realistic haptic rendering technique for scissor cutting that can be easily integrated into soft tissue models commonly used in surgical simulators. A virtual environment was created for combined graphic and haptic display of cutting. Two 2-dimensional deformable mass-spring-damper models were developed: The first is a large mesh that is used to calculate translational forces. The second is a smaller, one-element-thick mesh that is used to calculate cutting (angular) forces. This technique was implemented on the haptic scissors, a haptic interface that allows motion and force feedback in translation and cutting.

  View details for Web of Science ID 000222326200090

  View details for PubMedID 15544316

• The effect of visual and haptic feedback on computer-assisted needle insertion. Computer aided surgery Gerovich, O., Marayong, P., Okamura, A. M. 2004; 9 (6): 243-249

  Abstract

  We present a study evaluating the effects of visual and haptic feedback on human performance in a needle insertion task.A one-degree-of-freedom needle insertion simulator with a three-layer tissue model (skin, fat and muscle) was used in perceptual experiments. The objective of the 14 subjects was to detect the puncture of each tissue layer using varying haptic and visual cues. Performance was measured by overshoot error-the distance traveled by the virtual needle after puncture.Without force feedback, real-time visual feedback reduced error by at least 87% in comparison to static image overlay. Force feedback, in comparison to no force feedback, reduced puncture overshoot by at least 52% when visual feedback was absent or limited to static image overlay. A combination of force and visual feedback improved performance, especially for tissues with low stiffness, by at least 43% with visual display of the needle position, and by at least 67% with visual display of layer deflection.Real-time image overlay significantly enhances controlled puncture during needle insertion. Force feedback may not be necessary except in circumstances where visual feedback is limited.

  View details for PubMedID 16112974

• Performance analysis of steady-hand teleoperation versus cooperative manipulation 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2004) Emeagwali, I., Marayong, P., Abbott, J. J., Okamura, A. M. IEEE COMPUTER SOC. 2004: 316–322

  View details for Web of Science ID 000189492300044

• Vision-based assistance for ophthalmic micro-surgery 7th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2004) Dewan, M., Marayong, P., Okamura, A. M., Hager, G. D. SPRINGER-VERLAG BERLIN. 2004: 49–57

  View details for Web of Science ID 000224322400007

• Effect of sensory substitution on suture manipulation forces for surgical teleoperation 12th Conference on Medicine Meets Virtual Reality Kitagawa, M., Dokko, D., Okamura, A. M., Bethea, B. T., Yuh, D. D. I O S PRESS. 2004: 157–163

  Abstract

  Bilateral telemanipulation, which applies haptic feedback to the operator, is not yet available in most commercial robot-assisted surgical systems. We have shown in previous work that the lack of haptic (force or tactile) feedback is detrimental in applications requiring fine suture manipulation. In this paper, we study the effect of substituting direct haptic feedback with visual and auditory cues. Using the da Vinci robot from Intuitive Surgical, we observed the difference between applied forces during a knot tying procedure for four different sensory feedback substitution scenarios: no feedback, auditory feedback, visual feedback, and a combination of auditory and visual feedback. Our results indicate that visual feedback, which provides continuous force information, would improve robot-assisted performance during complex surgical tasks such as knot tying with fine sutures. Discrete auditory feedback gives additional useful support to the surgeon.

  View details for Web of Science ID 000222326200037

  View details for PubMedID 15544263

• Methods for haptic feedback in teleoperated robot-assisted surgery INDUSTRIAL ROBOT-AN INTERNATIONAL JOURNAL Okamura, A. M. 2004; 31 (6): 499-508

  View details for DOI 10.1108/01439910410566362

  View details for Web of Science ID 000224825400008

• Virtual remote center of motion control for needle placement robots. Computer aided surgery Boctor, E. M., Webster, R. J., Mathieu, H., Okamura, A. M., Fichtinger, G. 2004; 9 (5): 175-183

  Abstract

  We present an algorithm that enables percutaneous needle-placement procedures to be performed with unencoded, unregistered, minimally calibrated robots while removing the constraint of placing the needle tip on a mechanically enforced Remote Center of Motion (RCM).The algorithm requires only online tracking of the surgical tool and a five-degree-of-freedom (5-DOF) robot comprising three prismatic DOF and two rotational DOF. An incremental adaptive motion control cycle guides the needle to the insertion point and also orients it to align with the target-entry-point line. The robot executes RCM motion without having a physically constrained fulcrum point.The proof-of-concept prototype system achieved 0.78 mm translation accuracy and 1.4 degrees rotational accuracy (this is within the tracker accuracy) within 17 iterative steps (0.5-1 s).This research enables robotic assistant systems for image-guided percutaneous procedures to be prototyped/constructed more quickly and less expensively than has been previously possible. Since the clinical utility of such systems is clear and has been demonstrated in the literature, our work may help promote widespread clinical adoption of this technology by lowering system cost and complexity.

  View details for PubMedID 16192059

• A modular 2-DOF force-sensing instrument for laparoscopic surgery 6th International Conference on Medical Image Computing and Computer-Assisted Intervention Prasad, S. K., Kitagawa, M., Fischer, G. S., Zand, J., Talamini, M. A., Taylor, R. H., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2003: 279–286

  View details for Web of Science ID 000188592600035

• The Haptic Scissors: Cutting in virtual environments 20th IEEE International Conference on Robotics and Automation (ICRA) Okamura, A. M., Webster, R. J., Nolin, J. T., Johnson, K. W., Jafry, H. IEEE. 2003: 828–833

  View details for Web of Science ID 000187419900133

• Virtual fixture architectures for telemanipulation 20th IEEE International Conference on Robotics and Automation (ICRA) Abbott, J. J., Okamura, A. M. IEEE. 2003: 2798–2805

  View details for Web of Science ID 000187419900449

• Uniting haptic exploration and display 10th International Symposium on Robotics Research (ISRR 2001) Okamura, A. M. SPRINGER-VERLAG BERLIN. 2003: 225–238

  View details for Web of Science ID 000183013700015

• Robotic needle insertion: Effects of friction and needle geometry 20th IEEE International Conference on Robotics and Automation (ICRA) O'Leary, M. D., Simone, C., Washio, T., Yoshinaka, K., Okamura, A. M. IEEE. 2003: 1774–1780

  View details for Web of Science ID 000187419900285

• Steady-hand teleoperation with virtual fixtures 12th IEEE International Workshop on Robot and Human Interactive Communication Abbott, J. J., Hager, G. D., Okamura, A. M. IEEE. 2003: 145–151

  View details for Web of Science ID 000186827500025

• Methods for intelligent localization and mapping during haptic exploration IEEE International Conference on Systems, Man and Cybernetics (SMC 03) Schaeffer, M. A., Okamura, A. M. IEEE. 2003: 3438–3445

  View details for Web of Science ID 000186578600561

• Virtual remote center of motion control for needle placement robots 6th International Conference on Medical Image Computing and Computer-Assisted Intervention Boctor, E. M., Webster, R. J., Mathieu, H., Okamura, A. M., Fichtinger, G. SPRINGER-VERLAG BERLIN. 2003: 157–164

  View details for Web of Science ID 000188592600020

• Activation cues and force scaling methods for virtual fixtures 11th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Nolin, J. T., Stemniski, P. M., Okamura, A. M. IEEE COMPUTER SOC. 2003: 404–409

  View details for Web of Science ID 000182249800053

• Recognition of operator motions for real-time assistance using virtual fixtures 11th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems Ming, L., Okamura, A. M. IEEE COMPUTER SOC. 2003: 125–131

  View details for Web of Science ID 000182249800017

• Analysis of virtual fixture contact stability for telemanipulation IEEE/RSJ International Conference on Intelligent Robots and Systems Abbott, J. J., Okamura, A. M. IEEE. 2003: 2699–2706

  View details for Web of Science ID 000187883300437

• Spatial motion constraints: Theory and demonstrations for robot guidance using virtual fixtures 20th IEEE International Conference on Robotics and Automation (ICRA) Marayong, P., Li, M., Okamura, A. M., Hager, G. D. IEEE. 2003: 1954–1959

  View details for Web of Science ID 000187419900313

• Measurement, analysis, and display of haptic signals during surgical cutting PRESENCE-TELEOPERATORS AND VIRTUAL ENVIRONMENTS Greenish, S., Hayward, V., Chial, V., Okamura, A., Steffen, T. 2002; 11 (6): 626-651

  View details for Web of Science ID 000180224100006

• The effect of visual and haptic feedback on manual and teleoperated needle insertion 5th International Conference on Medical Image Computing and Computer-Assisted Intervention Gerovichev, O., Marayong, P., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2002: 147–154

  View details for Web of Science ID 000189412100019

• On the display of haptic recordings for cutting biological tissues 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2002) Chial, V. B., Greenish, S., Okamura, A. M. IEEE COMPUTER SOC. 2002: 80–87

  View details for Web of Science ID 000175458900011

• Vision assisted control for manipulation using virtual fixtures: Experiments at macro and micro scales 19th IEEE International Conference on Robotics and Automation (ICRA) Bettini, A., Lang, S., Okamura, A., Hager, G. IEEE. 2002: 3354–3361

  View details for Web of Science ID 000178573200530

• Effect of virtual fixture compliance on human-machine cooperative manipulation IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2002) Marayong, P., Bettini, A., Okamura, A. IEEE. 2002: 1089–1095

  View details for Web of Science ID 000179289100177

• Measurement of the tip and friction force acting on a needle during penetration 5th International Conference on Medical Image Computing and Computer-Assisted Intervention Kataoka, H., Washio, T., Chinzei, K., Mizuhara, K., Simone, C., Okamura, A. M. SPRINGER-VERLAG BERLIN. 2002: 216–223

  View details for Web of Science ID 000189412100027

• Modeling of needle insertion forces for robot-assisted percutaneous therapy 19th IEEE International Conference on Robotics and Automation (ICRA) Simone, C., Okamura, A. M. IEEE. 2002: 2085–2091

  View details for Web of Science ID 000178573200328

• Building a task language for segmentation and recognition of user input to cooperative manipulation systems 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2002) Hundtofte, C. S., Hager, G. D., Okamura, A. M. IEEE COMPUTER SOC. 2002: 225–230

  View details for Web of Science ID 000175458900029

• Analysis of suture manipulation forces for teleoperation with force feedback 5th International Conference on Medical Image Computing and Computer-Assisted Intervention Kitagawa, M., Okamura, A. M., Bethea, B. T., Gott, V. L., Baumgartner, W. A. SPRINGER-VERLAG BERLIN. 2002: 155–162

  View details for Web of Science ID 000189412100020

• Feature detection for haptic exploration with robotic fingers INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH Okamura, A. M., Cutkosky, M. R. 2001; 20 (12): 925-938

  View details for Web of Science ID 000174018800001

• Reality-based models for vibration feedback in virtual environments IEEE-ASME TRANSACTIONS ON MECHATRONICS Okamura, A. M., Cutkosky, M. R., Dennerlein, J. T. 2001; 6 (3): 245-252

  View details for Web of Science ID 000171123900005

• Feature-guided exploration with a robotic finger IEEE International Conference on Robotics and Automation Okamura, A. M., Cutkosky, M. R. IEEE. 2001: 589–596

  View details for Web of Science ID 000172615800095

• Vision assisted control for manipulation using virtual fixtures IEEE Conference on Intelligent Robots and Systems (IROS 2001) Bettini, A., Lang, S., Okamura, A., Hager, G. IEEE. 2001: 1171–1176

  View details for Web of Science ID 000176593900187

• Haptic surface exploration 6th International Symposium on Experimental Robotics Okamura, A. M., Costa, M. A., Turner, M. L. SPRINGER-VERLAG LONDON LTD. 2000: 423–432

  View details for Web of Science ID 000086068500041

• Haptic exploration of fine surface features International Conference on Robotics and Automation (ICRA '99) Okamura, A. M., Cutkosky, M. R. IEEE. 1999: 2930–2936

  View details for Web of Science ID 000081625700464

• Vibration feedback models for virtual environments IEEE International Conference on Robotics and Automation Okamura, A. M., Dennerlein, J. T., Howe, R. D. IEEE. 1998: 674–679

  View details for Web of Science ID 000074368900107

• Haptic exploration of objects with rolling and sliding 1997 IEEE International Conference on Robotics and Automation (ICRA97) - Teaming to Make an Impact Okamura, A. M., Turner, M. L., Cutkosky, M. R. IEEE. 1997: 2485–2490

  View details for Web of Science ID A1997BJ42X00394