Fanrui Fu

All Publications

• From microscope to head-mounted display: integrating hand tracking into microsurgical augmented reality. International journal of computer assisted radiology and surgery El Chemaly, T., Athayde Neves, C., Fu, F., Hargreaves, B., Blevins, N. H. 2024

  Abstract

  The operating microscope plays a central role in middle and inner ear procedures that involve working within tightly confined spaces under limited exposure. Augmented reality (AR) may improve surgical guidance by combining preoperative computed tomography (CT) imaging that can provide precise anatomical information, with intraoperative microscope video feed. With current technology, the operator must manually interact with the AR interface using a computer. The latter poses a disruption in the surgical flow and is suboptimal for maintaining the sterility of the operating environment. The purpose of this study was to implement and evaluate free-hand interaction concepts leveraging hand tracking and gesture recognition as an attempt to reduce the disruption during surgery and improve human-computer interaction.An electromagnetically tracked surgical microscope was calibrated using a custom 3D printed calibration board. This allowed the augmentation of the microscope feed with segmented preoperative CT-derived virtual models. Ultraleap's Leap Motion Controller 2 was coupled to the microscope and used to implement hand-tracking capabilities. End-user feedback was gathered from a surgeon during development. Finally, users were asked to complete tasks that involved interacting with the virtual models, aligning them to physical targets, and adjusting the AR visualization.Following observations and user feedback, we upgraded the functionalities of the hand interaction system. User feedback showed the users' preference for the new interaction concepts that provided minimal disruption of the surgical workflow and more intuitive interaction with the virtual content.We integrated hand interaction concepts, typically used with head-mounted displays (HMDs), into a surgical stereo microscope system intended for AR in otologic microsurgery. The concepts presented in this study demonstrated a more favorable approach to human-computer interaction in a surgical context. They hold potential for a more efficient execution of surgical tasks under microscopic AR guidance.

  View details for DOI 10.1007/s11548-024-03224-w

  View details for PubMedID 39162975

  View details for PubMedCentralID 4710572

• Deep Learning Method for Rapid Simultaneous Multistructure Temporal Bone Segmentation. Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery Neves, C. A., Chemaly, T. E., Fu, F., Blevins, N. H. 2024

  Abstract

  To develop and validate a deep learning algorithm for the automated segmentation of key temporal bone structures from clinical computed tomography (CT) data sets.Cross-sectional study.A total of 325 CT scans from a clinical database.A state-of-the-art deep learning (DL) algorithm (SwinUNETR) was used to train a prediction model for rapid segmentation of 9 key temporal bone structures in a data set of 325 clinical CTs. The data set was manually annotated by a specialist to serve as the ground truth. The data set was randomly split into training (n = 260) and testing (n = 65) sets. The model's performance was objectively assessed through external validation on the test set using metrics including Dice, Balanced accuracy, Hausdorff distances, and processing time.The model achieved an average Dice coefficient of 0.87 for all structures, an average balanced accuracy of 0.94, an average Hausdorff distance of 0.79 mm, and an average processing time of 9.1 seconds per CT.The present DL model for the automated simultaneous segmentation of multiple structures within the temporal bone from CTs achieved high accuracy according to currently commonly employed objective analysis. The results demonstrate the potential of the method to improve preoperative evaluation and intraoperative guidance in otologic surgery.

  View details for DOI 10.1002/ohn.764

  View details for PubMedID 38769857

• Automated Radiomic Analysis of Vestibular Schwannomas and Inner Ears Using Contrast-Enhanced T1-Weighted and T2-Weighted Magnetic Resonance Imaging Sequences and Artificial Intelligence. Otology & neurotology : official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology Neves, C. A., Liu, G. S., El Chemaly, T., Bernstein, I. A., Fu, F., Blevins, N. H. 2023

  Abstract

  To objectively evaluate vestibular schwannomas (VSs) and their spatial relationships with the ipsilateral inner ear (IE) in magnetic resonance imaging (MRI) using deep learning.Cross-sectional study.A total of 490 adults with VS, high-resolution MRI scans, and no previous neurotologic surgery.MRI studies of VS patients were split into training (390 patients) and test (100 patients) sets. A three-dimensional convolutional neural network model was trained to segment VS and IE structures using contrast-enhanced T1-weighted and T2-weighted sequences, respectively. Manual segmentations were used as ground truths. Model performance was evaluated on the test set and on an external set of 100 VS patients from a public data set (Vestibular-Schwannoma-SEG).Dice score, relative volume error, average symmetric surface distance, 95th-percentile Hausdorff distance, and centroid locations.Dice scores for VS and IE volume segmentations were 0.91 and 0.90, respectively. On the public data set, the model segmented VS tumors with a Dice score of 0.89 ± 0.06 (mean ± standard deviation), relative volume error of 9.8 ± 9.6%, average symmetric surface distance of 0.31 ± 0.22 mm, and 95th-percentile Hausdorff distance of 1.26 ± 0.76 mm. Predicted VS segmentations overlapped with ground truth segmentations in all test subjects. Mean errors of predicted VS volume, VS centroid location, and IE centroid location were 0.05 cm3, 0.52 mm, and 0.85 mm, respectively.A deep learning system can segment VS and IE structures in high-resolution MRI scans with excellent accuracy. This technology offers promise to improve the clinical workflow for assessing VS radiomics and enhance the management of VS patients.

  View details for DOI 10.1097/MAO.0000000000003959

  View details for PubMedID 37464458

• Improving Transcranial Acoustic Targeting: The Limits of CT Based Velocity Estimates and The Role of MR. IEEE transactions on ultrasonics, ferroelectrics, and frequency control Webb, T. D., Fu, F., Leung, S. A., Ghanouni, P., Dahl, J., Does, M. D., Pauly, K. B. 2022; PP

  Abstract

  Transcranial magnetic resonance (MR) guided focused ultrasound (tcMRgFUS) enables the non-invasive treatment of the deep brain. This capacity relies on the ability to focus acoustic energy through the in-tact skull, a feat that requires accurate estimates of the acoustic velocity in individual patient skulls. In current practice, these estimates are generated using a pre-treatment CT scan and then registered to an MR dataset on the day of the treatment. Treatment safety and efficacy can be improved by eliminating the need to register the CT data to the MR images and by improving the accuracy of acoustic velocity measurements. In this study we examine the capacity of MR to supplement or replace CT as a means of estimating velocity in the skull. We find that MR can predict velocity with less but comparable accuracy to CT. We then use micro CT imaging to better understand the limitations of Hounsfield Unit (HU) based estimates of velocity, demonstrating that the macrostructure of pores in the skull contributes to the acoustic velocity of the bone. We find evidence that detailed T2 measurements provide information about pore macrostructure similar to the information obtained with micro CT, offering a potential clinical mechanism for improving patient specific estimates of acoustic velocity in the human skull.

  View details for DOI 10.1109/TUFFC.2022.3192224

  View details for PubMedID 35853046

• Distortion-Free Diffusion Imaging Using Self-Navigated Cartesian Echo-Planar Time Resolved Acquisition and Joint Magnitude and Phase Constrained Reconstruction IEEE TRANSACTIONS ON MEDICAL IMAGING Dai, E., Lee, P. K., Dong, Z., Fu, F., Setsompop, K., McNab, J. A. 2022; 41 (1): 63-74

  Abstract

  Echo-planar time resolved imaging (EPTI) is an effective approach for acquiring high-quality distortion-free images with a multi-shot EPI (ms-EPI) readout. As with traditional ms-EPI acquisitions, inter-shot phase variations present a main challenge when incorporating EPTI into a diffusion-prepared pulse sequence. The aim of this study is to develop a self-navigated Cartesian EPTI-based (scEPTI) acquisition together with a magnitude and phase constrained reconstruction for distortion-free diffusion imaging. A self-navigated Cartesian EPTI-based diffusion-prepared pulse sequence is designed. The different phase components in EPTI diffusion signal are analyzed and an approach to synthesize a fully phase-matched navigator for the inter-shot phase correction is demonstrated. Lastly, EPTI contains richer magnitude and phase information than conventional ms-EPI, such as the magnitude and phase correlations along the temporal dimension. The potential of these magnitude and phase correlations to enhance the reconstruction is explored. The reconstruction results with and without phase matching and with and without phase or magnitude constraints are compared. Compared with reconstruction without phase matching, the proposed phase matching method can improve the accuracy of inter-shot phase correction and reduce signal corruption in the final diffusion images. Magnitude constraints further improve image quality by suppressing the background noise and thereby increasing SNR, while phase constraints can mitigate possible image blurring from adding magnitude constraints. The high-quality distortion-free diffusion images and simultaneous diffusion-relaxometry imaging capacity provided by the proposed EPTI design represent a highly valuable tool for both clinical and neuroscientific assessments of tissue microstructure.

  View details for DOI 10.1109/TMI.2021.3104291

  View details for Web of Science ID 000736740900007

  View details for PubMedID 34383645

• Evaluation of magnetohydrodynamic effects in magnetic resonance electrical impedance tomography at ultra-high magnetic fields MAGNETIC RESONANCE IN MEDICINE Minhas, A. S., Chauhan, M., Fu, F., Sadleir, R. 2019; 81 (4): 2264–76

  Abstract

  Artifacts observed in experimental magnetic resonance electrical impedance tomography images were hypothesized to be because of magnetohydrodynamic (MHD) effects.Simulations of MREIT acquisition in the presence of MHD and electrical current flow were performed to confirm findings. Laminar flow and (electrostatic) electrical conduction equations were bidirectionally coupled via Lorentz force equations, and finite element simulations were performed to predict flow velocity as a function of time. Gradient sequences used in spin-echo and gradient echo acquisitions were used to calculate overall effects on MR phase images for different electrical current application or phase-encoding directions.Calculated and experimental phase images agreed relatively well, both qualitatively and quantitatively, with some exceptions. Refocusing pulses in spin echo sequences did not appear to affect experimental phase images.MHD effects were confirmed as the cause of observed experimental phase changes in MREIT images obtained at high fields. These findings may have implications for quantitative measurement of viscosity using MRI techniques. Methods developed here may be also important in studies of safety and in vivo artifacts observed in high field MRI systems.

  View details for DOI 10.1002/mrm.27534

  View details for Web of Science ID 000462092100005

  View details for PubMedID 30450638

  View details for PubMedCentralID PMC6373455

• Functional magnetic resonance electrical impedance tomography (fMREIT) sensitivity analysis using an active bidomain finite-element model of neural tissue MAGNETIC RESONANCE IN MEDICINE Sadleir, R. J., Fu, F., Chauhan, M. 2019; 81 (1): 602–14

  Abstract

  A direct method of imaging neural activity was simulated to determine typical signal sizes.An active bidomain finite-element model was used to estimate approximate perturbations in MR phase data as a result of neural tissue activity, and when an external MR electrical impedance tomography imaging current was added to the region containing neural current sources.Modeling-predicted, activity-related conductivity changes should produce measurable differential phase signals in practical MR electrical impedance tomography experiments conducted at moderate resolution at noise levels typical of high field systems. The primary dependence of MR electrical impedance tomography phase contrast on membrane conductivity changes, and not source strength, was demonstrated.Because the injected imaging current may also affect the level of activity in the tissue of interest, this technique can be used synergistically with neuromodulation techniques such as deep brain stimulation, to examine mechanisms of action.

  View details for DOI 10.1002/mrm.27351

  View details for Web of Science ID 000454009000049

  View details for PubMedID 29770490

  View details for PubMedCentralID PMC6239993

• The effect of potassium chloride on Aplysia Californica abdominal ganglion activity BIOMEDICAL PHYSICS & ENGINEERING EXPRESS Fu, F., Chauhan, M., Sadleir, R. 2018; 4 (3)

  View details for DOI 10.1088/2057-1976/aab72e

  View details for Web of Science ID 000434520900037

• Direct detection of neural activity in vitro using magnetic resonance electrical impedance tomography (MREIT) NEUROIMAGE Sadleir, R. J., Fu, F., Falgas, C., Holland, S., Boggess, M., Grant, S. C., Woo, E. 2017; 161: 104–19

  Abstract

  We describe a sequence of experiments performed in vitro to verify the existence of a new magnetic resonance imaging contrast - Magnetic Resonance Electrical Impedance Tomography (MREIT) -sensitive to changes in active membrane conductivity. We compared standard deviations in MREIT phase data from spontaneously active Aplysia abdominal ganglia in an artificial seawater background solution (ASW) with those found after treatment with an excitotoxic solution (KCl). We found significant increases in MREIT treatment cases, compared to control ganglia subject to extra ASW. This distinction was not found in phase images from the same ganglia using no imaging current. Further, significance and effect size depended on the amplitude of MREIT imaging current used. We conclude that our observations were linked to changes in cell conductivity caused by activity. Functional MREIT may have promise as a more direct method of functional neuroimaging than existing methods that image correlates of blood flow such as BOLD fMRI.

  View details for DOI 10.1016/j.neuroimage.2017.08.004

  View details for Web of Science ID 000415673100010

  View details for PubMedID 28818695

  View details for PubMedCentralID PMC5696120

• Temperature- and frequency-dependent dielectric properties of biological tissues within the temperature and frequency ranges typically used for magnetic resonance imaging-guided focused ultrasound surgery INTERNATIONAL JOURNAL OF HYPERTHERMIA Fu, F., Xin, S., Chen, W. 2014; 30 (1): 56–65

  Abstract

  This study aimed to obtain the temperature- and frequency-dependent dielectric properties of tissues subjected to magnetic resonance (MR) scanning for MR imaging-guided focused ultrasound surgery (MRgFUS). These variables are necessary to calculate radio frequency electromagnetic fields distribution and specific radio frequency energy absorption rate (SAR) in the healthy tissues surrounding the target tumours, and their variation may affect the efficacy of advanced RF pulses.The dielectric properties of porcine uterus, liver, kidney, urinary bladder, skeletal muscle, and fat were determined using an open-ended coaxial probe method. The temperature range was set from 36 °C to 60 °C; and the frequencies were set at 42.58 (1 T), 64 (1.5 T), 128 (3 T), 170 (4 T), 298 (7 T), 400 (9 T), and 468 MHz (11 T).Within the temperature and frequency ranges, the dielectric constants were listed as follows: uterus 49.6-121.64, liver 44.81-127.68, kidney 37.3-169.26, bladder 42.43-125.95, muscle 58.62-171.7, and fat 9.2327-20.2295. The following conductivities were obtained at the same temperature and frequency ranges: uterus 0.5506-1.4419, liver 0.5174-0.9709, kidney 0.8061-1.3625, bladder 0.6766-1.1817, muscle 0.8983-1.3083, and fat 0.1552-0.2316.The obtained data are consistent with the temperature and frequency ranges typically used in MRgFUS and thus can be used as reference to calculate radio frequency electromagnetic fields and SAR distribution inside the healthy tissues subjected to MR scanning for MRgFUS.

  View details for DOI 10.3109/02656736.2013.868534

  View details for Web of Science ID 000330708100008

  View details for PubMedID 24417349