Bio
Dr. Christoph Leuze is director of the Visualization Core at the Stanford Wu Tsai Institute where his research focuses on data visualization, data processing and user interaction. His goal is to leverage Artificial Intelligence and Augmented Reality to automate creation of guidance and training applications by learning from expert users. He taught the first Stanford course on medical mixed reality development and founded the Stanford Medical Mixed Reality program, an institute-wide initiative to bring together academia, clinic and industry to establish and improve mixed reality applications for patient care.
Dr. Leuze has received multiple prizes for his work in Augmented Reality including the IEEE VR People's choice award for the best AR demo, the TechConnect Award for one of the most promising technological innovations for national security and the prize for the best 3D video at the Ars Electronica Art and Science Festival. Dr. Leuze has studied at Leipzig and Chiba University and received the Otto Hahn medal of the Max Planck Society for his PhD thesis at the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig.
Honors & Awards
-
Stanford Neuroscience Insitute Interdisciplinary Scholar Awards, Stanford Neuroscience Insitute (2015)
-
Otto-Hahn-Medal, Max Planck Society (2014)
Education & Certifications
-
Postdoc, Stanford University, Radiology (2017)
-
PhD, Max Planck Institute for Human Cognitive and Brain Sciences & University of Leipzig, Physics (2013)
-
MS, University of Leipzig, Physics (2008)
Projects
-
BrainVR (5/2016 - 10/2016)
-
Ars Electronica (7/2013 - 9/2013)
Production of a 3D movie about the brain for the Ars Electronica 2013 in Linz/Austria
Location
Linz, Austria
For More Information:
All Publications
-
HoloDIEP-Faster and More Accurate Intraoperative DIEA Perforator Mapping Using a Novel Mixed Reality Tool.
Journal of reconstructive microsurgery
2024
Abstract
Microsurgical breast reconstruction using abdominal tissue is a complex procedure, in part, due to variable vascular/perforator anatomy. Preoperative computed tomography angiography (CTA) has mitigated this challenge to some degree; yet it continues to pose certain challenges. The ability to map perforators with Mixed Reality has been demonstrated in case studies, but its accuracy has not been studied intraoperatively. Here, we compare the accuracy of "HoloDIEP" in identifying perforator location (vs. Doppler ultrasound) by using holographic 3D models derived from preoperative CTA. Using a custom application on HoloLens, the deep inferior epigastric artery vascular tree was traced in 15 patients who underwent microsurgical breast reconstruction. Perforator markings were compared against the 3D model in a coordinate system centered on the umbilicus. Holographic- and Doppler-identified markings were compared using a perspective-corrected photo technique against the 3D model along with measurement of duration of perforator mapping for each technique. Vascular points in HoloDIEP skin markings were -0.97 ± 6.2 mm (perforators: -0.62 ± 6.13 mm) away from 3D-model ground-truth in radial length from the umbilicus at a true distance of 10.81 ± 6.14 mm (perforators: 11.40 ± 6.15 mm). Absolute difference in radial distance was twice as high for Doppler markings compared with Holo-markings (9.71 ± 6.16 and 4.02 ± 3.20 mm, respectively). Only in half of all cases (7/14), more than 50% of the Doppler-identified points were reasonably close (<30 mm) to 3D-model ground-truth. HoloDIEP was twice as fast as Doppler ultrasound (76.9s vs. 150.4 s per abdomen). HoloDIEP allows for faster and more accurate intraoperative perforator mapping than Doppler ultrasound.
View details for DOI 10.1055/s-0044-1788548
View details for PubMedID 39038461
-
Leveraging the Apple Ecosystem: Easy Viewing and Sharing of Three-dimensional Perforator Visualizations via iPad/iPhone-based Augmented Reality.
Plastic and reconstructive surgery. Global open
2024; 12 (7): e5940
Abstract
We introduce a novel technique using augmented reality (AR) on smartphones and tablets, making it possible for surgeons to review perforator anatomy in three dimensions on the go. Autologous breast reconstruction with abdominal flaps remains challenging due to the highly variable anatomy of the deep inferior epigastric artery. Computed tomography angiography has mitigated some but not all challenges. Previously, volume rendering and different headsets were used to enable better three-dimensional (3D) review for surgeons. However, surgeons have been dependent on others to provide 3D imaging data. Leveraging the ubiquity of Apple devices, our approach permits surgeons to review 3D models of deep inferior epigastric artery anatomy segmented from abdominal computed tomography angiography directly on their iPhone/iPad. Segmentation can be performed in common radiology software. The models are converted to the universal scene description zipped format, which allows immediate use on Apple devices without third-party software. They can be easily shared using secure, Health Insurance Portability and Accountability Act-compliant sharing services already provided by most hospitals. Surgeons can simply open the file on their mobile device to explore the images in 3D using "object mode" natively without additional applications or can switch to AR mode to pin the model in their real-world surroundings for intuitive exploration. We believe patient-specific 3D anatomy models are a powerful tool for intuitive understanding and communication of complex perforator anatomy and would be a valuable addition in routine clinical practice and education. Using this one-click solution on existing devices that is simple to implement, we hope to streamline the adoption of AR models by plastic surgeons.
View details for DOI 10.1097/GOX.0000000000005940
View details for PubMedID 38957720
View details for PubMedCentralID PMC11216661
-
The Reconstructive Metaverse - Collaboration in Real-Time Shared Mixed Reality Environments for Microsurgical Reconstruction.
Surgical innovation
2024: 15533506241262946
Abstract
Plastic surgeons routinely use 3D-models in their clinical practice, from 3D-photography and surface imaging to 3D-segmentations from radiological scans. However, these models continue to be viewed on flattened 2D screens that do not enable an intuitive understanding of 3D-relationships and cause challenges regarding collaboration with colleagues. The Metaverse has been proposed as a new age of applications building on modern Mixed Reality headset technology that allows remote collaboration on virtual 3D-models in a shared physical-virtual space in real-time. We demonstrate the first use of the Metaverse in the context of reconstructive surgery, focusing on preoperative planning discussions and trainee education. Using a HoloLens headset with the Microsoft Mesh application, we performed planning sessions for 4 DIEP-flaps in our reconstructive metaverse on virtual patient-models segmented from routine CT angiography. In these sessions, surgeons discuss perforator anatomy and perforator selection strategies whilst comprehensively assessing the respective models. We demonstrate the workflow for a one-on-one interaction between an attending surgeon and a trainee in a video featuring both viewpoints as seen through the headset. We believe the Metaverse will provide novel opportunities to use the 3D-models that are already created in everyday plastic surgery practice in a more collaborative, immersive, accessible, and educational manner.
View details for DOI 10.1177/15533506241262946
View details for PubMedID 38905568
-
Suture Packaging as a Marker for Intraoperative Image Alignment in Augmented Reality on Mobile Devices.
Plastic and reconstructive surgery. Global open
2024; 12 (6): e5933
Abstract
Preoperative vascular imaging has become standard practice in the planning of microsurgical breast reconstruction. Currently, translating perforator locations from radiological findings to a patient's abdomen is often not easy or intuitive. Techniques using three-dimensional printing or patient-specific guides have been introduced to superimpose anatomy onto the abdomen for reference. Augmented and mixed reality is currently actively investigated for perforator mapping by superimposing virtual models directly onto the patient. Most techniques have found only limited adoption due to complexity and price. Additionally, a critical step is aligning virtual models to patients. We propose repurposing suture packaging as an image tracking marker. Tracking markers allow quick and easy alignment of virtual models to the individual patient's anatomy. Current techniques are often complicated or expensive and limit intraoperative use of augmented reality models. Suture packs are sterile, readily available, and can be used to align abdominal models on the patients. Using an iPad, the augmented reality models automatically align in the correct position by using a suture pack as a tracking marker. Given the ubiquity of iPads, the combination of these devices with readily available suture packs will predictably lower the barrier to entry and utilization of this technology. Here, our workflow is presented along with its intraoperative utilization. Additionally, we investigated the accuracy of this technology.
View details for DOI 10.1097/GOX.0000000000005933
View details for PubMedID 38919516
View details for PubMedCentralID PMC11199004
-
Increasing DIEA Perforator Detail in 3D Photorealistic Volume Rendering Visualizations with Skin-masking and Cinematic Anatomy.
Plastic and reconstructive surgery
2024
Abstract
Preoperative CT angiography (CTA) is increasingly performed prior to perforator flap-based reconstruction. However, radiological 2D thin-slices do not allow for intuitive interpretation and translation to intraoperative findings. 3D volume rendering has been used to alleviate the need for mental 2D-to-3D abstraction. Even though volume rendering allows for a much easier understanding of anatomy, it currently has limited utility as the skin obstructs the view of critical structures. Using free, open-source software, we introduce a new skin-masking technique that allows surgeons to easily create a segmentation mask of the skin that can later be used to toggle the skin on and off. Additionally, the mask can be used in other rendering applications. We use Cinematic Anatomy for photorealistic volume rendering and interactive exploration of the CTA with and without skin. We present results from using this technique to investigate perforator anatomy in deep inferior epigastric perforator flaps and demonstrate that the skin-masking workflow is performed in less than 5 minutes. In Cinematic Anatomy, the view onto the abdominal wall and especially onto perforators becomes significantly sharper and more detailed when no longer obstructed by the skin. We perform a virtual, partial muscle dissection to show the intramuscular and submuscular course of the perforators. The skin-masking workflow allows surgeons to improve arterial and perforator detail in volume renderings easily and quickly by removing skin and could alternatively also be performed solely using open-source and free software. The workflow can be easily expanded to other perforator flaps without the need for modification.
View details for DOI 10.1097/PRS.0000000000011359
View details for PubMedID 38351515
-
An orexigenic subnetwork within the human hippocampus.
Nature
2023
Abstract
Only recently have more specific circuit-probing techniques become available to inform previous reports implicating the rodent hippocampus in orexigenic appetitive processing1-4. This function has been reported to be mediated at least in part by lateral hypothalamic inputs, including those involving orexigenic lateral hypothalamic neuropeptides, such as melanin-concentrating hormone5,6. This circuit, however, remains elusive in humans. Here we combine tractography, intracranial electrophysiology, cortico-subcortical evoked potentials, and brain-clearing 3D histology to identify an orexigenic circuit involving the lateral hypothalamus and converging in a hippocampal subregion. We found that low-frequency power is modulated by sweet-fat food cues, and this modulation was specific to the dorsolateral hippocampus. Structural and functional analyses of this circuit in a human cohort exhibiting dysregulated eating behaviour revealed connectivity that was inversely related to body mass index. Collectively, this multimodal approach describes an orexigenic subnetwork within the human hippocampus implicated in obesity and related eating disorders.
View details for DOI 10.1038/s41586-023-06459-w
View details for PubMedID 37648849
View details for PubMedCentralID 6468104
-
An orexigenic subnetwork within the human hippocampus
NATURE
2023
View details for DOI 10.1038/s41586-023-06459
View details for Web of Science ID 001064862900002
-
Stereoscopic calibration for augmented reality visualization in microscopic surgery.
International journal of computer assisted radiology and surgery
2023
Abstract
Middle and inner ear procedures target hearing loss, infections, and tumors of the temporal bone and lateral skull base. Despite the advances in surgical techniques, these procedures remain challenging due to limited haptic and visual feedback. Augmented reality (AR) may improve operative safety by allowing the 3D visualization of anatomical structures from preoperative computed tomography (CT) scans on real intraoperative microscope video feed. The purpose of this work was to develop a real-time CT-augmented stereo microscope system using camera calibration and electromagnetic (EM) tracking.A 3D printed and electromagnetically tracked calibration board was used to compute the intrinsic and extrinsic parameters of the surgical stereo microscope. These parameters were used to establish a transformation between the EM tracker coordinate system and the stereo microscope image space such that any tracked 3D point can be projected onto the left and right images of the microscope video stream. This allowed the augmentation of the microscope feed of a 3D printed temporal bone with its corresponding CT-derived virtual model. Finally, the calibration board was also used for evaluating the accuracy of the calibration.We evaluated the accuracy of the system by calculating the registration error (RE) in 2D and 3D in a microsurgical laboratory setting. Our calibration workflow achieved a RE of 0.11 ± 0.06 mm in 2D and 0.98 ± 0.13 mm in 3D. In addition, we overlaid a 3D CT model on the microscope feed of a 3D resin printed model of a segmented temporal bone. The system exhibited small latency and good registration accuracy.We present the calibration of an electromagnetically tracked surgical stereo microscope for augmented reality visualization. The calibration method achieved accuracy within a range suitable for otologic procedures. The AR process introduces enhanced visualization of the surgical field while allowing depth perception.
View details for DOI 10.1007/s11548-023-02980-5
View details for PubMedID 37450175
View details for PubMedCentralID 4634572
-
Spatial Fidelity of Microvascular Perforating Vessels as Perceived by Augmented Reality Virtual Projections.
Plastic and reconstructive surgery
2023
Abstract
Autologous breast reconstruction yields improved long-term aesthetic results but requires increased resources of practitioners and hospital systems. Innovations in radiographic imaging have been used increasingly to improve the efficiency and success of free-flap harvest. Augmented reality (AR) affords the opportunity to superimpose relevant imaging on a surgeon's native field of view, potentially facilitating dissection of anatomically variable structures. To validate the spatial fidelity of AR projections of deep inferior epigastric perforator flap (DIEP) relevant anatomy, comparisons of 3D models and their virtual renderings were performed by four independent observers. Measured discrepancies between the real and holographic models were evaluated.3D-printed models of DIEP relevant anatomy were fabricated from CTA data from 19 de-identified patients. The corresponding CTA data was similarly formatted for the Microsoft Hololens to generate corresponding projections. Anatomic points were initially measured on 3D models, after which, the corresponding points were measured on the Hololens projections from two separate vantages. Statistical analyses, including Generalized Linear Modeling, were performed to characterize spatial fidelity regarding translation, rotation, and scale of holographic projections.Amongst all participants, the median translational displacement at corresponding points was 9.0 mm, 12.1 mm, and 13.5 mm between the real 3D model and V1, 3D model and V2, and between V1 and V2, respectively.Corresponding points, including topography of perforating vessels for the purposes of breast reconstruction can be identified within millimeters, but there remain multiple independent contributors of error, most notably the participant and location at which the projection is perceived.
View details for DOI 10.1097/PRS.0000000000010592
View details for PubMedID 37092985
-
Virtual Resection Specimen Interaction Using Augmented Reality Holograms to Guide Margin Communication and Flap Sizing.
Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery
2023
Abstract
Head and neck surgeons often have difficulty in relocating sites of positive margins due to the complex 3-dimensional (3D) anatomy of the head and neck. We introduce a new technique where resection specimens are 3D scanned with a smartphone, annotated in computer-assisted design software, and immediately visualized on augmented reality (AR) glasses. The 3D virtual specimen can be accurately superimposed onto surgical sites for orientation and sizing applications. During an operative workshop, a surgeon using AR glasses projected virtual, annotated specimen models back into the resection bed onto a cadaver within approximately 10minutes. Colored annotations can correspond with pathologic annotations and guide the orientation of the virtual 3D specimen. The model was also overlayed onto a flap harvest site to aid in reconstructive planning. We present a new technique allowing interactive, sterile inspection of tissue specimens in AR that could facilitate communication among surgeons and pathologists and assist with reconstructive surgery.
View details for DOI 10.1002/ohn.325
View details for PubMedID 36934457
-
The Impact of Occlusion on Depth Perception at Arm's Length.
IEEE transactions on visualization and computer graphics
2023; 29 (11): 4494-4502
Abstract
This paper investigates the accuracy of Augmented Reality (AR) technologies, particularly commercially available optical see-through displays, in depicting virtual content inside the human body for surgical planning. Their inherent limitations result in inaccuracies in perceived object positioning. We examine how occlusion, specifically with opaque surfaces, affects perceived depth of virtual objects at arm's length working distances. A custom apparatus with a half-silvered mirror was developed, providing accurate depth cues excluding occlusion, differing from commercial displays. We carried out a study, contrasting our apparatus with a HoloLens 2, involving a depth estimation task under varied surface complexities and illuminations. In addition, we explored the effects of creating a virtual "hole" in the surface. Subjects' depth estimation accuracy and confidence were a ssessed. Results showed more depth estimation variation with HoloLens and significant depth error beneath complex occluding surfaces. However, creating a virtual hole significantly reduced depth errors and increased subjects' confidence, irrespective of accuracy enhancement. These findings have important implications for the design and use of mixed-reality technologies in surgical applications, and industrial applications such as using virtual content to guide maintenance or repair of components hidden beneath the opaque outer surface of equipment. A free copy of this paper and all supplemental materials are available at https://bit.ly/3YbkwjU.
View details for DOI 10.1109/TVCG.2023.3320239
View details for PubMedID 37782607
-
Augmented Reality for Medical Training in Eastern Africa
IEEE COMPUTER SOC. 2023: 891-892
View details for DOI 10.1109/VRW58643.2023.00288
View details for Web of Science ID 000990508800273
-
Audiovisual augmentation for coil positioning in transcranial magnetic stimulation
COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING-IMAGING AND VISUALIZATION
2022
View details for DOI 10.1080/21681163.2022.2154277
View details for Web of Science ID 000898890200001
-
Remote Training for Medical Staff in Low-Resource Environments Using Augmented Reality.
Journal of imaging
2022; 8 (12)
Abstract
This work aims to leverage medical augmented reality (AR) technology to counter the shortage of medical experts in low-resource environments. We present a complete and cross-platform proof-of-concept AR system that enables remote users to teach and train medical procedures without expensive medical equipment or external sensors. By seeing the 3D viewpoint and head movements of the teacher, the student can follow the teacher's actions on the real patient. Alternatively, it is possible to stream the 3D view of the patient from the student to the teacher, allowing the teacher to guide the student during the remote session. A pilot study of our system shows that it is easy to transfer detailed instructions through this remote teaching system and that the interface is easily accessible and intuitive for users. We provide a performant pipeline that synchronizes, compresses, and streams sensor data through parallel efficiency.
View details for DOI 10.3390/jimaging8120319
View details for PubMedID 36547484
-
Changes In The Cerebello-thalamo-cortical Network After MR-guided Focused Ultrasound Thalamotomy.
Brain connectivity
2022
Abstract
Object In recent years, transcranial MR-guided focused ultrasound (tcMRgFUS) has been established as a potential treatment option for movement disorders, including essential tremor. So far, however, little is known about the impact of tcMRgFUS on structural connectivity. The objective of this study was to detect microstructural changes in tremor- and motor-related white matter tracts in essential tremor patients treated with tcMRgFUS thalamotomy. Methods Eleven patients diagnosed with essential tremor were enrolled in this tcMRgFUS thalamotomy study. For each patient, 3T MRI including structural and diffusion MRI were acquired and the Clinical Rating Scale for Tremor was assessed prior to the procedure as well as one year after the treatment. Diffusion MRI tractography was performed to identify the cerebello-thalamo-cortical tract (CTCT), the medial lemniscus (ML) and the corticospinal tract (CST) in both hemispheres on pre-treatment data. Pre-treatment tractography results were co-registered to post-treatment diffusion data. Diffusion tensor imaging (DTI) metrics, including fractional anisotropy (FA), mean diffusivity (MD) and radial diffusivity (RD), were averaged across the tracts in the pre- and post-treatment data. Results The mean value of tract-specific DTI metrics changed significantly within the thalamic lesion and in the CTCT on the treated side (p<0.05). Changes of DTI-derived indices within the CTCT correlated well with lesion overlap (FA: r=-0.54, p=0.04; MD: r=0.57, p=0.04); RD: r=0.67, p=0.036). Furthermore, a trend was seen for the correlation between changes of DTI-derived indices within the CTCT and clinical improvement (FA: r=0.58; p=0.062; MD: r=-0.52, p=0.64; RD: r=-0.61 p=0.090). Conclusions Microstructural changes were detected within the CTCT after tcMRgFUS and these changes correlated well with lesion-tract overlap. Our results show that diffusion MRI is able to detect the microstructural effects of tcMRgFUS, thereby further elucidating the treatment mechanism and ultimately to improve targeting prospectively.
View details for DOI 10.1089/brain.2021.0157
View details for PubMedID 35678063
-
Evaluation Challenges for the Application of Extended Reality Devices in Medicine.
Journal of digital imaging
2022
Abstract
Augmented and virtual reality devices are being actively investigated and implemented for a wide range of medical uses. However, significant gaps in the evaluation of these medical devices and applications hinder their regulatory evaluation. Addressing these gaps is critical to demonstrating the devices' safety and effectiveness. We outline the key technical and clinical evaluation challenges discussed during the US Food and Drug Administration's public workshop, "Medical Extended Reality: Toward Best Evaluation Practices for Virtual and Augmented Reality in Medicine" and future directions for evaluation method development. Evaluation challenges were categorized into several key technical and clinical areas. Finally, we highlight current efforts in the standards communities and illustrate connections between the evaluation challenges and the intended uses of the medical extended reality (MXR) devices. Participants concluded that additional research is needed to assess the safety and effectiveness of MXR devices across the use cases.
View details for DOI 10.1007/s10278-022-00622-x
View details for PubMedID 35469355
-
Phantom study of SPECT/CT augmented reality for intraoperative localization of sentinel lymph nodes in head and neck melanoma.
Oral oncology
1800; 125: 105702
Abstract
OBJECTIVE: To show that augmented reality (AR) visualization of single-photon emission computed tomography (SPECT)/computed tomography (CT) data in 3D can be used to accurately localize targets in the head and neck region.MATERIALS AND METHODS: Eight head and neck styrofoam phantoms were painted with a mixture of radioactive solution (Tc-99m) detectable with a handheld gamma probe and fluorescent ink visible only under ultraviolet (UV) light to create 10-20 simulated lymph nodes on their surface. After obtaining SPECT/CT images of these phantoms, virtual renderings of the nodes were generated from the SPECT/CT data and displayed using a commercially available AR headset. For each of three physician evaluators, the time required to localize lymph node targets was recorded (1) using the gamma probe alone and (2) using the gamma probe while wearing the AR headset. In addition, the surface localization accuracy when using the AR headset was evaluated by measuring the misalignment between the locations visually marked by the evaluators and the ground truth locations identified using UV stimulation of the ink at the site of the nodes.RESULTS: For all three evaluators, using the AR headset significantly reduced the time to detect targets (P=0.012, respectively) compared to using the gamma probe alone. The average misalignment between the location marked by the evaluators and the ground truth location was 8.6mm.CONCLUSION: AR visualization of SPECT/CT data in 3D allows for accurate localization of targets in the head and neck region, and may reduce the localization time of targets.
View details for DOI 10.1016/j.oraloncology.2021.105702
View details for PubMedID 34991004
-
Augmented Reality for Retrosigmoid Craniotomy Planning
JOURNAL OF NEUROLOGICAL SURGERY PART B-SKULL BASE
2021
View details for DOI 10.1055/s-0041-1735509
View details for Web of Science ID 000694675300004
-
Nanostructure-specific X-ray tomography reveals myelin levels, integrity and axon orientations in mouse and human nervous tissue.
Nature communications
2021; 12 (1): 2941
Abstract
Myelin insulates neuronal axons and enables fast signal transmission, constituting a key component of brain development, aging and disease. Yet, myelin-specific imaging of macroscopic samples remains a challenge. Here, we exploit myelin's nanostructural periodicity, and use small-angle X-ray scattering tensor tomography (SAXS-TT) to simultaneously quantify myelin levels, nanostructural integrity and axon orientations in nervous tissue. Proof-of-principle is demonstrated in whole mouse brain, mouse spinal cord and human white and gray matter samples. Outcomes are validated by 2D/3D histology and compared to MRI measurements sensitive to myelin and axon orientations. Specificity to nanostructure is exemplified by concomitantly imaging different myelin types with distinct periodicities. Finally, we illustrate the method's sensitivity towards myelin-related diseases by quantifying myelin alterations in dysmyelinated mouse brain. This non-destructive, stain-free molecular imaging approach enables quantitative studies of myelination within and across samples during development, aging, disease and treatment, and is applicable to other ordered biomolecules or nanostructures.
View details for DOI 10.1038/s41467-021-22719-7
View details for PubMedID 34011929
-
Augmented Reality Visualization Tool For The Future of Tactical Combat Casualty Care.
The journal of trauma and acute care surgery
2021
Abstract
The objective of this project was to identify and develop software for an Augmented Reality (AR) application that runs on the US Army Integrated Visual Augmentation System (IVAS) to support a medical caregiver during training and combat casualty care (TC3) scenarios. In this AR-TC3 application, human anatomy of individual soldiers obtained pre-deployment is superimposed on the view of an injured warfighter through the IVAS. This offers insight into the anatomy of the injured warfighter to advance treatment in austere environments.In this article, we describe various software components required for an AR-TC3-tool. These include a body pose tracking system to track the patient's body pose, a virtual rendering of a human anatomy avatar, speech input to control the application and rendering techniques to visualize the virtual anatomy and treatment information on the AR display. We then implemented speech commands and visualization for four common medical scenarios including injury of a limb, a blast to the pelvis, cricothyrotomy, and a pneumothorax on the Microsoft Hololens 1.The software is designed for a forward surgical care tool on the US Army IVAS, with the intention to provide the medical caregiver with a unique ability to quickly assess affected internal anatomy. The current software components still had some limitations with respect to speech recognition reliability during noise and body pose tracking. These will likely be improved with the improved hardware of the IVAS, which is based on a modified Hololens 2.LEVEL OF EVIDENCE: Basic science paper.
View details for DOI 10.1097/TA.0000000000003263
View details for PubMedID 33938509
-
Comparison of diffusion MRI and CLARITY fiber orientation estimates in both gray and white matter regions of human and primate brain.
NeuroImage
2020; 228: 117692
Abstract
Diffusion MRI (dMRI) represents one of the few methods for mapping brain fiber orientations non-invasively. Unfortunately, dMRI fiber mapping is an indirect method that relies on inference from measured diffusion patterns. Comparing dMRI results with other modalities is a way to improve the interpretation of dMRI data and help advance dMRI technologies. Here, we present methods for comparing dMRI fiber orientation estimates with optical imaging of fluorescently labeled neurofilaments and vasculature in 3D human and primate brain tissue cuboids cleared using CLARITY. The recent advancements in tissue clearing provide a new opportunity to histologically map fibers projecting in 3D, which represents a captivating complement to dMRI measurements. In this work, we demonstrate the capability to directly compare dMRI and CLARITY in the same human brain tissue and assess multiple approaches for extracting fiber orientation estimates from CLARITY data. We estimate the three-dimensional neuronal fiber and vasculature orientations from neurofilament and vasculature stained CLARITY images by calculating the tertiary eigenvector of structure tensors. We then extend CLARITY orientation estimates to an orientation distribution function (ODF) formalism by summing multiple sub-voxel structure tensor orientation estimates. In a sample containing part of the human thalamus, there is a mean angular difference of 19o±15o between the primary eigenvectors of the dMRI tensors and the tertiary eigenvectors from the CLARITY neurofilament stain. We also demonstrate evidence that vascular compartments do not affect the dMRI orientation estimates by showing an apparent lack of correspondence (mean angular difference=49o±23o) between the orientation of the dMRI tensors and the structure tensors in the vasculature stained CLARITY images. In a macaque brain dataset, we examine how the CLARITY feature extraction depends on the chosen feature extraction parameters. By varying the volume of tissue over which the structure tensor estimates are derived, we show that orientation estimates are noisier with more spurious ODF peaks for sub-voxels below 30m3 and that, for our data, the optimal gray matter sub-voxel size is between 62.5m3 and 125m3. The example experiments presented here represent an important advancement towards robust multi-modal MRI-CLARITY comparisons.
View details for DOI 10.1016/j.neuroimage.2020.117692
View details for PubMedID 33385546
-
Application of holographic augmented reality for external approaches to the frontal sinus.
International forum of allergy & rhinology
2020
Abstract
External approaches to the frontal sinus such as osteoplastic flaps are challenging because they require blind entry into the sinus, posing risks of injury to the brain or orbit. Intraoperative computed tomography (CT)-based navigation is the current standard for planning the approach, but still necessitates blind entry into the sinus. The aim of this work was to describe a novel technique for external approaches to the frontal sinus using a holographic augmented reality (AR) application.Our team developed an AR system to create a 3-dimensional (3D) hologram of key anatomical structures, based on CT scans images. Using Magic Leap AR goggles for visualization, the frontal sinus hologram was aligned to the surface anatomy in 6 fresh cadaveric heads' anatomic boundaries, and the boundaries of the frontal sinus were demarcated based on the margins of the fused image. Trephinations and osteoplastic flap approaches were performed. The specimens were re-scanned to assess the accuracy of the osteotomy with respect to the actual frontal sinus perimeter.Registration and surgery were completed successfully in all specimens. Registration required an average of 2 minutes. The postprocedure CT showed a mean difference of 1.4 ± 4.1 mm between the contour of the osteotomy and the contour of the frontal sinus. One surgical complication (posterior table perforation) occurred (16%).We describe proof of concept of a novel technique utilizing AR to enhance external approaches to the frontal sinus. Holographic AR-enhanced surgical navigation holds promise for enhanced visualization of target structures during surgical approaches to the sinuses.
View details for DOI 10.1002/alr.22546
View details for PubMedID 32362076
-
Evaluation of Different Visualization Techniques for Perception-Based Alignment in Medical AR
IEEE COMPUTER SOC. 2020: 45-50
View details for DOI 10.1109/ISMAR-Adjunct51615.2020.00027
View details for Web of Science ID 000713571300012
-
Comparison of head pose tracking methods for mixed-reality neuronavigation for transcranial magnetic stimulation
SPIE Medical Imaging
2020
View details for DOI 10.1117/12.2547917
-
Landmark-based mixed-reality perceptual alignment of medical imaging data and accuracy validation in living subjects
IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
2020
View details for DOI 10.1109/ISMAR50242.2020.00095
-
Multimodal characterization of the human nucleus accumbens
NEUROIMAGE
2019; 198: 137–49
View details for DOI 10.1016/j.neuroimage.2019.05.019
View details for Web of Science ID 000472495100013
-
Generalized diffusion spectrum magnetic resonance imaging (GDSI) for model-free reconstruction of the ensemble average propagator
NEUROIMAGE
2019; 189: 497–515
View details for DOI 10.1016/j.neuroimage.2019.01.038
View details for Web of Science ID 000461166900043
-
Multimodal image registration and connectivity analysis for integration of connectomic data from microscopy to MRI.
Nature communications
2019; 10 (1): 5504
Abstract
3D histology, slice-based connectivity atlases, and diffusion MRI are common techniques to map brain wiring. While there are many modality-specific tools to process these data, there is a lack of integration across modalities. We develop an automated resource that combines histologically cleared volumes with connectivity atlases and MRI, enabling the analysis of histological features across multiple fiber tracts and networks, and their correlation with in-vivo biomarkers. We apply our pipeline in a murine stroke model, demonstrating not only strong correspondence between MRI abnormalities and CLARITY-tissue staining, but also uncovering acute cellular effects in areas connected to the ischemic core. We provide improved maps of connectivity by quantifying projection terminals from CLARITY viral injections, and integrate diffusion MRI with CLARITY viral tracing to compare connectivity maps across scales. Finally, we demonstrate tract-level histological changes of stroke through this multimodal integration. This resource can propel investigations of network alterations underlying neurological disorders.
View details for DOI 10.1038/s41467-019-13374-0
View details for PubMedID 31796741
-
RNA-Sequencing Analysis Revealed a Distinct Motor Cortex Transcriptome in Spontaneously Recovered Mice After Stroke
STROKE
2018; 49 (9): 2191–99
View details for DOI 10.1161/STROKEAHA.118.021508
View details for Web of Science ID 000442858100035
-
Double diffusion encoding MRI for the clinic
MAGNETIC RESONANCE IN MEDICINE
2018; 80 (2): 507–20
Abstract
The purpose of this study is to develop double diffusion encoding (DDE) MRI methods for clinical use. Microscopic diffusion anisotropy measurements from DDE promise greater specificity to changes in tissue microstructure compared with conventional diffusion tensor imaging, but implementation of DDE sequences on whole-body MRI scanners is challenging because of the limited gradient strengths and lengthy acquisition times.A custom single-refocused DDE sequence was implemented on a 3T whole-body scanner. The DDE gradient orientation scheme and sequence parameters were optimized based on a Gaussian diffusion assumption. Using an optimized 5-min DDE acquisition, microscopic fractional anisotropy (μFA) maps were acquired for the first time in multiple sclerosis patients.Based on simulations and in vivo human measurements, six parallel and six orthogonal diffusion gradient pairs were found to be the minimum number of diffusion gradient pairs necessary to produce a rotationally invariant measurement of μFA. Simulations showed that optimal precision and accuracy of μFA measurements were obtained using b-values between 1500 and 3000 s/mm2 . The μFA maps showed improved delineation of multiple sclerosis lesions compared with conventional fractional anisotropy and distinct contrast from T2 -weighted fluid attenuated inversion recovery and T1 -weighted imaging.The μFA maps can be measured using DDE in a clinical setting and may provide new opportunities for characterizing multiple sclerosis lesions and other types of tissue degeneration. Magn Reson Med 80:507-520, 2018. © 2017 International Society for Magnetic Resonance in Medicine.
View details for PubMedID 29266375
View details for PubMedCentralID PMC5910247
-
Mixed-reality guidance for brain stimulation treatment of depression
IEEE. 2018: 377–80
View details for DOI 10.1109/ISMAR-Adjunct.2018.00109
View details for Web of Science ID 000487013100090
-
The separate effects of lipids and proteins on brain MRI contrast revealed through tissue clearing.
NeuroImage
2017
Abstract
Despite the widespread use of magnetic resonance imaging (MRI) of the brain, the relative contribution of different biological components (e.g. lipids and proteins) to structural MRI contrasts (e.g., T1, T2, T2*, proton density, diffusion) remains incompletely understood. This limitation can undermine the interpretation of clinical MRI and hinder the development of new contrast mechanisms. Here, we determine the respective contribution of lipids and proteins to MRI contrast by removing lipids and preserving proteins in mouse brains using CLARITY. We monitor the temporal dynamics of tissue clearance via NMR spectroscopy, protein assays and optical emission spectroscopy. MRI of cleared brain tissue showed: 1) minimal contrast on standard MRI sequences; 2) increased relaxation times; and 3) diffusion rates close to free water. We conclude that lipids, present in myelin and membranes, are a dominant source of MRI contrast in brain tissue.
View details for DOI 10.1016/j.neuroimage.2017.04.021
View details for PubMedID 28411157
-
The separate effects of lipids and proteins on brain MRI contrast revealed through tissue clearing.
NeuroImage
2017
Abstract
Despite the widespread use of magnetic resonance imaging (MRI) of the brain, the relative contribution of different biological components (e.g. lipids and proteins) to structural MRI contrasts (e.g., T1, T2, T2*, proton density, diffusion) remains incompletely understood. This limitation can undermine the interpretation of clinical MRI and hinder the development of new contrast mechanisms. Here, we determine the respective contribution of lipids and proteins to MRI contrast by removing lipids and preserving proteins in mouse brains using CLARITY. We monitor the temporal dynamics of tissue clearance via NMR spectroscopy, protein assays and optical emission spectroscopy. MRI of cleared brain tissue showed: 1) minimal contrast on standard MRI sequences; 2) increased relaxation times; and 3) diffusion rates close to free water. We conclude that lipids, present in myelin and membranes, are a dominant source of MRI contrast in brain tissue.
View details for DOI 10.1016/j.neuroimage.2017.04.021
View details for PubMedID 28411157
-
Early Non-invasive Detection of Acute 1,2-Dichloroethane-induced Toxic Encephalopathy in Rats.
In vivo
2016; 30 (6): 787-793
Abstract
To assess the acute effect of 1,2-dichloroethane (1,2-DCE) on rat brain using diffusion magnetic resonance imaging (dMRI).We performed dMRI on 30 male Sprague-Dawley rats, microstructural alterations were investigated by calculating the mean fractional anisotropy (FA) and apparent diffusion coefficient (ADC) changes in eight selected brain regions of interest. For the whole brain, clusters of 20+ voxels that differed significantly in FA and ADC between groups were marked. Hematoxylin-eosin staining was performed to confirm pathological changes.Brain images showed lesions with brain edema in the white matter in both hemispheres in all groups exposed to 1,2-DCE. Diffusivity values were significantly different after 1,2-DCE inhalation (p<0.05).Primarily cytotoxic edema occurred in acute 1,2-DCE-induced brain edema in rats. dMRI could be used for the early non-invasive detection of acute 1,2-DCE-induced toxic encephalopathy.
View details for PubMedID 27815462
-
Myelin and iron concentration in the human brain: A quantitative study of MRI contrast.
NeuroImage
2014
Abstract
During the last five years ultra-high-field magnetic resonance imaging (MRI) has enabled an unprecedented view of living human brain. Brain tissue contrast in most MRI sequences is known to reflect mainly the spatial distributions of myelin and iron. These distributions have been shown to overlap significantly in many brain regions, especially in the cortex. It is of increasing interest to distinguish and identify cortical areas by their appearance in MRI, which has been shown to be feasible in vivo. Parcellation can benefit greatly from quantification of the independent contributions of iron and myelin to MRI contrast. Recent studies using susceptibility mapping claim to allow such a separation of the effects of myelin and iron in MRI. We show, using post-mortem human brain tissue, that this goal can be achieved. After MRI scanning of the block with appropriate T1 mapping and T2* weighted sequences, we section the block and apply a novel technique, proton induced X-ray emission (PIXE), to spatially map iron, phosphorus and sulfur elemental concentrations, simultaneously with 1μm spatial resolution. Because most brain phosphorus is located in myelin phospholipids, a calibration step utilizing element maps of sulfur enables semi-quantitative ex vivo mapping of myelin concentration. Combining results for iron and myelin concentration in a linear model, we have accurately modeled MRI tissue contrasts. Conversely, iron and myelin concentrations can now be estimated from appropriate MRI measurements in post-mortem brain samples.
View details for DOI 10.1016/j.neuroimage.2014.02.026
View details for PubMedID 24607447
-
Layer-Specific Intracortical Connectivity Revealed with Diffusion MRI
CEREBRAL CORTEX
2014; 24 (2): 328-339
Abstract
In this work, we show for the first time that the tangential diffusion component is orientationally coherent at the human cortical surface. Using diffusion magnetic resonance imaging (dMRI), we have succeeded in tracking intracortical fiber pathways running tangentially within the cortex. In contrast with histological methods, which reveal little regarding 3-dimensional organization in the human brain, dMRI delivers additional understanding of the layer dependence of the fiber orientation. A postmortem brain block was measured at very high angular and spatial resolution. The dMRI data had adequate resolution to allow analysis of the fiber orientation within 4 notional cortical laminae. We distinguished a lamina at the cortical surface where diffusion was tangential along the surface, a lamina below the surface where diffusion was mainly radial, an internal lamina covering the Stria of Gennari, where both strong radial and tangential diffusion could be observed, and a deep lamina near the white matter, which also showed mainly radial diffusion with a few tangential compartments. The measurement of the organization of the tangential diffusion component revealed a strong orientational coherence at the cortical surface.
View details for DOI 10.1093/cercor/bhs311
View details for Web of Science ID 000329840900005
View details for PubMedID 23099298
-
Systematic changes to the apparent diffusion tensor of in vivo rat brain measured with an oscillating-gradient spin-echo sequence
NEUROIMAGE
2013; 70: 10-20
Abstract
As the oscillating gradient spin-echo sequence has shown promise as a means to probe tissue microstructure, it was applied here to diffusion-tensor imaging of in vivo rat brain. The apparent diffusion tensor (ADT) was estimated for motion-probing gradient (MPG) frequencies in the range 33.3-133.3 Hz, and regions-of-interest (ROIs) in the corpus callosum (CC), visual cortex (VC), cerebellar white matter (CBWM) and cerebellar grey matter (CBGM) were selected for detailed analysis. There were substantial, approximately linear changes to the ADT with increasing MPG frequency for all four ROIs. All ROIs showed clear increases in mean diffusivity. CBWM had a substantial decrease in fractional anisotropy, whereas the CC and VC had minor increases of the same parameter. All eigenvalues of the ADT tended to increase with frequency for the CBWM, CBGM and VC, but only the principal eigenvalue increased strongly for the CC. On the other hand, there was no evidence that the orientation of the principal eigenvector varied systematically with MPG frequency for any of the ROIs. The relationship between the behaviour of the eigenvalues and the behaviours of the mean diffusivity and fractional anisotropy is investigated in detail. Pixelwise linear fits to the MD from individual animals found elevated changes across the cerebellum. The data acquired for this work encompassed a range of effective diffusion-times from 7.5 ms down to 1.875 ms, and some ideas on how the results might be used to extract quantitative information about brain tissue microstructure are discussed.
View details for DOI 10.1016/j.neuroimage.2012.12.036
View details for Web of Science ID 000315703800002
View details for PubMedID 23274188
-
Quantitative measurement of changes in calcium channel activity in vivo utilizing dynamic manganese-enhanced MRI (dMEMRI)
NEUROIMAGE
2012; 60 (1): 392-399
Abstract
The ability of manganese ions (Mn(2+)) to enter cells through calcium ion (Ca(2+)) channels has been used for depolarization dependent brain functional imaging with manganese-enhanced MRI (MEMRI). The purpose of this study was to quantify changes to Mn(2+) uptake in rat brain using a dynamic manganese-enhanced MRI (dMEMRI) scanning protocol with the Patlak and Logan graphical analysis methods. The graphical analysis was based on a three-compartment model describing the tissue and plasma concentration of Mn. Mn(2+) uptake was characterized by the total distribution volume of manganese (Mn) inside tissue (V(T)) and the unidirectional influx constant of Mn(2+) from plasma to tissue (K(i)). The measurements were performed on the anterior (APit) and posterior (PPit) parts of the pituitary gland, a region with an incomplete blood brain barrier. Modulation of Ca(2+) channel activity was performed by administration of the stimulant glutamate and the inhibitor verapamil. It was found that the APit and PPit showed different Mn(2+) uptake characteristics. While the influx of Mn(2+) into the PPit was reversible, Mn(2+) was found to be irreversibly trapped in the APit during the course of the experiment. In the PPit, an increase of Mn(2+) uptake led to an increase in V(T) (from 2.8±0.3 ml/cm(3) to 4.6±1.2 ml/cm(3)) while a decrease of Mn(2+) uptake corresponded to a decrease in V(T) (from 2.8±0.3 ml/cm(3) to 1.4±0.3 ml/cm(3)). In the APit, an increase of Mn(2+) uptake led to an increase in K(i) (from 0.034±0.009 min(-1) to 0.049±0.012 min(-1)) while a decrease of Mn(2+) uptake corresponded to a decrease in K(i) (from 0.034±0.009 min(-1) to 0.019±0.003 min(-1)). This work demonstrates that graphical analysis applied to dMEMRI data can quantitatively measure changes to Mn(2+) uptake following modulation of neural activity.
View details for DOI 10.1016/j.neuroimage.2011.12.030
View details for Web of Science ID 000301218700040
View details for PubMedID 22227885
- Marker-less co-registration of MRI data to a subject’s head via a mixed reality device 26th Annual meeting of the International Society for Magnetic Resonance in Medicine 2018
- Holographic Visualization of Brain MRI with Real-Time Alignment to a Human Subject 25th Annual meeting of the International Society for Magnetic Resonance in Medicine 2017