Zi Yang
Postdoctoral Medical Fellow, Radiation Physics
Fellow in Radiation Oncology
All Publications
-
Deep learning-based overall survival prediction in patients with glioblastoma: An automatic end-to-end workflow using pre-resection basic structural multiparametric MRIs.
Computers in biology and medicine
2024; 185: 109436
Abstract
Accurate and automated early survival prediction is critical for patients with glioblastoma (GBM) as their poor prognosis requires timely treatment decision-making. To address this need, we developed a deep learning (DL)-based end-to-end workflow for GBM overall survival (OS) prediction using pre-resection basic structural multiparametric magnetic resonance images (Bas-mpMRI) with a multi-institutional public dataset and evaluated it with an independent dataset of patients on a prospective institutional clinical trial.The proposed end-to-end workflow includes a skull-stripping model, a GBM sub-region segmentation model and an ensemble learning-based OS prediction model. The segmentation model utilizes skull-stripped Bas-mpMRIs to segment three GBM sub-regions. The segmented GBM is fed into the contrastive learning-based OS prediction model to classify the patients into different survival groups. Our datasets include both a multi-institutional public dataset from Medical Image Computing and Computer Assisted Intervention (MICCAI) Brain Tumor Segmentation (BraTS) challenge 2020 with 235 patients, and an institutional dataset from a 5-fraction SRS clinical trial with 19 GBM patients. Each data entry consists of pre-operative Bas-mpMRIs, survival days and patient ages. Basic clinical characteristics are also available for SRS clinical trial data. The multi-institutional public dataset was used for workflow establishing (90% of data) and initial validation (10% of data). The validated workflow was then evaluated on the institutional clinical trial data.Our proposed OS prediction workflow achieved an area under the curve (AUC) of 0.86 on the public dataset and 0.72 on the institutional clinical trial dataset to classify patients into 2 OS classes as long-survivors (>12 months) and short-survivors (<12 months), despite the large variation in Bas-mpMRI protocols. In addition, as part of the intermediate results, the proposed workflow can also provide detailed GBM sub-regions auto-segmentation with a whole tumor Dice score of 0.91.Our study demonstrates the feasibility of employing this DL-based end-to-end workflow to predict the OS of patients with GBM using only the pre-resection Bas-mpMRIs. This DL-based workflow can be potentially applied to assist timely clinical decision-making.
View details for DOI 10.1016/j.compbiomed.2024.109436
View details for PubMedID 39637462
-
Exploring deep learning for estimating the isoeffective dose of FLASH irradiation from mouse intestinal histology images.
International journal of radiation oncology, biology, physics
2024
Abstract
Ultra-high dose rate (FLASH) irradiation has been reported to reduce normal tissue damage compared with conventional dose rate (CONV) irradiation without compromising tumor control. This proof-of-concept study aims to develop a deep learning (DL) approach to quantify the FLASH isoeffective dose (dose of CONV that would be required to produce the same effect as the given physical FLASH dose) with post-irradiation mouse intestinal histological images.84 healthy C57BL/6J female mice underwent 16 MeV electron CONV (0.12Gy/s; n=41) or FLASH (200Gy/s; n=43) single fraction whole abdominal irradiation. Physical dose ranged from 12 to 16Gy for FLASH and 11 to 15Gy for CONV in 1Gy increments. 4 days after irradiation, 9 jejunum cross-sections from each mouse were H&E stained and digitized for histological analysis. CONV dataset was randomly split into training (n=33) and testing (n=8) datasets. ResNet101-based DL models were retrained using the CONV training dataset to estimate the dose based on histological features. The classical manual crypt counting (CC) approach was implemented for model comparison. Cross-section-wise mean squared error (CS-MSE) was computed to evaluate the dose estimation accuracy of both approaches. The validated DL model was applied to the FLASH dataset to map the physical FLASH dose into the isoeffective dose.The DL model achieved a CS-MSE of 0.20Gy2 on the CONV testing dataset compared with 0.40Gy2 of the CC approach. Isoeffective doses estimated by the DL model for FLASH doses of 12, 13, 14, 15, and 16 Gy were 12.19±0.46, 12.54±0.37, 12.69±0.26, 12.84±0.26, and 13.03±0.28 Gy, respectively.Our proposed DL model achieved accurate CONV dose estimation. The DL model results indicate that in the physical dose range of 13 to 16 Gy, the biological dose response of small intestinal tissue to FLASH irradiation is represented by a lower isoeffective dose compared to the physical dose. Our DL approach can be a tool for studying isoeffective doses of other radiation dose modifying interventions.
View details for DOI 10.1016/j.ijrobp.2023.12.032
View details for PubMedID 38171387
-
Leveraging global binary masks for structure segmentation in medical images.
Physics in medicine and biology
2023
Abstract
Deep learning (DL) models for medical image segmentation are highly influenced by intensity variations of input images and lack generalization due to primarily utilizing pixels' intensity information for inference. Acquiring sufficient training data is another challenge limiting models' applications. Here, we proposed to leverage the consistency of organs' anatomical position and shape information in medical images. We introduced a framework leveraging recurring anatomical patterns through global binary masks for organ segmentation. Two scenarios were studied: (1) Global binary masks were the only input for the U-Net based model, forcing exclusively encoding organs' position and shape information for rough segmentation or localization. (2) Global binary masks were incorporated as an additional channel providing position/shape clues to mitigate training data scarcity. Two datasets of the brain and heart CT images with their ground-truth were split into (26:10:10) and (12:3:5) for training, validation, and test respectively. The two scenarios were evaluated using full training split as well as reduced subsets of training data. In scenario (1), training exclusively on global binary masks led to Dice scores of 0.77±0.06 and 0.85±0.04 for the brain and heart structures respectively. Average Euclidian distance of 3.12±1.43mm and 2.5±0.93mm were obtained relative to the center of mass of the ground truth for the brain and heart structures respectively. The outcomes indicated encoding a surprising degree of position and shape information through global binary masks. In scenario (2), incorporating global binary masks led to significantly higher accuracy relative to the model trained on only CT images in small subsets of training data; the performance improved by 4.3-125.3% and 1.3-48.1% for 1-8 training cases of the brain and heart datasets respectively. The findings imply the advantages of utilizing global binary masks for building models that are robust to image intensity variations as well as an effective approach to boost performance when access to labeled training data is highly limited.
View details for DOI 10.1088/1361-6560/acf2e2
View details for PubMedID 37607564
-
A deep learning approach for automatic delineation of clinical target volume in stereotactic partial breast irradiation (S-PBI).
Physics in medicine and biology
2023
Abstract
Accurate and efficient delineation of the clinical target volume (CTV) is of utmost significance in post-operative breast cancer radiotherapy. However, CTV delineation is challenging as the exact extent of microscopic disease encompassed by CTV is not visualizable in radiological images and remains uncertain. We proposed to mimic physicians' contouring practice for CTV segmentation in Stereotactic Partial Breast Irradiation (S-PBI) where CTV is derived from tumor bed volume (TBV) via a margin expansion followed by correcting the extensions for anatomical barriers of tumor invasion (e.g., skin, chest wall). We proposed a deep-learning model, where CT images and the corresponding TBV masks formed a multi-channel input for a 3D U-Net based architecture. The design guided the model to encode the location-related image features and directed the network to focus on TBV to initiate CTV segmentation. Gradient weighted Class Activation Map (Grad-CAM) visualizations of the model predictions revealed that the extension rules and geometric/anatomical boundaries were learnt during model training to assist the network to limit the expansion to a certain distance from the chest wall and the skin. We retrospectively collected 175 prone CT images from 35 post-operative breast cancer patients who received 5-fraction partial breast irradiation (PBI) regimen on GammaPod. The 35 patients were randomly split into training (25), validation (5) and test (5) sets. Our model achieved mean (standard deviation) of 0.94 (0.02), 2.46 (0.5) mm, and 0.53 (0.14) mm for Dice similarity coefficient, 95th percentile Hausdorff distance, and average symmetric surface distance respectively on the test set. The results are promising for improving the efficiency and accuracy of CTV delineation process during on-line treatment planning procedure.
View details for DOI 10.1088/1361-6560/accf5e
View details for PubMedID 37084739
-
Ensemble learning for glioma patients overall survival prediction using pre-operative MRIs.
Physics in medicine and biology
2022
Abstract
OBJECTIVE: Gliomas are the most common primary brain tumors. And approximately 70% of the glioma patients, whom diagnosed with glioblastoma, have an averaged overall survival (OS) of only ~16 months. Early survival prediction is essential for treatment decision-making in glioma patients. Here we proposed an ensemble learning approach to predict the post-operative OS of glioma patients using only pre-operative MRIs.APPROACH: Our dataset was from the Medical Image Computing and Computer Assisted Intervention (MICCAI) Brain Tumor Segmentation (BraTS) challenge 2020, which consists of multimodal pre-operative MRI scans of 235 glioma patients with survival days recorded. The backbone of our approach was a Siamese network consisting of twinned ResNet-based feature extractors followed by a 3-layer classifier. During training, the feature extractors explored traits of intra and inter-class by minimizing contrastive loss of randomly paired 2D pre-operative MRIs, and the classifier utilized the extracted features to generate labels with cost defined by cross-entropy loss. During testing, the extracted features were also utilized to define distance between the test sample and the reference composed of training data, to generate an additional predictor via K-NN classification. The final label was the ensemble classification from both the Siamese model and the K-NN model.MAIN RESULTS: Our method classify the glioma patients into 3 OS classes: long-survivors (>15 months), mid-survivors (between 10 and 15 months) and short-survivors (< 10 months). The performance is assessed by the accuracy (ACC) and the area under the curve (AUC) of 3-class classification. And the final result achieved an ACC of 65.22% and AUC of 0.81.SIGNIFICANCE: Our Siamese network based ensemble learning approach demonstrated promising ability in mining discriminative features with minimal manual processing and generalization requirement. This prediction strategy can be potentially applied to assist timelier clinical decision-making.
View details for DOI 10.1088/1361-6560/aca375
View details for PubMedID 36384039
-
Deep-learning and radiomics ensemble classifier for false positive reduction in brain metastases segmentation.
Physics in medicine and biology
1800
Abstract
Stereotactic radiosurgery (SRS) is now the standard of care for brain metastases (BMs) patients. The SRS treatment planning process requires precise target delineation, which in clinical workflow for patients with multiple (>4) BMs (mBMs) could become a pronounced time bottleneck. Our group has developed an automated BMs segmentation platform to assist in this process. The accuracy of the auto-segmentation, however, is influenced by the presence of false-positive segmentations, mainly caused by the injected contrast during MRI acquisition. To address this problem and further improve the segmentation performance, a deep-learning and radiomics ensemble classifier was developed to reduce the false-positive rate in segmentations. The proposed model consists of a Siamese network and a radiomic-based support vector machine (SVM) classifier. The 2D-based Siamese network contains a pair of parallel feature extractors with shared weights followed by a single classifier. This architecture is designed to identify the inter-class difference. On the other hand, the SVM model takes the radiomic features extracted from 3D segmentation volumes as the input for twofold classification, either a false-positive segmentation or a true BM. Lastly, the outputs from both models create an ensemble to generate the final label. The performance of the proposed model in the segmented mBMs testing dataset reached the accuracy (ACC), sensitivity (SEN), specificity (SPE) and area under the curve (AUC) of 0.91, 0.96, 0.90 and 0.93, respectively. After integrating the proposed model into the original segmentation platform, the average segmentation false negative rate (FNR) and the false positive over the union (FPoU) were 0.13 and 0.09, respectively, which preserved the initial FNR (0.07) and significantly improved the FPoU (0.55). The proposed method effectively reduced the false-positive rate in the BMs raw segmentations indicating that the integration of the proposed ensemble classifier into the BMs segmentation platform provides a beneficial tool for mBMs SRS management.
View details for DOI 10.1088/1361-6560/ac4667
View details for PubMedID 34952535
-
Dose kernel decomposition for spot-based radiotherapy treatment planning.
Medical physics
1800
Abstract
PURPOSE: Pre-calculation of accurate dose deposition kernels for treatment planning of spot-based radiotherapy, such as Gamma Knife (GK) and Gamma Pod (GP), can be very time consuming and may require large data storage with an enormous number of possible spots. We proposed a novel kernel decomposition (KD) model to address accurate and fast (real time) dose calculation with reduced data storage requirement for spot-based treatment planning. The application of KD model was demonstrated for clinical GK and GP radiotherapy platforms.METHODS: The dose deposition kernel at each spot (shot position) is modeled as the product of a shift-invariant kernel based on a reference kernel and spatially variant scale factor. The reference kernel, one for each collimator, is defined at the center of the commissioning phantom for GK and at the center of the treatment target for GP and calculated using the Monte Carlo (MC) method. The spatially variant scale factor is defined as the ratio of the mean tissue maximum ratio (TMR) at the candidate shot position to that at the reference kernel position; and the mean TMR map is calculated within the entire volume through parallel-beam ray tracing on the density image followed by averaging over all source directions. The proposed KD dose calculations were compared with the MC method and with the GK and GP treatment planning system (TPS) computations for various shot positions and collimator sizes utilizing a phantom and fourteen and twelve clinical plans for GK and GP, respectively.RESULTS: For the phantom study, the KD Gamma index (3%/1 mm) passing rates were greater than 99% (median 100%) relative to the MC doses, except for the shots close to the boundary. The passing rates dropped below 90% for 8 mm (16 mm) shots positioned within 1 cm (2 cm) of the boundary. For the clinical GK plans, the KD Gamma passing rates were greater than 99% (median 100%) compared to the MC and greater than 92% (median 99%) compared to the TPS. For the clinical GP plans, the KD Gamma passing rates were greater than 95% (median 98%) compared to the MC and greater than 91% (median 97%) compared to the TPS. The scale factors were calculated in sub-seconds with GPU implementation and only need to be calculated once before treatment plan optimization. The calculation of the dose kernel was also within sub-seconds without requiring beam-by-beam calculation commonly done in the TPS.CONCLUSION: The proposed model can provide accurate dose and enables real-time dose and derivative calculations by kernel shifting and scaling without pre-calculating or requiring large data storage for GK and GP dose deposition kernels during treatment planning. This model could be useful for spot-based radiotherapy treatment planning by allowing an efficient global fine search for optimal spots. This article is protected by copyright. All rights reserved.
View details for DOI 10.1002/mp.15415
View details for PubMedID 34932827
-
Robustness study of noisy annotation in deep learning based medical image segmentation
PHYSICS IN MEDICINE AND BIOLOGY
2020; 65 (17): 175007
Abstract
Partly due to the use of exhaustive-annotated data, deep networks have achieved impressive performance on medical image segmentation. Medical imaging data paired with noisy annotation are, however, ubiquitous, but little is known about the effect of noisy annotation on deep learning based medical image segmentation. We studied the effect of noisy annotation in the context of mandible segmentation from CT images. First, 202 images of head and neck cancer patients were collected from our clinical database, where the organs-at-risk were annotated by one of twelve planning dosimetrists. The mandibles were roughly annotated as the planning avoiding structure. Then, mandible labels were checked and corrected by a head and neck specialist to get the reference standard. At last, by varying the ratios of noisy labels in the training set, deep networks were trained and tested for mandible segmentation. The trained models were further tested on other two public datasets. Experimental results indicated that the network trained with noisy labels had worse segmentation than that trained with reference standard, and in general, fewer noisy labels led to better performance. When using 20% or less noisy cases for training, no significant difference was found on the segmentation results between the models trained by noisy or reference annotation. Cross-dataset validation results verified that the models trained with noisy data achieved competitive performance to that trained with reference standard. This study suggests that the involved network is robust to noisy annotation to some extent in mandible segmentation from CT images. It also highlights the importance of labeling quality in deep learning. In the future work, extra attention should be paid to how to utilize a small number of reference standard samples to improve the performance of deep learning with noisy annotation.
View details for DOI 10.1088/1361-6560/ab99e5
View details for Web of Science ID 000565759600001
View details for PubMedID 32503027
View details for PubMedCentralID PMC7567130
-
A web-based brain metastases segmentation and labeling platform for stereotactic radiosurgery
MEDICAL PHYSICS
2020; 47 (8): 3263–76
Abstract
Stereotactic radiosurgery (SRS) has become a standard of care for patients' with brain metastases (BMs). However, the manual multiple BMs delineation can be time-consuming and could create an efficiency bottleneck in SRS workflow. There is a clinical need for automatic delineation and quantitative evaluation tools. In this study, building on our previous developed deep learning-based segmentation algorithms, we developed a web-based automated BMs segmentation and labeling platform to assist the SRS clinical workflow.This platform was developed based on the Django framework, including a web client and a back-end server. The web client enables interactions as database access, data import, and image viewing. The server performs the segmentation and labeling tasks including: skull stripping; deep learning-based BMs segmentation; and affine registration-based BMs labeling. Additionally, the client can display BMs contours with corresponding atlas labels, and allows further postprocessing tasks including: (a) adjusting window levels; (b) displaying/hiding specific contours; (c) removing false-positive contours; (d) exporting contours as DICOM RTStruct files; etc. RESULTS: We evaluated this platform on 10 clinical cases with BMs number varied from 12-81 per case. The overall operation took about 4-5 min per patient. The segmentation accuracy was evaluated between the manual contour and automatic segmentation with several metrics. The averaged center of mass shift was 1.55 ± 0.36 mm, the Hausdorff distance was 2.98 ± 0.63 mm, the mean of surface-to-surface distance (SSD) was 1.06 ± 0.31 mm, and the standard deviation of SSD was 0.80 ± 0.16 mm. In addition, the initial averaged false-positive over union (FPoU) and false-negative rate (FNR) were 0.43 ± 0.19 and 0.15 ± 0.10 respectively. After case-specific postprocessing, the averaged FPoU and FNR were 0.19 ± 0.10 and 0.15 ± 0.10 respectively.The evaluated web-based BMs segmentation and labeling platform can substantially improve the clinical efficiency compared to manual contouring. This platform can be a useful tool for assisting SRS treatment planning and treatment follow-up.
View details for DOI 10.1002/mp.14201
View details for Web of Science ID 000534819700001
View details for PubMedID 32333797
View details for PubMedCentralID PMC7567132
-
Motion robust 4D-MRI sorting based on anatomic feature matching: A digital phantom simulation study.
Radiation medicine and protection
2020; 1 (1): 41-47
Abstract
Motion artifacts induced by breathing variations are common in 4D-MRI images. This study aims to reduce the motion artifacts by developing a novel, robust 4D-MRI sorting method based on anatomic feature matching and applicable in both cine and sequential acquisition.The proposed method uses the diaphragm as the anatomic feature to guide the sorting of 4D-MRI images. Initially, both abdominal 2D sagittal cine MRI images and axial MRI images were acquired. The sagittal cine MRI images were divided into 10 phases as ground truth. Next, the phase of each axial MRI image is determined by matching its diaphragm position in the intersection plane to the ground truth cine MRI. Then, those matched phases axial images were sorted into 10-phase bins which were identical to the ground truth cine images. Finally, 10-phase 4D-MRI were reconstructed from these sorted axial images. The accuracy of reconstructed 4D-MRI data was evaluated by comparing with the ground truth using the 4D extended Cardiac Torso (XCAT) digital phantom. The effects of breathing signal, including both regular (cosine function) and irregular (patient data) in both axial cine and sequential scanning modes, on reconstruction accuracy were investigated by calculating total relative error (TRE) of the 4D volumes, Volume-Percent-Difference (VPD) and Center-of-Mass-Shift (COMS) of the estimated tumor volume, compared with the ground truth XCAT images.In both scanning modes, reconstructed 4D-MRI images matched well with ground truth with minimal motion artifacts. The averaged TRE of the 4D volume, VPD and COMS of the EOE phase in both scanning modes are 0.32%/1.20%/±0.05 mm for regular breathing, and 1.13%/4.26%/±0.21 mm for patient irregular breathing.The preliminary evaluation results illustrated the feasibility of the robust 4D-MRI sorting method based on anatomic feature matching. This method provides improved image quality with reduced motion artifacts for both cine and sequential scanning modes.
View details for DOI 10.1016/j.radmp.2020.01.003
View details for PubMedID 36247384
View details for PubMedCentralID PMC9559608
-
Predicting real-time 3D deformation field maps (DFM) based on volumetric cine MRI (VC-MRI) and artificial neural networks for on-board 4D target tracking: a feasibility study
PHYSICS IN MEDICINE AND BIOLOGY
2019; 64 (16): 165016
Abstract
To predict real-time 3D deformation field maps (DFMs) using Volumetric Cine MRI (VC-MRI) and adaptive boosting and multi-layer perceptron neural network (ADMLP-NN) for 4D target tracking. One phase of a prior 4D-MRI is set as the prior phase, MRIprior. Principal component analysis (PCA) is used to extract three major respiratory deformation modes from the DFMs generated between the prior and remaining phases. VC-MRI at each time-step is considered a deformation of MRIprior, where the DFM is represented as a weighted linear combination of the PCA components. The PCA weightings are solved by minimizing the differences between on-board 2D cine MRI and its corresponding VC-MRI slice. The PCA weightings solved during the initial training period are used to train an ADMLP-NN to predict PCA weightings ahead of time during the prediction period. The predicted PCA weightings are used to build predicted 3D DFM and ultimately, predicted VC-MRIs for 4D target tracking. The method was evaluated using a 4D computerized phantom (XCAT) with patient breathing curves and MRI data from a real liver cancer patient. Effects of breathing amplitude change and ADMLP-NN parameter variations were assessed. The accuracy of the PCA curve prediction was evaluated. The predicted real-time 3D tumor was evaluated against the ground-truth using volume dice coefficient (VDC), center-of-mass-shift (COMS), and target tracking errors. For the XCAT study, the average VDC and COMS for the predicted tumor were 0.92 ± 0.02 and 1.06 ± 0.40 mm, respectively, across all predicted time-steps. The correlation coefficients between predicted and actual PCA curves generated through VC-MRI estimation for the 1st/2nd principal components were 0.98/0.89 and 0.99/0.57 in the SI and AP directions, respectively. The optimal number of input neurons, hidden neurons, and MLP-NN for ADMLP-NN PCA weighting coefficient prediction were determined to be 7, 4, and 10, respectively. The optimal cost function threshold was determined to be 0.05. PCA weighting coefficient and VC-MRI accuracy was reduced for increased prediction-step size. Accurate PCA weighting coefficient prediction correlated with accurate VC-MRI prediction. For the patient study, the predicted 4D tumor tracking errors in superior-inferior, anterior-posterior and lateral directions were 0.50 ± 0.47 mm, 0.40 ± 0.55 mm, and 0.28 ± 0.12 mm, respectively. Preliminary studies demonstrated the feasibility to use VC-MRI and artificial neural networks to predict real-time 3D DFMs of the tumor for 4D target tracking.
View details for DOI 10.1088/1361-6560/ab359a
View details for Web of Science ID 000482525700007
View details for PubMedID 31344693
View details for PubMedCentralID PMC6734921
-
The influence of unilateral contraction of hand muscles on the contralateral corticomuscular coherence during bimanual motor tasks
NEUROPSYCHOLOGIA
2016; 85: 199-207
Abstract
The mechanisms behind how muscle contractions in one hand influence corticomuscular coherence in the opposite hand are still undetermined. Twenty-two subjects were recruited to finish bimanual and unimanual motor tasks. In the unimanual tasks, subjects performed precision grip using their right hand with visual feedback of exerted forces. The bimanual tasks involved simultaneous finger abduction of their left hand with visual feedback and precision grip of their right hand. They were divided into four conditions according to the two contraction levels of the left-hand muscles and whether visual feedback existed for the right hand. Measures of coherence and power spectrum were calculated from EEG and EMG data and statistically analyzed to identify changes in corticomuscular coupling and oscillatory activity. Results showed that compared with the unimanual task, a significant increase in the mean corticomuscular coherence of the right hand was found when left-hand muscles contracted at 5% of the maximal isometric voluntary contraction (MVC). No significant changes were found when the contraction level was 50% of the MVC. Furthermore, both the increase of muscle contraction levels and the elimination of visual feedback for right hand can significantly decrease the corticomuscular coupling in right hand during bimanual tasks. In summary, the involvement of moderate left-hand muscle contractions resulted in an increase tendency of corticomuscular coherence in right hand while strong left-hand muscle contractions eliminated it. We speculated that the perturbation of activities in one corticospinal tract resulted from the movement of the opposite hand can enhance the corticomuscular coupling when attention distraction is limited.
View details for DOI 10.1016/j.neuropsychologia.2016.03.028
View details for Web of Science ID 000376546400021
View details for PubMedID 27018484