Md Tauhidul Islam
Assistant Professor of Radiation Oncology (Radiation Physics)
Radiation Oncology - Radiation Physics
Academic Appointments
-
Assistant Professor, Radiation Oncology - Radiation Physics
-
Member, Wu Tsai Neurosciences Institute
Research Interests
-
Data Sciences
Current Research and Scholarly Interests
My research program focuses on developing computationally efficient and clinically reliable artificial intelligence (AI) methods for biomedical imaging and high-dimensional molecular data, with the long-term goal of enabling precision diagnosis, prognosis, and treatment selection in cancer and neurological disease. Modern clinical and translational datasets—including radiology, pathology, ultrasound, cfRNA/ctDNA, and single-cell sequencing—are high-dimensional and heterogeneous, but are often limited in sample size, noisy, and affected by distribution shifts across cohorts and institutions. These challenges create a major barrier to translating deep learning advances into real clinical workflows.
The Islam Lab develops novel AI foundations that improve performance, interpretability, and robustness in data-constrained biomedical settings. A central theme of our work is designing representations and learning frameworks that exploit latent structure in complex biomedical data, rather than relying solely on scaling dataset size or model complexity. We develop methods for spatializing and reorganizing tabular omics features into semantically meaningful image-like representations, enabling convolutional architectures to learn feature neighborhoods and interactions more effectively. In parallel, we develop multi-modal learning strategies that integrate imaging and molecular measurements to improve clinical prediction and biological insight. These frameworks are designed to be computationally efficient and scalable, supporting real-world use in settings where GPU time, labeled data, and prospective cohorts are limited.
Another major direction of the lab is trustworthy and interpretable AI for clinical translation. In medical imaging and liquid biopsy, high accuracy alone is not sufficient; models must provide reliable uncertainty estimates, interpretable feature attribution, and stability across demographic and institutional shifts. We therefore integrate rigorous validation, null-controlled statistical testing, and interpretable learning objectives into our pipelines. Across projects, we emphasize reproducibility and open science through the release of codebases, benchmarking datasets, and deployable toolkits that support community adoption.
Our work is highly interdisciplinary and collaborative, integrating expertise in radiation oncology, medical physics, imaging science, genomics, and machine learning. By combining representation learning, multi-modal modeling, and clinically grounded evaluation, the Islam Lab aims to advance the scientific foundations of biomedical AI and accelerate the translation of AI technologies into patient-centered clinical impact.
2025-26 Courses
- AI and Data Driven Methods in Biomedical Imaging and Physics
BMP 254, RADO 254 (Spr) -
Prior Year Courses
2023-24 Courses
Stanford Advisees
-
Postdoctoral Faculty Sponsor
Sakib Mostafa -
Postdoctoral Research Mentor
Seraphina Shi
All Publications
-
Noninvasive imaging-based assessment of tumor-associated neutrophils for prognosis and immunotherapy response in gastric cancer: a multicenter study.
Journal of advanced research
2026
Abstract
Tumor-associated neutrophils (TAN) critically promote gastric cancer progression. However, current assessment relies on invasive biopsies that preclude serial monitoring. Noninvasive tools to quantify TAN infiltration are urgently required.To develop and validate a noninvasive, CT-based ensemble machine learning radiomic biomarker for mapping TAN infiltration in gastric cancer, and to assess its utility for prognosis stratification and the prediction of response to anti-PD-1 immunotherapy.In this multicenter study of 2,170 gastric cancer patients across eight cohorts, we developed EnmlbaRB, an ensemble machine-learning-based CT radiomic biomarker. Portal venous-phase scans were processed to extract features, with mRMR-Boruta algorithms identifying 11 radiomic signatures (six peritumoral and five intratumoral signatures). These were integrated via a five-tier heterogeneous stacking architecture supervised by the immunohistochemistry-derived CD66b + TAN status (high/medium/low). The validation spanned six independent cohorts, including 177 anti-PD-1-treated patients.External validation demonstrated robust performance: EnmlbaRB predicted TAN status with an AUC of 0.71 (95%CI: 0.65-0.78) and 80.74% specificity. Critically, TAN-Low patients exhibited significantly superior 5-year overall survival compared to TAN-High across all cohorts (e.g., SYSUCC cohort: 64.12% vs. 46.78%, p < 0.05). In the anti-PD-1 cohorts, the TAN-Low subgroups achieved 1.9-fold higher disease control rates (83.9% vs 44.1%; p < 0.001) and significantly prolonged median progression-free survival (>41.9 vs 6.2 months; HR = 0.162, p < 0.001), establishing clear clinical utility for immunotherapy stratification.This study is the first clinically validated noninvasive solution for mapping the TAN infiltration status in gastric cancer. EnmlbaRB effectively stratified the patients based on survival outcomes and immunotherapy responsiveness. This paradigm empowers clinicians to personalize therapeutic sequencing based on evolving TAN biology, thereby addressing the critical need for adaptive treatment strategies for advanced gastric cancer management.
View details for DOI 10.1016/j.jare.2026.01.082
View details for PubMedID 41633486
-
Deep radiomics for prognostic prediction in locally advanced non-small cell lung cancer by leveraging OmicsMap-based image representation.
Physics in medicine and biology
2026
Abstract
OBJECTIVE: Patients with locally advanced non-small cell lung cancer (LA-NSCLC) exhibit heterogeneous prognoses despite receiving standard treatments, highlighting the need for more reliable prognostic biomarkers. This study aims to develop and validate OmicsMap model, a deep radiomics biomarkers derived from computed tomography (CT) images for the prediction of progression-free survival (PFS) in LA-NSCLC patients.Approach: We retrospectively analyzed data from 329 LA-NSCLC patients who underwent definitive radiotherapy. The cohort was randomly divided into development (N=220) and independent testing set (N=109). The prognostic signature was derived from integrated radiomics features extracted from both the primary tumor and involved lymph nodes, and inter-patient radiomics feature interactions. To achieve this, high-dimensional radiomics data from all patients were transformed into structured 2D representations, termed OmicsMap, wherein radiomics feature interactions were encoded within the pixelated configuration. Deep radiomics features from the OmicsMaps were then extracted using a convolutional neural network for prognostic prediction. Model performance was evaluated by time-dependent area under the receiver operating characteristic curves (AUC). Kaplan-Meier (KM) curves were plotted and Hazard ratios (HR) were calculated via Cox proportional hazards model.Main results: The OmicsMap model achieved time-dependent AUCs of 0.76, 0.78 and 0.76 at 1, 2 and 3 years in the independent testing set, significantly outperforming the clinical model (AUC: 0.57, 0.57, 0.64; p < 0.05). The proposed model improved predictive discrimination with 7.69% increase in C-index over conventional radiomics approaches. It effectively stratified patients into high-risk and low-risk subgroups for both PFS (p < 0.001, HR = 0.380) and OS (p = 0.0021, HR = 0.525) in the testing set. Significance: The proposed OmicsMap model provides a novel paradigm for enhancing prognostic prediction in patients with LA-NSCLC. By improving risk stratification, the framework may help inform clinical decision-making and support future efforts toward more individualized management strategies. .
View details for DOI 10.1088/1361-6560/ae3b94
View details for PubMedID 41564543
-
Revealing neurocognitive and behavioral patterns through unsupervised manifold learning of dynamic brain data.
Nature computational science
2025
Abstract
Dynamic brain data are becoming increasingly accessible, providing a gateway to understanding the inner workings of the brain in living participants. However, the size and complexity of the data pose a challenge in extracting meaningful information across various data sources. Here we introduce a generalizable unsupervised deep manifold learning for exploration of neurocognitive and behavioral patterns. Unlike existing methods that extract patterns directly from the input data, the proposed brain-dynamic convolutional-network-based embedding (BCNE) captures brain-state trajectories by analyzing temporospatial correlations within the data and applying manifold learning. The results demonstrate that BCNE effectively delineates scene transitions, underscores the involvement of different brain regions in memory and narrative processing, distinguishes dynamic learning processes and identifies differences between active and passive behaviors. BCNE provides an effective tool for exploring general neuroscience inquiries or individual-specific patterns.
View details for DOI 10.1038/s43588-025-00911-9
View details for PubMedID 41345469
-
Non-Invasive Assessment of Structural and Mechanical Micro-Environmental Changes During Long Bone Regeneration Using a Multi-Modal and Multi-Parametric Ultrasound Imaging Technique in a Segmental Tibial Defect Sheep Model <i>In Vivo</i>
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2025; 72 (12): 3546-3557
Abstract
The underlying regeneration process of bony defects often exhibits multifaceted nature, which may not be completely characterized by imaging methods currently available to the clinic. In this paper, we present the first longitudinal study to use multi-modal and multi-parametric ultrasound (US) imaging to assess bone regeneration in situations of segmental defects.We derived two imaging markers from the proposed US imaging technique: the new-bone bulk volume and fibrovascular connective tissue area and computed their global and local statistics in a subject-specific manner.From a cohort of 5 sheep treated with baseline tissue engineered construct, the distance (mm) between surface reconstructions from multi-view 3-D US and CT was $0.30\pm 0.67$ (60 days post implantation) and $0.22\pm 0.43$ (120 days post implantation). From US elastography (USE), we discovered a new contrast mechanism between the soft tissue and fibrovascular connective tissue in axial normal strain elastograms and corroborated it using endpoint histology. From two sheep, we detected negative and positive correlations between the fibrovascular connective tissue area at 60 days post shell implantation and the area of bone mass that continued to form after 60 days post shell implantation.Based on our results, it is feasible to use the proposed multi-modal and multi-parametric US imaging technique to assess structural and mechanical micro-environmental changes in the sheep animal model.In the future, 3-D US and USE may become important quantitative tools for bone fracture healing diagnosis and prognosis.
View details for DOI 10.1109/TBME.2025.3572366
View details for Web of Science ID 001617899100005
View details for PubMedID 40397635
-
Computational pathology approach for assessment of prognosis and immunotherapy response in pan-gastrointestinal cancer.
Journal of translational medicine
2025; 23 (1): 1088
Abstract
Current cancer staging methods cannot accurately predict survival outcomes and therapeutic benefits in cancer patients. Digital pathomics, a rapidly evolving field, holds significant potential to revolutionize disease evaluation.This international multicenter study included 2463 patients with pan-gastrointestinal cancer from twelve cohorts of seven cancer centers, of whom 1653 patients were diagnosed with gastric cancer (GC). We proposed a deep learning pathomics signature (DLPS) by integrating information on three scales from the whole slide images (WSIs) of H&E-stained tissue, including pathomic features related to the nucleus, microenvironment, and single-cell spatial distribution. Next, we assessed the predictive accuracy of the DLPS for prognosis, chemotherapy response, and immunotherapy response.Our findings revealed that the DLPS was significantly associated with overall survival in gastric cancer (GC) and pan-gastrointestinal cancers, exhibiting good accuracy with area under the curve values ranging from 0.723 to 0.840 across GC cohorts. Multivariable Cox regression analyses indicated that the DLPS is an independent prognostic factor. The nomogram that integrated the DLPS and TNM stage showed improved performance in predicting cancer prognosis compared to that with TNM stage alone (P < 0.05). Importantly, GC patients with low-DLPS (but not those with high-DLPS) exhibited substantial benefits from adjuvant chemotherapy (P < 0.05). Furthermore, the objective response rate to anti-PD-1 immunotherapy was significantly higher in the low-DLPS group (29.6%) compared to the high-DLPS group (8.3%, P < 0.05).The DLPS has the potential to enhance prognosis assessment and identify patients who are likely to benefit from adjuvant chemotherapy and immunotherapy in pan-gastrointestinal cancers, as well as in other solid tumors.
View details for DOI 10.1186/s12967-025-06181-3
View details for PubMedID 41088294
View details for PubMedCentralID PMC12522295
-
Unveiling tissue heterogeneity through genomic interaction-encoded image representation of RNA-sequencing data.
American journal of human genetics
2025
Abstract
Genomic sequencing is essential for both biomedical research and clinical practice. While single-cell RNA sequencing (scRNA-seq) provides insights into biological processes at the cellular level, bulk RNA sequencing remains widely used for its scalability and cost-effectiveness. To explore biological heterogeneity, research efforts have been made toward inferring single-cell-like cellular compositions from bulk samples, i.e., deconvolving bulk samples into multiple cell types. However, existing deconvolution methods face two major limitations: (1) reliance on predefined gene signature matrices without accounting for inter-sample variability and (2) susceptibility to noise within biological systems. Here, we propose a cellular-component analysis (CCA) framework by leveraging a genomic-interaction-encoded image representation of RNA-seq data for substantially improved pattern discovery. The framework incorporates sample-specific gene-expression variability and derives signature patterns by utilizing a convolutional variational autoencoder and Gaussian mixture model. An image-domain linear decomposition of bulk RNA-seq data based on these sample-specific, interpretable gene-signature patterns is then performed for CCA and other downstream tasks, such as cancer subtype classification and biomarker discovery. We demonstrate that the proposed technique improves decomposition accuracy by over 14.1% in average Pearson correlation compared to existing techniques by using both simulation and experimental datasets. This approach offers an effective solution for tissue heterogeneity analysis and lays a foundation for a range of clinical and biological applications.
View details for DOI 10.1016/j.ajhg.2025.08.021
View details for PubMedID 40967222
-
Restoring mitochondrial quantity and quality to reverse the Warburg effect and drive neuroblastoma differentiation.
Proceedings of the National Academy of Sciences of the United States of America
2025; 122 (36): e2502483122
Abstract
Reduced mitochondrial quality and quantity in tumors is associated with dedifferentiation and increased malignancy. However, it remains unclear how to restore mitochondrial quantity and quality in tumors and whether mitochondrial restoration can drive tumor differentiation. Our study shows that restoring mitochondrial function using retinoic acid (RA) to boost mitochondrial biogenesis and a mitochondrial uncoupler to enhance respiration synergistically drives neuroblastoma differentiation and inhibits proliferation. U-13C-glucose/glutamine isotope tracing revealed a metabolic shift from the pentose phosphate pathway to oxidative phosphorylation, accelerating the tricarboxylic acid cycle and switching substrate preference from glutamine to glucose. These effects were abolished by electron transport chain (ETC) inhibitors or in ρ0 cells lacking mitochondrial DNA, emphasizing the necessity of mitochondrial function for differentiation. Dietary RA and uncoupler treatment promoted tumor differentiation in an orthotopic neuroblastoma xenograft model, evidenced by neuropil production and Schwann cell recruitment. Single-cell RNA sequencing of xenografts revealed that this strategy effectively eliminated the stem cell population, promoted differentiation, and increased mitochondrial gene signatures along the differentiation trajectory, potentially improving patient outcomes. Collectively, our findings establish a mitochondria-centric therapeutic strategy for inducing tumor differentiation, suggesting that maintaining/driving differentiation in tumor requires not only ATP production but also continuous ATP consumption and sustained ETC activity.
View details for DOI 10.1073/pnas.2502483122
View details for PubMedID 40911595
-
Multimodal Radiopathomics Signature for Prediction of Response to Immunotherapy-based Combination Therapy in Gastric Cancer Using Interpretable Machine Learning.
Cancer letters
2025: 217930
Abstract
Immunotherapy has become a cornerstone in the treatment of advanced gastric cancer (GC). However, identifying reliable predictive biomarkers remains a considerable challenge. This study demonstrates the potential of integrating multimodal baseline data, including computed tomography scan images and digital H&E-stained pathology images, with biological interpretation to predict the response to immunotherapy-based combination therapy using a multicenter cohort of 298 GC patients. By employing seven machine learning approaches, we developed a radiopathomics signature (RPS) to predict treatment response and stratify prognostic risk in GC. The RPS demonstrated area under the receiver-operating-characteristic curves (AUCs) of 0.978 (95% CI, 0.950-1.000), 0.863 (95% CI, 0.744-0.982), and 0.822 (95% CI, 0.668-0.975) in the training, internal validation, and external validation cohorts, respectively, outperforming conventional biomarkers such as CPS, MSI-H, EBV, and HER-2. Kaplan-Meier analysis revealed significant differences of survival between high- and low-risk groups, especially in advanced-stage and non-surgical patients. Additionally, genetic analyses revealed that the RPS correlates with enhanced immune regulation pathways and increased infiltration of memory B cells. The interpretable RPS provides accurate predictions for treatment response and prognosis in GC and holds potential for guiding more precise, patient-specific treatment strategies while offering insights into immune-related mechanisms.
View details for DOI 10.1016/j.canlet.2025.217930
View details for PubMedID 40675469
-
TME-guided deep learning predicts chemotherapy and immunotherapy response in gastric cancer with attention-enhanced residual Swin Transformer.
Cell reports. Medicine
2025: 102242
Abstract
Adjuvant chemotherapy and immune checkpoint blockade exert quite durable anti-tumor responses, but the lack of effective biomarkers limits the therapeutic benefits. Utilizing multi-cohorts of 3,095 patients with gastric cancer, we propose an attention-enhanced residual Swin Transformer network to predict chemotherapy response (main task), and two predicting subtasks (ImmunoScore and periostin [POSTN]) are used as intermediate tasks to improve the model's performance. Furthermore, we assess whether the model can identify which patients would benefit from immunotherapy. The deep learning model achieves high accuracy in predicting chemotherapy response and the tumor microenvironment (ImmunoScore and POSTN). We further find that the model can identify which patient may benefit from checkpoint blockade immunotherapy. This approach offers precise chemotherapy and immunotherapy response predictions, opening avenues for personalized treatment options. Prospective studies are warranted to validate its clinical utility.
View details for DOI 10.1016/j.xcrm.2025.102242
View details for PubMedID 40695288
-
RAFT-USENet: A Unified Network for Accurate Axial and Lateral Motion Estimation in Ultrasound Elastography Imaging
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
2025; 29 (7): 5109-5122
Abstract
High-quality motion estimation is essential in ultrasound elastography (USE) for evaluating tissue mechanical properties and detecting abnormalities. Traditional methods, such as speckle tracking and regularized optimization, face challenges including noise, oversmoothing of displacements and prolonged runtimes. Recent efforts have explored optical flow-based convolutional neural networks (CNNs). However, current approaches face, at least, one of the following limitations: 1) reliance on tissue incompressibility assumption, which compromises data fidelity and can introduce large errors; 2) dependence on ground truth displacement data for supervised CNN methods; 3) use of a regularizer not aligned with tissue physics by relying on first-order displacement derivatives only; 4) use of a L2-norm regularizer that oversmoothes motion estimates; and 5) substantially large sampling size, increasing computational and memory demands, especially for classical methods. In this paper, we develop RAFT-USENet, a physics-informed, unsupervised neural network to estimate both axial and lateral displacements. We design RAFT-USENet by substantially modifying the optical flow RAFT network to adapt it to high-frequency USE data. Extensive validation using simulation, phantom and in vivo data demonstrates that RAFT-USENet significantly outperforms existing methods, with normalized cross-correlation values of 0.94, 0.88, and 0.82 in simulation, breast phantom and in vivo datasets, respectively, compared to 0.79-0.88, 0.76-0.85, and 0.69-0.81 for the competing methods. Additionally, RAFT-USENet reduces computational time by 1.5-150 times compared to existing methods. These results suggest that RAFT-USENet may be a potentially reliable and accurate tool for clinical elasticity imaging applications.
View details for DOI 10.1109/JBHI.2025.3536786
View details for Web of Science ID 001522930100015
View details for PubMedID 40031265
-
VISTA immune checkpoint blunts radiotherapy-induced antitumor immune response.
Cell reports
2025; 44 (7): 115893
Abstract
Radiotherapy (RT) is a key treatment for solid neoplasms like head and neck cancer (HNC), but it can also activate and recruit immunosuppressive myeloid cells, causing treatment failure. In this study, we examine the role of V-domain immunoglobulin suppressor of T cell activation (VISTA) on myeloid cells during RT. We discovered high VISTA expression on myeloid cells in the tumor microenvironment (TME) of both murine and human HNC, with RT increasing VISTA+ myeloid cells in the TME and circulation. Compared to VISTA+/+ mice, VISTA-/- mice showed improved tumor control with RT, with their macrophages and neutrophils exhibiting antitumorigenic properties on sc-RNA-seq analysis, especially with RT. Combining anti-VISTA antibodies (active or silent Fc) with RT (fractionated or ablative) significantly decreased tumor volume compared to either treatment alone in multiple preclinical models (HNC, breast cancer, and colorectal cancer), enhancing systemic antitumor immune response with augmented intra-tumoral T cell function through myeloid repolarization. Targeting VISTA could improve the efficacy of RT.
View details for DOI 10.1016/j.celrep.2025.115893
View details for PubMedID 40580480
-
Tissue-origin transcriptomic interactions as indicators of inflammatory aging and aging gene discovery.
Computers in biology and medicine
2025; 195: 110534
Abstract
Aging is an inevitable process in living organisms, characterized by significant immunological and physiological alterations that increase susceptibility to diseases. Despite decades of research on the interplay between aging and immunity, identifying precise immunogenetic aging markers from high-dimensional (HD) transcriptomics data remains challenging due to the lack of effective pattern-discovery methods for such complex data. Recognizing the crucial role of gene-gene interactions in biological processes, and that aging is reflected in tissue-specific change in these interaction patterns, we first sought a holistic way to represent the intricate relationships among genes within the cells of each tissue. This is technically achieved by converting the scRNA-seq data of a cell into a semantically meaningful image representation, termed a "genomap", through the spatial encoding of the transcriptomic interactions. We then leverage this representation to unveil changes in interaction patterns associated with aging, rather than focusing solely on variations in gene expression profiles. This approach enables the construction of a highly accurate, tissue-specific inflammatory aging clock. Our clock significantly outperforms traditional aging-prediction methods based on blood tests, miRNA, and proteomics data. We illustrate a potential application of our framework in aging interventions via an in silico study targeting the Map3k1/Map2k4/JNK pathway, which may reverse aging-associated declines in B cell function and antibody production, thus mitigating chronic diseases linked to B cells deterioration. Thus, our strategy and pipeline may have broad implications for aging and developmental biology.
View details for DOI 10.1016/j.compbiomed.2025.110534
View details for PubMedID 40527241
-
Deep representation learning of protein-protein interaction networks for enhanced pattern discovery.
Science advances
2024; 10 (51): eadq4324
Abstract
Protein-protein interaction (PPI) networks, where nodes represent proteins and edges depict myriad interactions among them, are fundamental to understanding the dynamics within biological systems. Despite their pivotal role in modern biology, reliably discerning patterns from these intertwined networks remains a substantial challenge. The essence of the challenge lies in holistically characterizing the relationships of each node with others in the network and effectively using this information for accurate pattern discovery. In this work, we introduce a self-supervised network embedding framework termed discriminative network embedding (DNE). Unlike conventional methods that primarily focus on direct or limited-order node proximity, DNE characterizes a node both locally and globally by harnessing the contrast between representations from neighboring and distant nodes. Our experimental results demonstrate DNE's superior performance over existing techniques across various critical network analyses, including PPI inference and the identification of protein functional modules. DNE emerges as a robust strategy for node representation in PPI networks, offering promising avenues for diverse biomedical applications.
View details for DOI 10.1126/sciadv.adq4324
View details for PubMedID 39693438
View details for PubMedCentralID PMC11654695
-
Restoring Mitochondrial Quantity and Quality to Reverse Warburg Effect and Drive Tumor Differentiation.
Research square
2024
Abstract
Reduced mitochondrial quality and quantity in tumors is associated with dedifferentiation and increased malignancy. However, it remains unclear how to restore mitochondrial quantity and quality in tumors, and whether mitochondrial restoration can drive tumor differentiation. Our study shows that restoring mitochondrial function using retinoic acid (RA) to boost mitochondrial biogenesis and a mitochondrial uncoupler to enhance respiration synergistically drives neuroblastoma differentiation and inhibits proliferation. U-13C-glucose/glutamine isotope tracing revealed a metabolic shift from the pentose phosphate pathway to oxidative phosphorylation, accelerating the TCA cycle and switching substrate preference from glutamine to glucose. These effects were reversed by ETC inhibitors or in ρ0 cells lacking mtDNA, emphasizing the necessity of mitochondrial function for differentiation. Dietary RA and uncoupler treatment promoted tumor differentiation in an orthotopic neuroblastoma xenograft model, evidenced by neuropil production and Schwann cell recruitment. Single-cell RNA sequencing analysis of the orthotopic xenografts revealed that this strategy effectively eliminated the stem cell population, promoted differentiation, and increased mitochondrial gene signatures along the differentiation trajectory, which could potentially significantly improve patient outcomes. Collectively, our findings establish a mitochondria-centric therapeutic strategy for inducing tumor differentiation, suggesting that maintaining/driving differentiation in tumor requires not only ATP production but also continuous ATP consumption and sustained ETC activity.
View details for DOI 10.21203/rs.3.rs-5494402/v1
View details for PubMedID 39711563
View details for PubMedCentralID PMC11661309
-
Localization and recognition of human action in 3D using transformers.
Communications engineering
2024; 3 (1): 125
Abstract
Understanding a person's behavior from their 3D motion sequence is a fundamental problem in computer vision with many applications. An important component of this problem is 3D action localization, which involves recognizing what actions a person is performing, and when the actions occur in the sequence. To promote the progress of the 3D action localization community, we introduce a new, challenging, and more complex benchmark dataset, BABEL-TAL (BT), for 3D action localization. Important baselines and evaluating metrics, as well as human evaluations, are carefully established on this benchmark. We also propose a strong baseline model, i.e., Localizing Actions with Transformers (LocATe), that jointly localizes and recognizes actions in a 3D sequence. The proposed LocATe shows superior performance on BABEL-TAL as well as on the large-scale PKU-MMD dataset, achieving state-of-the-art performance by using only 10% of the labeled training data. Our research could advance the development of more accurate and efficient systems for human behavior analysis, with potential applications in areas such as human-computer interaction and healthcare.
View details for DOI 10.1038/s44172-024-00272-7
View details for PubMedID 39227676
View details for PubMedCentralID PMC11372174
-
Deciphering the Feature Representation of Deep Neural Networks for High-Performance AI
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
2024; 46 (8): 5273-5287
Abstract
AI driven by deep learning is transforming many aspects of science and technology. The enormous success of deep learning stems from its unique capability of extracting essential features from Big Data for decision-making. However, the feature extraction and hidden representations in deep neural networks (DNNs) remain inexplicable, primarily because of lack of technical tools to comprehend and interrogate the feature space data. The main hurdle here is that the feature data are often noisy in nature, complex in structure, and huge in size and dimensionality, making it intractable for existing techniques to analyze the data reliably. In this work, we develop a computational framework named contrastive feature analysis (CFA) to facilitate the exploration of the DNN feature space and improve the performance of AI. By utilizing the interaction relations among the features and incorporating a novel data-driven kernel formation strategy into the feature analysis pipeline, CFA mitigates the limitations of traditional approaches and provides an urgently needed solution for the analysis of feature space data. The technique allows feature data exploration in unsupervised, semi-supervised and supervised formats to address different needs of downstream applications. The potential of CFA and its applications for pruning of neural network architectures are demonstrated using several state-of-the-art networks and well-annotated datasets across different disciplines.
View details for DOI 10.1109/TPAMI.2024.3363642
View details for Web of Science ID 001262841000005
View details for PubMedID 38373137
-
Non-Invasive Imaging of Mechanical Properties of Cancers In Vivo Based on Transformations of the Eshelby's Tensor Using Compression Elastography.
IEEE transactions on medical imaging
2024; 43 (8): 3027-3043
Abstract
Knowledge of the mechanical properties is of great clinical significance for diagnosis, prognosis and treatment of cancers. Recently, a new method based on Eshelby's theory to simultaneously assess Young's modulus (YM) and Poisson's ratio (PR) in tissues has been proposed. A significant limitation of this method is that accuracy of the reconstructed YM and PR is affected by the orientation/alignment of the tumor with the applied stress. In this paper, we propose a new method to reconstruct YM and PR in cancers that is invariant to the 3D orientation of the tumor with respect to the axis of applied stress. The novelty of the proposed method resides on the use of a tensor transformation to improve the robustness of Eshelby's theory and reconstruct YM and PR of tumors with high accuracy and in realistic experimental conditions. The method is validated using finite element simulations and controlled experiments using phantoms with known mechanical properties. The in vivo feasibility of the developed method is demonstrated in an orthotopic mouse model of breast cancer. Our results show that the proposed technique can estimate the YM and PR with overall accuracy of (97.06 ± 2.42) % under all tested tumor orientations. Animal experimental data demonstrate the potential of the proposed methodology in vivo. The proposed method can significantly expand the range of applicability of the Eshelby's theory to tumors and provide new means to accurately image and quantify mechanical parameters of cancers in clinical conditions.
View details for DOI 10.1109/TMI.2024.3385644
View details for PubMedID 38593022
-
Development and interpretation of a pathomics-driven ensemble model for predicting the response to immunotherapy in gastric cancer.
Journal for immunotherapy of cancer
2024; 12 (5)
Abstract
Only a subset of patients with gastric cancer experience long-term benefits from immune checkpoint inhibitors (ICIs). Currently, there is a deficiency in precise predictive biomarkers for ICI efficacy. The aim of this study was to develop and validate a pathomics-driven ensemble model for predicting the response to ICIs in gastric cancer, using H&E-stained whole slide images (WSI).This multicenter study retrospectively collected and analyzed H&E-stained WSIs and clinical data from 584 patients with gastric cancer. An ensemble model, integrating four classifiers: least absolute shrinkage and selection operator, k-nearest neighbors, decision trees, and random forests, was developed and validated using pathomics features, with the objective of predicting the therapeutic efficacy of immune checkpoint inhibition. Model performance was evaluated using metrics including the area under the curve (AUC), sensitivity, and specificity. Additionally, SHAP (SHapley Additive exPlanations) analysis was used to explain the model's predicted values as the sum of the attribution values for each input feature. Pathogenomics analysis was employed to explain the molecular mechanisms underlying the model's predictions.Our pathomics-driven ensemble model effectively stratified the response to ICIs in training cohort (AUC 0.985 (95% CI 0.971 to 0.999)), which was further validated in internal validation cohort (AUC 0.921 (95% CI 0.839 to 0.999)), as well as in external validation cohort 1 (AUC 0.914 (95% CI 0.837 to 0.990)), and external validation cohort 2 (0.927 (95% CI 0.802 to 0.999)). The univariate Cox regression analysis revealed that the prediction signature of pathomics-driven ensemble model was a prognostic factor for progression-free survival in patients with gastric cancer who underwent immunotherapy (p<0.001, HR 0.35 (95% CI 0.24 to 0.50)), and remained an independent predictor after multivariable Cox regression adjusted for clinicopathological variables, (including sex, age, carcinoembryonic antigen, carbohydrate antigen 19-9, therapy regime, line of therapy, differentiation, location and programmed death ligand 1 (PD-L1) expression in all patients (p<0.001, HR 0.34 (95% CI 0.24 to 0.50)). Pathogenomics analysis suggested that the ensemble model is driven by molecular-level immune, cancer, metabolism-related pathways, and was correlated with the immune-related characteristics, including immune score, Estimation of STromal and Immune cells in MAlignant Tumor tissues using Expression data score, and tumor purity.Our pathomics-driven ensemble model exhibited high accuracy and robustness in predicting the response to ICIs using WSIs. Therefore, it could serve as a novel and valuable tool to facilitate precision immunotherapy.
View details for DOI 10.1136/jitc-2024-008927
View details for PubMedID 38749538
View details for PubMedCentralID PMC11097892
-
Bladder Cancer and Artificial Intelligence: Emerging Applications.
The Urologic clinics of North America
2024; 51 (1): 63-75
Abstract
Bladder cancer is a common and heterogeneous disease that poses a significant burden to the patient and health care system. Major unmet needs include effective early detection strategy, imprecision of risk stratification, and treatment-associated morbidities. The existing clinical paradigm is imprecise, which results in missed tumors, suboptimal therapy, and disease progression. Artificial intelligence holds immense potential to address many unmet needs in bladder cancer, including early detection, risk stratification, treatment planning, quality assessment, and outcome prediction. Despite recent advances, extensive work remains to affirm the efficacy of artificial intelligence as a decision-making tool for bladder cancer management.
View details for DOI 10.1016/j.ucl.2023.07.002
View details for PubMedID 37945103
-
Self-supervised deep learning of gene-gene interactions for improved gene expression recovery.
Briefings in bioinformatics
2024; 25 (2)
Abstract
Single-cell RNA sequencing (scRNA-seq) has emerged as a powerful tool to gain biological insights at the cellular level. However, due to technical limitations of the existing sequencing technologies, low gene expression values are often omitted, leading to inaccurate gene counts. Existing methods, including advanced deep learning techniques, struggle to reliably impute gene expressions due to a lack of mechanisms that explicitly consider the underlying biological knowledge of the system. In reality, it has long been recognized that gene-gene interactions may serve as reflective indicators of underlying biology processes, presenting discriminative signatures of the cells. A genomic data analysis framework that is capable of leveraging the underlying gene-gene interactions is thus highly desirable and could allow for more reliable identification of distinctive patterns of the genomic data through extraction and integration of intricate biological characteristics of the genomic data. Here we tackle the problem in two steps to exploit the gene-gene interactions of the system. We first reposition the genes into a 2D grid such that their spatial configuration reflects their interactive relationships. To alleviate the need for labeled ground truth gene expression datasets, a self-supervised 2D convolutional neural network is employed to extract the contextual features of the interactions from the spatially configured genes and impute the omitted values. Extensive experiments with both simulated and experimental scRNA-seq datasets are carried out to demonstrate the superior performance of the proposed strategy against the existing imputation methods.
View details for DOI 10.1093/bib/bbae031
View details for PubMedID 38349062
-
Revealing hidden patterns in deep neural network feature space continuum via manifold learning.
Nature communications
2023; 14 (1): 8506
Abstract
Deep neural networks (DNNs) extract thousands to millions of task-specific features during model training for inference and decision-making. While visualizing these features is critical for comprehending the learning process and improving the performance of the DNNs, existing visualization techniques work only for classification tasks. For regressions, the feature points lie on a high dimensional continuum having an inherently complex shape, making a meaningful visualization of the features intractable. Given that the majority of deep learning applications are regression-oriented, developing a conceptual framework and computational method to reliably visualize the regression features is of great significance. Here, we introduce a manifold discovery and analysis (MDA) method for DNN feature visualization, which involves learning the manifold topology associated with the output and target labels of a DNN. MDA leverages the acquired topological information to preserve the local geometry of the feature space manifold and provides insightful visualizations of the DNN features, highlighting the appropriateness, generalizability, and adversarial robustness of a DNN. The performance and advantages of the MDA approach compared to the existing methods are demonstrated in different deep learning applications.
View details for DOI 10.1038/s41467-023-43958-w
View details for PubMedID 38129376
View details for PubMedCentralID 8791835
-
Biology-aware mutation-based deep learning for outcome prediction of cancer immunotherapy with immune checkpoint inhibitors.
NPJ precision oncology
2023; 7 (1): 117
Abstract
The response rate of cancer immune checkpoint inhibitors (ICI) varies among patients, making it challenging to pre-determine whether a particular patient will respond to immunotherapy. While gene mutation is critical to the treatment outcome, a framework capable of explicitly incorporating biology knowledge has yet to be established. Here we aim to propose and validate a mutation-based deep learning model for survival analysis on 1571 patients treated with ICI. Our model achieves an average concordance index of 0.59±0.13 across nine types of cancer, compared to the gold standard Cox-PH model (0.52±0.10). The "black box" nature of deep learning is a major concern in healthcare field. This model's interpretability, which results from incorporating the gene pathways and protein interaction (i.e., biology-aware) rather than relying on a 'black box' approach, helps patient stratification and provides insight into novel gene biomarkers, advancing our understanding of ICI treatment.
View details for DOI 10.1038/s41698-023-00468-8
View details for PubMedID 37932419
-
Leveraging cell-cell similarity for high-performance spatial and temporal cellular mappings from gene expression data.
Patterns (New York, N.Y.)
2023; 4 (10): 100840
Abstract
Single-cell trajectory mapping and spatial reconstruction are two important developments in life science and provide a unique means to decode heterogeneous tissue formation, cellular dynamics, and tissue developmental processes. The success of these techniques depends critically on the performance of analytical tools used for high-dimensional (HD) gene expression data processing. Existing methods discern the patterns of the data without explicitly considering the underlying biological characteristics of the system, often leading to suboptimal solutions. Here, we present a cell-cell similarity-driven framework of genomic data analysis for high-fidelity spatial and temporal cellular mappings. The approach exploits the similarity features of the cells to discover discriminative patterns of the data. We show that for a wide variety of datasets, the proposed approach drastically improves the accuracies of spatial and temporal mapping analyses compared with state-of-the-art techniques.
View details for DOI 10.1016/j.patter.2023.100840
View details for PubMedID 37876896
View details for PubMedCentralID PMC10591141
-
Super-resolution biomedical imaging via reference-free statistical implicit neural representation.
Physics in medicine and biology
2023
Abstract
Supervised deep learning for image super-resolution (SR) has limitations in biomedical imaging due to the lack of large amounts of low- and high-resolution image pairs for model training. In this work, we propose a reference-free statistical implicit neural representation (INR) framework, which needs only a single or a few observed low-resolution (LR) image(s), to generate high-quality SR images. Approach. The framework models the statistics of the observed LR images via maximum likelihood estimation and trains the INR network to represent the latent high-resolution (HR) image as a continuous function in the spatial domain. The INR network is constructed as a coordinate-based multi-layer perceptron (MLP), whose inputs are image spatial coordinates and outputs are corresponding pixel intensities. The trained INR not only constrains functional smoothness but also allows an arbitrary scale in SR imaging. Main results. We demonstrate the efficacy of the proposed framework on various biomedical images, including CT, MRI, fluorescence microscopy images, and ultrasound images, across different SR magnification scales of 2×, 4×, and 8×. A limited number of LR images were used for each of the SR imaging tasks to show the potential of the proposed statistical INR framework. Significance. The proposed method provides an urgently needed unsupervised deep learning framework for numerous biomedical SR applications that lack HR reference images.
View details for DOI 10.1088/1361-6560/acfdf1
View details for PubMedID 37757838
-
Non-invasive tumor microenvironment evaluation and treatment response prediction in gastric cancer using deep learning radiomics.
Cell reports. Medicine
2023: 101146
Abstract
The tumor microenvironment (TME) plays a critical role in disease progression and is a key determinant of therapeutic response in cancer patients. Here, we propose a noninvasive approach to predict the TME status from radiological images by combining radiomics and deep learning analyses. Using multi-institution cohorts of 2,686 patients with gastric cancer, we show that the radiological model accurately predicted the TME status and is an independent prognostic factor beyond clinicopathologic variables. The model further predicts the benefit from adjuvant chemotherapy for patients with localized disease. In patients treated with checkpoint blockade immunotherapy, the model predicts clinical response and further improves predictive accuracy when combined with existing biomarkers. Our approach enables noninvasive assessment of the TME, which opens the door for longitudinal monitoring and tracking response to cancer therapy. Given the routine use of radiologic imaging in oncology, our approach can be extended to many other solid tumor types.
View details for DOI 10.1016/j.xcrm.2023.101146
View details for PubMedID 37557177
-
Assessment of compression-induced solid stress, fluid pressure and mechanopathological parameters in cancers in vivo using poroelastography.
Physics in medicine and biology
2023
Abstract
Compression-induced solid stress (SSc) and fluid pressure (FPc) during ultrasound poroelastography (USPE) experiments are correlated with two markers of cancer growth and treatment effectiveness: growth-induced solid stress (SSg) and interstitial fluid pressure (IFP). The spatio-temporal distributions of SSg and IFP are determined by the transport properties of the vessels and interstitium in the tumor microenvironment. Approach. We propose a new USPE method for the non-invasive imaging of the local cancer mechanical parameters and dynamics of fluid flow. When performing poroelastography experiments, it may be difficult to implement a typical creep compression protocol, which requires to maintain a constant normally applied force. In this paper, we investigate the use of a stress relaxation protocol, which might be a more convenient choice for clinical poroelastography applications. Main results. Based on our finite element (FE) and ultrasound (US) simulations study, we demonstrate that the SSc, FPc and their spatio-temporal distribution related parameters, interstitial permeability (IP) and vascular permeability (VP), can be determined from stress relaxation experiments with errors below 10% as compared to the ground truth and accuracy similar to that of corresponding creep tests, respectively. We also demonstrate the feasibility of the new methodology for in vivo experiments using a small animal cancer model.The proposed non-invasive USPE imaging methods may become an effective tool to assess local tumor pressure and mechanopathological parameters in cancers.
View details for DOI 10.1088/1361-6560/acdf39
View details for PubMedID 37327794
-
Non-invasive imaging of interstitial fluid transport parameters in solid tumors in vivo.
Scientific reports
2023; 13 (1): 7132
Abstract
In this paper, new and non-invasive imaging methods to assess interstitial fluid transport parameters in tumors in vivo are developed, analyzed and experimentally validated. These parameters include extracellular volume fraction (EVF), interstitial fluid volume fraction (IFVF) and interstitial hydraulic conductivity (IHC), and they are known to have a critical role in cancer progression and drug delivery effectiveness. EVF is defined as the volume of extracellular matrix per unit volume of the tumor, while IFVF refers to the volume of interstitial fluid per unit bulk volume of the tumor. There are currently no established imaging methods to assess interstitial fluid transport parameters in cancers in vivo. We develop and test new theoretical models and imaging techniques to assess fluid transport parameters in cancers using non-invasive ultrasound methods. EVF is estimated via the composite/mixture theory with the tumor being modeled as a biphasic (cellular phase and extracellular phase) composite material. IFVF is estimated by modeling the tumor as a biphasic poroelastic material with fully saturated solid phase. Finally, IHC is estimated from IFVF using the well-known Kozeny-Carman method inspired by soil mechanics theory. The proposed methods are tested using both controlled experiments and in vivo experiments on cancers. The controlled experiments were performed on tissue mimic polyacrylamide samples and validated using scanning electron microscopy (SEM). In vivo applicability of the proposed methods was demonstrated using a breast cancer model implanted in mice. Based on the controlled experimental validation, the proposed methods can estimate interstitial fluid transport parameters with an error below 10% with respect to benchmark SEM data. In vivo results demonstrate that EVF, IFVF and IHC increase in untreated tumors whereas these parameters are observed to decrease over time in treated tumors. The proposed non-invasive imaging methods may provide new and cost-effective diagnostic and prognostic tools to assess clinically relevant fluid transport parameters in cancers in vivo.
View details for DOI 10.1038/s41598-023-33651-9
View details for PubMedID 37130836
-
Learning image representations for content-based image retrieval of radiotherapy treatment plans.
Physics in medicine and biology
2023
Abstract
In this work, we propose a content-based image retrieval (CBIR) method for retrieving dose distributions of previously planned patients based on anatomical similarity. Retrieved dose distributions from this method can be incorporated into automated treatment planning workflows in order to streamline the iterative planning process. As CBIR has not yet been applied to treatment planning, our work seeks to understand which current machine learning models are most viable in this context.Our proposed CBIR method trains a representation model that produces latent space embeddings of a patient's anatomical information. The latent space embeddings of new patients are then compared against those of previous patients in a database for image retrieval of dose distributions. All source code for this project is available on github.The retrieval performance of various CBIR methods is evaluated on a dataset consisting of both publicly available image sets and clinical image sets from our institution. This study compares various encoding methods, ranging from simple autoencoders to more recent Siamese networks like SimSiam, and the best performance was observed for the multitask Siamese network.Our current results demonstrate that excellent image retrieval performance can be obtained through slight changes to previously developed Siamese networks. We hope to integrate CBIR into automated planning workflow in future works.
View details for DOI 10.1088/1361-6560/accdb0
View details for PubMedID 37068492
-
Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
2023
View details for DOI 10.1109/TNNLS.2023.3250490
View details for Web of Science ID 000947800700001
-
Cartography of Genomic Interactions Enables Deep Analysis of Single-Cell Expression Data.
Nature communications
2023; 14 (1): 679
Abstract
Remarkable advances in single cell genomics have presented unique challenges and opportunities for interrogating a wealth of biomedical inquiries. High dimensional genomic data are inherently complex because of intertwined relationships among the genes. Existing methods, including emerging deep learning-based approaches, do not consider the underlying biological characteristics during data processing, which greatly compromises the performance of data analysis and hinders the maximal utilization of state-of-the-art genomic techniques. In this work, we develop an entropy-based cartography strategy to contrive the high dimensional gene expression data into a configured image format, referred to as genomap, with explicit integration of the genomic interactions. This unique cartography casts the gene-gene interactions into the spatial configuration of genomaps and enables us to extract the deep genomic interaction features and discover underlying discriminative patterns of the data. We show that, for a wide variety of applications (cell clustering and recognition, gene signature extraction, single cell data integration, cellular trajectory analysis, dimensionality reduction, and visualization), the proposed approach drastically improves the accuracies of data analyses as compared to the state-of-the-art techniques.
View details for DOI 10.1038/s41467-023-36383-6
View details for PubMedID 36755047
-
Image classification using graph neural network and multiscale wavelet superpixels
PATTERN RECOGNITION LETTERS
2023; 166: 89-96
View details for DOI 10.1016/j.patrec.2023.01.003
View details for Web of Science ID 000924477900001
-
Bladder Cancer and Artificial Intelligence: Emerging Applications
Urologic Clinics North America
2023
View details for DOI 10.1016/j.ucl.2023.07.002
-
Flat lesion detection of white light cystoscopy with deep learning
2023
View details for DOI 10.1117/12.2650583
-
Sequential modeling for cystoscopic image classification
2023
View details for DOI 10.1117/12.2649334
-
Leveraging data-driven self-consistency for high-fidelity gene expression recovery.
Nature communications
2022; 13 (1): 7142
Abstract
Single cell RNA sequencing is a promising technique to determine the states of individual cells and classify novel cell subtypes. In current sequence data analysis, however, genes with low expressions are omitted, which leads to inaccurate gene counts and hinders downstream analysis. Recovering these omitted expression values presents a challenge because of the large size of the data. Here, we introduce a data-driven gene expression recovery framework, referred to as self-consistent expression recovery machine (SERM), to impute the missing expressions. Using a neural network, the technique first learns the underlying data distribution from a subset of the noisy data. It then recovers the overall expression data by imposing a self-consistency on the expression matrix, thus ensuring that the expression levels are similarly distributed in different parts of the matrix. We show that SERM improves the accuracy of gene imputation with orders of magnitude enhancement in computational efficiency in comparison to the state-of-the-art imputation techniques.
View details for DOI 10.1038/s41467-022-34595-w
View details for PubMedID 36414658
-
Small-Object Sensitive Segmentation Using Across Feature Map Attention.
IEEE transactions on pattern analysis and machine intelligence
2022; PP
Abstract
Semantic segmentation is an important step in understanding the scene for many practical applications such as autonomous driving. Although Deep Convolutional Neural Networks-based methods have significantly improved segmentation accuracy, small/thin objects remain challenging to segment due to convolutional and pooling operations that result in information loss, especially for small objects. This paper presents a novel attention-based method called Across Feature Map Attention (AFMA) to address this challenge. It quantifies the inner-relationship between small and large objects belonging to the same category by utilizing the different feature levels of the original image. The AFMA could compensate for the loss of high-level feature information of small objects and improve the small/thin object segmentation. Our method can be used as an efficient plug-in for a wide range of existing architectures and produces much more interpretable feature representation than former studies. Extensive experiments on eight widely used segmentation methods and other existing small-object segmentation models on CamVid and Cityscapes demonstrate that our method substantially and consistently improves the segmentation of small/thin objects.
View details for DOI 10.1109/TPAMI.2022.3211171
View details for PubMedID 36178991
-
Utilizing differential characteristics of high dimensional data as a mechanism for dimensionality reduction
PATTERN RECOGNITION LETTERS
2022; 157: 1-7
View details for DOI 10.1016/j.patrec.2022.03.015
View details for Web of Science ID 000792767300001
-
Implicit neural representation for radiation therapy dose distribution.
Physics in medicine and biology
2022
Abstract
OBJECTIVE: Dose distribution data plays a pivotal role in radiotherapy treatment planning. The data is typically represented using voxel grids, and its size ranges from 10^6--10^8. A concise representation of the treatment plan is of great value in facilitating treatment planning and downstream applications. This work aims to develop an implicit neural representation of 3D dose distribution data.APPROACH: Instead of storing the dose values at each voxel, in the proposed approach, the weights of a multilayer perceptron (MLP) are employed to characterize the dosimetric data for plan representation and subsequent applications. We train a coordinate-based MLP with sinusoidal activations to map the voxel spatial coordinates to the corresponding dose values. We identify the best architecture for a given parameter budget and use that to train a model for each patient. The trained MLP is evaluated at each voxel location to reconstruct the dose distribution. We perform extensive experiments on dose distributions of prostate, spine, and head and neck tumor cases to evaluate the quality of the proposed representation. We also study the change in representation quality by varying model size and activation function.MAIN RESULTS: Using coordinate-based MLPs with sinusoidal activations, we can learn implicit representations that achieve a mean-squared error of 10^{-6} and peak signal-to-noise ratio greater than 50 dB at a target bitrate of ~1 across all the datasets, with a compression ratio of ~32. Our results also show that model sizes with a bitrate of 1--2 achieve optimal accuracy. For smaller bitrates, performance starts to drop significantly.SIGNIFICANCE: The proposed model provides a low-dimensional, implicit, and continuous representation of 3D dose data. In summary, given a dose distribution, we systematically show how to find a compact model to fit the data accurately. This study lays the groundwork for future applications of neural representations of dose data in radiation oncology.
View details for DOI 10.1088/1361-6560/ac6b10
View details for PubMedID 35477171
-
Estimation of Mechanical and Transport Parameters in Cancers Using Short Time Poroelastography
IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE
2022; 10
View details for DOI 10.1109/JTEHM.2022.3198316
View details for Web of Science ID 000844127700001
-
Human-level comparable control volume mapping with a deep unsupervised-learning model for image-guided radiation therapy.
Computers in biology and medicine
1800; 141: 105139
Abstract
PURPOSE: To develop a deep unsupervised learning method with control volume (CV) mapping from patient positioning daily CT (dCT) to planning computed tomography (pCT) for precise patient positioning.METHODS: We propose an unsupervised learning framework, which maps CVs from dCT to pCT to automatically generate the couch shifts, including translation and rotation dimensions. The network inputs are dCT, pCT and CV positions in the pCT. The output is the transformation parameter of the dCT used to setup the head and neck cancer (HNC) patients. The network is trained to maximize image similarity between the CV in the pCT and the CV in the dCT. A total of 554 CT scans from 158 HNC patients were used for the evaluation of the proposed model. At different points in time, each patient had many CT scans. Couch shifts are calculated for the testing by averaging the translation and rotation from the CVs. The ground-truth of the shifts come from bone landmarks determined by an experienced radiation oncologist.RESULTS: The system positioning errors of translation and rotation are less than 0.47mm and 0.17°, respectively. The random positioning errors of translation and rotation are less than 1.13mm and 0.29°, respectively. The proposed method enhanced the proportion of cases registered within a preset tolerance (2.0mm/1.0°) from 66.67% to 90.91% as compared to standard registrations.CONCLUSIONS: We proposed a deep unsupervised learning architecture for patient positioning with inclusion of CVs mapping, which weights the CVs regions differently to mitigate any potential adverse influence of image artifacts on the registration. Our experimental results show that the proposed method achieved efficient and effective HNC patient positioning.
View details for DOI 10.1016/j.compbiomed.2021.105139
View details for PubMedID 34942395
-
Artificial intelligence in image-guided radiotherapy: a review of treatment target localization.
Quantitative imaging in medicine and surgery
2021; 11 (12): 4881-4894
Abstract
Modern conformal beam delivery techniques require image-guidance to ensure the prescribed dose to be delivered as planned. Recent advances in artificial intelligence (AI) have greatly augmented our ability to accurately localize the treatment target while sparing the normal tissues. In this paper, we review the applications of AI-based algorithms in image-guided radiotherapy (IGRT), and discuss the indications of these applications to the future of clinical practice of radiotherapy. The benefits, limitations and some important trends in research and development of the AI-based IGRT techniques are also discussed. AI-based IGRT techniques have the potential to monitor tumor motion, reduce treatment uncertainty and improve treatment precision. Particularly, these techniques also allow more healthy tissue to be spared while keeping tumor coverage the same or even better.
View details for DOI 10.21037/qims-21-199
View details for PubMedID 34888196
View details for PubMedCentralID PMC8611462
-
Geometry and statistics-preserving manifold emb e dding for nonlinear dimensionality reduction
PATTERN RECOGNITION LETTERS
2021; 151: 155-162
View details for DOI 10.1016/j.patrec.2021.07.012
View details for Web of Science ID 000697484500002
-
Artificial intelligence in image-guided radiotherapy: a review of treatment target localization
QUANTITATIVE IMAGING IN MEDICINE AND SURGERY
2021
View details for DOI 10.21037/qims-21-199
View details for Web of Science ID 000685217800001
-
Non-Invasive Assessment of the Spatial and Temporal Distributions of Interstitial Fluid Pressure, Fluid Velocity and Fluid Flow in Cancers In Vivo
IEEE ACCESS
2021; 9: 89222-89233
View details for DOI 10.1109/ACCESS.2021.3089454
View details for Web of Science ID 000673652200001
-
Self-Supervised Feature Learning via Exploiting Multi-Modal Data for Retinal Disease Diagnosis
IEEE TRANSACTIONS ON MEDICAL IMAGING
2020; 39 (12): 4023–33
Abstract
The automatic diagnosis of various retinal diseases from fundus images is important to support clinical decision-making. However, developing such automatic solutions is challenging due to the requirement of a large amount of human-annotated data. Recently, unsupervised/self-supervised feature learning techniques receive a lot of attention, as they do not need massive annotations. Most of the current self-supervised methods are analyzed with single imaging modality and there is no method currently utilize multi-modal images for better results. Considering that the diagnostics of various vitreoretinal diseases can greatly benefit from another imaging modality, e.g., FFA, this paper presents a novel self-supervised feature learning method by effectively exploiting multi-modal data for retinal disease diagnosis. To achieve this, we first synthesize the corresponding FFA modality and then formulate a patient feature-based softmax embedding objective. Our objective learns both modality-invariant features and patient-similarity features. Through this mechanism, the neural network captures the semantically shared information across different modalities and the apparent visual similarity between patients. We evaluate our method on two public benchmark datasets for retinal disease diagnosis. The experimental results demonstrate that our method clearly outperforms other self-supervised feature learning methods and is comparable to the supervised baseline. Our code is available at GitHub.
View details for DOI 10.1109/TMI.2020.3008871
View details for Web of Science ID 000595547500024
View details for PubMedID 32746140
-
A data-driven dimensionality-reduction algorithm for the exploration of patterns in biomedical data.
Nature biomedical engineering
2020
Abstract
Dimensionality reduction is widely used in the visualization, compression, exploration and classification of data. Yet a generally applicable solution remains unavailable. Here, we report an accurate and broadly applicable data-driven algorithm for dimensionality reduction. The algorithm, which we named 'feature-augmented embedding machine' (FEM), first learns the structure of the data and the inherent characteristics of the data components (such as central tendency and dispersion), denoises the data, increases the separation of the components, and then projects the data onto a lower number of dimensions. We show that the technique is effective at revealing the underlying dominant trends in datasets of protein expression and single-cell RNA sequencing, computed tomography, electroencephalography and wearable physiological sensors.
View details for DOI 10.1038/s41551-020-00635-3
View details for PubMedID 33139824
-
Estimation of Vascular Permeability in Irregularly Shaped Cancers Using Ultrasound Poroelastography
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING
2020; 67 (4): 1083–96
Abstract
Vascular permeability (VP) is a mechanical parameter which plays an important role in cancer initiation, metastasis, and progression. To date, there are only a few non-invasive methods that can be used to image VP in solid tumors. Most of these methods require the use of contrast agents and are expensive, limiting widespread use.In this paper, we propose a new method to image VP in tumors, which is based on the use of ultrasound poroelastography. Estimation of VP by poroelastography requires knowledge of the Young's modulus (YM), Poisson's ratio (PR), and strain time constant (TC) in the tumors. In our method, we find the ellipse which best fits the tumor (regardless of its shape) using an eigen-system-based fitting technique and estimate the YM and PR using Eshelby's elliptic inclusion formulation. A Fourier method is used to estimate the axial strain TC, which does not require any initial guess and is highly robust to noise.It is demonstrated that the proposed method can estimate VP in irregularly shaped tumors with an accuracy of above [Formula: see text] using ultrasound simulation data with signal-to-noise ratio of 20 dB or higher. In vivo feasibility of the proposed technique is demonstrated in an orthotopic mouse model of breast cancer.The proposed imaging method can provide accurate and localized estimation of VP in cancers non-invasively and cost-effectively.Accurate and non-invasive assessment of VP can have a significant impact on diagnosis, prognosis, and treatment of cancers.
View details for DOI 10.1109/TBME.2019.2929134
View details for Web of Science ID 000522351200015
View details for PubMedID 31331877
-
Non-invasive imaging of Young's modulus and Poisson's ratio in cancers in vivo.
Scientific reports
2020; 10 (1): 7266
Abstract
Alterations of Young's modulus (YM) and Poisson's ratio (PR) in biological tissues are often early indicators of the onset of pathological conditions. Knowledge of these parameters has been proven to be of great clinical significance for the diagnosis, prognosis and treatment of cancers. Currently, however, there are no non-invasive modalities that can be used to image and quantify these parameters in vivo without assuming incompressibility of the tissue, an assumption that is rarely justified in human tissues. In this paper, we developed a new method to simultaneously reconstruct YM and PR of a tumor and of its surrounding tissues based on the assumptions of axisymmetry and ellipsoidal-shape inclusion. This new, non-invasive method allows the generation of high spatial resolution YM and PR maps from axial and lateral strain data obtained via ultrasound elastography. The method was validated using finite element (FE) simulations and controlled experiments performed on phantoms with known mechanical properties. The clinical feasibility of the developed method was demonstrated in an orthotopic mouse model of breast cancer. Our results demonstrate that the proposed technique can estimate the YM and PR of spherical inclusions with accuracy higher than 99% and with accuracy higher than 90% in inclusions of different geometries and under various clinically relevant boundary conditions.
View details for DOI 10.1038/s41598-020-64162-6
View details for PubMedID 32350327
-
A Robust Method to Estimate the Time Constant of Elastographic Parameters
IEEE TRANSACTIONS ON MEDICAL IMAGING
2019; 38 (6): 1358–70
Abstract
Novel viscoelastic and poroelastic elastography techniques rely on the accurate estimation of the temporal behavior of the axial or lateral strains and related parameters. From the temporal curve of the elastographic parameter of interest, the time constant (TC) is estimated using analytical models and curve-fitting techniques such as Levenberg-Marquardt (LM), Nelder-Mead (NM), and trust-region reflective (TR). In this paper, we propose a new technique named variable projection (VP) to estimate accurately and robustly the TC and steady-state value of the elastographic parameter of interest from its temporal curve. As a testing platform, the method is used with a novel analytical model, which can be used for both poroelastic and viscoelastic tissues and in most practical experimental conditions of clinical interest. Finite element and ultrasound simulations and experimental results demonstrate that VP is robust to noise and capable of estimating the TC of the elastographic parameter with accuracy higher than that of typically employed curve-fitting techniques. The results also demonstrate that the performance of VP is not affected by an incorrect initial TC guess. For example, in simulations, VP can estimate the TC of axial strain and effective Poisson's ratio accurately for initial guesses ranging from 0.001 to infinite times of the true TC value even in fairly noisy conditions (30-dB signal to noise ratio). In experiments, VP always estimates the axial strain TC reliably, whereas the LM, NM, and TR methods fail to converge or converge to wrong solutions in most of the cases.
View details for DOI 10.1109/TMI.2019.2894782
View details for Web of Science ID 000470829000005
View details for PubMedID 30703014
-
An analytical poroelastic model of a spherical tumor embedded in normal tissue under creep compression
JOURNAL OF BIOMECHANICS
2019; 89: 48–56
Abstract
An analytical model for a spherical poroelastic tumor embedded in normal poroelastic tissues under creep compression is presented in this paper. The tissue is modeled as a cylindrical sample containing a spherical inclusion having different material properties. Analytical expression for the volumetric strain generated inside the inclusion during creep compression is obtained. Error analysis is carried out by comparing the results from the developed analytical model with corresponding results obtained from an established finite element software for a number of samples with different material properties. The error is found to be below 2.5% for the samples with a small inclusion and 7% in the samples with a large inclusion. The analytical solutions reported in this paper can greatly impact elasticity imaging techniques aiming at reconstructing mechanical properties of tumors such as Young's modulus, Poisson's ratio, interstitial permeability and vascular permeability.
View details for DOI 10.1016/j.jbiomech.2019.04.009
View details for Web of Science ID 000469156200007
View details for PubMedID 31000348
-
Non-Invasive Imaging of Normalized Solid Stress in Cancers in Vivo
IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE-JTEHM
2019; 7: 4300209
Abstract
The solid stress (SSg) that develops inside a cancer is an important marker of cancer's growth, invasion and metastasis. Currently, there are no non-invasive methods to image SSg inside tumors. In this paper, we develop a new, non-invasive and cost-effective imaging method to assess SSg inside tumors that uses ultrasound poroelastography. Center to the proposed method is a novel analytical model, which demonstrates that SSg and the compression-induced stress (SSc) that generates inside the cancer in a poroelastography experiment have the same spatial distribution. To show the clinical feasibility of the proposed technique, we imaged and analyzed the normalized SSg inside treated and untreated human breast cancers in a small animal model. Given the clinical significance of assessing SSg in cancers and the advantages of the proposed ultrasonic methods, our technique could have a great impact on cancer diagnosis, prognosis and treatment methods.
View details for DOI 10.1109/JTEHM.2019.2932059
View details for Web of Science ID 000494830100001
View details for PubMedID 32309062
View details for PubMedCentralID PMC6822636
-
A New Poroelastography Method to Assess the Solid Distribution in Cancers
IEEE ACCESS
2019; 7: 103404–15
View details for DOI 10.1109/ACCESS.2019.2929021
View details for Web of Science ID 000481692100004
-
A Model-Based Approach to Investigate the Effect of a Long Bone Fracture on Ultrasound Strain Elastography
IEEE TRANSACTIONS ON MEDICAL IMAGING
2018; 37 (12): 2704–17
Abstract
The mechanical behavior of long bones and fractures has been under investigation for many decades due to its complexity and clinical relevance. In this paper, we report a new subject-specific methodology to predict and analyze the mechanical behavior of the soft tissue at a bone interface with the intent of identifying the presence and location of bone abnormalities with high accuracy, spatial resolution, and contrast. The proposed methodology was tested on both intact and fractured rabbit femur samples with finite element-based 3-D simulations, created from actual femur computed tomography data, and ultrasound elastography experiments. The results included in this study demonstrate that elastographic strains at the bone/soft tissue interface can be used to differentiate fractured femurs from the intact ones on a distribution level. These results also demonstrate that coronal plane axial shear strain creates a unique contrast mechanism that can be used to reliably detect fractures (both complete and incomplete) in long bones. Kruskal-Wallis test further demonstrates that the contrast measure for the fracture group (simulation: 2.1286±0.2206; experiment: 2.7034 ± 1.0672) is significantly different from that for the intact group (simulation: 0 ± 0; experiment: 1.1540±0.6909) when using coronal plane axial shear strain elastography ( < 0.01). We conclude that: 1) elastography techniques can be used to accurately identify the presence and location of fractures in a long bone and 2) the proposed model-based approach can be used to predict and analyze strains at a bone fracture site and to better interpret experimental elastographic data.
View details for DOI 10.1109/TMI.2018.2849996
View details for Web of Science ID 000451903400015
View details for PubMedID 29994472
-
A New Method for Estimating the Effective Poisson's Ratio in Ultrasound Poroelastography
IEEE TRANSACTIONS ON MEDICAL IMAGING
2018; 37 (5): 1178–91
Abstract
Ultrasound poroelastography aims at assessing the poroelastic behavior of biological tissues via estimation of the local temporal axial strains and effective Poisson's ratios (EPR). Currently, reliable estimation of EPR using ultrasound is a challenging task due to the limited quality of lateral strain estimation. In this paper, we propose a new two-step EPR estimation technique based on dynamic programming elastography (DPE) and Horn-Schunck (HS) optical flow estimation. In the proposed method, DPE is used to estimate the integer axial and lateral displacements while HS is used to obtain subsample axial and lateral displacements from the motion-compensated pre-compressed and post-compressed radio frequency data. Axial and lateral strains are then calculated using Kalman filter-based least square estimation. The proposed two-step technique was tested using finite-element simulations, controlled experiments and in vivo experiments, and its performance was statistically compared with that of analytic minimization (AM) and correlation-based method (CM). Our results indicate that our technique provides EPR elastograms of higher quality and accuracy than those produced by AM and CM. Regarding signal-to-noise ratio and elastographic contrast-to-noise ratio, in simulated data, the proposed method provides an average improvement of 30% and 75%, respectively, with respect to AM and of 100% and 169%, respectively, with respect to CM, whereas, in experiments, the proposed approach provides an average improvement of 30% and 67% with respect to AM and of 230% and 525% with respect to CM. Based on these results, the proposed method may be the preferred one in experimental poroelastography applications.
View details for DOI 10.1109/TMI.2018.2792437
View details for Web of Science ID 000431544500010
View details for PubMedID 29727281
-
An analytical poroelastic model for ultrasound elastography imaging of tumors
PHYSICS IN MEDICINE AND BIOLOGY
2018; 63 (2): 025031
Abstract
The mechanical behavior of biological tissues has been studied using a number of mechanical models. Due to the relatively high fluid content and mobility, many biological tissues have been modeled as poroelastic materials. Diseases such as cancers are known to alter the poroelastic response of a tissue. Tissue poroelastic properties such as compressibility, interstitial permeability and fluid pressure also play a key role for the assessment of cancer treatments and for improved therapies. At the present time, however, a limited number of poroelastic models for soft tissues are retrievable in the literature, and the ones available are not directly applicable to tumors as they typically refer to uniform tissues. In this paper, we report the analytical poroelastic model for a non-uniform tissue under stress relaxation. Displacement, strain and fluid pressure fields in a cylindrical poroelastic sample containing a cylindrical inclusion during stress relaxation are computed. Finite element simulations are then used to validate the proposed theoretical model. Statistical analysis demonstrates that the proposed analytical model matches the finite element results with less than 0.5% error. The availability of the analytical model and solutions presented in this paper may be useful to estimate diagnostically relevant poroelastic parameters such as interstitial permeability and fluid pressure, and, in general, for a better interpretation of clinically-relevant ultrasound elastography results.
View details for DOI 10.1088/1361-6560/aa9631
View details for Web of Science ID 000422867100009
View details for PubMedID 29336354