Jinxi Xiang

All Publications

• A vision-language foundation model for precision oncology. Nature Xiang, J., Wang, X., Zhang, X., Xi, Y., Eweje, F., Chen, Y., Li, Y., Bergstrom, C., Gopaulchan, M., Kim, T., Yu, K. H., Willens, S., Olguin, F. M., Nirschl, J. J., Neal, J., Diehn, M., Yang, S., Li, R. 2025

  Abstract

  Clinical decision-making is driven by multimodal data, including clinical notes and pathological characteristics. Artificial intelligence approaches that can effectively integrate multimodal data hold significant promise in advancing clinical care1,2. However, the scarcity of well-annotated multimodal datasets in clinical settings has hindered the development of useful models. In this study, we developed the Multimodal transformer with Unified maSKed modeling (MUSK), a vision-language foundation model designed to leverage large-scale, unlabelled, unpaired image and text data. MUSK was pretrained on 50 million pathology images from 11,577 patients and one billion pathology-related text tokens using unified masked modelling. It was further pretrained on one million pathology image-text pairs to efficiently align the vision and language features. With minimal or no further training, MUSK was tested in a wide range of applications and demonstrated superior performance across 23 patch-level and slide-level benchmarks, including image-to-text and text-to-image retrieval, visual question answering, image classification and molecular biomarker prediction. Furthermore, MUSK showed strong performance in outcome prediction, including melanoma relapse prediction, pan-cancer prognosis prediction and immunotherapy response prediction in lung and gastro-oesophageal cancers. MUSK effectively combined complementary information from pathology images and clinical reports and could potentially improve diagnosis and precision in cancer therapy.

  View details for DOI 10.1038/s41586-024-08378-w

  View details for PubMedID 39779851

  View details for PubMedCentralID 9586871

• Deep learning-based diagnosis and survival prediction of patients with renal cell carcinoma from primary whole slide images. Pathology Chen, S., Wang, X., Zhang, J., Jiang, L., Gao, F., Xiang, J., Yang, S., Yang, W., Zheng, J., Han, X. 2024

  Abstract

  There is an urgent clinical demand to explore novel diagnostic and prognostic biomarkers for renal cell carcinoma (RCC). We proposed deep learning-based artificial intelligence strategies. The study included 1752 whole slide images from multiple centres. Based on the pixel-level of RCC segmentation, the diagnosis diagnostic model achieved an area under the receiver operating characteristic curve (AUC) of 0.977 (95% CI 0.969-0.984) in the external validation cohort. In addition, our diagnostic model exhibited excellent performance in the differential diagnosis of RCC from renal oncocytoma, which achieved an AUC of 0.951 (0.922-0.972). The graderisk for the recognition of high-grade tumour achieved AUCs of 0.840 (0.805-0.871) in the Cancer Genome Atlas (TCGA) cohort, 0.857 (0.813-0.894) in the Shanghai General Hospital (General) cohort, and 0.894 (0.842-0.933) in the Clinical Proteomic Tumor Analysis Consortium (CPTAC) cohort, for the recognition of high-grade tumour. The OSrisk for predicting 5-year survival status achieved an AUC of 0.784 (0.746-0.819) in the TCGA cohort, which was further verified in the independent general cohort and the CPTAC cohort, with AUCs of 0.774 (0.723-0.820) and 0.702 (0.632-0.765), respectively. Moreover, the competing-risk nomogram (CRN) showed its potential to be a prognostic indicator, with a hazard ratio (HR) of 5.664 (3.893-8.239, p<0.0001), outperforming other traditional clinical prognostic indicators. Kaplan-Meier survival analysis further illustrated that our CRN could significantly distinguish patients with high survival risk. Deep learning-based artificial intelligence could be a useful tool for clinicians to diagnose and predict the prognosis of RCC patients, thus improving the process of individualised treatment.

  View details for DOI 10.1016/j.pathol.2024.05.012

  View details for PubMedID 39168777

• Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Information Tian, K., Guan, Y., Xiang, J., Zhang, J., Han, X., Yang, W., ACM ASSOC COMPUTING MACHINERY. 2023: 7961-7970

  View details for DOI 10.1145/3581783.3611955

  View details for Web of Science ID 001199449107094