Sean Tsung

All Publications

• Evaluating Large Language Models and Retrieval-Augmented Generation Enhancement for Delivering Guideline-Adherent Nutrition Information for Cardiovascular Disease Prevention: Cross-Sectional Study. Journal of medical Internet research Parameswaran, V., Bernard, J., Bernard, A., Deo, N., Tsung, S., Lyytinen, K., Sharp, C., Rodriguez, F., Maron, D. J., Dash, R. 2025; 27: e78625

  Abstract

  Cardiovascular disease (CVD) remains the leading cause of death worldwide, yet many web-based sources on cardiovascular (CV) health are inaccessible. Large language models (LLMs) are increasingly used for health-related inquiries and offer an opportunity to produce accessible and scalable CV health information. However, because these models are trained on heterogeneous data, including unverified user-generated content, the quality and reliability of food and nutrition information on CVD prevention remain uncertain. Recent studies have examined LLM use in various health care applications, but their effectiveness for providing nutrition information remains understudied. Although retrieval-augmented generation (RAG) frameworks have been shown to enhance LLM consistency and accuracy, their use in delivering nutrition information for CVD prevention requires further evaluation.To evaluate the effectiveness of off-the-shelf and RAG-enhanced LLMs in delivering guideline-adherent nutrition information for CVD prevention, we assessed 3 off-the-shelf models (ChatGPT-4o, Perplexity, and Llama 3-70B) and a Llama 3-70B+RAG model.We curated 30 nutrition questions that comprehensively addressed CVD prevention. These were approved by a registered dietitian providing preventive cardiology services at an academic medical center and were posed 3 times to each model. We developed a 15,074-word knowledge bank incorporating the American Heart Association's 2021 dietary guidelines and related website content to enhance Meta's Llama 3-70B model using RAG. The model received this and a few-shot prompt as context, included citations in a Context Source section, and used vector similarity to align responses with guideline content, with the temperature parameter set to 0.5 to enhance consistency. Model responses were evaluated by 3 expert reviewers against benchmark CV guidelines for appropriateness, reliability, readability, harm, and guideline adherence. Mean scores were compared using ANOVA, with statistical significance set at P<.05. Interrater agreement was measured using the Cohen κ coefficient, and readability was estimated using the Flesch-Kincaid readability score.The Llama 3+RAG model scored higher than the Perplexity, GPT-4o, and Llama 3 models on reliability, appropriateness, guideline adherence, and readability and showed no harm. The Cohen κ coefficient (κ>70%; P<.001) indicated high reviewer agreement.The Llama 3+RAG model outperformed the off-the-shelf models across all measures with no evidence of harm, although the responses were less readable due to technical language. The off-the-shelf models scored lower on all measures and produced some harmful responses. These findings highlight the limitations of off-the-shelf models and demonstrate that RAG system integration can enhance LLM performance in delivering evidence-based dietary information.

  View details for DOI 10.2196/78625

  View details for PubMedID 41057043

• ASO Visual Abstract: Telemedicine Trends in Ambulatory Surgical Oncology-A Five-Year Analysis of Visit Volume and Utilization at a High-Volume Academic Center. Annals of surgical oncology Parameswaran, V., Miller, M., Pritchett, J., Demaerschalk, B., Hu, Y., Tsung, S., Zeng, Y. B., Rosengaus, L., Sharp, C., Lee, B., Shah, S. 2025

  View details for DOI 10.1245/s10434-025-17692-0

  View details for PubMedID 40517204

• Telemedicine Trends in Ambulatory Surgical Oncology: A Five-Year Analysis of Visit Volume and Utilization at a High-Volume Academic Center. Annals of surgical oncology Parameswaran, V., Miller, M., Pritchett, J., Demaerschalk, B., Hu, Y., Tsung, S., Zeng, Y. B., Rosengaus, L., Sharp, C., Lee, B., Shah, S. 2025

  Abstract

  Telemedicine is now a sustained modality of ambulatory surgical oncology care, yet its association with workforce utilization, patient volume, and visit type at high-volume academic centers remains understudied. Characterizing these patterns is essential for guiding clinical operations and long-term integration of telemedicine into surgical oncology practice.We conducted a retrospective cohort study across nine oncology subspecialties at Stanford Medicine's ambulatory surgical oncology clinics from January 2019 to December 2023 to compare yearly visit volumes and telemedicine use. The study included a total of 231,746 visits, including 50,667 new and 181,079 return visits. We measured overall visit volumes, telemedicine utilization, and their association with increase in unique patients served, including both new and return visits.In 2023, visit volumes increased by 44% (46,726 to 67,259), and the clinician workforce grew by 16.8% (107 to 125) compared with 2019. The number of unique patients served rose by 39% (20,620 to 28,711), while visits per patient remained stable (2.3 ± 2.1 to 2.3 ± 2.2). Telemedicine use increased from 0.5% (244/46,726) to 37% (24,906/67,259), correlating with serving more patients per year (r = 0.776, p = 0.030) and return visits (r = 0.796, p = 0.010), but not new visits (r = 0.432, p = 0.245).At this academic medical center, telemedicine use is associated with an expansion of the clinician workforce, an increase in patient volume, and more return visits rather than new visits, without contributing to overall higher healthcare utilization. This suggests that telemedicine can deliver a significant proportion of ambulatory surgical oncological visits while preserving access to care and operational efficiency.

  View details for DOI 10.1245/s10434-025-17592-3

  View details for PubMedID 40498347

  View details for PubMedCentralID 9331038