Huazhong University of Science and Technology Professor Chen Wei: Grey's Agent | Agent Insights

How can agents address the pain point of capturing patients' true preferences in medical LLMs? Can doctor-patient communication become more efficient? Can medical diagnoses become more precise? Can AI unlock new possibilities in healthcare? AI Hospital, developed by Professor Wei Chen at Huazhong University of Science and Technology, offers some compelling insights. Enjoy.

AI Hospital: Interactive Evaluation and Collaboration of LLMs as Intern Doctors for Clinical Diagnosis

Problem Solved: To effectively evaluate the practical utility of medical LLMs in real-world settings, the AI Hospital framework creates a simulated clinical environment. By replicating authentic doctor-patient interactions and decision-making processes, it enables a more comprehensive assessment of models' communication abilities and diagnostic accuracy, thereby gauging their potential effectiveness in actual healthcare environments.

Framework: The AI Hospital framework designs four key roles — patient, intern doctor, examiner, and attending physician. The intern doctor gathers clinical information through patient interactions and recommends medical examinations. The examiner provides test results based on actual medical data, which the intern doctor synthesizes to form a preliminary diagnosis. Finally, the attending physician evaluates the intern doctor's diagnostic report and the entire consultation process, examining the completeness of information collection and the soundness of the diagnosis.

Evaluation Methods: LLM performance is assessed through two modes: interactive consultation and one-step provision of complete information. Interactive consultation more closely resembles actual clinical practice, requiring the model to collect information through continuous dialogue with the patient before making a diagnosis. The One-Step method, which provides all information at once, sets a higher performance benchmark.

Applications: Hospital consultation agents, general-purpose medical LLMs.

Interactive process between intern doctor and attending physician during the diagnostic workflow

Results: The evaluation found that even advanced models like GPT-4 did not achieve the same performance in interactive consultation as in the One-Step method, revealing challenges in processing incrementally gathered information. This result highlights current limitations of LLMs in simulating real medical interactions and decision-making processes, suggesting that future research should focus on improving models' processing capabilities and adaptability in interactive environments to better serve actual diagnostic needs.

LLM test results in AI Hospital

---

Oasis Capital: Could you briefly introduce your background and research focus?

Prof. Chen: I did my PhD at Fudan University's Natural Language Processing Lab, on Zhongyu Wei's team. My main research direction was dialogue-based intelligent medical consultation. Multimodality is a major trend for NLP researchers looking to expand their work, and medical consultation is also transitioning from unimodal to multimodal approaches. Our current vision is to develop more practical medical applications based on large models, integrating patient records, medical imaging, test results, and more foundational data like genetic sequencing to enable doctors to make more precise diagnoses and treatment plans.

Oasis Capital: Please tell us about AI Hospital.

Prof. Chen: AI Hospital is primarily an innovation in evaluation methodology. It aims to better assess existing large models' capabilities in actual medical interactions. In the real world, doctors actively engage with patients through observation, listening, inquiry, and palpation — collecting information and combining it with test results before making a diagnosis. However, existing large models often lack this interactive capability. They tend to provide complete, lengthy responses all at once rather than adapting to patient needs through active inquiry and information gathering. We believe this behavioral pattern fails to align with patients' true preferences and the complexity of medical interactions.

To address this challenge, we developed AI Hospital, an interactive evaluation framework that simulates a real medical environment. It includes multiple roles: patient, intern doctor, examiner, and attending physician. Within this framework, we simulate multi-turn dialogue between doctor and patient, allowing the intern doctor to gradually build an understanding of the condition through questions and patient feedback. This interactive process not only more closely resembles actual clinical consultation but also better adapts to and satisfies patient needs.

Through AI Hospital, we can evaluate large language models' interactive capabilities and decision-making accuracy in simulated clinical diagnosis. Our evaluation focuses not only on the quality of information provided but also on how effectively they communicate with users. This assessment is crucial for guiding how we optimize these models' interactivity and practicality. In this way, we hope to advance the development of medical LLMs so they can better serve real clinical scenarios and provide medical services that more closely match patient expectations.

Oasis Capital: Why did you introduce a dispute resolution mechanism in AI Hospital?

Prof. Chen: In AI Hospital, we employ two evaluation methods to measure model performance: interactive evaluation and one-step (One-Step) evaluation. Interactive evaluation relies on the model collecting information through continuous dialogue with the patient, while one-step evaluation provides the model with all patient information at once for diagnosis generation. We found that although large models like GPT-4 perform well in one-step evaluation, their performance in interactive evaluation falls short, indicating limitations in simulating real doctors' diagnostic processes.

To address this, we introduced a dispute resolution mechanism. The core of this mechanism involves using multiple intern doctor models to process the same case, with each model proposing a diagnosis based on its own understanding and collected information. We then introduce a coordinating agent whose role is to integrate these intern doctors' diagnostic results, guiding them to gradually adjust and optimize their diagnoses through mutual comparison and discussion until consensus is reached.

This process simulates how real medical teams collaborate to solve problems. While this dispute resolution mechanism can improve diagnostic performance to some extent, we also recognize that it still cannot fully achieve the performance level of one-step evaluation. This indicates we still have a long way to go and need to continuously explore and improve to make our models more accurately simulate doctors' diagnostic processes and better serve actual medical needs.

Oasis Capital: You've been working on AI for Healthcare since 2019. What do you think is the biggest change that large models have brought to healthcare or hospital automation?

Prof. Chen: The biggest change is the transformation in model architecture and task paradigms. Early medical NLP mainly relied on traditional neural network models like RNNs and LSTMs. Back then, data volumes were small and model parameters were limited, so performance on many tasks naturally couldn't advance. With the emergence of models like BERT and GPT, we saw revolutionary changes in architecture and task processing approaches. The development of large models in particular has established consensus: as long as there is sufficient high-quality annotated data and enough parameters, model effectiveness and multi-task capabilities can improve.

Over the past year, both academia and industry in China have produced numerous medical LLMs, mostly text-based initially, and now increasingly multimodal — covering medical imaging, proteins, chemical molecules, drugs, and more. These large models have the potential to help doctors make more precise diagnoses and develop more personalized treatment plans. This extends beyond diagnosis to medical quality monitoring, patient service improvement, and even hospital daily management. As technology continues to advance and deeper industry collaboration develops, I believe medical LLMs can lead healthcare toward an entirely new era, dramatically improving the quality and efficiency of medical services we provide.

Oasis Capital: The biggest problem with the previous wave of medical AI imaging was transferability — for example, data trained at one hospital was useless at another. Multimodality is also converging on Transformer, including Vision Transformer and Sora. What's your view on transferability and generalizability?

Prof. Chen: I think large models can alleviate transferability issues, but may not fully solve them. There are significant differences between hospitals, including equipment purchased and evaluation standards — the standard values measured at different hospitals have slight variations. A value that reads as normal at one hospital might be considered abnormal at another.

If we want to achieve better transferability while different hospitals' data cannot be shared, I think federated learning is a promising direction. Each hospital can participate as an independent client in model training. Hospitals don't need to share raw data; they can process data locally and upload model gradient information to a central server. The server integrates these gradients and distributes updated model parameters back to participating hospitals. This enables collaborative training using data from multiple hospitals while protecting each hospital's privacy.

Oasis Capital: Is there industry consensus on how to do LLM evaluation well?

Prof. Chen: In healthcare, LLM evaluation is still insufficient. Even evaluations like Google's Med-PaLM are conducted on fixed datasets — while these metrics can indeed assess model performance, dataset performance doesn't necessarily match real-world performance. We also hope to develop more automated, lower-cost evaluation methods, because AI Hospital makes extensive calls to GPT-4. We're considering training these agent behaviors into our own smaller-parameter models, which would allow us to evaluate at lower cost. If evaluation is done well, we find the direction to optimize models.

Oasis Capital: How do you view overseas AI applications in hospital settings?

Prof. Chen: In healthcare, I believe models emphasizing generalizability carry certain risks. Hospitals and doctors tend to prefer specialized models over general-purpose ones. This is because medical professionals like doctors and nurses require high precision and specialization in their work. They need models that can provide explainable, evidence-based diagnostic recommendations, not just conversational abilities.

To meet medical professionals' needs, we should focus on improving models' specialization and accuracy. This means models must not only provide accurate diagnostic results but also explain their decision-making processes, offering doctors strong evidential support. Such models can help doctors make more informed clinical decisions and improve overall quality of medical services.

Currently, some foreign medical LLMs are developing in this direction, focusing on analyzing specific medical imaging types (like CT) or diagnosing specific organs. This specialized model design helps improve diagnostic accuracy and makes model outputs more aligned with clinical practice needs.

In summary, I believe in medical AI we should pursue specialization and explainability rather than focusing solely on conversational ability. By continuously optimizing and deepening models' specialized knowledge, we can better serve the healthcare industry and provide higher-quality medical services to patients.

Oasis Capital: Which technical iterations — such as instruction fine-tuning and retrieval augmentation — do you think will truly enable AI to land in healthcare? Which application could be deployed fastest and have the biggest impact?

Prof. Chen: In healthcare, technical iteration and innovation are crucial for AI's practical application. I believe both instruction fine-tuning and retrieval augmentation, among other techniques, can assist AI deployment in healthcare to varying degrees. These technologies are not mutually exclusive but can complement each other to jointly advance AI development.

The key lies in how to combine large models with effective methodologies and leverage high-quality data. As long as we can ensure data quality, AI applications in healthcare have tremendous potential. However, we must also face the reality that rare disease data collection is difficult, which can lead to model bias and unfairness. In such cases, doctors' professional judgment remains crucial — just as autonomous driving ultimately requires human driver supervision, final responsibility for medical decisions still rests with doctors.

From the perspective of AI deployment in healthcare, I believe tumor staging prediction and surgical plan selection are the most promising areas for rapid implementation with significant impact. From what I understand, approximately 30-40% of doctors misjudge tumor staging. If initial staging is inaccurate, subsequent treatment plans may not be optimal. As hospital information systems continue to upgrade and improve, combined with large volumes of real-world data and manual annotation, we can hope to use multimodal data for precise tumor staging and surgical plan selection.

In conclusion, AI applications in healthcare have broad prospects, but we need to focus on data quality, model fairness, and doctors' professional judgment alongside technological innovation. Through these comprehensive measures, we can expect AI to play a greater role in healthcare, providing patients with more precise and effective medical services.