Large Language Model Applied to ICD-10 Coding
Introduction
The International Classification of Diseases, 10th Revision (ICD-10), includes approximately 69,000 diagnostic codes and 72,000 procedure codes. These classifications are assigned by certified coding professionals after thoroughly reviewing medical records. Our hospital utilizes Large Language Model (LLM) to analyze medical records, such as discharge diagnoses and surgical reports. Expert domain knowledge in disease classification is incorporated into the model training process. The system then generates suggested ICD-10 diagnosis and procedure codes for the patient’s hospitalization, assisting in the disease classification and coding workflow. The features are as follows:
- A cross-disciplinary professional team collaborated to develop the AI coding model.
- NTUH possesses advanced deep learning technology and a large volume of training data, along with an outstanding team of coding specialists. It has pioneered the integration of expert knowledge into model training.
- Instruction Fine-tuning technology was applied, enabling the model to better understand the requirements of ICD-10 tasks, significantly improving prediction accuracy and overall performance.
- A systematic expert knowledge feedback process has been established, with periodic model updates.
- The interface is designed based on user habits, integrating and optimizing workflows.
Implementation Achievements
(一)Enhancing Disease Classification Accuracy and Efficiency
1. This project utilizes large language model (LLM) technology, integrated with domain knowledge from clinical coders, to provide ICD-10 diagnosis and procedure code suggestions, assisting in disease classification and coding operations. The project team continuously monitors the AI model’s performance, incorporating real-time feedback and ongoing program refinements. The full-code prediction F-score has reached 86.67%. The system has been successfully deployed across the entire NTUH healthcare system, including local, regional, and tertiary-level hospitals.
2. An innovative AI-based principal diagnosis prediction model has been developed, achieving an accuracy rate of 82.29%. The principal diagnosis is determined by disease classification experts as the main reason for the patient’s hospitalization. The model supports Diagnosis-Related Group (DRG) analysis and application, helping healthcare institutions secure appropriate reimbursement under the National Health Insurance system.
3. As a national-level teaching hospital, NTUH possesses a large volume of training data, enabling the model to cover a wide range of disease categories, including rare diseases.
(二) Strengthening Human-AI Collaborative Mechanism
1. Since the system is independently developed in-house, AI training data and disease classification logic can be dynamically updated. Classification rules often change due to emerging diseases or new medical technologies. Disease classification staff can update coding guidelines or code changes and feed them back into the AI system for retraining, thus improving prediction accuracy.
2. Coding results from the AI model are reviewed case-by-case in collaboration with disease classification professionals. Along with the existing 10% manual cross-check mechanism, the overall code consistency has reached 97.3%, significantly improving coding quality.
3. The AI system also flags potentially missing or more accurate codes, leading to a user satisfaction rate of 95%.
4. AI is an assistive tool that cannot replace coders, but it can shorten coding time and save 14% of labor hours.
(三) Future Development
1. Continue optimizing the AI-assisted coding model to reduce the workload of coding staff, allowing them to focus on complex cases or long-term planning.
2. This technology has been granted patent protection and is being developed toward a commercial application model.
Figure: NTUH disease classification coding model and applications