Assessing Disease Severity in Cutaneous Lupus Patients Using Natural Language Processing

BACKGROUND: Cutaneous lupus erythematous (CLE) is an autoimmune skin disorder that manifests as inflammatory cutaneous lesions commonly in photosensitive areas. It is often chronic in nature, with exacerbations that can lead to hyperpigmentation and scarring. One tool used to measure disease activity and damage in CLE patients is the Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI) score. There has been little work done previously using natural language processing (NLP) in dermatology to assess disease severity, though there is promising potential for its use given the role of narrative data in dermatology. OBJECTIVE: We aim to develop a NLP model that interprets physical examination (PE) documentation in CLE patients and computes disease severity scores in the form of CLASI activity and damage scores. METHODS: Dataset was derived from 50 patients enrolled in the UTSW CLE registry. 89 clinical exams of 24 patients were used in a training set, used to train the NLP model. 35 clinical exams of 26 patients were selected for a validation set, used to test the model's accuracy in prediction. An entity dictionary was defined that provided rules for labeling vocabulary pertinent to CLASI scores within the PE note. This was used to label the relationships between entities in the training and validation sets. The BERT (Bidirectional Encoder Representations from Transformers) model was trained to predict all entities and relationships in the notes, based on which the CLASI scores were calculated. After training, the model was applied to the validation set. In evaluation, scores generated from the model were compared to the ground-truth CLASI scores based on human annotation. RESULTS: The model-predicted scores had a correlation of 0.79 and 0.86 with the ground truth on the activity and damage scores, respectively, in the training set, and a 0.61 and 0.79 correlation in the validation set. The model had 0.84 and above for accuracy, recall, precision and F1 within the sub-goal of determining the category of score severity (high or low), for both training and validation sets. CONCLUSIONS: Using PE notes as the input, a BERT-based NLP model can be trained to predict CLASI scores in CLE patients. If successfully implemented, this algorithm can significantly increase the volume of real-world data available for CLE research by efficiently processing PE notes in the EHR. Future steps are to increase the size and representation of the training set to improve accuracy and external validity of BERT's predictions.