publications

2025

1. Evaluating Llama-3.1 for Adverse Drug Event Entity and Relationship Extraction Across Prompting Techniques

   Howard Prioleau, Saurav K Aryal, and Legand Burge

   In International Conference on Advances in Computing Research, 2025

   Abs

   Post-market adverse drug effects (ADEs) are often found in the clinical notes of Electronic Health Records. We evaluate open-weight large language model, Llama-3.1, for detection of ADEs, Named Entity Recognition, and Entity Linkage tasks across varying prompting techniques suited for text decomposition including, but not limited to zero-shot, few-shot and tree of thought. Finally, we propose a novel improvement to the tree of thought prompting technique which combines consistency and thought merging. Our proposed approach is then evaluated and compared with the other approaches on these tasks using standard metrics for each task (f1, precision and recall).

2. Entity Only vs. Inline Approaches: Evaluating LLMs for Adverse Drug Event Detection in Clinical Text (Student Abstract)

   Howard Prioleau, and Saurav Aryal

   In Proceedings of the AAAI Conference on Artificial Intelligence, 2025

2024

1. Ad-hoc Ensemble Approach for Detecting Adverse Drug Events in Electronic Health Records

   Saurav K Aryal, and Howard Prioleau

   Journal of Computing Sciences in Colleges, 2024

   Abs

   Adverse Drug Events (ADEs) are a significant concern in healthcare due to their prevalence and impact. However, implementing such personalized systems will not be without its own challenges, as ADEs are frequently documented in electronic health records (EHRs), and the inherent complexities of these records, particularly unstructured clinical notes, pose significant obstacles for their identification. In this study, we seek to address these challenges by using large language models (LLMs) to detect ADE occurrences in clinical text. We will employ the zero-shot classification capabilities of various pre-trained LLMs and then integrate them into an ensemble approach to consolidate their predictions. We conducted a series of experiments utilizing various ensemble strategies and model comparison techniques to gauge the effectiveness of these models in detecting and classifying an ADE in medical notes. Our findings highlight the potential and limitations of current state-of-the-art LLM and their maximal zero-shot potential when leveraging ensembling, emphasizing the importance of continuous fine-tuning to enhance model reliability and applicability in real-world scenarios.

2023

1. Howard University Computer Science at SemEval-2023 Task 12: A 2-Step System Design for Multilingual Sentiment Classification with Language Identification

   Saurav Aryal, and Howard Prioleau

   In Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), 2023

   Abs PDF

   The recent release of the AfriSenti-SemEval shared Task 12 has made available 14 new datasets annotated for sentiment analysis on African Languages. We proposed and evaluated two approaches to this task, Delta TF-IDF, and a proposed Language-Specific Model Fusion System using Language Identification. Both produced comparable or better classification performance than the current state-of-art models on this task: AfriBERTa, AfroXLMR, and AfroLM.

2. Zero-Shot Classification Reveals Potential Positive Sentiment Bias in African Languages Translations

   Hrishav Sapkota, Saurav Keshari Aryal, and Howard Prioleau

   2023

   PDF
3. Feature Importance Analysis for Mini Mental Status Score Prediction in Alzheimer’s Disease

   Howard Prioleau, and Saurav Keshari Aryal

   2023

   Abs PDF

   This research article proposes developing predictive models to forecast Mini-Mental State Exam (MMSE) scores using the 54 most important features identified from the current state-of-the-art model. The study employs the SHapley Additive exPlanations (SHAP) method to explore feature importance and interpret model performance. The analysis shows that the Automated Readability Index (ARI) is the most influential feature in predicting MMSE scores. This finding suggests that ARI’s capability to capture language impairment and morphosyntax is valuable in predicting cognitive decline in dementia patients. Although the analysis could not evaluate all features, this study provides a foundation for future investigations into features that may assist in predicting MMSE scores and the onset of Dementia.

4. Sentiment analysis across multiple african languages: A current benchmark

   Saurav K Aryal, Howard Prioleau, and Surakshya Aryal

   Social Impact of AI for Africa (SIAIA-23) at AAAI-23, 2023

   PDF
5. Baselining Performance for Multilingual Codeswitching Sentiment Classification

   Saurav K Aryal, Howard Prioleau, Surakshya Aryal, and Gloria Washington

   Journal of Computing Sciences in Colleges, 2023
6. Term Frequency Features vs Transformers: A Comparision for Sentiment Classification of African Languages

   Saurav K Aryal, Howard Prioleau, Ujjawal Shah, and Sameer Acharya

   Journal of Computing Sciences in Colleges, 2023
7. EXPERIMENTING WITH MULTIMODAL AUTOML: DETECTION AND EVALUATION OF ALZHEIMER’S DISEASE

   Ujjawal Shah, Saurav K Aryal, Howard Prioleau, and Legand Burge

   2023

   PDF
8. Evaluating Ensembled Transformers for Multilingual Code-Switched Sentiment Analysis

   Saurav K. Aryal, Howard Prioleau, Gloria Washington, and Legand Burge

   In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), 2023
9. Ensembling and Modeling Approaches for Enhancing Alzheimer’s Disease Scoring and Severity Assessment

   Saurav K. Aryal, Ujjawal Shah, Howard Prioleau, and Legand Burge

   In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), 2023
10. Benchmarking Current State-of-the-Art Transformer Models on Token Level Language Identification and Language Pair Identification

    Howard Prioleau, and Saurav K. Aryal

    In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), 2023

    Abs PDF

    With the rise of internet usage, code-switching, where multiple languages or dialects intermingle, has surged. Traditional linguistic analysis struggles with this mixed text, as they’re typically monolingual-focused. This paper delves into two core tasks for analyzing code-switched data: Token Level Language Identification (LID) and our newly proposed Language Pair Identification (LPI). We benchmarked and compared current state-of-art transformer models across both tasks to gauge their applicability to the tasks. Our results showed that state-of-the-art multilingual transformer models could achieve state-of-the-art performance on both tasks. The impressive performance on LPI suggests that this will be the first step to utilizing Language Pair Identification to assist in various facets related to Code-Switched corpora and classification performance.

2022

1. Acoustic-Linguistic Features for Modeling Neurological Task Score in Alzheimer’s

   Saurav K Aryal, Howard Prioleau, and Legand Burge

   In PACIFIC SYMPOSIUM ON BIOCOMPUTING 2023: Kohala Coast, Hawaii, USA, 3–7 January 2023, 2022

   Abs PDF

   The average life expectancy is increasing globally due to advancements in medical technology, preventive health care, and a growing emphasis on gerontological health. Therefore, developing technologies that detect and track aging-associated disease in cognitive function among older adult populations is imperative. In particular, research related to automatic detection and evaluation of Alzheimer’s disease (AD) is critical given the disease’s prevalence and the cost of current methods. As AD impacts the acoustics of speech and vocabulary, natural language processing and machine learning provide promising techniques for reliably detecting AD. We compare and contrast the performance of ten linear regression models for predicting Mini-Mental Status Exam scores on the ADReSS challenge dataset. We extracted 13000+ handcrafted and learned features that capture linguistic and acoustic phenomena. Using a subset of 54 top features selected by two methods: (1) recursive elimination and (2) correlation scores, we outperform a state-of-the-art baseline for the same task. Upon scoring and evaluating the statistical significance of each of the selected subset of features for each model, we find that, for the given task, handcrafted linguistic features are more significant than acoustic and learned features.

2. Sentiment Classification of Code-Switched Text using Pre-Trained Multilingual Embeddings and Segmentation

   Saurav K Aryal, Howard Prioleau, and Gloria Washington

   In CS & IT Conference Proceedings, 2022

   PDF