A Chatbot for Frequently Asked Questions using TF-IDF and Query Expansion Techniques

Ibrahim, Mohammed Salah; Hadi, Ahmed Adnan; Sallibi, Anas D.; Nafea, Ahmed Adil

doi:10.24271/psr.2025.487515.1805

A Chatbot for Frequently Asked Questions using TF-IDF and Query Expansion Techniques

Document Type : Original Article

Authors

Mohammed Salah Ibrahim ¹

Ahmed Adnan Hadi ²

Anas D. Sallibi ³

Ahmed Adil Nafea

¹ Department of Artificial Intelligence, College of Computer Science and IT, University of Anbar, Ramadi 31001, Iraq.

² Department of Computer Sciences, College of Sciences, University of Al Maarif, Ramadi 31001, Iraq.

³ Artificial Intelligence Sciences Department, College of sciences, Al-Mustaqbal University, Babil 51001, Iraq.

10.24271/psr.2025.487515.1805

Abstract

The Frequently Asked Questions (FAQs) chatbot plays an important role in providing information, especially in academic fields. FAQs are created to address common concerns frequently raised and answered by domain experts. Answers to such FAQs should be precise and related to the question asked. This paper presents a chatbot system that uses a Term Frequency-Inverse Document Frequency (TF-IDF ) method and enhances this using a semantic query expansion technique (TF-IDF-Expa) to rank and retrieve accurate responses. Two question-answering datasets are utilized to evaluate the system, SQuAD and MS-Marco. Moreover, 50 synthetic academic questions are used to show the system’s performance in real-time. Both methods, TF-IDF and TF-IDF-Expa, reported 100% accuracy; However, the confidence rate for TF-IDF-Expa was around 70% due to the additional information presented during query expansion, which can add ambiguity. The trade-off between accuracy and confidence raises concerns and suggests more optimisation chances for real-world applications.

Keywords

Generative AI applications

Frequently Asked Questions

Chatbots

Question-Answering

Query Expansion

Chatbot development