AI Insights

Revolutionizing Patient Care: How a Text-Based Chatbot is Transforming the Hospital Experience

May 23, 2024

article featured image

In today’s fast-paced healthcare environment, patients often struggle to get the information and support they need during their hospital stay. Overworked medical staff may not always be available to answer questions or provide guidance, leaving patients feeling anxious and unsupported.

This is where a text-based healthcare chatbot can make a significant difference. By providing 24/7 access to relevant information and personalized support, chatbots can help alleviate patient concerns, reduce the burden on healthcare workers, and improve overall patient satisfaction and outcomes.

Why is there a need for Chatbots in Healthcare?

Global health crises lay bare the weaknesses in medical systems around the world. Just a few years ago, the Covid-19 pandemic exposed the vulnerabilities in healthcare systems worldwide. A study by the World Health Organization found that 90% of countries experienced disruptions to essential health services due to the pandemic.

In the United States alone, hospitals saw a 21% increase in patient volume during Covid surges, while simultaneously facing critical staffing shortages as healthcare workers fell ill. A survey of over 1,300 healthcare professionals revealed that 93% experienced stress, 86% reported anxiety, and 76% reported exhaustion and burnout.

An unprecedented strain on medical resources can leave many patients without adequate care and support. Chatbots have emerged as a scalable solution to bridge this gap, capable of providing 24/7 access to health information and guidance. 

Conversational agents can reduce hospital visits, decrease healthcare costs, and improve medication adherence.

Our Goal

This project focused on solving this issue by creating solutions in multiple forefronts as below:

  1. Creating a chatbot to help users get answers at their fingertips
  2. Creating a dashboard for users to visualize insights on a particular disease
  3. Performing sentiment analysis on the data gathered from responses from patients to visualize the emotions into positive, negative, and neutral categories and improve the chatbot experience based on the results.

So How Do We Proceed?

Project Pipeline

Project Pipeline

Data Collection and Preprocessing

We started by web scraping reliable medical pages to collect data on 6 diseases for the chatbot’s knowledge base. We had individual .csv files with 21 columns of data for each disease, covering topics like symptoms, treatment, facts, expenses, and more.

Covid Symptoms (male, female, babies, common, old-people, critical-stage)
Diabetes Treatment ( general, babies, old people, critical-stage)
Respiratory Diseases Medication (general, babies, old people)
Heart Diseases Facts or Myths
Dengue Expenses
Diarrhoeal Diseases Funding
Survival Rate

After merging the CSV files, we preprocessed the data by:

  • Removing repeated columns and unnecessary data
  • Fixing spelling mistakes and removing unwanted symbols
  • Adding a “Condition” column to track the disease
  • Converting long-form content to short summaries
  • Removing points that users can’t understand

For patient experience, we used the Twitter API to gather 63,239 tweets without class and 29,706 tweets with class. The collected tweets needed cleaning, such as:

  • Removing non-English tweets using SpaCy’s lang_detect function
  • Removing hashtags, mentions, and URLs using the regex library
  • Dropping duplicate texts to get only unique tweets

The image below shows the conditions for cleaning the tweets:

The resulting dataframe contains the original tweets, cleaned tweets, and their classes.

NLP Sentiment Analysis

Sentiment Analysis determines the emotional tone behind words to understand attitudes, opinions, and emotions in online mentions.

Our project classified tweets based on patients’ experiences during treatment, such as their feelings and the helpfulness of hospital staff.

VADER (Valence Aware Dictionary and Sentiment Reasoner) was an excellent choice because our data was unlabeled for sentiments, and it handles preprocessing, acronyms, and emoticons. VADER is a lexicon and rule-based sentiment analysis tool attuned to social media sentiments. It provides the sentiment score and the degree of positivity, negativity, or neutrality.

Step 1: Active Learning For Labeling The Data As “Relevant” Or “Irrelevant”

  • Manually annotated 14,000 tweets
  • Applied active learning for text classification to identify relevant (patient experience) and irrelevant tweets
  • Utilized the Small-text library with a Transformer-based classifier
  • Leveraged HuggingFace for the machine learning pipeline
Active Learning Graph

Active Learning Graph

Step 2: Filtering The Data To Make The Corpus Customized To The Project Idea

  • The following was the distribution of tweets in the corpus of 62273 tweets
    • Relevant – 23001
    • Irrelevant – 39272
  • The irrelevant tweets were those containing any ads/promotions, government schemes or fundraisers, which were later dropped.
Number of Relevant and Irrelevant Tweets

Number of Relevant and Irrelevant Tweets

Number of Relevant and Irrelevant Tweets

Step 3: Performing Sentiment Analysis For 3 Sentiments Using Vader

  • We used VADER to perform Sentiment Analysis.
  • The data was classified into 3 sentiments namely:
    • Positive
    • Negative
    • Neutral
Number of tweets classified for each sentiment

Number of tweets classified for each sentiment

Step 4: Visualization using Python

Word Cloud is a data visualization technique used for representing text data in which the size of each word indicates its frequency or importance. In simpler terms, a ‘word cloud’ is a visual representation of word frequency.

Word clouds are increasingly being employed as a simple tool to identify the focus of written material. They are widely used for analyzing data from social network websites.

Word Cloud for each Sentiment

Word Cloud for each Sentiment

Data Visualization and Dashboarding

Data visualization, which was once a relatively obscure field, has now become ubiquitous in the fields of business intelligence and data journalism. There are numerous methods for effectively and beautifully presenting insights, as well as a plethora of blogs dedicated to creating and analyzing visualizations. Dashboards are what we utilized to represent data in a variety of ways.

Storytelling with Data

Data visualization exists, in large part, to help create a compelling narrative. In the Health Dashboard, we are trying to show the level of data based on the Disease Conditions. Simultaneously, data talks about Sentiment Analysis that is extracted from tweets data.

Innovative Data Visualization

Rather than limiting data to mundane functionality, we can take advantage of the opportunity to showcase innovative data visualization projects. To drill down into the data, the health dashboard makes use of GIF images, Tabular, Filters, Actions, and Word Cloud. In Addition, the Sentiment Analysis Dashboard uses Word Cloud, Bar Charts, Dual Axis Charts, and Filter Actions.


Users can visualize the symptoms, diagnosis, treatment, expenses, and FAQ’s in the Health Conditions Dashboard based on the disease they had chosen. It was developed to help patients and their families understand and learn more about the diseases and data used by our chatbot. Furthermore, the Sentiment Analysis Dashboard is intended to help doctors and medical administrators gain a better understanding of the experiences of patients and the general public when they visit hospitals.


Chatbot Modeling Rasa Framework

In a Chatbot implementation, we can integrate to provide advanced conversational capabilities for healthcare professionals. The mechanism involves training the Chatbot by using Generative AI technology, which enables it to understand and respond to natural language queries related to healthcare. By incorporating advanced NLP techniques, the ChatBot can unravel the intricacies of complex medical documents, unlocking hidden knowledge and providing valuable insights to users in a conversational manner.

We chose RASA to build our chatbot for its versatility, functionality, and scalability for future healthcare applications. As an open-source AI conversational chatbot tool, Rasa is regularly updated by a huge community of developers and users.

Now, how does Rasa work?


If you didn’t quite grasp the image above, no worries! We’ll break it down in a simple way. The Rasa structure is essentially divided into three main files: intents.yml, stories.yml, and domain.yml.

Intents are hard-coded examples of user inputs that train Rasa to anticipate user messages. A simple intent could be “Hi.” Entities are extracted from the user’s input and stored in slots for later use. For example, if a user mentions “Covid,” it’s captured as a ‘disease’ entity and stored for fetching data or incorporating into the conversation.

Stories contain generalized conversational flows between the user and chatbot. They can be triggered based on the user’s input to respond to their query. In this project, we have stories for Symptoms, Cure, Medication, Expenses, Greet, and Goodbye to make the user feel at home.

Lastly, domain.yml oversees the entire process, storing data related to entities, slots, stories, user responses, and actions.

Intents and Stories


Up until this point, we were developing our chatbot locally. But to progress to the next stage, we needed to deploy our assistant and make it accessible to the world. The Rasa stack can run on-premise or on your own cloud account, giving us flexibility in deploying our assistant.

These were the deployment methods we explored:

Using ngrok

ngrok is a cross-platform application that exposes local server ports to the internet. It offers various advantages as it is easy to use, fast, supports UTP and HTTPS, and also it can be used for free.

Using Heroku and Docker Compose

Heroku is a container-based cloud Platform as a Service(PaaS). Its popularity has always hinged on its simplicity, elegance, and usability. Docker Compose is a tool for running multi-container applications on a single host machine. Docker Compose networks the containers together, allowing the services to reach each other and making it possible to start or stop all containers with a single command.

Deployment process of the chatbot using ngrok on Telegram channel.

Deployment process of the chatbot using ngrok on Telegram channel.

Deployment process of chatbot using Heroku & Docker Compose on Telegram channel.

Deployment process of chatbot using Heroku & Docker Compose on Telegram channel.

Potential Benefits

When the three major outcomes of these projects are implemented at full scale, we could see the following benefits:

Large Knowledge Models: By applying Large Knowledge Models (LKMs), we can derive solutions from extensive language models to comprehensively understand medical terminologies, nuanced phrasing, and intricate medical documentation. We can customize this technology to meet the specific needs of our clients by tailoring the language models and training data to their unique healthcare scenarios.

Cost Estimation: The system can be configured to provide patients with cost-estimation in payor market and health insurance by asking questions in natural language, along with expansions of the NLP based chatbot into other healthcare processes. 

Instant Access to Health Information: The chatbot will help users get medical and health-related information at their fingertips, without needing to wait for doctors or health experts to answer general questions.

Enhanced Data Visualization and Transparency: The dashboard will enable users to visualize health and disease-related data utilized by the chatbot in real-time. It will also help answer questions related to the chatbot’s reasoning and provide better feedback on patient and user behavior to doctors and hospital administration.

Improved Emotional Support: The sentiment analysis provides insights into how users’ emotions are classified as positive, negative, or neutral. This will help determine the level of stress, anxiety, and nervousness among patients and users, allowing the chatbot and dashboard to work together to reduce these negative emotions.

Future Scope

Training on More Data: Improving the Rasa chatbot by training it on more data and incorporating advanced Natural Language Understanding (NLU) techniques to make it more robust and accurate in understanding and responding to user queries.

Integrating Sentiment Analysis: Incorporate sentiment analysis capabilities into the chatbot to enable it to respond to patient inquiries based on the sentiment detected in the preceding remark or query, allowing for more empathetic and personalized interactions.

Expanding Disease Database: Adding more comprehensive databases of diseases and their related attributes to expand the dataset used by the chatbot, making it more generalizable and applicable in a wider range of clinical settings.

Implementing Age-Based Filters: Improving the dashboard by implementing age-based filters using actions that dynamically change the attributes of the health conditions according to the selected age filter, showcasing relevant features while hiding others based on the user’s age group.

Potential Applications in Other Industries

While this project focused on developing a chatbot, dashboard, and sentiment analysis system for the healthcare industry, the underlying technologies and approaches have broad applicability across various sectors. Here are some potential applications in other industries:

Customer Service: Chatbots powered by natural language processing (NLP) can revolutionize customer service by providing instant, 24/7 support to customers. These chatbots can handle common queries, troubleshoot issues, and escalate complex problems to human agents when necessary. By leveraging sentiment analysis, businesses can also gauge customer satisfaction in real-time and proactively address concerns.

Education: Chatbots can serve as virtual tutors or learning companions, assisting students with their coursework, answering questions, and providing personalized feedback. Educational institutions can also use dashboards to visualize student performance data, identify areas for improvement, and make data-driven decisions to enhance the learning experience.

Finance: In the financial sector, chatbots can help customers with account inquiries, transaction history, and even provide personalized investment advice based on user preferences and risk tolerance. Sentiment analysis can be applied to news articles, social media posts, and market data to gauge market sentiment and inform investment strategies.

E-commerce: Online retailers can deploy chatbots to assist customers with product recommendations, order tracking, and returns or exchanges. By analyzing customer sentiment through reviews and feedback, e-commerce businesses can identify trends, improve product offerings, and optimize their marketing efforts.

Human Resources: HR departments can leverage chatbots to streamline employee onboarding, answer common HR-related questions, and provide guidance on company policies and benefits. Sentiment analysis can be used to monitor employee sentiment, identify potential issues affecting morale, and inform initiatives to improve employee engagement and retention.

Want to work with us too?

media card
A Success Story of Our Chatbot Revolutionizing Forest Restoration Efforts
media card
Leading Change in Crisis Management: Our DIMA Chatbot Success Story in Collaboration with DataCamp
media card
Ace Your Next Interview: How AI-Powered Mock Interviews are Revolutionizing Job Preparation
media card
AI-Powered Chatbots Initiative to Enhance Mental Health Support