A Review on the Determinants of a suitable Chatbot Framework- Empirical evidence from implementation of RASA and IBM Watson Assistant Frameworks

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 09 | Sep 2022 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 358
A Review on the Determinants of a suitable Chatbot Framework-
Empirical evidence from implementation of RASA and IBM Watson
Assistant Frameworks
Dr. N. Kalyani1, Tabassum Sultana2
1 Professor, Department of CSE, G.Narayanamma Institute of Technology and Science, Hyderabad, India
2Student, Department of CSE, G.Narayanamma Institute of Technology and Science, Hyderabad, India
--------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The rapid emergence and evolution of AI chatbots has been phenomenal. There are countlessframeworksoutthere
that are trying to catch up to each other in order to be the best. From modest start-ups to significant partnerships, these
conversational professionals are utilized in a variety of industries. On the market, there are a varietyofcode-basedand interface-
based chatbot development solutions. However, they lack the adaptability and agility required to create sincere conversations.
Chatbots are currently developed utilizing rule-based techniques, rudimentary machine learning algorithms, or retrieval-based
techniques, however the results are not adequate. It can be difficult to decide which one is most suited to your requirements. The
purpose of this paper is to look into the factors that influence the choice of a chatbot platform between RASA and IBM Watson
Assistant. This paper presents a survey of these frameworks for researchers in identifying the areas of development and
methodology. This study offers a critical examination of these frameworks, with current tactics thoroughly examined and
analyzed. 30 publications from well-known digital databases were analyzed usingasystematicreviewapproach. Inthispaper, an
extensive comparative analysis is carried out using evaluation models for chatbot performance. This survey concludes with
curiosity to know why would we prefer one over the other and what are the future aspects of each. The data is collected from
several resources including 50 respondents from 2 MNC’s dealing with chatbot providing services.
Key Words: Chatbots, RASA, Machine learning, Deep Learning, IBM Watson Assistant
1.INTRODUCTION
A chatbot is an artificial intelligence software. It can communicate with a customer in natural language via informative
applications, websites, and a variety of applications. It can have a simulated interaction with the user in such a way that they
don’t feel like they are talking to the machine directly. They are designed to help organizations maintain track of their client
interactions. It's ubiquitous on popular chat apps like Facebook Messenger,Telegram, RocketChat,andGoogleHangoutsChat,
among others. Despite the fact that chatbots appear to be a relatively new concept, 75 percent of web customers use courier
stages, according to research from the Global OnlineIndex.Itisa pieceofcorrespondenceprogrammingthatmimicswrittenor
voice communication with humans.
The chatbot established in the past maintains a rudimentary conversational stream with customers in the form of a simple
solicitation and response stream. As research progressed, chatbotshavebeen abletorecognizethecustomers'settingsandthe
flow of interactions and respond appropriately. According to Fortune Business Insights, the chatbot market will reach $721
million in 2022. This number may project to reach 3 billion dollars by the end of the decade, based on its current compound
annual growth rate (CAGR) of roughly 22%. Smaller firms are currently using chatbots in largenumbers.Addinga third-party
customer care bot powered by one of the popular chatbot builders is fairly simple. Larger companies, on the other hand, tend
to take a more strategic approach. This pushes them to create their own in-house solution, which prolongs the development
process. Conversational bots, according to 61 percent of executives, boost staff productivity by automatically followingupon
scheduled tasks. (According to Accenture, 2018). Chatbots are expected to provide consumers with 24-hour service (64
percent) and rapid responses (55 percent ). (2018, Drift). Chatbots or comparable technology will automate29% ofcustomer
service activities in the United States. (Tableau). During the COVID-19 epidemic, AI-powered chatbots played a crucial rolein
handling patient demands. The WorldHealthOrganizationestimatesthat4.2billion peoplemightpotentiallybereachedbythe
WHO Health Alert Messenger App and other related communication channels. 2020 (World Health Organization)
1.1 Chatbot Usage and Engagement Statistics
AI advancements enhance chatbots’ ability to mimic human agents in conversation. Contrasting with human-human
conversations, human-chatbot communication is distinguished by noticeable variances in bothcontentandquality. Ahuman–

chatbot conversation can last a long time. It's common for people to speak in brief sentences with few words or even in
poor language (Hill, Randolph Ford, & Farreras,2015). Whilechatbotsarelesscapable oflinguisticinterpretationthanhumans,
the fundamental difference between them and humans is how they detect empathy.However, breakthroughshave beenmade,
and chatbots are becoming increasingly sensitive to their users' emotions.(2018)(Fernandes).Artificial intelligenceandother
technologies are being used by businesses to help them make quicker, more informed choices,increaseefficiency,andprovide
more customized and relevant experiences for both consumers and staff. Conversational bots, like Apple's Siri, Google
Assistant, and Amazon's Alexa, are intelligent virtual assistants that enable third parties to create "skills" or original
conversational interactions by utilizing the AI, NLP, and ML APIs/services offered by these platform providers. Theconceptof
utilizing human language to communicate with machines emerged in the early 1950s. However, at the time, no one could
fathom computers that could react or operate like humans. The lastfewdecades,however,haveseena significantchangeinthe
situation. Even though people still have irrational expectations about AI, it can be said that we have advanced in our ability to
interact with robots. In a number of industries, including healthcare, business, education, and finance, AI technology is
currently used to provide virtual assistance (Tidio). With 1.4 billion people regularly utilizing chatbots, its use has spread to a
wide range of industries (Acquire). The fact that chatbots can respond to the majority of questions that customers may ask
them is one of the factors contributing to this technology's increasing popularity, per data from the chatbot industry. It is
nevertheless vital to have some knowledgeable customer support personnel onstandbyformorecomplexinquiries.However,
a chatbot service reduces costs and improvesresponsetimesforcommonissues.This allowscustomerserviceworkerstofocus
on harder tasks and get a bigger picture (IBM).
1.2 Technologies and Conversational Bot Frameworks
To create a chatbot based on natural language understanding, a variety of technologies and platforms are available (NLU).
Around the same time that the idea of a chatbot became popular, Alan Turing put up the Turing Test("Cancomputers think?").
(Turing, 2009, pp. 23–65). In order to serve as a psychoanalyst and respond to customer questions, Eliza, the first known
chatbot, was created in 1966. (Weizenbaum,1966).Itusedpatternmatchingalgorithmsanda template-basedresponsesystem
to react to the user's query (Brandtzaeg & Flstad, 2017, pp. 377–392). A Chatbot dubbed PARRY was created in 1972 in
addition to ELIZA. (Colby and colleagues, 1971). A prize-winning chatbot named ALICE was created in 1995. The annual
Turing Test award, the Loebner Prize, was conferred to it. It was the first chatbot that was widely recognized as a "human
Computer". (Wallace, 2009, pp. 181–210). It used pattern-matching and Artificial Intelligence Markup Language (AIML) to
define its core operations (Marietto et al., 2013). Current Chatbots developed as technology advanced include SmarterChild
(Moln'ar & Szuts, 2018), Siri, Amazon Alexia, IBM Watson, Cortana,andGoogleAssistant(Reisetal.,2018).Thedevelopment of
chatbots has significantly increased since 2016, leading to the creation of a wide range of conversational systems for use in
industry. The Scopus findings on Chatbot development history are shown in Fig. 1, which was modifiedfrom(Adamopoulou&
Moussiades, 2020).
Chatbots manage fictitious interactions and lack morals and independence(Murtarelli etal.,2021).Athoroughinvestigationof
the various Chatbot platforms, as well as the level of innovation and application of already-existing Chatbots,isrecommended
by Adamopoulou and Moussiades (2020). Hence the study aims to critically analyze two big chatbot frameworks- RASA and
IBM Watson. With the endless chatbot and AI helpers on the market, it is tough to determine which is better and compatible
with the platform on which it will be implemented. Although many reviews have been done on the design, trends and
applications of the chatbots, their primary focus has been to add to the body of knowledge on the variouschatbotsdesignsand
Chart -1- Shows the timeline when people searched Scopus using the phrases "chatbot," "conversation agent," or
"conversational interface" (Adamopoulou & Moussiades, 2020). (CW Okonkwo and A. Ade-Ibijola)

its approaches. This research, on the otherhand,proposestobefocusedontwoframeworks,showingtheon-the-groundreality
of their implementations pointing researchers into potential fields of future research.
This study contributes to the existing body of knowledge on chatbot frameworks in the following ways (specific to RASA and
IBM Watson).
 It offers structured and current information about prior research and their application areas.
 It addresses the primary obstacles related with the deployment of Chatbot systems.
 It outlines the key issues with the usage of Chatbot platform and help in the identification of critical areas thatrequire
enhancement.
Given the depth and breadth of prior research on chatbot platforms, the following research questions have been addressed
using a systematic literature review (SLR) (RQ).
RQ1 - What is the current research status or profile of the two chatbot frameworks?
RQ2 – What are their main advantages?
RQ3 - According to the literature, what are the hurdles observed during implementation
The remainder of the paper is structured as follows: The two Chatbot frameworks are explained in Section 2, the research
methodologies are covered in Section 3, and the results of the search are discussed in Section 4. Section 3 also covers the
results of the study's ramifications. Section 4 of the report concludes with suggestions for additional research.
2. Overview of RASA and IBM Watson ChatBot Platform
Fig -1: General Architecture of a ChatBot
The illustration (Fig 1) above depicts what happens within a chatbot. Despite the fact that practically every chatbot has these
features, they can be categorized into various distinct categories.
A chatbot can be categorized as a personalized bot, a customer support bot, or a functional bot. Each of thesecategorybotscan
respond to one of two deep learning models that can be used to decide the chatbot's design structure. The first one is the
Generative model. Generative models are intelligent Bots thatareusedseldombutare meanttocreatecomplicatedalgorithms.
These bots converse in a human-like manner. Figure 2 shows an example of Microsoft Tay (Deshpande et al., 2017)

Fig 2: Generative model
The Retrieval-based Model is the second. These are simple to construct utilizing pre-built APIs and established user
conversational context. (See Illustration 4) (Deshpandeandcolleagues,2017).Thismodel issimpletoimplement.Itsdrawback
is that the user's inquiry may not always fit into the existing question and answer database.
Fig 3: Retrieval-based Model
One of the most popular Artificial Intelligence chatbot platforms used by developers is Watson, which uses a retrieval-based
paradigm. It offers a variety of tools for creating bots. In terms of capability, integration, and scalability, IBM Watson holds its
own against other significant rivals in the hunt for an AI chatbot framework. It also includes machine learning capabilities. To
develop sophisticated AI chatbots for internal usage, large organizations are using IBM Watson. It has considerably enhanced
reasoning, pattern recognition, computational linguistics, andartificial intelligence.Additionally,itoffersdevelopersenhanced
cognitive capacities. More than 10 different languages are currently supported by Watson, and it also has a built-intranslator.
It contains a tone analyzer that can help with interpreting and identifying positive and negative responses from users and
customers. Its Technology stack and approach pretty much includes NLP, IBM’s Deep-QA software and ApacheUIMA. In2011,
IBM unveiled Watson as a chatbot (Watson Assistant |IBM Cloud, 2020). Onthegameshow"Jeopardy,"whereparticipantshad
to guess the questions that associated with answers they were given,Watsonwasable tounderstandreal humanlanguage well
enough to overcome two previouschampions.Yearslater, Watsonaidedcompaniesin creatingbettervirtual assistants.Watson
Health was developed to help medical professionals with diseasediagnosis (EleniAdamopoulou&LefterisMoussiades.,2020).
On the other hand, RASA is an open-source implementation of the Dual Intent and Entity Transformer (DIET) paradigm for
natural language processing (NLP), implements the DIET model. In order to boost efficiency, DIET employs a sequence model
that considers word order. It is also available in a smaller, more compact form with a plug-and-play, modular design. For
instance, DIET can be used for both intent categorization and entity extraction, or it can be used for a particular task, such as
entity extraction alone. This can be done by disabling intent categorization. Before DIET, Rasa's NLU pipeline utilized a bag of
words model with just one feature vector for each user communication. Currently, RASA consists of two modules: Rasa Core
and Rasa NLU. Rasa NLU analyses user input, categorizes intent, and extracts entities. Rasa core accepts the user's input and
uses several pipelines to provide a response. Rasa is a powerful and time-saving tool for creating complicated chatbots that
works right out of the box. In terms of development, it is clear and adaptable (Indiaai). Rasa is a chatbot structure to consider
for more aggressive roles, with an upgraded NLU motor capable of manipulating important AI associations via text or speech.
Rasa is also free and open source, unlike many other bot systems. It'squiterobustand iscommonlyused bydesignerstocreate
chatbots and context-aware coworkers. Inside the chatbot, one can design, interact, and execute,makingitandthedata itlinks

safer. Furthermore, it gives better control and freedom when running the chatbot. In terms of components, combinations,and
general adaptability, it ranks with the other major chatbots, suchasAmazonLex,DialogStream,andMicrosoftBotFramework.
3. Review Of Literature
Designing a chatbot's objectives, procedures, and user requirementsisthefirststepin creatingone.Thechatbotfunctionalities
are evaluated locally after being built using a programming language or a chatbot application framework. The chatbot is
subsequently made accessible to the public on a website or in a data center, and itislinkedtooneor morechannelstosend and
receive messages. (Eleni Adamopoulou & Lefteris Moussiades., 2020). The services a chatbot should provide to users and the
category it falls under dictate the algorithms or platforms utilized to build them (Nimavat & Champaneria,2017).Thebenefits
of making the proper decision include connectivity, efficiency, quick and easy production tweaks, and less labor for the
designer. It's important to remember that a chatbot is considered more effective if a user may use it directly without
downloading and installing anything. Computer languages like Python and Java, as well as a paid or free chatbot creation
platform, can be used to create a chatbot (Nayyar, 2019). RASA (Rasa, 2019), Botkit: Building Blocks for Building Bots (2019),
and Chatterbot are all open-source platforms. Some NLU cloud platforms (Braun et al., 2017) fueled by machine learning
include Google DialogFlow (Dialogflow, 2019),Facebook wit.ai(Wit.ai.,2019),MicrosoftLUIS(LUIS(LanguageUnderstanding)
- Cognitive Services - Microsoft Azure, 2019), and IBM Watson Conversation (IBM Watson Conversation, 2019).
The Rasa NLU and core were first presented under an open-source license by Bocklisch, T. etal.ina paper. Theobjectiveofthis
study was to create a communication system for non-expert technology enthusiasts based on machine learning and language
understanding. The bundle they created was small and included everything they needed to progress. 244versionsof Rasa had
been produced with a total of 18023 contributions, thanks to the efforts of 344 contributors. Rasa's API integrates principles
from scikit-learn [1] (consistent APIs with various backend implementations) and Keras [2], and both of these libraries are
(optional) components of a Rasa application. The technique to dialogue management used by Rasa Core is most similar to [3],
however it differs from the majority of previous research systems. End-to-end learning, as in [4] or [5,] in which natural
language understanding, state monitoring, dialog management, and answer production are all learned collaboratively from
discussion transcripts, is not currently enabled. Rasa's languageunderstandinganddialoguemanagementareentirelydistinct.
Rasa NLU and Core can be used separately, and learned conversation models can be applied to different languages. Rasa's
architecture is intended to be modular. This makes integratingwithothersystemsstraightforward.Forinstance,Rasa Corecan
be used as a dialogue manager in conjunction with NLU services other thanRasa NLU.Despitethefactthatthecode wasbuilt in
Python, both services may offer HTTP APIs, making it possible for programs written in other programminglanguagestoeasily
access them. In an experiment by Anran Jiao[6], the RASA NLU approach outperforms the NN method in terms of accuracy.
Additionally, the RASA NLU method excels at extracting all entities while analyzing a single word as a whole. However, it was
discovered that the NN approach has superior fidelity when classifying things from segmented words.HeintegratedtheRASA
NLU with neural network techniques to create an entity extraction systemthatcan recognizeintentionsandtheentitiesthatgo
along with them. He also created a practical framework to put the RASA NLU notion into practice. Lacerda[7] used the core of
Rasa in his work and proposed a new software stack called Rasa-ptbr-boilerplate for non-specialists who don't know much
about the internals of the chatbot and treat chatbots like a blackbox.
To be fair, the experiences of the top three chatbot platforms: Amazon, Google, and IBM are all extremely similar.Rasa wasthe
only chatbot platform that necessitates hands-on Python scripting. Although Rasa's documentation seems the most fun, IBM
Watson’s appeared to offer the most comprehensive and easy-to-follow resources. The IBM RedguideTM article (2012)
illustrates how Watson employs dynamic learning, natural language processing, hypothesis generation, and hypothesis
assessment to deliver prompt, accurate responses. It represents a cognitivesysteminaction.Withaccuracycomparabletothat
of a human, it can parse human language to identify connections between textpartsat ratesandscalesthataremuchfasterand
greater than those that a human alone could achieve. It can handle a high level of accuracy when it comes to knowing the
appropriate response to a question. Its pricing starts with the Lite edition, which is cost-free and supports up to 10,000
messages per month. The other fee-based programmes are Standard,Plus,Premium,andDeployAnywhere.Standardoffersan
infinite number of texts for $0.0025 each. Although the price of the Plus plan is not disclosed, one can get a free 30-day trial by
contacting IBM. The costs of the Premium and Deploy anywhere programmes vary.
3.1 Chatbot Performance Evaluation Models
Evaluation of chatbot performance can be done in a variety of ways. Personal assistants, question-answer bots, and domain-
specific bots are some of the different information retrieval (IR)usesforchatbots.The accuracy,precision,recall,andF-scoreof
the chatbot's responses must be measured by the evaluators after they have asked the chatbot questions and made requests
(Cahn, 2017). According to the user experience perspective, the bot's goal is probably to increase user satisfaction. Usersneed
to be surveyed (often via questionnaires on websites like Amazon Mechanical Turk), and bots need to be rated based on their

usability and pleasure. To get close to speech, bots should be put to the test by linguists for their ability to put together
complete, grammatical, and meaningful utterances. The bot that appears the most convincingly human—that is, one that best
passes the Turing Test—is also the most efficient from the perspective of artificial intelligence.
There are many measures that can be used to determine whether a chatbot will be effective, such as the Bleu Score, which
compares a created word sequence to a reference sequence. The BLEU score was first developed specifically for translation
assignments when it was first introduced by Kishore Papineniin2002.TheBLEUscore isdeterminedbycountingtheinstances
in which n-grams of user text coincide with n-grams of reference text. The chatbot's intelligence increaseswithitsBLEUscore.
b) The Turing test is a prominent way of evaluating a machine's capacity to demonstrateintelligentbehavioursimilartothatof
a human. The Turing test indicates that a machine has succeeded if the tester is unable to distinguish between human and
computer responses. c) Scalability: If a chatbot accepts a large number of users and new modules, it issaidtobemore scalable.
d) Interoperability, which refers to a system's capacity to share and utilize data. Multiple channels should be supported by an
interoperable chatbot, and users should be able to move between channels rapidly. e) Speed: When it comes to speed, a
chatbot's response rate measurement is crucial. Chatbots of high quality should be able torespondfast (Pooja Gambhir,2019).
In relation to IBM Watson Assistant, it can assist in overcoming the high learning curve and annoying language employed by
rival virtual agent technologies. Its chatbot design is easy and does not require the need for sophisticated decision trees or
scripting. Watson Assistant is found to be very easy to use and very scalable. The CEO of AdMed, Joan Francy states that the
interface allows anybody to build a chatbot, whilealsoallowingourdeveloperstofullyutilizeWatson'scapabilities(IBM,2019).
However, building a simple question- answer bot is very easy but establishing a communication with an external API can be a
complex process. According to a few respondents, the list of available integrations for the chatbotrevealsitsmajorflaw:itcan't
be used anyplace! Slack is a new addition to their offering, however Microsoft Teams is not supported (which has over 115
million users). Rasa's approach to dialogue management cannot becomparedtoIBM's.Thedialoguemanagement environment
in WA is highly strict. The dialogue has elements such as disambiguation, digression, numerousconditionedreplies,anda large
amount of business logic, resulting in a complicated environment. The Lite plan's metrics are notably lightonspecifics,making
it difficult to even notice where the chatbot's recognition is lacking. In order to make the chatbot user-friendly and useful, one
must first identify the shortcomings. IBM has a solution for it, which is called pay for Plus which could go down the expensive
road especially for small businesses. Furthermore, if the chatbot requires any third-party API connectors, one must join up for
IBM's Cloud Functions service, which, aside from time, costs a lot more money. Nonetheless,IBMWatsonisa market-leadingAI
solution that is acclaimed for helping businesses conduct quick and in-depth research. They can find patterns and insights
thanks to its powerful AI and machine learning capabilities
. Through extensive studies of complicated data, meaningful and actionable insights can be gleaned. It may sound mundane in
today's bot-building frameworks, but Rasa has claimed to be doing something a littledifferent. AccordingtoRasa Technologies,
the architecture enables the creation of contextual chatbots—true AI assistants that do more than repeat FAQ replies. Rasa is
more likely to be used for more ambitious applications, such as bots that can comprehend and respond to highly sophisticated
statements. That being said, the entrance hurdle is minimal. Onecanbeginusingtheframework forfree,asthereis minimal risk
in browsing the Rasa documentation to acquire an understandingforit. Accordingtothestudy[8],RASAismoreadaptablethan
other business softwares due to its scalability and open-source licensing. Jollity chatbot built in Rasa integratedwithTelegram
helps users cope with depression by giving an unseen friend on whom they can rely, as well as the ability to interact with the
bot throughout the day. Various criteria, including intent accuracy, narrative correctness, and confusion matrix, were used to
evaluate the system. Experiments showed that the system had a 90% accuracy rate for intent identification and could retrieve
pertinent responses (Kanakamedala Deepika,Veeranki Tilekya &JatrothMamatha,2020).Accordingtopeerspot,Rasa isplaced
fourth in Chatbot Development Platforms, whereas IBM Watson Assistant is ranked second. However, the majority of ones
selection will be dependent on their previous experience and what they want to accomplish with the bot. The most significant
difference between the frameworks is the development ecosystem and assistance that they best enable. Thus, considering the
advantages and disadvantages can help in motivating users and establishing acceptable standards that will encourage the
growth and application of chatbot technology.
4. Discussion Of Results
The purpose of this study was to conduct a systematic review of the literature on Chatbot technologies (Rasa and IBM Watson
Assistant) in order to gain a better understanding of their current status, benefits, obstacles, and future possibilities. Three
major research topics were identified in relation to the objectives.
RQ1 investigated the current state and profile of both the frameworks. A total of 30 published research publications were
analyzed in order to answer this question. This research also took into account a variety of verified Chatbot Development
Platform evaluations in order to uncover the real-world challenges that developers experience during implementation.

According to our findings, It is far too early to determine in the perspective of computing history if IBM Watson Assistant is
primitive. In a newly released benchmark (IBM,2020), Watson Assistant's new and enhanced intent recognition algorithm
outperformed commercial and open-source competitors. When trained on all training sets andtestedonin-scopeoccurrences,
the upgraded version of Watson Assistant achieves an average accuracy of 73.8 percent across all datasets. According to Jio
Haptik Technologies (2020), Watson Assistant outperforms Rasa by 9.3 percentage points. According to the study, Watson
Assistant competes with pretrained LMs across a wide range of datasets and situations, but trains far faster - an important
component in the usability of a commercial conversational AI service. Rasa, ontheotherhand,allowsopen-source modelstobe
added in the pipeline, such as Transformer-based (Vaswani etal.,2017)models.Rasa-basedchatbotshaveoutperformedanyof
the other open-source competitors.
RQ2 outlines the benefits of their use in the industry. The review has identified and highlighted numerous advantages gained
from the use of these Chatbot frameworks. Some of the advantages of WatsonAssistantincludetheeaseofuse,doesnotrequire
any coding knowledge. A high completion and containment rate is achieved through efficient and targeted encounters that
include rich media. It is designed for worldwide deployment and is built with IBM security, scalability, and flexibility. RASA,
however, falls into the open-source category, which allows for a balance of control over development processes. It takes a
flexible approach to Chatbot development in that the NLU pipeline may be tailored to the problem that the CA will address.
RQ3 identifies some of the primary issues that developers confront while implementing these frameworks. Various
publications verified chatbot platform evaluation websites, and responses from respondents were used in this research to
highlight some of the primary challenges of these frameworks.
5. Conclusion
Chatbots can aid in the implementation of major changes. Customers' interactions with businesses are affected by them. They
influence how client support is handled. They influence how leads are produced and how soon clientsarehelped.Chatbotsare
one of the most human-like ways for a product to communicate with customers and answer their inquiries. We have analysed
the performance of two of the most popular commercial services for developing task-oriented dialogue systems. In practise,
systems designed for designing and deploying virtual assistants must address a variety of scenarios and trade-offs. These
systems must train the best models in few-shot scenarios, strike a balance between training time and accuracy, and readily
adapt to a wide range of domains. In this review it has come to notice that Watson Assistant has outperformed Rasa at many
instances however the answer to which framework to prefer lies in the requirement for it. The stated benefits and constraints
can be experimentally studied in future works to see how they influence Chatbot development.
REFERENCES
[1] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel,P. Prettenhofer,R.Weiss,V.Dubourg,et
al. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12(Oct):2825–2830, 2011.
[2] F. Chollet et al. Keras, 2015.
[3] J. D. Williams, K. Asadi, and G. Zweig. Hybrid code networks: practical and efficient end-to-end dialog control with
supervised and reinforcement learning. arXiv preprint arXiv:1702.03274, 2017.
[4] T.-H. Wen, D. Vandyke, N. Mrksic, M. Gasic, L. M. Rojas-Barahona, P.-H. Su, S. Ultes, and S. Young. A network-based end-to-
end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562, 2016.
[5] A. Bordes and J. Weston. Learning end-to-end goal-oriented dialog. CoRR, abs/1605.07683, 2016.
[6] Anran Jiao, An Intelligent Chatbot System Based on Entity Extraction Using RASA NLU and Neural Network. Journal of
Physics Conference Series 1487(1):012014, 2020
[7] Lacerda, A. R. T. D. (2019). Rasa-ptbr-boilerplate: FLOSS project that enablesbrazilianportuguesechatbotdevelopmentby
non-experts.
[8] Singh, A., Ramasubramanian, K., &Shivam, S. (2019). Introduction to Microsoft Bot, RASA, and Google Dialogflow. In
Building an Enterprise Chatbot (pp. 281-302). Apress, Berkeley, CA.
[9] Bocklisch, T., Faulkner, J., Pawlowski, N., & Nichol, A. (2017). Rasa: Open-source language understanding and dialogue
management. arXiv preprint arXiv:1712.05181.

[10] IBM RedguideTM article -The Era of Cognitive Systems: An Inside Look at IBM Watson and How it Works-
https://www.redbooks.ibm.com/redpapers/pdfs/redp4955.pdf
[11] Cahn,B.J. (2017). [Thesis] Chatbot Literature Review. Retrieved from
https://static1.squarespace.com/static/569293741c1210fdda37b429/t/59160b6bff7c50104e601a85/1494616940469/
CHATBOT_thesis_final.pdf
[12] Pooja Gambhir (2019)-Review of Chatbot Designs and Trends, Conference: Artificial Intelligence and Speech Technology
2019, IGDTUW
[13] Henry Amm- https://www.adenin.com/blog/common-problems-building-ibm-watson-assistant-chatbot/
[14] Nayyar, 2019- Chatbots and the Open-Source Tools You Can Use to Develop Them
[15] Rasa 2019: open-source conversational A.I, Rasa website: https://rasa.com/. (Retrieved 18 November 2019)
[16] Botkit: Building blocks for building bots, https://botkit.ai/. (Retrieved 25 November 2019)
[17] Wit.ai., https://wit.ai/. (Retrieved 25 November 2019)
[18] Kanakamedala Deepika, Veeranki Tilekya, Jatroth Mamatha, 2020- Jollity Chatbot- A contextual AI Assistant, CFP20P17-
ART; ISBN: 978-1-7281-5821-1
[19] IBM,2020- Watson Assistant improves intent detection accuracy, leads against AI vendors cited in published study -
Watson Blog (ibm.com)
[20] Jio Haptik Technologies (2020), 2012.03929.pdf (arxiv.org)
[21] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia
Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NIPS), pages 5998–
6008, Long Beach, CA.
[22] Rakesh Kumar Sharma & Manoj Joshi (2020), An Analytical Study and Review of open Source Chatbot framework, RASA,
http://www.ijert.org ISSN: 2278-0181IJERTV9IS060723 (Thiswork islicensedunder a CreativeCommonsAttribution4.0
International License.)
[23] Drift, 2018- The ultimate guide to chatbots, Chatbots - The Beginners Guide to Chatbot Technology | Drift
[24] Jennifer Hill , W.RandolphFord andIngridG.Farreras-Real conversationswithartificial intelligence:Acomparison between
human–human online conversations and human–chatbot conversations, Computers in Human Behavior, Volume
49, August 2015, Pages 245-250
[25] Eleni Adamopoulou & Lefteris Moussiades- An Overview of Chatbot Technology, Artificial Intelligence Applications and
Innovations, Published 6 May 2020
[26] Deshpande, Alisha Shahane, Darshana Gadre, Mrunmayi Deshpande, Prachi M. Joshi, 2017- A Survey of various Chatbot
implementation Techniques, Computer Science
[27] Ketakee Nimavat, Tushar Champaneria- Chatbots: An overview. Types, Architecture, Tools and Future Possibilities,
October 2017, Conference: International Journal for Scientific Research & Development, At: Ahmedabad
[28] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017)Attentionisall youneed.In:
Advances in neural information processing systems, pp 5998–6008

A Review on the Determinants of a suitable Chatbot Framework- Empirical evidence from implementation of RASA and IBM Watson Assistant Frameworks

More Related Content

Similar to A Review on the Determinants of a suitable Chatbot Framework- Empirical evidence from implementation of RASA and IBM Watson Assistant Frameworks (20)

More from IRJET Journal (20)

Recently uploaded (20)

A Review on the Determinants of a suitable Chatbot Framework- Empirical evidence from implementation of RASA and IBM Watson Assistant Frameworks