Using Natural Language Processing(NLP), Voice Recognition and Internet of Things(IoT) Technologies in the Multi-Tier Architecture for Controlling Smart Home Operations
0 likes36 views
The document describes a multi-tier architecture that uses natural language processing, voice recognition, and internet of things technologies to control smart home operations. It introduces an IoT agent that is a web application for monitoring and managing smart home devices. The agent uses natural language processing for voice and text commands to make devices more user-friendly. The system architecture includes microcontrollers that connect sensors and actuators. Testing was done to analyze response times under different network conditions.
Using Natural Language Processing(NLP), Voice Recognition and Internet of Things(IoT) Technologies in the Multi-Tier Architecture for Controlling Smart Home Operations
- 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 222 Using Natural Language Processing(NLP), Voice Recognition and Internet of Things(IoT) Technologies in the Multi-Tier Architecture for Controlling Smart Home Operations Isha Pathania M.E. Dept. of Computer Science and Engineering, Chandigarh University, Gharuan -------------------------------------------------------------------***------------------------------------------------------------------------ Abstract - Along with the arrival of the IOT(Internet of Things),many devices are attaching with the Internet with the aim to provide help to many people. In this paper, I generally introduced an IoT Agent, which is usually a Web application for observing and managing a smart home to a certain extent. TheIoTAgent initiatesachatwhich is used to recognize the voice or text commands using NLP(Natural Language Processing). Along with the usage of Natural Language Processing( NLP), home devices are even much more user-friendly and managing them is a little bit easier, even when a command or the command/question isdissimilar fromthe presets, the system appreciates the user’s desires and act upon accordingly. The solution exploits some of the available APIS(Application Programming Interfaces), whose name are: the Dialogflow API for the well-organized integration of Natural Language Programming(NLP),the Message Queuing Telemetry Transport(MQTT) for the lightweight management of the Firebase and many actuators for the purpose of dynamic data storage. Key Words: Internet of Things, Natural Language Processing, Technology, Smart Home Operations, Voice recognition INTRODUCTION A growing Internet-based architecture is IOT(Internet Of Things), it usually enables the data exchange and also the services that are present in a global network. Along with the arrival of the IOT(Internet of Things),many devices are attaching with the Internet with the aim to provide help to many people. In this paper, we generally introduced an IoT Agent, which is usually a Web application for observing and managing a smart home to a certain extent. The IoT Agent initiatesachatwhich is used to recognize the voice or text commands using NLP(Natural Language Processing). Along with the usage of Natural Language Processing( NLP), home devices are even much more user-friendly and managing them is a little bit easier,evenwhena commandor the command/question isdissimilar fromthe presets, the system appreciates the user’s desires and act upon accordingly. The solution exploits some of the available APIS(Application Programming Interfaces), whose name are: the Dialogflow API for the well-organized integration of Natural Language Programming(NLP),theMessage Queuing Telemetry Transport(MQTT) for the lightweight management of the Firebase and many actuators for the purpose of dynamic data storage. Inthispaper,I have proposed an application for managingthe smart home to a certain extent. In my application I obtained the values of sensors and to manage the actuators I used natural language. In addition, any person could also even use the chatbot present in that main page of my application, with the purpose to communicate with it using free text. For the purpose of training of my NLP (Natural Language Processing) system, I had used an API which is Dialogflow Application Programming Interface. 1. System Architecture Figure 1 which is shown below depicts the comprehensive architecture of my ecosystem, which is comprised of many subparts. The user makes use of the IoT Agent and a web application which can host all of the functionality, could monitor and control the actuators and the sensors , that are unified into the smart home.
- 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 223 Figure 1. Main system architecture. Figure 2. IoT Agent’s welcome page
- 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 224 Figure 3. Generic interconnection of actuators and sensors with microcontrollers. 2. Natural Language Processing (NLP) System My system is an extension of the usage of the NLP(Natural Language Processing). NLP(Natural Language Processing) is a part of AI( Artificial Intelligence) which uses computational techniques to understand, it observe and renew human natural speech. The IoT Agent uses NLP(Natural Language Processing) for the use of voice-chat feature,and for its text-chat system too. Dialogflow is a very amazing solution for NLP’s convenient integration in the IoT system. Dialogflow API is also provided by Google and it uses techniques of machine-learning for the training process. 3. Microcontroller Implementation This section represent the most important logic of the microcontroller’s code which are at the maximum edge of my system. The code for both of the different microcontrollers is written at the Arduino IDE. Ihad divided my code into two separate code blocks.
- 4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 225 4. Web Application Implementation The main Web application is made by the use of the Angular framework. Angular is the common framework which is also developed by Google for the purpose of creating single-page applications which can be used for desktop, web or mobile also.
- 5. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 226 Figure 9. The IoT Agent login screen.
- 6. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 227 Figure 10. The IoT Agent Chatbot environment Figure 11. The IoT Agent Dashboard. Below is the sample of test commands, by using these commands we will find decision-making time of IOT Agent, response time of various controllers and total response time.
- 7. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 228 Table 1. Sample of test commands. Figure 12: Given below illustrates the results. Figure 12. Response time of NLP engine per network bandwidth Table 2. Data processing and decision-making time of the IoT Agent under different access networks. C./B. ADSL(D.:3.93) Fast 3G (D.:1.51 Slow 3G (D.:0.36 Mbps/U.:0.57 Mbps) Mbps/U.:0.45 Mbps) Mbps/U.:0.26 Mbps) C. 1 4 millisecond 7 millisecond 5 millisecond C. 2 2 millisecond 2 millisecond 2 millisecond C. 3 1 millisecond 1 millisecond 1 millisecond C. 4 1 millisecond 1 millisecond 1 millisecond C. 5 4 millisecond 4 millisecond 6 millisecond C. 6 3 millisecond 3 millisecond 7 millisecond C. 7 1 millisecond 1 millisecond 1 millisecond C. 8 1 millisecond 1 millisecond 1 millisecond C. 9 0.5 millisecond 0.5 millisecond 0.5 millisecond C. 10 0.5 millisecond 0.5 millisecond 0.5 millisecond
- 8. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 229 Figure 13 illustrates the results Figure 13. Data –making and data processing time of the IoT Agent. Table 3. Response time of various microcontrollers under many access networks. C./B. ADSL (D.:3.93 Mbps/U.:0.57 Mbps) Fast 3G (D.:1.51 Mbps/U.:0.45 Mbps) Slow 3G (D.:0.36 Mbps/U.:0.26 Mbps) C.1 20 millisecond 70 millisecond 730 millisecond C.2 30 millisecond 100 millisecond 720 millisecond C.5 40 millisecond 50 millisecond 156 millisecond C.6 30 millisecond 30 millisecond 720 millisecond Figure 14. Response time of microcontrollers under different access networks. Table 4: Total response time C./S. Response Time Of NLP Data Processing Time Response Time Of Microcontroller C. 1 2001 millisecond 5 millisecond 730 millisecond C. 2 2001 millisecond 2 millisecond 720 millisecond C. 3 2002 millisecond 1 millisecond 0 millisecond C. 4 2001 millisecond 1 millisecond 0 millisecond
- 9. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 230 C. 5 2003 millisecond 6 millisecond 156 millisecond C. 6 2003 millisecond 7 millisecond 720 millisecond C. 7 2002 millisecond 1 millisecond 0 millisecond C. 8 2002 millisecond 1 millisecond 0 millisecond C. 9 2002 millisecond 0 .5 millisecond 0 millisecond C. 10 2001 millisecond 0.5 millisecond 0 millisecond Figure 15. End-to-end Response Time For Many Sample Commands. Conclusions The IoT Agent initiates a chat which is used to recognize the voice or text commands using NLP(Natural Language Processing). Along with the usage of Natural Language Processing( NLP), home devices are even much more user-friendly andmanaging themis a little bit easier,evenwhenacommandor the command/question isdissimilar fromthe presets, the system appreciates the user’s desires and act upon accordingly. The solution exploits some of the available APIS(Application Programming Interfaces), whose name are: the Dialogflow API for the well-organized integration of Natural Language Programming(NLP),theMessage Queuing Telemetry Transport(MQTT) for the lightweight management of the Firebase and many actuators for the purpose of dynamic data storage. Inthispaper,I have proposed an application for managing the smart home to a certain extent. In my application I obtained the values of sensors and to manage the actuators I used natural language. In addition, any person could also even use the chatbot present in that main page of my application, with the purpose to communicate with it using free text. For the purpose of training of my NLP (Natural Language Processing)system, I had used an API called Dialogflow Application Programming Interface. My system is an extension of the usage of the NLP(Natural Language Processing). NLP(Natural Language Processing) is a part of AI( Artificial Intelligence) which uses techniques that are computational to understand, it observe and renew human natural speech .The IoT Agent uses NLP(Natural Language Processing) for the use of voice-chat feature, and for its text-chat system too. Dialogflow is a very amazing solution for NLP’s convenient integration in the IoT system. Dialogflow API is also provided by Google and it uses techniques of machine-learning for the training process. REFERENCES 1. Ashton, K. That “Internet of Things” thing. RFID J. 2009 2. Wortmann, F.; Flüchter, K. Internet of Things. Bus. Inf. Syst. Eng. 2015.
- 10. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 09 Issue: 03 | Mar 2022 www.irjet.net p-ISSN: 2395-0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 231 3. Xiaoyi, C. The Internet of Things. In Ethical Ripples of Creativity and Innovation; Palgrave Macmillan: UK, 2016 4. Rizvi, S.; Sohail, I.; Saleem, M.M.; Irtaza, A.; Zafar, M. A Smart home appliances power management System for the handicapped, Elder and Blind People. In Proceedings of the 4th International Conference on Computer and Information Sciences (ICCOINS), Kuala Lumpur, 2018. 5. Verspoor, K.M.; Cohen, K.B. Natural Language Processing. In Target Hub—Encyclopedia of Systems Biology; Springer: Berlin, 2013 6.Tharaniya Soundhari, M.; Brilly Sangeetha, S. Intelligent Interface based speech recognition for the Home automation In Proceedings of the 2nd International Conference on Innovations in Information Embedded and Communication Systems (ICIIECS’15), Coimbatore, India, 2015. 7. Milivojša, S., T.; Antić, M.; Smiljković, N. Implementation of Voice Control Interface for Smart Home Automation System. In Proceedings of the 2017 IEEE 7th International Conference on Consumer Electronics, Berlin, Germany, 3–6 September 2017. 8. Han, Y.; Hyun, J.; Jeong, T.; Yoo, J.-H,A Smart Home Control System based on Context and Human Speech. In the Proceedings of the 2016 18th International Conference on the Advanced Communication Technology , Pyeong Chang, Korea,2016