Appliance Control with IOT-Arduino of Voice Command Detection for Mobility Impaired People

ezwan7101@uitm.edu.my Abstract— People with physical mobility disabilities experience limited ability to perform daily routine. Integration between home appliances and Internet of Thing (IoT) can be used to support these people. The technology allows tasks to be performed much more easily and enable access to applications and ser-vices that would be a challenge with existing digital user interfaces. However, the developed system that is commercially available is costly and equipped with highly specific interfaces which may require considerable technical competency to conduct system interfacing. The purpose of this study is to develop a low cost IOT system for voice-enabled lamp control by utilizing Google Assistant that is available in commercial Android based mobile devices, allowing user to operate lighting appliances without moving. Personal assistance using Google platform embedded with Arduino microcontroller, ESP8266-based NodeMCU development board and integrated Blynk application allow for remote monitoring capability. The system is remotely controlled from a tablet or smartphone to assist disabled people with restricted mobility, in particular those with lower limb disabilities. The system had been tested and demonstrated that the lamp control can be easily accessed by voice command that is then validated using voice pitch analyzer. Test result indicated that capability of the integrated voice control system is verified up to a maximum distance of 24.5 meter, making it a potential implementation in any customized smart home or room system for people with mobile


Introduction
In 2017, more than one billion people worldwide are suffering from mobility impairment. This accounts for 15% of the world population and the figure is projected to increase due to global increase in ageing population and chronic health conditions [1]. For example, 23.8% of European population is expected to reach 65 years or older by year 2030, and researchers are looking for technological improvement in home environment that could help with performing tasks and health management [2]. Locally, 34% of Malaysian is physically disabled, making them one of the largest disabled group after learning disability [3]. Physical disability (also may be described as mobility impairment) may occur at birth due to developmental disabilities or as a result from accident, trauma or any underlying health issues. The limitation of movement due to disability significantly diminishes quality of life, even making the process of completing simple tasks or operating any equipment almost impossible. To improve living conditions, special requirements and preferences have to be met when modifying existing environment to cater for disabled people. Some examples may include considerations in home layout, accessibility of building (doors, window), ease of equipment operations, etc.
With respect to home environment, technological advancement has made its way into this sector by introducing a centralized smart home system that is controllable by user. This system typically involves an interconnecting network of home appliances that communicates with each other and manages collected data through the Internet of Things (IoT). Current technology chain framework that supports the IoT is making enormous progress, making this technology easier to set up and affordable. These improvements may be utilized to support people with special needs to boost their quality of life. However, employing these systems can be challenging due to difficulties for users to operate unfamiliar technology. A group of researchers have showed that for affected group with lower limb disability and mobility issues, voice-based smart home technology is effective in performing tasks [4]. This system based on voice-initiated command instruction can be seen as potential framework to assist mobility impaired people. The system allows capability to control basic operations of home electronic devices at any locations using designated application installed on the mobile phone, or it can be commanded directly in the home environment.
Current research project acknowledges the small number of research projects that have implemented home automation system for mobility impaired people to control lighting appliances with the voice command detection. Previous research has been conducted by Ashraf I. et al. [5] in order to build a mobile app that controls home appliances (including lighting system) for smart home application. The system used Microsoft Azure cloud database and Raspberry Pi database server to communicate between mobile apps and home apps in which the developed multiple GUI-based interface could be of use of disabled people. Another system developed for physically challenged people has been developed by A. Aswathi et al. [6] which implemented voice and gesture-based application with Arduino MEGA-2560. The hardware was embedded with speech recognition module and IOT modules using Wi-Fi adapter to provide wireless signal for implementing the IOT in their embedded system. It was concluded that the effectiveness of the developed system depends on surrounding environment, noise and user pronunciation, and suggested further improvements could be made on the system. Another framework proposed by S. Uma et al. [7] has implemented Node-RED technology and embedded with NodeMCU for IOT implementation in their lamp control design. Information fetching for turning on and off was done through Firebase cloud service.
Benchmark study were performed to examine the effectiveness and suitability of designed IOT frameworks. Table 1 summarised the comparison between existing frameworks. As for this study, an IOT-based voice control technology will be implemented on a smart home system to control lighting appliances. The proposed system provides an affordable and accessible option for home automation by making use of commercially available voice recognition system in smart phones. Nevertheless these interfacing technologies are costly, inefficient, and difficult to operate [13] [14]. The most significant issues with the currently existing systems are their high initial setup costs and unattractive user interfaces [15] [16].
The aim of this study is to develop the IoT voice-enabled device with low-cost controller and open-source platform as end product realization. The study uses the personal assistance using Google platform embedded with Arduino microcontroller, ESP8266based NodeMCU development board and integrated with Blynk application using If This Then That (IFTTT) as the interface between the mobile apps, voice command and the lamp appliance. Improvement in system development with simple open-source platform and user-friendly interface used allows people with physical disabilities control electrical devices without having to physically move in order to manually operate said devices. The controller interface's simple design makes it easy to use and also allows easy customization to fit any personal requirements by the caretaker or disabled users.
This paper is organized as follows: Section 1 demonstrates the introductory background related to the proposed study and literature review of previous related works. Section 2 discuss the system development methodology, in terms of hardware and software of the proposed prototype. Section 3 explains the results and discussion of our research, and Section 4 provides the conclusions of the study.

Material and method
The research method for developing this system is sequenced in stages that was adapted from [17], which comprised of design development, hardware construction, system development followed by integration and validation. Figure 1 illustrates the overall design concept of the proposed voice-enabled lamp control system, integrated with IOT. The system consists of three modules namely input module, controller module and output module. The input module utilizes IFTTT Platform to connect the applets to the Google Assistant voice recognition system, which then will transmit data to controller module that is comprised of Arduino UNO microcontroller and ESP8266based NodeMCU. The output module is composed of LCD to welcome and standby display message, four channels relay with potentiometer as the switching circuit, and light bulb (target device to be controlled). The Arduino Uno was selected as the main controller for processing and transmitting data in this study. The Blynk application provides an interface for monitoring the home environment and controlling the lamp appliance remotely through a button or voice command. ESP8266-based NodeMCU provides the connectivity interfaces between microcontroller and Blynk application. The smartphone that contains this application will transmit input data taken from user voice commands (voice recognition is performed by Google Assistant AI) to either turn ON or turn OFF. With voice recognition, mobility impaired user can control electrical devices remotely. The electrical device tested in the proposed system is a relay-operated lamp. To control the operation of the electrical system, electrical devices will require a relay that operates as a switch. Different types of electrical appliances could be added and customized in the future, depending on the user specifications. The user interface in the Blynk application used in this research is also configured to allow interactions with more than one electrical device.

2.1
Hardware design development and prototype The hardware components are arranged in Printed Circuit Board (PCB) prior to hardware assembly (see Figure 2) and designed in addition to the system architecture. Details description of each hardware components are described as follows:- • Arduino UNO microcontroller: Arduino is a microcontroller board based on the Atmega328P microcontroller, which is an 8-bit microcontroller. There are three different forms of memory for this microcontroller: flash memory, SRAM memory, and EEPROM memory. Six analogue input pins are included in the controller's 14-digit input/output pins (except that these 6-pins can be used as PWM outputs until it lights up. This type of light bulb is selected for this study due to its lower price when compared to LED light bulb. This light bulb will be the output to represent the home appliance that will be voice operated. • Power supply and battery 9V: The specification of power supply for this setup is either 50000mAh battery bank, power adapter connected to the socket or 9V battery in order to power up the Arduino UNO microcontroller board and operate the system.

System flowchart design
The flowchart diagram shown in Figure 3 outlines the algorithm sequence for the developed system. The program initiates by establishing the Wi-Fi connection on the mobile phone and the electrical appliance with the NodeMCU-ESP8266 microcontroller. The established connectivity remains in the standby mode until the system receives voice command from the user. When the smartphone's internal microphone receives the voice command, it will be interpreted by the voice recognition system (Google Assistant). Then IFTTT will verify the voice command and transfer the command over Wi-Fi to NodeMCU-ESP8266 through Blynk platform.
The IFTTT is designed to automate required function (in this case for Blynk App to send out recognized voice command to server) when a certain requirement is met (triggered by specific voice command). To use the IFTTT app, build an applet upon login, then specify the requirement for "This," i.e. the trigger, and then fill in the commands that the Google Assistant should reply to, and this command will be used to operate the appliance/relay associated with it. Additionally, Goggle Assistant's response upon command can also be typed in as desired. After configuring "This", the next step requires setting up "That", i.e. the desired action to be automated. In this case, we will need to send commands to the Blynk Server. For this, input the IP address of the Blynk server into the URL, followed by the Authentication token supplied by Blynk, and finally the pin number of the microcontroller to which the device to be controlled is connected. During operation, after all codes and data are transmitted to Arduino UNO (microcontroller), it reads the command and sends the signal to activate the General-Purpose Input Output (GPIO) pins. The respected relay then operates and turns the lamp appliance ON or OFF, depending on the voice command received.

IOT interfacing with Arduino and Blynk application
Blynk Application also serves as user interface that allow communications of information from button or voice command to the lamp appliance. In this study, this interface technology is utilized to visualize and support IOT by displaying data from activated input button or voice command and the control output of lamp appliance (see Figure 4).

Voice testing, verification, and validation
User voice recognition process is performed by the built-in personal assistant that is commercially available in Android smartphones, also known as Google Assistant. This functionality can be activated by pressing the home button or icon on the phone for two second. Upon activation, the microphone is ready to pick up any voice command from the user. When voice command is received, it was validated using a voice pitch analyzer to identify any spoken phrases. Upon identification, all the command is converted to text and displayed on the mobile phone or to be used in other functions (such as performing Internet search or operating applications).
Implementation of Google Assistant in the proposed appliance control system is done by introducing IFTTT statement command whereby it creates a automated communication between Google Assistant and NodeMCU-ESP8266 through Arduino library available in Blynk platform. IFTTT statement command provides a free software platform that allows web services, devices, and applications from variety of developers to connect with each other in order to automate simple tasks. This method was selected as it is convenient for users to set up their preferred conditional programming as well as the accessibility to various types of applications and services.
During voice identification process, the voice recognition system will convert the spoken command into text. Any phrases of either turn-on or turn-off command (or other custom commands that user has specified) that can be identified by the Google Assistant which is also assigned as IFTTT "trigger" will be verified with the user command. Once the phrase has been validated, the Google Assistant will respond with a confirmation message or a rejection text that is visible to the user. If it matches the requirement for IFTTT, the user command will be sent to Arduino to trigger the relay so that light gets ON.

Result and discussion
The complete prototype was tested and verified for its operational effectiveness. For voice command recognition, the experiment was conducted for four persons to sample with different tone voice. It is recorded with the date and time stamp to record the data with voice samples are tested in 16 different experiments. The system has been tested several times to simulate the frequency of daily use. After experiment was conducted, the IFTTT applet has successfully triggered and converts the voice command to text as shown in Figures 6-8.  Additionally, a test was carried out to determine the system's effectiveness by varying the distance between the speaker and the microphone on the smartphone and recording the respond time. Varying voice frequency is also added as a test variable in order to verify whether lamp control system can respond to different user voices. Voice frequency is measured using the voice pitch analyser apps that is available in the Google play store. The voice verification test was carried out at a maximum distance of 24.5 metres, without any physical obstructions. The user and the lamp are separated by a distance. This measurement is based on the average size of a typical living room terrace in relation to the kitchen. The time stamp successfully appeared during testing as shown in Figure 6 and full experiment results to demonstrate the system operational at different voice frequencies are shown in Table 3. The system response time was measured from the time the command is spoken, until the lamp is lit or turned off, with the average response time being approximately 9.65 seconds (see Figure 9). Among the responses received, shorter distance results the faster execution time of voice command.

Analysis of the system design
The proposed IOT-based lighting appliance voice control system had been successfully tested and verified. Since the lamp was demonstrated to be controllable by voice command up to 24.5m, the system will be capable of assisting mobility impaired person to operate electrical appliances remotely. The development of user control parameters through Blynk interface (see Figure 4) allows for selection and interface customization according to preference of the user and types of appliances to be controlled. Compared to the previous works [5][6][7][8][9][10][11][12], improvement was made in terms of utilizing available open-source platform, and customizable interface. The selected platform and interface are simple, and the ability to modify and add more types of appliances to be controlled allow for greater design flexibility. The suggested system, as presented, attempted to solve the constraints of the existing system and is accompanied with a real-world implementation scenario for validation purposes. The system performance was found satisfactory according to the obtained test results and voice verification.

Conclusion and future work
This research introduces the integration of voice recognition process from the Google Assistant into the smart home control framework through the use of IFTTT and Blynk platform for user-to-appliance communication. The voice-enabled lamp control with IoT technology for the mobility-impaired has been successfully developed and tested. The system is operational with varying voice frequency and voice to microphone distance with the average respond time of ten seconds. The device has been checked and its functionality has been confirmed. Verification of voice command had been done up to a maximum distance of 24.5 m, demonstrating the capability to remotely control appliances. Future work would involve incorporating a tracking database system to track the user activities and appliance usage. Also, current work can be extended to include additional electrical appliances or functions such as television, air conditioning or even water taps.