A Prototype Navigation System for Guiding Blind People Indoors using NXT Mindstorms

— People with visual impairment face enormous difficulties in terms of their mobility as they do not have enough information about their location and orientation with respect to traffic and obstacles on their route. Visually impaired people can navigate unknown areas by relying on the assistance of canes, other people, or specially trained guide dogs. The traditional ways of using guide dogs and long-cane only help to avoid obstacles, not to know where they are. The research presented in this paper introduces a mobile assistant navigation prototype to locate and direct blind people indoors. Since most of the existing navigation systems developed so far for blind people employ a complex conjunction of positioning systems, video cameras, location-based and image processing algorithms, we designed an affordable low-cost prototype navigation system for orienting and tracking the position of blind people in complex environments. The prototype system is based on the inertial navigation system and experiments have been performed on NXT Mindstorms platform.


INTRODUCTION
Mobility is one of the critical problems encountered by visual impaired persons in their daily life.Over decades, these people were using navigational aids like guide dogs, white cane, or electronic travel equipment.Dogs are very capable guidance for orienting the blind people outdoors, and offer impaired persons with the highest degree of mobility and independence.However this method necessitates an extensive training and selective breeding.Furthermore, trained dogs are only useful for about five years.According to [1], fully trained guide dogs cost between RM20000 and RM50000 per year for training, breed and support.
Due to the development of modern technology, many different types of navigational aids are now available to assist blind persons.They are frequently known as Electronic Travel Aid (ETA) which have been used widely to help blind people while they are walking from one point to another [2].ETA systems are efficient navigation systems; however they require hard training due to the complicated sensors which used in such systems.
Recently, robotic technology has been involved in guiding and navigating impaired people.The use of robotics is a promising alternative to guide dogs.A well designed navigation robotic system could enhance reliability and reduce high cost required for such complicated systems (i.e.ETA).A project named EyeBlind has started in 2012, which mainly aims to build a robotic system which should be able to guide blind people to reach their destinations with the least cost and the shortest route possible.In addition to track and localize the blind person's position, in order to help him/her in emergency situations.
Usually, navigation systems consist of three main parts to help blind people to travel with a greater degree of comfort and independency.First, sensing the instant environment for obstacles and hazards, second, providing information about location and orientation, and third, providing the optimal route towards the desired locations.The system presented in this paper aims to cover the aforementioned three main components in one system, as explained in Section 4.
In the first stage, we aim to study and analyze existing works, then develop a prototype using Lego NXT platform.While in the second stage, we aim to expand the proposed prototype to a real system using complicated hardware devices.
The main objective of this research is to propose, design, implement, and test a prototype navigation system to safely guide blind people to travel from one point to another using the shortest path possible.Our contribution lies on the following aspects: 1. Study, analyze, and categorize the existing systems used to guide blind people 2. Build an indoor prototype system to guide and navigate blind people when they travel between two points using the shortest path possible 3. Test the efficiency of the proposed prototype system indoors.This paper is organized as follows: Section 2 presents the motivation for this work, while Section 3 covers the existing navigation systems.In Section 4, the prototype system is presented and discussed.Section 5 presents the hardware and software platforms, whereas in Section 6 the prototype system is evaluated.Section 7 discuses related works and the proposed prototype system in terms of their strengths and weaknesses.And finally, Section 8 concludes the main work presented in this article and gives directions for future work.

II. MOTIVATION
While visually impaired persons travel between two points, they lack many useful inputs that can be seen by sighted traveler such as location, orientation, and possible PAPER A PROTOTYPE NAVIGATION SYSTEM FOR GUIDING BLIND PEOPLE INDOORS USING NXT MINDSTORMS obstacles ahead.Therefore they would rely on repetitive and predefined routes with the minimum obstacles.Guide dogs or even long canes may help detect the problem.However, blind people need more information to find the shortest path possible, and to safely travel between any two points.
Robotic systems can be designed to guide and navigate blind people to their destinations with no need for guidance from human or even trained dogs.The presented prototype system aims to assist blind people to get wherever they want to go within a specific environment, using voice commands system, and following shortest path possible.As for the motivational force that drives this project, it can be divided into the following main goals: 1. Develop a fully functional prototype robotic system; which will be able to guide blind people to reach their destinations in a short time, and in a safe manner, 2. Localize the blind person indoors, in case of emergency situations (injured, lost, etc…), 3. Provide an easy way of interaction between the robotic system and a blind person, 4. Test the performance of the proposed prototype robotic system, and test the ability of NXT robotic system to meet the requirements of the proposed system.

III. RELATED WORK
Most of the developed projects so far achieved for assisting blind people in navigation tasks combine research progress with technological innovations.Most of experiments have been either directed to indoor or outdoor environments, and considering that blind people have different knowledge of the environment where they act in either well-known, partially known or unknown surroundings.
There are many methods and devices used to assist and guide blind people.Several research works are being performed by several institutions throughout the world.This section reviews various navigational aids for blind individuals and categorizes them based on the technology used to guide blind people.Based on our knowledge, navigation systems can be divided into 5 categories as presented in Figure 1  First, GPS based systems are considered.A GPS receiver is used in this kind of systems to navigate blind persons, according to the longitude and latitude values received from GPS satellites.An electronic device was designed in [3], to navigate blind people with no need for any human assistant.The proposed system consists of the following components: GPS receiver, PIC microcontroller, and voice recorder.This work aimed to fabricate a device which is cheaper than using other complicated and expensive devices.
In [4], a mobile assistant system to locate and orient blind passengers in a Metrobus environment was introduced.A low cost system was developed based on using a mobile phone, GPS receiver, and a compass device to provide a good functionality.Interaction among these devices is achieved using Bluetooth communications.Furthermore, an audible interface was implemented to inform the blind person about his/her location within the Metrobus environment.Additionally, a new system was proposed in [5] to navigate blind people in indoor and outdoor environments, using GPS receiver and a wireless connection.
Second, RFID based systems are taken place in this section.RFID based systems consists of RFID tags and readers.Usually, RFID readers are required to be distributed over the navigational area, while the RFID tags might be attached to each robot device, or vice versa.The RFID reader detects the serial number stored in the RFID tag attached to each robot, and then transmits the location information to a base station in order to compute the robot's location.A navigational robotic system based on a combination of RFID and GPS systems was proposed in [6] which is called Smart-Robot (SR).While the implemented robotic system in [7] based on the RFID technology; RFID passive tags were deployed over the area of interest, whereas RFID reader was attached to the blind person.This work is similar to Blind Interactive Guide System (BIGS) presented in [8].BIGS is an indoor navigation system, and includes scattering the RFID tags and readers in the same way as in [7].
In [9], an RFID robotic system was introduced, which consists of an RFID reader and a laser range finder.RFID technology was used to determine the location of the blind person, whereas the laser range finder was used to find out and obstacles facing the blind person.On the other hand, SmartVision is a development of an electronic white cane that helps blind people moving around in both indoor and outdoor environments, providing contextualized geographical information using RFID technology for indoor environments and GPS for outdoors [10].
Third, sensor based systems are considered in this section.This kind of systems navigates blind people using sensor devices such as ultrasonic, infrared, light sensors, or a combination of the aforementioned sensors.In [11], an earlier tool designed in the early 1960, which consists of a sweep FM ultrasound emitter and three laterally displaced sensors.The proposed work aimed to warn the blind person from any obstacles and walls in his/her way.An assistance system was designed where the goal was to enhance interactions between a blind person and technological devices at home [12].
A sensor module was developed in [13] which can be handled like a flashlight by a blind user and can be used for searching tasks within a three-dimensional environment.The proposed work is based on stereo camera, and ultrasonic sensors.On the other hand, Navebelt system was also proposed to guide visually impaired people, which consists of a belt, a portable computer and an array of ultrasonic sensors placed on the front of the belt [14].In [15,16], an improvement has PAPER A PROTOTYPE NAVIGATION SYSTEM FOR GUIDING BLIND PEOPLE INDOORS USING NXT MINDSTORMS been added to the work presented in [14] through using a mobile robotic technology.
A system called GuideCane was proposed in [17] for navigating blind people indoors.This system consists of a computerized cane which can steer blinds around obstacles, whereas the system proposed in [18] includes a new approach to localize visually impaired people indoors, using cane-mounted sensors, and a foot-mounted pedometer.
Fourth, camera based systems.A camera device is used to guide and navigate blind people by capturing images around the navigational area and then processing them.The work presented in [19] includes a visual navigational system using ultrasonic sensor and USB camera.Also [20] includes a prototype assistive-guide robot (eyeDog) which provides the visually impaired person with autonomous vision-based navigation and laser-based obstacle avoidance function using USB webcam.A new robotic system based on using camera and laser range finders was proposed in [21], where sensors and a laptop were placed on a trolley walker.
Fifth, the Inertial Navigation systems are reviewed in this section, a navigational robotic system is proposed in [1], and a device called ROVI was produced.ROVI is a device that can be used to guide visually impaired people based on using ultrasonic sensors, and digital encoder.The ultrasonic sensor was deployed to estimate the distance between the blind person and obstacles, while the role of the digital encoder was to estimate the distance travelled by ROVI.The work presented in [19] involves a visual odometry system with an assisted inertial navigation filter to produce a precise and robust navigation system which does not rely on particular infrastructure.

IV. THE PROTOTPYE NAVIGATION SYSTEM
The main objective of this paper is to develop a prototype navigation system, which will be able to locate and orient blind people in several indoor environments, in which GPS signals are inaccessible.The system's requirements have been collected from [26].The proposed prototype system is based on two subsystems: the inertial navigation for mapping and ultrasonic sensors for obstacle avoidance.In this section, the prototype system is discussed.The navigation algorithm for the proposed robotic system is presented in Figure 2.
The blind navigation robotic system proposed in this paper consists of four main operations which can be described as follows:

A. Navigational task
It is very important to obtain the location information of the blind person in emergency situations, and in case if the blind person is lost.Since the GPS-based technology is inaccessible indoors, another localization technique has to be adopted instead.Therefore, digital encoders are used to convert the encoder values to a distance travelled by the robot.The encoder detects the hole and no holes on the encoder plate, as presented in Figure 3. Binary values of 0 and 1 are used to define the hole and no holes respectively.Whenever there is a transition from 1 to 0, the encoder tick is increased by 1.

B. Finding shortest path
To travel between two points (let's say a and b), usually, there are more than one route to reach the destination point b from source point a.Blind people should travel using the shortest path to get to their destinations as quick as possible, and to reduce the effort needed to reach there.The Dijkstra's algorithm is used to calculate the shortest path possible between the starting and ending points. Figure 4 depicts the navigational area test-bed, which includes 13 reference points, and all the possible routes between them.The map presented in Figure 4 is translated to a weighted graph, in order to be able to calculate the distance between any two points, since each reference point was considered as a node in that graph, as shown in Figure 5.

PAPER A PROTOTYPE NAVIGATION SYSTEM FOR GUIDING BLIND PEOPLE INDOORS USING NXT MINDSTORMS
At the first glance, the robot checks for all of possible routes from source to destination points.Then, the system compares and computes the shortest path based on the weights assigned to each line (the connection between any adjacent points), where the weight represents the distance between any two points.
In our experimental test-bed, the flag points (reference points) are presented using the numbers 1 to 13, while the intersection pionts are prsetned from a, b, c, to o, as shown in Figure 5.For instance, if the robot is planning to travel from point 1 to 7. Then the possible routes in accordance with the assigned weights would be computed as follows: As presnted above, the first route shows the lowest cost in terms of distance, therefore, the blind person has to follow this route in order to get to the distanation point 7 as quick as possible.Whereas, the second and third routes present high weights compared to the first route, and then the blind person should avoid travelling using routes 2 and 3, as he/she might require additional effort and time to reach there.

C. Obstacle avoidance
An ultrasonic sensor is attached to the robot system in order to detect any obstacle facing the blind person.This sensor can sense any obstacle facing the robot up to 2 meters.As soon as the robot device has detected any object in its way, it will warn the blind person via speaker.In this case the robot will ask the blind person to stop till the detected object moved away.The system will wait short random period of time and check again, if the obstacle has gone, the robotic system will ask blind person to carry on.The ultrasonic sensor will always look for any obstacles along the path.

D. User interaction
A microphone and speaker systems are deployed in this project in order to make the system easy to use.A blind person gives the destination point to the robot via a microphone, while the robot system guides the blind person via speaker.The supervisor (the person who has the authority to monitor blind people) also can track the position of the blind person using the inertial navigation system.

V. SOFTWARE AND HARDWARE PLATFORM
In this section, the hardware and software platforms are described in details.

A. Hardware platform
The main components for our prototype system are the NXT platform, and a laptop.NXT Mindstorms platform has been used widely as an experimental test-bed in several works, as in [23,24,25,26].Figure 6 shows NXT platform along with several sensors (including ultrasonic, touch, microphone, and light sensors).In order to build the proposed prototype navigation system, we used Lego Mindstorms NXT robotic platform.The standard Lego NXT kit contains all the essential mechanical parts and hardware (sensors, processor, and servo motors).One of the NXTs greatest advantages is its flexibility when it comes to programming languages; it supports different level of programming languages (from assembly to python, passing through C, C++, java and .Net).In our project, we used the following sensors: -Ultrasonic sensor: this sensor is able to detect the presence or absence of any solid object facing the robot, and hence warning the blind person from any object facing him/her, -Light sensor: in order to keep the robot stable while travelling between two points, a light sensor might be used to sense the reflected light coming from colored line fixed on floor, -Sound sensor: a blind person must be warned from obstacles in his/her way.Therefore, any message can be translated to a speech, and then transferred to blind person via sound sensor (speaker).

B. Software platform
A system was developed using Visual Basic .Net platform (VB.Net).VB.Net is an object oriented programming language that can be viewed as an evolution of the classic Visual Basic which is implemented on the .Net framework.Figure 7 presents the main screen for the prototype system.
The first part of the software platform involves starting the robot.As soon as the device has been powered on, the system beeps to notify the user that the robot is ready for navigation assistance.Through the speaker, the system may ask the user about the destination point (reference point).As soon as the robotic system obtains the destination point from the blind person, it will check for all possible routes between the source and destination points, and will take the shortest path.At this moment, the robot starts guiding the blind person with directions that he/she can follow to reach his/her destination point.As soon as the blind person has reached the requested destination, the robot will notify the blind person via a speaker.In the second part of the software platform, a voice command (speech recognition) system has been integrated in our project.The idea behind using the speech as an input mechanism is to help the blind person to interact with the robotic system in efficient way.Microsoft speech SDK can be used as a function to accomplish the interaction task.The Microsoft speech SDK can be implemented using C++, C#, or VB.Net.In our project we used the VB.NET programming language to develop the speech recognition subsystem.The blind person may tell the robot about his destination point using a microphone connected to a laptop.The system then recognizes the destination point, and start moving to the requested point.If the destination point was not recognized, the system will ask the blind person to repeat the destination point.The computations take place on a laptop computer that's connected to a robot via Bluetooth connection.

VI. PERFORMANCE EVALUATION
In order to evaluate the performance of the proposed prototype system, several experiments were conducted in a small environment (260 * 160 cm).The performance analysis is done for tracking accuracy, and computing the shortest path possible.The pilot experiments focused on reaching the destination points with the minimum input values from the blind user.

A. Finding shortest path
In this section, computing the shortest path is tested; the robot was placed on a reference point 1, and asked (via microphone) to travel to the reference point 12.The robot started computing all of the possible routes between source and destination points (1 and 12 respectively).Figure 8 shows a number of possible routes (routes 1, 2, and 3); each route is represented in a unique color and shape.

B. Evaluating localization accuracy
The localization accuracy has been tested through three experiments, as depicted in Figure 9.The first experiment took place (between the source and destination points 1 and 12, respectively).The dashed colored line indicates the robot's route.In the second experiment, the robot was asked to travel between the source and destination points 2 and 13, respectively.And finally, the robot travelled from points 4 and 6 in the third experiment.For all experiments, the localization error was between (5 -30) cm.Navigation includes updating the blind person's position and orientation when he/she travels between two points.Guiding blind persons is about augmenting them with contextual information, which usually involves optimal routing and obstacle prompting.Since blind people have difficulty knowing where they are or where are they going, and frequently feeling totally disoriented or isolated, enhancement navigational guidance is very critical for them.In this section, we discuss the existing approaches in terms of their strengths and weaknesses, and argue the proposed prototype system in this paper.
Currently, most of the widespread and used means by visually impaired people are the white stick and the guide dog; however both have some limitations.Guide dogs have been used widely to guide and direct blind people efficiently.However, guide dogs require extensive training before they can be matched with a user.The average cost of a guide dog in the United States, including the necessary training for both the dog and user, is $42,000 [27].On the other hand, the white stick tool alerts the blind from any obstacles facing him/her efficiently, but does not offer navigation, neither orientation facilities.
In section 2, we categorized the navigational systems into five main categories; each one has its own strengths and weaknesses.Most of the existing approaches offer reasonable efficiency in terms of guiding blind people indoors.In this section we summarize the limitations of the existing works as follows: 1. Inaccessibility: in complex environment: navigation systems based on GPS technology cannot be used indoors as in [3,4,5], since GPS signals are inaccessible indoors, 2. High cost: RFID based navigation systems require attaching an expensive RFID reader to each blind person, as in [6,7,8,9].As well as in [17,19,20] which require attaching an array of expensive sensors (such as a camera device) to each blind person, 3. Hard to use: the implemented systems in [6,7,8] require extra setup and training before using the navigation system, and this increases the complexity of this kind of systems, 4. Interaction: in order to offer good functionality, the blind person must find the navigation system easy to interact with.The systems implemented in [1,7,13,20,28], it's very hard for the blind person to interact with, due to the complicated sensors used, and the lack of the interaction functions offered.As a result, the presented prototype system in this paper offers reasonable localization and orientation information; however it has two limitations.First, the implemented system works efficiently in small environments, an extra work has to be added in order to expand the proposed prototype.Second, an extensive effort has to be devoted before using the proposed system in real situations, since a large number of reference points have to be pointed in order to make it possible to convert the infrastructure (environment) to a weighted graph, and this would increase the complexity for the presented navigation system.
On the other hand, the initial prototype was built using Lego NXT platform, the proposed system cannot be extended using this platform (NXT Mindstorms) for the following reasons: 1. Hardware limitation: Lego NXT platform is a good development kit for testing small projects, however, its inefficient solution in terms of cost and reliability when it comes to real situations, 2. Communication: the only communication method available for NXT is the Bluetooth.Bluetooth connection can work for few meters range, and can only establish connection between only two devices, 3. Location tracking issue: in case if the robot is lost, sometimes, it's difficult to find its location based on the inertial navigation system.Additional sensors could be added to the proposed robotic system to enhance the localization task in emergency situations.RFID reader can be attached to a robot while RFID tags must be distributed over the area of interest.However, integrating RFID reader with NXT platform is a complicated task due to NXT's hardware limitations.

VIII. CONCLUSION AND FUTURE WORK
Navigation systems have been widely developed over the past few years.Their range of applications is still growing up, from indoor to outdoor environments.This paper addressed the problem of robot navigation an indoor known environment, through using the shortest path possible between source and destination points.The prototype system presented in this paper will help the blind people to become less dependent on others to commute, if it's implemented in real situations.The navigation robotic system is a work in progress and has yet to be tested extensively.For future work, we aim to expand the proposed prototype to be implemented and tested by blind people.Furthermore, we aim to investigate more technologies which might be integrated to enhance the reliability and efficiency for the proposed navigation system, and to overcome some of the limitations discussed in section 7.

Figure 1 .
Figure 1.Classification of the existing navigation systems

Figure 2 .Figure 3 .
Figure 2. Flowchart for the prototype navigation system

Figure 7 .
Figure 7.The graphical user interface for the proposed navigation system