Application of Smartwatches in Elderly Care with Indoor Localization Functionality

—The recent achievements in mobile technology and wearable OS makes possible to create comfortably wearable and very capable smartwatches. They have many different sensors and powerful hardware combined with general purpose OS and all this available for reasonable price. It makes it ideal device for elderly care. Monitoring the elderly’s basic health condition is very straightforward, but using smartwatch as an indoor localization device, monitoring the motion activity, recognizing the typical motion patterns of wandering is not simple. Even those watches are really capable devices, they are not equipped with direct indoor localization sensors and we would like to avoid installing special equip-ment’s, markers, transmitters in the home of elderly. Using only a commercially available smartwatch hardware for indoor localization is a challenging task, several filtering and data processing algorithms needs to be combined in order to provide acceptable indoor localization function. The algorithms, their connection and fine-tuning methods are explained in this article.


Introduction
Over the last decades the average age of the population is shifting towards the older ages. Also, the high mobility of the younger population leaves elderlies without their families, without their informal caregivers. Taking care of the elderlies puts higher and higher burden to the society. Therefore, any support which extends the period of elderlies independent living conditions and reducing the amount of care services using IT equipment is welcomed from the society. Even with some chronic disease, for example with mild dementia the elderlies might continue their independent life but some kind of monitoring of their health and physical status is required to ensure their families or caregivers about their good condition. In this situation the recent developments of mobile technology can help to monitor the elderlies and in the case of sudden change of their health and mental condition the monitoring system can issue an alarm message to the caregivers. In this way the independent living period of elderlies can be expanded and the same time the human effort in their care / monitoring job can be reduced.
For the efficient monitoring of the elderlies several sensors are required. It is advisable to monitor their heart rate, activity, indoor and outdoor position and detect emergency conditions like fall detection or wandering. Considering the sensors and the elderly users the simple and easy to use sensors are essential. One of the most obvious devices to use as a sensor is a smartwatch, since elderlies are used to wear a watch therefore it is one of the most well accepted devices among possible sensors and a recent smartwatch is equipped with all features required for monitoring the health of the users.

Smartwatch in Elderly Care
Recently the application of smart devices in the field of elderly care is an intensively researched area. The current achievement of the smart mobile devices, IoT, low energy communication technologies and MEMS sensors makes possible to produce small, smart and wearable device for monitoring humans mental and health condition. For the elderly care the wearable devices are preferred [1] since they provide the most accurate results without compromising their user's privacy which would be the situation in case of using cameras. The [2] draws attention to the requirements and possible difficulties of using smartwatches in elderly care. Some of the main points are the price and bulky, not fashionable design of smartwatches. The price tag of the smartwatches is getting lower and lower so it will not cause significant problem in the near future but it should be mentioning the low price is true only for the mass-produced devices. This means only the off-the-self smartwatches can be used for the elderly care monitoring without any additional and custom hardware component in order to maintain the low price, the functionality of the smartwatch only can be expanded by software. The other significant problem is the less fashionable design of the smartwatches especially compared to regular watches. Hopefully with the miniaturization and evolution of the smart devices this disadvantage will disappear as well.
Many of the proposed application uses smartwatch only as a fall detection and heart rate sensor [3]. The [4] even steps further it proposes neural network for the fall detection and unusual activity detection.
Beside the heart rate, fall detection and basic motion detection the smartwatch can also provide some other functionality. In the case of early stage of dementia, the wandering can be a serious problem. If the mobile device has GPS the outdoor localization, geofencing [5] can be achieved easily.
Using activity recognition and indoor localization the wandering detection or significant changes in the activity, mood of elderly can be detected [5]. Analyzing the position of the user and time spent in each room of the house the wandering can be predicted. The Fig. 1 shows two different activity patterns. The diagram shows different type of rooms and total time spent daily in each room (the radial distance of the points). The change of the shape of the polygon might indicate the changes in the behavior of the elderly. The increased area of the polygon is the precursor of the wandering. In order to implement this functionality a reasonable precision (at least room precision) indoor localization algorithm is required and as mentioned earlier it must be able to work using only the sensors of the smartwatch without any additional hardware. The indoor position also can be a useful extension information in the case of fall detection, especially in larger buildings or nursing home. In this case knowing the location of the person in trouble might help to the emergency personal to find the elderly people in the fastest possible way.

The Elderly Care System
The elderly care system developed by us has several functions. It can continuously monitor the basic health state of an elderly by measuring heart rate, activity, step count. It also can detect some dangerous situation, so it has fall detection functionality built in. For elderlies living with early stage of dementia, it provides outdoor geofencing and also indoor localization and wandering prediction. All monitored data is continuously analyzed by the system and in the case of emergency situation or when immediate help is required an alarm message is issued to the caregiver or any family member.
The sensory part of the system is based on a smartwatch. Recently very capable watches with all required sensors and also with a versatile Android operation system are available for relatively low price. We are using Kingwear KW88 smartwatch [7], it has all required sensors and communication modules like heart rate, acceleration (motion and activity detection) sensors and Wi-Fi, 3G GSM communication module and a built in GPS receiver. These features make possible to implement all desired functionality like outdoor localization and geofencing, indoor localization, basic health monitoring and data communication. The smartwatch uses Android 7.0 operation system, which makes easy to use all sensors and features of the watch by developing a custom application.
The watch in general has another advantage, since elderlies are used to wear watches, they can comfortably wear it, they know how to handle it and they are not afraid of it as usually happens when they start using a new technology-oriented, modern device. The elderly care system is composed of several components, the block schema is shown on Fig. 2. The central component of the system is the Smartwatch server which is basically responsible for all data processing, storage and analysis functions. It receives data from the Smartwatch via the Wi-Fi (indoor mode) or 3G GSM (outdoor). The data transfer is done in blocks in order to reduce communication workload and reduce power consumption. The smartwatch automatically switches to outdoor communication mode when none of the predefined Wi-Fi network is available.
The indoor localization needs some preparational work. The digital representation of the floorplan needs to be created and the field strength map needs to be measured.
The field strength map (fingerprint map) can be created by using a mobile phone with our custom measurement application and walking thru all places of the indoor environment where the localization function will be used. The result of this activity is the building map which is stored on the server and used by the localization algorithm.

Fig. 2. Schematics diagram of the elderly care system
The server also provides interface for the caregivers, relatives and doctors. Using a standard web browser, they can access all information about the physical and mental health condition of the users of the system.
The Smartwatch is responsible for the data collection. The data collection has two modes: indoor and outdoor. The mode is determined by using the availability of some predefined Wi-Fi connections. If predefined Wi-Fi connections are available then the watch switches to indoor mode, if they are not available then outdoor mode will be used. In outdoor mode the watch uses the built-in GPS for position determination and checks the position against a predefined geofence polygon. The geofence polygon can be defined on a web interface simply drawing it onto a map.
The watch provides an SOS button for the immediate help request and detect fall event, geofence violation onboard and sends an alarm SMS to the caregivers and reports the alarm condition to the server.
Most component of such a system is straightforward to develop since the smartwatch can provide most of the data by using its sensors and a server application can store, process, analyze the data and can provide access to all information. There are two exceptions. The fall detection algorithm is based on the analysis of the acceleration sensor values and several examples can be found in the literature how it can be done [8][9] [10].
The indoor localization subsystem is the second exception which is not directly supported by the smartwatch, it has no specific sensors for this task and there is no commonly available method, standard or devices to implement it opposing to the GPS which is a well-known and widely used standard for outdoor localization. The complete indoor localization must be developed based on the available infrastructure, sensors which are not designed for the indoor localization purposes, so many data processing, filtering and proper localization estimation algorithm needs to be developed. In indoor mode the watch uses our Wi-Fi RSSI based localization algorithm and provides the user's position better than room accuracy. The position calculation and processing are done on the server, the watch is only responsible for the Wi-Fi scan collection.

The Indoor Positioning Algorithm
Since there is no widespread and standardized solution for the indoor localization many researchers working on creating a feasible solution. There are many proposed possible solutions but there is no generic approach, only specific solution for specific application is proposed.
There are a few technologies applicable for indoor localization, all of them have specific advantages and disadvantages. Probably the simplest solutions are based on proximity sensing. In this case there are markers or signal transmitters installed to the known position of the map. The localization device must be able to receive the signal or recognize the markers if it is close enough to them. If the proximity is detected the system will know that the position is close to the predefined position of the signal source. The signal type can be almost anything which can be emitted and detected easily. The most popular technology is based on RFID. This solution is well suited for robot or any autonomous vehicle indoor operations, since it is easy to install RFID tags on to the robots and readers in the surrounding environment [11]. The RFID technology has several application forms, depending on the placement of the transmitters and receivers, the number of used signal sources and receivers. Their performance varies depending on the configuration. In the best case it can provide sub meter accuracy [12] for the expense of large number of used RFID tags. This solution has some drawbacks, since the RFID tags needs to be installed and if we want to use mobile phone as a localization device a special receiver needs to be interfaced to the mobile device. The localization infrastructure cost can be lowered if the indoor localization doesn't need a special device to be installed or devices installed for other purposes can be reused as an indoor localization transmitter. The visible light communication is an example of using different purpose build infrastructure for localization [13]. The localization is based on sensing visible light emitted by a code modulated LED light sources with unique codes. It is a proximity-based position estimation but using visible light as a communication medium, since lights are very often used in public buildings even in daylight, the corridors, large internal spaces are often using continuous lighting. A drawback of this system is the additional light receiver, since a regular mobile device can't sense the visible light with the possibility of decoding the light source identification code.
The other group of indoor localization methods is the dead reckoning or more precisely the pedestrian dead reckoning. It continuously monitors the movement (walking) of the user using the built-in motion sensors of the mobile phone and estimates the distance travelled from the motion pattern [14]. The main advantage of this solution is that the only required device is a regular smartphone, no special transmitters needs to be installed and no additional devices needs to be interfaces to the mobile phone. The disadvantage is the long-term inaccuracy of the methods, since the errors are building up continuously. The precision can be improved by using some constrains from the building map [15].
A very popular method is using infrastructures build for other purposes than indoor localization for the position determination. For example, the wireless communication networks, the Wi-Fi network is a good candidate to use for localization. It is available virtually everywhere, so there is no additional installation and maintenance cost and practically all smart devices are capable to receive Wi-Fi signals. Using pattern matching [16] a simple indoor localization can be created. The disadvantage of this method is since the Wi-Fi network is not designed for indoor localization the signal fluctuations might cause significant error in position. In order to achieve stabile and precise results more filtering methods and algorithms needs to be applied.

Fig. 3. Localization algorithm data flow
Our indoor positioning algorithm is based on particle filter [17] but it is expanded with several filtering procedure in order to improve the precision of the localization. The localization relies only on the previously measured Wi-Fi field strength map (fingerprint map) stored in the building map database and the actual Wi-Fi field strength scan and motion information comes from the smartwatch. Both kind of data acquisition is natively supported by the Android operation system. The Wi-Fi scan API provides the RSSI and MAC address of the visible access points and the motion detection API determines if the user is walking or is in stationary position. All this information is transferred to the server where the actual localization algorithm takes place.
The data flow and main parts of the algorithm is shown on Fig. 3. The algorithm can be divided into two major part. The first part is the rough localization which consists iJIM -Vol. 15, No. 05, 2021 the building and floor determination. The second part is the fine localization which is done on one floor using particle filter algorithm and provides the final position information.
All of the methods are using the building map information, the digital floorplan as well as the field strength map. They are however accessing the AP information after a strong filtering and AP selection method.
In order to speed up the localization several internal fast data structures are created such as k-d tree for floor determination and quadtree for field strength vector search.

Building and floor determination
The building determination is quite simple; therefore, it is not shown on the figure. It searches for the MAC addresses scanned by the smartwatch in the fingerprint database which contains several building information. The building which has the most matching MAC addresses is selected for the localization. Once the building is determined some cached data structure is built up in the memory in order to make localization to real-time. One of the most important structure like this is the RSSI k-d tree. It is a multi-dimensional lookup data structure which is built from the AP information visible on each floor. Five consecutive Wi-Fi scans are used to determine the floor, all of them has a floor index from the k-d tree and majority voting is used to determine the final floor index. The majority voting over five scans provides acceptable stability in the floor determination and still provides quite fast response.
Once the floor is determined other cached data structures are built that specific floor in the memory. The basis of all other procedures is the AP filtering algorithm and data structure.

AP filtering
The AP filtering is important since the Wi-Fi based localization is suffered from many types of errors [18]. The AP filtering consists two stages: the concatenation of subnetwork APs information and filtering out the non-relevant APs from further processing.
One physical AP might serve many Wi-Fi networks, which is when scanned by the smartwatch looks like different APs, since they have different MAC address. However, from the localization point of view they should be handled as one AP since they only provide one location information physically, there is only one physical distance between the AP and the smartwatch and this distance must be considered only once. Therefore, it is essential to handle subnetwork APs as one unit thru all other stages of the localization algorithm.
The first part of the AP filtering is searching for the subnetwork APs and joining their information into one virtualized AP. This can be achieved by searching for high similarity between the measured RSSI maps of each AP. When the field strength difference is less than 5dBm over at least 80% of the measured position the two APs considered as the same physical AP with different subnetworks. Width this method the subnetworks can be recognized by 95% reliability by testing it on hundreds of measurements.
The second important step in the AP filtering is filtering out the irrelevant APs. In every Wi-Fi network very often appears weak or temporary AP from the outside of the environment (e.g. from neighbors of a building) and some devices (e.g. printers) provides local, temporary access points. They must be excluded from the localization in order to improve the stability of the position calculation. For filtering them out, the algorithm marks the three strongest APs in each measured position. The APs which was not given any mark will be left out by the AP filtering stage and will not be used for localization calculation.

Location determination
After preprocessing the Wi-Fi scans with the AP filtering algorithm, the particle filter-based position determination algorithm takes place. The particle filter [17] uses several position assumptions (particles) and an iterative procedure adjust their probability (weight) based on a two-stage predictor-corrector method. The rough data flow can be seen on Fig. 4.

Fig. 4. Particle filter operation
The algorithm has several stages but basically it is a predictor-corrector method. The first step is the resampling the particles based on the previous observation results. The observation is this case means the Wi-Fi field strengths vectors matching which is calculated at the last step of the algorithm. The particle represents a position takes higher weight factor if the measured field strengths vector has shorter Euclidean distance from the previously measured (stored in the building map) vector. The resample is done in random order but the probability for one particle to be selected in the resampled generation is proportional with the particles' weight factor. The next stage is the location prediction. Usually, the indoor localization uses dead reckoning in the predictor stage [19]. Using the inertial sensors of the mobile phone they try to estimate the actual motion vector (walking direction, step) of the user. And adjusting the last known position of the particles with the current motion vector will provide the new location estimation. However, in the case of smartwatch this method cannot be applied, since the smartwatch is not equipped with compass and gyro, it has only acceleration sensor. This means only the steps can be detected but the direction of the movement can't be accurately estimated. Therefore, the prediction stage consists a random movement estimation with random direction and random step length. This calculation is shown in equation (1).
In this equation the ⃗ is the position vector (with x,y position components) at the actual iteration step ( ) or the next ( + 1) iteration step. The ⃗( ) is a function which gives a random vector with random direction and length. In the generation of the random vector ⃗( ) the motion state of the user is considered. Since the limited sensory resources of the smartwatch the motion state has only two values, the user is walking or not walking. This information can be determined from the acceleration sensor and the Android operation system of the smartwatch provides an API function to access to this information. This motion state is represented by which is = when the user is walking and = when the user is not walking. With this modification the random vector calculation can be written as shown on equation (2): The ( , ) function is responsible for the random vector generation with a random length using the expected value ( ) and has a normal distribution with the variance ( ). When the user is walking, a random vector is generated. This vector uses the length of the average human step length which is calculated from the time step between each iteration step multiplied by the average human walking speed. The average human walking speed is 1.2m/s in our case [20]. For the walking we have found the variance = 4 gives the best result. In the non-walking case, the random vector has zero expected value and much lower variance = 2 which is determined to cover the positioning errors only since the user is not changing the position.
Since in our application the room precision, stability is required for the proper wandering detection, more filtering must be applied. The prediction alone might move thru the position over the wall into another room which will reduce the precision.
Therefore, as a last step of the prediction phase we apply the geometrical constrains over the predicted motion vectors as shown on Fig. 5. The figure shows a section of a floorplan at a certain stage of the particle filter. The floorplan is defined in digital vector format at the preparation stage of the indoor localization. It contains the geometry of the building as well as all properties of each geometry block. Using properties, the type of the geometry can be described and the pass-thru feature can be determined. The initial particles are shown in green color and predicted position is shown as small vectors. If the particle prediction trajectory crosses a wall the particle weight is reduced to the minimum value by the predictor (red particles), if it not crosses any geometry or the crossed geometry is a door or other open structure the probability is not changed by the predictor (green particles).

Fig. 5. Geometrical constrains
After each iteration step of the particle filter, the weighted average of the particles is calculated and this position is used as the final user position.

Evaluation
The evaluation of the smartwatch as a sensor in elderly care were done from two important aspects. One of the most important performance to be measured is the battery life time or daily operation time. This is important since many smart devices can run only for a day or less with one charge. In elderly care the whole day running is essential since charging the device in every few hours would be very inconvenient. The minimum expectation is the whole daytime operation with charging the device in each night. The Table 1 shows the operation time which is almost a daytime when all sensors are operating in indoor environment but without fall detection. The fall detection is continuously monitoring the acceleration sensors value and this background operation has significant effect on battery power. The only optimization what were done in this functionality is using the Android built in motion detection API. Using this method, the fall detection is only turned on when 'significant motion' (as per Android API) is detected. When the watch is relatively motionless the fall detection thread is suspended in order to save battery power. Even with this optimization the battery barely last for one day.
The outdoor, GPS based localization is heavy power consuming function. Turning on the GPS will reduce the battery lifetime to half or less. Therefore, the outdoor operation is limited, only few hours of outdoor operation can be powered from the battery.
The other evaluation factor is the precision of the indoor localization. This is critical since for proper wandering prediction at least room precision is required. The Table 2 contains not just the mean localization error, but also the percentage of missed room identification for each test scenarios.
The tests were done in two environments in an office and nursing home. The office as usual has many Wi-Fi access points installed compared with the nursing home. We had to install few more Wi-Fi APs in the nursing home to cover the whole area with acceptable precision. The mean localization error is almost the same in both environment the denser AP installation provides slightly better precision so the error cannot be further reduced by simply increasing the number of APs. The missed room percentage is measured by a one-day operation, moving the smartwatch between various rooms. Without the geometrical constrains the missed room percentage is very high, but using the constrains the Wi-Fi field strength fluctuation will not change the position over rooms.

Conclusion
The results of the implemented indoor localization algorithm show the smartwatch not just can be used in elderly care as a heart rate and activity sensor, but it also can be used as a localization device. The indoor localization works without any additional hardware device only the existing Wi-Fi network is needed. The indoor location information using the appropriate filtering and data processing algorithm has enough precision for stabile room localization and this precision is a good bases of the wandering detection.
The smartwatch also provides outdoor localization and geofencing functions efficiently but usage of the GPS significantly reduces the battery lifetime. We expect the rapid development if the smartwatches will solve this program in the near future.
Finally, it can be concluded that the smartwatch can be an important sensory element of an elderly care system it can provide essential information about the physical and mental health state of an elderly people.