Paper—User Identification Based on the Dynamic Features Extracted from Handwriting on Touchscreen... User Identification Based on the Dynamic Features Extracted from Handwriting on Touchscreen Devices

This research presents a methodology for user identification using ten English words handwritten on smartphone and mini-tablet. This research considers three features, namely Signature Precision (SP), Finger Pressure (FP), and Movement Time (MT) that were extracted from each of the ten English words using dynamic time warping. The features are then used individually and combined for the purpose of user identification based on the Euclidean distance and the k-nearest neighbor classifier. We concluded that the best identification accuracy results from the combinations of the SP and the FP features with average accuracies of 74.55% and 69% were achieved on small smartphone and Minitablet respectively using a dataset of 42 users Keywords—User identification, user identification on smartphone, security on smartphone, Dynamic time warping, dynamic features, mobile computing, and handwriting based finger on smartphone.


Introduction
The widely usage of smartphones makes them one of a frequent storage medium for the users' sensitive information such as personal data, email, and credit card numbers/passwords. As it is expected at any time to lost our smartphone devices or to be stolen, and this will lead to the disclosure, improper modification, or unavailability of information, which may incur expenses (loss) or missed profits for the users, the problem of securing the system from unauthorized user access to these data considers one of paramount importance [7] and [23].
On smartphone touchscreens, the methods used to unlock screens can be categorized as the most current access systems prompt for users to authenticate themselves such as: text-based password, graphical based password, or grid-based schemes [14]. This authentication method relies on the password /username's secrecy. If this secrecy is not breached, the assertion is that these tokens uniquely identify a legal user. The problems of user authentication associated with the integrity information that consists of maintaining password secrecy are well understood. Passwords that consist of common words, or terms associated with a particular user are generally considered weak because of the relative ease with which malicious users can guess them [13] and [21].
The strength of the authentication method will be influence by the input methodology of touchscreen devices and the different expectations of user for the users' patters and interaction models [7]. As mentioned in a study [20] over 3.3 million weak passwords, as number of their passwords was still "123456". Moreover, the additional cost makes biometric authentication techniques to be still unpopular on mobile devices [7].
Each user can be distinguished by his/her pattern in handwriting that will be an effective tool in the identification processes. In handwriting, text consists of characters, or words, therefore the different end goals of Optical Character Recognition (OCR), and user identification, they have a close relation. Where OCR is used to recognize the characters/ words, while the users focuses on the very special features that determine the patters which leads to identify the user. Researchers used different methods that target a specific category of identification like online and offline, or based on the data content like text dependent and text independent [22].
The main motivation of investigating the effect of dynamic features of handwriting on smartphones-based finger for user identification is preventing unauthorized users from breaching the unlock screen of smartphone touchscreens and then to prevent accessing to the system content. As mentioned early, the way of using touchscreen for unlock screen is preferred for users, as they are easy to remember; using text-based password, graphical based password, and grid-based schemes. The need to enhance the authentication users on smartphones is required in order to make the access on smartphones for an illegal user impossible; because of this need, we examined fingerbased features on smartphone touchscreens for user identification using ten English words. As the English words were used in our study because of their important features that we could analyses, and also for the common way used by users for signatures in the bank. Our study relay on the users' patterns and interaction methods on touchscreen, so breaching unlock screen for smartphones will not be easy even if the password also was stolen. In our study, the extracted features were analysed individually, and combined features in order to enhance the authentication users, and then to increase the identification accuracy.
The contributions of this research can be highlighted as follows. First, analysing the power of three dynamic features, namely Signature Precision (SP), Finger Pressure (FP), and Movement Time (MT) in identifying individuals. Secondly, the widely available consumer devices were used in our experiments. Thirdly, our research provided the research field with new database consist of 2520 trials using different English words that covers most of English characters. Fourthly, the way on how we analysed data is different from previous studies, as Dynamic Time Warping algorithm was used to calculate the similarity between trials, and Euclidean Distance was used to calculate distances between training dataset and testing dataset. As well as the two aspect of analysis for individually, and the combined features were considered in this research.
In the results, we concluded that the best identification accuracy results from the combinations of (SP and FP) features on small smartphone and Mini-tablet were 74.55% and 69% respectively . However, we found out that not all features help in increasing identification power.
The rest of this paper is organized as follows. Section 2 discussed the related work on which we prepared our study. Section 3 presented the study methodology. Section 4 explained the user identification processes. The data collection processes are described in Section 5. The experimental results are explained in Section 6. As well as the conclusions are presented in section 7.

Related Work
Handwriting on smartphones-based finger for user identification is increasing day by day because text or graphical based passwords and grid-based schemes are easy to recall and stolen from a malicious user. As it has attracted researchers recently for the importance of the results of the user identification on smartphones, especially when using handwriting based finger in some fields such as biometric authentication, and forensic expert decision-making systems. In addition, finger-based handwriting on smartphones for user identification has greater potential to provide a more dynamic movement for users on screen for using some a specific feature (e.g. SP, FP, MT) than other used in text-based passwords.
Number of researchers are working on the concept of user identification using offline text independent; some of them also introduced new ideas to provide more secure approach related to text or signature passwords. One of the studies that proposed approach in offline text independent for user identification was conducted by [18]. In their study (i.e. [18]), the handwriting image is segmented into word regions, and they used six public databases in the experiments, where their results showed that the accuracy was from 98.8% 100%. The most recently study conducted in the field of offline text independent for user identification using Arabic and English language was conducted by [1]. One of their (i.e. [1]) stages called segmentation stage, where in this stage the images were segmented into regions in order to be ready for the identification stage. The results referred to a superior performance on Arabic and a competitive performance on English.
Also, in the field of offline text dependent for user identification using handwriting. The researchers in [4] proposed a method for Kannada handwriting. After removing the noise from the scanned images, a KNN was used for 18 Kannada words that was written 5 times using 25 users. The results showed that the accuracy for the top 10 identifications was 90%, while the rest of the words resulted an accuracy rate ranging from 53%-75%.
Some of the researchers' efforts were put on an online text independent using velocity of the pen movement and sub-character level features that was conducted by [3]. Euclidean distance was used to calculate the similarity measures, and the Neural Network based classifier for classifying each curve primitive. Their data was collected from 12 users on 5 different scripts; Arabic, English, Hebrew, Devanagari, and Cyrillic. The results showed 85% for user identification. The most recently study conducted in the field of online text independent for user identification using Arabic and English language was conducted by [17]. They proposed a new approach to improve the performance of the Vector of Local Aggregate Descriptor formulation. For the evaluation of their system proposed system an SVM was applied on two publicity online databases (IAM, IBM-UBI). The results for the identification was 96.3%, 94.37% for IAM, and IBM-UBI respectively.
The most similar filed of studies to our work that is in the field of online text dependent, there is one study conducted by [6] to propose an approach for user identification and verification system based on short words. In their study, four Spanish word were captured by using a WACOM INTOUS A4 USB pen that were collected from 370 users, 3 times for each word. The dynamic features were used in the study: (x and y coordinates, timestamp, button status, azimuth, altitude, and pressure). After segmenting the strokes, two of features were excluded: (timestamp, and button status). They used DTW to compare between words. Dissimilarity measures of words were combined to create a single word-level measure, and an opinion level fusion was applied on these single word-level measure. The results showed 92.38% identification rate when using one word, and 96.64% when using more than one word.
While in the field of the combined methods, the researchers in [5] conducted a study for online and offline approaches. In Off-Line Handwriting they took the structure of number of pixels and the linear size, covered grid boxes of linear size, and in online they took the advantage of the dynamics of writing, the speed of movement of the pen, the accelerations, the exercised pressure, and the temporal order of the writing, which is a valuable factor in the task of writer identification. Their data collected from 110 users. The obtained results showed that the identification rate for one word was 82.7% in TOP1 and 97.3% in TOP10 for the off-line features, while it was 86.4% in TOP1 and 98.2% in TOP10 for the on-line features. However, the average identification rate of all words was about 70.3% in TOP1 and 85.9% in TOP10 for the off-line approach, and 73.1% in TOP1 and 86.5%in TOP10 for the on-line approach.
While other researchers concerned about the unlock screen-based coordinates on touchscreens. One of early researches focused on the possibility of applying keystroke dynamics and typing patterns for user identification. Keystrokes were used as samples by intercepting output from a keyboard [7]. But Mäntyjärvi, et al. in [12] examined identifying people by their gait using accelerometers worn. Also, the researchers in [11] and [8] examined user identification using gait recognition. The researchers Koreman, et al. in [9] proposed a multi-modal biometric for user identification.
Grid technique-based schemes are also examined in the body of the literature, which uses recall method. This technique allows a user draws the password on a 2D grid, and then the information of an occupied grid (e.g. coordinates) will be recorded. The user will be authenticated when drawing touches, the grid in the same order. In order to enter the password correctly and distinguishable, the drawing must be sufficiently away from the grid lines and intersections [7] and [10].
Some the latest studies were conducted for authentication users to unlock interfaces on smartphones using gestures. The researchers Xu, et al. in [19] examined an authentication biometrics for 32 users on slide, pinch, handwriting, and finger-based keystroke that involves a series of taps on the soft; on-screen keyboard. A classification algorithm of Support Vector Machine (SVM) was used in the analysis. The researchers considered the data of the position, pressure, and size of a touch, as well as a timestamp in order to calculate the accuracy and error rate as two straightforward metrics. They concluded in their study that touch operation can be a form of good biometrics. And they found that there is still room for the accuracy to reach up to 100%, and it is a promising solution to consider a join of a set of touch operations for making an authentication decision rather than using one at a time.
In what follows, we will present the study methodology used in this study.

The Study Methodology
This study was conducted on handwriting for 10 different English words on two smartphone devices, as follows:

Devices for data collection
Two devices used in the study are: 1) Samsung Galaxy Ace S 5830 (dimensions 112.4 x 59.9 x 11.5 mm, screen size 3.5 inches) represents Small screen size; 2) Samsung Galaxy Tab 2 (dimensions 193.7x122.4x10.5 mm, screen size 7 inches) represents Mini-Tablet of screen size.

Participants
A total of 42 participants took part in the experiments that consists people from age 20 to 65+ years old, this diversity of age shows that this study can be applied for all ages of users.

English words
The 10 different English words were chosen in this study were used to cover most letters used in the English literature, which cover 20 letters, as shown in Fig. 1. In what follows, we will discuss the processes on how the data was analysed considering the algorithms used in this study. http://www.i-jim.org

User Identification Processes
As each user of 42 has six samples of writing ten different English words on smartphones. On small smartphone the training dataset consists of (22) samples, and the testing dataset consists of (110) samples. On mini-tablet the training dataset consists of (20) samples, and the testing dataset consists of (100) samples. Fig. 2 below shows the numbered steps used in user identification study on each device of smartphone, as follows: 1. The data were collected from the users who were involved in this study based on their finger movement on the surface of touchscreens. 2. Feature Extraction. The collected data were entered into set of analysis processes to produce SP using (DTW), MT, and FP. 3. Part three of Fig. 2 reviews user identification processes in three subsections.

Fig. 2. Users identification processes
(3.1). Training. One trial of six was used in the training dataset for each user across ten of English words; as each feature has ten different English words, and this will be 30 if the three features were combined (SP, FP, and MT) and so on, as shown in Fig. 3 that shows feature factor of all trials we collected for study's features in the training and testing datasets. The matrix D [10 x N] in the Fig. 2 represents training dataset for one feature, where one feature means that (10 English words) and N users. (3.2). Testing. The remaining five trails of six for each user are in the matrix M [10 x Z] and considered to be testing part, where one feature means that (ten English words) and Z is (number of users multiplying with 5 trials of each user). (3.3). User identification. In this part the ED was used to calculate the distance between the matrices D [10 x N] and M [10 x Z], while KNN used to compare the trail in M [10 x Z] to which exemplar in D [10 x N] is the user belong to. In the next section, we will discuss the protocol used for the collected 10 English words from the 42 users.

Data set and Experimental Protocol
The dataset of this research consists of 42 participants, where each participant was asked to write repeatedly 10 different English words for six times for each word that producing a total of 60 trials per participant. Therefore, the total number of trials collected is 2520, which increases to 7560 when implemented the combination of three features. The English words used in the research are illustrated in the next paragraph.
This research includes ten English words (i.e., Cat Arm, School, Pun, Grass, Hi, Queen, White, Dark, and Jog) as shown in Fig. 1. Where each user was asked to write a word on the surface of smartphone' touchscreen as he/she write usually. Following the instruction of [16], the participants were asked to perform a task as accurately as possible.
In what follows, the results of this research will be discussed considering the user identification accuracy and performance variations across different users.

Results and Discussions
The results in this research were analysed individually, and in combined features for the accuracy of user identification, as follows:

Accuracy of user identification on individually features
In individually features, Fig. 4 shows FP on both smartphones devices has high accuracy result 70.91% and 70% on small smartphone and mini-tablet respectively compared to other individually features. This indicates that this feature (i.e. FP) could be used largely to provide stronger results accuracy. Then, the second-high accuracy can be SP. We will investigate if the combination of these two features (FP and SP) can enhance the accuracy in the next section.
The results also show that MT has the least discrimination power, which can be mainly due to how quickly the users perform the task, as this feature needs more investigation in next experiment. If partly, the users are not performing the tasks in same level of time, the MT will decrease the accuracy.

Accuracy of user identification on combined features
In term of combined features i.e. feature-level fusion, Fig. 5 shows the results on both smartphones for two combined features: SP and FP have high accuracy result compared to other combined features, where on small smartphone is 74.55% and on minitablet is 769%. This is for the high influence of features (e.g. FP, and SP) on results than any other influences (e.g. screen sizes). The second-high accuracy result of the combined features are when using all features together (i.e. SP and FP and MT). The results prove that the FP and SP have increased the influence also on accuracy for the combined features, this is because the MT has influenced negatively on the accuracy as mentioned early in the individually features section regarding the MT influence. This indicates that the combined features have high accuracy results compared to the individually features, and this provides evidence that the combination can enhance the accuracy results, this agrees with the study [2] and [14] in the field of the influence of the combination of features on the accuracy results.

SP
To the best of our knowledge we have not came across a study conducted for biometric identification on smartphones using 10 different English words that cover most of English letters, the way on how the data were analysed using DTW and ED algorithms to calculate the accuracy in order to prepare the features to the user identification processes, as well as the way on how we combined features to enhance the accuracy that were not considered in the previous studies.

Conclusion and Future Work
This research presents methodology for user identification using 10 different English words on smartphone and mini-tablet based finger movement on the surface of touchscreens, which improves user identification. The individually, and combined features were considered in this research to identify the user. The trials of extracted data were compared with trusted user values using ED and KNN.
The results provide evidence that the combined features can enhance and provide good level of accuracy results, where not all feature help increasing identification power and some of features has high accuracy result influence such as the combination of FP and SP. Therefore, not including the time element in the identification task led to decreasing the identification accuracy when MT was combined with other features (i.e. SP, and FP).
In the future work, we are planning to consider the percentage of power discrimination of users for the combined features to improve the user identification accuracy, as well as to consider the time condition when performing a task by users in the experiment.