The Design and Evaluation of a New Smartwatch User Interface

This paper discusses and presents a new prototype design for a smartwatch user interface. The user interface was designed aiming to adhere to some of the main universal design principles and be a more usable design. The prototype user interface was then compared with a Samsung Gear S3 smartwatch user interface. The comparison was done via an experiment and a series of realistic tasks. The aspects being investigated were task times, errors and subjective user satisfaction. The data collected were statistically analysed. The overall results showed that the prototype user interface fostered faster task times, fewer errors and more user satisfaction. Keywords—Smartwatches, User Interfaces, Interaction, Usability, Universal design, prototype


Introduction
The present-day smartwatch and other wearables have a similar physical appearance as traditional watches. The smartwatch and other wearables are revolutionized versions of traditional watches equipped with smart interfaces and complex features. The first computing in wearables began in 1955 and was developed by Edward O. Thorp in 1961. This was a 'cigarette pack sized' device to analyze the motion of roulette wheels [27]. This indicated the potential of wearables resulting in time with the introduction of the first smartwatch. This was the IBM Linux Watch launched in 2000 [19].
Many researchers have concentrated their efforts on producing novel gesture recognition for smartwatches and ways of expanding the small display of a smartwatch, with the aim of making the whole interaction and ownership experience of a smartwatch better (see Section 2 for more details).
Some researchers have also concentrated their efforts on using smartwatches in a very specialized context. For example, the work in [1] investigated the option of using a smartwatch for indoor localization, linked to care of the elderly.
However, to our knowledge there have been fewer efforts aiming to produce a universally designed and usable user interface by means of careful, more thoughtful and rigorously evaluated designs whilst working within the limits of the screen real estate of smartwatches.
In this paper we present a new universally designed and more usable smartwatch user interface. We will show with empirical evidence and statistically significant results, that certain changes to the current design used in a Samsung Gear S3 smartwatch can result in important and useful usability gains. These are achieved purely by careful design and not different gestures or attempts to increase the screen size.
The rest of this paper is structured as follows: Section 2 will present some relevant literature. Section 3 will present the new designs. Section 4 will describe an empirical experiment along with the statistical analysis and Section 5 will conclude with a discussion and conclusions.

Selected Literature Review
This section discusses some of the previous work that has been done concerning smartwatches, small screens and wearable devices. The selected literature will specifically not cover in detail the hardware development aspects as this is considered by the authors to be unnecessary in the context of our work. We therefore concentrate more on the interaction and user interface aspects of smart watches.

Overview of smartwatch interfaces
Smartwatches are categorized as mini versions of computers whose primary function is to collect data and present relevant information, such as notifications and emails [4]. Smartwatches are recognized as a device which is usually worn on the wrist by the user and used to get information conveniently on the wrist and stay connected to the things that matter through synchronization with other devices like smartphones [3]. Smartwatch use has led to certain characteristics of use. One example concerns a survey in [18] where it found that users use the smartwatch more often during lunchtime or when they cannot or are unable to use a smartphone. Similarly, it was found that 40% of women and 30% of men were likely to miss phone calls due to the way they carry the mobile phone which suggests that the mobile phone may not be easily accessible for them despite having it with them [6]. Similarly, wearables like smartwatches have been quicker to be accessed than mobile devices [2]. These aspects can give an indication concerning which features can be included in a smartwatch interface to enable users to easily access information through a smartwatch without accessing the smartphone.
In the context of wearable systems, the work in [24] suggested that it is important for interfaces to be aware of the user's context so that it can recognize and respond to the users' cognitive need and anticipate as required. They further concluded that the interface should be able to maintain the user's attention and not cause information overload.
A study in the context of tasks involving geographical maps, comparing the visualization techniques between various types of display found that among them, small screen devices performed worst in comparison to other bigger and medium sized displays [11]. They used the same content to be displayed on various displays. Some of the results suggest that content at the interface should be designed differently with respect to display size. The work in [21] also suggests that display size matters while designing interactions for various types of displays as interaction zones vary as per display size rather than visualization type.
Another study done to find out why and how people use smartwatches in their daily life, found that smartwatches are able to reduce the dependency on the mobile phone due to the usefulness of availability of notifications in social situations [3]. They also concluded that although smartwatches are useful at specific times, they would require additional functionality if the overall user base was to increase. This suggests that there are still more improvements to be done in smartwatch interfaces to create a rich content smartwatch.

Barriers and usability issues in smartwatch interfaces
Although small screen devices such as smartwatches have gained popularity and are expected to increase in their proliferation, smartwatches still have some limitations which hold them back in terms of usability and user satisfaction.
The design of an interface is linked to the size of the device which means the content of the interface should be designed in relation to the screen size. The screen size matters when it comes to interaction efficiency and effectiveness both in terms of accessibility and usability [22]. Smartwatches are usually worn on a wrist thus interaction is usually done using only one hand. One hand interaction can encounter some limitations and problems due to limited thumb reach, the fat-finger problem as well as when the other hand is occupied or not available [29].
With the aim of overcoming the barriers of small screen display size of smartwatches, various additional efforts and approaches of interaction methods have been investigated. Some examples include, the use of gesture-based input using the wrist as a joystick [7], motion gestures [17], performing tasks through touch-free interaction, solving the fat-finger problem [12], [15], using a camera in a smartwatch to capture finger gestures to use as input [28], use of multi-touch gestures such as pinch, performed on the back of a hand with mid-air gestures [25], gaze interaction to give commands to the smartwatch for tasks [8] and use of the user's skin for providing a larger display [9], [10].
A study of smartwatch usage and usability issues involving 30 participants from 20 to 43 years old, found the following usability issues [5]: • Text font and icons were small in size and easily blocked by fingers while interacting with the smartwatch interface. • It was difficult to read content immediately due to the small screen size and it became more difficult when the content was long. • Information shown on the smartwatch interface display did not provide enough or sufficient details.
• It was difficult to select the targeted object or icons accurately and perform touch gestures like tapping and swiping. • Excessive scrolling was required to find the target from a long list, resulting in lack of user control and need of repetitive interaction to perform the task. • The interaction was not responsive enough to touch input, especially in users' real situations, such as walking. • Difficult to learn and required initial learning to be familiar with the smartwatch.
• Due to different and complex UI design and content display layouts, there were problems in performing tasks easily.
As stated in [5], their study found that the users wanted larger displays in the smartwatch and the accuracy of input interaction to be improved. Further, users preferred to check the weather in the smartwatch. Also, they found that usability of small screen devices such as smartwatches were poor in terms of interacting with visually rich content. Similarly, their study also found that users were not satisfied with the current interface design of the smartwatch and expected new smartwatch interfaces to reduce usability issues.
Touch input is considered as a general or primary input method for smartwatches, but there are various alternative gestures related to the input approach being researched and developed over time. When the various input methods were compared, it was found that touch input modality was faster than other input methods even for linear list search [13]. Ref. [23] indicated that smartwatches have major constraints of having small screens which creates a challenge in defining new instances of input and output capabilities. They further stated that although research has been done, it does not seem to address the constraints of smartwatches properly or in a realistic way. Therefore, they concluded that the screen restrictions of smartwatches will require user interface designers to think of new interaction techniques and designs.
Ref. [20] considers that any user interface needs these following three features to fulfill the requirement of a usability approach: • The interface should be easy to learn and use by the users. • The interface should be able to enable the users to remember the patterns of operational methods of the interface after some time of non-use. • The design of the interface should help the users to minimize making errors while using it and should easily resolve the errors if it is made.
To create usable smartwatch interfaces, in [5] they propose usability principles that can be applied while designing smartwatch interfaces. They proposed several categories in their principles. Their 'Information Display' and 'Learnability' categories are similar to other general user interface guidelines concerning these issues. Their 'Control' category is more specific to smartwatches and contains aspects to do with being able to interact well using a fingertip etc. Their 'Interoperability' category deals mostly with connectivity issues between a smartwatch and a telephone. It also deals with some aspects of the user interface. Their 'Preference' category deals with subjective issues of a user's experience with the smartwatch.
The usability issues of smartwatch interfaces can be divided into two groups, i.e. hardware and software. A study conducted on the usability issues showed that screensize, device, size, resolution, and battery are related to the hardware of a smartwatch, while typography, navigation, button location, iconography, and interaction technique are related to the software of a smartwatch interface [14]. In Ref. [14] it was found that font size was important for older adults and button location should be appropriate with use of proper icon sizes and design. Similarly, interaction techniques should be represented in a simple and effective way for better usability [14].
As this selected literature review has shown, there have been research efforts over a number of years concerning smartwatches, small screens in general and wearable devices. The majority of these studies indicate that the design of smartwatches and particularly their user interfaces need further work if the smartwatch is to become a truly useful and user-friendly device. In line with what has been observed from previous works we will in the next sections describe a new user interface and the results of an evaluation of the new user interface.

New User Interface Design Prototype
Aiming to make improvements to smartwatch user interfaces we began by developing a prototype user interface for the Samsung Gear S3. This was then compared in an empirical experiment with the default Samsung Gear S3 user interface.
The new user interface was developed by being guided by the shortcomings found in the previous work done in this area (see previous main section). The new user interface was also developed in line with the Universal Design principles [26] applied to user interfaces and we note that in [5] their design guidelines have some overlap with the universal design principles.
Aiming for a fair comparison, four popular applications were used in both smartwatch interfaces: Message, Contacts, Call, and Weather.
Each application consisted of a series of screens which included a series of touch interactions for navigation. A screenshot of the new universally designed smartwatch interface prototype with its application is shown below.  (Figure 1, Screen 2). In the 'message' app option, an icon of a message is used followed by the text 'message' just below the icon. This helps the user to identify the type of option that can be selected. The other menu options can be retrieved by the user swiping up or down with touch gestures. Similarly, for a back or previous option, the back button of the smartwatch by default can be used which is situated on the top right side of the smartwatch (image not included). To help the user know the navigation direction through various apps, small arrows are shown at the top and bottom of the screen (See Figure 1, Screens 2-5).
To view a message, the user can touch the 'Message' option ( Figure 1, Screen 2) and go to the message section where the user can view the message of a contact person ( Figure 1, Screen 6). The back option can be generally achieved by pressing the back button of the smartwatch but nevertheless, the option to go back is also provided at the bottom of the message section of the contact (Figure 1, Screen 10). To go back to the main screen, the user can simply press the back button and reach the destination.
Similarly, the 'Contacts' option provides the detailed information of the contact person, including name, contact number and an icon showing the contact's gender ( Figure 1, Screen 7). Further, two options can be seen (Figure 1, Screen 7) which enable a user to either call or message the respective contact. Once the call option is tapped, a screen is shown calling the relevant person with an option to end the call if required (Figure 1, Screen 11). The messaging option takes the user to the message section of the 'Message' option of the relevant contact. This shows the interconnection between options. The screen displays the contact information of the first contact at first. Then the user can use swipe up or swipe down to access the other contacts as required.
Another option is called 'Call' (Figure 1, Screens 4, 8 and 11). This can be used to place a call by dialing a number. When the user taps the 'Call' option ( Figure 1,  Screen 4), a screen opens showing the number system in grid format with a delete button at the top right side and a button to place a call after dialing a number ( Figure  1, Screen 8). The button to place a call is shown in green as seen in most other interactive systems with text such as 'call' written with the icon representing the call function. Once the user presses the desired number and presses the 'call' button, a screen is displayed showing that the call is being placed (Figure 1, Screen 11). Further, the user can end the call by pressing the end call icon shown at the bottom of the screen (Figure 1, Screen 11).
The last option is called 'Weather' which is initialized to present the weather information to the user (Figure 1, Screens 5 and 9). When the user taps the 'weather' option, a screen opens (Figure 1, Screen 9) showing weather information for a location at the top of the screen, temperature on the left and time on the right. There is also an icon to represent the weather condition (e.g. a sun etc.) as well as an equivalent short text label relevant to that weather condition. Furthermore, the day of the week and date is shown at the bottom of the screen. This section includes information for the week which can be visualized by swiping left or right. To help the user to recognize the appropriate gestural interaction, a small arrow is presented to the left and right side of the screen indicating left and right swipe for interaction between various content sections.
The design features that are detailed above and viewable in Figure 1 were aiming to adhere to universal design principles. Universal Design Principle Three: Simple and Intuitive Use [26] states that the design should be understandable irrespective of who the user is. In our design above we have included clear labelling on all icons, rather than relying on users recognizing and/or remembering all icons. We have also given guidance on the direction of swiping, as many mobile-based apps can hide such visual cues causing designs to be unintuitive. The initial screen (Figure 1, Screen 1) also gives guidance regarding how to proceed next. These are all features that contribute to a user interface that should be simple to use and intuitive. These design aspects also link with Universal Design Principle Four: Perceptible Information [26]. This principle is about giving information to the users that is needed and is effective. As stated above the design of our prototype aims to achieve this as described. The design does not rely purely on iconic representations, but these are all labelled appropriately. Furthermore, since most smartwatches have a high degree of customization available concerning aspects such as colours, fonts and brightness levels these therefore feed into achieving Principle Four. Our designs further link with Universal De-sign Principle Five: Tolerance for Error [26]. In part, this principle concerns how various elements are arranged. In this case the elements concern the various screens and interaction steps within the apps. This has been achieved by trying to have clear and simple linked elements and where potential for error exists, we have tried to make it easy to recover from this. One example concerns the possibility of a user dialing a number by mistake. The screen that performs the dialing operation has a clear 'end' function. The relevant screens in addition have small arrows to show the user the correct swipe directions thus avoiding swiping in the wrong direction.
In the next section we present the details of an experiment that was carried out to evaluate if our aims had been met concerning an improved smartwatch user interface.

Participants
In this experiment, a total of 30 participants were included. The participants were university students studying at the master level and some were skilled workers (application developer, designer). The participants were approached with email invitations for participating in the experiment. When they accepted the invitation, then the location of the experiment was given as a choice between the participant's own place or university project room. The experiment was conducted with the presence of the experimenter and participant only.

Experimental design
The experiment for this investigation was conducted using a within users design. This in practice meant the participants were able to experience both user interfaces and in turn provide useful opinions on each interface from a perspective of having experienced both user interfaces.

Variables
The independent variables were the two smartwatch interfaces (Samsung Gear S3, new prototype smartwatch interface). The independent variables consisted of four tasks each which were designed to navigate through four different applications of the interface.
The defined dependent variables were performance and user satisfaction. The dependent measures for performance were task completion time and number of errors.
Errors were defined in two ways. The first one included a user tapping the incorrect option. The second one included a user swiping in the wrong direction. All the errors were recorded combining all the errors into a single score for statistical analysis.
The dependent measures for user satisfaction involved the use of a designed postexperiment questionnaire. The questionnaire was designed to elicit the opinions of the participants in terms usability, interface design, task design and overall experience while using both smartwatch user interfaces. The questionnaire was designed around participants rating their opinions on a Likert-type [16] scale. The scale used was from 1 to 7, where in each question a 7 score was the highest positive score available.

Apparatus and materials
For carrying out the experiment and completing the data collection, the following systems and materials were used: • Samsung Gear S3 with 1.3" circular screen size with (360x360 resolution), Exynos 7270 processor with 768MB RAM

Task design
A total of four tasks were designed to test four applications (see Figure 1) of the smartwatch interface as well as interaction styles used in the smartwatch interface. The main aim of the tasks was to evaluate how users intend to interact with the interface while searching for specific information or completing certain tasks. Furthermore, this research study aims to compare two smartwatch interfaces determining which interface is more usable.
The four tasks are shown in Tables 1 and 2 for each smartwatch interface. Open "Message" application and find the message of "MyCall". B Open "Contact" app and call the contact named "Amrit Sogn Oslo". C Open "Call" app and enter the number "48664579" and press "call" button. D Open "Weather" app and find the weather condition in "Wednesday" Open "Message" app and find the message of "Diana". B Open "Contact" app and call the contact named "Carol". C Open "Call" app and enter the number "48664579" and call button. D Open "Weather" app and find the weather condition in "Wednesday".
Randomization was used to minimize possible learning effects from the participants during the tasks. Therefore, the user interfaces and the tasks were randomized in their order of presentation to each participant. A snippet of the randomization undertaken is presented in Table 3.

Procedure
Before starting the experiment, a pilot test was performed by two students of Oslo Metropolitan University pursuing a Master level degree. The main goal of the pilot testing was to ensure the real experiment could be conducted without any bias and issues. The pilot test participants were provided tasks to perform between the two smartwatch interfaces. After the pilot test two main issues were discovered.
• It was observed that the pilot test participants were able to learn the pattern of tasks through previous tasks, thus potentially creating bias. Therefore, randomization as described in Table 3 was adopted. • During the design of the experiment, it was assumed that the participants may require training before the real experiment. However, during the pilot test, it was found that the participants were able to use the smartwatch interface easily. Therefore, the idea of initial training was removed from the experiment.
The actual experiment procedure was started after the participants were given details about the research study and experiment. The experiment was divided into three sections: pre-experiment, experiment execution and post-experiment. The whole experiment conformed with Norwegian ethical requirements for research of this kind.
The pre-experiment section involved welcoming the participants and then giving some explanation about the experiment. The participants were provided with participant information sheets to provide detailed information about the experiment and a consent form to provide details about ethical issues and confidentiality. Upon completion of reading this material, participants read and signed the consent form. Participants were then given a pre-experiment questionnaire to complete with the aim of gathering non-sensitive background information about the participants.
The experiment was carried out on an actual Samsung Gear S3 smartwatch using the tasks described in Tables 2 and 3. The participants were given the choice to wear the smartwatch on any wrist they wanted and in their preferred physical position. A stopwatch was used to record the time taken and any errors made were recorded manually.
After the experiment was completed, the participants were given a post-experiment questionnaire to complete.

Results
The data collected from the experiment was analysed statistically through a oneway repeated MANOVA test. This was with the aim of observing if there would be statistically significant differences between the two user interfaces for performance and user satisfaction. Performance involved task times and errors. User satisfaction was based on the responses given by the participants via the post-experiment questionnaire. The testing involved the correlation relationship check, normal distribution test, and one-way MANOVA test. All the data collected passed the correlation relationship check and normal distribution test. To keep the paper as brief as possible, only statistically significant results are discussed in this paper. Total time taken for task times, Table 4 presents the means and standard deviations. In all cases the prototype user interface fostered faster task times than the Samsung Gear S3 user interface. The one-way MANOVA test for total time taken shows that there were significant differences between the two smartwatch interfaces for Task 1 (F (1, 58) = 10.120; p < 0.05; ƞp 2 = 0.15), Task 2 (F (1, 58) = 23.413; p < 0.05; ƞp 2 = 0.29), Task 3 (F(1, 58) = 16.270; p < 0.05; ƞp 2 = 0.22), and Task 4 (F(1, 58) = 9.456; p < 0.05; ƞp 2 = 0.14). Clearly all four tasks were completed significantly quicker when carried out with the prototype user interface.
Total errors made for errors, Table 5 presents the means and standard deviations. In all cases the prototype user interface fostered fewer errors than the Samsung Gear S3 user interface. The one-way MANOVA test shows that there was not a significant difference in terms of total errors made during the completion of Task 1. However, there were significant differences in terms of errors made during the completion of Task  Concerning the questions about pleasantness of the user interface, the content being read with ease, the tidiness of the content, ease of performing the tasks, ease of learning the user interface, and stressfulness in using the user interface, no statistically significant results were observed.
The fourth question concerned whether the font size was of the right size for the participants. Overall, the font size in the prototype smartwatch interface (mean = 4.90; SD = 1.242) was scored more positively than the Samsung Gear S3 smartwatch interface (mean = 3.93; SD = 1.230). The difference was statistically significant, (F (1, 58) = 9.179; p < 0.05; ƞp 2 = 0.137).
This section has presented the statistically significant results from the data collected from the experiment. The performance data collected was statistically significant showing the prototype user interface fostering faster task times and fewer errors. The subjective opinions data from the post-experiment questionnaire was also in most cases statistically significant, showing the prototype user interface fostering more positive opinions. In the next section we will discuss the results and present some conclusions.

Discussion and Conclusion
This study aimed to find out whether the smartwatch interface if designed and developed using careful design and universal design principles can provide better usability and user satisfaction. In this study, a Samsung Gear S3 smartwatch interface was selected and compared with a newly developed universally designed smartwatch interface prototype.
The evaluation results in the previous section show categorically statistically significant results for performance aspects concerning task times and errors. The prototype user interface fostered faster times and fewer errors. There were also categorical-ly statistically significant results for subjective satisfaction aspects. Nine out of the 15 questions suggested participants were significantly more satisfied with the experience of using the prototype user interface. The other six questions were non-significant in nature. However, the scores given by the participants in these six questions were all more positive towards the prototype user interface when compared with the Samsung Gear S3 user interface scores. The actual Samsung Gear S3 user interface achieved no statistically significant results across any of the factors measured in the experiment.
These results stand out to suggest clearly that employing careful design, appropriate human computer interaction (HCI) techniques and universal design principles for smartwatch user interfaces will lead to a better-quality user interface and better user experience. We specifically concentrated on universal design principles three, four and five in the design effort. In line with these universal design principles, we designed the new smartwatch user interface to avoid ambiguity, guide the user's interaction flow and where there could be the possibility of some error happening, clear visible ways to recover. The results were categorical with the prototype user interface outperforming the Samsung Gear S3 user interface.
Our work makes some significant contributions to the field of HCI. Firstly, while investigating new gestures and methods for expanding screen real estate are interesting and should continue, our work suggests that good usability and user experience can be achieved without these specific areas of investigation. This should be of interest and of value to any corporation producing such products as it means they can make large improvements to their products by simply being more careful in how they design their smartwatch user interfaces. The second significant contribution we suggest is that to our knowledge no one has designed, evaluated and documented a new smartwatch user interface design where the design was strongly guided by the universal design principles.
In future work, we would like to have larger participant numbers for an experiment. The study provides some variation in the age groups only so we would like to add more participants with various impairments to achieve wider reaching conclusions. The interface designed in this study is simple with only four applications, we would therefore like to add more complexity and add more applications for testing and evaluation. Similarly, in this study only touch interaction style was used, in future other forms of interaction methods and various feedback methods can be applied and evaluated. Lastly, longer term studies could give further information regarding the user interface design.