An Instrumental Assessment of Touch‘n Go eWallet Mobile App

—This study assessed the instrumentality of Touch ’n Go eWallet mobile app at selected areas in University Malaysia Perlis (UniMAP) and Politeknik Tuanku Syed Sirajuddin (PTSS) Perlis, Malaysia, in July and August 2019. Fifteen staff from the two institutions was selected as participants. The pur-pose of the test was to assess the usability of the app and get user feedback to improve the instrumental quality of the application in order to meet user satisfaction and their experience. This report contains the participants’ feedbacks, task completion rates, ease or difficulty of task completion, time on task, errors, and recommendations for improvements. This study used 4 tasks to assess the instrumental quality of Touch ‘n Go eWallet mobile app. Overall, the outcome of the study revealed that the app is generally usable and instrumental to assisting users accomplish their electronic wallet goals. There is however some observed issues in the app that require fixing to enhance the instrumental quality of app.


Introduction
Touch 'n Go eWallet is an integrated mobile application. It uses a physical Touch 'n Go Card. The app allows users to add in Physical Touch 'n Go card number easier to track their transaction. Other functions available with the application to make daily use easier are as follows: i) Transfer money to other Touch 'n Go eWallet users; ii) Reload mobile prepaid; iii) Pay for utilities and postpaid bills; iv) Purchase movie and flight tickets; v) Dynamic QR code payment at participating Touch 'n Go eWallet merchants; and vi) Pay tolls via RFID feature and TNG Card feature. An instrumental assessment is the evaluation that ascertains how an application is instrumental to or successful in enabling users to achieve their goals in an interaction with or in their use of an application. Usability evaluation is one way this can be done. Usability testing for mobile applications is highly important for both entrepreneurs and users. A well-tested software application helps to establish reliable and fruitful collaboration between business owners and prospective customers. Evaluative improvement of a user interface is crucial for achieving high usability and instrumentality quality [1][2][3][4][5][6][7]. In this study, the researchers conducted a usability assessment of the Touch 'n Go eWallet mobile application using participants' mobile phone. The mobile phones have in-built video camera to record the participants' action, comments and navigation choices. The participants were free to choose any place for testing the mobile application. The usability testers were present in the selected areas. The session captured each participant's navigational choices, task completion rate, observations, questions and feedbacks.
Usability testing includes establishing a baseline for user performance, establishing and validating user performance measures, and identifying prospective design issues that need to be addressed to improve product effectiveness, efficiency, reliability, and productivity as well as the satisfaction of end-users. Usability testing offers the opportunity of finding out how well the application goals are being achieved [7][8][9][10][11][12][13][14][15]. The aims of this study are inter alia: i) To evaluate the usability of Touch 'N Go eWallet applications; ii) To recommend improvements to the design of the Touch 'N Go eWallet applications where necessary. This research will assist developers to identify the strength and weakness of the application. Implementation of the suggestions and recommendation made will go a long way to improve the application and increase business reputation and customer engagement.

Methodology
Sessions: Each participant was allowed to choose and decide where to conduct the testing/evaluation. This could be in their personal rooms that is conducive and quiet or in any other place. The participants' rooms should have at least one chair and one table in order to make the participants more comfortable. In addition, pens were needed. All user interaction will be captured by using the smartphone video camera. This will record how users perform the tasks given. The participants were informed that it is the eWallet application that is being tested and not their performance per se as participants, and that the goal of the assessment was to report anything that impedes their experience of using eWallet during their interaction activities and the carrying out of the tasks. Participants were also told to think-aloud while carrying out their tasks, reporting anything that impacts or impedes their experience. They were also told to feel free to comment on things they like or do not like about the eWallet application, thereby encouraging them to speak more. The test administrator recorded each session while watching the behavior of the participants. If any of the participant was not talking sufficiently, the test administrator will help such participant by asking him/her some brief questions, endeavoring to be as impartial as possible (e.g. by requesting "How do you find the link icon so far?" instead of "Do you discover the controls difficult?"). Thus, such questions are not to suggest answers to the particular participant. After the task sessions, the post-task questionnaire was administered and completed. Lastly, after task completion, users were asked to answer the questionnaire provided to collect their feedback. The measures collected are: i) Demographic Information, ii) Satisfaction Assessment (Objective metrics & Subjective metrics) and iii) Recommendations. The allocated of roles and responsibility for each test session is as follows: Two members of the research team were saddled with the following responsibilities: i) Briefing participants on the scope of the usability testing; ii) Observe each participant in each test sessions; iii) Interview participants after their testing; and iv) Recording of the participants' test sessions. On the other hand, the participants have the following responsibilities too: i) Test the functions in Touch 'n Go eWallet Application; and ii) Complete a data sheet form [16][17][18][19][20][21][22][23][24][25][26].
Participants: The targeted user group is the Touch 'n Go eWallet users. The test group consists of 15 persons (5 staff members from Tuanku Syed Sirajuddin Polytechnic and 10 staff members from UniMAP) from various units. The test was conducted from July 2019 to August 2019. Below is the demographic information for the selected participants: of the 15 participants, 1 (6.70%) was male while the other 14 were female (63.30%). Two of the participants' age ranged from 26 and below (13.30%), one participant fell into the 26-35 age bracket (6.70%). Most of the participants' (10) age ranged from 36-45 (66.70%). Lastly, two participants were aged 46 and above (13.30%).
Evaluation Tasks / Scenarios: The usability test for Touch 'n Go eWallet Application tested several functions. These include: i) Check Account Balance; ii) Perform money transfers to a favorite eWallet user; iii) Reload eWallet (FPX Online Banking, Credit or Debit Card, Touch 'n Go eWallet Reload Pin, Auto-Reload); and iv) Perform payment at selected attached merchants. The participants were not facilitated on how to use the application. It is assumed that they have previous experience with smartphone applications and are not technology illiterate. The participant's responsibilities are to complete a set of tasks given by the test facilitators and presented to them before the application is tested. They are to perform the several tasks at selected Touch 'n Go eWallet merchants to get their feedback regarding the application. The participants are free to test the application in their own way.
Usability Metrics: Usability metrics refers to user performance measures measured against specific performance goals necessary to satisfy usability requirements. This explains the instrumentality of the app (that is, how instrumental the app is to users in enabling them accomplish their tasks). In this study, task completion/success rates, adherence to dialog scripts, and error rates was used. Time-to-completion of tasks was collected [17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34]. Table 1 displays the metrics selected for this Touch 'n Go eWallet application. Testing Requirement: The hardware requirement for this test is a smartphone while the software requirements include the following: i) Touch 'n Go eWallet App; ii) Screen Recording App; iii) Smart Phone Video Camera.
Testing Analysis: In each session, the participants completed the tasks in a reasonable time and their actions were recorded. The testing proceeded even if the participant faced any failure. The failure was however recorded. The test tasks are as follows: Task 1: Check account balance-This task will demonstrate how the user will check the account balance in the Touch 'n Go eWallet; Task 2: Reload eWallet (FPX Online Banking, Credit or Debit Card, Touch 'n Go eWallet Reload Pin, Auto-Reload) -This task will demonstrate how the user will reload their Touch 'n Go eWallet account using selected reload option; Task 3: Perform money transfers -This task will demonstrate how the user will transfer money from their Touch 'n Go eWallet account to a favorite eWallet user; Task 4: Perform payment at selected attached merchants -This task will demonstrate how the user will perform payment at selected attached merchants.
Data Collection Instruments: There are several forms used in order to get feedback from the participants during the test: i) Data Collection Sheet: The data collection sheet is used to get feedback from the participants after all the tasks have been tested; ii) Consent Form: The Consent Form was created to obtain the consent of the users to participate in the study. The test procedure and their role as testers are clearly stated in the form. The testing activity is voluntary and no participant is coerced. Participants are informed that they can opt out of the study at any time they so wish.

Results
Task Completion / Success Rate: All participants (100%) successfully completed Task 1 (check account balance). This is the simplest task that participant tested since the account balance displayed at the top of screen with a big font size with "RM" label right after the apps is opened. In addition, all participants (100%) were able to accomplish Task 2 (reload money). This task requires participants to log in to their personal account to reload their eWallet in a secured way (mentioned in the apps). The account balance then updates right after the finishing of the reload process from a selected option (FPX Online Banking, Credit or Debit Card, Touch 'n Go eWallet Reload Pin, Auto-Reload). Similarly, all participants (100%) also successfully completed the Task 3 (perform money transfer). A task completion message is delivered by the application. All participants managed to finish Task 4 as well (perform payment at selected attached merchants). Task 4 appeared to be a challenging one because participants were required to present themselves at the selected merchant, but the process to be completed is an easy one. The payment process just needed a click on the menu "Pay" then the QR code will be displayed and scanned by the merchant cashier. The account will then be deducted and the balance displayed on the app. Table 2 shows the participants' task completion.  Time on Task: The screen recording app recorded the time on task for each participant. Some tasks were inherently more difficult to complete than others and this affected the average time on task. Task 1 required participants to check their account balance and it took the shortest time to complete (mean = 16.40 seconds). Completion times for task 1 ranged from 7 to 55 seconds with most participants completing their tasks in less than 20 seconds. There were two participants who took longer time to complete the first task with 28 and 55 seconds respectively. Task 2 required participants to reload money and the average time on task for this task was 150.13 seconds. Completion times for task 2 ranged from 62 to 260 seconds with most participants accomplishing their tasks in less than 200 seconds. There was however three participants who took more than 200 seconds to complete task 2. Task 3 required the participants to perform money transfer and this task took the longest time to complete (mean = 192 seconds). Howbeit completion times ranged from 37 to 535 seconds with most participants completing their tasks in less than 200 seconds. This task is more complicated compared to the other tasks. It widened the range in task completion times. One participant however took almost 9 minutes (completion time =535 seconds) to complete task 3. Task 4 required the participants to perform payment at selected attached merchants and its mean completion time was 34.80 seconds. The completion times ranged from 17 to 60 seconds with most participants completing their tasks in not more than 60 seconds (less than 4 minutes). Table 3 shows the task completion times per participant. Task Errors: In task 2, four (4) out of the 15 participants made typing error mistakes but it is in minimum errors range of 1-3 typing errors. These participants made the errors/mistakes during the keying in of their personal data while completing task 2 (this task required participant's personal banking data in order to reload their eWallet). The errors are categorized as non-critical and do not prevent successful completion of the task. The errors are just about user typing errors while they are trying to key in the data. Participants can make correction easily and proceed to complete the desired task. Table  4 shows the task errors per participants. Task Ratings: After the completion of each task, participants rated the interface design, ease of use and their experience based on the following nine criteria: i) Satisfaction with application interface -interface design; ii) Satisfaction with the use of text -interface design; iii) Satisfaction with the use of colour -interface design; iv) Satisfaction with the graphics and sound -interface design; v) Satisfaction with the menu provided -ease of use; vi) Easy to find help -ease of use; vii) Satisfaction with the outputexperience assessment; viii) Satisfaction with the application performance -experience assessment; and ix) Enjoy the application? -experience assessment. For this test, nine items were used based on 5-points Likert scale that ranged from 1. ("Strongly disagree'') to scale 5. ("Strongly agree''). Agree ratings combines the agree ratings and strongly agree ratings.

a) Interface Design
Application interface: All participants agreed they are satisfied with the application interface for task 1 ("Check Account Balance" with mean agreement rating = 5.00). Almost all the participants (93.3%) were satisfied with the interface during task 2 ("Reload Money" with mean agreement rating = 4.47). In task 3 ("Perform Money Transfer"), 93.4% of participants liked the interface (mean agreement rating = 4.27) meanwhile everyone were satisfied with the application interface in task 4 ("Perform payment at selected attached merchants", mean agreement rating = 4.33).
Use of text: All participants agreed they are satisfied with the use of text in task 1 ("Check Account Balance", mean agreement rating = 5.00) and 93.3% of participants were satisfied with the use of text during task 2 ("Reload Money", mean agreement rating = 4.33). In task 3 ("Perform Money Transfer"), all participants (100%) liked the use of text (mean agreement rating = 4.60) meanwhile everyone (100%) were satisfied with the use of text in task 4 ("Perform payment at selected attached merchants", mean agreement rating = 4.27).
Use of colour: All participants (100%) agreed they were satisfied with the use of color in all task. However, the mean task rating varies in each task. The mean task rating for task 1 is 5.00, that of task 2 is 4.60, the one of task 3 is 4.47 while that of task 4 is 4.53.
Graphics and sounds: All participants (100%) in task 1, task 2 and task 3 agreed they are satisfied with the graphics and sounds of the app with mean task ratings of 5.00, 4.47 and 4.73 respectively. However, in task 2, 6.7% of participants were dissatisfied with the graphics and sounds of the application with mean task rating of agreement of 4.27. Table 5 provides a summary of the users' satisfaction perception of the app's interface design. Overall, the mean interface design satisfaction perceptions are more than 4.00. Thus, it is considered that the users were satisfied with the interface design of the application. Table 5 shows the mean interface design satisfaction ratings per task for the application. Menu Navigation: Task 1 has the highest mean task rating with 4.80. Almost all participants (93.3%) were satisfied with ease of menu navigation of the application. Same percentage of participants perceived the menu navigation in task 2 as easy to use (mean agreement rating = 4.40). Meanwhile, 87.6% of participants were satisfied with the menu navigation ease in both task 3 and task 4 ("money transfer and payment", mean agreement rating = 4.13).
Use of text: 40.0% of the participants were satisfied with the ease of finding help while performing money transfer (mean agreement rating = 2.80) and only 26.7% found it is easy to find help when reloading money (mean agreement rating = 4.33); the mean agreement rating declined to 1.87 where only 13.3% of participants found it easy to find help while checking account balance. However, none of the participants could find help easily during task 4 while performing payment at selected attached merchants (mean agreement rating = 1.60). Overall, in the ease-of-use satisfaction ratings; mean agreement ratings for all tasks are less than 4.00. Thus, it is considered that the users do not think this application is easy to use. Table 6 shows the mean ease of use perception ratings per task for the application. Output: All participants agreed they were satisfied with the output for task 1 ("Check Account Balance", mean agreement rating = 5.00) and that of task 2 ("Reload Money", mean agreement rating = 4.67). In task 3 ("Perform Money Transfer"), 80.0% of participants were satisfied with the output (mean agreement rating = 3.80) meanwhile all participants (100%) were satisfied with the output in task 4 ("Perform payment at selected attached merchants", mean agreement rating = 4.40).
Performance: 100% of the participants were satisfied with the performance of application while performing money transfer (mean agreement rating = 5.00) and 93.4% of them found the performance of application satisfactory when reloading money (mean agreement rating = 4.33); the mean agreement rating of performance satisfaction while checking account balance is 4.67 with 100% participant agreement. The participants had the same level of satisfaction perception for task 4 while performing payment at selected attached merchants (mean agreement rating = 4.40).
Enjoyment: All participants enjoyed carrying out task 1 ("Check Account Balance") and task 2 ("Reload Money") with mean agreement rating = 5.00. In task 3 ("Perform Money Transfer"), 100% of participants so much enjoyed using the application (mean agreement rating = 4.53), similarly, all participants enjoyed carrying out task 4 ("Perform payment at selected attached merchants", mean agreement rating = 4.47). Table 7 shows the mean experience assessment ratings per task.
Overall, the mean experience assessment ratings for all tasks are more than 4.00. Thus, it is considered that the users enjoyed using this application.  Table 8 shows the overall mean ratings of the application per task. In all tasks, the average satisfaction was above 4.00 except for task 4 with a rating of 3.92. Nonetheless, there was a good performance rating in all tasks indicating that overall, the perceived instrumental quality of the application is good. Table 9 summarizes all the metrics captured in the study per task.   Table 9 indicates the mean combined ratings across the four task measures. The ratings across the four task measures are satisfactory and show that the app has good instrumental quality overall.

Issues and Recommendations
Fifteen (15) participants participated in testing the Touch 'n GO eWallet mobile application. Though overall the instrumental quality of the app is, however, a close scrutiny reveals some issues for require redesign. The major problems that were discovered based on the data collected during the testing and evaluation are: i. Icons: The significance of the icons was unclear for some test participants.
These icons were tightly placed which made it more difficult to distinguish them. Some of the icons for some participants were not suitable for the task given. Participant 8 felt that the icon of money transfers were not clear enough to represent the task. ii. Text: Some views were missing explanatory text which led to uncertainty amongst participants. Most of the participants felt that task 1: 'Check account balance' was hard to find. Using text for checking equilibrium in these applications is quite tiny. Likewise, the fonts are tiny. The font's color is white; the blue background is quite a reflection. iii. Help instructions: Help instructions cannot be found throughout the entire task.
Most respondents were unable to discover the help features they needed in any task. The introductory instruction only took place at the start of the applications setup. iv. Action order: Before tasks for money transfer executed, participants had to submit identification for security purpose, in order to do that, the task of capturing ID has to be fulfilled and executed, there should be caution in this task as it may take a long time when the capturing of images did not meet the specifications of the applications such as blurry picture, and picture reflection. If it takes a long time to recover the picture, the user may get bored and frustrated. The retake of picture was done approximately more than 3 times. Participants must use some techniques like turning off the lights and adjusting the picture capture angle. Most of the respondents experienced this (Participant 1, 2, 3, and 6).
Conversely, some features were so greatly favored by participants as captured in the evaluation: i. Navigation: The participants felt that navigation in the app was easy and most of the participants were very happy when they completed the tasks successfully. ii. Other Useful Features: Other features that interested many of the participants were the promotions given by the apps like Cashback for Toll, Get RM5 off any movie Ticket, PayDirect for 20% Toll Rebates, and much more. Although these tasks are not included in the list of tasks for the study, provision of more rebates may serve as incentives to lure more customers to join and use the application. iii. Recommendations: Performing continuous usability testing during production on an app design can save a great deal of time, effort, and money. It may also guarantee a better design that could have a significant effect on boosting app usage. Most participants appreciated the characteristics provided for the usability testing. Most respondents were between the ages 30 -40, so age variables may likely have altered their perception of the applications; also, most participants indicated that the font sizes for important characteristics should be expanded due to the tiny size of the smartphone and the tiny font size which they can hardly see in a glance. Often, they had to scroll up and down and forth to find the required feature. Next, a clear "help" feature should be provided in the apps. The only "help" feature was presented during the first installation and act as an introduction only. Participants were not able to discover the "help" after that. During the course of using the app, a help feature should be provided.

Conclusion
Since there are many eWallet applications in the store today, owners need to discover a function that draws consumers to use their app at the highest possible rate. Such apps should be interesting and should function correctly. Having too many features in an implementation can sometimes result in a catastrophe. The idea of Touch 'n Go eWallet is well conceived and timely as it is commonly implemented throughout Malaysia, so customers are requesting a shift from traditional Touch 'n Go cards to eWallet Touch 'n Go apps. The eWallet should drive and attract users in using the app and the design should be clear, clean and with minimalist display of features. This is the most important issue that should be dealt with in order for the app to work with people who have no previous e-wallet experience. When testing, it was realized that user were quite slow in comprehending the app's design because it was their first experience, but when they got used to playing with it, it got faster and quicker. On average, the findings acquired from the evaluation exceed 85% of each allocated task from the experiments performed. Thus, the eWallet basic function (Task 1 until Task 4) is accepted by participants involved in the testing session. This study used 4 tasks to assess the instrumental quality of Touch 'n Go eWallet mobile app using 15 participants. Overall, the outcome of the study revealed that the app is generally instrumental to assisting users accomplishing their electronic wallet goals. There is however some observed issues in the app that require fixing to enhance the instrumental quality of app.