Usability Testing of Mobile Flipboard Application on both Non-Users and Novice Users

This paper aims to discuss how journalism in a network has transformed the production and circulation of news and as gate keeping and motivation setting, from the action interdependence of ordinary people, journalists, organizations and advanced innovations (such as blogs, Twitter, Facebook and Flipboard). Finally, we analyze the interactions in the Flipboard mobile app. We carried out a study consisting of 13 participants in UUM Sintok, they were assigned five tasks to perform and questionnaires were distributed to gather their experience with the app. The results of this study showed that participants were satisfied with the application's aesthetic but find the complexity of features provided in the application to perform various functions very frustrating to use. Keywords—News aggregator, usability, testing, magazine, evaluation method, mobile application.


Introduction
Over the years there has been an enormous amount of effort put into satisfying users and get to know their exact needs and what they look for in a software and application. That's why there has been a wide range of usability testing done for a products and services. Before the coming of smartphones, usability testing was only limited to software and web applications. But since mobile smartphones was created, mobile applications also came into the picture. As mobile smartphones together with mobile apps becomes more popular and easily accessible, there are more applications being developed with various purposes and similar apps came from different company and developer. This led to competition for the survival of the fittest to see who can create a mobile application that is most efficient and is also able to satisfy the users enough for them to stick with the app without having the need to go for a similar app that also serves the same purpose. This eventually requires the need for usability testing of mobile applications to be able to get the exact requirements of what users expected in an application and also knows the features that is suitable for a group of people with special needs whenever they use the particular application.

Background
Flipboard is a news aggregation application that gives stretched out access to news articles and business stories from a wide scope of productions, including the New York Times, Guardian, Financial Times, and Forbes, among others [1]. It is a news aggregate site and distribution service having branches in New York, Vancouver and Beijing, their headquarter is in Palo Alto, California. The software was initially launched in July 2010, also referred to as Flipboard. It aggregates content from social media, news feeds, photo sharing sites and other websites, presents it in magazine format, and allows users to "flip" through the articles, images and videos being shared. Readers can also save stories into Flipboard magazines. As of March 2016, the company claims there have been 28 million magazines created by users on Flipboard. The service can be accessed through an internet browser, a Windows OS or Mac OS version of the Flipboard application, and iOS, Android and Windows Phone smartphone apps. Customer software is available free of charge and is available in 21 languages. It's a fantastic social app that enables users to share, group and gather content from various sources all in one location. It offers you the versatility to create your own streams, which are labeled 'magazines' to assemble relevant content that you will find online. Readers can also subscribe to general magazines from others or the ones managed by a specific brand. It is easy to use the device that is currently common among people.
But the issue is how people are satisfied with their current mobile Flipboard app? In order for a Flipboard mobile app to be able to deliver service to a consumer, things like features, layout and accessibility are essential. Therefore, in order to achieve both successful experiences and customer satisfaction, equal importance must be provided to the usability of such an application and user-centric issues must be at the core of the operation. Flipboard offering news services has limited functionality and this demonstrates that their interfaces are still incomplete and not user-friendly. Flipboard mobile app is a lightweight program that can run on mobile gadgets like smartphones, computers and tablets. Making use of smartphone, online news that can be watched at ease offers a comfortable platform for consumers and being able to readily get it when available. The app which represents the website with name Flipboard was utilized in carrying out usability testing on participants in this study.

Setting up a usability laboratory and field study
For the usability testing procedure, the evaluation is carried using two main methods of inquiry or environments which is the adoption of usability laboratory and field study to investigate the user's satisfaction towards Flipboard application in the context of efficiency and effectiveness. For the part of the usability laboratory, various quiet places were set up both inside the UUM library (well known as Perpustakaan Sultanahbahiyah) by using the postgraduate area section of the library which are usually more closed in, private and controlled from outside distractions. Some of the tests are also done at the smart reading rooms that are near the UUM library and also at the Library foyer during the time when it is most quiet; these places are used to serve the purpose of performing the test using the method Field testing. The other place used in achieving the field-testing method is the student residential hall of DPP Sime Darby and the Student Lounge located around there, which is well known to most students.
The users were given the permission to perform the tasks assign to them in any position they preferred whether it is by sitting or stand. There were two supervisors, one was entrusted with assigning the designated activities verbally to the users and the other was entrusted with recording the test segment with a smartphone camera. Following agreement between the researcher and the participants on the usability testing, no restrictions were placed on the individual during the test session to create a perfect atmosphere for the respondents. And we make sure the respondents were reassured that they are not being tested but the application they are doing task on.

Tools
The respondents were allowed to use their various mobile devices that can either be android or iOS to perform the tasks so as to make them more comfortable in doing the testing since they are used to their own mobile phones and will prefer it to another type of mobile devices they have never used before. We allowed them to use their various mobile devices also for the purpose to prove that the application to be tested on can easily adjust to various mobile device preferences and operating systems. The main recording done for each task the respondent is doing through screen recording capture on each respondent's phone. So, before the test started, the respondents have to install both Flipboard application and a screen record app (if they don't have screen record features on their devices) provided by us on their mobile devices. Apart from phone screen recording how the respondents are doing each task, we also take a picture of every respondents while performing the task to serve as proof of the test.

Task description
The 11 respondents in each session are required to test the Flipboard application by using the touch screen to accomplish the 5 experimental tasks. The researcher records the entire sessions on the phone video of the respondents while they are involved with the activities. The testing is done individually, from a participant following another. After concluding with the tasks, the respondents are required to complete the post-test questionnaire.
The following are the list of the five tasks used in the testing:  Register an Account  Add categories of news  Link social media accounts to the app  Follow a magazine page  Create a collection(playlist) of news in profile account The researcher advises respondents to use the "thinking aloud" technique prior to starting the study while they are in the task execution phase. The participants are also allowed to move effortlessly from one task to another if they choose, and this is intended to observe the actions of users when moving from one component to another. The post-test survey provides the additional data and participants' response to the Flipboard application.
The procedure of this experiment is based on the following steps: (adapted from the works of Lin, 2015)  Step 1: Carrying out of the 5 experimental tasks by the 11 respondents who evaluate the Flipboard app system. The respondents are selected randomly to evaluate the application.  Step 2: The researcher has recorded every experimental session using screen recording video app for mobile phone, comprising of when the errors happened and amount of time it took to do the normal task. The calculation consists of total amount of errors that respondents created when trying to finish a task and the time it took to perform a task is the operation time. With a stopwatch and the screen capturing app, the time to finish every task is documented. Successful tasks (whether a respondent finishes a task efficiently or not) are also documented. To fulfill each task, respondents are allocated up to four minutes (pilot testing was conducted to decide that 4 minutes is appropriate to complete the task).  Step 3: When the total 5 tasks are completed, the respondents are immediately given a post-test questionnaire so as to measure each factor of the usability test.  Step 4: Analysis of the numerical data that includes the operation time, error frequency and successful rate.

Preliminary evaluation
An initial evaluation was carried out to determine whether users find the Flipboard application as being efficient and effective as its help users get various news across the world easily and also able to share it with others. It was heavily based on the users' own words. This study chose to collect quantitative and qualitative data for the tests to reach in order to deem that the usability testing of Flipboard is successful or otherwise. The acceptability testing of the Flipboard can be obtained in a more indepth fashion by opting for a qualitative investigation and not quantitative alone. The users had informative opinion and information of the Flipboard after the testing procedure. The evaluation comprised direct questions related to the screen (visibility), terminology and app information, learning (Ease of use), application capabilities (efficiency of the app) and general impressions of the Flipboard App. The Flipboard mobile application was used throughout the evaluation period.
This study invited 13 students (10 male and 3 female) coming from different level of study at the Universiti Utara Malaysia. The users are categorized as experienced users in terms of using the application (i.e. on the scale of "3 years and above"), moderate users (i.e. on the scale of "1 -2 years"), and novice users (i.e. on the scale of "less than a year") in term of the number of years the users have been using the application. The users are categorized as frequently-use users in term of using the application daily. Table 1 above shows the user profiling of the participants. Though only 11 students were able to do the testing with recording, the other 2 male students just checked the application on their own and then filled the post-test questionnaire.

Results and Discussions
After the participants completed the task that was given to them, they were given a post-experiment questionnaire to provide more information based on their thoughts about the efficiency and effectiveness on the ease of use of the kindle. The results of this experiment were analyzed using SPSS V23. In this section, the reliability and descriptive statistics are presented accordingly.
Descriptive statistics was used due to its convenient and simplicity to analyse the collected data (Mwaura, K. R, 2013). These statistics took the forms of percentage, mean and frequency distribution The findings are presented by using tables, graphs, mean, frequencies and percentages. Table 2 illustrates the outcome of the task completion. Errors were made mostly in task 3 (Link social media) where the main error was how users is able to find the section where they can link their social media accounts to the app. The other task with most errors is task 5 (Create collection), this is mainly because participants don't recognize the button icon for creating playlist for news and also what to click. The button was not representing the function it is supposed to perform, the overall completion rate of both task 3 and task 5 is 82%. On the other hand, one of the participants committed an error while carrying out task 2 which is (Add categories), the error was because the participant went to the wrong place and choose wrong option to add categories so, he was unable to do it before he started another task. 6out of all the participants finished every task given and did not encounter any challenges.   Table 3 summarizes the findings, only 7 out of 11 participants accomplished task 3 without any errors in the task steps which are; start flipboard> go to my profile > settings > go to accounts > connect your social media accounts. Due to the complexity of the app interface in which the button does not indicate its purpose, the four participants were not able to locate the button to move ahead to the fourth step smoothly. So, the participant does not understand exactly where to click to progress the other step. This task has the maximum running time of 4 minutes and 48 seconds to be exact.
At another part, Task 1 had only one error because of incorrect passwords to register. Only one error was also committed in task 4, which is because of incorrect navigation. The steps to accomplish task 1 includes; start flipboard>get started> go to profile> Sign up> enter email and password> enter full name>enter username. See  table 2 for Summary of Completion, Errors, Time on Task.  Table 4 has shown the descriptive statistics for the screen measure. From the below table, the questions answered by the respondents in the present study about the screen visibility of the application, it shows that the majority of the respondents' answers chosen for the questions 'Agree'. Results have shown that the value of mean is above 4 and the value of standard deviation is below 1, (highest mean = 4.23, Sd = ~1.068) (lowest mean = 3.85, Sd = ~0.439). App Terminology (Information): Table 5 shows the descriptive statistics for the app information. From the below table according to the questions answered by the respondents in the present study about the ease of terminology of the application, it shows that the majority of the respondents' answers chosen for the questions is between 'Agree' and 'Uncertain'. Results have shown that the value of mean is below 4 and the value of standard deviation is below 1, (highest mean = 4.00, Sd = ~1.050) (lowest mean = 3.23, Sd = ~0.577). Learning: Table 6 below shows the descriptive statistics for the flexibility of learning how to use the application. From the below table according to the questions answered by the respondents in the present study about the ease of terminology of the application, it shows that the majority of the respondents' answers chosen for the questions is 'Agree'. Results have shown that the value of mean is below 4 and the value of standard deviation is below 1, (highest mean = 4.08, Sd = ~0.967) (lowest mean = 3.54, Sd = ~0.801). Application Capabilities: Table7 below shows the descriptive statistics for capabilities of the application. From the below table according to the questions answered by the respondents in the present study about the ease of terminology of the application, it shows that the majority of the respondents' answers chosen for the questions is mainly 'Agree'. Results have shown that the value of mean is below 4 and the value of standard deviation is above 1, (highest mean = 4.08, Sd = ~1.127) (lowest mean = 3.46, Sd = ~0.494). General Impressions: Table8 below shows the descriptive statistics for general impressions of the application. From the below table according to the questions answered by the respondents in the present study about the ease of terminology of the application, it shows that the majority of the respondents' answers chosen for the questions is mainly 'Agree'. Results have shown that the value of mean is below 4 and the value of standard deviation is below 1, (highest mean = 4.08, Sd = ~0.862) (lowest mean = 3.92, Sd = ~0.760). Thus, in conclusion, this area of research has been enriched by the findings of this research. The Flipboard application has shown a very low satisfaction in use while at the same time, providing aesthetically beautiful interface to the user. The study provides a depth analysis of usability test on four principles, which are ease of use, visibility, enjoyment and efficiency. Quantitative and qualitative methods are used to analyze the data; within the context of this research boundaries the objectives have been achieved, which is to investigate users' satisfaction towards the usability of the Flipboard application.

Screen (Visibility):
Even though the Flipboard app has a very advanced and interactive interface and layouts that makes it more attractive to the user, further work is needed to be done on the application, so that anyone of different age or academic background with either little or high knowledge in technology could be able to operate and use easily, so it could wind up satisfying most users and gain more frequent users of the app and could turn to be the best news aggregator app. This is because there were limits to the research effort, which are like the number of participants and furthermore some enhancement is required to be done in order to achieve this goal.