The Usability Analysis Online Learning Site for Supporting Computer programming Course Using System Usability Scale (SUS) in a University

—Computer programming subject is the most-feared subject by students because it is hard to learn. At present, various learning media for programming have been available; therefore, the learning process that used to be monotonous and challenging to learn has become exciting and easy to do. One of those media is the Website. Usability is a crucial aspect to be noticed for the users so they would be facilitated during the usage, acquiring the required knowledge, and be interested to always rely on the site in their learning process. This research was done to analyze the success of codesaya.com as an online learning site should it meets the usability criteria by implementing the System Usability Scale (SUS) as the research instrument. SUS is an effective and reliable instrument to measure the usability of various products and services. The research questionnaire consisted of 10 questions which disseminated toward 162 respondents divided into two groups, namely familiar users that amounted to 81 respondents and unfamiliar users that amounted to 81 respondents as well. The usability of the CodeSaya site acquired 72.1 of a score for familiar users and 70 of a score for unfamiliar users. The score acquired from the rating given by familiar respondents only a bit higher compared to the unfamiliar respondents. The rating in the alphabet resulted in B or Good. It can be concluded that this computer programming learning site has met the usability criteria; therefore, it can be implemented as a learning medium that is feasible to be used by the users. Referring to such results, the web-based learning of computer programming as the alternative of pedagogic approach and teaching technique can be applied based on the psychological needs of students in their adolescence period, allowing them to collaborate, stimulating creative and analytical thinking in solving problems.


Introduction
The teaching and learning of programming courses are often considered as a difficult topic for students and teachers due to its complexity and abstract. In the last 30 years, the scientific community has yet to quit the search for new pedagogic approaches and teaching techniques for the introduction course of computer programming [1]. Coding is a part of logical thinking and is one of the necessary skills known as 21 st -century skills. However, computer programming is hard to learn, and the program course has often resulted in a high drop-out rate [2]. One of the main factors of the difficulties faced in learning programming has been associated with the traditional approach to teaching the basics of programming, which unable to provide students with an attractive and simulation-rich environment in which the problems and concepts become the subject of investigation in a pleasuring and creative manner. [3] The studies of teaching and learning programs in various country and education contexts had revealed that novice programmers faced the same challenges in the efficiency of writing, debugging, and running the programs. These difficulties had to lead them to be involved in the teaching of programming to forwardly consider the most effective methods that could facilitate the novice programmers in learning the basic concepts of programming. The visual programming environments that support the development of the program through a drag-and-drop interface are some of the most popular coding tools to teach the novice programmers. In this paper, Papadakis [4] investigated the use of scratch and app inventor. Papadakis [4] wished that "his study will become useful guidance in the hand of every teacher in the middle school who teaches or designs the introduction course of programming in an effort of attracting more students into programming [2]". App Inventor and Scratch are the environments of block-based programming designed explicitly by considering the novices. A study had found that the programming environment of Scratch does not only help novice programmers in concentrating on syntax issues but also toward the design and development of comprehensive systems and algorithms [5]. Scratch offers easy modification in the constructs of programming (i.e., variables, loops, control statements, functions, etc.), this will become a good idea of integration in the course of smart robots such as Bee-bot and Kibo or internet-connected smart toys, i.e., Sphero [6]. Papadakis [3] teaches basic programming concepts to novice programmers. The proposed methodologies utilize various technologies, including Twitter, Python programming language, and Arduino board. By involving five students as part of the school activity program, the results of the case study were difficult to be generalized, the evaluation of the implementation through semi-structured interview showed pleasuring results. Through their involvement, students understood the basic concepts of programming and technology when they get involved in a practical work within the inter-discipliner authentic environment [18,19].
In Indonesia, one of the online learning media that have been developed and began to be introduced as a reference for students and teachers is CodeSaya.com Portal. A learning medium in the form of CodeSaya.com online site provides a service of providing materials related to basic concepts of programming languages that are presented online; therefore, the users can access the site swiftly. This site also provides a platform for the users to practice several instructions of programs that are discussed in sequences as well as giving a clear error message of the user program. This is one of the facilities that can be enjoyed by the users of this learning medium. Through this medium, the learning process does not have to be done face-to-face with a teacher in a specific place. The users only have to rely on their smartphone or computer to be able to understand the learning materials. The most common mistake is a syntax error, associated with the error in the code writing of the program and algorithmic error associated with the steps of program finishing. The difficulty in understanding syntax so the syntax error cannot be corrected. This condition is seen from some programs that still leave issues.
Learning media are inseparable from the usability factor, both the creation or the utilization. An international standard has expressed the definition of usability. ISO 9241-11 defines it as "the extent to which a product can be used by specific users to achieve specific goals with effectiveness, efficiency, and satisfaction in a specified context of use. This definition connects three criteria (effectiveness, efficiency, and satisfaction) [13].
System Usability Scale (SUS) is a questionnaire to measure the perception of usability. This instrument was created by John Brooke in 1986 and previously used to test the electronic office system. Despite its simplicity, SUS has become a questionnaire that is widely used for the assessment of the experienced usability [7] [8] [9]. The studies of Sauro [8] and Tullis [10] showed that the System Usability Scale (SUS) is a usability testing instrument that is valid and reliable. Tullis and Stetson [10] measured the usability of two websites by using five different surveys (SUS, QUIS, CSUQ, a variant of Microsoft's Product Reaction Cards, and one that have been used in their Usability Lab for several years), and found that SUS provides the most reliable result in various sample sizes. It is interesting to be noted that SUS is the most uncomplicated questionnaire to be learned, SUS (with only ten measurement scales), produces reliable results in all sample sizes. It is also interesting that SUS is the only questionnaire from those who were studied in which the entire questions discuss every aspect of user reaction toward the website comprehensively. There are some characteristics of SUS that attract users. First, it only consists of ten questions; therefore, it is relatively fast and easy to be completed by study participants and to be scored by administrators. Second, it is nonproprietary; thus, it is cost-efficient to use and value as very fast, soon after it be finished. Third, SUS it an agnostic technology, which means that it can be used by a large group of usability practitioner to evaluate almost every type of user interface, including Websites, cellphones, Interactive Voice Response (IVR) (both touch-tone), and expression), TV application, and many others. The last survey result is a single score, ranged from 0 to 100, and relatively easy-tounderstand by various people from other disciplines who work in a project team.
Usability is a quality attribute that assesses the easiness rate of a user interface to be used. Usability also refers to the method in improving the ease of use during the designing process. Users will abandon a web with poor usability. A study on usability had been done by Mostakhdemin [15] that gave Usability Considerations for Mobile-Learning Applications. Usability and pedagogic factors play a crucial role in mlearning. Dirin [16] developed a m-learning application in which the proposed mLUX's framework was implemented. The main goal of the proposed mLUX's framework was to ensure that the stakeholders, especially students, admitted that mlearning application was the learning medium that fulfilled the critical need for their education. This paper also argued that emotional factors, such as user pleasure, adapt-ability, and reliability, are the significant matter of design in learning. Hussain, A [17] reported the perception of the user regarding the usability of oBike cellular-a global sharing platform of a bicycle. The study results revealed that most of the participants found that the cellular application of oBike should be improved to enhance their satisfaction. Some suggested recommendations will ensure the improvement of the application should ensure they are being implemented.
As an ICT-based medium, CodeSaya.com Portal has some advantages to support the activity of teaching and learning. Even though sufficient studies are required to substantiate the assumption. The reason is, no matter how good the course of CodeSaya.com portal, it would be less objective if only be assessed internally by the administrator. Therefore, the author proposed the use of SUS to test the usability of the CodeSaya.com website. How is the effect of the web usability variable in the perspective of respondents? Therefore, the author proposed the utilization of SUS to test the usability of the Codesaya.com website. The results of this study are expected to become a benchmark and recommendation in the future development of the online learning site of CodeSaya to become a more effective, efficient, and reliable learning medium.

2
Literature Review Nielsen [7], "Usability is a quality attribute that assesses how easy user interfaces are to use. The word "usability" also refers to methods for improving ease-of-use during the design process. Five quality components define usability: 1. Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the design? 2. Efficiency: Once users have learned the design, how quickly can they perform tasks? 3. Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency? 4. Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?" 5. Satisfaction: How pleasant is it to use the design? System Usability Scale (SUS) is a questionnaire that can be used to measure the usability of a computer system from the subjective point of view of the users. John Broke developed SUS in 1986. Until present, SUS is widely used to measure usability and has shown various benefits, including [12]: 1. SUS can be used easily because the results are in the score range of 0-100. 2. SUS is very easy to use; it does not require complicated calculation. 3. SUS is free; it does not require additional cost. 4. SUS is proven as valid and reliable, even though with a small sample size.
SUS is a questionnaire consisting of 10 question items, as shown in Table 1. SUS questionnaire uses five points of Likert's scale. Respondents are asked to give ratings i.e., "Strongly Disagree," "Disagree," "Neutral," "Agree," and Strongly Agree" over 10 question items of SUS according to the subjective rating. If the respondents think that they do not find the right response scale, they have to fill the midpoint of the testing scale. Each question item has a contribution score. Every item contribution score will range between 0 to 4. For items 1, 3, 5, 7, and 9, their contribution score is the position of scale subtracted by 1. For items 2, 4, 6, 8, and 10, their contribution score is five subtracted by the position of the scale. The total contribution score is multiplied by 2.5 to acquire the overall value of system usability. The SUS scores are ranged between 0 to 100. This is the calculation formula for the SUS score: The whole SUS scores are acquired from the mean of individual SUS mean score. The SUS questionnaire is distributed through google form link toward students in System Department. 5 Agree I think that I would like to use this system frequently. I found the system unnecessarily complex. I thought the system was easy to use. I think that I would need the support of a technical person to be able to use this system. I found the various functions in this system were well integrated. I thought there was too much inconsistency in this system. I would imagine that most people would learn to use this system very quickly. I found the system very cumbersome to use. I felt very confident using the system. I needed to learn a lot of things before I could get going with this system.

Methodology
The research flow was started with questionnaire designing, data collecting, and data processing. The implemented questionnaire was a questionnaire regarding the usability rate of codesaya.com as a medium learning for programming.

Research object
The object of this research was the usability of an online learning site for python programming language, namely CodeSaya. In this research, the collecting of quantitative data was conducted by performing usability testing consisted of several scenarios to rate/measure the effectiveness, efficiency, and error criteria. The collecting of qualitative data was conducted by using the questionnaire to rate the satisfaction criterion.

Research respondents
This study tested 162 people who use the Codesaya learning site to learn programming languages (which in this context is Python). Respondents were divided into two, namely familiar users (two months of usage, the site was used in Algorithm and Computer Programming subject to learn Python) and unfamiliar users with two weeks of usage. The respondents were active students in the Computer Systems Department of Information Technology Faculty in Universitas Andalas. Respondents were asked to fill their identities while filling the questionnaire such as name, age, gender, and signature.

Research procedures
In this study, the performance measurement technique can measure effectiveness, efficiency, and error. The satisfaction criterion was measured through the questionnaire. The tasks given on the tested site were as follows: 1. Register and Login: Respondents conducted a registration process with the predetermined Username, Email, and Password until they succeed. Next, respondents will log in by entering the registered data until they can get into the Home menu. 2. Selecting the Material: Respondents selected the learning materials, in this case, respondents used the site to learn Python programming languages, and the respondents acted as the new user. 3. Learning: Respondents started the learning process to use the site. In this site, materials such as basic concepts and the application on programs as well as the discussion forum that can be utilized by users should some obstacles occur.

Questionnaire designing
After the study regarding the usability of a web, in general, was done, the next step was determining the way to measure web usability. In general, the criterion that determines that the web is useful is if the users can acquire and discover the things required by them from the website. According to the definition from Nielsen, there are five measurement criteria that work as the condition of which web will meet particular usability criteria; thus, the benchmark of a web usability fulfillment can be obtained.
The used questionnaire was the System Usability Scale. This SUS questionnaire was used for assessment after respondents done the given assignments. This questionnaire was provided as a post-test questionnaire that functioned to complete the data acquired from the subjective arguments of the respondents. SUS consists of 10 questions. In its original term, SUS uses English. However, there was a research or a paper that has made it in Bahasa, namely the research of Z. Sharfina and H.B. Santoso [13].
At each question point, there were five measurement scales from 1 to 5 (1 for the lowest, 5 for the highest). The answer options consisted of strongly disagree, disagree, hesitant, and strongly agree. SUS has 0 as the minimum score and 100 as the maxi-mum score. This SUS-implementation research resulted in the form of a single number with the result numbers ranged between 0 to 100. Then, this SUS score will be put in average/mean to acquire the final score of SUS.

Data processing
After data collecting was done toward the respondents, the data then be calculated. There are some rules in the calculation of SUS score when using the System Usability Scale (SUS). These are the rules during the score calculation of the questionnaire: a) For every question in odd numbers, the score of each question is acquired from the user score subtracted by 1. b) For every question in even numbers, the final score is acquired from 5 subtracted by the score of question acquired from the users. c) The SUS score is acquired from the score addition result of each question multiplied by 2.5.
The rule of score calculation applies to one respondent. For the next calculation, the mean of the SUS score of each respondent is found by adding whole scores divided by the number of respondents. This is the formula to calculate the SUS score: (2) In which: If the stage above has been reached, the previous formula is applied, the mean of the score is calculated. The method is based on the formula, namely, summing the scores of all respondents, then divided by the number of respondents. The mean score of SUS from the data result above is 85.

Validity and reliability tests of the instrument
The validation of the instrument was made to obtain valid data, so the instrument can be used to measure what should be measured. Validity shows the extent to which the relevancy of questions toward what being asked or what to measure in this study. The validity and reliability of the questionnaire were measured by testing 162 respondents. This effort was made to determine whether the instrument that will be distributed toward the respondents is valid or not as well as the questions associated with the research variables of usability, learnability, efficiency, memorability, error and user satisfaction. The validity test used Pearson's Product-Moment correlation formula presented as follows: (3) r =Correlation between x and y N = The number of samples  x =The amount of item score  y =The number of total item score The validity of the questionnaire was acquired by comparing r-count values with rtable values. The rule of decision is if r-count > r-table; so, the questionnaire is valid, on the opposite, if r-count < r-table; it means that the questionnaire is invalid. The validation test of this study used 162 respondents; the r-table value was 0.514. Therefore, the question item will be determined as valid if the value is > 0.514, and it would be determined as invalid if it is < 0.514.

Instrument reliability test
A reliable instrument means that if the instrument is used several times to measure the same object, it will generate the same measurement result. The answers to the questionnaire were processed through a statistical test by using a computer program. The reliability test was performed through Cronbach's Alpha test with a formulation as follows: (4) r1= Coefficient of reliability k=The number of question item Criteria: if ri-count > ri-table, thus, the instrument is reliable. If ri-count ≤ ri-table, thus, the instrument is unreliable. The reliability of the questionnaire is discovered by comparing the r-count value with the r-table value in which the used r-count is the Alpha value. It is determined as reliable if the r-count value is greater than minimum reliability value i.e., 0.7. The reliability test showed that this questionnaire is very reliable because it has more than 0.7 value, namely 0.968.

4.3
The analysis of SUS score SUS is a global assessment of usability aspects (effectiveness, efficiency, and satisfaction) which subjectively experienced by the users. The SUS score can indicate the acceptability range of the users. The mean score of SUS has to be higher than 70 (Brook, 2013) to be categorized as acceptable. The mean score of SUS of the Codesaya website was 70. It means that the questionnaire of the Codesaya website was good, as we can see from the figure presented below.

Fig. 3. SUS Score
The study was done by [8] also explained about the category of SUS score. To acquire A as the rank, the SUS score has to be mounted at least 81. The SUS score of Codesaya was 70 and 72.1; it means that the score can be categorized in rank B, as shown in Figure 4.

Fig. 4. Percentile Ranks
The calculation results performed through the SUS method then be converted into percentile ranks and letter grades. Percentile ranks show the usability rate in the form of a percentage (%), while letter grades show the usability rate from A to F classes where A is the best class, and F is the worst class. The rule for percentile ranks and letter grades are as follows: 1. Grade A: score >= 80.3, percentile >= 90 % 2. Grade B: 74 <= score < 80.3, 70 % <= percentile < 90 % 3. Grade C: 68 <= score < 74, 40 % <= percentile < 70 % 4. Grade D: 51 <= score < 68, 20 % <= percentile < 40 % 5. Grade F: score < 51, percentile < 20 % SUS can also be interpreted into adjective ratings to elucidate the usability rate of a system, which later be interpreted into the user acceptability range toward a system to determine whether the system is acceptable or not by the users.
Also, according to this data, the mean score from the SUS rating on the CodeSaya site for familiar users was amounted to 72.1, while the unfamiliar users acquired 70.0 of the score. Of both data above, the comparison of SUS scores on both respondent categories can be acquired as follows. According to the scale of SUS scores presented in Figure 3, 70, and 72.1, scores are categorized in the Good range for adjective ratings with B-C grade of scale and High acceptability range. Of the entire measurements, it can be concluded that the CodeSaya site was still categorized as a high mean class or almost categorized as acceptable. The score acquired from the rating done by familiar respondents only a bit higher than the unfamiliar respondents. It means the rating criteria for the users were almost the same, either for familiar or unfamiliar respondents.
About the utilization of ICT-based learning medium, i.e., a website that contains many elements of image animation, text, motion, and audio categorized as multimedia, thus, the learning will emerge learning motivations by the theory of motivation. The learning objects that are perceived as complicated or uncomplicated will generate a low motivation because students will feel bored, or even frustrated due to the feeling of the impossibility in finishing the assignment properly. The argument of Santrock [14] also stated that teenagers between the age of 15-18 were having a lot of cognitive, emotional, and social changes, capable of thinking in a more sophisticated manner, and often spent their time with peers. In view of the above, a web-based learning design can be implemented according to the psychological needs of students in the adolescent period.
Web-based learning is not punitive because in this theory; teachers are more emphasizing on the learning process rather than the new learning outcomes in this theory. This condition is substantiated by the establishment of a good environment (collaborative). Therefore, the increase in motivation and the minimum occurrence of errors are possible due to the pleasuring learning experienced by students.

Conclusion
After the measurement to acquire the Usability Scale on the platform of online learning of programming through CodeSaya that is used by students who take Algorithm and Computer Programming in Computer System Department of the Faculty of Information Technology in Andalas University was done, it can be concluded that the usability of Codesaya Portal was experienced by the user, highly-correlated with learnability, efficiency, memorability, errors, and satisfaction. In general, this rating ability in children is pragmatic. They frequently rate an object as "good" if it emerges a pleasure for them, while something "bad" is an annoying or unpleasant thing. It can be concluded that most of the students argued that the Usability of this CodeSaya Platform had reached the scale with 72.1 of the score for familiar users, while the unfamiliar users acquired 70.0 of the score; or acquired B (Good) based on the Alphabetical rating. Therefore, it can be determined that this website can be implemented as a learning medium that is easy to learn, understand, and operate by the users. The score acquired from the rating of familiar respondents only a bit higher than the unfamiliar users. It means that the measurement criteria for both familiar and unfamiliar users were equal. However, this website still requires more evaluations for its future developments. Because, based on the acquired scores, this website still leaves the impression that the users have yet to feel satisfied in using the website, so future improvements are required.
I suggest the improvement on the performance of this website, so the users will always use this medium as their learning platform because most of the respondents conduct the website evaluation through their mobile, due to that matter, respondents request a more mobile-friendly website for more comfortable use. The login should be made more accessible and simplified, so the users do not feel complicated. The validation results in the form of data of qualitative assessment on web products from the learning technology experts showed that the whole aspect of learning technology in the product of CodeSaya.com web has been sufficient; thus, they can be used as a new pedagogic approach and learning technique in the course of computer programming for teenagers. Teen students who participate in an online web-based are collaboratively showing the increase in motivation, depth concept comprehension on materials, and the increase in the desire to solve problems.