Hybrid Learning by Using Brilian Applications as One of the Learning Alternatives to Improve Learning Outcomes in College

— Brilian is a hybrid learning application that combines face-to-face learning and online learning. This study aims to analyze the effectiveness of hybrid learning of science management by using Brilian in college. This study was a pre-experiment with one group pre-test and post-test design. The sample of the study were 55 students of Information System study program, Institut Bisnis dan Informatika Stikom Surabaya, Indonesia who were divided into two groups. Before conducting hybrid learning by using Brilian, both groups of students were given a test of learning outcomes and after the learning was completed, students were given the same learning outcomes test again. The collected data was analyzed by using the Wilcoxon test; calculation of normalized gain (n-gain); and the Mann-Whitney U test. The results showed that hybrid learning of science management using Brilian is effective to improve students’ learning outcomes and is indicated by: (1) an increase in students’ learning outcomes which was that was statistically significant at α = 5%, (2) the n-gain mean was in high category, and (3) the n-gain mean was not different in the two students groups.


Introduction
The application of technology-based solutions to education has become a hot topic in entering the era of digital revolution. In particular, e-learning is one of the technology-based solutions that are considered to be most suitable for assisting the learning process [1], [2]. This thing is done maybe because by using e-learning, learning is seen to have many benefits, including: • Instructors and students can easily access the material • Learning time and discussion is not limited to when it must be done and how long it must end The third factor is suspected as the cause of e-learning is considered as the most appropriate to help the learning process at this time.
The need to interact easily with fellow students through cyberspace is because students are classified into Z generation who always want to be connected and communicate with others through social networks, across countries and cultures [4]. In addition, the Z generation is classified into a generation that feels comfortable and highly dependent on technology, and has multitasking capabilities with a variety of online products and sophisticated technology equipment [4]. Therefore, learning with e-learning in this Z generation era is predicted to be able to be developed and can be implemented in learning well.
Although e-learning has many advantages, there are also drawbacks, namely the difficulty of making learning designs that are appropriate to the needs of students. This can happen perhaps because online educational activities require greater pedagogical elements in achieving more effective and more useful learning outcomes [5]. In addition, making an e-learning design requires carefulness and readiness of resources, both the human resources involved, namely teachers and students, as well as resources in the form of needed infrastructure. However, according to Gilly Salmon [6], human touch is still needed in learning.
The research that was conducted by Salmon on learning at an open university, among others, concluded that the role of tutors in learning with e-learning is a key role and cannot be separated to support the success and satisfaction of students. Therefore, to make the learning with e-learning become more perfect, the learning should not exclude teachers, but still give the role of the teacher by using blended learning [7], [8], [9]. Blended learning or hybrid learning, hereinafter referred to as hybrid learning, is a combination of learning from two historically separate learning models, it is a combination of face-to-face learning model and online learning model [10].
The results of the previous study by Hariadi [11] concluded that learning outcomes could be optimal if we use the Internet as a medium in conventional learning, in the hybrid learning format. McCarthy [12] stated that online learning that invites face-toface discussion can improve the understanding of material that supports and the level of students' achievement in the group.
Based on the need to improve e-learning to become a hybrid learning and to meet the needs of technology-based learning that is suitable for students as Z generation, the Institut Bisnis dan Informatika Stikom Surabaya (Stikom Surabaya), Indonesia has created a hybrid learning application that is named Brilian. The Brilian Application has been applied at Stikom Surabaya since 2015, and has received many positive reactions from reviewers in the fields of education and Information Technology (IT), as well as from lecturers and students. The application put more emphasis on: ease of access, breadth of learning resources, interaction between lecturers and students, interaction between students, learning evaluation (collection of assignments and online examinations and is limited by certain time to familiarize discipline), openness of judgment, and emphasis on honesty from students' task collection [13]. However, the problem that arises is that hybrid learning by using Brilian has never been analyzed to find out whether it can effectively improve students' learning outcomes. Therefore, the purpose of this study is to analyze the effectiveness of science management learning by using Brilian to improve students' learning outcomes.

Hybrid learning
Hybrid learning is defined by Singh & Reed [14] as learning that uses more than one learning model, with the aim to optimize learning outcomes. Furthermore, Singh & Reed 14 states that hybrid learning is mixed learning that focuses on optimizing the achievement of learning goals by applying learning technology.
In line with Singh & Reed, Delialioğlu [15] defines hybrid learning as a combination of various learning models by incorporating technology as one of its elements. Meanwhile, Makhdoom, Zolaly, Heissam & Algaidi [16] define hybrid learning as a combination of face-to-face learning with asynchronous or synchronous learning.
Based on several hybrid learning definitions above, in this study hybrid learning is defined as a learning model that combines face-to-face learning and e-learning [13].

Brilian as hybrid earning application
Brilian is a hybrid learning application that has been developed at Stikom Surabaya with the aim to improve the quality of learning, which is built by optimizing Google Apps for Education (Gafe). By using the concept of hybrid learning, learning is not only done in the classroom, but also done in cyberspace so students can learn anywhere, anytime, with anyone, through any medias. In hybrid learning by using Brilian, lecturer' function is as facilitator, mentor, and consultant; thus in hybrid learning by using Brilian, students are required to learn actively. The Brilian application logo that has been developed is shown in Figure 1. In order to produce learning that can help lecturers as facilitators and be able to make students learn actively in class and in cyberspace, Brilian is built in 8 (eight) menus, they are: course, forum, assignment [17], announcement, score list, lecturer minutes, synchronous learning, and anti-plagiarism [18], as presented in Figure 2. The course menu contains learning plans, learning materials and reference materials that support learning. The forum menu is the discussion media, both discussions between students and between students and lecturers, and between groups of students and lecturers. The assignment menu is a place to measure the achievement of learning objectives because there are test questions and assignments that have been prepared by the lecturer. The announcement menu is a media for lecturers to give announcements to all students in the class. The score list menu displays student learning outcomes scores that have been achieved from the assignment process. The lecturer minutes is the media to record student attendance in learning. This menu links to the presence application that has been made in the academic administration and student affairs section. Synchronous learning is a menu for online learning, where students and lecturers can communicate with hangout facilities. The anti-plagiarism is a menu that can be used by lecturers to correct the originality of student work with the similarity check application. With these eight menus, Brilian hybrid learning can be used as a hybrid learning that has a full menu according to the needs of students, so that it is expected to excellence in achieving learning goals.

2.3
Steps of hybrid learning by using brilian Hybrid learning by using Brilian has 5 phases, they are: (1) Goal Orientation based on hybrid learning, (2) Assessing learning materials through the Internet, (3) Analyzing data from study results in groups, (4) Presenting results analysis, and (5) Providing feedback. In more detail, the five hybrid learning phases by using Brilian can be explained as follows.
• Phase 1: Destination orientation based on hybrid learning. In this phase, lecturers introduce learning objectives that must be achieved and learning materials that support the achievement of learning goals, both online and offline.
• Phase 2: Assessing learning materials through the Internet. In this phase, students learn the learning material in phase 1 for preparation to answer some of the raised problems by the lecturer. They can enrich knowledge by searching material from the internet.
• Phase 3: Analyzing data from the study results. In this phase, students answer several problems that must be solved by referring to the learning material and the study results.
• Phase 4: Presenting the analysis results. This phase is a demonstration of student learning by presenting or writing down the results of their analysis as response to the raised problems.
• Phase 5: Providing feedback. This phase aims to provide reinforcement of the learning outcomes that have been obtained, and to rectify the misconceptions that have happened to students. Therefore, in this phase students are given feedback by lecturers. In this phase, students are also given follow-up on learning outcomes that have been obtained through the provision of structured assignments, which are also carried out by using the Brilian application.

Methodology
This study was a pre-experimental study with one group pre-test and post-test design [19]: O1 X O2; with O1 states the initial test score before learning (pre-test), X is the applied learning, and O2 is the final test score after learning (post-test).
The study was conducted on a sample of 55 undergraduate students of Information Systems Study Program at the Stikom Surabaya, who programed science management courses in the even semester of the academic year 2017/2018. The study was divided into two groups, they are: group-1 (MS P1) that was consisted of 24 students and group-2 (MS R1) that was consisted of 31 students.
The research began by developing science management learning devices in the form of: Lesson Plans and Student Learning Materials; and Instruments in the form of science management learning outcomes test. This learning outcomes test consisted of 15 objective questions with 5 (five) options: 4 (four) distractor and 1 (one) correct answer; which refers to the cognitive domain according to Bloom's Taxonomy at a high level, that are: analysis, synthesis, and evaluation. The validity and reliability of Lesson Plans and Student Learning Materials, and science management test instruments are shown in Table 1. Before using hybrid learning by using Brilian, the two student groups were given tests of science management learning outcomes (pre-test) and after learning was carried out on the material and by the same lecturers, the two groups of students were again given a test of learning outcomes on the same material (post-test).
In order to analyze the effectiveness of hybrid learning by using Brilian, the data of pre-test and post-test scores that have been collected were analyzed by using SPSS version 24 data processing software through three stages, they are: • Testing the increase in student learning outcomes in the two groups after hybrid learning by using Brilian • Calculating the average n-gain in the two groups of students • Examining the similarity of two mean n-gain in both groups of students Hybrid learning by using Brilian is effective if: there is an increase in student learning outcomes, which is statistically significant at the significance level α = 5%; the average n-gain is at least moderate, and the average n-gain is not different in the two groups of students (consistent).
In order to test the increase in students' science management learning outcomes, first reserchers tested the assumption of normality toward the n-gain of the pre-test and post-test in both groups of students by using Kolmogorov-Smirnov, followed by the Wilcoxon test. While the average n-gain is calculated by using the formula: n-gain = [post-test score -score pre-test] / [100 -pre-test] [20] with the following categories: (a) high category if n-gain ≤ .3; (b) medium category if .3 < n-gain < .7; and (c) high category if n-gain 7 ≥ .7. In addition, in order to test whether the average n-gain is not different in the two groups of students, it was performed by using the Mann-Whitney U test.

4
Findings and Discussion

Findings
Based on the score of students' science management learning outcomes before the hybrid learning by using Brilian and after the learning in both groups of students: MS P1 and MS R1, the average score of the pre-test and post-test as shown in Table 2 Table 2 shows that the score of students' science management learning outcomes before hybrid learning by using Brilian in both groups of students is very low, it was less than 20 in the range of scores 0-100. Meanwhile, after learning, the average score of learning outcomes in the two student groups becomes high, it were 79.17 and 82. 10.
A summary of the normality assumption test result on n-gain of the pre-test and post-test score for the two groups of students: MS P1 and MS R1, are shown in Table  3.  Table 3 shows that the p-value for both groups was < .05; this means that the data in the two groups of students did not originate from the normal distribution population, or in other words the assumption of normality in the two groups of students was not fulfilled. Because the assumption of normality in the two student groups was not fulfilled, the Wilcoxon test was carried out; the results were summarized in Table 4.  Table 4 shows that the p-value for the two groups of students is < .05; this shows that there is a difference between the average score of the pre-test and post-test. Because the value of Z in the two groups of students is negative, it can be interpreted that the average score of students' science management learning outcomes after the hybrid learning by using Brilian is higher than before the learning process. In other words, there is an increase in student learning outcomes after the hybrid learning by using Brilian.
Summary of the calculation result of the average n-gain in the two student groups: MS P1 and MS R1 are shown in Table 5.  Table 5 shows that the average n-gain for the two groups of students was high category.
In the end, the summary of the Mann-Whitney U test results for the average n-gain in both groups: MS P1 and MS R1 are shown in Table 6.  Table 6 shows that p-value is > .05; this means that there is no difference in the average n-gain in the two student groups. Table 2 shows that before the application of hybrid learning by using Brilian, the score of science management in both groups of students was very low. This happened perhaps because students were not accustomed to get a hybrid learning by using Brilian. After the application of hybrid learning by using Brilian, the score of students' science management became high. These results were achieved perhaps because students have become familiar with hybrid learning by using Brilian; whose phases teach cognitive domain according to Bloom's taxonomy. Changes in the scores of the pre-test and post-test that were quite accurate in the two groups of students were supported by the results of statistical tests which showed that: there was an increase in student science management learning outcomes in both groups significantly at the significance level α = 5%, with the average n-gain in both groups were high, and the average n-gain were not different in the two groups of students; as shown in Table 4, Table 5, and Table 6. These results support the opinion of Caporarello [1], Zhang [21] who said that e-learning is one of the technology-based solutions that are considered the most suitable for assisting in the learning process. In addition, it also supports Goh's [3] opinion who said that e-learning is considered the most appropriate in helping students in the learning process through ease of interaction with fellow students. Likewise, the results support Grail [4] who says that the Z generation feels comfortable and highly dependent on technology and has multitasking capabilities with a variety of online products and advanced technology equipment. In addition, the results of the study also support the findings of Gambari, Shittu, Ogunlade & Osunlade [22] research that stated that there are significant differences between the performance of hybrid learning study groups, e-learning groups and (conventional) control groups in supporting integrated learning. These results are supported by research of: Wijaya [23]; Sjukur [24]; Syarif [25]; Manggabarani, Sugiarti, & Masri [26]; Simarmata, Djohar, Purba, & Djuanda [27]; Muflikasari [28] who concluded that hybrid learning can improve learning achievement. This is in line with the results of Nisa's research [29], which states that students who are taught with e-learning get better grades than students who are taught conventionally.

Discussion
Longtime before all of that, Chang et al. [30] had conveyed his research findings that learning which carried out through smartphones produced very effective results to meet the information needs of the training participants. As well. Rambe [31] states that the use of social media can enhance social learning, enhance digital literacy, and provide co-production of knowledge in the learning community. Botha & Butgereit [32] also stated that the right integration of technology into learning by providing information and tutors became a significant scaffolding. Scaffolding in learning is very helpful in increasing the understanding of the message and increasing learning outcomes. The results also support the results of research that has recently been carried out by: Astutik & Prahani [33]; Husamah, Fatmawati, & Setyawan [34]; Jatmiko, et al [35 ]; Madeali & Prahani [36]; Pandiangan, Sanjaya, & Jatmiko [37]; Prahani et al [38] which states that valid learning, models, and media can be used to improve and achieve learning goals. This finding is also in line with McCarthy's research [12] which states that hybrid learning, accompanied by face-to-face discussions, is useful to improve understanding of the taught material and the level of involvement in the group.

Conclusion
Based on the research results and discussion above, it can be concluded that: science management learning by using Brilian is effective to improve student science management learning outcomes that is indicated by: • An increase in the learning outcomes of both student groups that are significant at the significance level α = 5% • The average n-gain in the two groups of students in the high category • The average n-gain for the two groups of students is not different