Higher Order Thinking Skills for Improved Learning Outcomes Among Indonesian Students: A Blended Web Mobile Learning (BWML) Model

The purpose of this research was to produce a model that can be used as a reference in implementing hybrid learning and Problem Based Learning (PBL) in learning with the MoLearn application. This research was included in Research and Development/R&D type by developing the right learning model for the MoLearn application. The Blended Web Mobile Leaning (BWML) model test was conducted related to its validity and practicality. The result of research showed (1) quality test with average content validity (3.72) statistically in (rα = 0.25) and reliability in (α = 81), average construct validity (3.74), with validity of each aspect statistically in (rα = 0.20) and reliability in (α = 0.75). It can be concluded that the BWML model is qualified (valid in content and constructs, and reliable) by experts. (2) The feasibility test (practically used by students) had an average of 3.23 with a statistical aspect validity in rα = 0.91 and reliability in α = 0.99 This can mean that students stated that the model has novelty and is easy to use. The implication of this research is that a quality BWML model can be used to improve HOTs-based learning outcomes. Further research can be focused on looking at the effectiveness of the BWML model for improving HOTs-based learning outcomes


Introduction
Important problems are facing the world of education in Indonesia today, namely how to seek Higher Order Thinking Skills (HOTs) based learning outcomes improvement [1][2][3][4][5]. HOTs-based learning outcomes are student learning outcomes that involve high-level cognitive thinking activities from Bloom's taxonomy. Based on Bloom's taxonomic hierarchy, HOTs-based learning outcome indicators include analyzing, evaluating, and creating [6]. HOTs-based learning outcomes are very important because they provide superior student competencies to compete in the era of industrial revolution 4.0 and the demands of 21st-century learning.
On the other hand, 21 st century learning emphasizes the existence of learning innovations, including critical thinking skills, problem-solving skills, literacy, collaboration, decision making, creative thinking, responsibility, and being able to learn independently [1] [7][8][9]. Martin's research results, [10] show that on average Indonesian students are only able to recognize some basic facts and are not yet able to communicate and connect various topics, especially in applying complex and abstract concepts. The survey results show that the average score of student achievement is below the average international score. In line with the survey conducted by TIMSS, the survey conducted by the Program for International Student Assessment (PISA) the average achievement score at the HOTs level in Indonesia is still far below the international average. This fact is in line with the results of research [1-3] [5] that the learning process is still a teacher center and emphasizes more on the process of transferring knowledge.
The low learning outcomes of HOTs-based high school students are thought to have something to do with the learning process and model used. The learning model used is the conventional learning model. This model is considered unable to facilitate the development of HOTs-based high school student learning outcomes [1-3] [5].
Model Hybrid Learning is learning to provide the contents of a learning model in various media to keep up with current learning needs [24]. The application of Hybrid Learning can improve HOTs-based high school student learning outcomes but still needs improvement [24]. Models Hybrid Learning and Model PBL able to motivate students to conduct investigations and problem-solving in real-life situations and stimulate students to produce a product in improving learning outcomes based on HOTs [25]. Thus, educators need to be continuously encouraged to use e-learning technology and facilitate students with technology in improving academic learning outcomes [26].
To cover the weaknesses in the implementation of the Hybrid Learning Model and the PBL Model, it is very necessary to develop an Innovative Learning Model that can improve the learning outcomes of HOTs-based high school students. As an alternative solution that developing an Innovative Learning Model. This is in line with the findings of Hariadi's research [27] which suggests an innovative learning model that combines online learning and face-to-face learning. The innovative learning model developed is the BWML Model. The BWML model is expected to be able to improve the learning outcomes of HOTs-based high school students.
The BWML Model is a learning model that integrates the Hybrid Learning Model with the PBL Model which is supported by the use of the MoLearn application in every learning activity. The development of the model is Blended Web Mobile learning supported by the latest learning theories (constructivism, learning through observation, discovery learning, cognitive processes, metacognition, and scaffolding), as well as an empirical foundation from the latest research and scientific publications of researchers.
The MoLearn application is an application for Hybrid Learning that has been developed by the Dinamika University research team to improve the learning outcomes of HOTs-based high school students. In the MoLearn application, the teacher functions as a facilitator, mentor, and consultant so that students who learn to use the MoLearn application are required to learn actively. The addition of the MoLearn application with the Android version is because Mobile technology is considered an effective way to improve student skills such as positive thinking, collaborative thinking, communication, and is considered a major part of innovation in many fields of e-learning research [28]. Besides, research from Haerazi et al [29] states that learning with Mobile Apps can improve critical thinking skills in teaching writing skills.
The birth of the MoLearn application is based on the fact that current students are students in the Generation Z era. Among the characteristics of Generation Z are (1) Comfortable and highly dependent on technology, this is because Z-Generation grows surrounded by technology, (2) Multitasking with a variety of products online and sophisticated technology tools, and respect simplicity and interactive design, (3) Having a higher social responsibility with more information that can be accessed online, (4) Always connected, communicating through social networks, across countries and culture that indirectly affects the way of thinking and decision-making processes [30]. Educators must start thinking about a learning model that can align themselves with the needs of students, who always keep up with the fast development of gadgets today [31].
The description above strengthens the reasons for the need for the BWML model to improve learning outcomes for HOTs-based high school students. The main objective of this research is to produce a quality BWML model to improve HOTs-based learning outcomes. The BWML model has five phases, namely: (1) orientation based on IoTs and Big Data, (2) investigation, (3) analyzing, (4) presenting, and (5) evaluating. Each phase of the BWML model in the implementation of learning is carried out and supported by using the MoLearn application.

General background
The main objective of this research is to produce a quality BWML model (content and construct valid, reliable, and feasible/interesting). The main product in this research is the BWML model in the form of a BWML model book. Thus, this research is included in the type of development research (Research and Development). The development of the BWML model adapted Wademan's model development research design [8] [32]. The subjects in this study were high school students. The samples in this study were experts and students as the BWML user model. Experts perform validation regarding the content and constructs of the draft BWML model. The validity of content is the need for intervention and its design is based on current knowledge [32]. Construct validity is the fulfillment of a logically designed intervention [32]. The results of this expert's assessment are used as a reference for revising the draft BWML model. Students as users are taken from six high schools in four districts/cities in East Java Province. Students as test subjects use the BWML model learning with the MoLearn application, and after that are asked to fill out a student response questionnaire related to the application of the BWML model.

Instrument and procedures of research
The research instrument was a questionnaire for both experts and students which was adopted from Nieveen et al. [32]. The questionnaire for experts consists of two parts, namely (1) a questionnaire to measure the validity of content and (2) a questionnaire to measure construct validity. Questionnaire for students to measure the appropriateness or attractiveness of the BWML model of learning tools and activities. Research procedures include: (1) Preliminary research, which is carried out to obtain related data: (a) learning outcomes based on higher-order thinking skills, (b) PBL and hybrid learning models, (c) factors that support learning, (d) student and teacher opinions on learning. (2)

Data analysis data
Analysis uses descriptive statistics, namely the average score of the questionnaire that has been filled in by the expert. The criteria for the mean score used the Single Measures Interrater Coefficient Correlation (ICC) and Cronbach's coefficient alpha [8] [33] as in Table 1.  [34] This criterion is also used to analyze the quality of the developed BWML model.
iJIM -Vol. 15 (1) constructivism theory, (2) learning theory through observation, (3) discovery learning theory, (4) cognitive process theory, (5) metacognition theory, and (6) multi representation theory. Each phase is implemented and supported using the MoLearn application. Emphasis on the implementation of BWML with the percentages: 70% (on the job experience), 20% (mentoring and coaching), and 10% (classroom, course, and reading) [24] [35]. The internally organized strategy allows students to organize their thought processes, for example through investigations (phase 2: investigation). Gagne emphasized the importance of the role of cognitive strategies as one of the goals of teaching in schools. Learning how to think is also known as higher-order thinking skills. Students' knowledge of cognitive strategies in learning and thinking is an important component in achieving learning objectives, especially building HOTs-based learning outcomes. The hallmark of cognitive learning lies in learning to obtain and use representational forms (phase 2: investigation) that represent objects-the object faced, whether the object is a person, object, or event. These objects are represented or presented in a person through responses, ideas, or symbols that are all mental. The mental activity of thinking is exposed to objects that are initiated in consciousness, and physical objects as occur in observing, hearing, or feeling. Cognitive learning is closely related to the focus of this research, namely understanding the concept, which means that students must recall knowledge that has been learned in the past and utilize the potential of the environment as a learning resource.
Learning to think is faced with a problem that must be solved (phase 1: orientation), but without going through observation and reorganization in observation. The problems faced must be solved by mental operations, especially using certain concepts and rules and work methods. The ability of students to solve problems through representational work is one of the components of HOTs which is the focus of this study. Jonassen [36] and Chi, Glaser & Farr [37] that underlies phase 1: orientation. Jean Piaget studied how children think and the processes associated with intellectual development that are innately curious and try to understand the world around them. The child needs to understand the environment by investigating and constructing a theory that explains it (phase 2: investigation). Lev Vygotsky Theories [38][39] that the basis of Phase 5: evaluation. Problem learning by John Dewey describes a view of education, with the school as a mirror of the larger society and the classroom as a laboratory for the investigation and resolution of real-life problems (phase 2: investigation) [40][41]. Bruner [42] provides theoretical support to discovery learning. When discovery learning is applied to the sciences and social sciences, Bruner emphasizes the inductive reasoning and investigative processes that characterize the scientific method (phase 3: analyzing).

Syntax formulation for the BWML model
Syntax is the steps listed in the lesson plan and the steps that must be followed when the teacher implements the learning model in the classroom. BWML model has five phases, namely: (1) orientation based on IoTs and Big Data, (2) investigation, (3) analyzing, (4) presenting, and (5) evaluating. Each phase is implemented and supported using the MoLearn application. The BWML model aims to improve HOTsbased high school student learning outcomes and other goals, namely to generate motivation, activity, and student responses in learning. To achieve these goals, Model BWML was conducted through a collaborative and cooperative approach to scientific work (scientific approach), hybrid learning, application integration MoLearn, social interaction through the experience of independent study and group, and the grain problem-based contextual IoTs and Big Data. The learning objectives are explicitly contained in the syllabus and lesson plans (RPP) which are made by the teacher as general guidelines in implementing learning in the classroom. Good learning objectives are oriented specifically to students, contain a clear description of the assessment situation, and contain levels of performance achieved in the form of success criteria in learning. In general, the selection of subject matter must refer to basic competencies and predetermined indicators.

BWML model quality and advisability assessment
The results of the quality assessment of the BWML Model are presented in Table 3 regarding the quality of the model and Table 4 regarding the feasibility (attractiveness) of the model. Table 3 Table 2 shows that the construct validity and reliability of the BWML Model include: (1) Consistency in developing the BWML Learning Model, (2) Framework for the formation of the BWML Learning Model, and (3) Learning Description The BWML Model has an average validation score of 4.00, 3.56, and 3.67 with very valid criteria and rα .20 is greater than r table, so that each component is declared valid. As for the reliability of each component in terms of the α value, all of them are at a value of .75, so that each component is declared reliable. Ease of understanding learning device components 3.19 Ease in following the components of the process skills that are trained 3.25 Elements of attractiveness (fun and fun) in learning activities 3.00 Teacher guidance during the learning process 3.30 The relationship between BWML model learning and learning outcomes 3.06 Table 3 shows that after a large class trial was conducted, the feasibility of the BWML Model has an average validation score of 3.23 with very valid criteria and rα = .91 is greater than r table, so that each component is declared valid. As for the reliability of each component in terms of α value, all of them are at α = .99, so that each component is declared reliable. Thus it can be said that the BWML Model can be said to be feasible and attractive to be applied in senior high school. The social system in the BWML Model refers to a learning model based on Vygotsky's constructivist [38][39]. For a learning model can still be implemented must be supported by learning tools and the completeness of the facilities used. The facts show that an environment that provides a conducive atmosphere for teaching and learning activities will promote good instructional delivery and better learning outcomes. The support system for a learning model is all the means, materials, and tools for implementing the BWML Model using the MoLearn application. The support system in the BWML Model uses the MoLearn application, namely: (a) Learning tools refer to the BWML Model, namely: syllabus, lesson plans, worksheets, student teaching materials (BAM), HOTsbased learning outcome evaluation instruments. (b) MoLearn application as the main support in learning. (c) Learning media in the form of virtual labs and provided computers/laptops, as well as internet networks for access to data literacy. Gadgets and games have a positive impact on the world of education [43][44][45][46][47][48][49]. The instructional impact of the BWML Model using the MoLearn application is that students can improve HOTs-based learning outcomes.

Conclusion
This BWML model aims to improve HOTs learning outcomes and other goals, namely to generate motivation, activity, and student responses in learning. To achieve these goals, Model BWML was conducted through a collaborative and cooperative approach to scientific work (scientific approach), hybrid learning, application integration MoLearn, social interaction through the experience of independent study and group, and the grain problem-based contextual IoTs and Big Data. The result of research shows BWML model is qualified (valid in content and constructs, and reliable) by experts. (2) The feasibility test (practically used by students) an average of 3.23 with the validity of the statistical aspects in rα = .91 and reliability in α = .99 This means that students claim that the model is novel and easy to use. This research implies that a quality BWML model can be used to improve HOTs-based learning outcomes. Further research can be focused on the effectiveness of the BWML model to improve HOTs-based learning outcomes.

Acknowledgment
Our  Julianto Lemantara is a researcher and lecturer in the Department of Information System, Dinamika University. He graduated from STMIK Surabaya with bachelor's degrees in Information systems. He continued his study and completed his Master's degrees in Information Technology from Gadjah Mada University. He has research interests in the field of information systems, decision support systems, and data mining. Until February 2020, he has published his researches in Q2 International Journal, International Conference with Scopus Index, and Accreted National Journal (email: julianto@dinamika.ac.id) Binar Kurnia Prahani is Doctor of Science Education. He is a researcher and lecturer in Universitas Negeri Surabaya. His research interest in the education field, such as higher order thinking skills, the utilization of technology in education, the latest learning model, and science education. (e-mail: binarprahani@unesa.ac.id) Budi Jatmiko is a researcher and lecturer in Universitas Negeri Surabaya. He is also a professor in Department of physics, Universitas Negeri Surabaya. His research interest in the education field, such as higher order thinking skills, the utilization of technology in education, the latest learning model, and science education. (e-mail: budijatmiko@unesa.ac.id; budi@dinamika.ac.id).