Development of an ERP System Design Course to Improve Students' Learning Outcomes

This paper describes the design and evaluation of an ERP system design course, which utilizes experiential learning theory and an open ERP system to provide the students with a holistic and cost-free learning environment. The objective of the course is to familiarize students with the ERP selection and implementation life-cycle. When designing the pedagogical framework, the goal was to help students, especially those with little to no working experience, acquire the necessary knowledge and develop practical technical experience. Additionally, it was necessary to help students appreciate the depth and issues involved in the ERP system selection and implementation life-cycle. Lectures, case discussions, system demos with interactive labs, and group projects were used all together to enable in-depth learning of the proposed topics. We focus on an open-source ERP system to provide the students with the required handson experience and demonstrate that such an approach can be utilized even by low-resource universities. This paper includes details of the proposed framework. The proposed framework was taught and evaluated in a 1-semester graduate-level course at the University of Jeddah, in Jeddah, Saudi Arabia. The course design was indirectly assessed using a survey, and the results revealed the positive impact that the proposed framework had on students' learning outcomes. The assessment results in support that experiential-based learning using open-source ERP systems can improve the learning outcome. For future work, we recommend evaluating the impact of the proposed framework over a larger period and systematically compare it with the more traditional lecture-based approach to teaching this course. Keywords—Enterprise Systems · Enterprise Resource Planning (ERP) Enterprise systems education Open Source Odoo Pedagogy Experiential learning theory


Introduction
The importance and benefit of ERP systems are no longer associated with large enterprises only. Currently, Small and Medium Enterprises (SMEs) are starting to recognize such a system's potential benefits. However, the high cost of deployment is often the main reason that prevents SMEs from adopting ERP systems. This same reason prevents most educational institutions from incorporating hands-on technical aspects of ERP systems in their curriculum. Although major commercial ERP systems provide an educational license, the cost is still high, especially when considering that such a system will only be used in a handful of courses. The importance of teaching the technical aspects of ERP and the associated high cost led many researchers to suggest the utilization of open-source ERP systems in education [1,2]. This paper describes our experience in utilizing an open-source ERP system, Odoo, in an ERP system design course curriculum. This course is an elective course in the Professional Master of enterprise information technology program at the University of Jeddah in Jeddah, Saudi Arabia. The main objective of the course is to familiarize students with the ERP selection and implementation life cycle. As this course is part of an IT program, IT students are expected to have a good understanding of both the theoretical aspects of ERP systems and the technical details of selection and implementation. Therefore, it is important to explain the theoretical component and expose students to hands-on practical experiences. Thus, an open-source ERP system [3,4] was utilized as a part of an experiential learning framework that incorporates multiple pedagogical practices. The proposed pedagogical practices and framework reduces two of the main entry barriers of introducing ERP systems into the courses, i.e., cost and training time.
Various solutions exist to provide students with a comprehensive learning experience that covers theoretical, practical, and soft skills. In this work, the ERP system design course is designed with Kolb's experiential learning theory in mind [5]. It includes self-learning through reading, summarizing, critically evaluating case studies (individual work), and a group work project that focuses on selecting and implementing an ERP system. The main objective of this work is to describe the design and evaluation of an ERP system design course that utilizes Kolb's experiential learning theory [5], Ruhi's pedagogical ERP practices [6], and Odoo's open-source ERP system [4]. Evaluation of the proposed framework proved its success, and the cost-free and limited-required training facilitate fast adoption of this framework.
The rest of this paper is organized as follows. The next section (Sec 2) provides, which provides a comprehensive background on experiential learning theory and related work. The background section is then followed by the course objectives and learning outcomes section (Sec 3). This is then followed by the course content section (Sec 4), which provides details on the course content and pedagogical practices. The next two sections (Sec 5 and 6) describe the student and course assessments, respectively. These are then followed by the limitations section (Sec 7) and, finally, the concluding remarks (Sec 8).

Background
This section provides a comprehensive overview of Kolb's Experiential Learning Theory (ELT) and a summary of related work.

Experiential learning theory
Higher education institutions thrive on incorporating experiential learning into their curriculums, such as in [7,8], due to their significant impact on students' understanding and mastering course topics [9,27,28,29]. Incorporating experiential learning practices into the curriculums increases the students' level of understanding and increases their employment probability [10]. Kolb's experiential learning theory [5] emphasizes the importance of experiences in the learning process. It also places significant emphasis on reflecting on the experience. Kolb's experiential learning theory describes the learning process as a cycle of four phases: concrete experience, reflective observation, abstract conceptualization, and active experimentation. Figure 1 depicts the four phases of experiential learning in Kobl's theory. Although Kolb's theory has received significant attention from researchers and educators worldwide, it is very general and cannot be easily adapted by educators [11]. For example, many educators have attempted to incorporate this theory into ERP courses. However, most of these attempts lacked a proper theoretical foundation [6]. To overcome this gap, the authors in [6] have developed a framework that maps pedagogical practices to the various learning phases of experiential learning theory. This framework aims to help instructors design comprehensive enterprise system courses in business school, with an experiential learning-based approach.
We followed the proposed framework in this work, with minor modifications to make it better suited for an IT-based course. Details of our designed course and pedagogical practices are provided in Sections 3 and 4, respectively.

Related work
A plethora of research work proposes utilizing different learning theories and pedagogical practices to significantly improve the learning outcomes of ERP courses [12,13,14]. The work presented in [2] is among the most relevant studies to our work, as the researchers have utilized experiential learning theory and an open ERP system to improve the learning outcomes of an IS course. However, unlike our study, the introduction to open ERP [2] spanned 1-hour only during the entire semester and was not project-based as in our work. Nevertheless, the authors have reported enhanced learning outcomes after their 1-hour tutorial [2]. It is important to note that the work in [1] was among the first studies that shed light on the benefits of utilizing open-source ERP systems in education instead of the commercial ones, which is the mainstream in education.
Similarly to our work, the study by [15] has utilized the ERP-based experiential learning framework proposed by [6] to design an experiential learning pedagogical framework for an enterprise systems course. Our work draws from the theoretical framework proposed by [6] yet differs from [15] as we used it to design a course that focuses on the design and implementation of an ERP system. None of the related work has focused on designing an experiential-based curriculum for an ERP system design course to the best of our knowledge. In which the objective of the course differs significantly from regular business and IS-based courses.
Moreover, the work presented in [16] describes an experiential learning approach to an undergraduate ERP course by incorporating hands-on labs that use Microsoft Dynamics. The course is designed to provide practical hands-on labs on typical business processes in various functional areas. This includes sales, purchasing, accounting, finance, and human resources. Another study described in [17] has proposed a framework to improve teaching business courses using ERP and simulation. ERP systems are used as a tool for students to learn business processes, and simulations are used to provide dynamic learning environments. Like other reviewed work, these studies differ from our study's main objective as our goal is to focus on the ERP design and implementation phase while exposing students to the main functional areas of ERP systems. For this, students must understand the different issues of this phase and various decisions that must be made.
To sum, none of the reviewed related work had investigated the impact of utilizing the experiential learning process along with open-source ERP systems in an ERP system design and implementation course. Previous works either studied the impact of open-source ERP after a 1-hour introductory session only [2] or used commercial ERP systems [17]. More importantly, all previous studies used ERP systems to provide practical experience on various business processes [2,15,16,17]. None of the reviewed studied focused on providing practical experience in the design and implementation phase of ERP systems. Consequently, we have carefully designed the assessments to achieve our course objectives, detailed next.

Course Objectives and Learning Outcomes
ERP system design is a graduate-level elective course in a Master's degree program titled Enterprise Information Technology. The course aims to introduce students to the life-cycle of ERP selection and implementation. This includes addressing the issues and critical success factors for a successful ERP system implementation. It also includes highlighting the various decisions that must be made during ERP selection and implementation during the life-cycle. On successful completion of this course, the student should be able to: • CLO 1 Understand basic concepts, architecture, and terminologies of ERP systems.
• CLO 2 Understand the life-cycle of an ERP selection and implementation project.
• CLO 3 Understand the process views of organizations and tools and techniques used to model both as-is and to-be models. • CLO 4 Analyze a current architecture and perform an effective gap analysis before an ERP implementation. • CLO 5 Compare different ERP systems.
• CLO 6 Describe the selection and implementation of an ERP system. • CLO 7 Describe issues typical of ERP implementation projects.
• CLO 8 Demonstrate an ability to work independently and in a group.

Course Content
This course is intended to give a depth understanding of the main topics listed in the previous section. The course covers the following topics: To obtain a holistic learning experience, we adopted the experiential learning framework proposed in [6]. The framework is developed to help instructors design ERP courses with experiential learning practices, to support a holistic learning environment that covers several learning processes and cognitive modes. The framework covers four learning processes: concrete experience, reflective observation, abstract conceptualization, and active experimentation. Additionally, each learning process is associated with a cognitive model, which is: experiencing and feeling, watching and reflecting, thinking and analyzing, and doing and exploring. The framework also recommends a set of pedagogical practices and maps them to the four learning and cognitive modes.  Table 1 visualizes the selected pedagogical practices for the ERP design course, along with their mapping to the four experiential learning processes and cognitive modes. Ruhi's [6] 's original framework contains two additional pedagogical practices: simulations/interactive assessments and workshops/walk-through assignments. During the course design, these two practices were not included due to time constraints, and more weight was placed on the course project. As it is depicted in Table 1, for this course, we have selected four pedagogical practices, which are: lectures, case discussions, system demos and labs, and a group project. The pedagogical practices were selected in such a way to ensure that all of the four learning processes and cognitive modes were covered by the primary mapping at least once, by any of the pedagogical practices. Details of each pedagogical practice are provided next.
Lectures: Two textbooks were used as the main reference for the lecture notes and course topics [18,19]. These books were used as a reference to build the necessary material in order to introduce students to all the topics included in this course. Lectures are often characterized as being very theoretical and teacher-centered. However, even with this criticism, research has shown that lectures still have merit, despite the search for alternative approaches [20]. For this reason, we have incorporated lectures in this course as a means to introduce the basic concepts in ERP system design.
Case discussion: Discussion of case studies provides a deep learning experience to students. It has been shown that case studies are an effective approach to teaching ERP courses [21]. In this pedagogical practice, students were given access to case studies. One case study was done collaboratively in a class, where all students were asked to read the report and participate in the discussion. Further case studies were assigned to students, and each student prepared a presentation about and managed the discussion with their classmates.

System demos and interactive Labs:
The main objective here is to provide hands-on practical experience in ERP system design and implementation. In this pedagogical practice and the next one, an Open-source ERP system (Odoo) is utilized to provide this experiential learning experience with no additional high-cost or timeconsuming training [4]. In addition to Odoo being open source, Odoo provides many training material and resources that help developers in all aspects of the implementa-tion and configuration of ERP systems. Selected training materials were used in-class with students as interactive labs. These contained procedural steps need to be completed by the students in order to configure different parts of an ERP system. Additionally, students were urged to consume the available training material to understand the concepts and complete the project requirements fully.
Group project: The course project comprises a major component of the coursework. Group projects provide a good source for learning important skills, including hands-on technical skills and soft skills, including communication, teamwork, and ethics [22]. Teaching these soft skills is hard to deliver by lectures only. This is why experiential learning that incorporates multiple pedagogical practices is important [23,24,25,26,6].
We propose a course project that includes various student activities and learning modes, including reading, analytical thinking, writing, speaking, discussing, and presenting. The project tasks include managerial competencies related to ERP selection and technical competencies related to ERP implementation. The project is comprised of five tasks. The description of each task is provided in Table 2, where each sub-task assesses at least one learning outcome. Results of student assessments are presented in the next section. Out of the seven-course learning outcomes listed in Section 3, five were assessed in the described project. The next section details the student assessment results. Visualise the "as is" business processes using process mapping. (CLO3) Analyze the business processes (activity and flow) and redesign them using business improvement or reengineering steps using (CLO3) Visualise the "to be" business processes using process mapping. (CLO3)   3 Based on the selected businesses and information requirements, identify the architecture (modules) of the proposed ERP system (CLO4) Perform a thorough gap analysis (CLO4) Thoroughly assess and compare 5 potential ERP systems; 2 commericial ones, (e.g.,SAP) and 3 open-source (e.g., Odoo,ERP5) (CLO5) 4 Describe expected challenges during post and pre implementation phases, and list potential solutions to overcome these issues(CLO6) Implement and test the most suitable open-source ERP systems (CLO6) 5 Wrap your work in a comprehensive report,sing the given template and outline (CLO6) Present the complete case study (CLO6)  Table 3 shows a mapping of the course learning outcomes and student assessment methods. Students' performance was assessed using written exams, case discussions, and course projects. Written exams were used to assess student's understanding of basic concepts, architecture, and terminologies of ERP systems, where the students obtained an average point of 90.17 with a +/−0.8 standard deviation. Case discussions were used to assess students' understanding of the ERP implementation life-cycle and associated issues. Students obtained a very high grade, 96.87, with a +/ − 0.75 standard deviation in case discussion. Lastly, the group project was used to assess students' performance in the remaining learning outcomes. As described in the previous section, the project had five tasks where each task is associated with different learning outcomes. Tasks one and two in the project assess the students understanding to process and model the as-is and to-be models. Task three of the project assesses the students' ability to analyze the current architecture and perform gap analysis and their ability to compare several ERP systems. Tasks four and five assesses the students' ability to describe the selection and implementation life cycle. Overall, the students obtained an average grade of 88.57 with a +/ − 6.3 standard deviation. The variation of grades in the group project was higher than that in other individual-based assessments. This can be attributed to the difficulty of the project tasks compared to the other assessments. All project tasks assess students' ability to work within a group, while the written exams and case studies assess their ability to work individually. Table 4 provides summary statistics of students' results per assessment.

Course Indirect Assessment
We evaluated students' learning outcomes using a survey. The survey's objective was to indirectly assess the effectiveness and usefulness of employing experiential learning to teaching the ERP system design course. The survey consisted of only six questions, and all students in one semester filled the survey. The results of the survey are presented in Table 5. The survey assesses students' knowledge of ERP before and after the course. The first two questions ask the students to rate their knowledge and understanding of ERP systems and business processes before the course. These questions showed that the majority of students (43%) had little knowledge of ERP systems and business processes. 29% of them had very good knowledge of ERP systems, and the rest had little to no knowledge of ERP systems. The remaining questions assess the students understanding and knowledge of the ERP system after taking the course. The survey results show that most students gained knowledge of ERP in the given course, which indirectly assesses the effectiveness of employing an experiential learning approach in this context. Lastly, the last question assessed the difficulty of the project, where most students reported that it was neutral, and only a few (14%) reported that it was very difficult.
The findings of this study are consistent with previous related work. Previous related work reported the positive impact that open ERP systems have on students' learn-ing and understanding [2,15]. Additionally, our results are in line with the findings of [16,17], which demonstrates the benefits of applying experiential learning theory to various ERP-based courses.

Study Limitations
There are two main limitations to the presented study. First, the study spanned one semester only, which had one section of students. Second, the study did not compare the impact of the proposed teaching approach instead of the more traditional lecturebased approach. These two limitations bound the generalizability of the drawn conclusions. Nevertheless, the course assessment revealed positive student outcomes. These limitations can be solved in future studies by applying the proposed pedagogical practices to many sections, spanning multiple semesters. Additionally, a comparison of with and without (or pre-and post-) the proposed framework can be conducted to examine its impact on learning outcomes.

Conclusion
Documenting the pedagogical framework used to teach ERP system design courses is very important to repeat and spread successful practices. This is especially true when the course is designed to include hands-on experience, with no additional overhead cost and limited training time [30]. This paper has presented a framework for teaching ERP system design, based on Kolb's experiential learning theory [5] and Ruhi's pedagogical ERP framework [6]. Assessment results revealed the success of the proposed framework. Future work to improve the proposed framework includes incorporating more hand-on labs, evaluating the proposed framework on more sections over multiple semesters, and comparing the proposed framework's outcomes with student outcomes from a regular lecture-based ERP course. Two more pedagogical practices can be added to the course design. These are simulations/ interactive assessments and workshops/walk-through assignments. Some students requested the latter during course feedback. This can be achieved by designing labs (workshops) with different difficulties to help students in their learning process. Some students prefer labs during the class, so it is easier for them to follow and ask questions. Not like the demos available on the website. In such labs, the theoretical concepts can be reiterated and shown in practice. Also, for labs to be effective (student dependent), exercises with clear steps must be included, along with hints for more skilled users. Interactive/ formative assessments are also a good pedagogical practice that can be implemented at different stages during the course. This will help the instructor evaluate students' current state and provide prompt help and feedback as necessary. Finally, engaging industrial partners can also be added to the proposed framework, where such partners can guide and assess the project deliverables, thus enriching students' experience.