openSAP: Why are Enterprise MOOCs Working?

The rapid growth and changes in technologies demand a rapid transfer of knowledge in many areas. After more than five years of running openSAP, a list of success factors for Enterprise MOOCs has been identified.


INTRODUCTION
The rapid growth and changes in technologies demand a rapid transfer of knowledge in many areas. Not only in the education sector, but also in the economy, opportunities for a fast flow of information are very important. SAP meets these growing requirements with its Enterprise MOOC platform openSAP and thus meets the growing need to flexibly train employees, but also consultants, users and deciders. The aim is to convey knowledge about new technologies quickly and directly to the target group.

II. EASY ACCESS FOR AN OPEN COMMUNITY
In April 2013, SAP launched its first courses on the openSAP MOOC platform and was the first company that invest in MOOCs and an own platform. This made it possible to offer free courses for an open user community. The success confirms this effort. While around 55,000 learners had registered for the first course, more than 800,000 learners from all countries can now be counted among the user group of the offer. With the increasing interest, the range of courses has been steadily expanded and today more than 180 courses are offered on openSAP.
The target group of the courses offered on openSAP are developers, users, other stakeholders and last but not least SAP employees. However, the courses are open to all interested parties. Learners only need Internet access and a user account on openSAP to participate in the courses, which can be easily created as long as a user has a valid and active email account. Learners with an existing SAP account can also log in to openSAP via this account, which further minimizes the access hurdles.
The open nature of this learning format makes it possible to make knowledge quickly and easily available to all target groups, while the MOOCs approach gives the participants a great deal of freedom and flexibility. They can divide their learning times as freely as possible and they are not constrained to a fixed place of learning. This makes it possible to reach learners all over the world -across different time zones and independent of stationary training locations.
III. AUDIENCE AND USAGE Within the first five years, more than 3 million enrollments can be counted for the courses, which means an average of more than four courses per user. The average amount of enrollments per course (measured at the course end) is 9.400. Of all learners registered for the course, about 50 % participate actively in the course and 25 % complete the course successfully (Record of Achievement). More than 180.000 posts have been opened up in the course forums. Based on the learners age distribution (see chart 1) openSAP clearly belongs to lifelong learning. Based on surveys, that are conducted at the end of each course and have been answered more than 150.000 times, it can be stated that: 98 % of the survey participants are (very) satisfied with the openSAP learning experience. 95 % think the skills taught in the course will help them in their (future) work while 99 % plan to take additional openSAP courses in the future. The Net Promoter Score (average) is above 50, which can be considered a very good value.
IV. COURSE STRUCTURE AND FORMAT openSAP follows a traditional xMOOC format. Typical for this format is the structured preparation and structuring of the material into individual sections (usually one per week) which contain the individual learning items. The typical course duration is four to six weeks. Most courses are offering a linear learning path, however some elements may be optional. In a few courses, individual learning paths with alternative sections are offered. New content is activated at the beginning of each course week. If several courses run in parallel on the platform, the activation and the dating of the weekly homework takes place distributed on the individual week days. Whereas this was originally introduced in order to avoid peak loads (high access rates can be observed especially after the activation of new content and before the expiry of deadlines), today this mainly serves the purpose to accommodate learners who attend several courses at the same time.
Each week there is a repetitive sequence of video-based knowledge transfer and knowledge verification via automated self-tests. At the end of a week, usually a homework task with points follows. This structure is supplemented by the provision of collaboration rooms and a forum integrated into the learning platform.
There is a final examination at the end of the course, which usually grants 50 % of the total points available, while the remaining points can be achieved through the weekly homework. In general, the "graded" tasks only allow one attempt, which is limited to 60 minutes.
In addition to surveys based on the quiz module, it is also possible to leave textual feedback in an "I like, I wish" style based on the design thinking methodology.
Some courses offer peer assessment or team peer assessment tasks. These allow for more flexible tasks that go well beyond simple quizzes.
V. SUCCESS FACTORS In this chapter we identify different success factors that we assume are responsible for the sustainable success of openSAP and Enterprise MOOCs in general.

A. Provide enough team resources and an interdisplinary team
The courses are developed by a team of 20 people, who bring experts from the respective departments into the team to provide technical support and domain knowledge for specific courses. Seven teams are involved in course development and execution at the operational level. The Helpdesk team answers questions from users. These questions can be sent via a form contained in the portal. At the same time, data about the browser is used and the course context is attached to the helpdesk ticket. So that helpdesk tickets can be processed promptly, the employees are distributed in several time zones.
The communications department takes care of design and content. In addition to the brand-specific design of the platform, the design of course certificates and open badges, this also includes course communication in the form of course-specific and global news, which are available within the platform and the apps, but are also sent to the learners by e-mail. It is also responsible for releasing the slide decks used within the courses. In addition to compliance with the Corporate Style Guides, legal clarification of copyrights is also taken into account during this process. The Instructional Designers provide cross-course advice on media and, in particular, self-test creation and continuously monitor solution rates. Each course is supervised by a dedicated project manager over its entire duration, i.e. from conception, creation, active duration and cool-down. They are accompanied by content experts.
The operations team is responsible for uploading the content to the platform, including final quality control. This team is also responsible for sending course-and platformwide messages to all users. Even if the platform allows these activities to be carried out by the respective course supervisors, such centralization allows communication measures and deadlines to be coordinated. In particular, users who participate in several courses at the same time can be taken into consideration.

B. Ownership of production workflows
In traditional learning environments, workflows of production were often related to the development and updating of content involving external agencies. MOOCs and their primary video-based content delivery channel allow easy production workflows. Content experts can produce content themselves using screen recording software. Delivering the content in small content items also allows easy updating of course content, by extending or updating the courses.

C. Offer relevant topics
Courses are tagged with high level topics, where one course can be tagged with more than one topic.
A dedicated program manager has the task to combine the demands of the learner's community with the teams from within SAP, that want to offer courses. Surveys and the I like I wish forum help to get qualitative and quantitative feedback on desired content. As shown in Table 1, there is a good mix from different topics. The task of the program manager is also to orchestrate and distribute the courses over the year (courses are offered throughout the year, excluding breaks in summer and winter holidays).

D. Course promotion
Courses are published on the platform about six weeks before the start of the course and advertised in regular platform-wide messages, which are also sent to all users by email. From this point on it is possible to register for the courses.
While course announcements reach more than one third of the enrolled users (users that either read the mail or open the announcement in the app or on the web page) global announcements are opened up by an average of 20% of all users. Excluding users that deactivated mail notifications or have been marked inactive due to non-working emailaccounts this means around 120.000 users are reading a global news in average. Additional marketing is conducted through social media sites, SAPs website and the company's social communities. No paid promotion is used to gain new users.

E. The xMOOC Course format
The courses always follow a fixed schedule. The content is outlined on an introductory description page, the lecturers are introduced, and there is also information on the target group and any necessary access requirements. The courses are structured on a weekly basis. Each block consists of instructional videos followed by self-tests and a final, scored weekly test. In addition to the tests, access to systems is also offered where possible so that the acquired knowledge can be evaluated in a practical way. At the end of the course the participants can take a final test and receive a certificate after passing. A forum is available to the user during the course. Here the learners can communicate with each other and with the support team. This xMOOC typical format is well established in lifelong learning on many platforms like edX and openHPI, still The International Conference on E-Learning in the Workplace 2019, www.icelw.org ICELW 2013 Template #1 3 SAP was one of the pioneers in adapting and applying this to enterprise learning. The idea to use mobile apps to access the course content through mobile apps was sketched in the early beginnings of the project [5]. While most usage of the platform is conducted through desktop devices and a web browser (desktop web), there is also a significant usage through a web browser on a mobile device and the native apps, which are offered for iOS and Android. Most of the users, that are using the apps, or the mobile web page also use the web version on a desktop class device. For the app users, two thirds of the time spent within the apps is used watching videos (65% on iOS and 67,5 % on Android).

G. Allow easy coexistance with daily work
As shown in the activity heat map, openSAP is mainly used during working hours. Also, there is more usage in the early evening hours in the beginning of the week. This could be related to more motivation after a recreational weekend, but also to the fact that new content is published at the beginning of the week and leaners like to get things done. In general, the main usage seems to be after other office tasks have been finished, as it can be seen by the relative low usage on Monday morning. While there is more activity on workdays, the sessions duration is higher on Sundays, indicating that learners that learn in their private surrounding tend to learn in longer sessions and learning in the workplace is happening in smaller chunks.

Chart 3: Learners Activity heat map by local time and day
Concepts like the usage of personal learning objectives, that are described in [4] and have recently been tested on openSAP allow the learners to concentrate on those parts of the course that they are interested in, while still being able to track the progress of the learning objectives in a dashboard.

H. Regular course communication
Unlike webinars or onsite seminars, MOOCs require learners to proactively access the content themselves. This requires them to remember they are part of a running course and go to the website or open the mobile app. To minimize this hurdle, regular communication is key to success. This communication takes place in the course announcements that are send out by mail and are available online but also through automated emails.
The enrollment mail is sent out automatically after the user's enrollment and contains basic information about the course like deadlines and tips on how to learn. A few days before the course starts a reminder mail is send out. During the course an announcement mail is send out each week when new content is published or deadlines approach. High opening rates and activity after the sending prove the effectiveness of this communication channel.
Users that engage in the course forums also receive forum notification mails when there is activity in a thread, they participated in. They can also opt-in to a daily forum digest mail. It is also planned to send out this notification via Push Notification to the mobile apps.
Automated reminder mails have been tested in an AB experiment, described in [3] and proved that automated mails also increased the course activity of those users that received those mails significantly. It also showed that the course videos, but not the community contributed content in the forums is the trigger to re-enter the course.

I. Interactive and Peer-Assessments
Multiple-Choice tests are a good fit for self-tests and graded assessments, these assessment format have a limited flexibility. Enabling assessments that will lead to a more indepth reflection based on blooms revised taxonomy [6], while still allowing gradeability without requiring too much workload from the teaching team.
For a few coding introduction courses an open-source programming platform was used [7]. For other courses a (team) peer assessment workflow was added to the platform. This allowed assessments like "write a concept for an app" or "write an app", that led to over 1000 apps submitted in one course.

J. Providing benefits for the learner
This format offers several advantages for learners. Access to the resources is free of charge, there is no maximum number of participants and only any email address is required for registration. Participants also enjoy great flexibility in the choice of learning location and time, and access is possible on all devices. At the same time, the small-scale content allows the learning process to be spread over several learning sessions. This allows learners to work at their own working speed and, if necessary, repeat individual parts. Motivational learning is to be achieved through certificates, high-quality content and gamification.

K. Providing benefits for the provider
Early after the start of the pilot phase, it was foreseeable that the openSAP platform would become a long-term project due to the positive feedback, the significantly higher than expected number of users and the internal demand from potential course providers. This was accomplished by a corresponding organizational structure and continuity.
One of the main advantages for the provider is that knowledge can be disseminated much faster and more efficiently than by conventional means. New courses can be produced quickly and efficiently. Costs for classroom training can be avoided and the geographical range can be extended. In addition, the forum offers a valuable channel for collecting customer feedback within the courses. This not only involves feedback on the courses themselves, but also on the content of the company's products and services. At the end of a course, learners are asked for feedback in the form of a short survey. According to this survey, 99 percent of the participants want to attend further courses. The learning experience is assessed as positive by 98 percent, with 95 percent of learners stating that they find the offer relevant for their professional development. These figures also confirm the success of the model for SAP.

L. Co-Development and research
The cooperation with the Hasso Plattner Institute has also proven to be a goal-oriented approach for openSAP. As a result of the research cooperation, joint solution concepts such as the Team Builder [1], the (Team) Peer Assessment and offline capable apps have been developed. The focus here is on approaches that combine the requirements from the company's and the learner's point of view.
The deep integration into the existing company IT, such as the possible single sign-on of the users, allows an easy use and also the provision of internal course offers for certain target groups. Through scientific monitoring and evaluation, these measures can not only be evaluated and improved in themselves, they can also be strategically further developed. Learning analytics and reports, enabled by [2] are an important building block of this. Since the high level of activity on the platform allows quantitative testing of theoretical concepts, openSAP is also of interest from the Institute's point of view (HPI) and allows AB-Testing [3]. While this close and project centric view makes sense in the early years of the project, in later days a more product-based approach (while still being agile and user centric) should be followed to increase sustainability.
VI. CONCLUSION After more than five years of running openSAP, a list of success factors for Enterprise MOOCs has been identified. While these factors are important, they will only work if the fundamentals are set: Interesting content that is relevant for the learners, presented by authentic course hosts, and provided on a platform that enables a good learner experience. Providing a platform like openSAP should be understood as a process, that is focused on steady optimization. This includes course formats and concepts, but also the learning experience based on the platform. The large number of learners enables a set of quantitative methods like Learning Analytics, AB-Testing, Automated Quality Control, Polls, and Surveys that should be used to reach a win-win situation for enterprises and learners.