Adaptive Learning Environments: A Requirements Analysis in Business Settings

The design and development of an adaptive learning system (ALS) should be guided by a thorough analysis of users expectations and needs. A requirements analysis has been carried out by means of scenario-based semi-structured interviews in order to investigate the personalization and adaptation preferences of different stakeholder groups in business settings. Results show that an ALS has a decided advantage over a non-adaptive learning system by offering individual treatment of learners. The adaptation of content and learning activities to learner knowledge and learning goal, particularly determined by the job role, is perceived to be most relevant.


I. INTRODUCTION
The ability of an e-learning system to adapt to the broad range of learners' characteristics is promising. Nowadays, Learning Management Systems are used to deliver learning content to learners. However, usually they have weak or no adaptive functionality and hence do not consider the diversity of learners [1].
Web based training systems can be automatically adapted to the learner. They can take into account, for example, learner knowledge and abilities, learning goals, preferred learning style, and user device. Many types of adaptation features and techniques have been studied, e.g. [2,3,4]. As there are so many, it is not obvious which features and techniques are most wanted by both trainers and learners. To ensure the value of an adaptive learning system and its acceptance, the users' requirements, preferences, and expectations need to be incorporated.
The EU co-funded GRAPPLE 1 ("Generic Responsive Adaptive Personalized Learning Environment") project aims at delivering to learners an adaptive technologyenhanced learning environment incorporated into Learning Management Systems. To ensure target-oriented work from the beginning, a requirements analysis has been conducted in order to gather the needs and expectations of stakeholders towards adaptive functionalities. This paper is structured as follows. Firstly, the requirements elicitation methodology is described. Subsequently, the interview questions are explained and the results are summarized. Afterwards, essential and pertinent outcomes of the interviews are highlighted and further discussed, followed by a concluding section.

II. REQUIREMENTS ELICITATION METHODOLOGY
The purpose of the requirements analysis was to gather personalization and adaptation needs of corporate users with regards to an adaptive learning system (ALS). To develop a comprehensive view on users' requirements two groups of corporate stakeholders were consulted: learners and training providers. The first group consisted of corporate learners and the second one included those persons who are responsible for the provision of courses such as trainers, lecturers, authors, and human resource managers. The methodology chosen for eliciting requirements consisted of scenario-based semi-structured interviews. The provision of a common structure with key questions ensured consistency between interviews carried out by different persons and in different countries, while still allowing sufficient flexibility for shaping the flow of information and for explorative data gathering. To enhance interviewees' understanding and to contextualize the questioning, a scenario illustrating the use of an adaptive learning environment from different perspectives was created as a supporting tool for the interviews [5]. To allow for systematic data collection, an interview guide and manual were developed and distributed to all interviewers. The key questions (open and closed ended) were on the current usage and perceived benefits of adaptive learning systems as well as on needs and preferences w.r.t. adaptation features and technologies.
Interview documentation was carried out in two stages; through narrative interview summaries (providing an overview of the conversational content) and through interview data sheets (for quantitative analysis). The qualitative data was examined and aggregated by a combined approach of quantitative and qualitative content analysis condensing the data to its essential content through qualitative interpretation and quantitative description [6]. Frequency and statistical analyses of the consolidated interview data were carried out in order to reach a comprehensive and in-depth understanding of users' views.

A. The demo scenario
To aid the interviewees in understanding adaptation principles, a demo scenario was generated, which contained several practical illustrations of using an adaptive learning environment in a business context, from a learner's as well as from an author's perspective.
The scenario involved a business English course used by a publishing company for refreshing and training employees' skills. Use cases from a learning perspective illustrated the use of different adaptation technologies (adaptive navigation support, adaptive content and learn-ing activity selection, adaptive presentation) for personalization to different employees or groups of employees based on user characteristics (job role, preferences, knowledge) and user platform [3,7]. A use case from the authoring perspective exemplified the authoring process for realizing a course with such adaptive functionalities.
The scenario document was sent to the interviewees before the interview itself. At the beginning of the interview the scenario was recapitulated and also served as a point of reference during the conversation. In addition, an adaptive business English course [8] was created with the AHA! System [9]. The course followed the demo scenario and provided interviewees with a live demonstration of an adaptive learning environment.

III. THE INTERVIEWS WITH CORPORATE STAKEHOLDERS
During May and July 2008, 26 interviews with corporate users were conducted; 8 with learners and 18 with training providers. By distributing the interviews across six European countries and institutions, data from a broad range of users was gathered.
The conduction and analysis of interviews required much effort compared to other data collection techniques. The limited sample size of 26 therefore requires that the results and conclusions drawn from the collected data be handled with some care. Nevertheless, the semi-structured approach allowed for the gathering in-depth data in a flexible and open manner while still ensuring consistency between interviews.

A. Response to preliminary scenario
As mentioned in section A, the interviewees received a demo scenario and were requested to read it through prior to the interview. At the beginning of an interview stakeholders were asked to gauge the relevance of the scenario to their own situation. They noted that the scenarios were a useful introduction for the interviews in setting the scene and also during the interview itself to illustrate certain concepts of adaptation.

B. Current usage of learning systems
The initial part of the interview concerned interviewees' current use of learning systems and adaptive learning systems. This included questions on their experience with (adaptive) learning systems and the purposes and benefits of using them. In the following, we present each question and summarize the results. Questions indicated with OQ are open questions. Those indicated with Q are closed questions with given answer alternatives to choose from. Q_B1. Do you use any learning system? 17 out of 26 interviewees answered this question with "yes". The majority of training providers, i.e. 13 out of 18, used a learning system. In contrast, only half of the learners had experience with learning systems.
OQ_B2. Which learning system have you used? There were many different learning systems mentioned by the interviewees while answering this open question. This probably stems from the fact that the interviews were conducted in many different European countries and companies. This shows that there is no single de facto learning system, although Moodle appears to be the most popular one.
OQ_B3. How often do you use a learning system? Most training providers use learning systems daily. In contrast, the interviewed learners do not regularly use learning systems, some of them even very rarely. This may depend on the diffusion rate of e-learning in companies.
OQ_B4. How long have you been using learning systems? Besides the fact that training providers use a learning system more regularly than learners, most of them also have long-term experience with them. Also, most learners have been using a learning system for at least one year.
Q_B5. Do the learning systems you have used so far provide any adaptive features to users? Half of the interviewees stated that the learning system they are using provide adaptive functionalities. In the next questions we asked those saying 'yes' for more details.
OQ_B6. If yes to the previous question, which adaptive features do they provide? Regarding the previous question, it turned out that most of these adaptive functionalities are very weak and simple, e.g. customizing the graphical interface or activating links if the learner has visited certain pages.
OQ_B7. For what purposes or tasks are you using (or would you use) a learning system? Summarized, most interviewees (would) use learning systems for professional and further development. This purpose was stated by the majority of learners. The second most frequently mentioned item, "course provision and management", was unsurprisingly of more concern for training providers than learners. Table 1 summarizes the answers of both groups. OQ_B8. What are the benefits of using a learning system? The answers to this open question are summarized in Table 2. The main issue was time management for the learners (75%), while this category was of concern to half of the training providers. The mobility aspect was mainly mentioned by training providers. They also emphasized the benefit of course management.

C. Adaptivity needs and preferences
The second and main part of the interview concentrated on the opinion and requirements regarding adaptivity and queried reasons and tasks for using adaptive learning systems, benefits and essential features of adaptive learning systems, and adaptation features and dimensions.
OQ_C1. What do you think are the purposes or tasks which an adaptive learning system (ALS) is especially suited for? This question is an extension of question OQ_B7. In this case we wanted to find out for which purposes users would primarily use an ALS. Table 3 lists the top nine answers. The interviewees did not provide any concrete ideas but mainly stated that ALSs are suited for structured professional training in general. Furthermore, they see a purpose of ALSs in personalization tasks in general.  The top answer for this question is that interviewees see a benefit in personalization and the consideration of user needs. Similar to OQ_B8, 'time management' is also named in the sense of saving time. Table 4 presents the top benefits of an ALS in descending order by the number of interviewees who said so.
In the following, the interviewees' answers in terms of adaptation features and dimensions are presented. In both cases we first asked an open question in order to gather the interviewees' initial ideas and expectations. In the subsequent question we provided a comprehensive list of adaptation features and adaptation dimensions derived from the literature [e.g. 3,6], and asked the interviewees to rate the importance of each item on the list.

OQ_C3. To which characteristics of the user or environment would you expect a learning system to adapt?
Most interviewees expect adaptation to a learner's previous knowledge. 62.5% of the interviewed learners and 72.2% of the interviewed training providers said so. Adaptation to the access device was primarily mentioned by training providers (50%) but not by learners (12.5%). Furthermore, adaptation to the job role of the learner was mentioned with the same number of responses per group. Table 5 summarizes the results. Other frequently mentioned adaptation features were disabilities, learning style, interests, age, and motivation. Q_C4. I will now list features that are reported in the literature to function as source of adaptation, i.e. characteristics of learner or environment that may be considered by an ALS when adapting to the individual learner. Please indicate your opinion on the importance of adaptation to each of these features on a scale from 1 to 10 (1 being unimportant and 10 being very important)! In this question we asked every interviewee to rate fourteen different adaptation features. All adaptation features where rated to be quite important. "Learning goal/task" and "learner knowledge" were found to be most important (with average importance ratings above 9), while "user location" received the lowest rating with a mean of 5.38. Table 6 lists the features and their mean rating by the interviewees. In addition, the table presents the lowest rating (min.), the highest rating (max.) as well as the standard deviation (S.D.).  Table 7 lists the top six categorized answers of the interviewees to this question and the number of times they were mentioned. According to these answers, most interviewees expect adaptation in a way that provides them with relevant content. They also expect that the content and its media formats are adapted. Furthermore, it is expected that the system adapts its navigational interface. We presented both superordinate categories and more specific subcategories of adaptation. Table 8 lists the adaptation technologies (superordinate categories are in bold). In general, every adaptation dimension was rated quite high with mean importance value greater than 6. The dimension categories adaptive content selection, adaptive learning activity selection, and adaptive assessment were judged to be most important with an average rating of 8.23 each. Specifically, additional explanations (M=8.15) and prerequisite explanations (M=9.04) were highest rated in the adaptive content selection category. In the adaptive assessment category the highest assessed dimension was adaptive testing (M=8.31). Furthermore, the intelligent analysis of solutions was highly rated. Least importantly judged were adaptive hiding (M=6.12) and adaptive link generation (M=6.27) from the group of navigation support techniques.
A statistical group comparison of the importance judgements of learners and training providers showed that training providers placed significantly higher importance on adaptive content selection techniques (i.e. additional and comparative explanations, explanation variants, and sorting) than learners. Moreover, a significant difference could be found for adaptive collaboration support, which was again judged to be more important by training providers than by learners.
OQ_C7. What functionality is essential for an adaptive learning system to be useful to you? Interviewees stated most often, that it is essential for an adaptive learning system to consider a learner's previous knowledge or competencies (c.f. Table 9). This was followed by evaluation and monitoring, content personalization, user preferences, platform, and job role. IV. ANALYSIS OF THE INTERVIEWS Interviewees provided useful comments while talking in the interviews about adaptive learning environments. These comments were not necessarily related to particular questions of the interview guide, but were documented and reflected in the interview summaries. Hence, for the purpose of further investigating and elaborating users' requirements, we also analysed the interview summaries and extracted meaningful statements from them. Below, we provide a summarized analysis of the most pertinent topics and results of the conducted interviews.

A. Perceived benefits and disadvantages of adaptive learning systems
There is a common agreement among interviewees that an adaptive learning system has a decided advantage over a non-adaptive learning system by offering individual treatment of learners. Thus, the learning content is tailored to the learner's needs for a certain learning task instead of presenting unwanted, unnecessary, or tedious learning content.
Some interviewed users also emphasized that an adaptive learning system appears especially suited for the multi-faced nature of companies, especially large ones where personalization is difficult.
Some interviewees also think that an adaptive learning system may increase the learner's motivation, for example, if a learner has a limited understanding of the presented learning content and the adaptive system supports the learner by "linking familiar knowledge with the unknown".
Some of the interviewees appreciated that an ALS not only offers advantages to the learner but also to a teacher/tutor, such as the possibility to "check more efficiently the learning path" and to save time.
However, some interviewees fear that there is more effort needed by the author when creating an adaptive course. The reason is that many learning objects have to be created to serve different knowledge levels, learning styles, etc. and because the creation of rules is very complex and requires a lot of experience and insight.
In contrast to the fear of the complex and laborintensive work process described above, very few interviewees disagreed that creating an adaptive course is much harder than creating a conventional course. The advantage of an adaptive system is seen in the combined usage of learning objects from different expertise levels, rather than in a knowledge level context only. This could be more efficient and more motivating to employees and so improve their learning process.

B. Adaptation features
The most prominent features of adaptivity (with a mean rating greater than or equal to 7) in descending order as rated by the interviewees include learning goal/task, learner knowledge, language, user role, learning/cognitive style, platform, motivation and student qualification (c.f. Table 6).
The table of the most wanted adaptation features does not explicitly establish which features are essential but suggests a ranking. The reason for that is that some interviewees consider most adaptation features as important. One training provider declared that "an adaptive system should adapt to all necessary characteristics of the user and environment in order to enable learning without any barrier". Another thinks that there is almost no useless feature. Nevertheless, there are a few adaptation features which are emphasized by the interviewees during the conversations.
The majority of interviewees think it is very important that an ALS adapts to the learner's prerequisites and the resultant level of knowledge or skills.
For many corporate learners it is also particularly important that an adaptive system considers their job role, career path and learning goal, because these features are determined by the current position or future career levels. This is an essential function for corporate learners because they don't have to search for relevant courses, but get presented the most relevant ones. To the mind of some interviewees this saves a lot of their time, which is an important factor in corporate settings, and may enhance learner motivation.
However, the cost of providing adaptation to learner knowledge and learning goal were recognized by some training providers as essential issues.
Interestingly, the user role was only mentioned explicitly by two interviewees but high rated anyhow (cf. Table  6). It is possible that the user role and job role were used synonymously.
For some other interviewees mobility is a key factor. Thus, the adaptation to the platform in an easy and usable manner is important to abolish "technical stumbling blocks" and to guarantee accessibility with different devices (e.g. desktop PC, mobile phone, PDA, inside a car etc.).
Furthermore, it was declared that the language of the system should be adapted to the one preferred by the learner, because otherwise users would not be able to use the system properly.
The learning and cognitive style of a learner is a relevant adaptation criterion and mentioned explicitly by five interviewees. One learner stated during the interview that adaptation to the learning and cognitive style should be done in an intelligent way regarding principles of learning psychology, i.e. a media type should not be completely avoided but only reduced. Before adapting to the learning style the system should first evaluate what is best for the learner.
In the opinion of one learner learning should be fun like a game. A training provider underlines this opinion by stating that a "learning system should entertain the learner". Another one thinks that learning should be made less tedious. Therefore, motivation is also considered to be an important adaptation feature by some learners and training providers.
Some interviewees mentioned even new or slightly different adaptation features not provided in the list of features, namely, the amount of available time and the learner's handicaps or disabilities. This could be a daltonian, deaf learner, or a wearer of glasses who is supported by adapting the font size automatically or by the user. A further adaptation feature mentioned by one interviewee is the duration of a learning phase. Two training providers suggested taking the age of the learner into account as an adaptation feature. Table 8 lists the rating of the interviewees regarding adaptation technologies. Looking into the interview summaries some interviewees suggested performing a grading test to determine the learner's state of knowledge. The test results should be very specific in order to ensure a good classification of the learner's competences. These kinds of tests are equivalent to adaptive testing as they aim at recognizing knowledge gaps of a learner. But they also try to determine already known concepts. By doing so, both learners and tutors could receive feedback with regards to the learner's state of knowledge. This could not only be done at the beginning but also during the learning process when answering questions in a test. The intelligent analysis of student solutions in the form of specific error feedback is high-valued by some interviewees.

C. Adaptation technologies
One training provider thinks that "the anonymity of a learning system could be a chance to overcome a person's inhibitions, given that she is willing to work with an electronic device, because she does not have to reveal her knowledge deficit to a human being but only to a machine".
Furthermore, it seems to be particularly important in a corporate learning environment that knowledge gap analysis can be conducted and the system then adapts the learning content to the needs of an employee. One training provider gave an example: "A big company such as Coca Cola would be able to follow and update the professional learning path of its individual employees in tune with the direction of the company".
A pre-test could not only be used to gather information about a learner's state of knowledge but also about other characteristics like learning styles.
Some interviewees stated that adaptive navigation support is very important for them as it helps them to navigate through the offered courses in the LMS or a course itself. Direct guidance is emphasized as helpful especially for beginners. Moreover, link and map annotation are considered to be a useful specific technique of adaptive navigation support, although these adaptation dimensions were not highly rated in general.
A more important adaptation dimension is adaptive content selection, through which multi-faceted views could be created. One interviewee emphasized explanations variants, i.e. the delivery of content in a different difficulty level. Another preferred additional explanations and prerequisite explanations. One training provider mentioned that dimming irrelevant content could make the learner unsure. Thus hiding unnecessary learning content would be better. Nevertheless the learner should be able to access the hidden content by switching to a kind of overview of the course. In this case map annotation or adaptation would be a helpful feature. It is also considered to be helpful when the learner gets presented with an overview of what is required to complete a course or to achieve a goal satisfactorily.
Interestingly, the customization of the graphical interface (c.f. OQ_B6), an adaptation technology which already exists in some learning systems, was not highly rated by the interviewees (c.f. Table 8).
Many interviewees suggested that the adaptation process should always be transparent to the learner so that the learner always feels in control. Moreover, the learner should always have the choice to follow the recommendations of the adaptive system or not, or should even be able to deactivate certain adaptive functionalities.
In the opinion of some interviewees the system should never completely hide learning content. Therefore one interviewed learner completely rejected adaptive hiding as an adaptation technology. Besides a transparent adaptation process, it is desired by some interviewees that the learner has the authority over his or her profile to set it up themselves.

V. CONCLUSIONS
We presented a requirements analysis that was carried out within the EU co-funded GRAPPLE project which aims at delivering an adaptive technology-enhanced learning environment incorporated into Learning Management Systems. This appears to be the first study with this subject conducted in business settings.
A survey was conducted to determine the adaptation aspects seen as most relevant to corporate users. A requirements analysis methodology has been developed for semi-structured interviews with stakeholders that are corporate learners and training providers. In the interviews, scenarios involving the use of an adaptive training system were used as supporting tools to contextualize the conversation and to ensure interviewees' understanding of adaptation. Additionally, an adaptive business English course was created to illustrate adaptation principles to interviewees. Applying this methodology, 26 in-depth interviews were conducted in six European countries to gather personalization and adaptation needs.
In summary, there is a common agreement among the interviewees that an adaptive learning system has decided advantages over a non-adaptive learning system by offering individual treatment of the learners. Therefore an ALS appears especially suited for the multi-faceted nature of big companies. The adaptation to both learner knowledge (desirably determined by grading tests and adaptive testing) and learning goal, particularly determined by a job role and career path, is most relevant. Doing so, an ALS can facilitate target-oriented learning and lessen the time wasted on non-relevant learning content. Many interviewees have the opinion that the authoring process of an adaptive course is very complex and needs deep insight. Thus, authors should be supported in their work, for example, through wizards and templates.
In sum, the results from the requirements elicitation process provides useful information as to what prospective users expect and require from an ALS and thus can inspire and feed into the design and development process of such a system.