Maturity in Large Scale Corporate E-learning

—We present a framework of maturity indicators that can be used to assess the maturity of a company in terms of e-learning and to promote new approaches to corporate e-learning. This framework is the result of a study that included semi-structured interviews with prime companies of several sectors such as banking, insurance, retail, energy, telecommunications, pharmaceuticals, food and beverage, and transports. In common, all of these companies have annual training budgets about or higher than 6 million dollars and several years of experience using e-learning to promote corporate development and training. The result of this study is not a maturity path or a set of maturity levels but a list of maturity indicators, classified in Each dimension includes several indicators of maturity. Each company can be assessed for every indicator to determine its strength and maturity in each dimension. This framework helps companies assess their own maturity in several areas, self diagnose their e-learning practices, and design strategies to improve their practices. I. INTRODUCTION Each company develops its own strategy, procedures, and practices, and protects them from the outside in order to use them as a competitive advantage. When the time comes to access how adequate their approaches are, they need practical tools that can help them improve. This is also true in corporate e-learning: the companies build their e-learning initiatives but they need to evaluate them from time to time. What this paper provides is a tool to evaluate e-learning maturity in corporate environments. Each company is a living organism, with its own vision, strategy, resources, problems, and challenges. Just like animals and plants, companies also have a lifecycle[1, 2]: they are born, grow, age, and die, and the way each company deals with its own internal and external contexts, and with change, sets its success... or death. Companies, like humans, develop along four dimensions: physical, cognitive, social, and economical, and they also become mature. Maturity is about reaching a development level that maximizes skills and talents, optimizes the response to needs, reflects past experiences and recovers from the things that went wrong. Yet, maturity is not an ending stage. It is a process and a succession of stages, and each stage has specific conflicts, challenges, milestones, and expected outcomes. To understand maturity, and even maturity stages, we must look at indicators of maturity. This paper presents an approach to assess e-learning corporate maturity, in order to help companies …

INTRODUCTION Each company develops its own strategy, procedures, and practices, and protects them from the outside in order to use them as a competitive advantage. When the time comes to access how adequate their approaches are, they need practical tools that can help them improve. This is also true in corporate e-learning: the companies build their e-learning initiatives but they need to evaluate them from time to time. What this paper provides is a tool to evaluate e-learning maturity in corporate environments.
Each company is a living organism, with its own vision, strategy, resources, problems, and challenges. Just like animals and plants, companies also have a lifecycle [1,2]: they are born, grow, age, and die, and the way each company deals with its own internal and external contexts, and with change, sets its success... or death. Companies, like humans, develop along four dimensions: physical, cognitive, social, and economical, and they also become mature. Maturity is about reaching a development level that maximizes skills and talents, optimizes the response to needs, reflects past experiences and recovers from the things that went wrong. Yet, maturity is not an ending stage. It is a process and a succession of stages, and each stage has specific conflicts, challenges, milestones, and expected outcomes. To understand maturity, and even maturity stages, we must look at indicators of maturity.
This paper presents an approach to assess e-learning corporate maturity, in order to help companies evaluate where they are and what they can do to improve it.
To help companies assess their own maturity in terms of e-learning, we have conducted a qualitative study with prime companies that invest heavily in e-learning, so that we could extract from their practices elements that could help other companies progress.
In the next section, we discuss maturity models in general, in human resources management, and in elearning. Next, we present the aims, sample, and methodology of the study. We end by presenting the seven dimensions of maturity indicators we have found.

II. HUMAN RESOURCES MANAGEMENT CHALLENGES
& MATURITY IN E-LEARNING Measuring maturity is crucial to predict things such as crop stages, disease progressions, or psychosocial and human development[for example, 3]. In a corporate environment, maturity indicators are used to assess the performance of a company in a scale of maturity and to define the road towards development and improvement.
There are dozens of life-cycle models, each one with a different number of stages or phases and different criteria of classification. Each model is an attempt to understand and frame the complex phenomenon of corporate growth, development, and maturity. In an attempt to adjust more closely to the reality, some models are focused on small businesses [for example, 4,5] or in technology-based companies [6, for example, 7]. The usefulness of these models relies in the description of the main challenges that need to be faced in each phase.
Most studies on corporate life-cycles and maturity address problems that include areas such as product development and production, sales and marketing, strategic positioning, financial management, and human resource (HR) management.
Like information management [8,9] or customer experience management [10][11][12], human resources and training management also deal with their own specific maturation issues. In terms of HR management, each life cycle and maturation stage has specific problems in terms of recruitment, compensation, training, etc. The problems of development and training tend to be more complex in high-growth companies [13], big companies, companies with international businesses and expatriates, technologybased companies, companies that work in unstable sectors and, ironically, in sectors where the workers unions are strong, as we will discuss later.
The People Capability Maturity Model (PCMM) [14] is an example of a maturity framework of workforce practices. It is also an organizational change model designed to help improve workforce practices. It consists of five maturity or evolutionary stages through which the workforce practices and processes of an organization evolve (p.15). Each level of maturity represents a "new level of organizational capability created by the transformation of one or more domains of an organization's processes" (p. 17). There is also a set of workforce management practices at each level. For example, at maturity level 5, which is the highest level, individuals are encouraged to make continuous improvements to their personal work processes by analyzing their work and making the process enhancements needed.
Flynn [15] argues that, as strategy cannot be seen, we can only look at the practices and, based on them, let strategy make itself visible as a pattern of human resource practices. So, he proposes a maturity matrix for HR management that links human resource strategy and human resource practices and is applicable at all the stages of organizational development. His matrix combines four levels of organizational and HR maturity with HR frameworks such as organizational effectiveness, employee development, performance management systems, among others.
Bersin & Associates have also been designing a framework to understand maturity in HR management. O'Leonard & Harris [16] started by linking several independent processes, such as talent acquisition and rewards, performance management, career and succession management, leadership development, workforce planning and alignment with business strategy, among others, in a talent management framework. Based on this framework, they proposed a series of maturity stages of talent management and human resources management, each one with specific characteristics of leadership, people resources, technology, individual processes, and integration efforts [17][18][19].
Several authors have dedicated their attention to the construction of maturity models specific to e-learning [20-22, for example, 23,24]. Marshall [25][26][27][28][29][30] proposed an elearning maturity model (eMM), which is based on the Capability Maturity Model (CMM) [31] and on SPICE [32]. The core concept of the E-learning Maturity Model is capability, defined as the ability of an institution to sustain e-learning delivery and the support of learning and teaching as demands grows and staff changes [30]. The model is organized in five process categories (learning, development, support, evaluation, and organisation) and five process capabilities (delivery, planning, definition, management, and optimization). Each process is further broken intro practices and these are rated for performance in a not adequate to fully adequate scale.

III. AIMS, SAMPLE, AND METHODOLOGY
Our study aimed at identifying key indicators of maturity in e-learning and provide a battery of indicators of maturity that companies can use to self diagnose their elearning practices and maturities and design strategies to improve their practices. In order to do so, we have studied several companies and looked for good practices, off-thestream and disruptive practices, and experiences within elearning.
Most of the previous research on corporate maturity started by defining a number of stages or phases based on unique characteristics, and then "inserted" each company into those stages, putting a label on it. Our aim was not to propose a new model, nor to label each company by forcing it into a given stage, but rather to identify indicators of maturity and good practices.
The companies we have chosen are prime companies [33,34]. Prime companies are focused on longterm results, invest an above-average percentage of revenues in research and development, and keep being innovative. These companies balance flexibility and control, innovation and paper work. A prime company performs with excellence and assertively, with a clear vision and a sense of leadership in its own markets.
The data was collected in 14 prime companies, all of them market leaders with strong international operations, about 10.000 employees, and training budgets around or above 6 million dollars.
To reduce sector-related bias, we designed a multisector study that included banking, insurance, retail, energy, telecommunications, pharmaceuticals, food & beverage, and transports.
These companies share some challenges. Except for the energy and telecommunications companies, all the others live in a high competitive market where they are the leaders or challenge the leader.
We have conducted semi-structured interviews with training directors and/or e-learning managers. We then used NVIVO to analyze the data. We did not force the classification of the testimonials into a fixed structure of categories or themes, as we intended not to be fully constrained by the current literature, although it was a major input in the design of our interview protocol.
Our classification scheme also evolved as the interviews progressed. Our first interviews were quite exploratory and formed a temporary structure of hot topics that could become indicators of maturity and dimensions of maturity in corporate e-learning management. Often, we had to go back to the companies to collect more data on some aspects that the literature did not let us preview. We looked at this interactive process as a healthy way of maturing our study and of freeing ourselves from models that were not able to represent our data.

IV. INDICATORS OF MATURITY IN CORPORATE E-
LEARNING MANAGEMENT Our study allowed us to identify several critical indicators of maturity ( Figure 1). Due to the affinities among them, we were able to classify them in seven dimensions of maturity: strategy, structure, experience, learning design, learning products, learning process and people. Each dimension includes several indicators of maturity that we will describe next (see also Table 1).
These indicators are related. For instance, in the strategy dimension, we assess the attitudes towards elearning from the top management and the effect of that on the overall e-learning strategy. In the people dimension we access the ability of the training team to develop elearning projects and to conquer investment to them, even in the presence of internal blocking forces, such as an adverse attitude towards e-learning from top management.

A. Strategy
The strategy dimension includes the motives for using e-learning and the kind of objectives it follows, the link between e-learning and the business strategy, the ability to balance corporate idiosyncrasies and trends in the learning industry, as well as the importance, weight, and role of elearning in the overall training and development strategy.
It also includes indicators related to overseas operations of globalized companies, such as the degree of centralization/decentralization and convergence/divergence of international e-learning strategies and the ability to develop think-globally-actlocally e-learning strategies, procedures, and practices. This dimension also reflects the general use of e-learning in the company and the general attitudes towards elearning from top managers.

B. Structure
The dimension of structure includes three types of indicators: the organizational structure, the financial structure, and the compliance with legal requirements, norms, and standards.
The first type of indicators are focused on the internal organization and where training and e-learning are placed in the corporate structure and its evolution. For instance, some companies have a structure dedicated to top managers training and development and another structure dedicated to the rest of the workers. Other companies have corporate universities or subsidiaries dedicated to training and development. Some companies have tested several structures, while others work in a rigid structure and have never tested different structures and organizational models. There are also indicators related to the physical resources that support learning and their adequacy to the e-learning strategy.
The second type of structure related indicators are the financial ones, which include the financial investments in e-learning and the weight of e-learning in the training initiatives and budgets, the dependency on external funding, and the type of cuts that are made in e-learning initiatives during recession periods.
The last set of indicators is related to the certification of the training process and the compliance with legal requirements, norms, and standards, the internal procedures and how much they are formalized and monitored.

C. Experience
The experience dimension includes indicators to assess how sure and confident the company is regarding its elearning initiatives and the company's perception of its elearning practices as a group of isolated experiences or as a cruising-speed performance.
It also includes the perception of quality of the previous experiences in e-learning and the ability to cope with bad past experiences, how the company recovered from them, how those experiences are currently affecting initiatives and blocking innovation in e-learning, and the ability to develop experiments and test different approaches to elearning.

D. Learning Design
The learning design dimension includes indicators related to the tools used to create the courses and the platforms used to deliver them, and the adequacy of these tools and platforms to the overall e-learning strategy. It also assesses the degree of integration of those tools with the company's platforms of human resources management. In this dimension, we also evaluate the degree of use and the perceived easiness of use of those tools and platforms.
Other indicators included in this dimension are related to the degree, motives, and methodologies used to outsource nuclear activities in the learning design process, as well as to the methodology used to create the courses (for example, using agile development processes). This dimension is also focused on the degree of centralization/decentralization of the design process in the training management team/department (vs. making each worker a potential course builder or producer), the degree of innovation in terms of pedagogy (for example, the use of game based learning, flipped learning, etc.), and the general attitude towards current learning trends.

E. Learning Products
This dimension includes indicators of maturity that evaluate what kind of learning products are being created and offered: internally developed courses, courses created on demand by external companies, local versions of international courses, off-the-shelf courses, corporate MOOCs, blended-learning courses. It also includes the complexity, diversity, and depth of the company's learning portfolio. For example, the number of courses from a specific training area, the degrees available, the emphasis of the courses on soft skills rather than just on technical issues, as well as the technical support to the courses (for example, flash, video, text, etc.).

F. Learning Process
The learning process dimension includes four sets of indicators: The first is related to the needs assessment process and the ability to develop individualized elearning plans. The second is related to key performance indicators that have been created and are used to assess the e-learning initiatives and the tools and methods used in that evaluation.
The third set is related to the social dynamic of the courses and the existence of support systems, which includes the key performance indicators that are created for the online community, the existence of that community and supporting tutors, the role of the facilitators, the use of peer-assessment, and the use given to social platforms.
The fourth set of indicators related to the learning process is focused on the ways in which e-learning is being used to promote informal learning and online communities of practice within the company and to promote online coaching.

G. People
The people dimension includes three sets of indicators: The first is related to the target of e-learning initiatives(to whom are the courses created for) and the degree of openness of the courses inside the company: Can any worker have access to every course available? Is access blocked to some functions? Does it require superior authorization? Due to the weight of training regulations, this set also includes indicators that assess if learning is on-demand or imposed, its degree of voluntariness, and the restrictions of use imposed by labor unions, for instance, in terms of hours of access.
The second set of indicators is related to the maturity of the trainees or workers, their reactions and attitudes towards e-learning and training in general, as well as their basic and digital literacies.
The last set of indicators is related to the maturity, experience, and background of the training management team: how they became e-learning managers, what they do to keep updated, and how much research and development they do in house. It also includes their strength to deploy e-learning projects, even in contexts where the top management has an adverse attitude towards e-learning.
V. CONCLUSIONS In order to evaluate the maturity of corporate elearning, we have conducted a qualitative study with prime companies that have been investing heavily in elearning for several years. From that study, we were able to extract seven dimensions of maturity: strategy, structure, experience, learning design, learning products, learning process, and people.
Each dimension has several indicators of maturity against which each company can have its e-learning solution evaluated. Although the indicators are related to each other, some companies can be more mature in one dimension than in another. Moreover, they can appear as very mature overall but have specific areas that need to be improved.
These indicators of corporate maturity can thus help companies diagnose their e-learning's maturity and design strategies, so that they can improve their performance in this respect.

Strategy
Motives for using e-learning in the company and kind of objectives it follows Link between e-learning and business strategy Ability to balance corporate idiosyncrasies and trends in the learning industry Importance, weight, and role of e-learning in the overall training and development strategy Degree of centralization/decentralization and convergence/divergence of the international e-learning strategies Ability to develop think-globally-act-locally e-learning strategies, procedures, and practices Effective use of e-learning Top management attitudes towards e-learning

Structure
Training organizational structure and its evolution Physical resources to support e-learning and their adequacy to the e-learning strategy Financial investment in e-learning &weight of e-learning in training initiatives and budgets Degree of dependency on external funding Type of cuts that are made during recession periods Certification of the training process & compliance with legal requirements, norms, and standards Internal procedures in e-learning and extent to which they are formalized and monitored Experience

Confidence in current e-learning initiatives
Perception of the initiatives in e-learning as isolated experiences or as a cruising-speed performance Perception of quality of previous experiences in e-learning Ability to cope with and recover from bad experiences

Learning Design
Adequacy of tools and platforms used Degree of integration of e-learning tools with HR management tools Degree of use and perceived easiness of use of the tools and platforms Degree, motives and methodologies used to outsource nuclear activities of learning design Methodology to create and update the courses Degree of centralization/decentralization of the design process Innovation in terms of pedagogy General attitudes towards learning trends

Learning Products
Types of learning products that are created or used