A Learner-Centered Design, Implementation, and Evaluation Approach of Learning Environments to Foster Acceptance

—Errors in requirements specifications have been identified as a major contributor to costly software project failures. Hence it may be highly beneficial if designers and/or developers of learning environments could verify requirements by predicting learners’ acceptance and usage based on evaluations during the earliest phases of the learning environment development. Previous findings of such an approach in the field of CRM software implementation showed that pre-prototype user acceptance tests did have almost equal informational value than their prototype counterparts that ensure hands-on experience with the system. More concrete, we focus on interventions of specific design features with their inherent design characteristics during the pre-implementation phase of learning environments. They may help minimizing learners’ initial resistance by providing a realistic preview of the system to enable potential learners to develop accurate perceptions and find out how the system may help them to improve their learning process. Consequently, this conceptual paper proposes a framework that examines the influence of specific design features with their inherent design characteristics of learning environments on user acceptance, particularly on the antecedents of perceived usefulness and perceived ease of use. Doing so, we will be able to extrapolate the influence of specific design features with their inherent design characteristics on specific antecedents of perceived usefulness and perceived ease of use in order to enhance designers’ and/or developers’ ability to identify and improve critical design features during the earliest phases of learning environment development.

D. Mueller, V. Zimmermann IMC AG, Saarbruecken, Germany Abstract-Errors in requirements specifications have been identified as a major contributor to costly software project failures.Hence it may be highly beneficial if designers and/or developers of learning environments could verify requirements by predicting learners' acceptance and usage based on evaluations during the earliest phases of the learning environment development.Previous findings of such an approach in the field of CRM software implementation showed that pre-prototype user acceptance tests did have almost equal informational value than their prototype counterparts that ensure hands-on experience with the system.More concrete, we focus on interventions of specific design features with their inherent design characteristics during the pre-implementation phase of learning environments.They may help minimizing learners' initial resistance by providing a realistic preview of the system to enable potential learners to develop accurate perceptions and find out how the system may help them to improve their learning process.Consequently, this conceptual paper proposes a framework that examines the influence of specific design features with their inherent design characteristics of learning environments on user acceptance, particularly on the antecedents of perceived usefulness and perceived ease of use.Doing so, we will be able to extrapolate the influence of specific design features with their inherent design characteristics on specific antecedents of perceived usefulness and perceived ease of use in order to enhance designers' and/or developers' ability to identify and improve critical design features during the earliest phases of learning environment development.

I.
INTRODUCTION Virtual learning environments (VLE) are systems that use modern information and communication technologies to support education and training efforts [55].However, the benefits of such systems can only be ensured if learners accept and use the system in an appropriate manner [54].Therefore it is important to investigate the drivers or determinants of VLE success to assist designers and/or developers in building VLEs that are useful and accepted by the end-user, being the learners [59].In this paper, design features as drivers or determinants of VLE success are of particular interest.Design features are defined as specific functionalities of VLEs (here: personal portals) such as navigation bars, chat-, and recommender functionalities as well as mini-applications within the personal portal itself (here: Widgets).The design features in turn possess specific characteristics, which can positively influence user acceptance and system success [4,19,74,75].To better extrapolate those critical design features, [2] suggest methods such as prototyping, user testing, and use of early system capabilities.These principles are consistent with approaches to Human Centered Design (HCD) such as recommended in ISO 13407 [35].It describes a process cycle that starts with the analysis of the context of use (e.g.user, task, and environment), subsequently derives requirements which are incorporated into comprehensive prototypes that finally are evaluated in terms of their acceptance and potential usage.Yet, as the manipulation of design features in a field setting may be difficult and expensive [70], [50] and [63] suggest simulation and agent-based modeling approaches that are considered as low-cost alternatives in order to investigate the impact of specific design features on VLE adoption and use.These approaches can be used to manipulate specific design features and isolate the effects of these features on various determinants of VLE adoption [70].
The conceptual paper is structured as follows: section II gives a concise overview of personal portals as futureoriented VLEs.Section III A undertakes a comprehensive overview of the literature on technology acceptance and its consideration of design characteristics.This literature review provides the underlying rationale for section III B that addresses the detected gap of research by providing a way towards a (n integrated) learner-centered design and evaluation approach to foster acceptance.Based on section III B, section IV presents a learner-centered design and evaluation approach by focusing on design features, their inherent characteristics as pre-implementation interventions (section IV A), and their influence on the antecedents of PU, and PEOU (section IV B).Section V is concerned with delineating a roadmap towards a comprehensive research model to bring more actionable guidance into the design process of VLEs.Section VI finally draws some overall conclusions.

II. PERSONAL PORTALS AS FUTURE-ORIENTED
ENVIRONMENTS FOR LEARNING As one of three types of VLEs detected during the consolidation procedure of 21 best practice case studies within the EU-funded eContentplus Best Practice Network ICOPER [33], Personal Learning Environments (PLE) were considered as a promising and future-oriented learning approach on which we therefore will focus in the subsequent discussion.Regarding their technical implementation, PLEs are mostly realized in the form of per-A LEARNER-CENTERED DESIGN, IMPLEMENTATION, AND EVALUATION APPROACH OF LEARNING ENVIRONMENTS TO FOSTER ACCEPTANCE sonal portals such as [51].In order to generate comprehensive prototypes (e.g.mock-ups and/or executable applications/systems) for subsequent acceptance evaluations as described in HCD processes such as ISO 13407 (see section I), we therefore extrapolate those requirements in the subsequent explanations that are driven by the context of use.For PLEs, the context of use may be characterized as follows: the concept of PLEs constitutes, that its learning environment as well as its inherent tools and content are owned and controlled by the learner [5].Hence, the learning environment allows the learner not only to learn with the peer, but to control her learning resources, manage her learning activities she participates in, and to integrate her learning from different institutions/sources [6].More concrete: the PLE concept may be "a response to pedagogic approaches which require that learner's e-learning systems need to be under the control of the learners themselves" [48].Therefore the learner within the PLE has to be enabled to modify its configuration depending on her own preferences [52].Yet, according to [47], personal portals are not specially designed for educational purposes, but they possess flexible features to configure the existing components in a learning environment.
Thus we raise the question which specific features of personal portals are crucial for learners' adoption and contribute to the success of the learning environment?To answer this question, we firstly undertake a comprehensive overview of the literature on technology acceptance and its consideration of design characteristics.This literature review in turn provides the underlying rationale for section III B.

A. Design Science and Technology Acceptance Research
Based on theories in social psychology, such as the theory of reasoned action (TRA: see [28]) and the theory of planned behavior (TPB: see [27]), the TAM has been validated as a powerful and parsimonious research framework that predicts individual adoption and use of new ITs [18].
TAM posits that individuals' behavioral intention to use an IT, and in this context, a VLE, is determined by two beliefs: perceived usefulness (PU), defined as "the extent to which a person believes that using an IT will enhance his or her job performance", and perceived ease of use (PEOU), defined as "the degree to which a person believes that using an IT will be free of effort" [70].
studies [4,19,42,46,67,68], some of them focusing on the acceptance of VLEs [37,44,60], have looked explicitly at the role of specific design characteristics as antecedents of PU, and/or PEOU as suggested by [18] so far.Yet, as these studies either used a dummy variable to represent different IS [19,67], or adopted a single overall construct, such as perceived system quality [37,46,60], system capability [44], or output quality [68] to substitute for specific design characteristics, TAM in itself rarely propose concrete guidelines about design options (information-, and system related: see [74,75]) that can impact the acceptance of information technologies [4,59,72].Some noteworthy exceptions in the broader field of eLearning are: To begin with the influence of more informationrelated design characteristics on behavioral beliefs such as (the antecedents of) PU as well as PEOU, [61] found information quality (here: e-learning materials presentation types such as text, audio, and video and specific configuration of them), and more concrete the most media-rich presentation interface (text-audio-video) to have the highest levels of PU.Moreover, [80] found two dimensions of content quality, namely content richness, and update regularity, to be significant predictors of PU within mandatory settings of VLE use, whereas content richness was the stronger predictor of PU when the VLE use was voluntary.
Referring to more system-related design characteristics and their influence on behavioral beliefs such as (the antecedents of) PU as well as PEOU, [76] examined how dimensions of usability (information relevance, clarity of terminology, and screen design) influenced PU and PEOU of a digital library application.Their results showed mixed effects with information relevance showing a stronger effect on PU than PEOU, and terminology as well as screen design having only significant effects on PEOU.The difference in the effects of these variables provides insight into the nature of the design characteristics as relevance is considered to be more informationrelated, whereas screen design is considered to be more system-related [4,74,75].Beside that, [24] found terminology to be a significant predictor of PEOU whereas [53] found screen design and relevance to be a significant predictor of PEOU.Yet, in addition to the suggestions above, we consider terminology and relevance as a more information-related design characteristic with a potential impact on (the antecedents of) PU [4].A reverse effect was detected by [49] who found interactivity and control (e.g.control of the rhythm of learning/learning sequence, etc.) not to be a significant predictor of PU.Yet, these findings support our understanding of interactivity and control to be an antecedent of PEOU as it reflects more system-related design characteristics [4].In the context of search engines as learning assisted tools, [62] found two dimensions of system quality, namely search quality, and search efficiency to be significant predictors of two antecedents of PEOU, namely Perceived Enjoyment (PE), and Computer self-efficacy (CSE).In contradiction to that, [41] found both, PU, and PEOU to be affected by the same system-related design characteristics, namely functionality, interactivity, and response time.Hence, we consider these findings not to provide iJAC -Volume 2, Issue 3, August 2009 A LEARNER-CENTERED DESIGN, IMPLEMENTATION, AND EVALUATION APPROACH OF LEARNING ENVIRONMENTS TO FOSTER ACCEPTANCE actionable guidance to practitioners as designers and/or developers may not be empowered with a comprehensive instrument that recommends specific design-related tasks (and their prioritization) based on a precisely defined and validated causal chain of object-based and behavioral beliefs.
To sum up, these more or less simplistic operationalizations at a holistic level, with focus on a limited number of design features, and with almost no distinction between specific kinds of design characteristics (information-and/or system-related) do not comprehensively extrapolate the effects of specific design features with their inherent characteristics on the antecedents of PU and PEOU.More concrete, a deeper understanding of specific design characteristics contributing to the antecedents of both PEOU, and PU is needed in order to improve our understanding which design features and their inherent characteristics have an impact on which determinants of PU and PEOU in order to provide welldefined and actionable levers to designers and/or developers so that they may foster the acceptance of VLEs [76].
These findings go in line with [79] who postulate that one of the most common criticisms of TAM is its lack of actionable guidance to practitioners ("imagine talking to a manager and saying that to be adopted technology must be useful and easy to use […].The more important questions are what makes technology useful and easy to use" [79]).

B. Towards a learner-centered design and evaluation
approach to foster acceptance In order to address this gap of research, to bring more actionable guidance to practitioners, and consequently foster learners' acceptance, we suggest the following approach: Firstly, we build on the TAM3 model [70] as it supports our approach in the following way: In contradiction to TAM which is considered as a powerful and parsimonious research framework (parsimony dictates whether some factors should be deleted because they add little value to our understanding of a phenomenon [11]), the key strength of TAM3 is seen in its comprehensiveness and potential for actionable guidance (comprehensiveness ensures whether all relevant factors are included in a theory [11]) [70].This is due to fact as TAM3 delineates a comprehensive research model that integrates general determinants of both, PU [68], and PEOU [65].Moreover, the determinants of PU and PEOU don't show any crossover effects, the determinants of PU will not influence PEOU and vice versa [70].Consequently, our research on the general determinants of PU and PEOU presents pointers to constructs that could be levers for greater adoption of VLEs [70].In this sense, TAM3 improves our understanding of learner reactions to new VLEs.Based on [70], we further suggest that design features with their inherent characteristics may have a potential influence on the following antecedents of PU, and PEOU, all of them comprehensively represented in TAM3 [70].For PU these are: Job Relevance (JR: "the degree to which an individual believes that the target system is applicable to his or her job" [68]), Output Quality (OQ: "the degree to which an individual believes that the system performs his or her job tasks well" [68]), and Result Demonstrability (RD: "the degree to which an individual believes that the results of using a system are tangible, observable, and communicable" [22]).For PEOU this is: Perceived Enjoyment (PE: "the extent to which the activity of using a specific system is perceived to be enjoyable in its own right, aside from any performance consequences resulting from system use" [65]).Objective usability (OU: "comparison of systems based on the actual level (rather than perceptions) of effort required to completing specific tasks" [65]) was excluded from the original scale [70], as it is not evaluated as a perceptual measure, but "as a ratio of time spent by the subject to the time spent by an expert on the same set of tasks" [70].However, usability will be integrated as a system-related characteristic (see section IV B) to measure its influence on the antecedents of PEOU [4,75].Furthermore, beyond the findings of [70] we suggest to include computer self-efficacy (CSE: "the degree to which an individual believes that he or she has the ability to perform a specific task/job using the computer" [13,14]), as well as computer anxiety (CA: "the degree of an individual's apprehension, or even fear, when she/he is faced with the possibility of using computers" [65]).
Secondly, based on the aforementioned findings we build on the literature on user satisfaction [3,36,38,74,75,77], as it, in contrast to the more behavioral beliefs of the technology acceptance literature (PU, and PEOU), preferably deals with more object-based beliefs (information-, and system-related characteristics).Consequently, such approaches are considered as more appropriate to provide designers with guidelines to build improved VLEs [26].However, object-based approaches that measure specific system characteristics of specific design features are mostly weak direct predictors of system usage [4].Consequently, we additionally ground our work on [4] who integrated the technology acceptance and the user satisfaction literature on a theoretical and conceptual level.This is due to the fact that [4] verified information and system characteristics as direct determinants of PU and PEOU, respectively, which in turn showed a significant influence on BI.In this sense [4] developed a model that explicitly distinguishes the object-based beliefs found in the user satisfaction literature from behavioral beliefs in the technology acceptance literature.Consequently, and most important, their model suggests that the technology acceptance and the user satisfaction literature are not competing approaches to understand VLE usage, but they rather "represent complementary steps in a causal chain from key characteristics [of specific design features] to [the antecedents of the more behavioral] beliefs" [4], means PU and PEOU that ultimately determine BI.Hence, the approach suggested by [4] provides more actionable guidance to practitioners [79] as one of its key benefits is that designers and/or developers "now have a way to assess system and information characteristics and then reliably investigate their impacts on ultimate usage through the proposed causal chain.This can help with management activities such as task prioritization and resource allocation" [4].Furthermore, the proposed research model also provides a mechanism for understanding and assessing the relative influence of detailed system and information characteristics, and in this way provides important guidance to system designers [4,79].
Thirdly, beside insights into how and why learners make a decision about the adoption and use of VLEs, we build upon interventions of specific design features (with A LEARNER-CENTERED DESIGN, IMPLEMENTATION, AND EVALUATION APPROACH OF LEARNING ENVIRONMENTS TO FOSTER ACCEPTANCE their inherent characteristics) and their influence on the general determinants of PU and PEOU as they "represent a focus on the major managerial levers that can foster adoption" [66,69].Therefore, interventions help managers to make effective decisions about applying specific design features to influence both, object-based, and behavioral beliefs of VLEs, and, consequently, the acceptance and effective utilization as well as the success of new learning environments [1,57,75].Our approach is in line with [39] who postulates that managers need to develop and implement effective interventions in order to maximize employees' IT adoption and use.Additionally, [70] underline "that unless organizations can develop effective interventions to enhance IT adoption and use, there is no practical utility of our rich understanding of IT adoption."Thus, we want to address this gap of research by identifying and linking feature-based interventions with specific object-based and behavioral determinants of VLE acceptance and usage.

A. Design features as pre-implementation interventions
According to [58,73] interventions can be separated into two categories, namely pre-implementation, and post-implementation interventions.Whereas preimplementation interventions focus on phases leading to the actual roll-out of a VLE, post-implementation interventions contain phases following the actual deployment of the VLE [58].As the subsequent explanations focus on pre-implementation interventions, post-implementation interventions will not be further discussed.
Referring to pre-implementation interventions, "they represent a set of organizational activities that take place during system development and deployment periods and can potentially lead to greater acceptance of a system" [70].Beside design features and their inherent characteristics, further examples of pre-implementation interventions are: user participation, management support, and incentive alignment [70].These interventions help designer and/or developers to minimize initial resistance to a new VLE, and consequently provide a realistic preview of it so that potential learners can develop accurate perceptions and how the system may help them perform their learning processes [70].
Thus proactive implementation of interventions is necessary to minimize such resistance.Beside approaches suggested by [2] (see chapter I) that concentrate on prototyping, we suggest the application of pre-implementation interventions already in the stage of pre-prototyping (e.g.paper prototyping, video mockups) as past work showed that BI and PU measured before hands-on experience were highly correlated with, and not significantly different from the same measures taken up to three months after implementation/hands-on experience [21].The interventions in turn are based on the general determinants of PU, and PEOU, comprehensively presented in the TAM3 model, as they present "constructs that could be levers" for actionable guidance [70].As the special focus of the subsequent explanations lies on design characteristics as pre-implementation interventions, other kinds of pre-implementation interventions will not be further discussed.

B. The characteristics of specific design features and their influence on the antecdents of PU and PEOU
According to the Institute of Electrical and Electronics Engineers the term feature is defined as "a distinguishing characteristic of a software item (e.g., performance, portability, or functionality)" [34].Yet, in this paper we define design features as specific functionalities of a personal portal (see section I).More concrete, these functionalities have two basing points: the framework of the personal portal (e.g.navigation bar, chat, recommender functionality) as well as the applications within the personal portal itself (here: Widgets).The design features in turn possess specific characteristics, which can positively influence user acceptance and system success [4,19,74,75].These characteristics, that are either referred to as system characteristics within the technology acceptance literature [70] or as quality characteristics (here: focus on information-and system-related quality characteristics) within the user satisfaction literature [74], are "those salient features of a system that can help individuals develop favorable (or unfavorable) perceptions regarding the usefulness or ease of use of a system" [70], and in this sense constitute crucial levers to positively influence user acceptance and system success [74].
Design characteristics can be broadly categorized into information-, system-, and service-related characteristics [4,74,75], whereas we consider service-related characteristics as not relevant for this paper.
Given these initial findings, we drill down to the influence of specific design features and their inherent characteristics on specific antecedents of PU and PEOU in order to enhance our ability to identify and improve specific design features and/or a specific configuration of them to enhance certain determinants of PU and PEOU.
Such an approach would be in line with [32] who call for "identifying the IT artifacts [here: design features] and their [inherent] characteristics that influence and enhance" object-based as well as behavioral beliefs.In this sense, our approach doesn't solely focus on manipulating specific characteristics (e.g.accuracy, flexibility) on the antecedents of PU and PEOU [10], all of them comprehensively represented in TAM3 [70], but represents the influence of specific design features (e.g.recommender functionality) with their inherent characteristics (e.g.information-related: personalization, completeness; system-related: response time) in order to be able to detect to which extent this specific feature leverages the object-based (information-and system-related) as well as the behavioral beliefs (antecedents of PU and PEOU) of the VLE as a whole.For example, the introduction of a new recommender functionality probably will not only improve the system-, but also the information-related characteristics of the VLE (e.g. more personalized and complete information; faster response time of queries), In doing so, our approach will be at the nexus of HCI and IS and provide actionable guidance to practitioners as we will -based on a comprehensive causal chain -be able to answer the question which specific design feature will drive which object-based (here: design characteristics) and behavioral (here: antecedents of PU and PEOU) beliefs, which in turn drive BI, and in this sense foster acceptance.

V. ROADMAP TOWARDS A COMPREHENSIVE RESEARCH MODEL TO BRING MORE ACTIONABLE GUIDANCE INTO THE DESIGN PROCESS OF LEARNING
ENVIRONMENTS In order to provide more actionable guidance to practitioners, means designers and/or developers, we suggest the following research model that will be compiled during the next months: Firstly, we build on the categorization of information and system related design characteristics suggested by [4,74,75].Yet, to better fit the contextual requirements (here: VLEs with focus on personal portals), the specific constructs and their containing items/statements will be extended and/or modified based on the relevant literature on VLE success metrics such as described in [81].
Secondly, based on the findings and suggestions of [4] and [70] we model the information-related design characteristics (= object-based beliefs) on the antecedents of PU (behavioral beliefs) as well as the system-related design characteristics on the antecedents of PEOU, all of them comprehensively represented in TAM3 [70].The suggested correlations are summarized in Tab.I, where X indicates a particular intervention of a specific design feature with its inherent characteristics that potentially may influence particular antecedents of PU and/or PEOU.
Thirdly, all further effects within the TAM3 model are in accordance with [70].Yet, to better fit the contextual requirements (here: VLEs with focus on personal portals), the specific constructs and their containing items/statements will be modified based on the relevant TAM literature about VLEs.
Fourthly, the development of the survey instrument for both, design characteristics, and TAM constructs is patterned in accordance with [22], whose approach has successfully been adopted by [4].In doing so, we proactively address TAM-related limitations such as single measurement scales [79] and consider the principles of correspondence while designing our research models as it involves object-based and behavioral beliefs [4].Yet, the aforementioned relationships of object-based and behavioral beliefs can only be understood by examining appropriate mediating factors related to behavioral beliefs [4], such as (the antecedents of) PU, and (the antecedents of) PEOU, precisely specified in TAM3 [70].Consequently, as already mentioned above we will investigate the effects of specific design features with their inherent characteristics as antecedent of the PU-and PEOU-related constructs.
Finally, the biggest challenge in developing the measurement scale of design characteristics is considered to be the development of "a way to theorize about the characteristics of a system and the information it produces to arrive at ways to predict and understand them across contexts.In other words, researchers need to develop theories about information and system characteristics to overcome the legitimate complaint that existing sets of system and information characteristics are arbitrary" [4,12].To address this concern, we will elaborate and evaluate information-and system-related characteristics of personal portals as well as their influence on the antecedents of PU and PEOU in the following contexts of eLearning: higher education institutions and corporate organizations.
VI. CONCLUSION This conceptual paper investigated theoretical perspectives and empirical findings of the user satisfaction as well as the technology acceptance literature by applying [4] conceptualizations of object-based and behavioral beliefs to better explain the impact of specific design features with their inherent characteristics on the antecedents of PU and PEOU, which in turn drive the BI, means acceptance of VLEs.
With respect to the conceptualizations of object-based beliefs, we build on the categorization of information and system related design characteristics suggested by [4,74,75] that will be contextualized based on the relevant literature on VLE success metrics such as described in [81].
With respect to the conceptualizations of behavioral beliefs, we build on the TAM3 model [70] as it supports our approach in the following way: In contradiction to TAM which is considered as a powerful and parsimonious research framework, the key strength of TAM3 is seen in its comprehensiveness and potential for actionable guidance [70].In doing so, we developed a modified research model that comprehensively examines to which extend specific design characteristics of specific design features, means object-based beliefs, have an impact on the antecedents of PU, and PEOU, means behavioral beliefs, and finally towards BI to use a VLE.
Consequently, we believe that our model is able to provide more actionable guidance to practitioners as decision makers will be able to apply comprehensive designrelated interventions that can be simply derived from the causal chain that is mapped in our suggested model.

iJAC -Volume 2 ,
Issue 3, August 2009 A LEARNER-CENTERED DESIGN, IMPLEMENTATION, AND EVALUATION APPROACH OF LEARNING ENVIRONMENTS TO FOSTER ACCEPTANCE and these improvements in turn will positively drive corresponding antecedents of PU and PEOU, and they in turn BI.

TABLE I .
POTENTIAL INFLUENCE OF INFORMATION-AND SYSTEM-RELATED DESIGN CHARACTERISTICS ON SELECTED ANTECEDENTS OF PU AND PEOUA LEARNER-CENTERED DESIGN, IMPLEMENTATION, AND EVALUATION APPROACH OF LEARNING ENVIRONMENTS TO FOSTER ACCEPTANCE