The Efficacy of Deliberate Practice Delivered Using Asynchronous Training Technology

.


INTRODUCTION
At a time when learning technology is proliferating and corporations are spending millions of dollars on training, there is a need to quantify technology supported training applications, particularly among senior workforce members. Corporations make significant financial investments and human resource commitments to train workers in our knowledge-based economy, and with expanding elearning strategies and instructional methods available, choosing the best form of instruction is increasingly complex. Current research suggests some 70 percent of what is learned in traditional training is forgotten within weeks (Thalheimer, 2010).
Despite the proliferation of e-learning options for workplace training, the literature reveals continued attitudinal and institutional barriers to the adoption of online education regardless of the nature of the corporate training (Czeropski, 2012). A survey of European corporate managers found support for e-learning as a training costeffective strategy, with caveats that included limits on the ability to quantify the contribution, cultural resistance, and lack of learner motivation (Beamish, Armistead, Watkinson & Armfield, 2002).
Examining current literature, Gunawardena, Linder-VanBerschot, LaPointe and Rao (2010) identified a need for statistically-based studies (inferential and naturalistic) that help increase understanding of variables in corporate e-learning processes that lead to learning gains, transfer, and overall satisfaction among learners.
Deliberate practice is a form of training that includes coached, intensive exercises conducted over a period of time, aimed at perfecting skills and pushing learners beyond their level of ability (Ericsson, 2008). Much of the existing literature examines deliberate practice using simulation technology in educational contexts. For example, a meta-analysis (McGaghie et. al, 2011) found simulation-based medical education using deliberate practice was more effective than traditional clinical education for particular skill acquisition goals. However, in a corporate learning context involving seasoned employees requiring "just-in-time" training to enhance performance, there is comparatively little research.
This study looked closely at a specific corporate context using readily available e-learning tools to determine whether flexible, self-directed online practice modules and asynchronous mentoring by video and online platforms could be practical, effective ways of delivering deliberate practice training with minimal workflow disruption among senior staff.
Training was designed according to the deliberate practice theoretical framework with customized practice modules increasing in intensity and complexity (Ericsson, 2007) and delivered using highly interactive e-learning platforms, unfettered by traditional seminar schedules which interrupt workflow.
Comparing gains between the pretest and posttest scores, the deliberate practice group improved by an average of 6.7 points, while the seminar group improved by an average 2.9 points.

II. MAIN RESEARCH
Despite aging workforce demographics, few quantitative research studies have been undertaken to determine the impact of deliberate practice in corporate learning among experienced, older employees. Research suggests that experienced workers often reach a performance plateau that is difficult to overcome (Williams van Rooij, 2012).
The Council for Adult and Experiential Learning, an advocacy group for older workers, recently recommended new policy commitments to more robust workforce education that makes "ample use of new technologies and social web tools to improve the efficiency and effectiveness of the labor exchange process to meet the demands of today's employers and job seekers" (Heidkamp & Heldrich, 2012, p. 4).
Extensive experience is often thought to be a necessary component of expertise; however, research has suggested SHORT PAPER THE EFFICACY OF DELIBERATE PRACTICE DELIVERED USING ASYNCHRONOUS TRAINING TECHNOLOGY that while important, extensive experience does not necessarily lead to superior achievement. In fact, the performance plateau can be a stubborn barrier. Ericsson (2007) found that in many professional settings, well-performing staff became habituated to executing routine work proficiently without further improvement. However, among professionals who had reached this plateau, additional improvement emerged when deliberate, sustained efforts to change specific aspects of performance took place (Ericsson, 2008).
Deliberate practice applications are well documented in learning situations involving physical routines and skills; however, the workplace often demands complex skill and greater cognitive load. Gramm (2013) asserted that in order to serve the modern workforce, deliberate practice should be embedded in the job, with experiences "designed to include practice and reflection, build tacit knowledge, and design rich feedback" (para. 16). The challenge for trainers continues: How to find efficient, effective ways of supporting the novice to expert trajectory in the modern workplace setting.
In addition to the attention to experience level, this study adds to the literature by using an authentic workplace environment in which the training was conducted, and a skill domain rarely studied in the deliberate practice literature. The author attempted to quantify performance gains in writing competency among corporate communication professionals tasked with writing high quality copy under deadline pressure.
A limited group of 22 senior professionals were randomly selected from the company's staff database for the study. Senior staff was defined as those employees having worked for 10 years or more within the company and within their job category. 11 members received seminar training, while the other half received deliberate practice training supported by integrated learning technology.
Using a two group pretest/posttest design, improvement in communications writing was examined. Outcome measures were a reliable computer-delivered pretest and posttest that evaluated high level business writing skills required for interdepartmental written communication and report writing. Additional qualitative data were gathered in pre and post training interviews to determine perceptions, reactions, and overall satisfaction with the training approaches.
Training was delivered over a 14-day period. Within five business days following the administration of the pretest, the deliberate practice group underwent training. The deliberate practice modules were pilot tested and formatively evaluated to ensure that they met learning goals and the deliberate practice theoretical framework for increasingly complex activities and timely feedback (Ericsson, 2008). Some of the deliberate practice training was coached, while some involved self-guided video-assisted practice modules accessible online at the convenience of the participant. A coach/mentor was assigned for feedback and a prescribed regimen of deliberate practice activities was completed over the training period. Participants engaged in post-exercise self-evaluations after they received feedback from their mentor, adding to the qualitative data gathered in the study.
The video-assisted exercises required significant design and set up work prior to the training; however, once launched, the exercises were completed with minimal trainer involvement. Consistent with other studies in current literature involving simulation exercises and deliberate practice in the medical field, subjects showed rapid improvement with little faculty intervention (Issenberg et. al, 2002).
Synchronous and asynchronous discussions took place using Webcam technology, Wikis and discussion boards accessible on the company Intranet. A dashboard-like menu of e-learning tools allowed mentors to easily monitor activities and to deliver timely feedback. Data including duration of practice, modules completed, and feedback were available.
Mentors followed a prescribed protocol for delivering feedback, engaging in discussions with the trainees for evaluations and customizing subsequent exercises according to the deliberate practice framework. Criteria included ensuring that each exercises pushed participants out of their comfort zones and that they were authentic to daily work tasks. Within five business days of completing the deliberate practice training, participants wrote the posttest.
The seminar group underwent four days of traditional seminar meetings over the same two week period. The group received the same number of coached sessions compared with the deliberate practice group. Pretests and posttests were delivered on the same schedule for both groups in order to keep their schedules as similar as possible given the dramatically different training approaches. Mentor feedback was confined to the seminar sessions but was given according to the same guidelines as were used in the deliberate practice group.

III. ACHIEVED RESULTS
When the difference between pretest/posttest scores was compared, the DP group was significantly higher as shown by the t-test (p < 0.001). When the difference in only post scores were compared, the deliberate practice group was also significantly higher (p = 0.001). (Table 1). When an ANCOVA was performed to control for pretest scores, a difference in the mean posttest scores was evident. The deliberate practice group improved by an average of 6.7 points, while the seminar group improved by an average 2.9 points, a difference of almost 4 points (Table  II). To gather qualitative data, open-ended survey questions were administered after training to both groups. Deliberate practice participants expressed high levels of engagement, motivation, and appreciation for the break in "old-style" seminar training formats, with flexibility permitted by online access to many of the training components. Words such as "engaged," "energized," "motivated," "realistic," and "improvement" appeared regularly in the data. Participants expressed the general sentiment that they viewed themselves, their work, and the company more positively after the training regimen.
Deliberate practice participants also expressed appreciation that their workflow was rarely interrupted, and that the training tasks were "authentic," while those in the seminar group used less positive descriptors.
The seminar group's surveys showed repeated modifiers such as "difficult to schedule," "tiring," the vaguely positive "interesting," and the often repeated sentiment that the work was "repetitive" to describe their training experience.
The qualitative data showed high contrast in positive and negative perceptions about the training between the treatment groups. Participants in the deliberate practice group actually proposed an ongoing mentorship program that would include many of the e-learning elements experienced in the study.

IV. METHODOLOGICAL ASSUMMPTIONS, LIMITATIONS
AND DELIMITATIONS The two-group pretest/posttest design of this quantitative study will reduce threats to validity (Cohen et al., 2007). However, because it is impossible to make the randomly selected groups equal in all characteristics, the threat of selection bias is a factor worth attention.
The treatment group might have contained highly motivated employees, while the control group might have been skewed unintentionally toward participants with significantly less motivation or with attitudinal resistance to the training process. Even with randomization of the treatment groups, the inability to determine assignment bias means that the statistical power is weak.
The small sample size also weakens the significance of the research. Similar studies on larger pools of participants across more diverse learning and training domains would contribute to the literature.
In addition, there is no way to determine whether the process of pre-testing influenced the results because the methodology includes no baseline measurement against groups that went entirely untreated.
The methodological assumptions of the research included assumed differences in training protocols, differences in participant ability levels, differences in participant cultural backgrounds, and differences in participant attitudes towards training The researcher assumed there would be no change in the underlying conditions or traits that the researcher was attempting to measure between the pretest and the posttest. The allotted timeframe provided adequate time to execute both training modules, but not an excessive amount of time. Extended timeframes could have invited unexpected changes in underlying conditions between the administrations of the two tests.
The researcher assumed that the two training protocols for the study would be quantitatively different. It is impossible to describe seminar training as equivalent in all characteristics to the coached, feedback-enhanced, and intensive deliberate practice protocol. However, careful design of the training protocols ensured that both met best practices and standards of corporate training. In other words, the seminar training was well-designed, effective seminar training; not a poorly designed alternative.
The deliberate practice training protocol was designed in accordance to the standards outlined in the significant body of research that applies the theory to professional environments (Ericsson, 2008;Farmer & Williams, 2005;Kolb, 2007). However, even with such care, the training duration and nature was fundamentally different between the two groups. Moreover, the researcher could not control for the differing styles, approaches, and abilities of multiple mentors. Although the coaching was monitored throughout the 14-day training period, the need for multiple mentors further limited the research.
The study might have been further limited by the fact that observed differences between the groups may not necessarily result from the training and instruction, but from other unexpected factors. For example, those chosen for the study may have felt more special than those who were not chosen, or may have been more competent in the usage of e-learning tools. The Hawthorne effect (Cohen et al., 2007) might have had an unintended impact on the results.

V. CONCLUSIONS
Scholarly literature contains examples of expertise attained as a result of reputation, perceived skills, or extensive experience (Heller, 2000). Researchers have generally viewed superior performance as a predictable progression from novice to expert (Hoffman, 1992). Corporate literature documents flat performance among experienced professionals who rely on automated responses once high levels of competency are achieved (Koys, 2006). Many professionals maintain a stable level of performance with minimal effort for years, perhaps because they lack the motivation or the opportunity to improve their skills. Such professionals present a serious challenge for instructional designers and corporate trainers serving diverse populations of highly competent adult learners (Moore et al., 2007).
The lack of commitment to well-designed, integrated training hampers the ability of experienced employees to improve performance (Wang & Wu, 2008). Deliberate practice presents a promising and challenging tool for professional trainers in corporate contexts, particularly as e-learning is increasingly accepted in workplace training.
Current research supports the effectiveness of deliberate practice training (Ericsson, 2008) compared with traditional, over-used training treatments such as seminars and classroom sessions laden with visual presentations and overflowing with information that may never apply to authentic work situations, or be quickly forgotten.
This study suggests that deliberate practice training should play an important role in corporate training contexts. The approach is well suited to adult learners and the integrated use of technology, and it is readily customizable to diverse learner populations (Farmer & Williams, 2005;Gram, 2013;Roblyer, 2003).

SHORT PAPER THE EFFICACY OF DELIBERATE PRACTICE DELIVERED USING ASYNCHRONOUS TRAINING TECHNOLOGY
In addition, deliberate practice may be a valuable training tool when managers seek a less passive, more readily transferable method of employee training which incorporates the demonstrated benefits of mentoring, immediate feedback, and on-the-job instruction rather than timeconsuming, costly, seminar approaches (London, 2011).
In this study, deliberate practice training led to statistically significant performance improvement when compared with seminar training. Such experimental research approaches across larger populations and wider ranges of skill domains and learning contexts will further enhance the body of deliberate practice literature.
Critics of deliberate practice research have noted that sufficient efforts have not been made to design studies using sensitive, valid measurement instruments, nor have sufficient efforts been made to quantify the nature and amount of deliberate practice exercises needed to achieve improvement, particularly among experienced or "expertlevel" employees (Heller, 2000).
There is a need for further research into the nature and duration of deliberate practice training, its use in actual employment training settings, and its marriage with learning technology. Further research should use mixedmethod designs and larger sample sizes involving increasingly diverse learner populations. Such study would enhance the evolving theoretical framework for this type of training and help meet the needs of the ever-expanding adult learning population.