Sustainability, Social Impact, Learning and Training Innovation in Online Experimentation

Experimentation is an essential ingredient in any learning strategy about Sciences and Technologies (SciTech in the following). That is why the research on online experimentation is paramount to help confront the skill shortage in SciTech disciplines, by stimulating the development of more effective online learning and training approaches. Nonetheless, in this sector "functional" aspects are often overrated by researchers and no specific attention is paid to the sustainability of their proposals or to the possibility to analyze and improve the achieved social impact. A better integration among these aspects is then essential to move online experimentation research out of its infancy, to improve its perceived value and to increase its diffusion. Keywords—online experimentation, sustainability, social impact, online learning, online training. 1 Requirements for Online Experimentations In systems engineering, functional requirements define how systems react to a set of inputs to produce the desired outputs. However, functional correctness is necessary but not sufficient to define effective and successful systems, or even suitable ones. Horses, for example, are functionally perfect for transportation services, but not suitable and not effective in modern urban environments. This concept applies well to the current status of online experimentation research [1] (OER from now on), which is able to exhibit the functional correctness and feasibility of a large number of valuable online experiments. However, nothing or little is said about their suitability, effectiveness or successfulness. Several concepts and tools, borrowed from the business sector, can be usefully exploited to find which non-functional aspects must be considered to transform "functioning" online experiments into effective and sustainable ones. For example, referring to the suitability of the (possibly emerging) technologies adopted for a specific online experimentation, very useful conclusions can be drawn from the Hype Cycle [2]. Gartner's Group introduced it for providing a graphical representation of the maturity level, adoption degree and social application of the most relevant technologies at a given time. From the analysis of Figure 1, for example, we can see that "4D printers" are less mature and widespread than "volumetric displays" to represent some 6 http://www.i-jim.org Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp... phenomena, and that both of them are in a much less mature phase with respect to virtual reality, which is almost ready for large scale adoption and diffusion in 2016. Fig. 1. Hype Cycle for Emerging Technologies. Gartner. July 2016 It should be observed that Figure 1 represents a simplified view of the whole 2016's report [3], which includes more than 2000 technologies. A detailed explanation about how to use such business report in innovation initiatives, is out of the scope of this introductive paper but, in the opinion of the authors, this kind of considerations must become an essential part in any successful OER proposal. In this sense, referring to the specific topic of the "Experiment@ International Workshop 2016", which was "The Emerging Technologies on the Internet of Everything" [4] (IoE in the following), a second relevant source of information about nonfunctional requirements for online experimentation research is provided by the: "Navigating the IoE Roadmap of Challenges" report [5], published by "TM Forum" in 2016, which is a global association for digital business. As we can easily deduct from the table of contents shown in Figure 2, this report is oriented to managers and not to researchers, but in order to produce effective and sustainable IoE experiments, the latter cannot overlook the main concerns (monetization, brand management, trust, vision, etc.) typically expressed by managers working in the IoE field. Not considering these concerns, researchers could produce solutions which turn out to be not suitable/acceptable for the common user, not compatible with the most widespread IoE devices or simply not available out of their lab. iJIM ‒ Vol. 11, No. 5, 2017 7 Special Focus Paper—Sustainability, Social Impact, Learning and Training Innovation in Online Exp... Fig. 2. TM Forum Report on Internet of Everything 2016. Table of


Requirements for Online Experimentations
In systems engineering, functional requirements define how systems react to a set of inputs to produce the desired outputs. However, functional correctness is necessary but not sufficient to define effective and successful systems, or even suitable ones. Horses, for example, are functionally perfect for transportation services, but not suitable and not effective in modern urban environments.
This concept applies well to the current status of online experimentation research [1] (OER from now on), which is able to exhibit the functional correctness and feasibility of a large number of valuable online experiments. However, nothing or little is said about their suitability, effectiveness or successfulness.
Several concepts and tools, borrowed from the business sector, can be usefully exploited to find which non-functional aspects must be considered to transform "functioning" online experiments into effective and sustainable ones. For example, referring to the suitability of the (possibly emerging) technologies adopted for a specific online experimentation, very useful conclusions can be drawn from the Hype Cycle [2]. Gartner's Group introduced it for providing a graphical representation of the maturity level, adoption degree and social application of the most relevant technologies at a given time. From the analysis of Figure 1, for example, we can see that "4D printers" are less mature and widespread than "volumetric displays" to represent some phenomena, and that both of them are in a much less mature phase with respect to virtual reality, which is almost ready for large scale adoption and diffusion in 2016. It should be observed that Figure 1 represents a simplified view of the whole 2016's report [3], which includes more than 2000 technologies.
A detailed explanation about how to use such business report in innovation initiatives, is out of the scope of this introductive paper but, in the opinion of the authors, this kind of considerations must become an essential part in any successful OER proposal. In this sense, referring to the specific topic of the "Experiment@ International Workshop 2016", which was "The Emerging Technologies on the Internet of Everything" [4] (IoE in the following), a second relevant source of information about nonfunctional requirements for online experimentation research is provided by the: "Navigating the IoE Roadmap of Challenges" report [5], published by "TM Forum" in 2016, which is a global association for digital business.
As we can easily deduct from the table of contents shown in Figure 2, this report is oriented to managers and not to researchers, but in order to produce effective and sustainable IoE experiments, the latter cannot overlook the main concerns (monetization, brand management, trust, vision, etc.) typically expressed by managers working in the IoE field. Not considering these concerns, researchers could produce solutions which turn out to be not suitable/acceptable for the common user, not compatible with the most widespread IoE devices or simply not available out of their lab.  Table of Contents   2 The Relevance of Business Models As stated in the same TM Forum report, a third specific aspect to be considered for evaluating the sustainability of an OER solution, is the "business model" or, at least, a rough version of it. In research scenarios, this concept is not to be interpreted literally, i.e. by including earnings, profits, etc. Rather, it can be useful to define and describe the main involved stakeholders; the scenario (e.g. school, home, etc.) in which the solution should be considered; the main cost drivers (e.g. connectivity, maintenance cost, consumables, etc.) needed to deploy/operate it in a real context; the main goals to achieve; some ideas about competitors (if any) or alternative approaches to achieve the same goals. Particular attention should be focused also on how to build up a capable and motivated partnership that is interested in the proposed solution.
It is worth to mention that even great OER solutions can fail, or remain socially irrelevant, if these aspects are not taken into account.
Again, a complete description of how to design and implement a successful business model for OER solutions is out of the scope of this paper but it is worth to mention two methodologies, among others, named "business model canvas" [6] and "E3Value" [7], because of the quantity and quality of good literature and case descriptions available online in the field of technology innovation. The former methodology, initially proposed by Alexander Osterwalder [6] and based on his earlier work on Business Model Ontology, can be defined as a strategic management and lean startup template for developing or documenting business models. It is based on a visual chart with elements describing the firm's or product's value proposition, its infrastructure, customers and finances. An example is given in Figure 3, where a simplified version of the Google business model is represented. http://www.i-jim.org

Fig. 3. Business model canvas: simplified version of the Google business model
The latter methodology [7] stems from the conceptual modeling research field and it has been designed to help defining how value is created and exchanged within a network of actors, which can be very helpful for researchers working with online experimentation.

The Role of Social Innovation
As shown in Figure 4, around 40% of employers in Europe have experienced difficulties finding employees with the required skills [8]. The problem is even more pronounced in Japan and in India, where the need for technical skills is stringent. This lack of professional profiles with technical skills is commonly referred to as "skill shortage": it has a strong negative impact on our Society and requires to be confronted. For these and other relevant reasons, public funding programs are often adopted by advanced countries to stimulate the development of innovative learning and training approaches, including online experimentation. On the other hand, these approaches are likely to fail without synergic activities of social innovation [9]. As stated by Phills, Deiglmeier and Miller in their article for the Stanford Social Innovation Review, social innovation is: "a novel solution to a social problem that is more effective, efficient, sustainable, or just than existing solutions and for which the value created accrues primarily to society as a whole rather than private individuals". NESTA [10] defines social innovation as: "innovation that is explicitly for the social and public good. It is innovation inspired by the desire to meet social needs which can be neglected by traditional forms of private market provision and which have often been poorly served or unresolved by services organized by the state". In this sense, online experimentation researchers should consider social innovation as an opportunity to design not only new learning/training activities, but also new social contexts in which such activities can be effectively carried out and can produce positive outcomes. The Social Innovation Learning Model, adopted by the Stanford Center for Social Innovation, gives useful hints and examples in this direction but, obviously, it should be noted that a research project integrating social innovation and new learning/training approaches can be really challenging to define and deploy, even for strong research groups.
Charter schools, in US, represent a real-life example in this sense, but there is a strong debate about their effectiveness and their overall results.

Conclusion
Online experimentation represents a stimulating and increasingly interesting research field, with several opportunities, even for small research groups.
A non-negligible risk is to produce good research results with irrelevant social impact.
To overcome this problem, a suitable set of non-functional elements should be considered to integrate the functional characteristics of online experiments and to support their adoption in real-life contexts.
Combining social innovation with online experimentation research, in this scenario, can be challenging, but results can be worth the effort.