Non-Traditional Labs and Lab Network Initiatives: A Review

Lab-based education has always played an important role in teaching students. Making remote and virtual labs communicate with one another by creating networks of labs can enhance the traditional way of learning as well as reduce the costs of implementing and using labs. This paper provides a review of the literature on non-traditional labs and lab network initiatives up to 2020. With the term ‘non-traditional labs’, we mean virtual, remote and hybrid labs, whereas with the term ‘lab network’, we indicate a set of two or more cooperating labs typically connected through the internet. In this study, we used a recent and comprehensive framework for data collection, organization and analysis to gather information on 40 non-traditional labs and lab network initiatives. Thanks to this framework, the outcomes of our work highlight interesting trends of lab-based education, which pertain to didactical, organizational and technical aspects.


Introduction
Nowadays, there is the need to provide students with more and more technical skills rather than just theoretical knowledge. Lab-based education may play a key role in achieving this goal, as it is considered a meaningful learning tool, especially in scientific subjects [1]. Although the first reliable guide to early lab technique can be dated back to 1556, with the work of the German scientist Georgius Agricola, De Re Metallica, it is in the nineteenth century that we may see significant changes in laboratory layout, instrumentation and working styles, which include team research [2]. Lab-based education has always been considered essential, as it provides training in observation, arouses students' interest, and offers a consistent level of technical and practical knowledge; these very same intentions are still accepted even after a few centuries of lab history. In the Second Handbook of Research on Teaching, Shulman and Tamir named five purposes that may be accomplished with lab-based education [3]: 1. Skills -manipulative, inquiry, investigative, organizational, communicative.
virtual and remote labs may be educational hindrances, while others see them as useful supplemental tools [15]. In recent years, societal and technological changes have increased the use of distance learning. For instance, the current trend of Industry 4.0 is constantly asking for new digital competences and technological knowledge, especially on topics like Internet of Things, Auto-ID and RFID (standing for automatic identification and radio frequency identification, respectively), Virtual and Augmented Reality, as well as 3D printing and cyber-physical systems; to this extent, e-learning is a requirement more than just an option [16], [17]. Moreover, disruptive events can compromise the traditional learning, as the COVID-19 pandemic that is currently undermining all the certainties that academia and, more in general, educational institutions had about ways of providing students with learning paths: online teaching overcome the impossibility to educate students when infrastructures are not available [18].
If, on the one hand, digitizing and providing remote access to labs might decrease investment costs, as stated by [4], on the other one hand higher costs for online course development, instruction, and support services might bring online programs to higher tuition fees than on campus programs [19]. A possible solution towards the economic and financial sustainability of NTLs is sharing resources among different campuses to extend the amount of resources and their availability, therefore reducing the amount of investment needed [20]. Networks of labs -meaning two or more labs, linked together whether they are hands-on, virtual, online, or remote -can reduce the number and the cost of physical resources as well as the risks of negative outcomes, including the misuse of equipment, reagents, or materials during a training session. These benefits are immediately obtainable using networks of virtual labs; however, similar results are also achieved through developing networks of physical labs. Indeed, resource sharing could reduce purchasing and operating costs while risks can be avoided or limited by allowing only remote access to students and/or unexperienced users [21]. Students and teachers find these solutions very attractive because of their portability, ease of use and high efficiency.
Today, networks of labs are used in many areas, such as physics, chemistry, fuel industry, nuclear research, biology, engineering, and others. With the term 'lab network initiative (LNI)', we mean a project that produced a lab network. Much research has been produced on NTLs [6]. The topic is wide, as labs can be investigated from different points of view (e.g., organizational, didactical, and technical), and the possibility of building lab networks significantly increases research possibilities. As an example, the literature on this topic may range from describing specific lab characteristics to discussing lab relevance as well as advantages and disadvantages. However, nowadays it still exists a gap concerning the discussion on lab networks. The research gap is likely due to the fact that it is quite complex to assess the characteristics of lab network initiatives from a technical point of view because, so far, very few studies focused on both didactical and technical aspects. The possible cause of this fact could be traced to the lack of a common standards to evaluate the architecture of the existing networks of labs [22]. However, a proposal to organize information on technical aspects of NTLs and LNIs has been recently provided by [23].
The goal of this paper is to try to answer to the following research questions: • (RQ1) Does it exist a survey that collect relevant information on a wide set of NTLs and LNIs? • (RQ2) What is the state of the art of NTLs and LNIs? RQ1 is answered by a literature review of existing research; RQ2, instead, is answered by applying the structure designed by [23] for collecting and organizing information on NTLs and LNIs. This structure is adopted since it identifies the relevant information needed, from organizational, didactical, and technical perspectives. Specifically, we collected and organized information from 40 different NTLs and LNIs, and this information was validated by direct contact with several lab network personnel. Moreover, we will use descriptive statistics to understand and explain characteristics and trends of NTLs and LNIs over time to the benefit of teachers, students, and practitioners. Another important goal of our research is to provide existing and future NTLs and LNIs with relevant information on recent developments on this topic, giving suggestions concerning possible opportunities and threats as well as a guidance to understand which predictors should be considered to maximize the success and duration of NTLs and LNIs.
The remainder of the paper is organized as follows. The next section critically addresses the existing literature, which contains general issues about NTLs and LNIs and their usage within engineering education. Section 3 introduces the methods we used, and section 4 presents the results of our survey and a discussion of the findings. Finally, section 5 provides some concluding remarks together with guidance for future work.

An Overview of The Existing Literature
A large amount of research has been produced on LNIs as well as NTLs, especially in terms of their didactical and technical aspects. In the following, we describe the most relevant papers on this topic, report what they discuss and categorize them by focus (i.e., didactical, or technical aspects).
For the sake of clarity, before starting the analysis, we would like to establish common ground about the terms that will be used in the paper. Even though there is a lot of work on this topic, the terminology is not always consistent, and the definitions of types of laboratories sometimes can be confusing or even contradictory. We have searched within the existing options and have chosen suitable definitions that, in our opinion, are the most effective to convey our research. The definitions are presented in Table 1 and are mostly taken from Zutin et al. [10], if not otherwise indicated. The categories used in this paper, which are traditional and non-traditional labs as well as and lab network initiatives, are reported on the left-hand side of the table, and in the following section we will proceed to describe existing NTL and LNI solutions presented in the literature in accordance with these definitions. It is a traditional real laboratory where students can perform experiments and manipulate real objects while being directly located with the tools in the same room [24].

Non-traditional labs (NTLs)
Online laboratory It is an environment in which a student can run experiments or simulations over the internet. In the literature, the term 'webbased lab' is also used.

Remote laboratory
It is an online laboratory that provides real experiments through online connection. This definition entails the control of real equipment and the realization of real measurements at a distance.

Virtual laboratory
It is an online laboratory that provides applications or software simulations.

Hybrid laboratory
It is an online laboratory that links virtual and remote laboratory technologies. It provides real hardware experiments and software simulations as well.

Lab network Initiatives (LNI) Network of labs
It is an environment that combines at least two isolated online laboratories connected to each other. It can also be presented in the literature as 'lab sharing initiatives' [21] or 'federated labs'; however, in this paper, we propose to use the term 'lab network initiatives'.

Didactical aspects
Several literature reviews compare or describe different LNIs in terms of didactical issues [22], [25]. The interest in this topic began decades ago when, in the late 90s, Aktan et al. [26] provided a survey in which their laboratory network was compared with four pre-existing ones. The focus was not much on the type of experiments done through the labs, nor on their structure and architecture, but rather on the number and kind of funding organizations that supported the development of the labs.
A noteworthy and more recent example of an evaluation of existing labs can be found in Ma and Nickerson [22]. To compare labs in terms of teaching effectiveness, the authors analyzed more than 1,000 works, which were eventually refined to 60 articles through different selection criteria. Specifically, based on a proposal of Accreditation Board for Engineering and Technology (ABET), the authors developed their own evaluating method, namely the four-dimensional goal model for laboratory education. Its main dimensions are: 1. Design skills -Extent to which lab activities help students understand and solve problems related to key elements taught in class. 2. Social skills -Extent to which lab activities increase students' ability to solve open-ended problems through the design and construction of new artifacts or processes. 3. Professional skills -Extent to which students learn how to productively perform engineering-related activities in groups. 4. Conceptual understanding -Extent to which students become familiar with the technical skills they will be expected to have when practicing in the profession.
The authors noticed a general lack of interest in social skills. Even if social skills are explicitly identified and adequately stressed by several different sources, they are not discussed as often as other educational goals. Moreover, an important difference suggested by this study is that even if simulated and remote labs show similar interests in conceptual understanding and professional skills, design skills are an educational goal that is mostly disregarded in remote labs, with respect to simulated labs.
Balamuralithara and Woods [27] aimed to provide guidelines to make a proper choice between virtual and remote labs. A total of 15 labs (seven virtual and eight remote ones) were evaluated and compared according to the following parameters: (i) costs; (ii) equipment and facilities; (iii) hands on experience; (iv) reality and actual control; (v) accessibility; (vi) level of instructor's supervision; (vii) support and teamwork; (viii) educational benefits; (ix) safety issues and maintenance. For an informed and judicious choice, previous research has advised the following items: pedagogical needs of experiment -'it is very important to identify the objectives and expected outcomes of an experiment in order to select the suitable online lab'; economical and resources factors -'it is important to consider the price of simulation software, period of license and availability of expertise to change and develop the simulation software'; and participants -'novice or mature' [27].
Seiler [21] presented a literature review on current trends of LNIs, and he also discussed different approaches for lab sharing and their positives and negatives. Specifically, the author introduced the latest developments and trends in remote and virtual technologies as well as their applications in engineering education. After analyzing some existing LNIs, the author noticed that the key enabling technologies (to create a labs network) are multiple and heterogeneous, so he also advocated the need for a shared and common standard in this field. A similar conclusion was reached by Ma and Nickerson [22], who claimed that 'there is no standard criteria to evaluate the effectiveness of lab work'. Seiler also claimed that to establish a wider use of remote labs, a more generalized approach is necessary. The approach should include the development of a common language for lab integration as well as for comprehensive soft-and hardware toolboxes, together with documentation for the automated plugs and play distribution of remote and virtual labs.
The same reasons motivated Heradio et al. [25] to review past and present applications in LNIs. To give a panoramic view of the main issues from the past 18 years, the authors analyzed 6 cross-institutional lab-sharing networks in terms of their effects on control engineering education. By doing so, the authors discovered that the set of tools available for the development of virtual laboratories is becoming more powerful and more capable of generating a wide range of complex simulations. According to the authors, due to the technological factors, we are moving from 'proof-of-concept VRLs towards mature and affordable online labs'. As a result, virtual and remote labs will soon become extremely popular in control education.
Patterson [28] has discussed the need for concurrently providing traditional laboratories and NTLs, hence students might benefit from a combination of different approaches, rather than relying on just one of those, pursuing optimum education. Drysdale et al. [29] have further broadened the horizon discussing the suitability of providing traditional and non-traditional lab experiences, and the need for tailoring curricula to new skills and requirements of NTLs (i.e., knowledge about managing hardware and software design). The latest example of contrasting different types of laboratories is provided by Hernández-de-Menéndez et al. [30]. In their paper, the authors describe the state of the art of laboratories in engineering and compare 18 hands-on, virtual, and remote laboratories. To assess the educational effectiveness of selected labs, the authors used a method named KIPPAS -Knowledge and understanding, Inquiry skills, Practical skills, Perception, Analytical skills, and Social and scientific communication. As a result, the authors claimed that virtual and remote laboratories have the potential to substitute traditional labs in the near future. However, it is opinion of the authors that to get the most out of both solutions, a proper mix of physical and virtual labs is the best approach for experimental engineer education.
In recent years, however, the research focus on this topic has moved from investigating the usefulness of providing lab experience over distance and the validity of different solutions to assessing the reliability of specific solutions and looking for future directions (e.g., by means of surveys). For instance, the work of Foust [31] has summarized and discussed, from the curricula standpoint, the common practices used at various US colleges when providing Instrumentation and Measurement Laboratory class into the Mechanical Engineering degree, compared with the lab provided by the York College of Pennsylvania. Also, concerning curricula, Burghardt et al. [32] have identified, by means of a survey, possible teaching and learning methods and technologies to be introduced in future Internet of Things and Industry 4.0 learning environments.
Other recent studies analyze the pedagogical scenario and how students assimilate notions and knowledge disseminated by laboratories provided over distance [33]- [35], as well as the students' acceptance and intention to use NTLs [36], [37]. Also, the work of Parkhomenko, Gladkova and Parkhomenko [38] rigorously classifies and characterizes didactic and pedagogic scenario of different NTLs. The authors provide a recommendation system aimed at helping teachers, students, and developers of new labs with information about features and possibilities of NTLs, considering the users the labs are aimed at, and the curricula they cope with.

Technical aspects
Concerning the technical aspects of the LNIs, relatively few studies can be found in the literature. Actually, the number of papers that provide some technical details are not so few, but the analysis is generally limited to a single lab network, as in the works of Sáenz et al. [8], Zutin et al. [10], and Krneta et al. [39]. Conversely, we found just a handful of studies that analyze and/or compare more than one LNI from a technical point of view. This may be due to the fact that researchers and practitioners are often more focused on their own labs rather than on other ones as well as the difficulty of finding reliable technical information about different labs.
One of the studies that addresses this topic is by Gomes and Bogosyan [4]. The authors analyzed online experimentation approaches and collected information on support technologies that could open the opportunity to implement new functionalities in labbased education. To do so, a great number of papers were reviewed, and relevant information was extracted. By doing so, this paper offered an overview of current trends in remote laboratories and related technologies, with examples in several areas of industrial electronics: (i) the characterization of remote labs, (ii) their usage benefits, (iii) evolution, (iv) components, (v) communications among them, (vi) integration of LMS, (vii) topologies and platforms, (viii) remote laboratory applications for engineering education and (ix) research in various electronics fields of education.
Ponta et al. [40] compared three web-services-based architectures of remote laboratories -DIBE ISILab (Internet Shared Instrumentation Laboratory), HPI DCL (Distributed Control Laboratory) and MIT iLab-according to user interactions and interoperability between remote labs. All the analyzed architectures collected in a web service interface with all the functionalities exposed by the lab and use work sessions to structure measurements and store data sent or received from the instruments. The authors also stated that 'structuring remote laboratory functions as a set of services has the major advantage of allowing the sharing of the physical experimental setup, while leaving the possibility of customizing the client application interface'.
Another important contribution was made by Maiti et al. [41]. By using program logic analysis, the authors aimed to examine key inputs, outputs and impacts for the creation of remote access laboratories. They studied six lab network initiatives (Netlab, LiLa, iLab, Visir, Labshare and WebLab) to identify commonalities or differences among them. Among the initiatives, system architectures, availability of scheduling systems, capabilities, procedures to create new experiments as well as programming language were analyzed and compared. Specifically, to evaluate differences, the authors made use of the following criteria: 1. Origin of the remote access laboratory 2. Innovations to increase the students learning 3. Replication of classroom experience 4. Pedagogy.
The work even highlighted mutual influences and shared components of the examined labs; for instance, LiLa is used by others as a booking system, and Netlab initiates co-operative experiments.
An important survey and comparison of LNIs is discussed in the paper by Krneta et al. [39]. This study describes three labs networks developed by three different universities, all funded by the Tempus IV Programme project. Specifically, the LNI of the University of Maribor, the Faculty of Engineering of the University of Porto, and the WebLab by the University of Deusto are presented. All the LNIs are evaluated both from a technical and non-technical perspective. The experiments are described in detail, and the authors provide a deep description of many technical features, such as architecture, design software and kind of connection to lab server. The collected information is extensive and covers the most relevant technical aspects, but it has also some limitations. Most of all, the paper of Krneta et al. [39] does not propose any evaluation criteria (to assess the technical quality of a lab network), and all considered lab networks are related to the same funding project. As we will discuss in the following section, all these drawbacks will be overcome by the data-collecting-strategy adopted in our work.
Another important contribution was made by Abramov et al. [42]. The authors described the process they followed to create their own LNI. As a preliminary activity, the authors conducted research on already affirmed lab networks mainly in the biomedical and bioengineering fields to identify the main technologies used for their development. Specifically, they investigated the following four lab networks: Hhmi Bio Interactive, VIRTUAL LABS, Virtual Biology Lab and WOW Biolab. They characterized labs focusing on the following elements: (i) general and didactical aspects (i.e., which is the coordinating organization); (ii) sponsors; (iii) teaching fields; (iv) users of labs; (v) technical perspective (i.e., which is the technology); (vi) presence of a repository; and (vii) kind of user interface. Moreover, the authors listed the advantages and disadvantages of each laboratory. Having considered the biomedical simulation market, the authors concluded that the previously existing LNIs were developed in the past and they no longer met modern standards. However, they noted that among those networks of labs, there is a good material and technical base; moreover, many reference materials, teaching aids, active links and illustrations are also present. The paper is essential to quantify the benefits of using remote laboratories within teaching and learning processes; however, a consistent and rigorous structure for evaluating LNIs is missing.
A good example of a structured survey can be found in the work by Potkonjak et al. [43]. The authors developed an effective method to give an overview of the state-ofthe-art virtual laboratories and identified relevant technologies and current trends. Their method is based on four evaluation criteria, namely: • (C1) the user interfaces for each piece of equipment must be identical to the corresponding real devices. • (C2) the behavior of the virtual system (e.g., its state and control variables) must be equivalent to the system behavior in the physical paradigm. • (C3) visualization must be provided in such a way that makes students feel like they are looking at a real authentic thing. • (C4) a 3D laboratory space must be created which allows for communication and collaboration among students and with the lab supervisor (or expert in the field).
To provide a structured and comprehensive comparison of existing labs, the authors integrated their method with additional and relevant data (e.g., institutions, funding and teaching fields) to analyze more than 20 LNIs. The author considers all STE disciplines (science, technology, and engineering), with the emphasis on engineering and robotics.
The development and integration of remote laboratory management systems was a crucial task for Zutin et al. [44]. The authors presented a new paradigm to deliver remote lab infrastructure as a service. The authors investigated two lab network initiatives, namely WebLab Deusto and the MIT iLab, with the goal to minimize the requirements needed by online lab developers to connect their equipment to the internet. The authors provided information on architecture with a high level of accuracy. The differences between the connection solutions, federation models as well as the provided tools and applications were analyzed and compared.
Recent research on technical aspects of NTLs and LNIs has also moved towards investigating the reliability of implemented solutions, often by means of very precise and detailed descriptions, while also trying to look for future directions. As an example, Strenger and Frerich [45] suggest ways to overcome issues related to the digitalization of laboratories in engineering education by analyzing 10 different laboratories over a time period of 8 years; the work of Uckelmann et al. [46] proposes a general concept for assessing safety and security in federated labs, and the work of Hernández-de-Menéndez, Vallejo Guevara and Morales-Menendez [30] describes various types of university laboratories worldwide to present an updated state of the art on the subject.

Results
As it emerges from the literature review, no study has already performed a detailed analysis of existing labs, dealing at the same time with general and context information as well as with technical aspects. Generally, authors dealing with the describing NTL and LNI solutions limit their interest to just specific macro-category of their interest, identified as didactical and technical, or focusing on very specific problems of the own solution implemented. In few words, a holistic approach collecting data about NTLs and LNIs seems to lack, and hence it lacks a full overview of what have been proposed by institutions providing NTLs and by LNIs. An exception is the study of Hernándezde-Menéndez, Vallejo Guevara and Morales-Menendez [30], providing a discussion of both technical and didactical solutions implemented worldwide. However, the review seems to be more narrative and, in the opinion of the current authors, lacking on a systematic approach. Also, the fact that recent studies do not further justify the need or prove the reliability of NTLs, but rather they move the focus to the analysis of specific NTL or LNI solutions, suggests us two considerations. The former, the field is mature. The latter, it is timely to organize the existing material and depicting a comprehensive state of the art on this topic.

Methods
In the previous section, we reported examples of scientific works comparing different labs. The conveyed extensive literature review of online laboratories has revealed several publications in the area, with many papers discussing solutions also from a technical perspective. Although we acknowledge the value of these papers, we must notice the following limitations in the previous literature.
First, the number of LNIs analyzed in each work is often limited to a handful of them, with the sole notable exceptions of the works by Potkonjak et al. [43] and by Hernández et al. [30]. Quite often, in fact, scientists emphasize the largest and most famous NTLs or LNIs, without giving proper credit to the less renowned ones. Thus, huge labs, such as WebLab Deusto, UniLabs, MIT iLab and LiLa are examined almost always, as opposed to smaller but equally interesting labs, such as Ironmaking, USARsim, SCY and NeReLa, just to name a few of them.
Second, none of the examined papers conducted a deep research in terms of both didactical and technical features. Furthermore, criteria used to analyze LNIs are often limited (typically in the range from five to ten) and not clearly organized. Both these causes do not help readers to fully understand LNIs and NTLs.
To overcome these limitations, we used the method for evaluating LNIs introduced by Romagnoli et al. [23]. Here, the authors delimit three dimensions of relevant information: • General information -Which collects information on the project from the organizational point of view (such as project stakeholders, duration, activities) and it reports the sources where relevant information can be retrieved. • Context information -Which deals with lab's availability and characteristics, as well as with the didactic and thematic aspects of the labs, and with the kind of experiments and users. • Technical information -Which addresses directly the technical solutions implemented.
Every dimension contains a group of key attributes (KAs), either of a quantitative or qualitative nature, used to categorize the investigated laboratory. More precisely, KAs are described through a series of hierarchical multi-dimensional issues, which convey low-level (i.e., non-aggregated) information describing the implemented solution in detail.
As already stated in the previous section, Potkonjak et al. [43] performed a similar analysis using a well-structed framework to evaluate 20 LNIs. So, before applying the comprehensive method by Romagnoli et al. [23], we extended the original set of LNIs from 20 to 40. This was made following a two-way approach, as described next.
At first, to collect relevant material on existing laboratories, we used a bibliographic and abstract database as well as article citation tracking tools, such as Scopus and Google Scholar. We queried the Scopus database by using following strings: (TITLE-

ABS-KEY ['remote AND laboratory*']); (TITLE-ABS-KEY ["virtual AND laboratory*"]); (TITLE-ABS-KEY ['networks AND laboratories*']).
Over 500 records were identified and screened. Afterwards, we analyzed the abstracts to select only relevant material. We analyzed the laboratories reported in the selected papers and after gathering 40 LNIs, and we also referred to the websites of the specific lab network or to the linked project. Our focus was on the networks that publicly shared their architecture and technical details so that the selected framework for data collection and analysis could be used to its full potential. We aimed to trace commonalities and/or peculiarities concerning the way the investigated labs were built.
Next, we relied on contributions from a number of partner institution members of the research project reported in the acknowledgements section. We asked partner institutions to contribute to our research; we suggested them to include relevant LNIs and/or other laboratories of their interest not yet included in our list. After receiving our partners' suggestions, we stopped the list and started collecting information on the identified LNIs. As we did at step one, we collected relevant literature dealing with technical implementation details, and we also referred to each lab's website to get more detailed information. Specifically, we searched information on each key attribute, one by one, for each laboratory of the list using a specific string in Scopus database (TITLE-ABS-KEY [*name of a specific lab*]).
Finally, to cross-check our work, we contacted responsible people from every laboratory to see if they agree on the correctness of the data we collected or to allow them to amend those data. To do so, we identified contacts of one or two responsible persons from a related lab website. We contacted people in three different calls, between 2019 and 2020. Thus, the collected material can be considered quite complete. After all the information was regulated and reviewed by the labs' responsible people, we performed descriptive statistical studies. The results are presented in the following section.

4
Results and Discussion

Completeness and reliability of collected information
We collected and analyzed information from 40 LNIs following the framework described in Romagnoli et al. [23], which we will refer to as 'LNI framework' from now on. The LNI framework is very broad and detailed, as it covers general, didactical, and technical aspects.
However, since some of the selected laboratories were developed decades ago, it was not always possible to collect all data required to fill out all the attributes of the method.
In this regard, it is worth noting that each attribute can be categorized as either an 'always-required' field or as a 'required-on-condition' field. As an example, specifying whether the laboratory belongs to a network is mandatory, but the number of laboratories belonging to the network must be provided only in case of membership.
For this reason, on-condition data that resulted in a null score when their top-level criterion was not met will not be considered in the statistics presented next.
The percentage of the completeness of the gathered data and the URLs of all investigated labs are reported in Table 2. The same table also shows the presence of a network of labs and the number of years of activity (up to year 2020). As Table 2 shows, some of the labs (e.g., Virtual Electric Machine Laboratory) were created decades ago and are no longer active. Consequently, we experienced some difficulties in terms of data gathering and verification because it was not always possible to find the desired data in the literature, or, as an alternative, it was not possible to contact people in charge of the lab. Another reason for the low percentage of completeness is due to the fact that not all lab developers are willing to disclose their project activities or to share details of these projects' general structures and adopted technologies (indeed, N/A stands for not available). As it can be seen from Table 2, the percentage of valid attributes ranges from 90% to 16%, with 22 labs with 60% or more valid attributes, and only 5 labs with less than 30%. A similar analysis is made in Table 3, which reports the percentage of completeness of each category included in the LNI framework. The highest amount of information (90%) is related to the 'context information' area, which includes generic fields, such as the presence of a lab network, the number of affiliated labs, targeted users, teaching fields, types of laboratories and free access availability. Conversely, and as expected, we were less successful with 'technical information'. Likewise, the 'client layer' attribute scored only 41% and physical/virtual lab layers only 34%, due to the very detailed technicalities that are present in the LNI framework; those peculiarities were not publicly shared so often. We aimed to make the research as accurate as possible, with adequate information of each considered lab. To do so, we contacted people in charge of each laboratory, and of the 40 labs, we were able to contact 31 of them (77% of the total). Nine were no longer in place at the time of the analysis. A total of 21 active labs replied (68% of 31 contacted) and verified the collected information. Seven LNIs (23%) asked for minor amendments, which were implemented and then rechecked. The average percentage of changes on the information we filled in is equal to 8%; therefore, we consider our study accurate enough. Figure 1 shows the didactical fields covered by each lab. The research showed that almost all the laboratories cover more than one application domain. Indeed, it must be stressed that the percentages displayed do not equal 100%. This figure shows that the 90% of laboratories have interest in STEM (science, technology, engineering, and math), and most of them are also multidisciplinary. We can also deduce that engineering is currently the leading field, with a share of 52.5 % of the total implemented fields of applications from 1995 to date. This is because the development of practical and operating skills is a fundamental part of engineering education. In this regard, the possibility to visualize an experiment, maneuvering and dynamically fine tuning its parameters, is essential for the assimilation and sedimentation of purely theoretical notions learned in class. Moreover, labs allow students to work with unique, expensive, and potentially dangerous equipment that cannot be easily installed and made available for training activities in didactical labs.  (ii) researchers (17%); (iii) high schools (16%); (iv) companies (10%); (v) others (e.g., private laboratories, NGOs, and governments) (15%); and (vi) primary schools (5%). It turned out that most often these labs are used at universities, a fact that is not surprising considering the high costs needed to install and maintain real laboratories, compared to the ease of use and implementation of virtual and remote ones.

Organizational and context information
A similar analysis was made to the different financial options used by the labs, aiming to see if there is a link among funding, project duration and success. During the research, we identified three kind of funding sources, namely 'public', 'private nonprofit' and 'private commercial'. By public agencies, we mean those entities who invest money generated by the government to offer public services, such as public universities, maintained by public money. By private non-profit, we refer to private organizations, such as universities or foundations, that further social causes and provide benefits to public. Lastly, by private commercial organizations, we refer to standard businesses which make profits from their own activity, e.g., Logic Design Inc. and Microsoft Inc.  Table 4 reports the three main types of funding organizations: the average duration of the projects (expressed in years), the number of laboratories that were funded by a specific kind of an organization and their involvement into a network. First, we could find founding information for 22 labs or lab networks. Out of these, 15 initiatives (i.e., 68.2%) were founded by public bodies, with an average duration of 6.4 years. Among these, two subgroups can be identified -single state and international fundingwhose definition is self-explanatory. The difference in the average duration of these two clusters is limited, as the single-state projects lasted 1.2 years more than the international funded ones (6.9 against 5.7). However, if we separate lab networks from single labs, the differences become more visible. Publicly funded single labs experienced much longer duration if funded at the single-state level (average duration of 7.4 years) instead of by international bodies (3.3 years). On the contrary, publicly funded network of labs, whose average duration is slightly longer than its non-networked counterpart (6.9 against 5.9), experience longer average durations by 1.5 years when funded by international bodies.
Four projects (18.2% of the total) have been registered as funded by non-profit organization (two private universities and two private foundations); their average duration is almost twice the average duration of a publicly funded project (i.e., almost eleven years), and a difference in duration may be noticed between university funding and foundations.
Privately funded projects are the least frequent ones, with less than 15% of the entries, yet they show the longest duration, with an average duration of more than 23 years. In this case, and due to the low number of entries, the two-year difference in average duration between networked and non-networked labs looks relatively small.
As it can be seen from Table 4, public bodies are the main funders of laboratories in our database. The reason behind this may be that the public is also the main funder of education at the world level, and as such, there is a need to make education and training more efficient. Most of the projects with public international funds were funded by the European Commission (e.g., projects such as TEMPUS or Erasmus+). Yet it is opinion of the authors that private commercial organizations, whose ultimate aim is profit generation, mostly promote projects with a higher attractiveness in terms of economics returns. Thus, the high duration of these laboratories may be due to a restrictive selection of funded projects and to the continuity of funding that these projects received.  Figure 3 reports the start and end year of each examined lab. As a convention, 2020 was used as the ending year for all the labs that were still active at the time of this analysis. As we saw from Table 4, labs that were co-funded by companies show the longest lifespan (e.g., iLab and TEALsim, funded by Microsoft, and Robologix, funded by Logic Design).
As it can be seen in Figure 3, projects that started between 2006 and 2010 are characterized by a lifespan that is significantly lower than the average values of Table 4, regardless of the type of funding (e.g., Lab2go and Labshare). Since this fact looks peculiar, we hypothesized a possible explanation -the economic and financial crisis that exploded exactly in those years. As a matter of fact, the average lifespan of these nine labs is approximatively seven years, shorter than the average lifespan of all the other labs, equal to nine years.
Concerning public funded projects, two distinctive elements can be highlighted. First of all, they have the tendency to stop functioning shortly after the closure of the project, which can be the moment when public funding ends. Probably, the short duration of most public funding programs is the main cause of failure and the consequent closure of lab initiatives. Many funding projects behind NTLs and LNIs, in fact, have a duration of two or three years, which is definitely long enough to develop a prototype solution but maybe not enough to develop and promote a self-sustainable opportunity. A second interesting fact is the opportunities that can be opened up by labs networking. As it can be seen from Table 4, in fact, publicly funded labs that were networked (e.g., LiLa, Go-Lab, NeReLa, WebLab and many others) experienced a longer duration than non-networked labs (e.g., Robomosp, Pallete, VccSSe and RoblabUA).

Didactical and technical information
The trend of using different types of experiments over the last three decades is presented as a bar chart in Figure 4. The figure reports the number of experiments performed per type of experiment and per starting year of the lab, as the starting years of the labs have been clustered in 5-year periods. In detail, we considered four different types of experiments with increasing involvement of the experimenter: i. Repository -Strictly speaking, this category does not contain experiments, as a repository is defined as a database of performed experiments and obtained results to which students can only refer without any interaction. ii. Batch experiments -In this type of experiment, users fully specify the experimental setting in advance and submit the specification as a request, receiving the experiment results in return, typically after that the experiment has been completed. iii. Sensor experiments -This is a classical experiment, with real-time data stream, but with no real time interaction between experiment and experimenter. iv. Interactive experiments -Users fully control and monitor the experiment during its execution, typically by (slightly) altering the experimental settings or stopping an experimental run due to real-time information.
Please note that, just as in Figure 1, the sum of the bars (i.e., the number of experiments) does not correspond to the number of active labs in a certain time period due to the fact that the same lab can perform different types of experiments. As we may see from Figure 4, a clear trend over time cannot be spotted. Indeed, the number of different experiments seems to reach a peak between 2001 and 2010 and then decrease in the following decade (2011-2020). However, the last time period has a slightly shorter duration than the previous one, as we performed our research at the beginning of 2020; also, more recent NTLs or LNIs may not have published their results yet. However, Figure 4 suggests a couple of considerations. First, the figure suggests that the different types of experiments have been leveling out in time. Starting from a clear dominance of sensor and interactive experiments, the use of repositories and batch experiments have been relatively increasing over time, up to the level situation of the 2016-2020 period. Also, another interesting consideration may be given if we also consider the results of Figure 5, which we will explain below. The comparison of Figures 4 and 5 suggests that the experiments might have become more specialized over time: that is, an increase in active labs has not been met by a corresponding increase in types of experiments, as if more focused experiments were proposed by NTLs and LNIs.
As we anticipated, Figure 5 reports the number of active labs per 2-year period and per type of lab (i.e., virtual, or remote). We note that in Figure 5, simulations and serious games were considered in the virtual category, as their number did not justify a separate lab category. Also, we stress that, in the remainder of the paper, we chose to report the number of active labs per year instead of the starting year of labs. In detail, a lab which started in 2001 and it is still active will generate bars in just one-time period in Figure  4  Finally, we note that after 2011, the decrease of remote labs was less pronounced than that of their virtual counterparts; indeed, remote labs decreased for a short period but returned to their peak value over the last 5 years, reducing the gap between virtual and remote labs to its minimum value. Without deeply investigating the reasons behind these numbers, the higher percentage of remote labs could be due to the continuous increase in communication technologies and to the added value provided by real experiments, even if done remotely. Concerning didactical issues, our focus was on the main functionalities, if any, provided by the learning platform used by the labs. In detail, we considered the following aspects: • Access manager -This functionality investigates whether logs and activities of the experimenters are tracked -that is, if it is possible to check when users are logged in, for how long, to which experiments, or if it is possible to trace back results to experimentation sessions and users. • Course selection -Here, we verify whether professors or teachers can access the experiment repositories and didactical materials to create new modules, whether experiments or labs activities can be linked to specific modules and whether the results can be linked to the final evaluation. • Pedagogical scenario -The question here is whether it is possible to create specific learning paths for courses or single users.
• Learning analytics -This aspect investigates if data about learners and their contexts are collected, analyzed and reported. • Learning record repository or learner record store -This characteristic investigates if a repository for learning records is in place to collect results and other important interactions or information from connected systems. Figure 6 shows the number of laboratories that have implemented those functionalities over the years. As it can be seen, the growth of interest in the listed functions is slowly but steadily increasing over time. In this regard, it is interesting to note that some of these functionalities were already available before 2000. As an example, learning analytics and learning record repositories are among the functionalities that were implemented first. Moreover, Figure 6 shows that no clear dominance of didactic functionalities can be spotted in our research, with a slightly higher presence of learning analytics and data repositories. Still, the trend over time is that the total number of NTLs and LNIs has increased, with a corresponding increase in didactical functionalities and applied technologies. It is, therefore, likely to forecast that this upward trend of the suggested functionalities will continue in future years and that the transition to NTLs and lab networks may also increase. Larger student audiences may then be reached through online teaching, be it virtual or remote. In such a future scenario, the personalization of learning paths and new technologies will likely be adopted more and more.
If we shift focus to the technical features, we considered several lab peculiarities, organized in 5 clusters, as reported in Figure 7. The figure presents different kinds of physical and virtual lab applications over the years. Specifically, we focused on labs that implemented the following technical solutions: • Gamification -The application of game principles into non-game contexts to improve user engagement, foster learning, and support evaluation. • Sensors, actuators, and controllers -Possible devices and equipment used to operate the lab.
• Smart devices -Devices endowed with some onboard intelligence, such as, for example, identity and kind, memory and status tracking, communication capabilities, reasoning, and learning. • Smart sensors -Sensors that can collect input from the surrounding environment and, upon detection of specific inputs, perform data processing before their transmission. • Haptics -Sensitive devices able to actuate real equipment or to interact with virtual reality models by returning feedback information from the reaction force to the action performed by the user.
As it can be seen from Figure 7, the interest of the proposed technologies has also slightly increased over time, even if few labs implement these technologies.

Conclusion
The field of education over distance has been very prolific in the last decades, with a variety of topics and concerns tackled in scientific literature to reveal the importance of this field. Laboratories over distance, also referred to as NTLs (non-traditional labs), are often developed in medium-to-long term projects aiming at digitalizing traditional hands-on labs, upgrading existing solutions or introducing new one ex-novo. These projects often suggest the creation of lab networks, hence we labelled them as LNIs (lab network initiatives).
The present paper arises from the need of coping with two unanswered research questions; RQ1 refers to the need for a study that collects relevant information on a wide set of NTLs and LNIs from different standpoints, namely (i) organizational, (ii) didactical and (iii) technical. From a review of literature on the topic, we proved that, even if the interest of the research community has grown significantly in recent years, and a lot of material has been provided on NTLs and LNIs, a systematic approach for collecting relevant information on these topics was still missing. Therefore, RQ2 directly follows from the previous one and refers to the possibility of depicting an updated picture of NTLs and LNIs. Very recently, Romagnoli et al. [23] proposed a comprehensive classification framework to collect and organize the information of both NTLs and LNIs. This structure can help researchers and practitioners to get better insight into NTLs and lab networks. The method proposed in that study covers organizational, didactical, and technical perspectives, aiming to define the recent developments and to give future outlooks of provided solutions. The present study builds on those results, as it (i) collects and organizes data on 40 different NTLs and LNIs; (ii) reports some descriptive statistics on the types of users as well as teaching fields that labs are focused on; and (iii) takes a step further and analyzes the trends of NTLs and LNIs applications, considering the implemented didactical and technical aspects used to improve labs usability and results.
We found that laboratories are often used to teach more than a single subject. Indeed, remote, and virtual laboratories are frequently implemented to teach STEM, engineering topics in particular. As for the targeted audience, networks of labs are generally implemented by universities, both for didactical and research purposes. Still, some uses of NTLs and their networks also targeted toward high schools' pupils or company staff, whereas undergraduate students are a rare target according to our research, as we found only a single issue implemented by primary schools to teach environmental sciences.
As for the funding the labs received, we considered three categories of funding organizations. Almost 70% of the labs were publicly funded, with the remaining percentage split between private funding by non-profit (18.2%) and commercial organizations (13.6%). Non-publicly funded labs, however, experienced much longer durations on average, especially those ones funded by commercial organizations (i.e., profit-oriented ones). As a matter of fact, publicly funded NTLs and LNIs experienced an average duration of 6.4 years -that is to say that most of them have probably halted their work shortly after the project funding had stopped. We noted that although the limited dimension of the considered sample does not allow reliable statistics, publicly funded labs are almost equally partitioned in belonging vs. non-belonging to networks and in single state vs. international funding. Also, although single-state funded labs do not seem to benefit from being part of a network, the opposite occurs for the internationally funded ones, where belonging to a network seems to double the average duration of the lab (i.e., from 3.3 to 7.5 years). Still, the average duration of privately funded labs is significantly higher than that of publicly funded ones, and this suggests that the effectiveness, as well as the selection process and the support of the former group, is superior to that of the latter one.
Another trend that we noted is the fact that interest in remote and virtual labs, as well as the interest of different didactical and technical functionalities, had been growing rather constantly up to 2011, until the economic crisis hit the market. We provide charts to show the trends of using haptics, serious games, smart sensors, and actuators. As for didactical functions new tools and methodologies, such as self-regulated learning and collaborative learning in didactics, are becoming more important and can be supported by new generation labs. For these reasons, we studied the frequency of using pedagogical scenarios, course selection and learning analytics tools, as it seems it will be mandatory in the near future.
In conclusion, we mention the limitations of the present paper. Firstly, the proposed method of collecting material relied on published information, with the corrections of lab owners provided as notes and comments to the lab information we collected and formalized. This approach did not always allow us to collect all the available information needed for our structure and, therefore, some lab data were not considered in full. Access to the confidential material describing the lab implementation, as well as data-entries directly edited by lab owner could overcome this issue. Secondly, another limitation we accepted is the fact that that we did not evaluate the educational outcomes of those labs. Although this objective was clearly out of the scope of our paper, it clearly affects the didactical point of view on NTLs and LNIs. Finally, even if we collected lab data from different points of view, we do not provide any method of combining that information. As an example, assessing the didactical effectiveness of NTLs and LNIs while also considering the technical solutions implemented could be a very interesting research area. This is still a very open issue, especially considering the different natures of labs, and therefore the amount of data to be considered. Moreover, it would be interesting to find some useful indicators to assess successful initiatives not only in terms of project timespan. This would be especially useful to publicly funded labs, for discriminating between labs and projects that assure their financial sustainability beyond the project funding. Indeed, the authors are working on some of these topics for future research.

8 Authors
Giovanni Esposito is Mechanical Engineer earning his PhD in Industrial Engineering at the University of Parma (Italy). He is currently involved as a research fellow in the international research project DigiLab4U (http://digilab4u.com/), funded by the German BMBF, and he also act as local manager of the project for the University of Parma. His research interests comprehend digitalization of laboratories, and of industrial systems towards the Industry 4.0, supply chain modelling towards sustainability of business, and project management. His research also entails RFID application for supply chain management. He is co-author of 3 journal papers and other 7 conference papers (already published or to be published soon). He is a member of the RFID lab of the University of Parma (http://www.rfidlab.unipr.it/), the research centre of the University of Parma specialized in RFID UHF Class1 Gen2 applications. He acts as technology transfer experts and as a member of SMILE-DIH (https://www.smiledih.eu/?lang=en), the digital innovation hub of the University of Parma. He visited the Stuttgart University of Applied Sciences (Germany) as young researcher. His ORCID iD is https://orcid.org/0000-0001-5150-0855 Davide Mezzogori graduated with distinction in 2015 in Management Engineering and received a Ph.D. in industrial engineering in 2019 at the University of Parma (Italy). From 2019 he is a post-graduate research fellow at the Department of Architecture and Engineering of the University of Parma, where he is involved in the DigiLab4U international project (http://digilab4u.com/), for the development of a serious game for supply chain and operation management education. His main research interests concern the application of Machine and Deep Learning algorithms to Industrial problems, as well as the development of metaheuristics for operation management. He applied these techniques both academically and industrially; obtained results were published on prestigious International Journals and/or presented at International Conferences.
Davide Reverberi was born in 1992 in Parma (Italy). He completed his higher study as accountant, and he received his Bachelor and Master Degrees in Management Engineering at the University of Parma. During his university studies he worked in an industry for tomatoes transformation as person in charge of inbound logistic. He is now research fellow at University of Parma at the Department of Engineering and Architecture. Since May 2019 he is involved in the Open Digital Laboratory For You (Digi-Lab4U) with a focus on remote-access to laboratories and on lab-based learning. During this period he followed four different workshop on technical and educational topics for laboratories. He is currently co-author in five international scientific publications, one of those accepted on an international journal. Another publication is under revision.
Giovanni Romagnoli was born in November 1982 in Scandiano (RE), Italy, the town where he still lives with his family. He is a father-of-three. He received his master degree (with honours) in Mechanical Engineering for the Food Industry at the University of Parma, Italy. Since 2013, Giovanni Romagnoli is employed as Research Fellow at the Department of Engineering and Architecture of the University of Parma. Since December 2018, he acts as a local manager on the project DigiLab4U (http://digilab4u.com/), he has then become local scientific coordinator of the same project on December 2019. His research interests include virtual and remote lab network initiatives, RFID and supply chain management, production planning & control systems, improvements and applications of lean manufacturing, and led to the realization of more than 40 works published on International Journals or presented at International Conferences. He is member of the editorial board of the International Journal of Industrial Engineering Computations, and he's listed in the scientific committee of two International Conferences. Some of his work have been awarded with international prizes by the scientific community. His ORCID iD is: https://orcid.org/0000-0002-9891-0314 Email: giovanni.romagnoli@unipr.it Maria Ustenko was born in 1994 in Moscow (Russia). She received her master's degree in nanotechnologies and circuitry at the Peoples Friendship University of