Remote Laboratory System: A Model Proposal and Implementation on X-Ray Laboratory Machines

—This work presents the development and implementation of a remote X-ray laboratory, located at CNR – ISM, Rome, Italy. The remote laboratory model is based on a innovative, flexible and scalable modular architecture which makes it easier to update, expand or deal with problems. The developed system, accessible for con-trol only to authenticated users, provides a remote control of the laboratory equipment, a real time visual feedback of the machinery and the opportunity to retrieve and make a preliminary analysis of stored data. These features offer the possibility to researchers of carrying out real research experiments remotely, avoid the exposure to ionizing radiation produced by the X-Ray equipment. The system also provides the opportunity of carrying out experiments programmatically, optimizing the use of available machine time and providing the possibility of running experiments in special environment-dependent conditions. Eventually, the system allows researchers located outside of laboratory site to run experiments in a collaborative way.


I. INTRODUCTION
The remote control of scientific equipment represents an extraordinary opportunity to allow the access of external users to laboratory equipment. In particular, when researchers have to deal with potentially risky environments, or to modify parameters in long time experiments, or need particular experimental conditions, or require an external collaboration, the possibility to carry out experimental procedures remotely provides a better utilization of experimental time and safer way to work. This is the case of X-Ray laboratories, which are submitted to radioprotection restrictions due to the possible exposure of users to ionizing radiations.
Indeed, checks of the experimental procedures, as Xray beam alignment, which would otherwise require specific cautions, can be done by means of an internet connection.
On the other hand, external researchers interested in the use of X-ray instruments may share the device time with the in-house researchers, so creating collaborative networks. The present work aims to describe flexible remote system architecture to control laboratory devices, in particular X-ray machines.
Remote labs are not a recent innovation, but most of the studies related to them have academic and educational purpose [11][12][13][14]17] and some of them only are devoted to operative research situations [15,16]. However, the main scope of the latter studies was to simplify the access of external users who need to execute routinely experiments by means of software packages, as in the case of the European Synchrotron Radiation Facility beamline for protein crystallography; or in the case of the National Crystallography Service (NCS) Grid network, UK, where it is possible to remotely monitor ordinary crystallographic studies and change a limited set of experimental parameters. Yet, in both these examples, the possibility of working remotely is a consequence of the standardization of the operations required to perform the experiments.
In our case, instead, the principal target is to provide aversatile system architecture that allows to utilize any kind of experimental device, and in particular any X-ray instrument in all its possible configurations as if it was "on your workbench", having a total control from an user interface and a real time visual feedback of the configuration of the devices.
In the present implementation, we use unconventional Xray machinery for various purposes, namely as, diffractometers, reflectometers, small angle scatterers, tomographers, spectrometers. For this reason, a wide variety of samples can be studied by using one or more techniques in a synergic way, thus obtaining better and more reliable information.
In the present paper, we focus on the description of the architecture and the software solutions for the facilities of the X-Ray laboratory of Istituto Struttura della Materia, CNR (National Research Council, Rome, ITALY).

A. Core Components
The framework we present is based on a modular architecture model that can contribute to improve similar systems [1,17]. The core components in this model are: 1) The laboratory instruments 2) The hardware servers 3) The instrument controllers 4) The NAT gateway 5) The centralized data system 6) The Visualization System 7) The Web Server 8) Software Package The modular architecture has the advantage to permit the detection and correction of malfunctions and errors.
Modules can be easily replaced without changing the framework set up and upgrades can be done on each single module without modifying the general architecture and preserving the communication protocols among the single modules.
1) Laboratory Instruments. The laboratory instruments represent the machines to be remotely controlled in an integrated way. The laboratory where this model has been implemented is provided with several X-ray instruments, namely diffractometer, tomographer, spectrometer, reflectometer, equipped with ancillary devices and diagnostics.
An example of such X-ray instruments is reported in Fig.1. It is composed by: 1) Water-Cooled X-ray Tungsten anode source (Philips, model PW2214/20). 2) Collimation Slits.
3) Sample holder. 4) EG&G ultra-pure germanium solid-state detector (SSD), cooled by X-cooler cryostatic unit. 5) Translational/rotational stages of the sample holder and arms motors controllers The continuous spectrum (white) X-ray beam is collimated by two slits upstream the sample. The beam impinges on the sample in the optical centre of the Xray machine arms. Other two slits downhill the sample select the portion of the diffracted (or reflected) beam, thus defining the acceptance angle of the detector. The detector is connected to a spectroscopic amplifier (Ortec ADCAM 92X Spectrum Master) that processes the signal coming from the detector and permits to visualize it, through a MultiChannelAnalyser, onto a computer screen.
Diffraction conditions can be changed by modifying the arms inclination or by moving the sample with the motorized sample holder.
In this example, the experimental equipment can be considered as a cluster of devices that must be controlled individually (in our case: X-ray detector, step motors, etc.).
The model does not impose any theoretical limitation to the number and kind of instruments that can be controlled remotely.

2) Hardware Servers and Instrument Controllers.
Each computer, directly connected by means of hardware interfaces to the devices (step motors, X-ray detector, etc.) is called Hardware Server, and runs a server application for each device. Then, a different computer, called Instrument Controller PC, hosts a main application (Control Software, fig. 2) which is in communication with the server applications on the Hardware Server. This main application integrates in a single software all the devices functions, allowing synchronizing tasks and programmatically executing actions that, in our lab implementation, are: beginning and stopping acquisition, moving arms or sample holder, etc.
The goal of this structure is to have a scalable, robust, and simple to understand system, taking advantage from a client/server structure, i.e. not to be forced to implement all the low-level codes on a single, heavy and complicated application. A basic scheme is reported in Fig. 2.
A NAT [4] (Network Address Translation) gateway has been implemented to interface the external world with the internal net services. It allows to run multiple services (ftp server, web server, application interfaces etc.) over a single IP address, using different ports.
The NAT gateway is also used as a firewall for security. In the present implementation, the gateway has been realised by installing open BSD [5], an open-source unix-like operating system, on a Pentium 3 platform.
The Web server has been implemented using an Apache distribution on a Penthium V machine, running an Ubuntu Server operating system [9]. The ftp server in vsftpd, included in the Ubuntu package.  To handle the resources remotely, a web site has been developed. A screenshot of the site frontpage is reported in Fig. 4. 4) Centralized data system. The experimental data are collected from different instruments and stored to be available at any time to authenticated users. The storage should be furthermore persistent over time, data should be simply retrievable, well organized, and satisfy integrity constraints (avoid data duplicates).
The easiest and most straightforward way to guarantee all these requirements is to use a single machine to store all the data. This means that each Instrument Server doesn't save data locally, but has its own memory space in the centralized data system located in the internal laboratory network.
Precautions have been taken to avoid possible connections problems: 1) Installation of UPS units to prevent system crash due to interruptions of the power supply. 2) Preence of a backup unit to prevent loss of data caused by memory damages (Maxtor oneTouch II).
The centralized data system must be accessible outside the laboratory network only for registered users who wants to download experimental data remotely. This function has been implemented installing an ftp server [9].

5) Visualization System.
When performing a remote experiment, a feedback on the control of remote equipment is needed. It is given primarily by software interfaces accessible by internet connections, which indicate in real-time, the actual value of the experimental parameters. Also, an on-line visualization feedback is given by a group of remote (Sony Network Cameras SNC-RZ30P).
A visualization control of the experiments allows remote checking of malfunctioning or damages, and verifying the experimental conditions, for instance the Xray instrument arms positions.
Furthermore, this is an important utility for external users who want to collaborate with an internal operator, giving them detailed instructions by phone and verifying their accomplishment in real time.

6) Software Package.
A software package has been expressly written to implement the conceptual framework, and represents a core component of the system.
The package implements the following functions: communication between the instruments and hardware server; communication between remote users and local applications; data upload/download; preliminary data analysis; running of automatic tasks to perform complex and long time experiments. Most of the software has been written in LabVIEW 7.1 ® .

B. Laboratory Software package architecture
When a software application needs to integrate several functions to control several hardware devices (as in the present case) and to be able to fulfil scalability 1 requirements, a modular architecture system is a suitable solution, because it focuses on the interaction and communication between the modules/parts of the system, rather than on building and upgrading of a single software application, which implies serious testing and validating issues that in the modular architecture case are limited to the single module.
The client-server architecture has been widely used in this project, both for end_user-laboratory_instruments communication, which will be described in the remote system architecture section, and for main_application-server_applications communication inside the laboratory system. A detailed technical description of the latter can be found in [3], while the following sections will describe the underlying model of communication between the server application installed on the hardware server, which controls the hardware of the Xray equipment, and the main client application installed on the instrument controller.
These parts of the model only concern the communication between modules of the software package and not the communication between external user and the laboratory system. 1) Client-Server Architecture.
The idea underlying the inner laboratory client-server architecture is to create a modular, scalable system, a representation of which is given in fig. 2.
The client consists of a main application, the Control Software, installed on the instrument controller computer.
Each part of the laboratory instrument (i.e. motor, detector, ecc) is controlled by a different device on the hardware server. For each hardware device a server application has been created (server app in fig1.) that communicates with the devices by means of drivers.
The communication between the main client application (Control Software ) and the server applications is carried out by means of proper communication protocol over an ethernet connection.
In the present project, the laboratory equipment consists of a cluster of devices such as: motors, detectors, temperature or water flow sensors etc.

2) Main Client Application.
1 Scalability is a desirable property of a system, a network, or a process, which indicates its ability to either handle growing amounts of work in a graceful manner, or to be readily enlarged. The client was named XMCA (Acronym of "X-Ray Multichannel Analyzer") and built in LabVIEW 7.1 ® graphical development environment. Its main characteristics are: a) Total control over laboratory equipment (controlled devices: axes stepper motor; detector; multichannel buffer. Possibility to control: the High Voltage X-ray power supply; calorimeters; variable temperature chambers; cooling water currents etc.) b) Possibility to run automatic experiments whose parameters change programmatically c) Preliminary data analysis.
A deeper description can be found in [3].

3) Server application and protocols .
The server application was written in LabVIEW ® , unlike the detector server application, which was developed by ORTEC ®.
The communication protocols in use are: a) NI Data Socket [2] protocol for communications with server apps written in LabVIEW ® . b) Proprietary ORTEC ® protocol.
The main application can communicate easily with the server application: in the case of Data Socket, Lab-VIEW ® a control over remote application objects and variables by DataSocket, whose use is transparent to the programmer; in the case of the ORTEC ® protocol, ActiveX controls are adopted: they communicate with a server application (MCB Server) installed on the same computer, where the main application (instrument controller) resides, and are used by LabVIEW ® routines. For a deeper description of the ORTEC Protocols see [10].

C. Remote system architecture
In order to provide a versatile system, the possibility of connecting and controlling remotely every instrument is of primary importance. The framework used to accomplish this task can be described by means of client-server architecture model and connections, user permission model, and UML model. 1) Client-Server Architecture.
The laboratory system consists of several connected machines, which compose a Local Area Network, as seen in fig.3 and fig. 5. Some of these machines are available and accessible from outside by means of a redirection protocol (in the present work Network Address Translation has been used. [4]).
The reason for using a redirection protocol is the following: a) Enable multiple hosts on a private network to access to and to be accessed to/from the Internet using a single public IP address, thus saving ip addresses, often not easily available, b) Prevent malicious activity initiated by outside hosts from reaching those LAN machines which should not be seen from outside, c) Simple control of users and traffic control is much simpler because all the traffic passes through a unique machine, d) As a consequence, the end user, who just connects to one internet address, sees the laboratory system as a black box, which hides the complex structure of the framework.
In fig. 5, the Global system architecture is represented: on the right side the software elements are depicted (browser, web server, remote panels etc.) and elements which deal with data (internet connection); on the left side, the hardware elements are shown (NAT Gateway, Instrument Controllers etc).
A typical connection sequence develops this way: the remote-client (for instance home computer, with a browser and the LabVIEW ® Run-Time engine plug-in installed) 2 connects to the laboratory web site (Apache web server on Ubuntu machine); it authenticates and, if the authentication is successful, the data flow is redirected, so that the communication is carried out only between the remote-client and the instrument controller 3 .
The server (LabVIEW ® Web Server) sends the application front panel (remote panel) to the client, whom can visualize it by means of the aforementioned plug-in. By this configuration, each machine, called instrument controller, actually works as a server, allowing a remote user to control the main application.
To accomplish this task, a web server must be installed on each instrument controller: the choice has been to utilize LabVIEW ® Web Server, which supports the use of remote panels technology [18].
The instrument server machines, instead, are not visible from outside.  2) Basic UML model. Not all users can access the system and control the machines, so a hierarchical model has been created to rule the site access. Taking advantage from the UML formalism, system users have been divided into three categories: Controller, Watcher and Common, as shown in Fig. 6.
Users who belong to the Common group can enter and browse the site, but can't control or visualize the control system, nor use the visualization system.
Users who belong to the Watcher group can enter and browse the site, can't take control of the program interface but can visualize it and use the vision system.
Users who belong to the Controller group can entirely control the system, i.e. they can control the system and look what's going on in the lab with the vision system.
As it's a hierarchical model, users who belongs to the highest level (Controller) inherit the permissions of users at lower level (Watcher and Common).
Users can be recognized by means of an userpassword authentication system. Password and username must be requested to the administrator by means of an online request form.
3) An example of Remote use. The use of the laboratory devices by an external remote user is very simple, provided that both JAVA runtime environment and the LabVIEW ® Runtime plug-in have been installed, as illustrated by the following steps: a) Connection to the site. The user types the site address (http://rxlab.artov.ism.cnr.it) and follows the links to 'instrument control' on the left hand menu b) Site log in. The user logs in by his personal username and password ( fig. 7) c) Instrument and camera selection. The user follows the link to 'camera' and selects the proper camera d) Instrument login. It represents a second security level: the user must provide also an instruments password. e) Instrument control interface and cameras. The user can control the instrument by means of the remote interface (remote instruments GUI in fig  8), and also visualize the sample, instruments and other parts of the machinery by means of a pan-tilt camera (remote instrument visualization in fig. 8)

D. Security issues in the X-ray laboratory implementation of the model 1) Password sniffing.
According to what discussed above, all users can navigate the site but not all can control the instruments. The clients must authenticate, using a username and a password.
In the present configuration of the system, the authentication credentials sent through the net are not encrypted, so that a malicious user could -through the use of a so-called sniffer-gather and use them to control the instruments.
Aware of this risk, the use of encrypted communications protocols (i.e. https) is being taken into account.
Anyway, it is worth stressing that the damages a malicious user could produce can only consist of an improper use of the interface or system. 2) Improper use of the Interface/ System. All actions that consists of an improper use of the interface or system can be referred to as Denial of Service trials. They can roughly be grouped as follows: 1) Repeated clicks on the load/save/clear buttons, so that a huge amount of data is sent through the net, thus creating traffic congestions, till the possible freezing of the interface. 2) Random, or even pernicious, movements of mechanical moving parts of the spectrometers/diffractometers that may damage the mechanical components.
It must be remarked that, to face those problems, the following precautions have been taken: 1) A maximum number of data can be loaded, to prevent memory overflows, http://www.i-joe.org

REMOTE LABORATORY SYSTEM: A MODEL PROPOSAL AND IMPLEMENTATION ON X-RAY LABORATORY MACHINES
2) All the mechanical parts have limit switches that do not allow the moving parts to reach improper positions.

III. RESULTS AND FUTURE PROSPECTS
A. From "educational purpose" to "real case" use A huge amount of experiments, model, ideas and technological innovations can be found in literature about remote laboratories [19], most of which are addressed to educational purposes (high schools, university etc.).
In this work, the aim was to create a robust, scalable, and working system to use in a real working context, i.e. the X-ray laboratory of CNR (National Research Council of Italy), Institute for Materials Science.
This system is currently operative and permits the participation of many people located on site or remotely to the same experiment; furthermore, it widely extends the possibilities of collaboration that nowadays play a pivot role to make interdisciplinary studies and to participate in international projects.

B. Radioprotection
The European Radioprotection legislation carried out by EURATOM 4 impose many constraints about the age, the effective dose and the operation protocols of workers who make use of machines emitting ionizing radiation [7]. When working remotely, possible dangers coming from ionizing radiations are prevented.

C. Full time use of the devices
When a research group is deeply involved in many collaboration and external projects, an optimization of the working time of the devices is required. The remote use has permitted the "full time" use of the instruments, as the presence of operator who controls the system is no longer required, and the experiments can be run 24 hours a day.

D. Special experimental conditions
Some studies carried out by instruments like AFM require special operational conditions, as the absence of electrical noise or vibrations [8], often induced by the use of other equipment located in the same building. It is often impossible to completely filter this noise or vibration, so that it is often necessary to perform the experiments at night or during not-working time.
Also in this case, the remote use has allowed to perform these experiments to be carried out or monitored when most of workers were absent and noise, vibrations or other kind of disturbance were not present.

E. Validation
This model has been validated in its implementation by several researchers belonging to Italian research in-4 European Atomic Energy Community (Euratom): "The Euratom Supply Agency's mission is to ensure a regular and equitable supply of nuclear fuels for Community users. The Euratom Supply Agency acts under the supervision of the European Commissioner which shall issue directives to it, possesses a right of veto over its decisions and appoints the Director-General". EURATOM also gives directives about the use of electrical equipment or natural substances emitting ionizing radiation. stitutes and Universities by connecting to the laboratory site url (http://rxlab.artov.ism.cnr.it) Those people has been using and testing the system for more than one year, ordinarily executing remote experiments and monitoring their development, sometimes with the help of a local operator for the experimental setup (sample positioning, axis alignment etc.).
After a first testing phase, the system showed to be reliable for what concerns the machines control and automation, i.e. the experiment always ran without inconveniences, data were stored and could be downloaded properly.
Some problems instead have been encountered in data flow redirection, in real time visualization and in remote interface visualization, so that an improvement of the network-related functions was planned.

F. Future prospects
The present implementation already allows internal and external researchers to work in a collaborative way. Anyway, it can be improved and expanded in some of its parts, which is possible without great architectural modifications or re-projecting efforts, thanks to its scalable and modular nature.
First of all, we intend to enhance the security by means of encrypted communication protocols as https, replacing or modifying the portal framework and the web-server configuration, to have an integrated control and managing of users authentication.
Another aspect is the redirection of the data flow. Protocols or architectures different from NAT can be used (for example, virtual servers on Linux/Unix machines, or reverse proxies), which allow users behind firewalls to connect to the laboratory without changing parameters on their browsers.

IV. CONCLUSIONS
The implementation of this remote system has proven to be useful for a number of reasons: it has provided the opportunity to carry out experiments programmatically, thus allowing studies requiring the variation of the dynamic parameters; it allowed the optimization of the machine time, thus filling the holes of "machine dead time" present before the remote system implementation. It also has permitted to increase the external collaborations with researchers located far from the x-ray laboratory.
Moreover, the remote system has permitted to carry out experiments in particular conditions, for example at night, i. e. when vibrations produced by the regular activity could produce interference with the measurements.
The modular architecture of this remote system has proven be an important characteristic. Thanks to modularity, it is possible to update, expand the system or deal with problems, without any shutdown of the whole system or introducing significant changes to its architecture.
The use of LabVIEW ® has played an important role in the development of a fast and practical solutions, as it easily allows the visualization of the instrument control interface remotely, through the use of Remote Panel technology [18].
Regarding future work, an effort will be done to enhance security, using encrypted communication algorithms and modifying both the portal framework and the web-server configuration, to obtain an integrated system for control and user authentication. Another future improvement is the redi-rection of data flow, which could be realised by protocols or architectures different from NAT.