Improved Quality and Reliability in Telecommunication Network Testing by means of a Remote Online Test System

—Increasing demand for high quality and reliable operation of the telecommunication network services and its applications have been the main challenges of the telecommunication network providers in the last decade. In this paper, a remote online testing facility (Intact – Intelligent Test Automation and Configuration Tool) is introduced that proposes a solution for telecommunication network providers to meet demands and to ensure business services as well as network related services (circuit switched and packet switched) operate efficiently. This paper presents a particular solution of remote testing based on the automated functional “End to End” testing approach. “End to End” starts by configuring service properties, subscriber record and service logic on one hand. On the other hand several hundred call scenarios can be executed fully automatically. Moreover the call detail records written by the network elements are verified automatically too. Hence a test description and web automation language was developed which is very easy to learn. This testing approach improves test speed and quality leading to reduced risk and investment on test procedure. As a result, the proposed solution of a remote automated online test system aims to overcome higher cost, longer time to market periods and increased effort in testing.


I. INTRODUCTION
The telecommunication network operators face challenges of providing the customers with reliable, faster and more cost-effective services that depend on developing and maintaining a consistent set of quality system requirements and measurement techniques [1,2]. Development of the test systems has been a strategic approach in the sector in order to overcome the challenges and meet the requirements.
Integration and implementation of test systems comprise of a combination of various computing systems, telecommunication equipment and test engineers [3]. Existing test systems available for large scale telecommunication networks demonstrate disadvantages of being inefficient and high cost in operation.
The scope of this paper is to present a novel testing facility "Intact -Intelligent Test Automation and Configuration Tool" that proposes a remote online solution for the telecommunication network providers. The tool works based on a particular solution of "End to End" test approach that starts by configuring service properties, subscriber record and service logic fulfilling automated test calls and verification of the call detail records.

II. THE TEST METHODOLOGY
The test methodology was established based on automation approach. The Intact was designed to overcome intensive and expensive testing. Hence, the targets were defined as: • Test automation should improve quality.
• Test automation should be easy to run.
• Test automation should reduce risk and costs.
• Automated tests should be easy to write and maintain. • Automated tests should require minimal maintenance even "System Under Test" (SUT) changes continuously. • Test automation should provide Key Performance Indicators (KPIs) to determine status in an early phase of the project.
The tests are designed for both fixed and mobile network services that require huge amount of test complex test cases. The telecommunication services are defined in the Intact tool are mainly: • Intelligent Network Services • Service Node applications • Supplementary services • Private Branch Exchange services • and any kind of telecommunication services which can be initiated by a smartphone.
During the test development phase, the problems that occur in the test automation are reported as: • The used tools rarely provided an "End to End" view.
In complex set-ups with various options and services automation tools cover individual functionality. There is no tool which covers all services or provides an "End to End" view. • Many tools were expensive and they do not offer an interface to integrate the tools in a smarter environment. • Integration of simulators and other tools into the SUT environment was always very complex and expen- ously to be updated due to lack of compatibility, especially for "Graphic User Interface" (GUI) related activities, e.g. record and play back. • Test tools generally have their own language, syntax, graphical user interface, parameter definition and configuration. This makes it very expensive to create an "End to End" view for all use cases. A single automation engineer needs to know all of these languages in order to create "End to End" use cases.
Therefore, the automation test methodology of the Intact is established based on integration to the existing development cycle with the following approaches: • Test automation starts in the requirements definition phase of a project. Please keep in mind "Use cases" are exactly the same as "Test cases". When you define your "Requirements", you can derive them from "Use cases". And "Test cases" are "Use cases" with some detailed value settings. • Test automation has a scope and target. There are many stakeholders in the company and project. It must be clear to every stakeholder which goals the company and/or project follows.
• Test automation needs good quality of documentation, especially in terms of good "Use cases". • Test automation needs "Proof of Concept". If a test suite for an "End to End" test cannot be established/integrated within a short timeframe then there is something wrong! • Be aware of next steps: Test automation must be embedded into the whole development process. Also future versions of the product have to be supported and changes to existing test cases have to be reduced to a minimum. • Don't try to automate 100 % of your test cases. 80% is good enough. 20% manual testing will keep the know-how of your test team up to date.

III. THE AUTOMATED REMOTE TESTING: INTACT
Basic philosophy of the Intact is to make life easier for all testers of telecommunication services as stated in detail Intact Handbook [4]. Intact is an "End to End" testing tool for people who are sitting in test labs and making one phone call after the other in order to test a telecommunication service. The idea was to create software which performs all actions a human being would perform during test of a telecommunication service beginning with provisioning and configuration, execution of the test case and taking necessary notes and reports following an "End to End" view. All this shall be performed automatically. Following graphical representation gives a high level overview of the remote online configuration of the test tool.
Intact is designed for "End to End" black box testing. Everything a telecommunication user can do via his mobile or a graphical user interface are automated (Fig.1  The basic philosophy is: "Reduce boring manual test activity to a minimum". On the other side, a language and syntax are created in order to be able to test all telecommunication use cases, voice, SMS, USSD, MMS, IPsession, IP-event and provisioning activities via web browser or clients.
The second topic is: "Create a common environment, language and syntax for all use cases".
The third aimed topic is "Keep the language simple to understand, readable and learnable for everybody".
The Intact provides a wide range of functionality in order to overcome test automation problems of the past: • All tools and frameworks integrated in the developed tool are license free. • A simple alarming is integrated to support active probing respectively continuous testing. • The set-up is very simple and does not require complex and expensive integration effort.
The main approach was to keep "System Under Test" untouched. Applications and adapted tools were developed in order to control user, customer and administration equipment and devices. The following figure represents a good high level overview of Intact's integration (Fig.2). Fig.2 presents clearly that Intact does not interact with system under test. It only interacts with the triggering equipment e.g. mobile and fixed network phones (customer premises equipment); graphical user interfaces; CDRs/EDRs.
Each test case / test script has a specific amount of phones in use. The simplest test case is A calls B. In order to configure it we have to define two phones: E.g. +436641234567 and +436641234568.
A major concept is the unpredictable selection of used phones. This means when there are two phones assigned, which of the phones will be A or B is not known. This is intentional. We want to keep it unpredictable so we can detect additional faults which one would never find by same behavior. In the first executed test A can be +436641234567 and B +436641234568 and in the next test execution it can be vice versa. Moreover, it is also possible to do a dedicated assignment.

IV. CONCLUSION
The basic principles of a telecommunication service are generally the same: • It starts with a configuration action at customer or administrator GUI and ends: o o In a verification of values The Intact follows these basic principles. In principle we can define the Intact as a software program with an execution environment where soft bots (agents) are triggered by a test plan written in a common syntax (based on telecommunication terms in English). The soft bots themselves trigger telephones. Thus there are soft bots to control web GUIs. All actions triggered by the test plan are recorded an reported by a separate reporting and formatting module.
The test achievements during the test automation and testing are provided in the Intact to evaluate the quality at each step of the test. The useful metrics and key performance indicators that are used in the tests are: • Level of Automation With the quality management system integrated in the organization and the test automation system as stated in the ISO 9001(2008) and TL 9000 Quality Management System Requirements Handbook 5.5 (2013), continuous improvement is the basics of the whole process. The automation process is evaluated, documented and always aimed to improve quality to achieve the desired key per-formance indicators. The international standard ISO/IEC 15504(2004) [5][6][7][8][9] under the general title Information technology-Process assessment is considered for this work as the framework with compliance to the rules stated in the ISO/IEC directives.
The metrics and key performance indicators of the system provide consistency, efficiency and quality while reducing redundancy and improve customer satisfaction. As a conclusion, it enables increased competitiveness together with improved quality in operation.