Stimuli Generation for Quality-of-Experience Evaluation of Mobile Applications

Lars Wischhof; Jan Andreas Krahl; Robert Müllner

doi:10.3991/ijim.v16i06.28691

Authors

Lars Wischhof Munich University of Applied Sciences HM https://orcid.org/0000-0003-1074-9771
Jan Andreas Krahl Munich University of Applied Sciences HM
Robert Müllner Telefónica Germany GmbH & Co. OHG

DOI:

https://doi.org/10.3991/ijim.v16i06.28691

Keywords:

quality of experience, qoe, mobile device, mobile applications

Abstract

The assessment of Quality-of-Experience (QoE) corresponding to a given set of Quality-of-Service (QoS) parameters is of high importance for any mobile network operator. Methods for subjective QoE assessment such as MUlti Stimulus test with Hidden Reference and Anchors (MUSHRA) or Subjective Assessment Methodology for Video Quality (SAMVIQ) are based on collecting user feedback for one or several stimuli, i.e. differently processed versions of a source.

Generating these stimuli corresponding to a given set of QoS parameters for a QoE assessment of a mobile application is a time-consuming, non-trivial task. Therefore, in this paper we propose a novel publicly available open-source framework for stimuli generation for QoE assessment of mobile applications. It is based on the combination of network emulation and automatic user-interface control of a real mobile device, whose behavior is recorded and post-processed to generate suitable stimuli.

The article presents the basic concept of the framework, describes its open-source implementation, and concludes with an initial evaluation of the framework based on typical types of mobile applications.

Author Biographies

Lars Wischhof, Munich University of Applied Sciences HM

Lars Wischhof is a professor at the department of computer science and mathematics at Munich University of Applied Sciences HM, Munich. He received his Dipl.-Ing. degree in computer engineering from Technische Universität Berlin, and a Dr.-Ing. from the Institute of Communications at Technische Universität Hamburg-Harburg, Germany. His research interests are in applied mobile communication, in particular vehicular and pedestrian communication. He is member of IEEE, ACM and VDE. (email: wischhof@ieee.org)

Jan Andreas Krahl, Munich University of Applied Sciences HM

Jan Andreas Krahl is currently working towards his Bachelor of Science degree at Munich University of Applied Sciences HM. He is student research assistant at the department of computer science and mathematics (email: krahl.jan@hm.edu).

Robert Müllner, Telefónica Germany GmbH & Co. OHG

Robert Müllner graduated from University of Munich, Germany, with a diploma degree in physics, worked for Siemens AG and Nokia Siemens Networks on simulation models and product development for mobile networks. His research interests are Quality of Service and Quality of Experience, radio resource management and performance efficiency. Since 2010, he is with Telefónica working on radio design and quality strategies with focus on Quality of Experience (email: robert.muellner@telefonica.com).