A. QoS versus QoE
The QoS was defined by the ITU as [6] “Totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service”. Another definition is that QoS is a set of techniques that offers to applications the service they need, from end to end. The goal of quality of service is to provide priority to networks, including dedicated bandwidth, controlled jitter, low latency and improved loss characteristics. So that service providers could offer the best possible service to their clients.
In other words, Quality of Service, is the prioritization of some services using the network, such as VoIP telephony, messaging, video conferencing or video surveillance. It allows to classify the different types of applications according to their importance, in order to assign more or less bandwidth, and thus optimize the network.
To evaluate the quality on offered service, QoS was the only metric used in the past. However, since it relies only on technical measures related to network performance, it doesn’t really reflect users’ assessments. For the same service two users could have different appreciations. This is due to the performance of the used device, the expectations or the feeling of the users at that moment, their social and intellectual environment and their emotional state. Thus, a new metric appears named QoE or Quality of experience.
The International Telecommunication Union (ITU) defines the quality of experience in ITU-T P10/G100 [7] “the overall acceptability of an application or service, as perceived subjectively by the end-user”. The European Telecommunications Standards Institute (ETSI) [8] defines the QoE as "the performance of a user when using what is presented by a communication service or application user interface".
By definition, quality of service and quality of experience are two performance indicators for a service but from different ways. For QoS – Quality of Service it takes into account the network characteristics/behavior Performance guarantees given by network provider based on measurements. Regarding QoE – Quality of Experience: it considers the impact of network behavior on end users as some imperfections may go unnoticed and may render application useless. It is not captured by network measurements. The table 1 presents a comparison between QoS and QoE.
Table 1: QoS and QoE comparision
All the definitions presented above consider that QoE is a subjective measurement provided by the end user that reflects the degree of satisfaction of the used service, While QoS is an objective measure provided by clear measurement methods based on indicators. This type of evaluation incorporates the end-to-end system and especially the user’s appreciation. This makes its meaning more complete but also exposes it to several factors that may affect the results.
B. QoE measurment factors
QoE is a multidimensional measurement which can be affected by a variety of factors. By definition, the factors that can affect the QoE are: “Any characteristic of a user, system, service, application, or context whose actual state or setting may have influence on the Quality of Experience for the user”. Services and applications, the human experience can be influenced by various factors that have an impact on QoE. In this part, we define the general factors that can alter the QoE and the specific factors to our context among them. Factors that may have an impact on QoE can be classified into 4 categories. They are shown is the figure 1:
2.2.3.1 The context factor
As mentioned in the previous section, QoE represents the end-user’s perception of quality. For example, quality perception of a multimedia service depends strongly on the viewing context in which consumption takes place. The viewing environment (physical setting) has a considerable influence as it determines lighting conditions, viewing distances, screen quality…
Six categories from different contexts are defined:
- Physical context (location and space)
- Temporal context (time of day, frequency of use, . . . )
- Social context (inter-personal relations during experience)
- Economic context
- Task context (multitasking, interruptions, task type)
- Technical and information context (relationship between systems)
According to K. Brunnström et al. "Context Influence Factors (CIFs) are factors that embrace any situational property to describe the user’s environment”.[10]
2.2.3.2 The user or human factor
A Human Influence Factor (HIF) is any variant or invariant property or characteristic of a human user [10]. The characteristic can describe the demographic and socioeconomic back-ground, the physical and mental constitution, or the user’s emotional state. "QoE is how the user feels about how an application or service was delivered, relative to their requirements” [11]. This is strongly influenced by the user’s internal states and predispositions. Common examples of human factors include not only gender, age, level of expertise, but also the psychological situation when using the service. Indeed, the properties related to the emotional and mental constitution of the user can play a major role in the final assessment of the user. Because of their complexity and lack of empirical evidence, we still do not know how human factors affect QoE.
2.2.3.3 The system factor
The properties of the technical system directly influence QoE. The term refers to both the entire chain of communication between the service provider and the end-user (eg network, terminal equipment), and technical characteristics of the service provided.
We can therefore classify them as sub-factors of system as follows:
- Media factors: When dealing with multimedia content, the configuration of the source of media, such as encoding and compression settings, the rate sampling, the resolution of the scene, the frequency of the images, has a high impact on perceived overall quality.
- Network factors: These factors refer to the transmission of data over a network and are closely related to network QoS parameters, including packet loss, delay, jitter, bandwidth, and error rate. The effect of these parameters on the perceived quality depends mainly on the type of multimedia application but evolves with time and / or with the location of the user.
- Device factor: User device performance may affect the whole user experience. These factors include, for example, the resolution display, colors, brightness. For example, if a high-quality image and high resolution is displayed on a low-resolution screen with few colors, most of the original intent of the image may be lost.
2.2.3.4 The content factor
For different types of content, there are different requirements system. For video or gaming, for example, the amount of movement and the bandwidth audio can influence the overall QoE. The content itself and its type of influence strongly the overall QoE of the system because for different content characteristics, different system properties are needed.
C. QoE evaluation models
There are various approaches to quantifying the QoE of a provided service. These approaches are classified according to the perceived quality assessment, if evaluated directly by humans or automatically by technical factors. In the first case, specific evaluation processes are used, called subjective tests, while in the second case, mathematical formulas or algorithms are exploited, called models goals. There is a third category of evaluation of QoE called “hybrid method”; based on the use of automatic goal estimator, relying however on the subjective tests available. The next figure 2 presents the main classification of the QoE models:
Subjective methods Subjective tests are usually based on controlled experiments with human participants who directly evaluate their experience with an application or service. Different techniques can be used for subjective evaluation. For example, users can rate their experience using an absolute rating scale or they can compare images and / or videos by specifying which is better. In all cases, the results are based on users’ opinions, past experiences, expectations, user perception, judgment and description skills, etc. One of the most popular subjective assessment methods is "Mean Opinion Score "(MOS) [13]. This method is based on laboratory tests under good conditions. specific, detailed by the ITU in [14]. The quality is then evaluated by the users based on feedback surveys of the experience lived on a qualitative scale (bad, poor, fair, good and excellent) numbered from (1) to (5) as shown in figure 3. Then, the MOS is calculated as the arithmetic mean of all the individual scores mentioned by the test subjects. The QoE is then attributed to this statistical value.
2.3.2 Objective methods
According to ITU, the principle of objective quality assessment is the estimation of subjective quality only from the measurement of objective quality or indices. Depending on the type of input data used for quality of experience assessment the objective method was classified:
- Media layer models: These models use the multimedia signal to calculate the quality of experience (QoE), following comparisons and do not require any information about the system being tested.
- Packet layer models: These models predict QoE only from packet header information and do not have access to the multimedia signals.
- Parametric planning models: These models use parameters of quality planning for networks and terminals to predict QoE. They require prior knowledge of the tested system.
Objective measures can be classified according to the availability of the original signal. Three major model approaches have been identified:
- Completed reference The QoE Estimation Algorithm requires access to both the reference input data and the degraded output data
- No reference The QoE estimation algorithm requires only access to the degraded output data.
- Reduced reference The QoE estimation algorithm requires access to degraded output data and some features from the original signal that the quality assessment system will use as secondary information to help evaluating the quality of the degraded output data.
2.3.3 Hybrid methods
The third type of QoE assessment method is a hybrid model that is located between the two categories subjective and objective. It works as a quality estimator automatic and objective, relying however on the scores resulting from the subjective tests carried out previously. These hybrid methods are based on learning tools called Machine Learning (ML), and they use subjective test scores like input parameters to form a QoE model.
This model then matches the network parameters (for example, packet loss rate, delay, jitter, etc.) to MOS score values. This model offers the possibility of predicting and / or estimating quality in time real.
This type of solution, lying between the subjective and objective methods, presents a significant advantage in the field of prediction and estimation of quality experience but remains very complex to implement and the learning stage is very long and hardworking. Furthermore, the learning stage of the neural network needs a huge amount of data, but in our case, we can’t use hybrid models because of lack of data.
D. QoE in the Cloud
A cloud computing environment must be elastically scalable; in other word it must have the ability to flexibly expand as the offered load and the business demands change. However, this feature requires the development of a diverse set of algorithms, like those outlined below. The study of Elastic Scalability and QoE Assessment for Cloud Services are prerequisites for the construction of an intelligent QoE Management and Control mechanism for the cloud resources. Various research works have been carried out dealing with the resource allocation problem and VM management to achieve better utilization of computing resources while avoiding overload situations considering QoE. Many research works concentrated on measuring the performance of cloud computing through measuring parameters such as availability, reliability, scalability, response time. Table 2 shows a literature review of different metrics related to IAAS cloud services.
In [15], cloud resource allocation is considered alongside dynamic resource provisioning using feedback control mechanism on the infrastructure level performance metrics. In [16], the proposed approach considers dynamic service level agreement (SLA). To improve users’ perceived QoE, various studies have come up with metrics, which are directly related to the performance of services such as video streaming. Mean Opinion Score (MOS) that evaluate the user QoE is calculated in [13] as metrics consisting of network bandwidth,video-bit rate, Round Trip Time (RTT), page load times and video interruptions.[17]
S. Dutta et al. inn [18] a novel scaling method that closely considers users’ QoE, in fact the solution has considered QoE’s feedback as a criterion to scale up/down cloud resources. the proposed solution is to build a QoE-aware resource management of virtual instances; to automatically provision and scale network services (NS) in an elastic way. H. Qian et al. in [19] firstly identifies interactions among the cloud entities and afterwards evaluates the QoE for the End-Users in this complicated environment. Work in [20] studied host reliability issues, from the perspective of the End-Users. IN [21] proposes a three-step approach to map SLA and QoS requirements of business processes to cloud infrastructures. [22] considers QoE in the cloud with power management issues, since it studies a service cloud environment with mobile devices. Emmanouil Kafetzakis et al. in [23], proposes a unified QoE-aware management framework, directly targeting to cloud computing environments. In [24] authors, proposes a methodology to estimates the QoE from the end-to-end response time and adjusts the estimated score according to the evaluation context. Sunny Dutta et al. proposes in [25] an approach that enables a cloud-infrastructure to automatically and dynamically scale-up or scale-down resources of a virtualized environment aiming for efficient resource utilization and improved quality of experience within the ETSI NFV MANO framework for cloud-based 5G mobile systems. Some companies like Infovista [26] or Compuware [27] offer proprietary, closed assessment solutions for monitoring quality at an IaaS level. W. Cai et al. [28] studied the popular cloud vendors for gaming applications. Though gaming applications involve video streaming, the Quality of Experience (QoE) for gaming are very different from the QoE of video streaming service. Besides, both works focus on the infrastructure services such as computing, storage and networking.