Service Migration Algorithm Based On Markov Decision Process With Multiple QoS Attributes


 With the rapid development of mobile internet cloud computing, the traditional network structure becomes non-suitable for advanced network traffic requirements. A service migration decision algorithm is proposed in the Software Defined Network(SDN) to satisfy differential Quality of Service(QoS) requirements. We divide services into real-time ones and non-real-time ones due to their different requirements on time delay and transmission rates, and construct the revenue function on two QoS attributes i.e. time delay and available transmission rates. We use the Markov decision process to maximize the overall benefits of users and network system to achieve the best user experience. The simulation results show that our proposed algorithm achieves better performance in terms of overall benefits than the exiting algorithms only considering single service and single QoS attribute.

Compared with work [10] and [11], the authors in [12] considered the two-dimensional user moving model to enrich the revenue function for the user. [

Application scenario
We consider wireless cellular network under SDN architecture shown in Fig.1 that each cell corresponds to a service area with a server. As shown in Fig.2, when the user terminal moves from service area 1 to service area 2, it will send a service migration request to server 2 in service area 2. The united cloud formed by these distributed servers will make a trade-off between the QoS requirement and the system overhead for service migration. A decision will be made whether the current server should migrate service to others If the migration request has been accepted, the service data will be migrated from server 1 to server 2. Otherwise, the user keeps communicating with server 1.

Define the action
The action is the decision made according to the revenue function under the current condition. It makes the state transferred from s to s through action a.
When action a equals 0, the real-time and non-real-time services do not migrate. When action a equals 1, the real-time and non-real-time services migrate. In this paper, we assume that the real-time services and non-real-time services migrate simultaneously or not.

Define the state transition probability
To calculate the overall benefits by summing instant benefits of state space, we need to know the probability of each state. If the current state is already known, then we can calculate the probability of each state through state space transition probability since the next state is associated with the current one. As Another example is that The transition probability above is just   Combining the state transition probability matrix 1 P and 2 P , we can get the global state transition probability matrix P as:

Instant benefit
Instant revenue function depends on the current state, and a revenue function can be generated from a given state s . The specific instant revenue function is defined considering the following parts.
(1) The time delay benefits: The  (2) Part of available rates benefit: Now, we consider available rates, which is another attribute that influences the QoS of user service. Real-time services and non-real-times service of the user have their different transmission rate requirements. The user's benefit reaches the maximum when the service rates equal the rates the service requires. The data rate higher than the required rate will cause a waste of server resources, and the data rate lower than the required rate will lead to insufficient user experience.
Combining the above demand relationship and refer to the relevant model in [16], we can get the available rates revenue function b(s, a): (4) Instant revenue function: According to (6) and (7), we can get the revenue function:

Optimization model
We have defined the instant revenue function above, the goal of our model is to maximize the overall benefits. The overall revenue function is: where  is a service migration decision policy constituted of a series of actions under specific states, the initial state is s 0 ,   Proof: For each state is associated with the previous state, so the overall revenue is related with the initial state and the actions of each state, we extract the first term from the expectation of (11), and obtain (14) as follows: We expand the first half of (14), it follows: and then transform the last half of (14) into (16): End of proof. Different  will lead to different values of overall revenue function by adopting different policies. Our goal is to find out a policy *  to maximize the summation of overall benefits for all states, and that is the migration algorithm we get last to make the user experience and network overhead optimized.

Model solving
This paper uses a numerical iterative method to solve the Markov decision model, the detailed process is as follows: Numerical iterative algorithm: