Predictive Mobility and Cost-Aware Flow Placement in SDN-Based IoT Networks: A Q-Learning Approach

doi:10.21203/rs.3.rs-3119875/v1

Download PDF

Research Article

Predictive Mobility and Cost-Aware Flow Placement in SDN-Based IoT Networks: A Q-Learning Approach

https://doi.org/10.21203/rs.3.rs-3119875/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 25 Jan, 2024

Read the published version in Journal of Cloud Computing →

You are reading this latest preprint version

Software-Defined Networking (SDN) has emerged as an innovative method of networking that offers effective Nanyang Institute of Technology management and remarkable flexibility. However, current SDN-based solutions address networks' static nature or concentrate primarily on backbone networks, where network dynamics have minimal impact. This research presents a new approach, specifically the mobility-aware adaptive flow entry placement scheme for SDN-based Internet of Things (IoT) environments, to address the mobility aspect of networks. The proposed scheme employs the Q-learning algorithm to predict the next possible location of end devices, while the cost-sensitive AdaBoost algorithm is utilized to select heavy and active flows. Consequently, efficient flow rules for incoming flows can be dynamically implemented without the need for intervention from the controllers. Extensive computer simulations demonstrate that this approach significantly enhances match probability and prediction accuracy, while concurrently reducing the number of table misses and resource expenditure compared to existing schemes.

SDN

OpenFlow

Flow rule placement

Q-learning

Mobility

Matching probability

The development of Software-Defined Networking (SDN) has been aimed at enhancing the controllability of communication networks by decoupling the data plane from the control plane [1, 2]. In this regard, SDN centralizes network management, facilitates programmability, and accelerates innovation through the abstraction of SDN. As depicted in Fig. 1, the structure of SDN involves the implementation of OpenFlow as a communication interface between the control plane and data plane, whereby the OpenFlow switch utilizes flow tables to process incoming packets [3–5]. The OpenFlow controller in the control plane plays a crucial role in managing the entire network with a global view while defining how to deal with the flow using the flow table. The users’ needs are abstracted over the application layer which interacts with the controller through the northbound interface. The OpenFlow switch performs a lookup with one or more flow tables and forwards the flow to the controller through the channels [6–9].

The broad implementation of the Internet of Things (IoT) presently generates enormous volumes of data from numerous devices linked to SDN [10]. The management of such extensive data becomes a significant challenge, especially when data types and characteristics are diverse [11, 12]. Furthermore, it is essential to facilitate real-time transactions for time-critical applications. Delays in communication for such applications may result in catastrophic outcomes or significantly impede performance. Several approaches have been proposed to address the real-time challenge of SDN [13]. In SDN, incoming packets are managed based on the flow rule maintained in the switch's flow table. The controller can reactively (in response to arrive packets) or proactively add, update, and delete table rules. The controller can probe and handle flow entries at any time. Incoming packets matched with a flow entry are processed by the instructions kept in the entry. The probability of a table hit is dependent on the table configuration. If a flow entry handling table-miss exists, the instructions in that entry determine how to handle unmatched packets [14, 15]. The instruction options include dropping, transferring to another table, or communicating with the controller over the control channel via the packet-in message, resulting in a new flow entry. By default, flow entries for table-miss do not exist [16, 17]. A switch configuration can, however, override the default flow table and specify alternative behavior using the OpenFlow configuration protocol. With the increasing deployment of IoT devices, centralized controllers will play a critical role in SDN, especially for scalability and reliability [18].

The employment of OpenFlow has enabled network management to become more adaptable and efficient. Nonetheless, the current implementation of Software-Defined Networking (SDN) still faces limitations in addressing heterogeneous data and the constantly changing connectivity of Internet of Things (IoT) applications, particularly within a mobile environment [19, 20]. Additionally, the current SDN-based flow rule placement schemes make assumptions either about the static nature of the network or the backbone networks, which ultimately results in lower dynamism of terminal devices. In the typical IoT environment deployed in everyday life, both stations and mobile devices [21]. Therefore, the flow table must be dynamically updated in such an environment by allowing the switch to exchange user information with the backbone network. Given this context, it is paramount to devise a new model for flow rule placement that considers user mobility [22]. While two feasible approaches for flow rule placement exist, namely reactive and proactive, achieving the optimal trade-off between them represents a central research focus.

This paper proposes a novel mobility-aware flow rule placement scheme that minimizes communication delay by considering the mobility of the nodes. It capitalizes on the merit of both the reactive and proactive flow rule placement approach. In the proposed scheme, the position of the end devices is predicted using Q-learning (QL) algorithm to alleviate the computation overhead of the switch. The controller decides the placement of flow entries based on the prediction. The AdaBoost scheme is employed to classify the flow, which also helps the Q-learning in predicting the position of the nodes. NS3 and Matlab evaluate the efficiency of the proposed scheme. It reveals that the proposed scheme significantly increases the number of high-level flow rules in the flow table compared to the existing schemes. Furthermore, it indicates that the communication delay and load of the controller are substantially decreased. The main contributions of the proposed scheme, called mobility-aware flow rule placement with Q-learning (MAPQ), are as follows.

Even though AdaBoost has long been used in various fields, little research has been reported on efficiently and adequately deciding a strong classifier. A novel approach is developed which effectively achieves the final classifier considering the cost of error of the classification. Here the flows are classified into two types.
The QL algorithm is combined with the cost-sensitive AdaBoost in the proposed MAPQ scheme, and the approach is generic such that it can be readily applied to other application problems employing AdaBoost to improve the performance.
A scheme is developed which predicts the position of the end node using the QL algorithm to select the flow entry needed to be appropriately adjusted. Here the flows are classified based on the usage history to reduce the computation overhead required for proactive prediction.
A new approach based on the QL algorithm is developed in which the next state and the corresponding reward are not obtained as soon as a specific action is chosen but after some event occurs. This approach reduces computation overhead while supporting high accuracy of prediction with QL.

The rest of the paper is structured as follows. Section 2 discusses the work related to flow rule placement for SDN, and Section 3 presents the proposed scheme based on cost-sensitive AdaBoost and QL. In Section 4, the performance of the proposed scheme is presented through simulation results and subsequently contrasted with the current approaches. Finally, Section 5 provides a conclusive summary of the paper with some remarks.

2.1 Terminology

The basic terminologies of SDN are defined as follows:

Definition 1 (Flow entry). This flow entry constitutes the fundamental element of a flow table, encompassing the domain that specifies the circumstances required for a match (Match Field), direction pertaining to actions for the incoming packet that has been matched (Instructions), a record of statistics associated with matched packets (Counter), the maximum duration or idle time prior to the expiration of an entry (Time-out), among others. Table I showcases the basic fields of a flow entry.

Table 1. The basic fields of a flow entry.

Match Field

Priority

Counter

Instruction

Time-out

Definition 2 (Hard time-out, T_H). Once a flow entry has been assigned to the flow table, it is subject to eviction upon the expiry of T_H, regardless of the number of matches.

Definition 3 (Idle time-out, T_I). A flow entry must be removed from the flow table when the Time Interval TI has elapsed since the last match.

Definition 4 (Stale entry). The entry is unmatched before T_I expires.

Definition 5 (Elephant flow). This large flow takes up more than a certain percentage of the link traffic during a given time interval [23]. Assume a certain length of time, t. Large flows are those sending packets more than a given threshold (say P%) in the previous period of t, where P is usually 0.1% of the link capacity and its corresponding flow entry is a long-lived entry.

Definition 6 (Mice flow). Contrary to elephant flow, mice flow stands for a flow taking less than P% of the link capacity in the previous period of t. In SDN, the majority of flows are mice flow.

2.2 Related Work

A comprehensive investigation of multiple extant works concerning the placement of flow rules for SDN has been explicated in reference [19]. The current schemes for flow rule placement primarily address two main obstacles: resource constraint and signaling overhead. Concerning the matter of resource constraint, two methods have been suggested, namely, flow eviction and flow compression.

A. Flow Eviction

The researchers were motivated to develop effective flow eviction methods due to the flow table's limited size and declining performance caused by increasing stale entries. Openflow network implements a time-out mechanism to flush flow entries, but different time-out values have been used, ranging from 5 [24] to 60 seconds [22]. However, a fixed time-out value is inadequate for dynamically changing flows. To address this issue, the Adaptive Hard Time-out Method (AHTM) [25] was proposed, which utilizes the M/G/C/C/FCFS queuing system to model the flow table and analyze truncation time and blocking probability to determine the optimal value of T_H.

In [26], another scheme targeting dynamic T_H was proposed in which the flow is classified into four kinds and the flow duration time, flow type, and utilization of the flow table are taken into account to determine its value. SmartTime [27] combined the adaptive idle time-out scheme with a proactive eviction strategy and utilized the Ternary Content Addressable Memory (TCAM) to minimize the number of table misses.

DIFANE [29] proposed a distributed flow management architecture that efficiently handles wildcard rules and quickly reacts to network dynamics such as policy/topology change and host mobility. Instead of the time-out approach, Multiple Bloom Filters (MBF) [30] is used for logging data, where the importance of each flow entry is encoded and the entry with the lowest importance value is evicted upon a table miss. Flowmaster [31] uses a learning predictor based on the Markov model to identify flow entries expected to become idle and evicts an existing flow entry based on the number of transitions and transition probability using the time-out mechanism. A proactive approach is proposed based on predicting the probability of matching entries using HMM [32]. The scheme in [33] employs HMM and fuzzy logic to select the victim entry proactively.

B. Flow Compression

Applying a wildcard mechanism in compression reduces the number of flow entries present in a flow table while preserving the original semantics. This process involves the aggregation of specific flow entries into a single entry if they share common characteristics. Notably, the OpenFlow protocol does not require adjustment during flow compression.

Optimal Routing Table Constructor (ORTC) [34], which employs a binary tree to determine the minimal number of prefixes for each rule, represents one of the earliest methods for table aggregation. Once constructed, the routing table derived by ORTC remains unaltered. ORTC has also been implemented using structures other than binary trees [35].

The Palette distribution framework, introduced in [36], decomposes and aggregates large SDN tables into a predetermined number of smaller tables of the same semantics, which are then distributed across switches. The Espresso heuristic, presented in [37], proposes the compression of a minimum-size set of prefix-based match fields to achieve logic minimization within flow tables. DIFANE [29] employs decision trees to subdivide flow rules and assign them to corresponding switches, thereby circumventing controller bottlenecks.

Much attention has been paid to flow rule placement in addressing signaling overhead. Reactive and proactive categories of flow rule placement are explained in the following sections.

C. Reactive Rule Placement

A reactive approach to rule placement involves the placement of rules on flow-related events, such as table miss. As the OpenFlow specification outlines, the switch notifies the controller of its arrival for each new flow if there is no corresponding flow entry. Subsequently, the controller installs a new flow rule for the switch.

It should be noted that an increase in table misses leads to more frequent communication between switches and controllers for exchanging packet-in and packet-out messages. Moreover, setting up new flow rules takes considerable time, resulting in additional latency.

An example of reactive flow rule placement is presented in Fig. 2. Assuming that a flow enters the switch (Arrow 1), and its flow rule is not found in the flow table, the switch sends an OpenFlow message to the controller, and the controller sends the flow rule placement message to the switch (Arrow 2). After the switch deals with the flow (Arrow 3), it handles subsequent flows (Arrow 4,5).

Reactive flow rule placement necessitates continuous interaction between the switch and the controller. In [38], integer linear programming (ILP) and greedy heuristic algorithms were employed for flow rule placement. The controller puts unused links into sleep mode to minimize energy consumption, while considering the constraints on link capacity and space of the table, keeping the rules. An energy-efficient traffic forwarding scheme was proposed in [39], considering dynamic traffic in the network. The traffic management policies minimizing unnecessary traffic are presented in [40], and the routing strategy of [41] is based on Open Shortest Path First (OSPF), utilizing the global view of the network.

It is important to note that reactive rule placement typically involves high communication load and latency for the devices involved. Additionally, due to the limitations in the computation speed of the controller, handling flow messages quickly is challenging. Therefore, it is typically employed for elephant flow.

D. Proactive Rule Placement

The proposed methodology involves pre-emptively placing rules before the arrival of new flows. It is based on predicting the usage of flow rules before their insertion into the flow table, as illustrated in Fig. 3. This approach can effectively reduce delays in setting up flow rules and minimize the total number of signaling messages.

Proactive flow rule placement is often integrated with access control, with ILP being utilized in [42] to design and analyze the dependency graph for rule placement in the switch. The graph is also helpful for predicting access control rules before the corresponding flow arrives [43–44]. DevoFlow [22] further reduces switch-controller interaction by detecting significant flows ahead of time. The switch only places flow rules associated with mice flow, without invoking the controller.

Figure 3 serves as an example of proactive flow rule placement, where flow rules are installed by the controller (Arrow 1), matched by the contents of a flow table (Arrow 2), and quickly forwarded without setup delays (Arrow 3).

In recent times, there has been an increased consideration of user mobility in managing SDN flow table due to the growing popularity of IoT. A flow rule placement scheme was proposed in [45] which improves on the existing method by taking into account the occupation time of a flow rule in a table while ensuring the processing of the flow with the local switch. In [46], a dynamic environment allows users to join and exit the network freely. Here, an online flow-based routing approach is presented, which permits the dynamic reconfiguration of the existing flows and the adaptation of the link rate, considering the user's demand and mobility. [47] presents a software-defined IoT (SD-IoT) architecture for intelligent urban sensing, which customizes data acquisition, transmission, and processing for the user through well-defined APIs. Various applications coexist in the common infrastructure to reduce overall maintenance costs. [48] formulated an SDN-assisted system using MapReduce to support big data processing in wireless sensor networks (WSN). It allows for the dynamic loading and implementation of user-specified maps and reduces energy consumption in the nodes. In [49], a mobility-aware adaptive flow rule placement, Mobi-Flow, is presented as a solution to the issue of user mobility. It consists of two components: the path estimator and the flow manager. The path estimator predicts the future position of the user in the network, while the flow manager determines the activation of access points and places the corresponding flow rules according to the result of the path estimator.

Next, the proposed MAPQ scheme is presented, further enhancing the Mobi-Flow approach using cost-sensitive AdaBoost and Q-learning. Frequent use of reactive flow rule placement is not beneficial for mice flow. The proposed MAPQ scheme adopts proactive flow rule placement to resolve the issue.

This section presents the proposed MAPQ scheme for mobility-aware flow rule placement with SDN. Then its function is analyzed, providing more space for high-level flow rules and assuring the efficiency of the flow table management.

3.1 Overview

As depicted in Fig. 4, the operation of MAPQ involves a tripartite process comprising flow rule classification, position prediction, and flow entry update. The first step of flow rule classification utilizes the cost-sensitive AdaBoost methodology to categorize the flow rules into two distinct types. The accuracy of this particular classification process holds substantial sway over the performance of the MAPQ scheme, wherein mice or inactive flow are excluded from consideration. The proactive rule placement policy allocates the remaining flows to the switches. The Q-learning model is leveraged to prognosticate the node's position relative to the corresponding flow entry, thereby facilitating the scheduling of rule placement in advance. Subsequently, the controller updates the flow entries contingent upon the node position. The notations employed in this study are summarized in Table II.

Table 2. The list of notations.

Parameter	Description
l_i	ith position of the device in a sequence of positions
t_i	arrival time of the device corresponding to ith position
w_ij	weight of the visit from l_i to l_j
h_i	ith basic classifier
P_ij	transition probability from l_i to l_j, i ≠ j
α	learning rate
γ	discount factor

3.2 Flow Classification

Flow classification is the first step of MAPQ, which classifies the flows into elephant flow or mice flow. The traditional machine learning algorithm uses a training dataset, (X^tra,Y^tra)={(x₁,y₁),(x₂,y₂),…,(x_n,y_n)}, to generate a classifier. Then it is employed to identify the category of the test dataset, (X^tes,Y^tes). With AdaBoost, a trained classifier is used to differentiate two kinds of flows; elephant (active) flow and mice (inactive) flow. Usually, the distribution of the training dataset is the same as that of the test dataset. As for the training dataset of a different distribution, it is regarded as a redundant dataset. Unlike traditional machine learning algorithms, the cost-sensitive algorithm considers the minimization of the misclassification cost to construct a full-scale classifier.

Based on the cost-sensitive scheme framework, the improved AdaBoost algorithm, known as cost-sensitive AdaBoost, has been utilized in the context of MAPQ. AdaBoost is a conventional machine learning algorithm that comprises multiple weak classifiers. These classifiers are assigned weights based on the classification error, and a robust classifier is produced by aggregating the weak classifiers utilizing these weights. The operational sequence is outlined as follows.

1) Initialize the weight of each training data to be the same as 1/m, where m is the total number of the training data. Thus, the initial weights are

$${w}_{1}={w}_{2}=\dots ={w}_{m}=\frac{1}{m}$$

2) Update the weights of the training data with n iterations.

A. Choose a weak classifier of the minimal error rate as the tth basic classifier, h_t, and calculate its error rate.

$${\epsilon }_{t}={Pr}_{i\sim{D}_{t}}\left[{h}_{t}\left({x}_{i}\right)\ne {y}_{i}\right]$$

$$=\sum _{i=1}^{m}{w}_{i}^{t}[1-I({h}_{t}\left({x}_{i}\right)={y}_{i}\left)\right]$$

where w_i^t denotes the weight for the tth iteration and I(•) is the indicator function.

$$I\left(y=x\right)=\left\{\begin{array}{c}1, y=x\\ 0,y\ne x\end{array}\right.$$

B. Calculate the weight of the basic classifier for the final ensemble.

$${\alpha }_{t}=\frac{1}{2}ln\left(\frac{1-{\epsilon }_{t}}{{\epsilon }_{t}}\right)$$

C. Update the weight of each training data.

$${w}_{(t+1)}\left(i\right)=\frac{{w}_{t}\left(i\right)exp\left(-{\alpha }_{t}{y}_{i}{h}_{t}\left({x}_{i}\right)\right)}{{Z}_{t}}$$

where Z_t is the normalization factor.

$${Z}_{t}=2\sqrt{{\epsilon }_{t}\left(1-{\epsilon }_{t}\right)}$$

3) Combine the basic classifiers using the following equation into a strong classifier.

$$H\left(x\right)=sign\left(\sum _{t=1}^{T}{\alpha }_{t}{h}_{t}\left(x\right)\right)$$

The overall steps are summarized in Procedure 1.

Procedure 1: AdaBoost

Input: (X^tra,Y^tra)={(x₁,y₁),(x₂,y₂),…,(x_n,y_n)} where x_i∈X,y_i∈{−1,1}.

Initialize:${w}_{1}={w}_{2}=\dots ={w}_{m}=\frac{1}{m}$

1. FOR t = 1 to n

2. Train weak learners using the current weight distribution, D_t

3. Obtain weak hypothesis h_t: X→{−1,1}.

4. Select h_t of smallest weighted error:

${\epsilon }_{t}={Pr}_{i\sim{D}_{t}}\left[{h}_{t}\left({x}_{i}\right)\ne {y}_{i}\right].$5.

${\alpha }_{t}=\frac{1}{2}ln\left(\frac{1-{\epsilon }_{t}}{{\epsilon }_{t}}\right)$.

6. Update for i = 1,…,m:

${w}_{(t+1)}\left(i\right)=\frac{{w}_{t}\left(i\right)exp\left(-{\alpha }_{t}{y}_{i}{h}_{t}\left({x}_{i}\right)\right)}{{Z}_{t}}$

where Z_t is a normalization factor used to make w_(t+1) a weight.

7. Compute the final hypothesis:

$H\left(x\right)=sign\left(\sum _{t=1}^{T}{\alpha }_{t}{h}_{t}\left(x\right)\right)$

Output: H(x)

AdaBoost is similar to traditional machine learning algorithms in adopting the hypothesis that the training and test dataset distributions are the same. The cost-sensitive AdaBoost preserves AdaBoost for the training dataset of the same distribution, and a cost-sensitive function is applied to the training dataset of different distributions for flow classification. To differentiate the importance of the data, each data is associated with a cost: the higher the cost, the higher the importance of correctly identifying the data. Let {(x₁,y₁,c₁),…, (x_m,y_m,c_m)} be a sequence of training dataset, where c_i∈[0,+∞) is an associated cost. c(y_i,y_j) is the cost of predicting class y_j when the true class is y_i for a sample. Its value is decided below.

$$c\left({y}_{i},{y}_{j}\right)=\left\{\begin{array}{c}\frac{p}{N}\times acc {y}_{i}\ne {y}_{j}{ and y}_{i}=-1\\ \frac{1-p}{N}\times \left(1-acc\right) {y}_{i}\ne {y}_{j}{ and y}_{i}=1 \\ 0 {y}_{i}={y}_{j} \end{array}\right.$$

where p is the number of negative samples. c(0,0) is the cost of correctly predicting negative value for actual negative value, Cost_TN. Four combinations exist regarding the correctness of the positive/negative value prediction as listed in Table III.

Table 3. The combinations of prediction of the values.

Prediction	Actual Negative	Actual Positive
Negative	Cost_TN:c(0,0)	Cost_FN:c(1,0)
Positive	Cost_FP:c(0,1)	Cost_TP:c(1,1)

The update of the weight is done as follows.

$${w}_{(t+1)}\left(i\right)=\frac{{w}_{t}\left(i\right)\text{e}\text{x}\text{p}(-{\alpha }_{t}{y}_{i}{h}_{t}\left({x}_{i}\right)c\left({y}_{i}, {h}_{t}({x}_{i}\right)))}{{Z}_{t}}$$

The final ensemble formula is

$$H\left(x\right)=sign\left[\sum _{y\in \left\{\text{0,1}\right\}}L(x,y)(\sum _{\tau :h\left(x\right)=y}{a}_{\tau }{h}_{\tau }\left(x\right))\right]$$

where L(x,y) is the real loss of class_i. Here class_I is the actual class of x.

$$L\left(x,{\text{y}}_{i}\right)={\sum }_{j}p\left(j|x\right)c\left({y}_{i},{y}_{j}\right)$$

As the cost factor is involved in the proposed AdaBoost scheme, it can be regarded as a cost-sensitive boosting algorithm, summarized in Procedure 2. For the AdaBoost algorithm, selecting a suitable value for the weight update parameter is essential in transforming a weak classifier into a strong one. When the cost parameters are added to the weight updating formula of AdaBoost, the data distribution is affected by the parameter. The efficiency of AdaBoost is not guaranteed if the cost is considered without proper weight updates. The value of the weight update parameter needs to be decided to minimize the overall training error of the combined classifier. The process of flow classification is shown in Fig. 5.

Procedure 2: Cost-Sensitive AdaBoost

Input: (X^tra,Y^tra)={(x₁,y₁),(x₂,y₂),…,(x_n,y_n)} where x_i∈X,y_i∈{−1,1}.

Initialize:${w}_{1}={w}_{2}=\dots ={w}_{m}=\frac{1}{m}$

1. FOR t = 1 to n

2. Train weak learners using the current weight distribution, D_t.

3. Get weak hypothesis h_t: X→{−1,1}.

4. Select h_t of the lowest weighted error:

${\epsilon }_{t}={Pr}_{i\sim{D}_{t}}\left[{h}_{t}\left({x}_{i}\right)\ne {y}_{i}\right].$

5. Choose

${\alpha }_{t}=\frac{1}{2}ln\left(\frac{1-{\epsilon }_{t}}{{\epsilon }_{t}}\right)$.

6. Update for i = 1,…,m:

${w}_{(t+1)}\left(i\right)=\frac{{w}_{t}\left(i\right)\text{e}\text{x}\text{p}(-{\alpha }_{t}{y}_{i}{h}_{t}\left({x}_{i}\right)c\left({y}_{i}, {h}_{t}({x}_{i}\right)))}{{Z}_{t}}$

Where Z_t is a normalization factor used to make w_(t+1) a weight.

7. Compute the final hypothesis:

$H\left(x\right)=sign\left[\sum _{y\in \left\{\text{0,1}\right\}}L\left(x,y\right)(\sum _{\tau :h\left(x\right)=y}{a}_{\tau }{h}_{\tau }\left(x\right))\right]$

Output: H(x)

3.3 Prediction of Location

The QL algorithm is used to predict the future position of the end device. The QL model consists of 5-tuple < L,A,R,P,γ>, where L is a finite set of states, and A is a finite set of actions. The controller obtains feedback information which R denotes, and P is the transition probability between the states. The feedback for each end device is the same, and the discount factor γ is applied to the calculation of the reward value. First, two tuples, <H, P>, need to be elaborated.

H: A set of data showing the movement history of an end-device with three tuples, <L,T,S>, where L={l₁, l₂,…,l_n} is the location visited by the device, T={t₁,t₂,…,t_n} is the arrival time, and S={s₁,s₂,…,s_n} is the duration of stay at each location. n is the number of positions visited.
P: Transition probability from one location to another as P_ij the transition probability from l_i to l_j, i ≠ j.

The QL algorithm is typically composed of a series of actions carried out by agents and states, as well as a reward function and a transfer function. In the proposed QL model, the controller and SDN are considered the agent and environment, respectively, in which the agent operates. The agent initially assesses the state of the SDN and determines the appropriate course of action to interact with it. Once an action is taken, a reward is obtained and the environment transitions to a new state. Through ongoing interactions with the environment, the agent observes a sequence of state-action-reward events and assesses the reward in conjunction with the corresponding state and action pair. The ultimate objective is to obtain the expected maximum cumulative discounted reward, which the agent seeks to achieve by calculating the optimal action-value function.

The QL algorithm is a set of actions executed by agents and states, along with reward and transfer functions. Within the proposed QL model, the controller and SDN are identified as the agent and environment in which the agent conducts operations. At the outset, the agent evaluates the condition of the SDN and determines the most appropriate course of action for engaging with it. A reward is obtained upon taking action, and the environment moves to a new state. Through continual interactions with the environment, the agent observes a sequence of state-action-reward occurrences and evaluates the reward in conjunction with the corresponding state and action pair. The primary objective is attaining the expected maximum cumulative discounted reward, which the agent accomplishes by computing the optimal action-value function.

$$Q\left(s,a\right)=\left(1-\alpha \right)Q\left(s,a\right)+\alpha \left[R\left(s,a\right)+\gamma max{Q}^{{\prime }}\left({s}^{{\prime }},{a}^{{\prime }}\right)- Q(s,a)\right]$$

where Q(•) is the expected reward, and α is the learning rate set to be the same value as QLAODV [48]. γ (> 1) is the discount factor, determining the importance of the reward. In addition, Q${\prime }$(s${\prime }$, a${\prime }$) is the maximum reward given the new state, s${\prime }$, and all possible actions at s${\prime }$. The proposed scheme defines the state, action, and reward as follows.

Definition 7 (State). The state is deemed the location that the end device has formerly accessed, with each position signifying a semantic locale such as the office, home, station, etc. As previously mentioned, L denotes this.

Definition 8 (Action). An action possessing a particular state can be delineated as the duration during which the device remains stationary in its present position (state). This duration can be calibrated in seconds, ranging from one to ten seconds, and is therefore classified as an action space, A, comprising of the values {1, 2,...,10}. This is because the device's state is assessed at intervals of ten seconds. It should be noted that a decrease in the number of actions enhances the precision of prediction since the number of predicted actions is curtailed.

Definition 9 (Reward). The reward is given for a transition between two states with an action. The reward with s_i, R_i, is represented as follows.

$${R}_{i}=\sum _{j=1}^{n}{w}_{ij}{P}_{ij}$$

where w_ij is the probability of a visit to l_j from l_i.

$${w}_{ij}=P\left({l}_{t+1}={l}_{j}|L\right)=\frac{N(a{l}_{j},L)}{N(a,L)}$$

Here a = l₁,…,l_i is a sequence of i recent positions visited and N(•, L) is the number of occurrences of one sequence of the visited locations, L.

In Software-Defined Networking (SDN), the controller regularly verifies the location of the end device and its connectivity to a switch every 10 seconds. The OpenFlow protocol facilitates the collection of network information, thereby resulting in no additional burden in executing the suggested approach for acquiring network information.

Prediction operation

For predicting the following location, the Q-values are evaluated. Here the input is the current location, l_i, and the output is a vector, Q(l_i)=(Q(l_i,a₁), Q(l_i,a₂),…,Q(l_i,a₁₀)), consisting of 10 estimated Q-values for each possible action. The following action from li, ai, is chosen from the vector.

$$\begin{array}{c}{a}_{i}=\text{arg}\text{max }Q({l}_{i},{a}_{j})\\ j\text{ϵ}[1,\dots ,10]\end{array}$$

Given the possibility of encountering a local optimum, the decision-making process employs the ԑ-greedy policy to determine the subsequent action. Specifically, an action is randomly and uniformly selected with a small probability of ԑ, while the best action is chosen from the Q-table based on Eq. (15) with a probability of (1-ԑ). The controller selects the action with the highest value with a probability of (1-ԑ), and a random action with a probability of ԑ, thereby facilitating exploration of the environment with a low probability. It is noteworthy that the parameter, ԑ, is not constant, but rather varies based on the operational status of the scheme. Specifically, ԑ is set to 1/c, where c denotes the number of successful predictions. The value of ԑ decreases gradually as the frequency of successful predictions increases, thereby promoting exploration during the initial stages of training, while the degree of exploitation in the later stages is contingent on experience.

Update operation: The model gets < L, T, S > of the end device delivered by the switch. Then the state of the end device is updated, and the corresponding reward for the action is computed. Lastly, the last state, current state, previous action, and its corresponding reward are logged in the training dataset for future use.

Train: The data are periodically sampled from the training dataset to perform experience replay, which utilizes previous data to remove the samples' associations. With the input consisting of past states, the Q-value of the pair of (s,a) is updated with the previous pair using the Bellman equation, while other Q-values are unchanged. The updated results are stored as the output. The key to using the Bellman equation is that if the Q-value of all the subsequent states is known, the optimal strategy is the action maximizing the expected value of R(s, a) + γ×max (Q (s,a) − Q(s, a)).

The proposed QL algorithm for reinforcement learning exhibits a significant deviation from other such algorithms in that the subsequent state and corresponding reward are not immediately procured upon the selection of a specific action. Rather, they are obtained post the relocation of the end device by the controller. This implies that the parameters of QL do not undergo any alteration until the end device is relocated. Moreover, the choice of the next action is not contingent on the current action. The process of predicting the location of a device is explicated in the following manner.

Procedure 3: Prediction of location

Input: H and current location, l_i

Initialize: the number of predictions, t = 0.

1. Decide optimal action according to the action function as shown in Eq. (15);

2. Take action based on the ԑ-greedy policy;

3. Determine the next state, l_j;

4. Compute reward

${R}_{i}=\sum _{j=1}^{n}{w}_{ij}{P}_{ij}$

5. Update discrete Q-value

$Q\left(s,a\right)=\left(1-\alpha \right)*Q\left(s,a\right)+\alpha \left[R\left(s,a\right)+\gamma *max{Q}^{{\prime }}\left({s}^{{\prime }},{a}^{{\prime }}\right)-Q(s,a)\right]$

6. Formulate continuous actions through ԑ-greedy policy

7. t = t + 1;

8. End while

Output: Sequence of predicted locations

3.4 Rule Placement

In an SDN-based IoT environment, the network comprises numerous end devices that are connected through distinct switches. These switches maintain flow tables that require consistent updates to optimize the hit ratio, considering the movement of the end devices. The proposed MAPQ scheme employs a proactive flow rule placement approach to update the flow table. Therefore, it is imperative to update the future location of the end device based on its past locations. Subsequently, the flow rule linked to the end device is positioned in the flow table of the corresponding switch.

In this section, a computer simulation is performed to assess the proposed approach and contrast it with the existing methods, namely the standard hard time-out scheme and Mobi-Flow scheme [49] which employs a location prediction mechanism in conjunction with an adapted time-out.

4.1 Simulation Setting

The simulation is implemented on Intel Core i7 process, 2.50GHz PC with 16GB RAM, and Matlab R2015a. The prediction accuracy of a scheme, Acc, indicates the rate of correct prediction of the flow entry having the most negligible matching probability. When the hard time-out is up, the predicted flow entry is checked whether it has the smallest matching number. Then the number of successful predictions, n_s, is incremented by 1. After the entire operation is over, the accuracy is calculated as:

$$Acc=\frac{{n}_{s}}{{n}_{e}}$$

where n_e is the number of evicted flow entries. Table IV lists the simulation parameters. The flows are generated according to an exponential distribution with λ = 1, and the contents of the flow entries are decided randomly.

Table 4. The simulation parameters.

Parameter	Value
Number of switches	3
Number of flow entries per table	100 ~ 200
Upper bound on controller-to-switch delay	3.043ms
Upper bound on end-to-end delay	0.341ms
Average packet arrival rate per switch	0.005 ~ 0.025mpps (million packets per second)
Average packet service rate per switch	0.30mpps
Flow-table lookup time	33.333µsec

A virtual network of tree topology of 25 switches and 200 hosts of Fig. 6 is constructed using NS3 with the following elements: Open vSwitch, open daylight controller, and end-host nodes with the service provider (SP) enabled. The host nodes, the switch, and the controller are running on Ubuntu 16.04 LTS, and all the switches are connected to a controller.

4.2 Experiment Results

Suppose that n and m denote the number of incoming flows and flow entries of a flow table, respectively. The performance metrics adopted in the simulation are Precision, Recall, F1-score, and Garbage rate (Grate). The number of data correctly classified as positive while they are positive is defined as true positive (tp). The number of data mistakenly classified as a positive class for a negative one is defined as false positives (fp). tn and fn are defined similarly. Precision, obtained by Eq. (17), is the ratio of True Positives to all the Positives. Recall, defined as Eq. (18), represents the portion of correctly classified positive data from the positive dataset. F1_score of Eq. (19) is a parameter obtained by the harmonic mean of precision and recall. fp and fn deteriorate the performance of a classifier. Grate, defined as Eq. (20), is the portion of false classification out of entire classifications used to show the overall inaccuracy.

Precision = TP/(TP + FP) (17)

Recall = TP/(TP + FN) (18)

F1-score = 2•Precision•Recall/(Precision + Recall) (19)

Grate= (FP + FN)/(TP + TN + FP + FN) (20)

Table V compares the Precision and Grate of the classification of the proposed cost-sensitive AdaBoost approach and that of the traditional AdaBoost. Our analysis reveals that the cost-sensitive AdaBoost consistently outperforms AdaBoost in Precision and Grate. AdaBoost cannot filter bursty error readings, resulting in lower precision when error cost is not considered. Our simulation results demonstrate that incorporating error cost significantly enhances classification precision.

Table 5. The comparison of the metrics.

Method	Recall	Precision	F1-score	Grate
Adaboost	0.74	0.91	0.81	0.05
Cost-sensitive Adaboost	0.56	0.95	0.71	0.03

The comparison of table misses for the various schemes is presented in Fig. 7, with (8000,100) for (n,m). T_I and T_H are 18s and 30s, respectively. Notably, the number of table misses exhibits an increase when n is increased from 8000 to 16000 while m remains unchanged, as evidenced in the middle of the figure. However, it is worth mentioning that the number of table misses associated with the proposed scheme is considerably lower than that of other schemes. Furthermore, upon increasing the number of flow entries to 200, there is a significant reduction in table misses. It is noteworthy that the proposed MAPQ scheme substantially reduces the number of table misses compared to the other schemes, owing to its ability to predict the locations of the end devices with great accuracy.

The impact of T_H and T_I on the frequency of table misses is analyzed in Fig. 8, utilizing a sample size of 300 matches and employing the (8000,100) configuration. The data depicted in the figures reveal that the proposed scheme demonstrates a significant superiority over alternative schemes, irrespective of the values assigned to T_H and T_I.

Subsequently, to assess the efficacy of the flow table, an analysis is conducted on the number of obsolete flow rules and the volume of packets dispatched to the controller. It is worth noting that the (n,m) values employed correspond to those illustrated in Fig. 7, whereby T_I and T_H are 18s and 30s, respectively. Remarkably, the proposed approach consistently outperforms other methods. Specifically, when utilizing (16000,100), the number of inactive flow rules is inferior to that of (8000,100), given that the former facilitates support for additional flows with the same flow table size. However, when the number of flow entries is raised to 200, the number escalates due to an increase in unmatched flow entries. As illustrated in Fig. 9, the proposed mechanism consistently results in a notably reduced number of packets transmitted to the controller.

Figure 10 presents a comparison of the table occupancy rates among various schemes. This rate is the ratio of occupied flow entries to the total number of entries in the flow table. Notably, the table occupancy rate decreases when n increases from 8000 to 16000 while m remains constant. This phenomenon can be attributed to the utilization of more flow entries with an increase in incoming flows. However, the proposed scheme exhibits a significantly lower rate compared to other schemes, owing to a reduction in the number of idle flow entries. Notably, when the number of flow entries increases to 200, the rate also increases, albeit being smaller than that of (8000,100), since more unmatched flow entries exist. Note that the proposed scheme shows a much lower occupancy rate than the other schemes, regardless of the operational condition, due to efficient flow classification and hybrid flow rule placement with the flow table.

Table VI compares the prediction accuracy of the MAPQ scheme and Mobi-Flow. The (n,m) values used for the comparison are (8000,100) and T_I is set at 30 seconds. It is important to note that Mobi-Flow employs an order-k Markov predictor, whereas MAPQ utilizes Q-learning for flow prediction. The table demonstrates that the proposed scheme consistently achieves greater accuracy than Mobi-Flow across all tested scenarios.

Table 6 The comparison of prediction accuracy.

Method	1	2	3
MAPQ	0.815	0.832	0.809
Mobi-Flow	0.684	0.732	0.726

Finally, the performance of the schemes is investigated in terms of average transmission delay and the ratio of packet drop in the switch with 300 matchings (8000,100). The comparison summary is presented in Table VII. The findings indicate that the proposed scheme is superior as it accurately predicts the end-device location, leading to reduced table misses. This reduction is achieved by minimizing the unnecessary flow entry placement, which enables the overhead of flow setup. As a result, the proposed scheme necessitates the smallest delay and packet drop rate, which, in turn, ensures the maintenance of free space in the SDN buffers (such as the datapath buffer that retains unmatched packets) while keeping the current connections undisturbed.

Table 7. The comparison of average delay and rate of packet drop.

Scheme	MAPQ	Mobi-Flow	Table Miss
Average delay(ms)	3.29	4.21	7.35
Rate of packet drop	0.389	0.602	0.764

This paper introduces an innovative flow rule placement approach that is compatible with Software-Defined Networking (SDN) and that is designed to support Internet of Things (IoT) applications. The proposed scheme employs the Q-learning algorithm to predict the location of the end device, enabling the proactive placement of flow entries. The controller exploits the forecast outcome to determine and adjust the flow rules accordingly. Moreover, the cost-sensitive AdaBoost algorithm is utilized to filter out small or infrequently matched flows. A computer simulation is conducted to demonstrate that the proposed scheme provides significant enhancements in match probability and the number of table misses in contrast to existing schemes.

Future work will refine the proposed scheme by incorporating additional factors that affect prediction accuracy, such as idle time-out. The paper focuses on the proactive placement of flow rules based on predicted end-device locations and plans to explore a hybrid flow rule placement scheme to capture the dynamic nature of networking conditions more effectively. An analytical model will also be developed to achieve optimal designs for specific conditions by establishing relationships between the factors considered and the desired performance criteria.

Acknowledgments

Thanks to Dr. Hanyao Huang of Nanyang Institute of Technology for his help in our work.

Authors’ contributions

Methodology: Gan Huang; Resources: Hanyao Huang and Ihsan Ullah; Software: Ihsan Ullah; Supervision: Kyung Tae Kim; Writing original draft: Gan Huang; Writing review editing Hanyao Huang, Kyung Tae Kim; All authors read and approved the final manuscript.

Authors’ information

Gan Huang received the B.S. degree in Electronic and Information Engineering from Chuzhou University, China, in 2012, the M.S. degree in Computer Science from Anhui Polytechnic University, China, in 2016, and the Ph.D. degree in Computer Engineering from Sungkyunkwan University, South Korea, in 2021.

From 2021 to 2022, he was a postdoctoral researcher at Sabanci University, Turkey. He is currently a lecturer with the School of Mathematics and Computer Science, Zhejiang A&F University, China. His research interests include Software-defined networking (SDN), Quality of Experience (QoE), Ubiquitous and Distributed Computing, Cloud and Edge Computing, Mobile Computing, Wireless Sensor Networks, IoT, Machine Learning, Computer Networking, and Network Security.

Ihsan Ullah received the B.S. and M.S. degrees in computer science from the University of Peshawar, Pakistan, in 2001 and 2004, respectively, and the Ph.D. degree in computer engineering from Sungkyunkwan University, Suwon, South Korea, in 2019. From September 2019 to August 2020, he was a Postdoctoral Research Fellow with the Ubiquitous Computing Technology Research Institute (UTRI), Sungkyunkwan University.

Since 2020, he has been a Research Professor with the School of Computer Science and Engineering, Korea University of Technology and Education, Cheonan, South Korea. His research interests include data aggregation, data fusion, virtual network embedding, network slicing (5G), the IoT (Internet of Things), artificial intelligence, machine learning, cloud computing, and wireless sensor networks.

Hanyao Huang received the B.S. degree from the University of Electronic Science and Technology of China, Chengdu, China, in 2015, and the Ph.D. degree from Sungkyunkwan University, Korea, in 2022.

He is a lecturer with the Department of Computer and Software Engineering, Nanyang Institute of Technology, Nanyang, China. His current research interests include Wireless Networks, Internet of Things Technology, and Machine Learning.

Kyung Tae Kim received the Ph.D. degree from College of Information and Communication Engineering at Sungkyunkwan University, Korea in 2013. He is currently a research professor at the College of Computing and Informatics at Sungkyunkwan University, Korea. His current research interests include Edge computing, Artificial Intelligence Systems, and Internet of Things technology.

Funding

This research was supported by Basic Science Research Programs through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2022R1I1A1A01053800).

Availability of data and materials

The data used to support the findings of this study are available from the corresponding author upon request.

Ethics approval and consent to participate

No ethical approval is required, and the authors express their consent to participate in the paper.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Bannour S, Souihi, Mellouk A, “Distributed SDN, Control (2018) Survey, Taxonomy, and Challenges. Ieee Commun Surv Tutorials 20(1):333–354
Basit S, Qaisar SH, Rasool, Ali M (2017) SDN Orchestration for Next Generation Inter-Networking: A Multipath Forwarding Approach. Ieee Access 5:13077–13089
Adami B, Martini A, Sgambelluri L, Donatini M, Gharbaoui P, Castoldi, Giordano S (2017) “An SDN orchestrator for cloud data center: System design and experimental evaluation,” Transactions on Emerging Telecommunications Technologies, vol. 28, no. 11, Nov,
Aguado V, Lopez J, Marhuenda OG, de Dios, Fernandez-Palacios JP (2015) “ABNO: A Feasible SDN Approach for Multivendor IP and Optical Networks,” J Opt Commun Netw, vol. 7, no. 2, pp. A356-A362, Feb 1,
Bastam M, Sabaei M, Yousefpour R (2018) “A scalable traffic engineering technique in an SDN-based data center network,” Transactions on Emerging Telecommunications Technologies, vol. 29, no. 2, Feb,
Abdelmoniem M, Bensaou B, Abu AJ (2018) “Mitigating incast-TCP congestion in data centers with SDN,” Annals of Telecommunications, vol. 73, no. 3–4, pp. 263–277, Apr,
Chekired L, Khoukhi, Mouftah HT (2018) “Decentralized Cloud-SDN Architecture in Smart Grid: A Dynamic Pricing Model,” Ieee Transactions on Industrial Informatics, vol. 14, no. 3, pp. 1220–1231, Mar,
Chen J, Chen JB, Ling JC, Zhou JL, Zhang W (2018) “Link Failure Recovery in SDN: High Efficiency, Strong Scalability and Wide Applicability,” J Circuits Syst Computers, vol. 27, no. 6, Jun 15,
Chen N, Wang M, Zhang N, Shen XM, Zhao DM (2017) “SDN-Based Framework for the PEV Integrated Smart Grid,” Ieee Network, vol. 31, no. 2, pp. 14–21, Mar-Apr,
Al-Rubaye S, Aulin J (2017) “Grid Modernization Enabled by Sdn Controllers: Leveraging Interoperability for Accessing Unlicensed Band,” Ieee Wireless Communications, vol. 24, no. 5, pp. 60–67, Oct,
Awad MK, El-Shafei M, Dimitriou T, Rafique Y, Baidas M, Alhusaini A (2017) “Power-efficient routing for SDN with discrete link rates and size-limited flow tables: A tree-based particle swarm optimization approach,” International Journal of Network Management, vol. 27, no. 5, Sep-Oct,
Kim Y, Choi SI, Lee, Kim HJ (2017) Enhanced Flow Table Management Scheme With an LRU-Based Caching Algorithm for SDN. Ieee Access 5:25555–25564
Metter M, Seufert F, Wamser T, Zinner, Tran-Gia P (2017) “Analytical Model for SDN Signaling Traffic and Flow Table Occupancy and Its Application for Various Types of Traffic,” Ieee Transactions on Network and Service Management, vol. 14, no. 3, pp. 603–615, Sep,
Qiu XF, Zhang K, Ren QZ (2017) Global Flow Table: A convincing mechanism for security operations in SDN. Comput Netw 120:56–70 Jun 19
Luo SX, Yu HF, Li LM “Practical flow table aggregation in SDN,” Comput Netw, vol. 92, pp. 72–88, Dec 9, 2015.
Dargahi T, Caponi A, Ambrosin M, Bianchi G, Conti M (2017) A Survey on the Security of Stateful SDN Data Planes. Ieee Commun Surv Tutorials 19(3):1701–1725
Wei YK, Zhang XN, Xie L, Leng SP (2016) “Energy-Aware Traffic Engineering in Hybrid SDN/IP Backbone Networks,” Journal of Communications and Networks, vol. 18, no. 4, pp. 559–566, Aug,
Qi QL, Wang WD, Gong XY, Que XR (2017) Rules Placement with Delay Guarantee in Combined SDN Forwarding Element. KSII Trans Internet Inf Syst 11(6):2870–2888
Nguyen X-N, Saucez D, Barakat C, Turletti T (2016) Rules placement problem in openflow networks: a survey. IEEE Commun Surv Tutorials 18(2):1273–1286
Cheng YH, Cao G, Wang XS, Pan J (2013) “Weighted Multi-source TrAdaBoost,” Chinese Journal of Electronics, vol. 22, no. 3, pp. 505–510, Jul,
Jordehi R (2015) Enhanced leader PSO (ELPSO): a new PSO variant for solving global optimisation problems. Appl Soft Comput 26:401–417
Curtis R, Mogul JC, Tourrilhes J, Yalagandula P, Sharma P, Banerjee S "DevoFlow: scaling flow management for high-performance networks." pp. 254–265
Estan C, George Varghese (2003) New directions in traffic measurement and accounting: Focusing on the elephants, ignoring the mice. ACM Trans Comput Syst (TOCS) 21(3):270–313
Gude N, Koponen T, Pettit J, Pfaff B, Casado M, McKeown N, Shenker S (2008) NOX: towards an operating system for networks. ACM SIGCOMM Computer Communication Review 38(3):105–110
Zhang L, Lin R, Xu S, Wang S "AHTM: Achieving efficient flow table utilization in software defined networks." pp. 1897–1902
Zhang L, Wang S, Xu S, Lin R, Yu H "TimeoutX: An adaptive flow table management method in software defined networks." pp. 1–6
Vishnoi R, Poddar V, Mann, Bhattacharya S "Effective switch memory management in OpenFlow networks." pp. 177–188
Kim T, Lee K, Lee J, Park S, Kim Y-H, Lee B (2014) A dynamic time-out control algorithm in software defined networks. Int J Future Comput Communication 3(5):331
Yu M, Rexford J, Freedman MJ, Wang J (2010) Scalable flow-based networking with DIFANE. ACM SIGCOMM Computer Communication Review 40(4):351–362
Challa R, Lee Y, Choo H "Intelligent eviction strategy for efficient flow table management in OpenFlow Switches." pp. 312–318
Kannan K, Banerjee S (2020) "Flowmaster: Early eviction of dead flow on sdn switches." pp. 484–498. Huang, Gan, and Hee Yong Youn. " Management of Flow Table of SDN for Proactive Eviction Using Fuzzy Logic " Frontiers of Computer Science 14, no. 4 : 1–10
Huang G and Hee Yong Youn. "Proactive eviction of flow entry for SDN based on hidden Markov model." J Inform Sci Eng.
Draves RP, King C, Venkatachary S, Zill BD "Constructing optimal IP routing tables." pp. 88–97
Liu Y, Zhao X, Nam K, Wang L, Zhang B "Incremental forwarding table aggregation." pp. 1–6
Kanizo Y, Hay D, Keslassy I "Palette: Distributing tables in software-defined networks." pp. 545–549
Braun W, Menth M "Wildcard compression of inter-domain routing tables for OpenFlow-based software-defined networking." pp. 25–30
Giroire F, Moulierac J, Truong Khoa P (2014) "Optimizing rule placement in software-defined networks for energy-aware routing." In 2014 IEEE Global Communications Conference, pp. 2523–2529. IEEE,
Markiewicz A, Tran PN, Timm-Giel A (2014) "Energy consumption optimization for software defined networks considering dynamic traffic." In 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet), pp. 155–160. IEEE,
Vawter I, Pan D, Ma W (2014) "Emulation performance study of traffic-aware policy enforcement in software defined networks." In 2014 IEEE 11th International Conference on Mobile Ad Hoc and Sensor Systems, pp. 775–780. IEEE,
Caria M, Jukan A, Hoffmann M (2016) SDN partitioning: A centralized control plane for distributed routing protocols. IEEE Trans Netw Serv Manage 13(3):381–393
Zhang S, Ivancic F, Lumezanu C, Yuan Y, Gupta A, Malik S "An adaptable rule placement for software-defined networks." pp. 88–99
Kang N, Liu Z, Rexford J, Walker D "Optimizing the one big switch abstraction in software-defined networks." pp. 13–24
Katta N, Alipourfard O, Rexford J, Walker D "Infinite cacheflow in software-defined networks." pp. 175–180
Optimized rule placement (2014) for mobile users in SDN-enabled access networks." In 2014 IEEE Global Communications Conference, pp. 4953–4958. IEEE,
Amokrane A, Langar R, Boutaba R, Guy Pujolle (2015) Flow-based management for energy efficient campus networks. IEEE Trans Netw Serv Manage 12(4):565–579
Liu J, Li Y, Chen M, Dong W, Jin D (2015) Software-defined internet of things for smart urban sensing. IEEE Commun Mag 53(9):55–63
Anadiotis, Angelos-Christos G, Morabito G, Palazzo S (2015) An SDN-assisted framework for optimal deployment of MapReduce functions in WSNs. IEEE Trans Mob Comput 15(9):2165–2178
Bera S, Misra S, Mohammad S (2018) Obaidat. "Mobi-Flow: Mobility-aware adaptive flow-rule placement in software-defined access network. IEEE Trans Mob Comput 18(8):1831–1842

Competing interest reported. This research was supported by Basic Science Research Programs through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2022R1I1A1A01053800).

Download PDF

Journal Publication

published 25 Jan, 2024

Read the published version in Journal of Cloud Computing →

Editorial decision: Major revision
05 Sep, 2023
Reviews received at journal
04 Aug, 2023
Reviewers agreed at journal
29 Jul, 2023
Reviewers invited by journal
26 Jul, 2023
Editor assigned by journal
30 Jun, 2023
Submission checks completed at journal
30 Jun, 2023
First submitted to journal
28 Jun, 2023

You are reading this latest preprint version

Predictive Mobility and Cost-Aware Flow Placement in SDN-Based IoT Networks: A Q-Learning Approach

Status:

Journal Publication

Version 1

Abstract

Figures

1 Introduction

2 Related Work

2.1 Terminology

2.2 Related Work

3 The Proposed Scheme

3.1 Overview

3.2 Flow Classification

3.3 Prediction of Location

3.4 Rule Placement

4 Performance Evaluation

4.1 Simulation Setting

4.2 Experiment Results

5 Conclusion

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1