The paradigm Internet of Things (IoT) connects several million devices that can gather information which is stored and processed in the Cloud. This data is analyzed for inferring knowledge and performing predictions and analysis. Generally, in the cloud platform, users are charged based on the resources for storage and computing used. For real-world scheduling applications, these machines are not always available during certain periods of time owing to their stochastic or deterministic causes. The problem of cloud computing scheduling can be as challenging as the Non-deterministic Polynomial (NP) optimization problem, which can be an NP-hard problem. The continuous development in cloud computing and its complexity have made the problem even more challenging. Also, the problem of scheduling optimization is becoming an indispensable topic in academia. In this paper, a new hybrid metaheuristic technique based on Firefly Algorithm (FA), Particle Swarm Optimization (PSO) and Tabu Search (TS) is proposed to enhance task scheduling in resource intensive IoT systems. For the proposed algorithm, there can be a new and complete scheme to handle the task scheduling problems that were designed. To this, the TS algorithm can be incorporated aiming to lookout for the local optimum of every individual. For the purpose of improving solution quality, in every hybrid algorithm step, there has been an effective heuristic that has been proposed. This heuristic can bring down the other overtime costs by means of efficiently using the slack of the operation. Deep Reinforcement Learning (DRL) can be employed to solve problems in resource allocation and time scheduling thus making it easy to handle several tasks and resource heterogeneity. The experimental results demonstrated that the proposed methods (PSO-TS & FA-TS) achieved better performance compared to the other methods.
Research Article
Swarm Optimized Deep Learning Scheduling in Cloud for Resource-intensive Iot Systems
https://doi.org/10.21203/rs.3.rs-2984667/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Cloud computing
NP-complete problem
scheduling techniques
Metaheuristic algorithms
The concept of Internet of things (IoT) is becoming more pertinent in various application as there has been a large growth in mobile devices, data, analytics, cloud computing, ubiquitous communication, and embedded communication. IoT does not contain only a computer network but also includes a network device consisting of different sizes and types, buildings, people, animals, instruments, toys, cameras, home appliances, vehicles, smartphones, and much more. All these are used to communicate and share information using stipulated protocols to achieve reorganization, tracing, positioning, safe control, process control, administration, online upgrade, and personal online monitoring [1].
There are different IoT applications which by employing the cloud and wireless sensor network permitting two-way communication and enabling actuation. An IoT application is basically a wireless system which can gather data from different devices through internet to make effective decisions employing such sensor data. The IoT uses sensing technology, computing technology, and wireless communication technology to connect devices to achieve interaction with the real world. The cloud computing provides flexible computing along with storage services that can be used for supporting a large amount of data processing. So, cloud is efficient in analyzing the data acquired from IoT devices and sensors. There are platform-neutral tools for resolving other key issues like bottlenecks to communication, formats of data interchange, and interoperability that can maximize the future market potential [2]. IoT has been generating data of an unprecedented amount that can impose a major strain on the Internet infrastructure. So, ways to alleviate this pressure are being identified. Cloud computing can help in connecting devices, but it can be different from the IoT in certain ways that can pose a challenge in the future when more data are generated [3].
Cloud computing scheduling represents the procedure of mapping jobs/tasks to resources and systems (such as memory, bandwidth, and CPU time) in an efficient manner. Therefore, the allocation of resources for each of these jobs in an efficient manner has a major role in cloud computing. Based on job dependency, the scheduling is classified as Independent (or Task) and Dependent (or Workflow) Scheduling. The primary objective of workflow scheduling is to automatically process the passing of data and files among the cloud resources while maintain a sequence of connected instructions that is constraints. It is an NP-hard problem, thus, the research tends to focus on the generation of near-optimal or approximate solutions using metaheuristic methods. The multi-objective nature of a scheduling problem can become a challenge in workflows [4].
Reinforcement Learning (RL) refers to a machine learning paradigm. The RL is based on the concept of an agent taking action in any type of setting to increase the belief of cumulative reward. This means, the computational approach in which an agent, who acts in a setting or situation with a certain level of uncertainty, is able to relate to situations and actions thus maximizing their numerical reward value that has been deemed as RL. At the start, the agent will not be aware of the actions to be taken and with the passage of time, will discover the actions that can result in long-term benefits by trying different options again and again. Among the challenging cases, actions can impact immediate reward and in subsequent situations, all future rewards as well. By making a suitable decision that considers all indirect consequences of such actions, a certain level of planning or foresight is required [6]. The RL is opitmized using metaheuristics to improve performance.
Optimization process identifies the best solution to a problem within a finite time. When compared to other traditional optimization or iterative approaches, metaheuristics makes certain suppositions to the problems of optimization and in most cases, can attain good solutions with minimal computation effort. For the metaheuristic algorithm to be efficient, a right equilibrium between exploitation and exploration is required. Based on the search approach, metaheuristics are classified as single-solution-based local search algorithm and population-based search algorithms. A basic single-solution-based metaheuristic like Tabu Search (TS) has better local search ability while population-based metaheuristics like ant colony algorithm FA, PSO have better global search ability [5].
Shadroo et al., [7] had proposed a two-step scheduling using deep learning in IoT. The initial step decides the location of executing the task with clustering. The next step includes the task in accordance with execution location. For the clustering, Self-Organizing Map (SOM), hierarchical SOM and an Autoencoder, a method of deep learning is proposed. There is a comparison made and presented in the article as a method for future extraction in deep learning that can improve clustering and reduce the rate of tasks in fog and cloud thus reducing cost.
Pandiyan et al., [8] had made a proposal of a dynamic management framework for the IoT devices in the cloud (DMFIC) for evaluating and scheduling requests or sensor data to permit the coordination of large data using high resolutions in a manner that was cost-efficient. This was done by anticipating different queues to send a suitable communication to the users. The proposed framework was implemented in a smart home application. The results of the experiment proved that the DMFIC algorithm could efficiently manage cloud sensor data.
Shah and Zhao [9] had proposed a Network Function Virtualization (NFV) method that was applied for accessing various network resources with RL technique used for solving the resource allocation problem. The IoT network traffic makes use of a substrate network that was available by means of the NFV for the transmission of data. This data transmission will require the IoT network to be translated into various virtual requests along with a Service Function Chain (SFC) mapping the substrate network that serves these requests. In the proposed resource allocation based on Deep RL (DRL), a virtual layer is created consisting of network resources will act in the form of a common repository. There are virtual agents that serve the SFCs to interact with the whole environment. They also collectively receive rewards and update policies using learned experiences. This proves the proposed scheme is capable of solving the problem of optimization using suitable reward design, action space, and state. The results of the simulation had demonstrated the multi-agent DRL scheme to have outperformed other reference schemes.
Miao et al., [10] had proposed a Federated Learning (FL)-based Secure data Sharing (FL2S) technique for the IoT. Additionally, for improving the quality of data sharing, there is a DRL is applied to choose members with sufficient capabilities of computation and datasets of high quality. By means of mixing member selection with task decomposition dependable data sharing was realized with all data privacy preserved. The results of the experiment proved the proposed FL2S to achieve better data sharing accuracy for different IoT applications. Harshala S. and Shriram R. [11] had proposed the Firefly algorithm for optimization of the hyperparameters of DRL. The proposed method achieves lesser makespan, better resource utilization, and optimized cost for resource allocation in cloud.
Rjoub et al., [12] had capitalized on the above concept and had presented four deep and RL approaches on learning-based scheduling for automating the scheduling of large-scale workloads on the resources of cloud computing. The RL, Deep Q Networks, DRL with LSTM (DRL-LSTM), and RNN-LSTM. The experiments were conducted on real-world datasets (Google Cloud) and the experiments proved the DRL-LSTM had outperformed all other three methods. These experiments further showed that DRL-LSTM was able to minimize CPU usage and reduced RAM memory usage cost. Harshala et al. [13] presented that Lasso and Ridge’s regression provides an optimal solution for auction-based resource allocation for a multi-dimensional resources in the cloud environment.
Saravanan and Anbalagan [14] proposed the DRL based cloud resource allocation system. The Lion Optimization algorithm was used to improve the accuracy of resource allocation. When compared to the existing systems, the proposed system showed better performance at a lower cost.
Deep learning has accelerated the progress of the RL using deep learning algorithms in the RL and is termed as Deep Reinforcement Learning (DRL). For covering the seminal and other recent developments observed in the DRL, innovative ways that ensure neutral networks to be used for bringing use close to the development of other autonomous agents. To obtain a better and more comprehensive survey of all other recent efforts in the DRL, along with the applications of DRL they are forwarded to areas like natural language processing. The perception of deep learning function is combined with DRL and this is a method of AI that is closer to human thinking therefore regarded as the real AI [15, 16].
Taking an RL setup with a learning entity that interrelates with the whole space taken into consideration, this may be described by the newest of variables wherein S refers to the actual set of states. The set of actions is denoted by A, \({p(s}_{0})\) refers distribution of the initial states, \(r:S\times A⟶R, 0\left({s}_{t+1}\right|{s}_{t,}{a}_{t,})\) will be the transition probabilities and \(\gamma ϵ\left[\text{0,1}\right]\) which will be a discount factor [24].
The mapping of the states to actions: \(\pi :S⟶A\) is guided by the deterministic policy. At the start of each episode, this will be denoted by the initial state which is \({s}_{0}\). For every time step t, the entity will perform an action that is founded on the current state:\({a}_{t}=\pi \left({s}_{t}\right)\). The action executed will get a reward \({r}_{t}=r({s}_{t},{a}_{t,})\), and distribution p(.|\({s}_{t},{a}_{t})\) will help in sampling the new state of the environments. The total return will be: \({R}_{t}={\sum }_{i=T}^{\propto }{\gamma }^{i-t}{r}_{i}\). The goal of the agents was to attempt the maximization of expected return E[\({R}_{t}|{s}_{t},{a}_{t,}\)] with an optimal policy \({\pi }^{*}\) expressed as the policy \({\pi }^{*}\), so that Q\({\pi }^{*}\)(s,a) \(\ge {Q}^{\pi }(s,a)\) for each s \(ϵ S\) ; a \(ϵ A\) and any other policy π. The optimal Q-function, \({Q}^{*}\), is the optimal policy with Q-function which will satisfy the Bellman equation as in (1) [24]:
$${Q}^{*}\left(s,a\right)={E}_{\stackrel{´}{s{\prime } p(.|s,a\left)\right)}}[r\left(s,a\right)+ {\gamma }_{{a}^{{\prime }}ϵ A}^{max}{Q}^{*}\left(s{\prime },a{\prime })\right)]$$
1
Deep Q-Networks (DQN) is expressed in the form of a model-free reinforcement learner, for representing the discrete action spaces. In the case of a DQN, a neural network Q will be kept, and this will approximate \({Q}^{*}\). \({\pi }_{Q}\left(s\right)={argmax}_{aϵA}Q(s,a)\) signifies a new greedy policy w.r.t. Q. A – which will take a random action with the probability \(ϵ\) and action \({\pi }_{Q}\left(s\right)\) with the probability 1-\(ϵ\). The episodes were created at the time of training with the help of greedy policy. The Replay buffer will store transition tuples \({: {(s}_{t},a}_{t},{r}_{t},: {s}_{t+1})\) that were experienced in training. The training of the Neural network will be intertwined with the creation of new episodes. The Loss \(\mathcal{L}\) sampled from a replay buffer is given by \(\mathcal{L}\) =E \({\left(Q\right({{s}_{t},a}_{t})-{y}_{t})}^{2}\)where \({y}_{t}={r}_{t+}{\gamma }_{{a}^{{\prime }}ϵ A}^{max}Q\left({{s}_{t+1},}_{{a}^{\mathcal{L}}}\right)\) and tuples \({: {(s}_{t},a}_{t},{r}_{t},: {s}_{t+1})\).
Changes are slower with target network when compared to the main network used for measuring the targets \({y}_{t}\). These weights of target networks are set as the current weights of its main network. Polyak-averaged parameters may also be utilized.
Two neural networks are termed as an Actor and a Critic. Actor neural network will be a target policy π S\(\to\)A, and the critic neural network will be an action-value function approximator Q: S ×\(A\to\)R. Critic network Q (s, a|\({\theta }^{Q}\)) and Actor-network µ (s|\({\theta }^{{\mu }}\)) can be adjusted using weights \({\theta }^{\text{Q}}\) and \({\theta }^{{\mu }}\).
The behavioral policy employed for the generation of episodes that can be a noisy variant of the target policy \({\pi }_{b}\)(s)= π(s) + \(\aleph \left(\text{0,1}\right)\). The Q function in DQN is utilized for training of a critic neural network, however, in the place where a target \({y}_{t}\) has been calculated as \({y}_{t}={r}_{t}\)+\({\gamma }^{Q}\left({s}_{t+1, }{\pi }\right({s}_{t+1, }\left)\right)\), wherein, \(\gamma\) will be the discounting factor. Loss \({\mathcal{L}}_{a}\)= \(-{E}_{a}\) \(Q(s,{\pi }\left(\text{s}\right)\)was used for training actor networks.
The metaheuristic algorithms iterates through the space of parameter values for maximizing task performance and minimizing the number of training episodes. The parameters: discounting factor \(\gamma\); polyak-averaging coefficient \(\tau\); learning rate for networks are optimized. In this work, the Firefly Mutated algorithm, PSO-TABU, and Firefly-TABU are used for optimizing the parameters.
3.1 Hybrid PSO-TS Algorithm
PSO simulate the behavior of a flock of birds in a multi-dimensional space looking for an optimum one. The swarms made up of particles which are randomly initialized and used to search for optimal solutions through updating during the iterations. The candidate solutions encoded as particles traverse within an area of search (both current and possible solutions). It includes a movement to a promising area to obtain the global optimum and includes two approaches, cognitive and social [17].
Tabu search (TS) was based on the idea that if problem-solving has to be qualified as intelligent should integrate an adaptive memory with responsive exploration. This feature of adaptive memory of the TS permits the execution of certain processes that can search for a solution space that is both effective and economical. As all other local choices have been guided by information collated at the time of the search, the TS will contrast along with memoryless designs that are dependent on the semi-random processes implementing a new form of sampling [18].
The drawback of PSO is that it may be caught into a local solution and in certain cases slow convergence. For preventing the particles from jumping into various local location regions, the TS has been incorporated to it to allocate a new Tabu attribute to the regions. So, there can be a synergy of the TS and PSO duly employed in a hybridized fashion in Deep Reinforcement Learning (DRL). For accelerating the convergence of the PSO, the proposal was to combine it with the TABU search algorithm. At every epoch, the PSO provides the best solution, along with other paths that are examined by the application of the TABU [19, 20]. The objective function is thus used for the PSO to minimize Mean Square Error (MSE) calculated using Eq. (2):
Wherein: r refers to the sample size and T is the sampling time, e(𝑘) = d(𝑘) − m(𝑘) will be the difference between different quantities of the desired output d(𝑘) and the computed output with the m(𝑘) process under control.
The primary basis of the proposed PSO-TS is shown below: the good solutions are identified using the PSO. After this, a “critical path” is presumed to be a “TABU”. To improve the PSO solution, the TS process tries to enhance the quality of the gbest. The PSO and the TS is iteratively repeated until such time a stop position is met as per Fig. 1.
3.2 Hybrid FA-TS Algorithm
The Firefly Algorithm (FA) imitates the social behavior of fireflies. The two basic functions of flashing light of the fireflies are attracting mates and also to attract their potential prey. For the FA, all search strategies will be multi-agent that are regulated using randomization, and an effectual local search which the choice of the best possible solutions [21]. For the purpose of this work, a new hybrid FA-TS combination technique has been proposed for solving task scheduling using limited constraints. The primary objective function for this is the reduction of makespan while improving the solutions and their quality. Here, a complete algorithm scheme for task scheduling was designed with the TS local search aiming to look out for a local optimum for each individual and moving directly the TS to a much more promising space [22].
The hybrid FA-TS approach proposed was used to determine a global optimal value that had a high success rate working effectively and efficiently. The primary advantage of this was it made use of real and random numbers. The metaheuristic algorithms, in most cases, face the challenge of dealing with certain stochastic test functions. But the FA was better in dealing with them even if there were noisy problems of optimization. The main workload for critical resources and the total workload of all the resources and makespan were dealt with.
Even though there is a hybrid approach that has been proposed, the work does have some limitations. There is plenty of attention needed to make a choice of parameters since they can affect the results to a significant extent and may also yield poor results. The flowchart of the proposed FA-TS is depicted in Fig. 2.
3.3 PSO-Mutated
For the purpose of improving the PSO and its performance, and to mitigate local minima issue, different mutation operators for the PSO are proposed. Some works mutate its gbest particle and some of its local best particle using varied methods that can avert the PSO from being stagnated in local minima. Certain variants of the PSO were discussed based on mutation operators. A variant uses Cauchy mutation. The global best particle to be compared to the original one based on its fitness and the one that was fitter was chosen. The Eq. (3) below was used to mutate the particle.
\({gbest}_{i} \left(i\right)={gbest}_{i} \left(i\right)+W\left(i\right)*\) N (\({x}_{min}, {x}_{max}\)) (3)
Wherein, N refers to a cauchy distributed function using a scale parameter t = 1, N (\({x}_{min}, {x}_{max}\)) is a random number in (\({x}_{min}, {x}_{max}\)) of domain of test function and given by (4),
$$W\left(i\right)=\left({\sum }_{j=1}^{popsize}V\left[j\right]\left[i\right]\right)∕popsize$$
4
Wherein, \(V\left[j\right]\left[i\right]\)refers to the \({i}^{th}\) velocity vector of the \({j}^{th}\) particle, and the popsize was used for population.
3.4 Firefly Mutated
The strength of the optimization algorithm is based on the way in which the algorithm can explore new solutions in the minimum possible time and the manner in which the solutions are exploited to ensure they are made better and more efficient. The FA is a step movement that includes both exploitation and exploration effectively with good outcome in a short period of time. Thus, the mutation is included in the FA. Based on mutation, the manner in which new solutions are looked at by means of a step-by-step manner emitting brighter light. The features of better fireflies helps faster coverage and avoiding in the local optima. In order to perform a new mutation operator that chooses certain elements with a probability pm (mutation probability) and this is useful in helping the fireflies escape from the local optima. To decide if the firefly has to participate in mutation, it will not be dependent on the mutation probability \({p}_{m}\) and this is computed based on each iteration. The selection of the \({p}_{m}\) can significantly impact the performance of an algorithm. The generic values of \({p}_{m}\) will be 0.001 to 0.05. The mutation probability in the MFA is computed by using (5)
$$MP={f}_{new}-{f}_{old}$$
5
Wherein, \({f}_{new}\)fnew is the fitness for the new firefly and \({f}_{old}\) is the fitness of the original firefly. For the generation that goes through \({n}_{m}\) mutation operations, an average mutation progress value which is \(\stackrel{\sim}{M}P\) has been given by (6)
$$\stackrel{\sim}{M}P= \frac{1}{{n}_{m}}\sum MP$$
6
The rates of mutation were adjusted before the end of every generation with the average progress values, on the basis of equations (5) and (6) [23].
The simulations are carried out using CloudSim platform and Matlab. The task size was set to be between [500, 4000] with a latency of 5s to 10s. the transmission delay ranges from 1 s to 5 s and expected latency ranges from 5 s to 10 s. Any arriving rate is applicable. The processing capacity of the virtual machine is [1000, 2000]. The size of the waiting slot is 5, and the length of the backlog queue is set at 5. The DRL, Firefly DRL, PSO-Tabu DRL, PSO-Mutated DRL, Firefly Tabu DRL, Firefly Mutated DRL as shown in Tables 1 to 3 and Figs. 3 to 5.
| Number of Tasks | DRL | Firefly DRL | PSO-Tabu DRL | PSO-Mutated DRL | Firefly Tabu DRL | Firefly Mutated DRL |
|---|---|---|---|---|---|---|
| 50000 | 15.9 | 17.7 | 18.3 | 19 | 19.8 | 20.3 |
| 100000 | 13 | 14.3 | 14.8 | 15.3 | 15.8 | 16.3 |
| 150000 | 12.7 | 13.9 | 14.3 | 14.7 | 15.3 | 15.8 |
| 200000 | 12.3 | 13.4 | 13.8 | 14.3 | 14.6 | 15 |
| 250000 | 12.1 | 13.3 | 13.6 | 14.1 | 14.6 | 15 |
From the Fig. 3, it can be observed that the Firefly Mutated DRL has higher optimization ratio on makespan by 24.31%, 22.52%, 21.75%, 19.78% & 21.4% for DRL, by 13.68%, 13.07%, 12.79%, 11.26% & 12.01% for Firefly DRL, by 10.36%, 9.64%, 9.96%, 8.33% & 9.79% for PSO-Tabu DRL, by 6.61%, 6.32%, 7.21%, 4.77% & 6.18% for PSO-Mutated DRL and by 2.49%, 3.11%, 3.21%, 2.7% & 2.7% for Firefly Tabu DRL when compared with 50000, 100000, 150000, 200000 and 250000 number of tasks respectively.
| Number of Tasks | DRL | Firefly DRL | PSO-Tabu DRL | PSO-Mutated DRL | Firefly Tabu DRL | Firefly Mutated DRL |
|---|---|---|---|---|---|---|
| 50000 | 82.6 | 84.3 | 84.9 | 85.5 | 86.1 | 86.7 |
| 100000 | 81.4 | 83.4 | 83.9 | 84.4 | 85.1 | 85.8 |
| 150000 | 81.8 | 83.8 | 84.3 | 84.8 | 85.5 | 86.2 |
| 200000 | 84.9 | 86.6 | 87.2 | 87.9 | 88.8 | 89.3 |
| 250000 | 83.6 | 85.5 | 86.2 | 86.8 | 87.4 | 88 |
From the Fig. 4, it can be observed that the Firefly Mutated DRL has higher resource utilization by 4.84%, 5.26%, 5.23%, 5.05% & 5.13% for DRL, by 2.81%, 2.83%, 2.82%, 3.06% & 2.88% for Firefly DRL, by 2.09%, 2.23%, 2.22%, 2.37% & 2.06% for PSO-Tabu DRL, by 1.39%, 1.64%, 1.63%, 1.58% & 1.37% for PSO-Mutated DRL and by 0.69%, 0.81%, 0.81%, 0.56% & 0.68% for Firefly Tabu DRL when compared with 50000, 100000, 150000, 200000 and 250000 number of tasks respectively.
| Number of Tasks | DRL | Firefly DRL | PSO-Tabu DRL | PSO-Mutated DRL | Firefly Tabu DRL | Firefly Mutated DRL |
|---|---|---|---|---|---|---|
| 50000 | 4.2 | 4.7 | 4.9 | 5.1 | 5.3 | 5.4 |
| 100000 | 4.8 | 5.3 | 5.5 | 5.6 | 5.8 | 5.9 |
| 150000 | 4.8 | 5.3 | 5.4 | 5.6 | 5.8 | 6 |
| 200000 | 5.2 | 5.7 | 5.9 | 6 | 6.2 | 6.4 |
| 250000 | 5.3 | 5.7 | 5.8 | 6 | 6.2 | 6.4 |
From the Fig. 5, it can be observed that the Firefly Mutated DRL has higher computation cost optimization by 25%, 20.56%, 22.22%, 20.68% & 18.8% for DRL, by 13.86%, 10.71%, 12.38%, 11.57% & 11.57% for Firefly DRL, by 9.7%, 7.01%, 10.52%, 8.13% & 9.83% for PSO-Tabu DRL, by 5.71%, 5.21%, 6.89%, 6.45% & 6.45% for PSO-Mutated DRL and by 1.86%, 1.71%, 3.38%, 3.17% & 3.17% for Firefly Tabu DRL when compared with 50000, 100000, 150000, 200000 and 250000 number of tasks respectively.
Cloud computing can be a promising and contemporary technology that can deliver computational resources of a large scale over the network with a pay-as-you-go model. Scheduling algorithms can benefit the cloud and the cloud provider. The operating performance of the scheduling algorithm was inseparably related to scheduling efficiency. For the purpose of this work, PSO-TABU Search (TS), Firefly Mutated, and Firefly-TS, PSO-Mutated were proposed. To increase the speed of convergence of the PSO, the proposal was to ensure the PSO algorithm with the TS to identify a better solution within the time taken for computation accuracy. For improving PSO performance, the TS based procedure embedded inside the PSO. The hybrid FA-TS is a technique that can solve a new and complete scheme of scheduling designed and the TS algorithm can be incorporated to search for the local optimum. To improve the solution quality, an effective heuristic was proposed to bring down makespan and improve resource utilization. The mutation operator to the basic PSO is like the GA. In case the historic optimal position \({P}_{i}\) keeps remains unchanged or has minimally changed, the Particle swarm can heavily aggregate. For this, \({P}_{i}\) can be retained and different dimensions of the particles are used to improve search accuracy and convergence speed. The Mutated Firefly Algorithm (FA) will explore the search space adding features to the firefly with lower brightness from brighter ones. To determine the extent of features to be added, mutation probability is calculated. The less bright a firefly is, the higher is its mutation probability. Experimental results demonstrate the effectiveness of the proposed Firefly Mutated DRL.
Ethical Approval
Not applicable.
Competing interests
The authors declare no competing interests.
Authors' contributions
Harshala Shingne - Conceptualization, Methodology, Software, Data curation,
Writing - Original draft preparation, Visualization, Investigation, Software Validation.
Shriram R – Writing – Reviewing and Editing
Funding
Not applicable.
Availability of data and materials
Data sharing is not applicable to this article as no new data were created or analyzed in this study.
- Patel, K. K., & Patel, S. M. (2016). Internet of things-IOT: definition, characteristics, architecture, enabling technologies, application & future challenges. International journal of engineering science and computing, 6(5).
- Patil, C., & Chaware, A. (2021). Integration of Internet of Things, Cloud Computing. In IOP Conference Series: Materials Science and Engineering (Vol. 1022, No. 1, p. 012099). IOP Publishing.
- Kaur, C. (2020). The cloud computing and internet of things (IoT). Int. J. Sci. Res. Sci. Eng. Technol, 19-22.
- Kaur, A. (2015). A review of workflow scheduling in cloud computing environment. Int. J. Comp. Sci. Eng, 4(2), 47-50.
- Gao, H., Kwong, S., Fan, B., & Wang, R. (2014). A hybrid particle-swarm tabu search algorithm for solving job shop scheduling problems. IEEE Transactions on Industrial Informatics, 10(4), 2044-2054.
- Garí, Y., Monge, D. A., Pacini, E., Mateos, C., & Garino, C. G. (2021). Reinforcement learning-based application autoscaling in the cloud: A survey. Engineering Applications of Artificial Intelligence, 102, 104288.
- Shadroo, S., Rahmani, A. M., & Rezaee, A. (2021). The two-phase scheduling based on deep learning in the Internet of Things. Computer Networks, 185, 107684.
- Pandiyan, S., Lawrence, T. S., Sathiyamoorthi, V., Ramasamy, M., Xia, Q., & Guo, Y. (2020). A performance-aware dynamic scheduling algorithm for cloud-based IoT applications. Computer Communications, 160, 512-520.
- Shah, H. A., & Zhao, L. (2020). Multiagent Deep-Reinforcement-Learning-Based Virtual Resource Allocation Through Network Function Virtualization in Internet of Things. IEEE Internet of Things Journal, 8(5), 3410-3421.
- Miao, Q., Lin, H., Wang, X., & Hassan, M. M. (2021). Federated deep reinforcement learning based secure data sharing for Internet of Things. Computer Networks, 197, 108327.
- Shingne, H., Sountharrajan, S., Karthiga, M., & Rajan, C. (2020). Lasso and ridge regression for optimized resource allocation in cloud computing. J Adv Res Dynam Contr Syst, 12, 1740-1747.
- Rjoub, G., Bentahar, J., Abdel Wahab, O., & Saleh Bataineh, A. (2020). Deep and reinforcement learning for automated task scheduling in large-scale cloud computing systems. Concurrency and Computation: Practice and Experience, e5919.
- Shingne, H., & Shriram, R. (2023). Heuristic deep learning scheduling in cloud for resource-intensive internet of things systems. Computers and Electrical Engineering.
- Saravanan, S., & Anbalagan, P. 2020, Enhanced cloud resources allocation using Deep reinforcement learning and Lion optimization. Journal of Seybold Report ISSN NO, 1533, 9211. VOLUME 15 ISSUE 9 2020, pp 1350-1363
- Zhang, Y., Wang, X., Wang, J., & Zhang, Y. (2020). Deep reinforcement learning based volt-var optimization in smart distribution systems. IEEE Transactions on Smart Grid, 12(1), 361-371.
- Zhang, Z., Zhang, D., & Qiu, R. C. (2019). Deep reinforcement learning for power system applications: An overview. CSEE Journal of Power and Energy Systems, 6(1), 213-225.
- Wang, D., Tan, D., & Liu, L. (2018). Particle swarm optimization algorithm: an overview. Soft Computing, 22(2), 387-408.
- Glover, F., Laguna, M., & Martí, R. (2018). Principles and strategies of tabu search. In Handbook of Approximation Algorithms and Metaheuristics, Second Edition (pp. 361-377). Chapman and Hall/CRC.
- Panda, M. R., Priyadarshini, R., & Pradhan, S. K. (2016, December). Autonomous mobile robot path planning using hybridization of particle swarm optimization and Tabu search. In 2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC) (pp. 1-7). IEEE.
- Toshev, A. (2019). Particle swarm optimization and tabu search hybrid algorithm for flexible job shop scheduling problem-analysis of test results. Cybernetics and Information Technologies, 19(4), 26-44.
- Rohaninejad, M., Kheirkhah, A. S., Vahedi Nouri, B., & Fattahi, P. (2015). Two hybrid tabu search–firefly algorithms for the capacitated job shop scheduling problem with sequence-dependent setup cost. International Journal of Computer Integrated Manufacturing, 28(5), 470-487.
- Kharat, M. G., Khadke, S. S., Raut, R. D., Kamble, S. S., Kamble, S. J., & Kharat, M. G. (2016). Application of Hybrid Firefly Algorithm-Tabu Search Technique to Minimize the Makespan in Job Shop Scheduling problem. International Journal of Applied Industrial Engineering (IJAIE), 3(2), 1-21.
- Arora, S., Singh, S., Singh, S., & Sharma, B. (2014, December). Mutated firefly algorithm. In 2014 International Conference on Parallel, Distributed and Grid Computing (pp. 33-38). IEEE.
- Sehgal, A., La, H., Louis, S., & Nguyen, H. (2019, February). Deep reinforcement learning using genetic algorithm for parameter optimization. In 2019 Third IEEE International Conference on Robotic Computing (IRC) (pp. 596-601). IEEE.
No competing interests reported.