Robust Particle Swarm Optimization for Active/Reactive and Reserve Scheduling in a Grid-Connected Microgrid with Energy Storage Systems

---------------------------------------------------------------------------------------------------------------------Abstract: Background: In recent years, simultaneous participation in electrical energy and ancillary services markets has been very profitable for distributed energy resources (DERs). Moreover, the presence of renewable generations along with energy storage systems (ESS) is bringing a significant contribution to modern distribution systems. High penetration of non-predictable power sources in microgrids (MGs), due to the uncertainties of these products, increases the need for ancillary services and the management and coordination of these technologies combined with the ESSs. Results: For the first time, this paper develops a robust particle swarm optimization model to handle the uncertain renewable power production involved in the joint active/reactive and reserve scheduling of a smart MG. The robust optimization approach has a medium priority compared to deterministic and stochastic ones. The objective function utilized for the optimal joint active/reactive and reserve scheduling of an MG is defined as maximizing social welfare, which is accomplished based on a max-min optimization model. The robust optimal solution can be achieved in such a way that the maximizer at the outer level makes an optimal decision against the worst-case objective function, which is acquired based on the minimizer at the inner level considering the uncertainty neighborhood. Conclusions: The effectiveness of the proposed method is examined on a 33-bus MG test system. Simulation results prove that the proposed RPSO model can help MG operators to reduce scheduling costs to obtain a higher social welfare. The consideration of more uncertainty in renewable energy resources production leads to higher operation costs, especially reserve costs. Integration of robustness against uncertainty in the joint active/reactive and reserve management in the smart MGs leads to a more robust operation at the expense of higher costs.

The set of buses that have DER.
The set of buses that contain DL.
The set of variables for outer maximization

Introduction
In recent years, to hinder the destructive effects of environmental degradation and depletion of fossil fuel energy sources, distributed energy resources (DERs), including renewable energy sources (RESs), energy storage systems (ESSs), demand response (DR) programs and other policies with green characteristics are rapidly emerging in the world power industry [1]. However, the mismanagement of these resources in the network can create numerous challenges in the system operation, such as worsening the network reliability and flexibility, deteriorating the voltage profile of the feeders, and increasing power losses of the network [2][3][4]. In line with this concern, smart grids equipped with advanced communication infrastructure have been introduced to attain the best possible amalgamation of diverse energy resources in distribution systems [5].
Microgrid (MG) consists of generators of electrical energy (renewable generators, micro turbines, fuel cells, etc.), electrical loads and sometimes ESS, which can be a consumer or supplier of electricity with the coordination of its members [6]. Developing the DERs in the distribution systems, the power system can be divided into multi-MGs in which all these technologies are coordinately controlled by a central or local controller through wide-area monitoring systems [7].
Due to the high integration rate of the RESs such as photovoltaic (PV) cells and wind turbines (WTs) into the grid, the uncertain parameters will substantially increase in the distribution ESSs and demand response (DR) programs can successfully alleviate the influence of uncertainties and intermittencies of renewable generations in the electric power systems [8,9]. The ESSs with small-capacity fast-response characteristics are appropriate for smoothing rapid fluctuations with low range, and the large-capacity ESSs with slow response are compatible with slow variations with large-scale extent [10]. Power loss reduction in electrical networks is another important ESS application, as reported in [11]. The grid power losses are generally minimized based on the optimal charging/discharging scheduling of the ESSs by shifting energy usage from off-peak periods to on-peak hours. Moreover, optimal operation of the ESS (i.e., charging-discharging management) reduces operational costs and improves voltage stability [12]. Many studies only have addressed the optimum active power scheduling of ESSs and have not investigated their reactive power capability to absorb or inject reactive power to the grid [13]. Active users equipped with a battery energy storage system (BESS) can contribute effectively to alleviate the overvoltage and undervoltage troubles, by exchanging a certain amount of active power with the grid [14]. The authors of [15] have developed a smart home energy management (HEM) strategy in which electric vehicles (EVs) and ESS can participate in reactive power compensation at the point of grid connection.
Considering the uncertainties in the problem modeling could affect the optimum decision variables and objective function value. Hence, uncertain parameters append an additional aspect to any global optimization search. Optimization problems by considering the uncertainties are typically dealt with using both stochastic [16][17][18][19][20] and robust models [21,22]. Stochastic optimization requires the probability distributions of uncertain parameters along with numerous scenarios to address uncertainties and yield a guaranteed solution [23]. However, the probabilistic distribution of some uncertain parameters is challenging to be acquired [24]. Accordingly, the stochastic model of the MG operation will be complicated, and needs to improve its computational burden.
Alternatively, describing the lower and upper bounds for each uncertain parameter is usually easier than obtaining scenarios for the uncertain variables. A robust optimization approach finds an optimal decision that optimizes its performance in the worst case. This procedure is defined as a min-max or a max-min optimization. In the max-min one, the maximizer makes an optimal decision against the worst case obtained by the minimizer. This means that at any point in the solution space, an inner optimizer is applied to seek the minimum objective function value in the corresponding uncertainty intervals, while an outer optimizer is utilized to determine the maximum minimal objective function. While, in the min-max optimization, this trend will be reversed. In non-convex optimization problems under uncertainty, it is impossible to achieve an exactly desired solution. The robust optimization aims to find a robust solution where the worst-case solution still achieves a possible performance.
Under the restructuring of electric power systems, the utilization of DERs and DR programs is becoming the most beneficial way to provide ancillary services to comply with the grid code requirement and to improve the power quality [25][26][27]. Also, the distribution system operator (DSO) is willing to acquire the required ancillary services for the power system security from the active users. In the literature, the potentials of different DER technologies, ESS, and active loads in the smart distribution grid have been investigated in various fields. Currently, in most power systems, non-dispatchable DERs are not capable of participating in the ancillary services market.
However, as reported in [28], a new framework is being defined, enabling non-dispatchable DERs to participate in such a market. Thus, equipping modern distribution networks with the DERs as new reactive power and reserve resources can lead to economic and technical benefits [26,29].
The authors of [30] have proposed an energy management model based on the virtual power plant (VPP) scheme to examine the capability of the WTs and synchronous distributed generations in the coupled energy and reserve market. Authors of [31] have developed a stochastic optimal bidding framework of a renewable-based MG to contribute in the energy and reserve markets. In [32], stochastic-based programming has been introduced for optimal energy management of the renewable-based MG with a parking lot and DR programs to take part in both energy and reserve markets. In [27], the optimum active and reactive power management of the EVs has been proposed as an appropriate policy to alleviate the voltage drop and line overloading as well as to reduce network losses. In [33], a two-stage robust optimization is developed for active/reactive power dispatch in the active distribution systems using relaxed optimal power flow, and then a column-and-constraint generation algorithm has been employed to solve the robust optimization model.
Since the payment for reactive power support is much lower than that for electrical energy provision; the coupled active and reactive power market clearing is not convenient for the bulk power systems [34]. Furthermore, the computing burden and complexity in the coupled market are more significant than that of the separate energy and reactive power markets. However, in the active distribution networks and MGs, since the necessary electrical energy is provided locally by the DERs, the concurrent optimization of active-reactive power scheduling is utilized in many studies [18,[35][36][37].
In the literature, coupled energy/reserve management [29,38,39] and active/reactive power scheduling in the smart MGs have been addressed considering the problem's uncertainties.
However, few papers have been reported on the concurrent active/reactive and reserve scheduling in the MG with the ESS. DERs such as diesel generators (DGs) along with WTs and PVs as RES units are the new technologies that should be optimally managed in coordination with ESSs.
Concisely, this paper presents a novel robust model to simultaneously address the active/reactive power and reserve scheduling in the grid-connected MG in the presence of ESS units and dispatchable loads to maximize social welfare (SW). In the proposed robust model (as a max-min optimization), a maximization level makes an optimal robust decision against the worst-case solution obtained by another minimization layer, which makes solving them more challenging than classical optimization problems. To the best of our knowledge, in this paper, it is the first time that a robust PSO [40] is implemented for optimal active/reactive and reserve management of the smart MGs. As stated in [41,42], DC power flow is generally utilized to model the ESS management.
However, DC power flow does not yield an accurate and realistic operation model of the network with ESS. Accordingly, AC power flow is implemented in this paper to analyze the network and ESSs in order to present a realistic and more practical model and results. The effectiveness of the proposed method is examined on a 33-bus MG test system.
Eventually, the main contributions of this paper are briefly summarized as follows:  A robust model is proposed to simultaneously manage the active/reactive power and reserve scheduling in the grid-connected MG.
 A Robust particle swarm optimization (RPSO) model is presented to handle the uncertain renewable power production involved in the scheduling of a smart MG.
 The presence of the ESS and dispatchable loads are analyzed in the proposed model.

Problem formulation for robust scheduling of MG
Here, the proposed formulation for robust joint active/reactive and reserve scheduling of MG is presented based on a max-min optimization model.

Objective function
The objective function for the optimal joint active/reactive and reserve scheduling of an MG is defined as maximizing social welfare (SW), which is performed based on a max-min optimization problem. The robust optimal solution can be achieved in such a way that the maximizer at the outer level makes an optimal decision against the worst-case objective function, which is acquired based The costs of procuring active/reactive power from the UPN are given as follows [43]: The payment for provided reserve by the UPN is expressed as follows: The reserve cost of a DER is given as: The bid function of a DL that corresponds to its benefit from energy consumption is calculated as follows:

Constraints
In the proposed formulation, a set of technical constraints that are expressed in the following should be satisfied.
The robust joint active/reactive and reserve scheduling of the MG has been examined regarding a set of various technical constraints, which are expressed in the following.

Constraints on ESS
The operational constraints of the ESS can be displayed by Eqs. (15) to (21). The active power charge and discharge of the ESS should be restricted by Eqs. (15) and (16) (21).

DL Constraints
As a normal load behavior, it is assumed that DLs preserve a constant power factor (cos ( )) during the scheduling horizon.

Robust Particle Swarm Optimization (RPSO)
Here, the PSO approach is extended to an RPSO framework to tackle the max-min optimization problem. The proposed RPSO is accomplished based on an inner minimization search around any point visited by a particle to substitute the SW of the visited point with its worst value. By employing this method, all parts of the PSO formulation remains unchanged. Pseudo codes of the presented RPSO are given in the following. For all particles (p=1, …, Npop) 8: Scheduling the base condition based on solving bellow problem:

Simulation results
The proposed robust joint active/reactive and reserve scheduling in a grid-Connected MG is examined on a modified 33-bus MG as a case study displayed in Fig. 2. Two diesel generators (DGs), two ESSs along with four RESs, including two PVs and two WTs, are considered in this case study. Furthermore, it is supposed there is a DL located at bus 24. The hourly energy prices of the UPN reflecting the day-ahead prices are listed in Table 1. as the only coefficient of the reactive cost function of the UPN are assumed to be 1% of the hourly energy prices of the UPN.  Table 2 provides the techno-economic data of the DGs, and the operational data of the ESSs are presented in Table 3. Table 4 represents the reserve prices offered by the UPN, the DGs and the DL. The coefficients of the active cost function The coefficients of the reactive cost function  Table 3. The data of the ESSs.  The hourly forecasted active power generations of PVs and WTs are delineated in Figs The proposed method is examined on two scenarios as listed in table 5. The simulation results are discussed in detail in the following. As observed, scenario S2 shows a higher range of uncertainty neighborhood around the forecasted power production of RESs.  Fig. 5 illustrates the hourly charge and discharge power of the ESSs in scenario S1. The percentage of hourly load can also be plotted in Fig. 5. As expected, ESSs are charged during the low load period while in other hours, they are mostly discharged. Fig. 6 displays the SOC values of the ESSs in scenario S1. According to Fig. 6., the SOC initially increases and then decreases until the final SOC restriction for both ESSs is satisfied. Fig. 7 shows the hourly charge and discharge power of the ESSs in scenario S2, and their corresponding SOC values are depicted in Fig. 8.
Similar to the obtained results in scenario S1, in low load period, the ESSs are charged, their SOC levels increase, and then they are discharged subsequently their SOC levels go down.   examining Figs. 9 and 10, it can be observed that the reserve curves for scenarios S1 and S2 are similarly varied. Moreover, the hourly reserve values in S2 are higher than the ones in scenario S1, because a higher range of uncertainty neighborhood has been defined in the S2 compared to scenario S1. Hence, the higher reserve cost in scenario S2 can be expected (as demonstrated in Fig. 21).     The hourly voltage profiles of the network buses in scenarios S1 and S2 are plotted in Figs. 18 and   19, respectively. As can be observed, all voltages are within the permissible range. However, in peak hours, the voltage of buses with ESS, DG and RES is approaching the upper voltage limit.
On the other side, buses 18 in S1 and 15 in S2 have the lowest voltages in the most hours.   than the corresponding values in scenario S1. This is due to that scenario S2 takes a higher uncertain range into consideration for the output power production of the RESs. Thus, this situation leads to higher operating costs in the worst case and therefore, it will result in a decrease in the SW. Since considering a more uncertain production range of RESs imposes more cost, component z2 has lower amount respect to component z1 for both scenarios. Concerning the computed third component of SW, z3, it can be inferred that more reserve cost should be paid by the system operator in scenario S2. Fig. 20. The different components of the SW in scenarios S1 and S2.

Conclusion
In this paper, a max-min optimization model to simultaneously manage the active/reactive power and reserve scheduling in the grid-connected MG in the presence of ESS units and dispatchable loads has been presented to maximize social welfare. In the proposed robust problems, a maximization level makes an optimal robust decision against the worst-case solution obtained by another minimization layer, which makes solving them more challenging than classical optimization problems. The PSO approach is extended to a robust PSO framework to tackle the max-min optimization problem considering the uncertainty of wind speed and solar radiation. AC power flow is implemented in this paper to analyze the network and ESSs in order to present realistic and more practical models and results. The effectiveness of the proposed method has been evaluated on a 33-bus MG test system. In the simulation, two scenarios with different uncertainty ranges of RESs production have been regarded. The simulation results evidence that the ESSs are charged during low-cost off-peak hours and are discharged at on-peak high-cost hours. Moreover, they can reduce operational costs, and consequently, the social welfare will be significantly improved. The consideration of more uncertainty in RESs production leads to higher operation costs, especially reserve costs. It should be mentioned that integration of robustness against uncertainty in the joint active/reactive and reserve management in the smart MGs leads to a more robust operation at the expense of higher costs. The future research work will be addressed to utilize the EVs' participation in the robust concurrent active/reactive power and reserve management.