Stochastic control for optimal power flow in islanded microgrid

International Journal of Electrical and Computer Engineering (IJECE)
Vol. 9, No. 2, April 2019, pp. 1045~1057
ISSN: 2088-8708, DOI: 10.11591/ijece.v9i2.pp1045-1057  1045
Journal homepage: http://iaescore.com/journals/index.php/IJECE
Stochastic control for optimal power flow in islanded microgrid
Thang Diep-Thanh1
, Quang Nguyen-Phung2
, Huy Nguyen-Duc3
1,2
Institute for Control Engineering and Automation, Hanoi University of Science and Technology, Vietnam
3
School of Electrical Engineering, Hanoi University of Science and Technology, Vietnam
Article Info ABSTRACT
Article history:
Received Dec 19, 2017
Revised May 23, 2018
Accepted Jun 25, 2018
The problem of optimal power flow (OPF) in an islanded mircrogrid (MG)
for hybrid power system is described. Clearly, it deals with a formulation of
an analytical control model for OPF. The MG consists of wind turbine
generator, photovoltaic generator, and diesel engine generator (DEG), and is
in stochastic environment such as load change, wind power fluctuation, and
sun irradiation power disturbance. In fact, the DEG fails and is repaired at
random times so that the MG can significantly influence the power flow, and
the power flow control faces the main difficulty that how to maintain the
balance of power flow? The solution is that a DEG needs to be scheduled.
The objective of the control problem is to find the DEG output power by
minimizing the total cost of energy. Adopting the Rishel’s famework and
using the Bellman principle, the optimality conditions obtained satisfy the
Hamilton-Jacobi-Bellman equation. Finally, numerical examples and
sensitivity analyses are included to illustrate the importance and effectiveness
of the proposed model.
Keywords:
Hamilton-Jacobi-Bellman
equation
Hybrid Power System
Markov Process
Microgrid
Optimal Power Flow
Copyright © 2019 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Quang Nguyen-Phung,
Institute for Control Engineering and Automation,
Hanoi University of Science and Technology,
1 Dai Co Viet, Hai Ba Trung, Hanoi, Vietnam.
Email: quang.nguyenphung@hust.edu.vn
1. INTRODUCTION
In the last decades, the earth’s climate has changed much more than before, particularly the carbon
dioxide level has increased faster and faster so that CO2 level surpassed 404 ppm for the first time in
recorded history as mentioned in climate.nasa.gov. In order to reduce carbon dioxide level, renewable energy
source is a great solution for energy generation. From this point of view, it is expected that the renewable
electricity such as wind and solar would play an important role to provide the electrical energy not only in the
near future but also to islanded microgrid [1], [2]. Concerning the microgrid (MG) of which the concept was
proposed and has been recently emerged, developed in many institutions to introduce a smart grid [3]. In
reality, the installation of DG with small-scale power generation technologies has rapidly increased in many
countries at a reduced cost and a higher efficiently. MG can operate in grid-connected, and islanded operating
modes [4], [5].
The principle roles of the microgrid control structure are [4] and [5] as voltage, frequency, and angle
regulation for operating modes; proper load sharing and distributed energy resources; microgrid
synchronization with the main grid (utility grid); power flow control between the microgrid and the main
grid, or within microgid; and optimizing the microgrid operating cost. In islanded operating mode, the local
loads should be supplied by the DG units that the control is carried out as controlled voltage sources (CVS)
[6]. Moreover, the MG operation can be subject to several problems such as: voltage oscillations, frequency
oscillations, and power quality issues [7]. Therefore, the hierarchical control structure consists of four levels,
namely, the primary, secondary, tertiary, and emergency control [4]. Literature [4] shows that the primary
control is in normal operation, and droop control methods involve an inherent trade-off between power

 ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 9, No. 2, April 2019 : 1045 - 1057
1046
sharing and voltage and frequency regulation, and include fundamental control hardware. The secondary
control is in off-normal operation, and compensate for the voltage and frequency deviations after the primary
control achieves the power sharing. Particularly, the tertiary control is used to restore the secondary reserve,
and manages the power flow between the microgid and the grid, and within the microgrid for the case of
islanded mode that leads to the economic issues. Therefore, the optimal power flow is also to be called
energy management that implements in a SCADA system that produces the lowest cost of energy.
To solve the problem of optimal power flow in the tertiary control level, basically we must consider
the uncontrollable nature of wind, solar power as well as load change raises uncertainty for power system
operation on the one part, the application and integration of DGs into the system is complex on the other part.
Among these aspects are, the investigation of impact of distributed generation on the power fluctuations from
penetration of wind, photovoltaic power is presented in [8], [9]. According to [10], [11] the authors have
considered the hybrid power system whose the energy storage/thermal unit has a high potential for providing
regulation power to meet the reverse requirements. In the context of uncertainty, the work in [12] focus on
the design optimization of MG by using the stochastic method to model energy systems such that the
probability of occurrence of incident solar source called Stochastic Programming method. In the literature,
there are many researches that have been aimed to solve the problem optimality.
One of important works as in [13], which examines the optimal problem as the investigation of
optimal power flow by adopting the interval optimization in which the wind power is defined in range of
values; the interval power flow problem for DC power flow can be formulated as a non-convex and nonlinear
programming. In consequence, to ensure system reliability, the forecasting uncertainty must be considered
into generation scheduling, and interval power flow provides promising approach to achieve the boundary
information of system statuses under uncertainties. More importantly, the contribution in [13] is a landmark
for class of optimal power flow problem. For the load flow problem which is typically formulated as a set of
non-linear equations as functions of bus voltages has taken some advantage. The contributions of optimal
power flow are considred in [14], the control of MG in [15], and quality of power (power factor, total
harmonic distortion) as in [16]. Stochastic demand has been developed and presented in [17]-[19], those
authors used stochastic, fuzzy, and probability programming techniques to model the uncertainties. Although
the works in [17]-[19] have been specified to load flow problem with some algorithms, the aforementioned
methods are typically dependent on predefined probability distribution function or membership function of
uncertainty. Furthermore, it is hard to identify accurate probability distribution functions due to data
availability and stochastic nature of the uncertainty.
The research in all these directions [8]-[22] for the case of power flow was based on non-convex,
and non-linear programming and the DG units have known locations and running all the time. To add to the
complexity of the problem, in the real life systems, the operation of these DG units, undergoes different
scenarios according to the strategies of the electricity producers and the needs of the consumers. Therefore,
some uncertainties are introduced in the operation of such units and thus, stochastic modeling of systems
involving DG units becomes of great interest. The sources of the uncertainties in the operation of the DG
systems at a certain hour of the day include. These uncertainties affect the modeling and evaluation of the
system capacity, power losses.
It’s not similar to the reviewed papers above, the aim of this paper design is to focus on
investigation of the power flow of hybrid wind-PV-diesel power system under uncertainties such as load
change, wind power fluctuation, sun irradiation power disturbance, and the diesel engine generator fails and
is repaired at random times. The objective function is to minimize the cost of electricity production.
Therefore utilizes the energy balance equation in real time to formulate the optimal power flow problem as
optimal control problem of a linear system by using dynamic programming. By adopting the Rishel’s
formalism [20], the DEG failure and repair times are characterized by Markov process and the cost function
is described at terminal time. To do this it, will be expedient to derive an algorithm similar to the Bellman
principle where the optimality conditions satisfy Hamilton-Jacobi-Bellman equations, and the value function
is convex. In addition, one of the principle reasons for introducing feedback into an optimal control for power
flow problem is to make the resulting system relatively insensitive to fluctuations that can deal with
uncertainties of power system considered.
2. HYBRID ISLAND MICROGRID
For the simple system as shown in Figure 1, we consider the hybrid power system on the islanded
AC microgrid including AC loads, photovoltaic (PV), wind turbine generator (WTG), and diesel engine
generator (DEG). Also we consider a stochastic control problem for the DEG subject to failures and repairs
in continuous time.

Int J Elec & Comp Eng ISSN: 2088-8708 
Stochastic control for optimal power flow in islanded microgrid (Thang Diep-Thanh)
1047
Wind
Turbine
Load
1
PV
array
Load
2
Diesel
Engine
Load
3
Tie-line
Figure 1. Simplified schematic of an islanded microgrid
As in Figure 1, the system consists of three stochastic sources such that WTG as stochastic wind
power, PV as stochastic solar power, and includes the DEG that is subject to failures and repairs. On the
other hand, the load demand is stochastic based on forecasting in short-term. The DGs are connected to AC
bus by power electronic devices used synchronization as AC sources as DEG, WTG, and PV with invert DC
voltages into AC called inverter. The DEG is used for the conversion of mechanical energy into electrical
energy as controllable source.
Now, the problem is considered in finite time in order to find the DEG power generation. Such a
DEG is subject to random failures and repairs and can inject the power into MG. The DEG has two possible
operational states. The dynamic of the DEG (i.e., jump from one state to another) is characterized by a two-
state stochastic process  (s): s 0 defined as:
0 if the DEG is under repair
( )
1 if the DEG is fully functioning
s

 
 (1)
The process (s) is a controlled process, as it will depend on the DEG action when the DEG is
available generation. Thus, (s): T  s  t is a Markov process, with value in 0, and 1. Let  , ,P F
denote the underlying probability space and let ( )( ),s
t t sF s c t t= £ £ be the sigma algebra. Let us define
F(t) and G(t) as the functioning and repair time distributions of the DEG, respectively. Given that the DEG is
operational, the probability of a failure in an interval of length t is t. The probability that the failed DEG
is repaired during such a time interval is t. Thus, the parameters  and  are the failure and repair rates for
the DEG. Now, the dynamics of the DEG state are given by:
( ) ( ) ( )ijPr |t t j t i p t o tc cé ù+ D = = = D + Dê úë û if j  I (2)
In this case, p01 = , p10 =  are the transition probabilities.
Based on Markov properties [21], the transition probabilities are defined by
( ) ( )
( )
( )
ij
ij
1 if
Pr |
if
p t o t i j
t t j t i
p t o t i j
c c
ìï + D + D =ïïé ù+ D = = = íê úë û ï D + D ¹ïïî (3)
with
( )0lim 0t O tD ® D =
, jj iji j
p p¹
= - å for all i,j . (4)
As mentioned in Section 1, the OPF problem is considered in the tertiary control level, in what
following, we then describe the modeling of the considered microgrid. First of all, the total power generation
of DGs satisfies the demand such as:

 ISSN: 2088-8708
1048
( ) ( ) ( ) ( )DEG WTG PVP t P t P t D t+ + =
(5)
where D(t) refers to the load demand at time t; PWTG(t) is the non-dispatchable (WTG) output power
generation at time t; PPV(t) is the PV output power generation at time t; and PDEG(t) is the DEG output power
generation at time t. As mentioned above, the produced power by PV and WTG depends on the
environmental conditions, and the demand depends on the power consumption habits. Meanwhile, the DEG
is subject to random events such as failure and repair. As result, the fluctuations in loads, PV, and WTG
output power are adjusted by control in the DEG output power under failure and repair events. Therefore, the
expression of the power balance in Equation (5) for randomness in such DG associated with demand can be
described as follows:
( ) ( ) ( ) ( )( )DEG WTG PVP t D t P t P t= - +
(6)
The power generation of DEG must evolve the fluctuations in loads and DGs in Equation (6). In
literature the optimal power flow (OPF) problem has been investigated based on linear and non-linear
programming such [10], and [13]. In contrast, we will formulate the considered problem as an optimal
control one in order to find out the optimal policies for operation of microgrid in the next section.
3. MATHEMATICAL MODEL
In this section, we consider an optimal control problem for the system described in Section 2, and
assume that the DEG is subject to random events such subject to failures and repairs in continuous time
called jump linear. The problem is considered in finite time (i.e., 0 < T < ) in order to find the DEG output
power. In the literature, Rishel in [20] was established the necessary and sufficient condition of the stochastic
control problem by the dynamic programming. In Rishel’s work, the discrete jump is characterized by
Markov process and the cost function described at terminal time. For more large, Rishel in [22] presents the
necessary conditions by use the unnormalized conditional probabilities and expectations of partially observed
jump Markov process, and the cost function defined by the integral on a finite interval. An extension of
Rishel’s formalism, Boukas in [23] was established the conditions for optimality for partially observed
controlled with the non-homogenous Markov process. For a semi-Markov process, the transition rates depend
on the duration of the system. So those transition rates can be used to model phenomena like maintenance in
real time.
Clearly, to formulate a new model, by adopting Rishel’s framework [20] we shall assume that the
power generation in the real time, and the state equation is represented as follows:
( )
( ) ( ) ( ) ( )( )( )
i
i
WTG PV
dX t
U t D t P t P t
dt
= - - +
(7)
( )
( ) ( )( ), ,
i
i
dX t
f t X t U t
dt
=
(8)
Constraints:
( ) ( ) ( )W W
min max
TG TG
WTGP t P t P t£ £
(9)
( ) ( ) ( )min max
PV PV
PVP t P t P t£ £
(10)
( ) ( ) ( )min max
DEG DEG
DEGP t P t P t£ £
(11)
where ( ) ( )DEGU t P t= is the control variable in equation (6) and ( ) [ )0,U t +
Î Â = + ¥ in [kW], D(t) is load
demand; PWTG(t), PPV(t) and PDEG(t) are within their forecasted upper bounds ( ) ( ) ( )W
max max max, ,TG PV DEG
P t P t P t and

1049
lower bounds ( ) ( ) ( )W
min min min, ,TG PV DEG
P t P t P t (in [kW]), ( )X t is the difference between cumulative electric
production and cumulative load demand, called surplus at time t if X(t) is positive and backlog if X(t) is
negative, shortly called energy in [kWh] and ( ) ( ), ,X t Î Â= - ¥ +¥ i is the discrete state index i : in
this case i = 0 and 1, ( ), ,i
f t X U is the state function and satisfies the Lipschitz condition [24]:
( ) ( )1 2 1 2, , , ,i i
f t X U f t X U K X Xr- £ -
(12)
where K is constant.
The behavior of the state variable ( )X t will be specified shortly in the subsection 3.2.
Let us define cost function (cost-to-go or cost-to-arrive) which is given by
   , ; , , | ( ) , ( )
T
i
t
J X t U E G s X U ds X t X t i
  
   
  

(13)
where ( , ( ), ( ))G t X t U t is the running cost function: (.)G C X C X+ + - -
= + with C+
representing a unit
surplus cost at time t, C-
the unit backlog cost at time t , ( )max ,0X X+
= , and ( )min 0,X X-
= - .
Thus, the function ( ), ,i
J t X U is called an overall cost of the system. To simplify things, we make
the following assumptions in this paper to describe the hybrid power system:
(A.1) The total power generation satisfies the power demand in finite time considered T, i.e, at any time t:
( ) ( ) ( ) ( )DEG WTG PVP t P t P t D t+ + ³
(A.2) The stochastic power values D(t), PWTG(t) and PPV(t) are forecasted values in [kW].
DEFINITION 3.1.
(1) A control variable ( ) ( ) ( ){ }, , 0Y X t U X t U t= = ³ is called an admissible control;
(2) A control ( ),t XS is the set of admissible control Y with initial vector X(t) = X. 
Our motivation is to obtain admissible control ( ) ( ), ,U X t X tÎ S that optimizes the cost function
(13). In what following, we will build the model that satisfies the contrary (3-12) and optimization of (13) by
using the dynamic programming approach.
We formulate the power flow problem defined above. Under appropriate conditions, the optimal
control policy is to satisfy (8-12) in order to determine the OPF ( ),U X t which minimize the cost function
described in (13). These policies are characterized by a target production level subject to capacity constraints.
We denote ( ),i
V X t the value function, i.e.:
( ) ( )(.) ( ,.)
, inf , ;i i
U t
V X t J X t U
Î S
=
(14)
This function will be used to establish the optimality conditions. For simplicity in the presentation of
the model, we use only the sign ( ),i
V X t . Based on the dynamic programming principle, the following
theorem is used for the generalization of the value function in (14):
Theorem 3.1 Control problem satisfies the system of partial differential equations:
( ) ( ) ( ) ( ) ( )ji
(.) ( ,.)
0 inf , , , , , , ,i i i j
t t
U t
j
XG t X U V X t f t X U V X t p V X t
Î S
ì üï ïï ïï= + + +í ý
ï ïï ïþïî
å
(15)

 ISSN: 2088-8708
1050
at time t the initial and boundary conditions are satisfied:
( ) ( )( ) for , ; , ( ) 0i
X t X X t Q V T X T= Î =
(16)
where the terms  ,i
tV X t , and  ,i
XV X t denote the gradient of value function respect to time t and state
variable X, respectively, and [ ]0, .Q t T= ´ Â 
The following theorem presents the necessary and sufficient conditions for which an optimal
solution exists.
Theorem 3.2. Let ( ),i
V X t QÎ be a solution to (15). Then for all ,X t Q :
(i) for every admissible control system  ,U X t :
( ) ( ), ,i i
V X t J X t;U£
(17)
(ii) If there exists an admissible system  ,*
U X t such that
( ) ( ) ( ) ( ){ }(.) (.)
, arg min , , , , ,* i i
X
U
U X t G t X U f t X U V X t
Î W
Î +
(18)
almost everywhere in t, then ( ) ( ), ,i i *
V X t J t X;U= , and ( ),*
U X t is the optimal solution.
Proof. The Proofs of these theorems are adopted the Rishel’s framework as in [37] and [30]. 
REMARKS. The system of partial differential Equation (15) is well known the Hamilton-Jacobi-Bellman
(HJB) equations associated with optimal control problem under study. The optimal feedback control (15) is
designed to drive the system to the optimal point (hedging point or balance point), and corresponds to the
value function described by Equation (15). Then, when the value function ( ),i
V X t is available, an optimal
policy can be obtained as in Equation (18). However, an analytical solution of equation (15) is so hard to
find. Thus, the numerical solution of HJB Equation (18) is adopted from Kushner’s method [25] to be
represented in the next section.
3.1. The feedback control
Let  t be the random parameter such as the stochastic wind power, demand, or solar radiation
(called disturbance or noise dependent on context). Let   ,t X t be the mapping of  ,U X t such that
    , ,t X t X t  . In the Equation (7), the disturbance  t may consist of D(t), PWTG(t), and PPV(t).
Hence, the feedback control of (7) is represented in the Figure 2:

1051
      ,. , ,X t f t X t U t
System    ,U t t X t
  ,t X t
 t
Disturbance

Figure 2. The closed-loop policies [26]
3.2 Behavior of cumulative production
By definition, the energy in equation (7) whose value is described as follows:
( ) ( )( )( )
0
( ) ( , ) ( )
t
o WTG PV
t
X t X U t X D s P s P s dsé ù= + - - +ê úë ûò
(19)
In the Figure 3, there is two kinds of regions: (1) the total power generation is less than the demand
called backlog in the interval (0, t1), (2) it is more than the demand called surplus when t < t2. The balance
point (hedging point) is at time t* = t1 and t = t2 where the production meets the demand.
Cumulative
Electricity Production
and Demand
t
0
( , )
t
WTG PV
t
U s X P s P s ds
0
t
t
D s ds
t1 t2
-
+
Figure 3. Energy production strategy (this behavior is modified from Gershwin’s framework in [27])
4. NUMERICAL APPROACH
In this section, we develop the numerical method for solving the optimality conditions represented
in the previous section. This method is based on Kushner’s approach [25]. By adopting the algorithm in [28],
the HJB equation (15) which includes the gradient of value function of Vi
(t, X) can be solved. Let X > 0 and
t > 0 denote the length of the finite difference interval of the variable X and t respectively. The first-orders

 ISSN: 2088-8708
1052
partial derivative of the value functions ().i
tV and ().i
XV in equation (15) are replaced by the expressions:
( )
( ) ( ), ,
,
i i
i
t
V t t X V t X
V t X
t
+ D -
=
D (20)
( )
( ) ( )
( )
( ) ( )
, ,
if , , 0
,
, ,
otherwise
i i
i
X i i
V t X X V t X
f t X U
XV t X
V t X V t X X
X
ìï + D -ïï ³ï Dïï= í
ï - - Dïïïï Dïî (21)
Using X and t, and after manipulations, the HJB equations can rewritten as follows:
( )
( )
()ji
(.)
(.)(., ) (.)( ,.) (.)
0 min (.) ,. .
ii
i j
U
j
V X X VV t t V
G f t p V
t XD
DD
DD
D
ì üï ï± D+ D -ï ïï= + + +í ý
ï ïD Dï ïþïî
å
m m
(22)
The next theorem shows that ( ),i
v t XD is an approximation to ( ),i
v t X for small step size X.
Theorem 4.1 Let ( ),i
V t XD denote a solution to HJB equation (22). Assume that there are constants Cg and
Kg such that
( ) ( )0 , 1 gKi
gV t X C XD£ £ +
(23)
then
0
lim ( , ) ( , )i i
V t X V t XD
D®
=
(24)
Proof. The proof of this theorem is adopted from the one in [29] for the case of deterministic control
problem. In this study, we make use of the policy improvement technique to obtain a solution of the
approximating optimization problem. Let XGD
and UGD
be the grids of the states and control vectors belonging
to the control space, the algorithm is represented as follows:
Step 1 (Initialization): Choose e +
Î Â . Let k = 1 and ( ), 0k
V t XD = ,( ) [ ]0, , Xt X t T GD
Î ´ and
k
UU GD
Î (initial policy).
Step 2: For a given k
UU GD
Î , compute
( ) ( )1
, , ,k k
V t X V t X-
D D= ( ) [ ]0, , Xt X t T GD
Ú Î ´
Step 3: Compute the corresponding value function to obtain the control policy ( )( ),U t X t
Step 4: Convergence test
( )( ) ( )( )1
min min , ,k k
V t X t V t X td -
D D= - ; ( )( ) ( )( )1
max max , ,k k
V t X t V t X td -
D D= -
If max mind d e- £ , then stop, else k = k+1 and go to the step 2.

1053
5. NUMERICAL EXAMPLE
The proposed model in Section 3 is for application of hybrid wind/photovoltaic/diesel engine
generator generation system such as follows:
Figure 4 presents the power flow of microgrid including DG1 as DEG, DG2 as WTG, and DG3 as
PV.
DG3
DG1
DG2
Bus
D(t)
PPV(t)
PWTG(t)
PDEG(t)
Figure 4. The power flow of MG
In this example, the forecast of load demand, PV power, and wind power generations are
represented in Figure 5 to Figure 7. The constraints of system parameters in equations (4-7) for MG presents
in the Table 1, these parameters also obey the Assumptions A.1 and A.2. In the Table 1, the failure rate is
 = 0.0001 (i.e., the Mean Time to Failure is 10000 time units) and the repair rate is  = 0.09 (i.e., the Mean
Time to repair is 11.11 time units) that means the probability that the DEG fails is about 1.1%, i.e., for 8760
hours in operation the DEG could be repaired in 97.3 hours (see in [27] for more details).
Table 1. Parameters of Optimal Power flow Model
No Description Lower Upper
1 Time interval [h] 0 24
2 WTG power [kW] 0 500
3 PV power [kW] 0 600
4 DEG power [kW] 1000 3500
5 Load demand [kW] 0 3500
6 C- Backlog cost [$/kWh] 10 -
7 C+ Surplus cost [$/kWh] 1 -
8 Failure rate  0.0001 -
9 Repair rate  0.09 -
The results are illustrated in Figure 8 and Figure 9. Figure 8 represents the optimal production of
DEG output power versus time t in the interval [0, 24h]. Figure 9 represents the cumulative electricity
production of MG and the load demand versus time t. This figure shows that, the effectiveness of optimal
control gives the birth to optimal power flow of MG that satisfies the load demand, thus the characteristic is
linear instead of being non-linear as in Figure 3. Meanwhile, Figure 10 presents the value function which is
convex function.
As the results, the novel model to hybrid wind-diesel-photovoltaic system is valid to find out the
optimal power output of DEG.

 ISSN: 2088-8708
1054
Figure 5. PV power forecast [kW] Figure 6. Wind power forecast [kW]
Figure 7. Demand forecast [kW] Figure 8. Optimal output power of DEG [kW]
Figure 9. Cumulative production [kWh] Figure 10. Value Function
0 5 10 15 20 25
0
50
100
150
200
250
300
350
400
450
500
SolarPowerForecast/kW
Time periods/h
0 5 10 15 20
0
100
200
300
400
500
600
WindPowerForecast/kW
Time periods/h
0 5 10 15 20 25
3100
3150
3200
3250
3300
3350
3400
3450
3500
LoadDemandForecast/kW
Time periods/h
0 5 10 15 20
2000
2200
2400
2600
2800
3000
3200
3400
3600
3800
Optimal Diesel Power Flow
OptimalPowerFlowofDEG/kW
Time periods/h
0 5 10 15 20
0
1
2
3
4
5
6
7
8
9
x 10
4
CumulativeProduction&Demand
Time periods/h
Demand
Optimal Production
-5
0
5
10
0
5
10
15
20
0
2
4
6
8
10
12
x 10
5
Surplus Level x
DEG at mode 1 "fully-functioning"
Time Periods/h
ValuefunctionV(t,x,1)

1055
6. SENSITIVITY ANALYSIS
As mentioned above, the conventional implementation of state DEG in this study is based on the
selection of successor states and the execution of related actions. Since it is natural to use state machines to
model these applications, the systemic optimal models built focused on a variety of production organization
scenarios, production interactions and DEG states, as well as on stochastic market demand dynamics, WTG
power, and PV power.
The new proposed model was built with a view to stochastic generation sources in parallel
generating electricity, and was applied to single-DEG, WTG, and PV of islanded microgrid. The effects of
the cost variable on the model are illustrated in Table 2. Consider the HPS in islanded mode, and with a fixed
demand (D = 3000kW), WTG power (PWTG = 1200kW), and PV power (PPV = 500kW).
Table 2 Sensitivity Analysis for Varying c-
and c+
Case c+ c- z* V(t,z*) million
1
2
3
100
100
100
1000
2000
3000
0
0
0
88,04
162,5
237,0
4
5
6
200
400
600
1000
1000
1000
0
0
0
88,04
88,04
88,04
From a practical and a mathematical point of view, the use of the HJB equation in order to
determine the optimal process cost offers a comprehensive optimization method, which renders the efficient
performance of parallel machines and interactive applications feasible. The general stationary probability
distribution functions for the operation mode and the repair mode, respectively, were formally investigated
by determining whether the machine had sufficient average capacity to meet demands. The results obtained
after the simulation of various inventory costs as shown in Table 1 and Table 2 can be interpreted as shown
below.
Table 2 presents the surplus and backlog costs, the hedging point z*, and the optimal costs at time
t = 11.48h which is arbitrarily chosen from within the time interval (0, 24). The surplus and backlog costs
(C+
, C-
) vary between 100 and 600 units and 1000 and 3000 units, respectively. Case 1-3 in Table 2 shows
that an increase in the backlog cost from 1000 to 3000 leads to the value function V1
(t, z*) increasing from
88,04 to 237,0M. Meanwhile, the hedging point z*(t) does not change in order to stabilize the energy stock.
For case 4-6, when the inventory costs c+
are higher than 100, the value functions v1
(.) does not change.
The Markov model proves to be particularly suitable for modeling hybrid power systems, and its
optimal control, which is based on Bellman principle, can effectively be framed with a varying demand
policy that aims to optimize a given objective function. We can also see that the performance of the
optimization method is stable for different manufacturing system structures. Although some assumptions
have been made with regard to the processing time, the accuracy of the method is quite acceptable for general
applications.
7. CONCLUSIONS
We have provided an analysis of an optimal power flow problem for hybrid power generation
system, which underlies the well-known Markov properties. The discrete DEG states are characterized by
Markov processes, and so DEG is subject to random failures and repairs, with exponential distributions of
functioning and repair times. Using Markov processes in continuous time, simultaneously with Rishel’s
framework and a dynamic programming approach, a new model of the stochastic control problem in a
deterministic horizon is formulated without a discounted rate. While the dynamic programming approach is
used to make decisions in stages over time, Rishel assumptions are used to model the discrete DEG states.
In our model, we used the definitions of Markov processes found in Rishel [20] to characterize the
discrete events of the DEG such a breakdown and DEG repairs. The main objective of this new model is to
find the DEG power generation with minimizing the surplus cost of energy. Hence, the optimality conditions
have been established and showed that the optimal generation control is the bang-bang control. On the other
hand, the new model is to deal with optimal power flow control with uncertain power injection.
We applied our proposed model to real word hybrid power system: single DEG with DEG having
exponential distributions of failure and repair times. The HJB equation was involved by using numerical
approach. To valid our proposed model, a sensitivity evaluation has been implemented on a sample path.

 ISSN: 2088-8708
1056
REFERENCES
[1] D. Thiam, “Renewable decentralized in developing countries : Appraisal from microgrids project in Senegal,”
Renew. Energy, vol. 35, no. 8, pp. 1615-1623, 2015.
[2] T. N. A. Kavousifard, “Impact of thermal recovery and hydrogen production of fuel cell power plants on
distribution feeder reconfiguration,” IET Gener. Transm. Distrib., vol. 6, no. November 2011, pp. 831-43, 2012.
[3] T. Tanabe et al., “Optimized operation and stabilization of microgrids with multiple energy resources,” 7th
Internatonal Conf. Power Electron. ICPE’07, no. September 2007, pp. 74-78, 2008.
[4] J. M. Guerrero, J. C. Vasquez, J. Matas, L. G. De Vicuña, and M. Castilla, “Hierarchical control of droop-
controlled AC and DC microgrids - A general approach toward standardization,” IEEE Trans. Ind. Electron.,
vol. 58, no. 1, pp. 158-172, 2011.
[5] J. C. Vasquez, J. M. Guerrero, J. Miret, M. Castilla, and L. G. De Vicuña, “Hierarchical Control of Intelligent
Microgrids,” IEEE Ind. Electron. Mag., no. December 2010, pp. 23-29, 2010.
[6] X. Tang, X. Hu, N. Li, W. Deng, and G. Zhang, “A novel frequency and voltage control method for islanded
microgrid based on multienergy storages,” IEEE Trans. Smart Grid, vol. 7, no. 1, pp. 410-419, 2016.
[7] M. Reza Ebrahimi, M. Naser Hashemnia, M. Ehsan, and A. Abbaszadeh, “A Novel Approach to Control the
Frequency and Voltage of Microgrids in Islanding Operation,” Int. J. Eng. Technol., vol. 4, no. 5, pp. 562-566,
2012.
[8] Y. T. Tan, “Impact on the power system with a large penetration of photovaltaic generation,” The University of
Machester Institute of Science and Technology, 2004.
[9] F. Vallee, J. Lobry, and O. Deblecker, “System Reliability Assessment Method for Wind Power Integration,”
Power Syst. IEEE Trans., vol. 23, no. 3, pp. 1288-1297, 2008.
[10] W. El-Khattam, Y. G. Hegazy, and M. M. a. Salama, “Stochastic power flow analysis of electrical distributed
generation systems,” 2003 IEEE Power Eng. Soc. Gen. Meet. (IEEE Cat. No.03CH37491), vol. 2, pp. 1141-1144,
2003.
[11] C. Wang, M. Liu, and L. Guo, “Cooperative Operation and Optimal Design for Islanded Microgrid,” IEEE,
pp. 1-8, 2011.
[12] M. Lennard, A. Date, and X. Yu, “Islanded microgrid energy system parameter estimation using stochastic
methods,” Sol. Energy, vol. 147, pp. 300-313, 2017.
[13] T. Ding, Power System Operation with Large Scale Wind Power Integration. Springer, 2017.
[14] F. Laouafi, A. Boukadoum, and S. Leulmi, “A Hybrid Formulation between Differential Evolution and Simulated
Annealing Algorithms for Optimal Reactive Power Dispatch,” TELKOMNIKA (Telecommunication, Computing,
Electronics and Control), vol. 16, no. 2, pp. 513–524, 2018.
[15] J. Zhu and W. Pan, “Intelligent Control of Wind/Photovoltaic Microgrid Information Fusion.,” TELKOMNIKA
(Telecommunication, Computing, Electronics and Control), vol. 15, no. 1, pp. 264-272, 2017.
[16] S. M. Antony and G. Immanuel, “A novel single phase bridgeless AC/DC PFC converter for low total harmonics
distortion and high power factor,” Int. J. Power Electron. Drive Syst., vol. 9, no. 1, pp. 17-24, 2018.
[17] M. K. Kim, D. H. Kim, Y. T. Yoon, S. S. Lee, and J. K. Park, “Determination of available transfer capability using
continuation power flow with fuzzy set theory,” 2007 IEEE Power Eng. Soc. Gen. Meet. PES, pp. 1-7, 2007.
[18] D. Villanueva, J. L. Pazos, and A. Feijóo, “Probabilistic load flow including wind power generation,” IEEE Trans.
Power Syst., vol. 26, no. 3, pp. 1659-1667, 2011.
[19] P. Zhang and S. T. Lee, “Probabilistic Load Flow Computation Using the Method of Combined Cumulants and
Gram-Charlier Expansion,” IEEE Trans. Power Syst., vol. 19, no. 1, pp. 676-682, 2004.
[20] R. Rishel, “Control of Systems with Jump Markov Disturbances,” IEEE Trans. Automat. Contr., vol. 20, no. 2,
pp. 241-244, 1975.
[21] S. M. Ross, Introduction to Probability Models, 11th ed. Academic Press, 2014.
[22] R. Rishel, “Unormalized conditional probabilities and optimality for partially observed controlled jump Markov
processes,” Lect. notes Control Inf. Sci., pp. 326-343, 1982.
[23] E.-K. Boukas, “Commande optimale stochastique appliquée aux systèmes de production,” École Polytechnique de
Montréal, 1987.
[24] W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Second. Springer.
[25] H. J. Kushner and Paul Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, Second.
Springer, 2001.
[26] D. P. Bertsekas, “Dynamic Programming and Optimal Control: Volume I.” Athena Scientific, Belmont,
Massachusetts, p. 543, 2017.
[27] S. B. Gershwin, “Manufacturing Systems Engineering,” p. 350, 1994.
[28] E. Charlot, J. P. Kenné, and S. Nadeau, “Optimal production, maintenance and lockout/tagout control policies in
manufacturing systems,” Int. J. Prod. Econ., vol. 107, no. 2, pp. 435-450, 2007.
[29] H. Yan and Q. Zhang, “A numerical method in Optimal production and Setup Scheduling of Stochastic
Manufacturing Systems,” IEEE Trans. Automat. Contr., vol. 42, no. 10, pp. 1452-1455, 1997.

1057
BIOGRAPHIES OF AUTHORS
Thang Diep received the B.S degree from Department of Electric Power System, Hanoi University
of Science and Technology (HUST), Hanoi, Vietnam, in 2002, the Master’s degree in Electrical
Engineering from Institut National Polytechnique de Grenoble, Grenoble, France, in 2004, and the
Master of Engieering in Mechanical Engineering from École de Technologie Supérieure, Montreal,
Québec, Canada, in 2011. He is currently with Institute for Control Engineering and Automation at
HUST as Ph.D student. His Research Interests are Hybrid Power System, MG, Optimal Power Flow,
Optimal Control, Stochastic Optimal Control, Smart Grid.
Quang Nguyen-Phung received his Dipl.-Ing. (Uni.), Dr.-Ing. and Dr.-Ing. habil. degrees from TU
Dresden, Germany in 1975, 1991 and 1994 respectively. Prior to his return to Vietnam, he had
worked in Germany industry for many years, contributed to create inverters REFU 402 Vectovar,
RD500 (REFU Elektronik); Simovert 6SE42, Master Drive MC (Siemens). From 1996 to 1998, he
served as lecturer of TU Dresden where he was conferred as Privatdozent in 1997. He joined Hanoi
University of Science and Technology in 1999, as lecturer up to now. He is currently a professor of
HUST and honorary professor of TU Dresden. He was author/co-author of more than 160 journal
and conference papers; 8 books with three among them was written in German and one in English
entitled “Vector Control of Three-Phase AC Machines – System Development in the Practice”
published by Springer in 2008, and 2nd edition in June 2015. His Research Interests are Electrical
Drive Systems, Motion Control, Robotic Control, Vector Control of Electrical Machines, Wind and
Solar Power Systems, Digital Control Systems, Modeling and Simulation.
Huy Nguyen-Duc received the B.S degree from Department of Electric Power System, Hanoi
University of Science and Technology (HUST), Hanoi, Vietnam, in 2001, and the Ph.D degree in
Electrical Engineering from Department of Electrical Engineering, École de Technologie
Supérieure, Montreal, Québec, Canada, in 2011. He is currently with the Department of Electric
Power System at HUST. He is a member of the IEEE. His Research Interests are Power System
Operation and Control, Application of Artificial Intelligence Technique in Power Systems.

Stochastic control for optimal power flow in islanded microgrid

More Related Content

What's hot (16)

Similar to Stochastic control for optimal power flow in islanded microgrid (20)

More from IJECEIAES (20)

Recently uploaded (20)

Stochastic control for optimal power flow in islanded microgrid