时滞随机系统的最优控制问题及应用
|
摘要
近些年来,人们逐渐认识到现实世界中的一些问题的发展不仅依赖于当前的状态,而且还依赖于其过去的历史。这类问题应该用状态对过去有依赖的系统方程来刻画,我们称之为随机微分延迟方程(简记为SDDE).由于其在工程、生命科学及金融等领域的广泛应用(可参见[3；11；43；44；47；33]等),SDDE成为现代研究的热点问题。本文将致力于在金融及其他领域中常见的受控的延迟系统的研究。
对于一个系统而言,当观测与调控之间有时间差或者控制有滞后性时就会出现系统延迟,我们称之为时滞系统。2000年,(?)ksendal和Sulem[47]研究了一类财富方程为SDDE的最优控制问题。在他们的模型中,不仅当前值X(t)而且X(t-δ)及过去值在某种意义下的平均值都会影响t时刻财富的增长。由于所选模型的特殊性,他们能够将无穷维问题化为有限维问题来处理,得到了问题的最大值原理,并将结果应用于金融中的相关问题。但是他们的条件,相对来说,是比较强的。
在实际当中,观测过去的数据可能是离散的有限个点,因此我们首先考虑当前发展依赖于以往有限个点的系统,而且控制也具有延迟性,这些延迟还可以是时变的。
我们研究一类时滞系统的最优控制问题,其中状态及控制变量都有延迟。在相对比较一般的条件下,我们得到了这类受控的时滞系统的最大值原理。作为应用,我们用得到的结果处理了经济当中一类带有时滞的生产消费选择问题,并得到了问题的显示解。借助于数值计算,我们给出了不同的延迟时间对结果的影响情况。我们的主要创新在于引入新型的倒向随机微分方程(简称BSDE)一超前BSDE(参见Peng,Yang[55])作为伴随方程来处理时滞最优控制问题。据我们所知,这是首次采用这种方法来研究时滞控制系统的最优控制问题。
我们也考虑了具有时变延迟的正倒向系统,即用递归效用函数代替一般的效用函数且延迟δ不在是常数,而是时间t的函数。这部分结果推广了1995年Xu[65]中的结论。Hu,Peng[29],Peng,wu[54]和Yong[69]等相继研究了完全耦合的正倒向随机微分方程(简称FBSDE).在处理线性二次(LQ)问题(参见[62；67])和金融中的大户投资问题(参见Cvitanic和Ma[15])时都会遇到这类方程。而我们在探索时滞系统的最大值原理问题时遇到了一类新型的FBSDE,其正向方程为时滞随机微分方程,倒向方程为超前倒向随机微分方程.我们称其为推广的FBSDE,并给出了这类新型FBSDE解的存在唯一性条件.
早在1973年Kolmanovskii and Maizenberg就讨论了时滞随机系统的LQ问题。我们在此基础上考虑状态与控制都有延迟的LQ问题,并且用FBSDE和值函数两种方法来寻找最优反馈控制。另外,我们也将关于推广的FBSDE的结果应用于时滞线性二次非零和随机微分对策问题。
考虑一般情况,我们处理一类由随机泛函微分方程(简称SFDE)刻画的时滞系统的递归最优控制问题,其中系统及递归效用函数的变动都依赖于过去的一段状态而不仅仅是有限个点。对此类问题,我们证明了其值函数仍满足Bellman型的动态规划原理。由于对过去状态依赖形式的复杂性,完整形式的Ito公式和Dynkin公式都很难得到,故想进一步得到值函数满足的Hamilton-Jacobi-Bellman方程并不是件容易的事情。为此我们引入Mohammed [44]中提到的弱无穷小生成元及Fuhrman等[24]中用到的联合二次变差作为工具,得到了上述问题值函数对应的一个无穷维的HJB方程。而且我们也证明了值函数就是这个无穷维偏微分方程的粘性解。
最后,作为理论的应用,我们考虑一类具有延迟效用的广告问题即广告费用支出与产生效果之间会有一定的时间差。Gozzi和Marinelli在[26]中用动态规划原理的方法讨论过这类问题。我们采用与他们不同的方法-无穷维最大值原理来处理这类问题,这种方法是由Hu和Peng[28]首次提出的。这部分结果也是我们论文2.3部分在无穷维空间中的一个推广
本论文由5章内容组成,下面我们将列出主要结果。
第1章：介绍第2章-第5章所要研究的问题。
第2章：我们研究状态方程为如下形式的时滞系统的最优控制问题：首先我们给出此类SDDE解的存在唯一性。接着我们考虑控制域为凸集时的情形,得到了如下的最大值原理。
定理2.2.6.令u(·)为时滞随机最优控制问题(2.3)-(2.5)的最优控制,X(·)为其相应的最优轨线。则我们有如下论断：其中对任意0≤t≤T.Hamilton函数定义为:(p(·)，z(·))为如下超前BSDE的解
更进一步,我们可以得到如下的控制最优的充分条件.
定理2.2.8.假设u(·)∈A,令X(·)为其相应的轨线,p(t)和z(t)为伴随方程(2.12)的解。如果(H2.5)-(H2.6)及(2.13)(或(2.16))成立,则u(·)为时滞控制问题(2.3)-(2.5)的最优解。
利用得到的最大值原理,我们研究了一类时滞的生产消费选择最优化问题.生产资本x(t)满足如下方程：问题是如何选取消费率c(t)最大化代价泛函
下面的命题给出了最有控制问题的显示解,而且从Figure 1和Figure 2中我们可以看出不同的时间延迟对结果的影响情况。
命题2.2.9.对生产消费选择问题(2.17)-(2.18),最优消费率为c*(t)=(?),其中p(t)如式(2.20)中的形式。
接着,我们考虑了控制域非凸时的递归最优控制问题。假设延迟是时变的,且扩散项σ不含控制,即系统由如下的时滞正倒向方程来描述：
定理2.3.6.假设u(·)为最优控制,(x(·),y(·),z(·))为其相应的最优轨线。则对所有0≤t 第3章：我们考虑时滞最优化问题中遇到的一类推广的FBSDE：
定理3.1.3.若(H3.1)及(H3.2)成立,则推广的FBSDE (3.1)有唯一适应解(X,Y,Z).
我们研究了两类时滞LQ问题,寻找它们的最优控制的显示形式。
在问题3.1中,我们假设只有状态有延迟。借助于FBSDE的结果,我们有
定理3.2.1.控制
为问题3.1的唯一最优控制,其中(χt,yt,zt)为如下的推广的FBSDE的解
第二种情况,即问题3.2,我们假设系统的状态及控制都有延迟。通过一类更为复杂的FBSDE我们给出其最优控制的形式。
定理3.2.2.控制为问题3.2的唯一最优控制,其中(x(t),y(t),z(t))为推广的FBSDE (3.14)的解。
我们引入两种方法来寻找最优反馈控制。这两种方法是从解决最优控制问题的经典方法-最大值原理和动态规划原理这两个不同的角度出发的。
从最大值原理的角度出发,我们有：
定理3.3.1.假设存在矩阵值过程(Kt,Ht),t∈[0,T],满足广义的矩阵Riccati方程(3.19).则时滞线性二次最优控制问题问题3.3的最优反馈控制为且最优值函数为
从动态规划原理的角度出发,我们有：
定理3.3.3.设(3.25)中引入的满足如下带边值条件的方程组：对任意t∈[0,T]及θ,ζ∈[-δ,0],
则对问题3.4,
为[s,T](s≥δ)上的最优控制且值函数为
在第3章的最后,我们讨论时滞线性二次非零和随机微分对策问题。
定理3.4.1.当且仅当(u1(·),u2(·))具有如下形式时,(u1(·),u2(·))为对策问题Problem 3.5的一个Nash均衡点。其中为推广的FBSDE(3.29)的解。
第4章：我们给出了由如下SFDE刻画的时滞系统的递归最优控制问题的动态规划原理：
定理4.2.10.若A4.4-A4.8成立,则(4.9)式定义的时滞最优控制问题的值函数u(s,φ)具有如下的优良性质：对任意0≤(?)≤T-s,(?)
我们得到了值函数满足的HJB方程—一类无穷维的偏微分方程。
定理4.3.9.如果假设我们问题中的值函数则u(s,φ)为如下的Hamilton-Jacobi-Bellman偏微分方程的解：这里我们以▽0u(t,x)记▽xu(t,x)({0}).
我们有：
定理4.3.11.如(4.9)式定义的u(s,φ)为HJB方程(4.33)的一个粘性解。
第5章：在最后一章,我们研究一类广告模型中的时滞随机控制问题作为我们理论的应用。这类时滞的广告模型可以重新定义于Hilbert空间。我们利用无穷维的最大值原理处理该类问题：
定理5.3.1.Let(u(·),X(·),Y(·),Z(·))为问题5.1的最优控制及相应的轨线。则如下的最大值原理成立：其中(p(·),q(·),k(·))为如下伴随方程的解
这里A是一个强连续半群的无穷小算子,A*为它的伴随算子。若给定某些效用函数我们可以给出最优控制的显示形式。
It has been recognized in recent years that the description of many real world prob-lems should be modelled by stochastic dynamical systems whose evolution depend on the past history of the state. Such models are often referred to as stochastic differential delay equations (SDDEs for short). Due to their wide applications in engineering, life science and finance (see e.g. [3; 11; 43; 44; 33]), SDDEs become a popular topic in modern research. This thesis is dedicated to study the controlled systems with delay which arising in finance and other areas.
A delay term may arise in a control problem when there is a time lag between observation and regulation or the aftereffect of control. In 2000,(?)ksendal and Sulem [47] investigated a class of problems where the wealth X(t) at time t is given by a SDDE. In their model, not only the present value X(t) but also X(t -δ) and some sliding average of previous values affect the growth at time t. Because of the specific structure of the dependence of the past that they considered, they are able to reduce the problem to finite dimension. They proved maximum principles for such models and applied them to solving some problems related to finance. But the assumptions they need is relatively stronger.
In practice, the observation for the history of the state may be finite points. So, at first, we consider the system involving finite delayed points. Moreover, the delay points can be time-varying.
We pay attention to the systems involving both delays in state variable and in control variable. We derive the maximum principle for this kind of controlled systems with delay under more general conditions. As an application, we apply our result to a produce and consumption choice problem with delay in economics and the explicit solution of the problem is given. Moreover, by the numerical results, we show the effects of different time delays. The main novelty of our method is we introduce a new type backward stochastic differential equations(BSDEs for short)-anticipated BSDE (see Peng and Yang [55]) as our adjoint equation. To our best knowledge, it is the first attempt to study stochastic optimal control problem with delay in this way.
We also consider the forward-backward systems with time-varying delay i.e. the stochastic delayed systems with recursive utility and the delayδis a function of time t. This result extends that of [65].
Fully coupled forward-backward stochastic differential equation (FBSDE for short) was studied in Hu, Peng [29], Peng, Wu [54] and Yong [69] etc. It can be encountered in linear-quadratic(LQ) problem(see[62; 67]) and mathematic finance when we consider large investor (see Cvitanic and Ma[15]). A new type of forward-backward stochastic differential equations with Ito stochastic delay equations as forward equations and anticipated BSDEs as backward equations came out during our studying the maximum principle for the systems with delay. We obtain the existence and uniqueness results of the general FBSDEs under some suitable conditions.
The LQ regulatory problem involving stochastic delay equations was studied in Kolmanovskii and Maizenberg [34]. We consider the LQ problem with delays in state and control, and find the feedback regulator by FBSDE method and value function method independently. Also we apply our results on the general FBSDEs to deal with the LQ nonzero sum stochastic differential game problem with delay.
In general case, we treat a stochastic recursive optimal control problem in which both the controlled state dynamics and the recursive utility may depend on segment of the history, not only several points, i.e. the system is described by stochastic functional differential equation(SFDE for short). For such problems we prove that the value function satisfies a Bellman-type dynamic programming principle. Because of various forms of path dependency, the closed form of Ito's formula or Dynkin formula are generally hard to obtained, so the work to derive the Hamilton-Jacobi-Bellman equation is not easy. Using the weak infinitesimal generator introduced in Mohammed [44] and the joint quadratic variations in Fuhrman etc. [24], we obtain an infinite-dimensional HJB equation. It is shown that the value function is a viscosity solution of the HJB equation.
Finally, as an application of our results, we consider a class of dynamic advertising problems under uncertainty in the presence of carryover and distributed forgetting effects, which is also discussed in Gozzi and Marinelli [26]. We deal with the problem using maximum principle in infinite dimensional which is a method different from theirs. An this method was introduced firstly in Hu and Peng [28]. This section is also a generalization of our results in Section 2.3.
The thesis consists of five chapters. In the following, we list the main results.
Chapter 1:We introduce problems studied from Chapter 2 to Chapter 5.
Chapter 2:We study the stochastic optimal control problem for the system with delay as following: We give the existence and uniqueness of the solution for this type of SDDEs. And then we study the stochastic optimal control problem in which the domain of the control is convex. We have the following stochastic maximum principle.
Theorem 2.2.6. Let u(·) be an optimal control of the optimal stochastic control problem with delay subject to (2.3)-(2.5), X(·) is the corresponding optimal trajectory. Then we assert with notation (?)for all 0< t< T. And Hamiltonian function is defined by: (p(·),z(·)) is the solution of the following anticipated BSDE:
Moreover, we obtain the following sufficient optimality result.
Theorem 2.2.8. Suppose u(·)∈A and let X(·) be the corresponding trajectory, p(t) and z(t) be the solution of adjoint equation (2.12). If (H2.5)-(H2.6) and (2.13) (or (2.16)) hold for u(·), then u(·) is an optimal control for stochastic delayed optimal problem (2.3)-(2.5).
And we we apply our maximum principle to study a kind of production and con-sumption choice optimization problem with delay. The capital of the investorχ((?)) satisfies: The problem is how to choose the consumption rate c(t) to maximize the performance function
The explicit solution of the optimal control problem is given by the following proposition and the numerical results with differential delays are shown in Figure 1 and Figure 2.
Proposition 2.2.9. For the production and consumption choice problem (2.17)-(2.18), the optimal consumption rate is c*(t)= (?), where p(t) is of the form (2.20).
Consequently, we assume that the domain of the controls is non-convex, the prob-lem is with recursive utility and the delays are time-varying. In this case the control variable does not enter the diffusion coefficientσ, i.e. the system is described as the following forward-backward equation with delay:
Theorem 2.3.6. Let u(·) be an optimal control and (χ(·), y(·), z(·)) be the corresponding trajectory. Then, for all0≤t Chapter 3:We consider one kind of generalized FBSDE encountered in the op-timization problem:
Theorem 3.1.3. Let (H3.1) and (H3.2) hold. Then there exists a unique adapted solution (X, Y, Z) of the general FBSDE (3.1).
We study two kinds of LQ problems with delay. We seek the optimal control of the problem.
In Problem 3.1, there is only delay in state. By the result of the generalized FBSDE, we have
Theorem 3.2.1. The control is the unique optimal control of Problem 3.1, where (xt,yt,zt) is the solution of the following general FBSDE
In the second case, Problem 3.2, it is assumed that the system is with delays both in state and in control. We also find the optimal control by a more complex FBSDE.
Theorem 3.2.2. The control is the unique optimal control of Problem 3.2 where (x(t),y(t), z(t)) is the solution of a generalized FBSDE (3.14).
In order to find the optimal feedback regulator, we introduce two method. They are from different view of two main classical methods in optimal control-maximum principle and dynamic programming prindiple.
By maximum principle, we have:
Theorem 3.3.1. Suppose there exists matrix (Kt,Ht),t∈[0,T], satisfying the gener-alized matrix Riccati equation system (3.19). Then the optimal linear feedback regulator for the delayed linear quadratic optimal problem Problem 3.3 is and the optimal value function is
By dynamic programming principle, we have:
Theorem 3.3.3. Let Eo(t),E1(t,θ), E2(t,θ,(?)),t∈[0,T],θ,(?)∈[-δ,0] which we intro-duced in (3.25) satisfy the following set of equations: with boundary conditions for all t∈[0, T] andθ, (?)∈[-δ,0]. Then for the control problem Problem 3.4, is the optimal control on[s,T](s≥δ)and the ualue function is
At last in Chapter 3,we discuss the linear-quadratic nonzero sum stochastic dif-ferential game problem with delay.
Theorem 3.4.1.(u1(.),u2(.))isαNash equilibrium point for the above game Problem 3.5,if and only if(u1(·),u2(·) has the form with (xt,yt1,yt2,zt1,zt2)is the solution of the general FBSDE(3.29).
Chapter 4:We giVe the dynamic programming principle for one kind of stochastic recursive optimal control problem described by SFDE:
Theorem 4.2.10. Let A4.4-A4.8 hold. Then the value function u(s,ψ) defined in (4.9) for our optimal control problem with delay has the following virtue:for each 0≤τ≤T-s,
We obtain the HJB equation for the value function. It is an infinite dimensional partial differential equation.
Theorem 4.3.9.If we assume that the value function u(s,ψ)∈Clip1,2([0,T]×C)(?) D(S) in our problem,then u(s,ψ)solves the following Hamilton-Jacobi-Bellman partial differential equation with terminal condition u(T,ψ)=Φ(ψ). Here we denote (?)χu(t,χ)({0}) by (?)0u(t,χ).
And also we have:
Theorem 4.3.11.u(s,χ) as defined by (4.9) is a viscosity solution of HJB equation (4.33).
Chapter 5:In the last chapter, we investigate a stochastic delayed problem arising in advertising model as an application of our theoretical results. The advertising model can be reformulation in Hilbert space. Using maximum principle in infinite dimension we solve the problem:
Theorem 5.3.1. Let (u(·),-X(·),Y(·),Z(·)) be an optimal control and its corresponding trajectory of Problem 5.1. Then the following maximum condition holds, that is where(p(·),q(·),k(·)) be the corresponding solution of
Here A is an infinitesimal generator of a strongly continuous semigroup, and A* is its adjoint operator. We can get the explicit solution of optimal control for some given utility functions.
对于一个系统而言,当观测与调控之间有时间差或者控制有滞后性时就会出现系统延迟,我们称之为时滞系统。2000年,(?)ksendal和Sulem[47]研究了一类财富方程为SDDE的最优控制问题。在他们的模型中,不仅当前值X(t)而且X(t-δ)及过去值在某种意义下的平均值都会影响t时刻财富的增长。由于所选模型的特殊性,他们能够将无穷维问题化为有限维问题来处理,得到了问题的最大值原理,并将结果应用于金融中的相关问题。但是他们的条件,相对来说,是比较强的。
在实际当中,观测过去的数据可能是离散的有限个点,因此我们首先考虑当前发展依赖于以往有限个点的系统,而且控制也具有延迟性,这些延迟还可以是时变的。
我们研究一类时滞系统的最优控制问题,其中状态及控制变量都有延迟。在相对比较一般的条件下,我们得到了这类受控的时滞系统的最大值原理。作为应用,我们用得到的结果处理了经济当中一类带有时滞的生产消费选择问题,并得到了问题的显示解。借助于数值计算,我们给出了不同的延迟时间对结果的影响情况。我们的主要创新在于引入新型的倒向随机微分方程(简称BSDE)一超前BSDE(参见Peng,Yang[55])作为伴随方程来处理时滞最优控制问题。据我们所知,这是首次采用这种方法来研究时滞控制系统的最优控制问题。
我们也考虑了具有时变延迟的正倒向系统,即用递归效用函数代替一般的效用函数且延迟δ不在是常数,而是时间t的函数。这部分结果推广了1995年Xu[65]中的结论。Hu,Peng[29],Peng,wu[54]和Yong[69]等相继研究了完全耦合的正倒向随机微分方程(简称FBSDE).在处理线性二次(LQ)问题(参见[62；67])和金融中的大户投资问题(参见Cvitanic和Ma[15])时都会遇到这类方程。而我们在探索时滞系统的最大值原理问题时遇到了一类新型的FBSDE,其正向方程为时滞随机微分方程,倒向方程为超前倒向随机微分方程.我们称其为推广的FBSDE,并给出了这类新型FBSDE解的存在唯一性条件.
早在1973年Kolmanovskii and Maizenberg就讨论了时滞随机系统的LQ问题。我们在此基础上考虑状态与控制都有延迟的LQ问题,并且用FBSDE和值函数两种方法来寻找最优反馈控制。另外,我们也将关于推广的FBSDE的结果应用于时滞线性二次非零和随机微分对策问题。
考虑一般情况,我们处理一类由随机泛函微分方程(简称SFDE)刻画的时滞系统的递归最优控制问题,其中系统及递归效用函数的变动都依赖于过去的一段状态而不仅仅是有限个点。对此类问题,我们证明了其值函数仍满足Bellman型的动态规划原理。由于对过去状态依赖形式的复杂性,完整形式的Ito公式和Dynkin公式都很难得到,故想进一步得到值函数满足的Hamilton-Jacobi-Bellman方程并不是件容易的事情。为此我们引入Mohammed [44]中提到的弱无穷小生成元及Fuhrman等[24]中用到的联合二次变差作为工具,得到了上述问题值函数对应的一个无穷维的HJB方程。而且我们也证明了值函数就是这个无穷维偏微分方程的粘性解。
最后,作为理论的应用,我们考虑一类具有延迟效用的广告问题即广告费用支出与产生效果之间会有一定的时间差。Gozzi和Marinelli在[26]中用动态规划原理的方法讨论过这类问题。我们采用与他们不同的方法-无穷维最大值原理来处理这类问题,这种方法是由Hu和Peng[28]首次提出的。这部分结果也是我们论文2.3部分在无穷维空间中的一个推广
本论文由5章内容组成,下面我们将列出主要结果。
第1章：介绍第2章-第5章所要研究的问题。
第2章：我们研究状态方程为如下形式的时滞系统的最优控制问题：首先我们给出此类SDDE解的存在唯一性。接着我们考虑控制域为凸集时的情形,得到了如下的最大值原理。
定理2.2.6.令u(·)为时滞随机最优控制问题(2.3)-(2.5)的最优控制,X(·)为其相应的最优轨线。则我们有如下论断：其中对任意0≤t≤T.Hamilton函数定义为:(p(·)，z(·))为如下超前BSDE的解
更进一步,我们可以得到如下的控制最优的充分条件.
定理2.2.8.假设u(·)∈A,令X(·)为其相应的轨线,p(t)和z(t)为伴随方程(2.12)的解。如果(H2.5)-(H2.6)及(2.13)(或(2.16))成立,则u(·)为时滞控制问题(2.3)-(2.5)的最优解。
利用得到的最大值原理,我们研究了一类时滞的生产消费选择最优化问题.生产资本x(t)满足如下方程：问题是如何选取消费率c(t)最大化代价泛函
下面的命题给出了最有控制问题的显示解,而且从Figure 1和Figure 2中我们可以看出不同的时间延迟对结果的影响情况。
命题2.2.9.对生产消费选择问题(2.17)-(2.18),最优消费率为c*(t)=(?),其中p(t)如式(2.20)中的形式。
接着,我们考虑了控制域非凸时的递归最优控制问题。假设延迟是时变的,且扩散项σ不含控制,即系统由如下的时滞正倒向方程来描述：
定理2.3.6.假设u(·)为最优控制,(x(·),y(·),z(·))为其相应的最优轨线。则对所有0≤t
定理3.1.3.若(H3.1)及(H3.2)成立,则推广的FBSDE (3.1)有唯一适应解(X,Y,Z).
我们研究了两类时滞LQ问题,寻找它们的最优控制的显示形式。
在问题3.1中,我们假设只有状态有延迟。借助于FBSDE的结果,我们有
定理3.2.1.控制
为问题3.1的唯一最优控制,其中(χt,yt,zt)为如下的推广的FBSDE的解
第二种情况,即问题3.2,我们假设系统的状态及控制都有延迟。通过一类更为复杂的FBSDE我们给出其最优控制的形式。
定理3.2.2.控制为问题3.2的唯一最优控制,其中(x(t),y(t),z(t))为推广的FBSDE (3.14)的解。
我们引入两种方法来寻找最优反馈控制。这两种方法是从解决最优控制问题的经典方法-最大值原理和动态规划原理这两个不同的角度出发的。
从最大值原理的角度出发,我们有：
定理3.3.1.假设存在矩阵值过程(Kt,Ht),t∈[0,T],满足广义的矩阵Riccati方程(3.19).则时滞线性二次最优控制问题问题3.3的最优反馈控制为且最优值函数为
从动态规划原理的角度出发,我们有：
定理3.3.3.设(3.25)中引入的满足如下带边值条件的方程组：对任意t∈[0,T]及θ,ζ∈[-δ,0],
则对问题3.4,
为[s,T](s≥δ)上的最优控制且值函数为
在第3章的最后,我们讨论时滞线性二次非零和随机微分对策问题。
定理3.4.1.当且仅当(u1(·),u2(·))具有如下形式时,(u1(·),u2(·))为对策问题Problem 3.5的一个Nash均衡点。其中为推广的FBSDE(3.29)的解。
第4章：我们给出了由如下SFDE刻画的时滞系统的递归最优控制问题的动态规划原理：
定理4.2.10.若A4.4-A4.8成立,则(4.9)式定义的时滞最优控制问题的值函数u(s,φ)具有如下的优良性质：对任意0≤(?)≤T-s,(?)
我们得到了值函数满足的HJB方程—一类无穷维的偏微分方程。
定理4.3.9.如果假设我们问题中的值函数则u(s,φ)为如下的Hamilton-Jacobi-Bellman偏微分方程的解：这里我们以▽0u(t,x)记▽xu(t,x)({0}).
我们有：
定理4.3.11.如(4.9)式定义的u(s,φ)为HJB方程(4.33)的一个粘性解。
第5章：在最后一章,我们研究一类广告模型中的时滞随机控制问题作为我们理论的应用。这类时滞的广告模型可以重新定义于Hilbert空间。我们利用无穷维的最大值原理处理该类问题：
定理5.3.1.Let(u(·),X(·),Y(·),Z(·))为问题5.1的最优控制及相应的轨线。则如下的最大值原理成立：其中(p(·),q(·),k(·))为如下伴随方程的解
这里A是一个强连续半群的无穷小算子,A*为它的伴随算子。若给定某些效用函数我们可以给出最优控制的显示形式。
It has been recognized in recent years that the description of many real world prob-lems should be modelled by stochastic dynamical systems whose evolution depend on the past history of the state. Such models are often referred to as stochastic differential delay equations (SDDEs for short). Due to their wide applications in engineering, life science and finance (see e.g. [3; 11; 43; 44; 33]), SDDEs become a popular topic in modern research. This thesis is dedicated to study the controlled systems with delay which arising in finance and other areas.
A delay term may arise in a control problem when there is a time lag between observation and regulation or the aftereffect of control. In 2000,(?)ksendal and Sulem [47] investigated a class of problems where the wealth X(t) at time t is given by a SDDE. In their model, not only the present value X(t) but also X(t -δ) and some sliding average of previous values affect the growth at time t. Because of the specific structure of the dependence of the past that they considered, they are able to reduce the problem to finite dimension. They proved maximum principles for such models and applied them to solving some problems related to finance. But the assumptions they need is relatively stronger.
In practice, the observation for the history of the state may be finite points. So, at first, we consider the system involving finite delayed points. Moreover, the delay points can be time-varying.
We pay attention to the systems involving both delays in state variable and in control variable. We derive the maximum principle for this kind of controlled systems with delay under more general conditions. As an application, we apply our result to a produce and consumption choice problem with delay in economics and the explicit solution of the problem is given. Moreover, by the numerical results, we show the effects of different time delays. The main novelty of our method is we introduce a new type backward stochastic differential equations(BSDEs for short)-anticipated BSDE (see Peng and Yang [55]) as our adjoint equation. To our best knowledge, it is the first attempt to study stochastic optimal control problem with delay in this way.
We also consider the forward-backward systems with time-varying delay i.e. the stochastic delayed systems with recursive utility and the delayδis a function of time t. This result extends that of [65].
Fully coupled forward-backward stochastic differential equation (FBSDE for short) was studied in Hu, Peng [29], Peng, Wu [54] and Yong [69] etc. It can be encountered in linear-quadratic(LQ) problem(see[62; 67]) and mathematic finance when we consider large investor (see Cvitanic and Ma[15]). A new type of forward-backward stochastic differential equations with Ito stochastic delay equations as forward equations and anticipated BSDEs as backward equations came out during our studying the maximum principle for the systems with delay. We obtain the existence and uniqueness results of the general FBSDEs under some suitable conditions.
The LQ regulatory problem involving stochastic delay equations was studied in Kolmanovskii and Maizenberg [34]. We consider the LQ problem with delays in state and control, and find the feedback regulator by FBSDE method and value function method independently. Also we apply our results on the general FBSDEs to deal with the LQ nonzero sum stochastic differential game problem with delay.
In general case, we treat a stochastic recursive optimal control problem in which both the controlled state dynamics and the recursive utility may depend on segment of the history, not only several points, i.e. the system is described by stochastic functional differential equation(SFDE for short). For such problems we prove that the value function satisfies a Bellman-type dynamic programming principle. Because of various forms of path dependency, the closed form of Ito's formula or Dynkin formula are generally hard to obtained, so the work to derive the Hamilton-Jacobi-Bellman equation is not easy. Using the weak infinitesimal generator introduced in Mohammed [44] and the joint quadratic variations in Fuhrman etc. [24], we obtain an infinite-dimensional HJB equation. It is shown that the value function is a viscosity solution of the HJB equation.
Finally, as an application of our results, we consider a class of dynamic advertising problems under uncertainty in the presence of carryover and distributed forgetting effects, which is also discussed in Gozzi and Marinelli [26]. We deal with the problem using maximum principle in infinite dimensional which is a method different from theirs. An this method was introduced firstly in Hu and Peng [28]. This section is also a generalization of our results in Section 2.3.
The thesis consists of five chapters. In the following, we list the main results.
Chapter 1:We introduce problems studied from Chapter 2 to Chapter 5.
Chapter 2:We study the stochastic optimal control problem for the system with delay as following: We give the existence and uniqueness of the solution for this type of SDDEs. And then we study the stochastic optimal control problem in which the domain of the control is convex. We have the following stochastic maximum principle.
Theorem 2.2.6. Let u(·) be an optimal control of the optimal stochastic control problem with delay subject to (2.3)-(2.5), X(·) is the corresponding optimal trajectory. Then we assert with notation (?)for all 0< t< T. And Hamiltonian function is defined by: (p(·),z(·)) is the solution of the following anticipated BSDE:
Moreover, we obtain the following sufficient optimality result.
Theorem 2.2.8. Suppose u(·)∈A and let X(·) be the corresponding trajectory, p(t) and z(t) be the solution of adjoint equation (2.12). If (H2.5)-(H2.6) and (2.13) (or (2.16)) hold for u(·), then u(·) is an optimal control for stochastic delayed optimal problem (2.3)-(2.5).
And we we apply our maximum principle to study a kind of production and con-sumption choice optimization problem with delay. The capital of the investorχ((?)) satisfies: The problem is how to choose the consumption rate c(t) to maximize the performance function
The explicit solution of the optimal control problem is given by the following proposition and the numerical results with differential delays are shown in Figure 1 and Figure 2.
Proposition 2.2.9. For the production and consumption choice problem (2.17)-(2.18), the optimal consumption rate is c*(t)= (?), where p(t) is of the form (2.20).
Consequently, we assume that the domain of the controls is non-convex, the prob-lem is with recursive utility and the delays are time-varying. In this case the control variable does not enter the diffusion coefficientσ, i.e. the system is described as the following forward-backward equation with delay:
Theorem 2.3.6. Let u(·) be an optimal control and (χ(·), y(·), z(·)) be the corresponding trajectory. Then, for all0≤t
Theorem 3.1.3. Let (H3.1) and (H3.2) hold. Then there exists a unique adapted solution (X, Y, Z) of the general FBSDE (3.1).
We study two kinds of LQ problems with delay. We seek the optimal control of the problem.
In Problem 3.1, there is only delay in state. By the result of the generalized FBSDE, we have
Theorem 3.2.1. The control is the unique optimal control of Problem 3.1, where (xt,yt,zt) is the solution of the following general FBSDE
In the second case, Problem 3.2, it is assumed that the system is with delays both in state and in control. We also find the optimal control by a more complex FBSDE.
Theorem 3.2.2. The control is the unique optimal control of Problem 3.2 where (x(t),y(t), z(t)) is the solution of a generalized FBSDE (3.14).
In order to find the optimal feedback regulator, we introduce two method. They are from different view of two main classical methods in optimal control-maximum principle and dynamic programming prindiple.
By maximum principle, we have:
Theorem 3.3.1. Suppose there exists matrix (Kt,Ht),t∈[0,T], satisfying the gener-alized matrix Riccati equation system (3.19). Then the optimal linear feedback regulator for the delayed linear quadratic optimal problem Problem 3.3 is and the optimal value function is
By dynamic programming principle, we have:
Theorem 3.3.3. Let Eo(t),E1(t,θ), E2(t,θ,(?)),t∈[0,T],θ,(?)∈[-δ,0] which we intro-duced in (3.25) satisfy the following set of equations: with boundary conditions for all t∈[0, T] andθ, (?)∈[-δ,0]. Then for the control problem Problem 3.4, is the optimal control on[s,T](s≥δ)and the ualue function is
At last in Chapter 3,we discuss the linear-quadratic nonzero sum stochastic dif-ferential game problem with delay.
Theorem 3.4.1.(u1(.),u2(.))isαNash equilibrium point for the above game Problem 3.5,if and only if(u1(·),u2(·) has the form with (xt,yt1,yt2,zt1,zt2)is the solution of the general FBSDE(3.29).
Chapter 4:We giVe the dynamic programming principle for one kind of stochastic recursive optimal control problem described by SFDE:
Theorem 4.2.10. Let A4.4-A4.8 hold. Then the value function u(s,ψ) defined in (4.9) for our optimal control problem with delay has the following virtue:for each 0≤τ≤T-s,
We obtain the HJB equation for the value function. It is an infinite dimensional partial differential equation.
Theorem 4.3.9.If we assume that the value function u(s,ψ)∈Clip1,2([0,T]×C)(?) D(S) in our problem,then u(s,ψ)solves the following Hamilton-Jacobi-Bellman partial differential equation with terminal condition u(T,ψ)=Φ(ψ). Here we denote (?)χu(t,χ)({0}) by (?)0u(t,χ).
And also we have:
Theorem 4.3.11.u(s,χ) as defined by (4.9) is a viscosity solution of HJB equation (4.33).
Chapter 5:In the last chapter, we investigate a stochastic delayed problem arising in advertising model as an application of our theoretical results. The advertising model can be reformulation in Hilbert space. Using maximum principle in infinite dimension we solve the problem:
Theorem 5.3.1. Let (u(·),-X(·),Y(·),Z(·)) be an optimal control and its corresponding trajectory of Problem 5.1. Then the following maximum condition holds, that is where(p(·),q(·),k(·)) be the corresponding solution of
Here A is an infinitesimal generator of a strongly continuous semigroup, and A* is its adjoint operator. We can get the explicit solution of optimal control for some given utility functions.
引文
[1]Y. Alekal, P. Brunovsky, D. Chyung and E. Lee, The quadratic problem for systems with time delays, IEEE Transactions on Automatic Control,16(6) (1911), 673-687.
[2]F. Antonelli, Backward-forward stochastic differential equations, Ann. Appl. Probab.,3(1993),777-793.
[3]M. Arriojas, Y. Hu, S. E. A. Monhammed, and G. Pap, A delayed Black and Scholes formula, Stochastic Analysis and Applications,25(2)(2007),471-492.
[4]M. Basin, J. Rodriguez-Gonzalez, R. Martinez-Zuniga, Optimal control for linear systems with time delay in control input based on the duality principle, American Control Conference, Proceedings of the 2003,3,2003.
[5]G. Barles and E. Lesigne, SDE,BSDE and PDE, Pitman Research Notes in Math-ematics Series,364(1997),47-80.
[6]G. Barles, R. Buckdahn, E. Pardoux, Backward stochastic differential equations and integral-partial differential equations. Stochastics Stochastics Rep.,60(1997), 57-83.
[7]R. Bellman, On the theory of dynamic programming, Proc. Nat. Acad. USA, 38(1952),716-719.
[8]A. Bensoussan, Lectures on stochastic control In:Lecture Notes in Mathematics, 972, Berlin:Springer-Verlag,1982.
[9]J. M. Bismut, An introductory approach to duality in optimal stochastic control, SIAM REW.,20(1)(1978),62-78.
[10]M. Chang, T. Pang and M. Pemy, Optimal control of stochastic functional differ-ential equations with a bounded memory, Stochastic:An international Journal of Probability and Stochastic Process,80(1)(2008),69-96.
[11]M. Chang, Stochastic Control of Hereditary Systems and Applications, New York: Springer,2008.
[12]A. Chojnowska-Michalik, Representation. theorem for general stochastic delay equations, Bull.Acad. Polon. Sci. Ser. Sci. Math. Astronom. Phys.,26(7)(1978), 635-642.
[13]M. G. Crandall and P-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc.,277(1983),1-42.
[14]M. G. Crandall, H. Ishii and P-L. Lions, User's guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc.,27(1992), 1-67.
[15]J. Cvitanic and J. Ma, Hedging options for a large investor and forward-backward SDE's, Ann. Appl. Probab.,6(2)(1996),370-398.
[16]G. Da Prato and J. Zabczyk, Stochastic equations in infinite dimensions. Cam-bridge UP,1992.
[17]D. Duffie, Dynamic Aasset Picing Theory, Princeton University Press,1992.
[18]D. Duffie and L. G. Epstein, Stochastic differential utility, Economicrica, 60(2)(1992),353-394.
[19]I. Elsanousi, B. Larssen, Optimal consumption under partial observations for a stochastic system with delay, Dr. Scient. thesis, University of Oslo,2001.
[20]N. El Karoui, S. Peng, and M. C. Quenez, Backward stochastic differential equa-tion in finance, Math. Finance,7(1997),1-71.
[21]G. Feichtinger, R. Hartl and S. Sethi, Dynamical optimal control models in ader-tising:recent developments, Management Sci.,40(1994),195-226.
[22]W. H. Fleming, Generalized solutions in optimal stochastic control, in Differential Games and Control Theory 2, (Pro,2nd Kingston Conference, Kingston, RI, 1976), Lecture Notes in pure and Appl. Math.,30(1977),147-165.
[23]M. Fuhrman and G. Tessitore, Nonlinear Kolmogorov equation in infinite di-mensional spaces:the backward stochastic differential equations approach and applications to optimal control, Ann. probab.,30(3)(2002),1397-1465.
[24]M. Fuhrman, F. Masiero and G. Tessitore, Stochastic equations with delay:opti-mal control via BSDEs and regular solutions of Hamilton-Jacobi-Bellman equa-tions, arXiu:math/08061837ul[math.PR].
[25]G. Gandolfo, Economic Dynamics, Springer-Verlag,1996.
[26]F. Gozzi and C. Marinelli, Stochastic optimal control of delay equations arising in advertising models, Da Prato, Giuseppe (ed.) et al., Stochastic partial differential equations and applications-VII. Papers of the 7th meeting, Levico, Terme, Italy, January 5-10,2004. Boca Raton, FL:Chapman and Hall/CRC. Lecture Notes in Pure and Applied Mathematics 245(2006),133-148.
[27]S. Hamadene, Nonzero sum linear-quadratic stochastic differential games and backward-forward equations, Stochastic Anal. appl.17(1999),117-130.
[28]Y. Hu and S. Peng, Maximum principle for semilinear stochastic evolution control systems, Stochastic and Stochastic Reports,33(1990),159-180.
[29]Y. Hu and S. Peng, Solution of forward-backward stochastic differential equations, Prob. Theory Rel. Fields,103(2)(1995),273-283.
[30]N. Ikeda and S. Watanabe, Stochastic Differential Equations and Diffusion Pro-cesses, North-Holland Press, Kodansha,1989.
[31]A. F. Ivanov and A. V. Swishchuk, Optimal control of stochastic differential delay equations with application in economics, Conference on Stochastic modelling of Copmlex Systems, Australia,2005.
[32]I. Karatzas, S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag,1991.
[33]Y. I. Kazmerchuk, A. V. Swishchuk, and J. H. Wu, The pricing of options for secu-rities markets with delayed response, Mathematics and Computers in Simulation 75(2007),69-79.
[34]V. B. Kolmanovskii and T. L. Maizenberg, Optimal control of stochastic systems with aftereffect, Avtomat. i Telemeh,1(1973), pp.47-61.
[35]V. B. Kolmanovskii and L.E. Shaikhet, Control of systems with aftereffect, Ttanslations of Mathematical Monographs Vol.157, America Mathematical So-ciety,1996.
[36]H. J. Kushner, Necessary conditions for continuous parameter stochastic optimiza-tion problems, SIAM Journal on Control and Optimization,10(1972),550-556.
[37]B. Larssen, Dynamic programming in stochastic control of systems with delay Stochastics and Stochastics Reports,74(2002),651-673.
[38]B. Larssen, N. H. Risebro, "When are HJB equations for control problems with stochastic delay equations finite dimensional?", Dr. Scient. thesis, University of Oslo,2003.
[39]A. E. B. Lim and X. Zhou, Linear-quadratic control of backward stochastic dif-ferential equations, SIAM J. Control Optim.,40(2001),450-474.
[40]J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differen-tial equations explicitly-a four step scheme, Probab. Theory and Related Fields, 98(1994),339-359.
[41]J. Ma and J. Yong, Forward-backward stochastic differential equations and their applications, Lecture Notes in Mathematics, Springer,1999.
[42]R. C. Merton, Continuous-time Finance, Bssil Blackwell,1991.
[43]S. E. A. Mohammed, Stochastic Functional Differential Equations, Pitman,1984.
[44]S. E. A. Mohammed, Stochastic differential equations with memory:theory, ex-amples and applications, Stochastic Analysis and Related Topics 6, The Geido Workshop,1996, Progress in Probability,.Birkhauser,1998.
[45]M. Nerlove and J. K. Arrow, Optimal advertising policy under dynamic condi-tions, Economica,29(1962),129-142.
[46]D. Nualart, The Malliavin Calculus and Related Topics, Probability and Its Appli-cations, New York and Berlin:Springer-Verlag,1995.
[47]B.(?)ksendal, and A. Sulem, A maximum principle for optimal control of stochastic systems with delay, with applications to finance, In Menaldi JM, Rofman E, Sulem A(eds):Optimal Control and Partial Differential Equations,ISO Press, Amsterdam,2000,64-79.
[48]E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, System and Control Letters,14(1991),55-61.
[49]S. Peng, A general stochastic maximum principle for optimal control problems, SI AM J. Control Optim.,28(1990),966-979.
[50]S. Peng, Probabilistic interpretation for systems of quasilinear parobolic partial differential equations, Stochast. Rep.,37(1991),61-74.
[51]S. Peng, A generalized dynamic programming principle and Hamilton-Jacobi-Bellmen equation, Stochastics Stochastics Rep.,38(1992),119-134.
[52]S. Peng, Backward stochastic differential equation and application to optimal control, Applied Mathematics and Optimization,27(1993),125-144.
[53]S. Peng, Backward stochastic differential equations-stochastic optimization theory and viscosity solution of HJB equations, Topics on Stochastic Analysis, J. Yan, S. Peng, S. Fang, and L. Wu, eds., Science Press, Beijing,1997,85-138(in Chinese).
[54]S. Peng, Z. Wu, Fully coupled forward-backward stochastic differential equations and applications to the optimal control, SI AM J. Control Optim.,37 (3) (1999), 825-843.
[55]S. Peng, and Z. Yang, Anticipated backward stochastic differential equation, An-nals of Probability,37 (2009),877-902.
[56]F. P. Ramsey, A Mathenatical Theory of Savings, Economic J.,38(1928),543-549.
[57]J. Shi and Z. Wu, The maximum principle for fully coupled forward-backward stochastic control systems, Acta Automatica Sinica,32(2)(2006),161-169.
[58]R. B. Vinter and R. H.Kwong, The infinite time quadratic control problem for linear systems with state and control delays:an evolution equation approach, SIAM J. Contr. Optimiz.,19(1981),139-153.
[59]M. L. Vidale and H. B. Wolfe, An operations research study of sales response to advertising, Operations Res.,5(1957),370-381.
[60]G. Wang and Z. Wu, The maximum principles for stochastic recursive optimal control problems under partial information, IEEE Transactions on Automatic Control,54(6)(2009),1230-1242.
[61]Z. Wu, Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems, Systems Sci. Mathe. Sci.,11 (3)(1998),249-259.
[62]Z. Wu, Forward-backward stochastic differential equations, linear quadratic stochastic optimal control and nonzero sum differential games, J. Syst. Sci.& Complexity,18(2) (2005),179-192.
[63]Z. Wu and Z. Yu, FBSDE with Poisson process and linear quadratic nonzero sum differential games with random jumps, Applied Mathematics and Mechanics, 26(2005),1034-1039.
[64]Z. Wu, Z. Yu, Dynamic programming principle for one kind of stochastic recursive optimal control problem and Hamilton-Jacobi-Bellman equation. SIAM J. Control Optim,47(2008),2616-2641.
[65]W. S. Xu, Stochastic maximum principle for optimal control problem of forward and backward system, Journal of Australian Mathematical Society,37(B)(1995), 172-185.
[66]W. S. Xu, Maximum principle for a stochastic optimal control problem and ap-plication to portfolio/consumption choice, Journal of Optimization Theory and Applications,98(1998),719-731.
[67]Z. Yu and S. Ji, Linear-quadratic nonzero-sum differential game of backward stochastic differential equations, Proceedings of the 27th Chinese Control Confer-ence,2008,562-566.
[68]Z. Yu, Some backward problems in stochastic control and game theory, Ph.D. Thesis, Shandong University,2008.
[69]J. Yong, Finding adapted solution of forward backward stochastic differential equations-method of continuation, Prob. Theory Rel. Fields,107(4)(1997),537-572.
[70]J. Yong and X. Y. Zhou, Stochastic Controls:Hamiltonian Systems and HJB Equations, New York:Springer-Verlag,1999.
[2]F. Antonelli, Backward-forward stochastic differential equations, Ann. Appl. Probab.,3(1993),777-793.
[3]M. Arriojas, Y. Hu, S. E. A. Monhammed, and G. Pap, A delayed Black and Scholes formula, Stochastic Analysis and Applications,25(2)(2007),471-492.
[4]M. Basin, J. Rodriguez-Gonzalez, R. Martinez-Zuniga, Optimal control for linear systems with time delay in control input based on the duality principle, American Control Conference, Proceedings of the 2003,3,2003.
[5]G. Barles and E. Lesigne, SDE,BSDE and PDE, Pitman Research Notes in Math-ematics Series,364(1997),47-80.
[6]G. Barles, R. Buckdahn, E. Pardoux, Backward stochastic differential equations and integral-partial differential equations. Stochastics Stochastics Rep.,60(1997), 57-83.
[7]R. Bellman, On the theory of dynamic programming, Proc. Nat. Acad. USA, 38(1952),716-719.
[8]A. Bensoussan, Lectures on stochastic control In:Lecture Notes in Mathematics, 972, Berlin:Springer-Verlag,1982.
[9]J. M. Bismut, An introductory approach to duality in optimal stochastic control, SIAM REW.,20(1)(1978),62-78.
[10]M. Chang, T. Pang and M. Pemy, Optimal control of stochastic functional differ-ential equations with a bounded memory, Stochastic:An international Journal of Probability and Stochastic Process,80(1)(2008),69-96.
[11]M. Chang, Stochastic Control of Hereditary Systems and Applications, New York: Springer,2008.
[12]A. Chojnowska-Michalik, Representation. theorem for general stochastic delay equations, Bull.Acad. Polon. Sci. Ser. Sci. Math. Astronom. Phys.,26(7)(1978), 635-642.
[13]M. G. Crandall and P-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. Amer. Math. Soc.,277(1983),1-42.
[14]M. G. Crandall, H. Ishii and P-L. Lions, User's guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc.,27(1992), 1-67.
[15]J. Cvitanic and J. Ma, Hedging options for a large investor and forward-backward SDE's, Ann. Appl. Probab.,6(2)(1996),370-398.
[16]G. Da Prato and J. Zabczyk, Stochastic equations in infinite dimensions. Cam-bridge UP,1992.
[17]D. Duffie, Dynamic Aasset Picing Theory, Princeton University Press,1992.
[18]D. Duffie and L. G. Epstein, Stochastic differential utility, Economicrica, 60(2)(1992),353-394.
[19]I. Elsanousi, B. Larssen, Optimal consumption under partial observations for a stochastic system with delay, Dr. Scient. thesis, University of Oslo,2001.
[20]N. El Karoui, S. Peng, and M. C. Quenez, Backward stochastic differential equa-tion in finance, Math. Finance,7(1997),1-71.
[21]G. Feichtinger, R. Hartl and S. Sethi, Dynamical optimal control models in ader-tising:recent developments, Management Sci.,40(1994),195-226.
[22]W. H. Fleming, Generalized solutions in optimal stochastic control, in Differential Games and Control Theory 2, (Pro,2nd Kingston Conference, Kingston, RI, 1976), Lecture Notes in pure and Appl. Math.,30(1977),147-165.
[23]M. Fuhrman and G. Tessitore, Nonlinear Kolmogorov equation in infinite di-mensional spaces:the backward stochastic differential equations approach and applications to optimal control, Ann. probab.,30(3)(2002),1397-1465.
[24]M. Fuhrman, F. Masiero and G. Tessitore, Stochastic equations with delay:opti-mal control via BSDEs and regular solutions of Hamilton-Jacobi-Bellman equa-tions, arXiu:math/08061837ul[math.PR].
[25]G. Gandolfo, Economic Dynamics, Springer-Verlag,1996.
[26]F. Gozzi and C. Marinelli, Stochastic optimal control of delay equations arising in advertising models, Da Prato, Giuseppe (ed.) et al., Stochastic partial differential equations and applications-VII. Papers of the 7th meeting, Levico, Terme, Italy, January 5-10,2004. Boca Raton, FL:Chapman and Hall/CRC. Lecture Notes in Pure and Applied Mathematics 245(2006),133-148.
[27]S. Hamadene, Nonzero sum linear-quadratic stochastic differential games and backward-forward equations, Stochastic Anal. appl.17(1999),117-130.
[28]Y. Hu and S. Peng, Maximum principle for semilinear stochastic evolution control systems, Stochastic and Stochastic Reports,33(1990),159-180.
[29]Y. Hu and S. Peng, Solution of forward-backward stochastic differential equations, Prob. Theory Rel. Fields,103(2)(1995),273-283.
[30]N. Ikeda and S. Watanabe, Stochastic Differential Equations and Diffusion Pro-cesses, North-Holland Press, Kodansha,1989.
[31]A. F. Ivanov and A. V. Swishchuk, Optimal control of stochastic differential delay equations with application in economics, Conference on Stochastic modelling of Copmlex Systems, Australia,2005.
[32]I. Karatzas, S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag,1991.
[33]Y. I. Kazmerchuk, A. V. Swishchuk, and J. H. Wu, The pricing of options for secu-rities markets with delayed response, Mathematics and Computers in Simulation 75(2007),69-79.
[34]V. B. Kolmanovskii and T. L. Maizenberg, Optimal control of stochastic systems with aftereffect, Avtomat. i Telemeh,1(1973), pp.47-61.
[35]V. B. Kolmanovskii and L.E. Shaikhet, Control of systems with aftereffect, Ttanslations of Mathematical Monographs Vol.157, America Mathematical So-ciety,1996.
[36]H. J. Kushner, Necessary conditions for continuous parameter stochastic optimiza-tion problems, SIAM Journal on Control and Optimization,10(1972),550-556.
[37]B. Larssen, Dynamic programming in stochastic control of systems with delay Stochastics and Stochastics Reports,74(2002),651-673.
[38]B. Larssen, N. H. Risebro, "When are HJB equations for control problems with stochastic delay equations finite dimensional?", Dr. Scient. thesis, University of Oslo,2003.
[39]A. E. B. Lim and X. Zhou, Linear-quadratic control of backward stochastic dif-ferential equations, SIAM J. Control Optim.,40(2001),450-474.
[40]J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differen-tial equations explicitly-a four step scheme, Probab. Theory and Related Fields, 98(1994),339-359.
[41]J. Ma and J. Yong, Forward-backward stochastic differential equations and their applications, Lecture Notes in Mathematics, Springer,1999.
[42]R. C. Merton, Continuous-time Finance, Bssil Blackwell,1991.
[43]S. E. A. Mohammed, Stochastic Functional Differential Equations, Pitman,1984.
[44]S. E. A. Mohammed, Stochastic differential equations with memory:theory, ex-amples and applications, Stochastic Analysis and Related Topics 6, The Geido Workshop,1996, Progress in Probability,.Birkhauser,1998.
[45]M. Nerlove and J. K. Arrow, Optimal advertising policy under dynamic condi-tions, Economica,29(1962),129-142.
[46]D. Nualart, The Malliavin Calculus and Related Topics, Probability and Its Appli-cations, New York and Berlin:Springer-Verlag,1995.
[47]B.(?)ksendal, and A. Sulem, A maximum principle for optimal control of stochastic systems with delay, with applications to finance, In Menaldi JM, Rofman E, Sulem A(eds):Optimal Control and Partial Differential Equations,ISO Press, Amsterdam,2000,64-79.
[48]E. Pardoux and S. Peng, Adapted solution of a backward stochastic differential equation, System and Control Letters,14(1991),55-61.
[49]S. Peng, A general stochastic maximum principle for optimal control problems, SI AM J. Control Optim.,28(1990),966-979.
[50]S. Peng, Probabilistic interpretation for systems of quasilinear parobolic partial differential equations, Stochast. Rep.,37(1991),61-74.
[51]S. Peng, A generalized dynamic programming principle and Hamilton-Jacobi-Bellmen equation, Stochastics Stochastics Rep.,38(1992),119-134.
[52]S. Peng, Backward stochastic differential equation and application to optimal control, Applied Mathematics and Optimization,27(1993),125-144.
[53]S. Peng, Backward stochastic differential equations-stochastic optimization theory and viscosity solution of HJB equations, Topics on Stochastic Analysis, J. Yan, S. Peng, S. Fang, and L. Wu, eds., Science Press, Beijing,1997,85-138(in Chinese).
[54]S. Peng, Z. Wu, Fully coupled forward-backward stochastic differential equations and applications to the optimal control, SI AM J. Control Optim.,37 (3) (1999), 825-843.
[55]S. Peng, and Z. Yang, Anticipated backward stochastic differential equation, An-nals of Probability,37 (2009),877-902.
[56]F. P. Ramsey, A Mathenatical Theory of Savings, Economic J.,38(1928),543-549.
[57]J. Shi and Z. Wu, The maximum principle for fully coupled forward-backward stochastic control systems, Acta Automatica Sinica,32(2)(2006),161-169.
[58]R. B. Vinter and R. H.Kwong, The infinite time quadratic control problem for linear systems with state and control delays:an evolution equation approach, SIAM J. Contr. Optimiz.,19(1981),139-153.
[59]M. L. Vidale and H. B. Wolfe, An operations research study of sales response to advertising, Operations Res.,5(1957),370-381.
[60]G. Wang and Z. Wu, The maximum principles for stochastic recursive optimal control problems under partial information, IEEE Transactions on Automatic Control,54(6)(2009),1230-1242.
[61]Z. Wu, Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems, Systems Sci. Mathe. Sci.,11 (3)(1998),249-259.
[62]Z. Wu, Forward-backward stochastic differential equations, linear quadratic stochastic optimal control and nonzero sum differential games, J. Syst. Sci.& Complexity,18(2) (2005),179-192.
[63]Z. Wu and Z. Yu, FBSDE with Poisson process and linear quadratic nonzero sum differential games with random jumps, Applied Mathematics and Mechanics, 26(2005),1034-1039.
[64]Z. Wu, Z. Yu, Dynamic programming principle for one kind of stochastic recursive optimal control problem and Hamilton-Jacobi-Bellman equation. SIAM J. Control Optim,47(2008),2616-2641.
[65]W. S. Xu, Stochastic maximum principle for optimal control problem of forward and backward system, Journal of Australian Mathematical Society,37(B)(1995), 172-185.
[66]W. S. Xu, Maximum principle for a stochastic optimal control problem and ap-plication to portfolio/consumption choice, Journal of Optimization Theory and Applications,98(1998),719-731.
[67]Z. Yu and S. Ji, Linear-quadratic nonzero-sum differential game of backward stochastic differential equations, Proceedings of the 27th Chinese Control Confer-ence,2008,562-566.
[68]Z. Yu, Some backward problems in stochastic control and game theory, Ph.D. Thesis, Shandong University,2008.
[69]J. Yong, Finding adapted solution of forward backward stochastic differential equations-method of continuation, Prob. Theory Rel. Fields,107(4)(1997),537-572.
[70]J. Yong and X. Y. Zhou, Stochastic Controls:Hamiltonian Systems and HJB Equations, New York:Springer-Verlag,1999.