Special Applied Math Seminar: SOC-MartNet: A Martingale Neural Network for the Hamilton-Jacobi-Bellman Equation without Explicit inf H in Stochastic Optimal Controls

In this talk, we will present a martingale based neural network, SOC-MartNet, for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations where no explicit expression is needed for the Hamiltonian $\inf_{u \in U} H(t,x,u, z,p)$, and stochastic optimal control problems (SOCP) with controls on both drift and volatility.We reformulate the HJB equations into a stochastic neural network learning process, i.e., training a control network and a value network such that the associated Hamiltonian process is minimized and the cost process becomes a martingale. To enforce the martingale property for the cost process, we employ an adversarial network and construct a loss function based on the projection property of conditional expectations. Then, the control/value networks and the adversarial network are trained adversarially, such that the cost process is driven towards a martingale and the minimum principle is satisfied for the control. Numerical results show that the proposed SOC-MartNet is effective and efficient for solving HJB-type equations and SOCP with a dimension up to d=500 in a small number of training epochs (<100).(Joint work with Shuxin Fang and Tao Zhou).