Journal Home Page

Cumulative Index

List of all Volumes

Complete Contents
of this Volume

Previous Article

Next Article

Journal of Convex Analysis 30 (2023), No. 3, 897--916
Copyright Heldermann Verlag 2023

Dynamic Programming For Data Independent Decision Sets

Paul Dommel
Faculty of Mathematics, University of Technology, Chemnitz, Germany

Alois Pichler
Faculty of Mathematics, University of Technology, Chemnitz, Germany

Multistage stochastic optimization problems are oftentimes formulated informally in a pathwise way. These formulations are appealing in a discrete setting and suitable when addressing computational challenges, for example. But the pathwise problem statement does not allow an analysis with mathematical rigor and is therefore not appropriate.
R. T. Rockafellar and R. J.-B. Wets [Nonanticipativity and L1-martingales in stochastic optimization problems, Math. Programming Study 6 (1976) 170--187] address the fundamental measurability concern of the value functions in the case of convex costs and constraints. This paper resumes these foundations. The contribution is a proof that there exist measurable versions of intermediate value functions, which reveals regularity in addition. Our proof builds on the Kolmogorov continuity theorem.
It is demonstrated that verification theorems allow stating traditional problem specifications in the novel setting with mathematical rigor. Further, we provide dynamic equations for the general problem. The problem classes covered include Markov decision processes, reinforcement learning and stochastic dual dynamic programming.

Keywords: Multistage stochastic optimization, stochastic processes, measurability.

MSC: 90C08, 90C15, 60G07.

[ Fulltext-pdf  (149  KB)] for subscribers only.