Dynamic optimization and its relation to classical and quantum constrained systems

https://doi.org/10.1016/j.physa.2017.02.075Get rights and content

Highlights

  • Dynamic optimization problem.

  • Constrained systems and its applications.

  • Dirac’s method in finance.

  • Quantum mechanics and its applications.

Abstract

We study the structure of a simple dynamic optimization problem consisting of one state and one control variable, from a physicist’s point of view. By using an analogy to a physical model, we study this system in the classical and quantum frameworks. Classically, the dynamic optimization problem is equivalent to a classical mechanics constrained system, so we must use the Dirac method to analyze it in a correct way. We find that there are two second-class constraints in the model: one fix the momenta associated with the control variables, and the other is a reminder of the optimal control law. The dynamic evolution of this constrained system is given by the Dirac’s bracket of the canonical variables with the Hamiltonian. This dynamic results to be identical to the unconstrained one given by the Pontryagin equations, which are the correct classical equations of motion for our physical optimization problem. In the same Pontryagin scheme, by imposing a closed-loop λ-strategy, the optimality condition for the action gives a consistency relation, which is associated to the Hamilton–Jacobi–Bellman equation of the dynamic programming method. A similar result is achieved by quantizing the classical model. By setting the wave function Ψ(x,t)=eiS(x,t) in the quantum Schrödinger equation, a non-linear partial equation is obtained for the S function. For the right-hand side quantization, this is the Hamilton–Jacobi–Bellman equation, when S(x,t) is identified with the optimal value function. Thus, the Hamilton–Jacobi–Bellman equation in Bellman’s maximum principle, can be interpreted as the quantum approach of the optimization problem.

Introduction

We have recently witnessed the increasing application of ideas from physics to finance and economics, such as path integral techniques applied to the study the Black–Scholes model in its different forms  [1], [2], [3], [4], [5], [6], [7]. Some developments have also been used to try to understand the Black–Scholes equation as a quantum mechanical Schrödinger equation  [8], [9], [10]. In the last few years, constrained systems techniques, through Dirac’s method  [11], [12] have been used to explain some features of stochastic volatility models  [13], [14] and the multi-asset Black–Scholes equation  [15]. In this paper, we apply these same constrained methods to understand (from a physical point of view) a particular issue: the dynamic optimization problem.

We start by analyzing the dynamic optimization problem for a single-state variable x and a control variable u. By identifying the state variable x as the coordinate of a physical particle and the Lagrange multiplier λ as its canonical momentum px, we can map the theory in the Hamiltonian phase space. Here, the model presents constraints thus, it is necessary (to study the system in a correct way) to use Dirac’s method of constrained systems. The application of this method implies that constraints are of second-class character according to Dirac’s classification. Thus, the dynamic optimization problem can be seen as a second-class physically constrained system.

We also analyze the role of open-loop and closed-loop strategies in the context of Pontryagin’s framework. We explicitly show that the only consistent strategies that permit the Pontryagin equations to be obtained correctly from the optimization of cost functionals are open-loop λ-strategies (λ=λ(t)). For closed loop λ-strategies (λ=λ(x,t)), the optimization of the cost functional gives a consistency relation which is related to the Hamilton–Jacobi–Bellman equation.

After that, we explore the quantum side of this classically constrained system. By quantizing it according to the standard rules of quantum mechanics and imposing the constraints as operator equations over the wave function, we arrive at a set of partial differential equations for the wave function. After defining the wave function as Ψ(x,t)=eiS(x,t), these equations map into some partial differential equations for the S function. For right-hand side quantization order, these equations give origin to the Hamilton–Jacobi–Bellman equation of the dynamic programming theory. Thus, Bellman’s maximum principle can be considered as the quantum view of the optimization problem.

To make this paper self-contained for non-physicist readers coming from the optimization field, we start with a brief digression on classical and quantum physics in Section  2.

Section snippets

Hamiltonian quantum and classical mechanics

In physics, quantum-dynamic behavior is defined by the Hamiltonian operator. For the simple case of a nonrelativistic one-dimensional particle subjected to external potential U(x), the Hamiltonian operator reads Ȟ=12mP̌x2+U(x)=ħ22m2x2+U(x), where P̌x=iħx is the momentum operator. The wave function at time t (given that the wave function at t=0 is Ψ0) is thus Ψ(x,t)=eiħȞtΨ0(x), which can be written as a convolution according to Ψ(x,t)=K(x,t|x0)Ψ0(x)dx, where the propagator K admits

The Pontryagin approach

Consider an optimal control problem that is commonly used in financial applications (see, for example,  [18]). We want to optimize the cost functional A[x,u]=t0t1F(x,u,t)dt, where x represents a state variable (for example, the production of a certain article) and u is a control variable (such as the marketing cost). The state variable must satisfy the market dynamic ẋ=f(x,u,t). The problem is to determine how to obtain the production trajectory x=x(t) and the control path u=u(t) to optimize

The optimization problem as a classically constrained system

From a structural point of view, the optimization problem is then characterized completely by the Lagrangian multiplier λ. For a open-loop λ-strategy, the optimization of the action (16) gives a system of coupled ordinary differential equations: the Pontryagin equations (18), (19) for both, open or closed-loop optimal u-strategies. For a closed-loop λ-strategy instead, the optimization of the action (16) gives a partial differential equation for λ: the consistency relation (34), which is

Dynamic optimization and quantum mechanics

Until now, we have studied the dynamic optimization problem from a classical point of view, and we have seen that it is equivalent to a classical physically constrained system. However, what happens at the quantum level? To explore that view, we will quantize our classical system and study its consequences.

Again, consider the classical Hamiltonian H0(x,u,λ,px,pu)=F(x,u,t)+f(x,u,t)px. Now, we have to quantize the classical Hamiltonian (53). For this purpose, we replace px,pu with the appropriate

Conclusions

In this article, we have examined the structure of the dynamic optimization problem from a physical perspective, and we conclude that the correct analysis of the optimization problem must be done either in the phase-space or using the classical Hamiltonian approach. Due to the presence of constraints in the theory, we must apply Dirac’s method for constrained systems. Dirac’s analysis implies that the theory has two second-class constraints. One of these constraints fixes the momentum

References (21)

  • M. Contreras et al.

    A quantum model for option pricing: when Black–Scholes meets Schrödinger and its semi-classic limit

    Physica A

    (2010)
  • M. Contreras et al.

    Option pricing, stochastic volatility, singular dynamics and constrained path integrals

    Physica A

    (2014)
  • E. Bennati et al.

    A path integral approach to derivative security pricing I

    Int. J. Theor. Appl. Finance

    (1999)
  • M. Otto, Using path integrals to price interest rate derivatives, 1999....
  • D. Lemmens, M. Wouters, J. Tempere, A path integral approach to closed-form option pricing formulas with applications...
  • B.E. Baaquie, Quantum Finance: Path Integrals and Hamiltonians for Option and Interest Rates, Cambridge,...
  • B.E. Baaquie

    A path integral to option price with stochastic volatility: Some exact results

    J. Phys. I France

    (1997)
  • V. Linetsky

    The path integral approach to financial modelling and option pricing

    Comput. Econ.

    (1998)
  • Jan W. Dash

    Quantitative Finance and Risk Management: A Physicist Approach

    (2004)
  • E. Haven

    A Black–Scholes Schrödinger option price: “bit” versus “qubit”

    Physica A

    (2003)
There are more references available in the full text version of this article.

Cited by (11)

  • The analysis of the dynamic optimization problem in econophysics from the point of view of the symplectic approach for constrained systems

    2021, Chaos, Solitons and Fractals
    Citation Excerpt :

    Historically, the word econophysics was used for the first time in Kolkata in 1995 [1], when Indian economists and physicists considered as possible the formulation of some of the economic problems and their solutions using traditional physics approaches from physics. Today [2], many of these approaches usually applied in theoretical physics like quantum mechanics, statistical mechanics, mathematical physics and more, are being also used to understand many problems in economy and financial markets [3,4]. Many times we came across approaches based on Hamilton’s principle and more generally, Dirac-Bergman constrained systems algorithm [5], as we will explain in a few moments.

  • Second class constraints and the consistency of optimal control theory in phase space

    2021, Physica A: Statistical Mechanics and its Applications
    Citation Excerpt :

    Unfortunately, the Rothe–Scholtz method works only when the Dirac matrix is constant [11,12]. Now, considering the optimal problem in terms of a Hamiltonian formulation, unfortunately the occurrence of second class constraints can be proven [5–8]. In fact, the Dirac matrix is equal to the second derivative of the Hamiltonian with respect to the control variable.

  • The quantum dark side of the optimal control theory

    2019, Physica A: Statistical Mechanics and its Applications
View all citing articles on Scopus
View full text