We consider the problem of steering a linear dynamical system with complete state observation from an initial Gaussian distribution in state-space to a final one with minimum energy control. The ...system is stochastically driven through the control channels; an example for such a system is that of an inertial particle experiencing random "white noise" forcing. We show that a target probability distribution can always be achieved in finite time. The optimal control is given in state-feedback form and is computed explicitly by solving a pair of differential Lyapunov equations that are nonlinearly coupled through their boundary values. This result, given its attractive algorithmic nature, appears to have several potential applications such as to quality control, control of industrial processes, as well as to active control of nanomechanical systems and molecular cooling. The problem to steer a diffusion process between end-point marginals has a long history (Schrödinger bridges) and the present case of steering a linear stochastic system constitutes such a Schrödinger bridge for possibly degenerate diffusions. Our results provide the first implementable form of the optimal control for a general Gauss-Markov process. Illustrative examples are provided for steering inertial particles and for "cooling" a stochastic oscillator. A final result establishes directly the property of Schrödinger bridges as the most likely random evolution between given marginals to the present context of linear stochastic systems. A second part to this work, that is to appear as part II, addresses the general situation where the stochastic excitation enters through channels that may differ from those used to control.
We take a new look at the relation between the optimal transport problem and the Schrödinger bridge problem from a stochastic control perspective. Our aim is to highlight new connections between the ...two that are richer and deeper than those previously described in the literature. We begin with an elementary derivation of the Benamou–Brenier fluid dynamic version of the optimal transport problem and provide, in parallel, a new fluid dynamic version of the Schrödinger bridge problem. We observe that the latter establishes an important connection with optimal transport without zero-noise limits and solves a question posed by Eric Carlen in 2006. Indeed, the two variational problems differ by a
Fisher information functional
. We motivate and consider a generalization of optimal mass transport in the form of a (fluid dynamic) problem of
optimal transport with prior
. This can be seen as the zero-noise limit of Schrödinger bridges when the prior is any Markovian evolution. We finally specialize to the Gaussian case and derive an explicit computational theory based on matrix Riccati differential equations. A numerical example involving Brownian particles is also provided.
We consider the problem of steering an initial probability density for the state vector of a linear system to a final one, in finite time, using minimum energy control. In the case where the dynamics ...correspond to an integrator (ẋ(t) = u(t)) this amounts to a Monge-Kantorovich Optimal Mass Transport (OMT) problem. In general, we show that the problem can again be reduced to solving an OMT problem and that it has a unique solution. In parallel, we study the optimal steering of the state-density of a linear stochastic system with white noise disturbance; this is known to correspond to a Schrödinger bridge. As the white noise intensity tends to zero, the flow of densities converges to that of the deterministic dynamics and can serve as a way to compute the solution of its deterministic counterpart. The solution can be expressed in closed-form for Gaussian initial and final state densities in both cases.
Monge–Kantorovich optimal mass transport (OMT) provides a blueprint for geometries in the space of positive densities—it quantifies the cost of transporting a mass distribution into another. In ...particular, it provides natural options for interpolation of distributions (displacement interpolation) and for modeling flows. As such it has been the cornerstone of recent developments in physics, probability theory, image processing, time-series analysis, and several other fields. In spite of extensive work and theoretical developments, the computation of OMT for large-scale problems has remained a challenging task. An alternative framework for interpolating distributions, rooted in statistical mechanics and large deviations, is that of the Schrödinger bridge problem (SBP), which leads to entropic interpolation. SBP may be seen as a stochastic regularization of OMT, and can be cast as the stochastic control problem of steering the probability density of the state-vector of a dynamical system between two marginals. The actual computation of entropic flows, however, has received hardly any attention. In our recent work on Schrödinger bridges for Markov chains and quantum channels, we showed that the solution can be efficiently obtained from the fixed point of a map which is contractive in the Hilbert metric. Thus, the purpose of this paper is to show that a similar approach can be taken in the context of diffusion processes which (i) leads to a new proof of a classical result on SBP and (ii) provides an efficient computational scheme for both SBP and OMT. We illustrate this new computational approach by obtaining interpolation of densities in representative examples such as interpolation of images.
We show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite- and infinite-dimensional maximum entropy problems concerning probability ...distributions, spectral densities, and covariance matrices. These include Burg's spectral estimation method and Dempster's covariance completion, as well as various recent generalizations of the above. We then apply this orthogonality principle to the new problem of completing a block-circulant covariance matrix when an a priori estimate is available.
The purpose of this work is to pose and solve the problem to guide a collection of weakly interacting dynamical systems (agents, particles, etc.) to a specified terminal distribution. This is ...formulated as a mean-field game problem, and is discussed in both non-cooperative games and cooperative games settings. In the non-cooperative games setting, a terminal cost is used to accomplish the task; we establish that the map between terminal costs and terminal probability distributions is onto. In the cooperative games setting, the goal is to find a common optimal control that would drive the distribution of the agents to a targeted one. We focus on the cases when the underlying dynamics is linear and the running cost is quadratic. Our approach relies on and extends the theory of optimal mass transport and its generalizations.
In this letter, we connect some recent papers on smoothing of energy landscapes and scored-based generative models of machine learning to classical work in stochastic control. We clarify these ...connections providing rigorous statements and representations which may serve as guidelines for further learning models.
In the early 1930s, Erwin Schrödinger, motivated by his quest for a more classical formulation of quantum mechanics, posed a large deviation problem for a cloud of independent Brownian particles. He ...showed that the solution to the problem could be obtained through a system of two linear equations with nonlinear coupling at the boundary (
Schrödinger system
). Existence and uniqueness for such a system, which represents a sort of bottleneck for the problem, was first established by Fortet in 1938/1940 under rather general assumptions by proving convergence of an ingenious but complex approximation method. It is the first proof of what are nowadays called Sinkhorn-type algorithms in the much more challenging continuous case. Schrödinger bridges are also an early example of the maximum entropy approach and have been more recently recognized as a regularization of the important optimal mass transport problem. Unfortunately, Fortet’s contribution is by and large ignored in contemporary literature. This is likely due to the complexity of his approach coupled with an idiosyncratic exposition style and due to missing details and steps in the proofs. Nevertheless, Fortet’s approach maintains its importance to this day as it provides the only existing algorithmic proof, in the continuous setting, under rather mild assumptions. It can be adapted, in principle, to other relevant optimal transport problems. It is the purpose of this paper to remedy this situation by rewriting the bulk of his paper with all the missing passages and in a transparent fashion so as to make it fully available to the scientific community. We consider the problem in
R
d
rather than in
R
and use as much as possible his notation to facilitate comparison.
The subject of this work has its roots in the so-called Schrödginer bridge problem (SBP) which asks for the most likely distribution of Brownian particles in their passage between observed empirical ...marginal distributions at two distinct points in time. Renewed interest in this problem was sparked by a reformulation in the language of stochastic control. In earlier works, presented as Part I and Part II, we explored a generalization of the original SBP that amounts to optimal steering of linear stochastic dynamical systems between state-distributions, at two points in time, under full state feedback. In these works, the cost was quadratic in the control input, i.e., control energy. The purpose of the present work is to detail the technical steps in extending the framework to the case where a quadratic cost in the state is also present. Thus, the main contribution is to derive the optimal control in this case which in fact is given in closed-form (Theorem <xref ref-type="disp-formula" rid="deqn1a-deqn1c">1 ). In the zero-noise limit, we also obtain the solution of a (deterministic) mass transport problem with general quadratic cost.
PDF
