The Kuramoto model of coupled phase oscillators is often used to describe synchronization phenomena in nature. Some applications, e.g., quantum synchronization and rigid-body attitude ...synchronization, involve high-dimensional Kuramoto models where each oscillator lives on the n-sphere or SO(n). These manifolds are special cases of the compact, real Stiefel manifold St(p,n). Using tools from optimization and control theory, we prove that the generalized Kuramoto model on St(p,n) converges to a synchronized state for any connected graph and from almost all initial conditions provided (p,n) satisfies p≤23n−1 and all oscillator frequencies are equal. This result could not have been predicted based on knowledge of the Kuramoto model in complex networks over the circle. In that case, almost global synchronization is graph dependent; it applies if the network is acyclic or sufficiently dense. This paper hence identifies a property that distinguishes many high-dimensional generalizations of the Kuramoto models from the original model.
This work focuses on the representation of model-form uncertainties in molecular dynamics simulations in various statistical ensembles. In prior contributions, the modeling of such uncertainties was ...formalized and applied to quantify the impact of, and the error generated by, pair-potential selection in the microcanonical ensemble (NVE). In this work, we extend this formulation and present a linear-subspace reduced-order model for the canonical (NVT) and isobaric (NPT) ensembles. The symplectic reduced-order basis is randomized on the tangent space of the Stiefel manifold to provide topological relationships and capture model-form uncertainty. Using the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS), we assess the relevance of these stochastic reduced-order atomistic models on canonical problems involving a Lennard-Jones fluid and an argon crystal melt.
Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric ...(generalized) eigenvalue problems, in nonlinear eigenvalue problems, and in electronic structures computations, to name a few problems. In statistics and machine learning, they arise, for example, in various dimensionality reduction techniques such as canonical correlation analysis. In deep learning, regularization and improved stability can be obtained by constraining some layers to have parameter matrices that belong to the Stiefel manifold. Solving problems on the generalized Stiefel manifold can be approached via the tools of Riemannian optimization. However, using the standard geometric components for the generalized Stiefel manifold has two possible shortcomings: computing some of the geometric components can be too expensive and convergence can be rather slow in certain cases. Both shortcomings can be addressed using a technique called Riemannian preconditioning, which amounts to using geometric components derived by a preconditioner that defines a Riemannian metric on the constraint manifold. In this paper we develop the geometric components required to perform Riemannian optimization on the generalized Stiefel manifold equipped with a non-standard metric, and illustrate theoretically and numerically the use of those components and the effect of Riemannian preconditioning for solving optimization problems on the generalized Stiefel manifold.
Uncertainty quantification (UQ) tasks, such as model calibration, uncertainty propagation, and optimization under uncertainty, typically require several thousand evaluations of the underlying ...computer codes. To cope with the cost of simulations, one replaces the real response surface with a cheap surrogate based, e.g., on polynomial chaos expansions, neural networks, support vector machines, or Gaussian processes (GP). However, the number of simulations required to learn a generic multivariate response grows exponentially as the input dimension increases. This curse of dimensionality can only be addressed, if the response exhibits some special structure that can be discovered and exploited. A wide range of physical responses exhibit a special structure known as an active subspace (AS). An AS is a linear manifold of the stochastic space characterized by maximal response variation. The idea is that one should first identify this low dimensional manifold, project the high-dimensional input onto it, and then link the projection to the output. If the dimensionality of the AS is low enough, then learning the link function is a much easier problem than the original problem of learning a high-dimensional function. The classic approach to discovering the AS requires gradient information, a fact that severely limits its applicability. Furthermore, and partly because of its reliance to gradients, it is not able to handle noisy observations. The latter is an essential trait if one wants to be able to propagate uncertainty through stochastic simulators, e.g., through molecular dynamics codes. In this work, we develop a probabilistic version of AS which is gradient-free and robust to observational noise. Our approach relies on a novel Gaussian process regression with built-in dimensionality reduction. In particular, the AS is represented as an orthogonal projection matrix that serves as yet another covariance function hyper-parameter to be estimated from the data. To train the model, we design a two-step maximum likelihood optimization procedure that ensures the orthogonality of the projection matrix by exploiting recent results on the Stiefel manifold, i.e., the manifold of matrices with orthogonal columns. The additional benefit of our probabilistic formulation, is that it allows us to select the dimensionality of the AS via the Bayesian information criterion. We validate our approach by showing that it can discover the right AS in synthetic examples without gradient information using both noiseless and noisy observations. We demonstrate that our method is able to discover the same AS as the classical approach in a challenging one-hundred-dimensional problem involving an elliptic stochastic partial differential equation with random conductivity. Finally, we use our approach to study the effect of geometric and material uncertainties in the propagation of solitary waves in a one dimensional granular system.
Distributed optimization aims to effectively complete specified tasks through cooperation among multi-agent systems, which has achieved great success in large-scale optimization problems. However, it ...remains a challenging task to develop an effective distributed algorithm with theoretical guarantees, especially when dealing with nonconvex constraints. More importantly, high-dimensional data often exhibits inherent structures such as sparsity, which if exploited accurately, can significantly enhance the capture of its intrinsic characteristics. In this paper, we introduce a novel distributed sparsity constrained optimization framework over the Stiefel manifold, abbreviated as DREAM. DREAM innovatively integrates the ℓ2,0-norm constraint and Stiefel manifold constraint within a distributed optimization setting, which has not been investigated in existing literature. Unlike the existing distributed methods, the proposed DREAM not only can extract the similarity information among samples, but also more flexibly determine the number of features to be extracted. Then, we develop an efficient Newton augmented Lagrangian-based algorithm. In theory, we delve into the relationship between the minimizer, the Karush–Kuhn–Tucker point, and the stationary point, and rigorously demonstrate that the sequence generated by our algorithm converges to a stationary point. Extensive numerical experiments verify its superiority over state-of-the-art distributed methods.
For a projection-based reduced order model (ROM) of a fluid flow to be stable and accurate, the dynamics of the truncated subspace must be taken into account. This paper proposes an approach for ...stabilizing and enhancing projection-based fluid ROMs in which truncated modes are accounted for a priori via a minimal rotation of the projection subspace. Attention is focused on the full non-linear compressible Navier–Stokes equations in specific volume form as a step toward a more general formulation for problems with generic non-linearities. Unlike traditional approaches, no empirical turbulence modeling terms are required, and consistency between the ROM and the Navier–Stokes equation from which the ROM is derived is maintained. Mathematically, the approach is formulated as a trace minimization problem on the Stiefel manifold. The reproductive as well as predictive capabilities of the method are evaluated on several compressible flow problems, including a problem involving laminar flow over an airfoil with a high angle of attack, and a channel-driven cavity flow problem.
Matrix completion, aiming at restoring a low-rank matrix from observed entries, indicates the connection with the subspace clustering due to the low-rank property. However, it is expensive to ...incorporate subspace learning into the pervasive surrogate of matrix completion, the nuclear norm. In this paper, we design an orthogonal subspace exploration model for matrix completion, which can be easily integrated due to the succinct formulation. Then, we propose a non-convex surrogate with tractable solutions for low-rank matrix completion, so that the subspace exploration can be performed simultaneously. Compared with the existing surrogates (e.g., nuclear norm, Schatten-p norm, max norm, etc.), the proposed surrogate is differential such that the optimization is still simple even after the subspace exploration is incorporated. Although the surrogate is non-convex, a parameter-free algorithm that is proved to converge into the global optimum is developed. The optimization consists of closed-form solutions so that the orthogonal subspace exploration will not distinctly bring additional costs and the algorithm empirically converges within dozens of iterations. Experiments illustrate the efficiency and superiority of our model.
•A tractable orthogonal subspace exploration is designed for matrix completion.•A smooth non-convex surrogate is proposed to replace the classic nuclear norm.•The optimization is proved to converge into the global minimum.•The optimization of surrogate usually converges within dozens of iterations.•The surrogate is expandable so it is still fast with additionl subspace exploration.
PDF
