Oswin So 蘇雋文 (苏隽文)

Publications

2025

1. RSS 2025

   Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL

   Songyuan* Zhang, Oswin* So, Mitchell Black, Zachary Serlin, and Chuchu Fan

   Robotics: Science and Systems , 2025

   Website arXiv Cite Abstract

   Tasks for multi-robot systems often require the robots to collaborate and complete a team goal while maintaining safety. This problem is usually formalized as a Constrained Markov decision process (CMDP), which targets minimizing a global cost and bringing the mean of constraint violation below a user-defined threshold. Inspired by real-world robotic applications, we define safety as zero constraint violation. While many safe multi-agent reinforcement learning (MARL) algorithms have been proposed to solve CMDPs, these algorithms suffer from unstable training in this setting. To tackle this, we use the epigraph form for constrained optimization to improve training stability and prove that the centralized epigraph form problem can be solved in a distributed fashion by each agent. This results in a novel centralized training distributed execution MARL algorithm which we name Def-MARL. Simulation experiments on 8 different tasks across 2 different simulators show that Def-MARL achieves the best overall performance, satisfies safety constraints, and maintains stable training. Real-world hardware experiments on Crazyflie quadcopters demonstrate the ability of Def-MARL to safely coordinate agents to complete complex collaborative tasks compared to other methods.

   @inproceedings{zhang2025solving,
     title={Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL},
     author={Zhang, Songyuan* and So, Oswin* and Black, Mitchell and Serlin, Zachary and Fan, Chuchu},
     booktitle={Robotics: Science and Systems},
     year={2025},
   }

2. RSS 2025

   Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functions

   Yin* Ji, Oswin* So, Eric Yu, Chuchu Fan, and Panagiotis Tsiotras

   Robotics: Science and Systems , 2025

   arXiv Cite Abstract

   A common problem when using model predictive control (MPC) in practice is the satisfaction of safety beyond the prediction horizon. While theoretical works have shown that safety can be guaranteed by enforcing a suitable terminal set constraint or a sufficiently long prediction horizon, these techniques are difficult to apply and thus rarely used by practitioners, especially in the case of general nonlinear dynamics. To solve this, we make a tradeoff between exact recursive feasibility, computational tractability, and applicability to black-box dynamics by learning an approximate discrete-time control barrier function and incorporating it into variational inference MPC (VIMPC), a sampling-based MPC paradigm. To handle the resulting state constraints, we further propose a new sampling strategy that greatly reduces the variance of the estimated optimal control, improving the sample efficiency and enabling real-time planning on CPU. The resulting Neural Shield-VIMPC (NS-VIMPC) controller yields substantial safety improvements compared to existing sampling-based MPC controllers, even under badly designed cost functions. We validate our approach in both simulation and real-world hardware experiments.

   @inproceedings{yin2025safe,
     title={Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functions},
     author={Ji, Yin* and So, Oswin* and Yu, Eric and Fan, Chuchu and Tsiotras, Panagiotis},
     booktitle={Robotics: Science and Systems},
     year={2025},
   }

3. Automatica

   Comment(s) on “Control barrier functions for stochastic systems” [Automatica 130 (2021) 109688]

   Oswin So, and Chuchu Fan

   Automatica

   Website Cite Abstract

   It is shown that Theorem 3 of Clark (2021) is flawed. The proof of Theorem 2 in Clark (2021) relies on the same proof technique and is thus also flawed.

   @article{so2025comment,
     title={Comment(s) on “Control barrier functions for stochastic systems” [Automatica 130 (2021) 109688]},
     author={So, Oswin and Fan, Chuchu},
     journal={Automatica},
     year={2025},
   }

4. ICLR 2025

   Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control

   Songyuan Zhang*, Oswin So*, Mitchell Black, and Chuchu Fan

   The Thirteenth International Conference on Learning Representations

   Website Cite Abstract PDF

   Control policies that can achieve high task performance and satisfy safety contraints are desirable for any system, including multi-agent systems (MAS). One promising technique for ensuring the safety of MAS is distributed control barrier functions (CBF). However, it is difficult to design distributed CBF-based policies for MAS that can tackle unknown discrete-time dynamics, partial observability, changing neighborhoods, and input constraints, especially when a distributed high-performance nominal policy that can achieve the task is unavailable. To tackle these challenges, we propose DGPPO, a new framework that simultaneously learns both a discrete graph CBF which handles neighborhood changes and input constraints, and a distributed high-performance safe policy for MAS with unknown discrete-time dynamics. We empirically validate our claims on a suite of multi-agent tasks spanning three different simulation engines. The results suggest that, compared with existing methods, our DGPPO framework obtains policies that achieve high task performance (matching baselines that ignore the safety constraints), and high safety rates (matching the most conservative baselines), with a constant set of hyperparameters across all environments.

   @inproceedings{zhang2025discrete,
     title={Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control},
     author={Zhang*, Songyuan and So*, Oswin and Black, Mitchell and Fan, Chuchu},
     booktitle={The Thirteenth International Conference on Learning Representations},
     year={2025},
   }

2024

1. T-RO

   GCBF+: A neural graph control barrier function framework for distributed safe multi-agent control

   Songyuan Zhang*, Oswin So*, Kunal Garg, and Chuchu Fan

   IEEE Transactions on Robotics

   Website arXiv Cite Abstract

   Distributed, scalable, and safe control of large-scale multi-agent systems (MAS) is a challenging problem. In this paper, we design a distributed framework for safe multi-agent control in large-scale environments with obstacles, where a large number of agents are required to maintain safety using only local information and reach their goal locations. We introduce a new class of certificates, termed graph control barrier function (GCBF), which are based on the well-established control barrier function (CBF) theory for safety guarantees and utilize a graph structure for scalable and generalizable distributed control of MAS. We develop a novel theoretical framework to prove the safety of an arbitrary-sized MAS with a single GCBF. We propose a new training framework GCBF+ that uses graph neural networks (GNNs) to parameterize a candidate GCBF and a distributed control policy. The proposed framework is distributed and is capable of directly taking point clouds from LiDAR, instead of actual state information, for real-world robotic applications. We illustrate the efficacy of the proposed method through various hardware experiments on a swarm of drones with objectives ranging from exchanging positions to docking on a moving target without collision. Additionally, we perform extensive numerical experiments, where the number and density of agents, as well as the number of obstacles, increase. Empirical results show that in complex environments with nonlinear agents (e.g., Crazyflie drones) GCBF+ outperforms the handcrafted CBF-based method with the best performance by up to 20% for relatively small-scale MAS for up to 256 agents, and leading reinforcement learning (RL) methods by up to 40% for MAS with 1024 agents. Furthermore, the proposed method does not compromise on the performance, in terms of goal reaching, for achieving high safety rates, which is a common trade-off in RL-based methods.

   @article{zhang2024gcbf+,
     title={GCBF+: A neural graph control barrier function framework for distributed safe multi-agent control},
     author={Zhang*, Songyuan and So*, Oswin and Garg, Kunal and Fan, Chuchu},
     journal={IEEE Transactions on Robotics},
     year={2024},
   }

2. RPCBF: Constructing Safety Filters Robust to Model Error and Disturbances via Policy Control Barrier Functions

   Luzia Knoedler*, Oswin So*, Ji Yin, Mitchell Black, Zachary Serlin, Panagiotis Tsiotras, Javier Alonso-Mora, and Chuchu Fan

   2025 IEEE International Conference on Robotics and Automation (ICRA) (Under Review)

   Website Cite PDF

   @inproceedings{knoedler2025rpcbf,
     title={RPCBF: Constructing Safety Filters Robust to Model Error and Disturbances via Policy Control Barrier Functions},
     author={Knoedler*, Luzia and So*, Oswin and Yin, Ji and Black, Mitchell and Serlin, Zachary and Tsiotras, Panagiotis and Alonso-Mora, Javier and Fan, Chuchu},
     booktitle={2025 IEEE International Conference on Robotics and Automation (ICRA) (Under Review)},
     year={2024},
   }

3. NeurIPS 2024

   Solving Minimum-Cost Reach Avoid using Reinforcement Learning

   Oswin So*, Cheng Ge*, and Chuchu Fan

   Thirty-Eighth Conference on Neural Information Processing Systems , 2024

   Website Cite Abstract PDF

   Current reinforcement-learning methods are unable to directly learn policies that solve the minimum cost reach-avoid problem to minimize cumulative costs subject to the constraints of reaching the goal and avoiding unsafe states, as the structure of this new optimization problem is incompatible with current methods. Instead, a surrogate problem is solved where all objectives are combined with a weighted sum. However, this surrogate objective results in suboptimal policies that do not directly minimize the cumulative cost. In this work, we propose RC-PPO, a reinforcement-learning-based method for solving the minimum-cost reach-avoid problem by using connections to Hamilton-Jacobi reachability. Empirical results demonstrate that RC-PPO learns policies with comparable goal-reaching rates to while achieving up to 57% lower cumulative costs compared to existing methods on a suite of minimum-cost reach-avoid benchmarks on the Mujoco simulator.

   @inproceedings{so2024solving,
     title={Solving Minimum-Cost Reach Avoid using Reinforcement Learning},
     author={So*, Oswin and Ge*, Cheng and Fan, Chuchu},
     booktitle={Thirty-Eighth Conference on Neural Information Processing Systems},
     year={2024},
   }

4. ARC

   Learning safe control for multi-robot systems: Methods, verification, and open challenges

   Kunal Garg, Songyuan Zhang, Oswin So, Charles Dawson, and Chuchu Fan

   Annual Reviews in Control

   arXiv Cite Abstract

   In this survey, we review the recent advances in control design methods for robotic multi-agent systems (MAS), focussing on learning-based methods with safety considerations. We start by reviewing various notions of safety and liveness properties, and modeling frameworks used for problem formulation of MAS. Then we provide a comprehensive review of learning-based methods for safe control design for multi-robot systems. We start with various types of shielding-based methods, such as safety certificates, predictive filters, and reachability tools. Then, we review the current state of control barrier certificate learning in both a centralized and distributed manner, followed by a comprehensive review of multi-agent reinforcement learning with a particular focus on safety. Next, we discuss the state-of-the-art verification tools for the correctness of learning-based methods. Based on the capabilities and the limitations of the state of the art methods in learning and verification for MAS, we identify various broad themes for open challenges: how to design methods that can achieve good performance along with safety guarantees; how to decompose single-agent based centralized methods for MAS; how to account for communication-related practical issues; and how to assess transfer of theoretical guarantees to practice.

   @article{garg2024learning,
     title={Learning safe control for multi-robot systems: Methods, verification, and open challenges},
     author={Garg, Kunal and Zhang, Songyuan and So, Oswin and Dawson, Charles and Fan, Chuchu},
     journal={Annual Reviews in Control},
     year={2024},
   }

5. ICRA 2024

   How to train your neural control barrier function: Learning safety filters for complex input-constrained systems

   Oswin So, Zachary Serlin, Makai Mann, Jake Gonzales, Kwesi Rutledge, Nicholas Roy, and Chuchu Fan

   2024 International Conference on Robotics and Automation (ICRA)

   Website arXiv Cite Abstract

   Control barrier functions (CBF) have become popular as a safety filter to guarantee the safety of nonlinear dynamical systems for arbitrary inputs. However, it is difficult to construct functions that satisfy the CBF constraints for high relative degree systems with input constraints. To address these challenges, recent work has explored learning CBFs using neural networks via neural CBF (NCBF). However, such methods face difficulties when scaling to higher dimensional systems under input constraints. In this work, we first identify challenges that NCBFs face during training. Next, to address these challenges, we propose policy neural CBF (PNCBF), a method of constructing CBFs by learning the value function of a nominal policy, and show that the value function of the maximum-over-time cost is a CBF. We demonstrate the effectiveness of our method in simulation on a variety of systems ranging from toy linear systems to an F-16 jet with a 16-dimensional state space. Finally, we validate our approach on a two-agent quadcopter system on hardware under tight input constraints.

   @inproceedings{so2024train,
     title={How to train your neural control barrier function: Learning safety filters for complex input-constrained systems},
     author={So, Oswin and Serlin, Zachary and Mann, Makai and Gonzales, Jake and Rutledge, Kwesi and Roy, Nicholas and Fan, Chuchu},
     booktitle={2024 International Conference on Robotics and Automation (ICRA)},
     year={2024},
   }

2023

1. RSS 2023

   Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learning

   Oswin So, and Chuchu Fan

   Robotics: Science and Systems , 2023

   Website arXiv Cite Abstract

   Tasks for autonomous robotic systems commonly require stabilization to a desired region while maintaining safety specifications. However, solving this multi-objective problem is challenging when the dynamics are nonlinear and high-dimensional, as traditional methods do not scale well and are often limited to specific problem structures. To address this issue, we propose a novel approach to solve the stabilize-avoid problem via the solution of an infinite-horizon constrained optimal control problem (OCP). We transform the constrained OCP into epigraph form and obtain a two-stage optimization problem that optimizes over the policy in the inner problem and over an auxiliary variable in the outer problem. We then propose a new method for this formulation that combines an on-policy deep reinforcement learning algorithm with neural network regression. Our method yields better stability during training, avoids instabilities caused by saddle-point finding, and is not restricted to specific requirements on the problem structure compared to more traditional methods. We validate our approach on different benchmark tasks, ranging from low-dimensional toy examples to an F16 fighter jet with a 17-dimensional state space. Simulation results show that our approach consistently yields controllers that match or exceed the safety of existing methods while providing ten-fold increases in stability performance from larger regions of attraction.

   @inproceedings{so2023solving,
     title={Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learning},
     author={So, Oswin and Fan, Chuchu},
     booktitle={Robotics: Science and Systems},
     year={2023},
   }

2. ICRA 2023

   MPOGames: Efficient Multimodal Partially Observable Dynamic Games

   Oswin So, Paul Drews, Thomas Balch, Velin Dimitrov, Guy Rosman, and Evangelos A. Theodorou

   2023 IEEE International Conference on Robotics and Automation (ICRA)

   arXiv Cite Abstract

   Game theoretic methods have become popular for planning and prediction in situations involving rich multi-agent interactions. However, these methods often assume the existence of a single local Nash equilibria and are hence unable to handle uncertainty in the intentions of different agents. While maximum entropy (MaxEnt) dynamic games try to address this issue, practical approaches solve for MaxEnt Nash equilibria using linear-quadratic approximations which are restricted to unimodal responses and unsuitable for scenarios with multiple local Nash equilibria. By reformulating the problem as a POMDP, we propose MPOGames, a method for efficiently solving MaxEnt dynamic games that captures the interactions between local Nash equilibria. We show the importance of uncertainty-aware game theoretic methods via a two-agent merge case study. Finally, we prove the real-time capabilities of our approach with hardware experiments on a 1/10th scale car platform.

   @inproceedings{so2022mpogames,
     title={MPOGames: Efficient Multimodal Partially Observable Dynamic Games},
     author={So, Oswin and Drews, Paul and Balch, Thomas and Dimitrov, Velin and Rosman, Guy and Theodorou, Evangelos A.},
     booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
     year={2023},
   }

2022

1. ML4PS 2022

   Data-driven discovery of non-Newtonian astronomy via learning non-Euclidean Hamiltonian

   Oswin So, Gongjie Li, Evangelos A Theodorou, and Molei Tao

   Machine Learning and the Physical Sciences Workshop NeurIPS , 2022

   arXiv Cite Abstract

   Incorporating the Hamiltonian structure of physical dynamics into deep learning models provides a powerful way to improve the interpretability and prediction accuracy. While previous works are mostly limited to the Euclidean spaces, their extension to the Lie group manifold is needed when rotations form a key component of the dynamics, such as the higher-order physics beyond simple point-mass dynamics for N-body celestial interactions. Moreover, the multiscale nature of these processes presents a challenge to existing methods as a long time horizon is required. By leveraging a symplectic Lie-group manifold preserving integrator, we present a method for data-driven discovery of non-Newtonian astronomy. Preliminary results show the importance of both these properties in training stability and prediction accuracy.

   @inproceedings{so2022data,
     title={Data-driven discovery of non-Newtonian astronomy via learning non-Euclidean Hamiltonian},
     author={So, Oswin and Li, Gongjie and Theodorou, Evangelos A and Tao, Molei},
     booktitle={Machine Learning and the Physical Sciences Workshop NeurIPS},
     year={2022},
   }

2. NeurIPS 2022

   Deep Generalized Schrodinger Bridge

   Guan-Horng Liu, Tianrong Chen*, Oswin So*, and Evangelos A Theodorou

   Thirty-Sixth Conference on Neural Information Processing Systems , 2022

   arXiv Cite Abstract

   Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling the collective behavior of individual agents interacting stochastically with a large population. In this work, we aim at solving a challenging class of MFGs in which the differentiability of these interacting preferences may not be available to the solver, and the population is urged to converge exactly to some desired distribution. These setups are, despite being well-motivated for practical purposes, complicated enough to paralyze most (deep) numerical solvers. Nevertheless, we show that Schrödinger Bridge - as an entropy-regularized optimal transport model - can be generalized to accepting mean-field structures, hence solving these MFGs. This is achieved via the application of Forward-Backward Stochastic Differential Equations theory, which, intriguingly, leads to a computational framework with a similar structure to Temporal Difference learning. As such, it opens up novel algorithmic connections to Deep Reinforcement Learning that we leverage to facilitate practical training. We show that our proposed objective function provides necessary and sufficient conditions to the mean-field problem. Our method, named Deep Generalized Schrödinger Bridge (DeepGSB), not only outperforms prior methods in solving classical population navigation MFGs, but is also capable of solving 1000-dimensional opinion depolarization, setting a new state-of-the-art numerical solver for high-dimensional MFGs. Our code will be made available at https://github.com/ghliu/DeepGSB.

   @inproceedings{liu2022deep,
     title={Deep Generalized Schrodinger Bridge},
     author={Liu, Guan-Horng and Chen*, Tianrong and So*, Oswin and Theodorou, Evangelos A},
     booktitle={Thirty-Sixth Conference on Neural Information Processing Systems},
     year={2022},
   }

3. Multimodal Maximum Entropy Dynamic Games

   Oswin So, Kyle Stachowicz, and Evangelos A. Theodorou

   arXiv preprint (in submission)

   arXiv Cite Abstract Video

   Environments with multi-agent interactions often result a rich set of modalities of behavior between agents due to the inherent suboptimality of decision making processes when agents settle for satisfactory decisions. However, existing algorithms for solving these dynamic games are strictly unimodal and fail to capture the intricate multimodal behaviors of the agents. In this paper, we propose MMELQGames (Multimodal Maximum-Entropy Linear Quadratic Games), a novel constrained multimodal maximum entropy formulation of the Differential Dynamic Programming algorithm for solving generalized Nash equilibria. By formulating the problem as a certain dynamic game with incomplete and asymmetric information where agents are uncertain about the cost and dynamics of the game itself, the proposed method is able to reason about multiple local generalized Nash equilibria, enforce constraints with the Augmented Lagrangian framework and also perform Bayesian inference on the latent mode from past observations. We assess the efficacy of the proposed algorithm on two illustrative examples: multi-agent collision avoidance and autonomous racing. In particular, we show that only MMELQGames is able to effectively block a rear vehicle when given a speed disadvantage and the rear vehicle can overtake from multiple positions.

   @article{so2022multimodal,
     title={Multimodal Maximum Entropy Dynamic Games},
     author={So, Oswin and Stachowicz, Kyle and Theodorou, Evangelos A.},
     journal={arXiv preprint (in submission)},
     year={2022},
   }

4. RSS 2022

   Decentralized Safe Multi-agent Stochastic Optimal Control using Deep FBSDEs and ADMM

   Marcus A. Pereira, Augustinos D. Saravanos, Oswin So, and Evangelos A. Theodorou

   Robotics: Science and Systems , 2022

   arXiv Cite Abstract Video

   In this work, we propose a novel safe and scalable decentralized solution for multi-agent control in the presence of stochastic disturbances. Safety is mathematically encoded using stochastic control barrier functions and safe controls are computed by solving quadratic programs. Decentralization is achieved by augmenting to each agent’s optimization variables, copy variables, for its neighboring agents. This allows us to decouple the centralized multi-agent optimization problem. How- ever, to ensure safety, neighboring agents must agree on what is safe for both of us and this creates a need for consensus. To enable safe consensus solutions, we incorporate an ADMM- based approach. Specifically, we propose a Merged CADMM- OSQP implicit neural network layer, that solves a mini-batch of both, local quadratic programs as well as the overall con- sensus problem, as a single optimization problem. This layer is embedded within a Deep FBSDEs network architecture at every time step, to facilitate end-to-end differentiable, safe and decentralized stochastic optimal control. The efficacy of the proposed approach is demonstrated on several challenging multi- robot tasks in simulation. By imposing requirements on safety specified by collision avoidance constraints, the safe operation of all agents is ensured during the entire training process. We also demonstrate superior scalability in terms of computational and memory savings as compared to a centralized approach.

   @inproceedings{pereira2022decentralized,
     title={Decentralized Safe Multi-agent Stochastic Optimal Control using Deep FBSDEs and ADMM},
     author={Pereira, Marcus A. and Saravanos, Augustinos D. and So, Oswin and Theodorou, Evangelos A.},
     booktitle={Robotics: Science and Systems},
     year={2022},
   }

2021

1. ICRA 2022

   Maximum Entropy Differential Dynamic Programming

   Oswin So, Ziyi Wang, and Evangelos A. Theodorou

   2022 International Conference on Robotics and Automation (ICRA)

   arXiv Cite Abstract Video

   In this paper, we present a novel maximum entropy formulation of the Differential Dynamic Programming algorithm and derive two variants using unimodal and multimodal value functions parameterizations. By combining the maximum entropy Bellman equations with a particular approximation of the cost function, we are able to obtain a new formulation of Differential Dynamic Programming which is able to escape from local minima via exploration with a multimodal policy. To demonstrate the efficacy of the proposed algorithm, we provide experimental results using four systems on tasks that are represented by cost functions with multiple local minima and compare them against vanilla Differential Dynamic Programming. Furthermore, we discuss connections with previous work on the linearly solvable stochastic control framework and its extensions in relation to compositionality.

   @inproceedings{so2021maximum,
     title={Maximum Entropy Differential Dynamic Programming},
     author={So, Oswin and Wang, Ziyi and Theodorou, Evangelos A.},
     booktitle={2022 International Conference on Robotics and Automation (ICRA)},
     year={2021},
   }

2. RSS 2021

   Variational Inference MPC using Tsallis Divergence

   Ziyi Wang*, Oswin So*, Jason Gibson, Bogdan Vlahov, Manan S Gandhi, Guan-Horng Liu, and Evangelos A Theodorou

   Robotics: Science and Systems , 2021

   arXiv Cite Abstract

   In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using thenon-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive PathIntegral Control, Cross Entropy Method, and Stein VariationalInference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

   @inproceedings{wang2021variational,
     title={Variational Inference MPC using Tsallis Divergence},
     author={Wang*, Ziyi and So*, Oswin and Gibson, Jason and Vlahov, Bogdan and Gandhi, Manan S and Liu, Guan-Horng and Theodorou, Evangelos A},
     booktitle={Robotics: Science and Systems},
     year={2021},
   }

3. Spatio-Temporal Differential Dynamic Programming for Control of Fields

   Ethan N Evans, Oswin So, Andrew P Kendall, Guan-Horng Liu, and Evangelos A Theodorou

   arXiv preprint (in submission)

   arXiv Cite Abstract

   We consider the optimal control problem of a general nonlinear spatio-temporal system described by Partial Differential Equations (PDEs). Theory and algorithms for control of spatio-temporal systems are of rising interest among the automatic control community and exhibit numerous challenging characteristic from a control standpoint. Recent methods focus on finite-dimensional optimization techniques of a discretized finite dimensional ODE approximation of the infinite dimensional PDE system. In this paper, we derive a differential dynamic programming (DDP) framework for distributed and boundary control of spatio-temporal systems in infinite dimensions that is shown to generalize both the spatio-temporal LQR solution, and modern finite dimensional DDP frameworks. We analyze the convergence behavior and provide a proof of global convergence for the resulting system of continuous-time forward-backward equations. We explore and develop numerical approaches to handle sensitivities that arise during implementation, and apply the resulting STDDP algorithm to a linear and nonlinear spatio-temporal PDE system. Our framework is derived in infinite dimensional Hilbert spaces, and represents a discretization-agnostic framework for control of nonlinear spatio-temporal PDE systems.

   @article{evans2021spatio,
     title={Spatio-Temporal Differential Dynamic Programming for Control of Fields},
     author={Evans, Ethan N and So, Oswin and Kendall, Andrew P and Liu, Guan-Horng and Theodorou, Evangelos A},
     journal={arXiv preprint (in submission)},
     year={2021},
   }

4. L4DC 2021

   Adaptive Risk Sensitive Model Predictive Control with Stochastic Search

   Ziyi Wang, Oswin So, Keuntaek Lee, and Evangelos A. Theodorou

   Learning for Dynamics & Control Conference , 2021

   arXiv Cite Abstract

   We present a general framework for optimizing the Conditional Value-at-Risk for dynamical systems using stochastic search. The framework is capable of handling the uncertainty from the initial condition, stochastic dynamics, and uncertain parameters in the model. The algorithm is compared against a risk-sensitive distributional reinforcement learning framework and demonstrates outperformance on a pendulum and cartpole with stochastic dynamics. We also showcase the applicability of the framework to robotics as an adaptive risk-sensitive controller by optimizing with respect to the fully nonlinear belief provided by a particle filter on a pendulum, cartpole, and quadcopter in simulation.

   @inproceedings{wang2021adaptive,
     title={Adaptive Risk Sensitive Model Predictive Control with Stochastic Search},
     author={Wang, Ziyi and So, Oswin and Lee, Keuntaek and Theodorou, Evangelos A.},
     booktitle={Learning for Dynamics & Control Conference},
     year={2021},
   }