Research
You can also find my research on my Google Scholar page.
Current Projects
Smoothing Methods for Deep DC Optimal Power Flow
Optimal Portfolio Rebalancing with Reinforcement Learning
Measure-Flow Hamiltonian Neural Networks for the Vlasov Equation
Preprints and Working Papers
Algorithmic Collusion and a Folk Theorem from Learning with Bounded Rationality (Submitted)
Álvaro Cartea*, Patrick Chang*, Jose Penalva*, Harrison Waldon*
The Algorithmic Learning Equations: Evolving Strategies in Dynamic Games
Álvaro Cartea*, Patrick Chang*, Jose Penalva*, Harrison Waldon*
Publications
DARE: The Deep Adaptive Regulator for Closed-Loop Predictive Control (ICML FoRLaC 2024)
Fayçal Drissi*, Harrison Waldon*, Yannick Limmer, Uljad Berdica, Jakob Foerster, Álvaro Cartea
Abstract: A fundamental challenge in continuous-time optimal control (OC) is the efficient computation of adaptive policies when agents act in unknown, uncertain environments. Traditional OC methods, such as dynamic programming, face challenges in scalability and adaptability due to the curse-of-dimensionality and the reliance on fixed models of the environment. One approach to address these issues is Model Predictive Control (MPC), which iteratively computes open-loop controls over a receding horizon. However, classical MPC algorithms typically also assume a fixed environment. Another approach is Reinforcement Learning (RL) which scales well to high-dimensional setups but is often sample inefficent. Certain RL methods can also be unreliable in highly stochastic continuous-time setups and may be unable to generalize to unseen environments. This paper presents the Deep Adaptive Regulator (DARE) which uses physics-informed neural network based approximations to the agent’s value function and policy which are trained online to adapt to unknown environments. To manage uncertainty of the environment, DARE optimizes an augmented reward objective which dynamically trades off exploration with exploitation. We show that our method effectively adapts to unseen environments in settings where ``classical’’ RL fails and is suited for online adaptive decision-making in environments that change in real time.
Rough Transformers for Continuous and Efficient Time-Series Modelling (ICLR TS4H 2024)
Fernando Moreno-Pino*, Álvaro Arroyo*, Harrison Waldon*, Xiaowen Dong, Álvaro Cartea
Abstract: Time-series data in real-world medical settings typically exhibit long-range dependencies and are observed at non-uniform intervals. In such contexts, traditional sequence-based recurrent models struggle. To overcome this, researchers replace recurrent architectures with Neural ODE-based models to model irregularly sampled data and use Transformer-based architectures to account for long-range dependencies. Despite the success of these two approaches, both incur very high computational costs for input sequences of moderate lengths and greater. To mitigate this, we introduce the Rough Transformer, a variation of the Transformer model which operates on continuous-time representations of input sequences and incurs significantly reduced computational costs, critical for addressing long-range dependencies common in medical contexts. In particular, we propose multi-view signature attention, which uses path signatures to augment vanilla attention and to capture both local and global dependencies in input data, while remaining robust to changes in the sequence length and sampling frequency. We find that Rough Transformers consistently outperform their vanilla attention counterparts while obtaining the benefits of Neural ODE-based models using a fraction of the computational time and memory resources on synthetic and real-world time-series tasks.
Forward robust portfolio selection: The binomial case (Probability, Uncertainty and Quantitative Risk, 2024)
Harrison Waldon
Abstract: We introduce a new approach for optimal portfolio choice under model ambiguity by incorporating predictable forward preferences in the framework of Angoshtari et al. The investor reassesses and revises the model ambiguity set incrementally in time while, also, updating his risk preferences forward in time. This dynamic alignment of preferences and ambiguity updating results in time-consistent policies and provides a richer, more accurate learning setting. For each investment period, the investor solves a worst-case portfolio optimization over possible market models, which are represented via a Wasserstein neighborhood centered at a binomial distribution. Duality methods from Gao and Kleywegt; Blanchet and Murthy are used to solve the optimization problem over a suitable set of measures, yielding an explicit optimal portfolio in the linear case. We analyze the case of linear and quadratic utilities, and provide numerical results.
* denotes equal contribution
Theses
The Algorithmic Learning Equations (PhD Dissertation)
Advisor: Thaleia Zariphopoulou
Abstract: This thesis presents the Algorithmic Learning Equations (ALEs) to study tacit algorithmic collusion. The ALEs are a set of differential equations that characterizes the finite-time and asymptotic behavior of state-dependent learning algorithms in stochastic and repeated games. The ALEs are derived rigorously, drawing upon stochastic approximation theory. The ALEs are analyzed to show, numerically and theoretically, that decentralized, self-interested learning algorithms can learn to collude. The final chapter of this thesis presents preliminaries using inverse reinforcement learning to detect collusion in a data-driven way. The contents of this thesis are primarily drawn from joint work with Professor Álvaro Cartea, Professor José Penalva, and Patrick Chang during various research visits to the Oxford-Man Institute of Quantitative Finance in 2022 and 2023.