Code

Listed below are the primary bits of code that I’ve used throughout my research, with more recent work near the top.

Reinforcement Learning

• Acme. A library of reinforcement learning (RL) building blocks that strives to expose simple, efficient, and readable agents.

Bayesian optimization

• pybo. A modular approach to Bayesian optimization. This builds on reggie for modelling and is used inside benchfunk for benchmarking various approaches.

• reggie. A set of routines for Bayesian regression. Used heavily within pybo. This is heavily based on our earlier pygp code and plays the same role within pybo.

• benchfunk. Code for benchmarking Bayesian optimization algorithms. This includes a collection of benchmark functions as well as tools to run and collect the results of multiple solvers.

Other code

• mwhutils. Various machine learning and linear algebra utility functions that are used across my different projects. This has mostly been superceded by the utilities inside reggie.

• pygp. This implements code for inference with Gaussian processes, although it has been largely been replaced by reggie.

• pychud. This implements efficient rank-one updates and downdates to the Cholesky decomposition. Based on simplified wrappers to Linpack implementations of these procedures.

• mogmdp. This implements an EM-based strategy for finding the optimal policy for MDPs given as a mixture of Gaussians. This is based on the approach described here.

• qllp. A python translation of Emtiyaz Khan’s Matlab code for approximating the logistic log-partition function using piecewise quadratic bounds. Emti’s original code can be found here.