A PDE Interpretation of Prediction with Expert Advice
We study the problem of prediction of binary sequences with expert advice in the online setting, which is a classic example of online machine learning. We interpret the binary sequence as the price history of a stock, and view the predictor as an investor, which converts the problem into a stock prediction problem. In this framework, an investor, who predicts the daily movements of a stock, and an adversarial market, who controls the stock, play against each other over N turns. The investor combines the predictions of n ≥ 2 experts in order to make a decision about how much to invest at each turn, and aims to minimize their regret with respect to the best-performing expert at the end of the game. We consider the problem with history-dependent experts, in which each expert uses the previous d days of history of the market in making their predictions. The prediction problem is played (in part) over a discrete graph called the d dimensional de Bruijn graph.
In the first part of the talk we focus on an appropriate continuum limit and using methods from optimal control, graph theory, and partial differential equations, we discuss strategies for the investor and the adversarial market. The proposed strategies are asymptotically optimal for n=2 and d at most 4.
We prove that the value function for this game, rescaled appropriately, converges as N → ∞ at a rate of O(N−1/6) (for C4 payoff functions) to the viscosity solution of a nonlinear degenerate parabolic PDE. It can be understood as the Hamilton-Jacobi-Issacs equation for the two-person game. As a result, we are able to deduce asymptotically optimal strategies for the investor. For Lipschitz payoff functions the rate of convergence is O(N−1/6logN). Our results extend those in the first part of the talk (for n=2, d at most 4, where the convergence rate is O(N−1/2)).
In the last part of the talk we show that the optimality conditions over the de Bruijn graph correspond to a graph Poisson equation, and we establish O(N−1/2) optimal strategies for all n and d.
This is joint work with Robert Kohn and Jeff Calder.