Compressed Conditional Mean Embeddings for Model-Based Reinforcement Learning-Supplementary Material

Our algorithm uses, as a subcomponent, a vector-valued version of the matching pursuit algorithm (Mallat and Zhang 1993). The use of matching pursuit for vector-valued regression is not new (Lever and Stafford 2015) but we provide details for completeness. This adaptation can handle targets in general vector spaces V. Since the algorithm only computes inner products between vectors in the target space V it can be kernelized, ie we can learn an RKHS-valued function. This is a straightforward extension of the scalar case, we derive the method here for clarity. Suppose we wish to regress a vector-valued function f∗: X→ V, given a data sample D={xi, vi} m i= 1 where vi= f∗(xi)+ ϵ where ϵ is zero-mean noise, f∗(xi)= E [Vi| xi]. Suppose we are given a dictionary G={g1,..., gn}, where gi: X→ R, of candidate real-valued functions, and we aim to find an estimateˆffor f∗ of the form,