Part of Advances in Neural Information Processing Systems 12 (NIPS 1999)
Nicol Schraudolph, Xavier Giannakopoulos
Stochastic meta-descent (SMD) is a new technique for online adap(cid:173) tation of local learning rates in arbitrary twice-differentiable sys(cid:173) tems. Like matrix momentum it uses full second-order information while retaining O(n) computational complexity by exploiting the efficient computation of Hessian-vector products. Here we apply SMD to independent component analysis, and employ the result(cid:173) ing algorithm for the blind separation of time-varying mixtures. By matching individual learning rates to the rate of change in each source signal's mixture coefficients, our technique is capable of si(cid:173) multaneously tracking sources that move at very different, a priori unknown speeds.