Part of Advances in Neural Information Processing Systems 12 (NIPS 1999)
Nebojsa Jojic, Brendan J. Frey
Invariance to topographic transformations such as translation and shearing in an image has been successfully incorporated into feed(cid:173) forward mechanisms, e.g., "convolutional neural networks", "tan(cid:173) gent propagation". We describe a way to add transformation invari(cid:173) ance to a generative density model by approximating the nonlinear transformation manifold by a discrete set of transformations. An EM algorithm for the original model can be extended to the new model by computing expectations over the set of transformations. We show how to add a discrete transformation variable to Gaussian mixture modeling, factor analysis and mixtures of factor analysis. We give results on filtering microscopy images, face and facial pose clustering, and handwritten digit modeling and recognition.