Part of Advances in Neural Information Processing Systems 13 (NIPS 2000)
Arno Schödl, Irfan Essa
We present techniques for rendering and animation of realistic scenes by analyzing and training on short video sequences. This work extends the new paradigm for computer animation, video tex(cid:173) tures, which uses recorded video to generate novel animations by replaying the video samples in a new order. Here we concentrate on video sprites, which are a special type of video texture. In video sprites, instead of storing whole images, the object of inter(cid:173) est is separated from the background and the video samples are stored as a sequence of alpha-matted sprites with associated veloc(cid:173) ity information. They can be rendered anywhere on the screen to create a novel animation of the object. We present methods to cre(cid:173) ate such animations by finding a sequence of sprite samples that is both visually smooth and follows a desired path. To estimate visual smoothness, we train a linear classifier to estimate visual similarity between video samples. If the motion path is known in advance, we use beam search to find a good sample sequence. We can specify the motion interactively by precomputing the sequence cost function using Q-Iearning.