Part of Advances in Neural Information Processing Systems 3 (NIPS 1990)
Daniel Schwartz, Vijay Samalam
We have produced a VLSI circuit capable of learning to approximate ar(cid:173) bitrary smooth of a single variable using a technique closely related to splines. The circuit effectively has 512 knots space on a uniform grid and has full support for learning. The circuit also can be used to approximate multi-variable functions as sum of splines.
An interesting, and as of yet, nearly untapped set of applications for VLSI imple(cid:173) mentation of neural network learning systems can be found in adaptive control and non-linear signal processing. In most such applications, the learning task consists of approximating a real function of a small number of continuous variables from discrete data points. Special purpose hardware is especially interesting for applica(cid:173) tions of this type since they generally require real time on-line learning and there can be stiff constraints on the power budget and size of the hardware. Frequently, the already difficult learning problem is made more complex by the non-stationary nature of the underlying process. Conventional feed-forward networks with sigmoidal units are clearly inappropriate for applications of this type. Although they have exhibited remarkable performance in some types of time series prediction problems (for example, Wiegend, 1990 and Atlas, 1990), their learning rates in general are too slow for on-line learning. On-line performance can be improved most easily by using networks with more constrained architecture, effectively making the learning problem easier by giving the network a hint about the learning task. Networks that build local representations of the data, such as radial basis functions, are excellent candidates for these type of problems. One great advantage of such networks is that they require only a single layer of units. If the position and width of the units are fixed, the learning problem is linear