Neural Network Design: Chapter 17 Radial Basis Networks
Download as PPTX, PDF3 likes453 views
The document presents a chapter on Radial Basis Function (RBF) networks, discussing their applications in high-dimensional data interpolation and kernel methods like support vector machines. It outlines the architecture and training process of RBF networks, including the calculation of RBF centers and output layer adaptations through supervised learning. The chapter also references historical examples and provides further reading on deep RBF networks.
Neural Network Design: Chapter 17 Radial Basis Networks
- 1. Chapter 17 Radial Basis Function Network Jason Tsai (蔡志順) February 8, 2020 Mozilla Community Space Taipei
- 2. *Copyright Notice: Most materials from this presentation are taken from the book “Neural Network Design 2nd edition” authored by Martin T. Hagan, Howard B. Demuth, Mark Hudson Beale and Orlando De Jesús. The other quoted sources are mentioned in the respective slides. This presentation itself adopts Creative Commons license.
- 3. Radial Basis Function *Figure adopted from https://bit.ly/381JXsV
- 5. In Brief • Radial basis functions (RBFs) were originally used for interpolating unstructured data as an approximator in high dimensions. It’s also used in kernel methods, e.g. support vector machine (SVM). • It’s local and suffers from the “curse of dimensionality”. • A classical two-layer RBF network is commonly trained in two stages. First the RBF centers (weights) and scaling parameters (biases) in RBF layer are calculated, and then the weights (and biases) of the output layer are adapted. The RBF centers can be trained through unsupervised or supervised learning. The regular output layer is adapted by supervised learning.
- 6. Miscellaneous RBFs *Figure adopted from https://bit.ly/37cPOfb
- 7. Multivariable Interpolation *Figure adopted from https://bit.ly/2tm8WZi
- 8. RBF Network Architecture *Figure adopted from https://bit.ly/2FJ9lYj
- 9. RBF Network Gaussian Basis Function When Gaussian function is used: (Sometimes b is omitted.)
- 12. Parameters Matter! *Figure adopted from https://bit.ly/2QVlCiO
- 13. XOR Example
- 15. XOR Example Decision Regions
- 16. Two categories Example *Figure adopted from https://bit.ly/2FJ9lYj
- 17. Two categories Example (Cont’d) *Figure adopted from https://bit.ly/2FJ9lYj
- 18. Training RBF center: • Subset of training set • Use Kohonen competitive layer, k-means, …, etc. to perform clustering • Use subset selection techniques, such as orthogonal least squares • Set bias with degree of overlap between RBFs Learning: • Define cost function, such as linear least squares • Train or fine-tune with gradient descent backpropagation
- 21. OLS Algorithm
- 22. Backpropagation
- 23. Deep RBFN in History: LeNet-5 *Figure adopted from https://bit.ly/3706EgX
- 24. Further Reading • Pourya Habib Zadeh, Reshad Hosseini, Suvrit Sra, "Deep-RBF Networks Revisited: Robust Classification with Rejection", arXiv preprint (2018), https://arxiv.org/abs/1812.03190 • Andrew Hryniowski, Alexander Wong, "DeepLABNet: End-to-end Learning of Deep Radial Basis Networks with Fully Learnable Basis Functions", arXiv preprint (2019), https://arxiv.org/abs/1911.09257