Two 75 minutes lectures per week (3-0-3)
Designation: Elective Course
Course Level: Undergraduate
Prerequisite(s) by Topic:
- Algorithms and Problem-Solving
- Fundamental Data Structures
Introduction to neural computing: Real vs. artificial neurons; Threshold logic; Training a linear threshold unit, the perceptron rule; Multilayer feed-forward networks and the back propagation algorithm; The Hopfield net; Self-organizing maps; Radial basis functions; Adaptive resonance theory; Applications of Neural Networks (ANN).
Simon Haykin, Neural Networks: A Comprehensive Foundation, 2nd Edition, Prentice Hall PTR, 1999.
Reference(s) and Other Material:
- C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995,
- D. P. Mandic and J. A. Chambers, Recurrent Neural Networks for Prediction, John Wiley & Sons, 2001.
- D.E. Rummelhart, J.L. McClelland, et al.; Parallel Distributed Processing, MIT Press, 1986.
- F.M. Ham & I. Kostanic. Principles of Neurocomputing for Science and Engineering, McGraw Hill, 2001 .
- J. Hertz, A. Krogh & R.G. Palmer. An Introduction to the Theory of Neural Computation, Addison Wesley, 1991 .
- Kevin Gurney. An Introduction to Neural Networks , UCL Press, 1997 .
- Laurene Fausett. Fundamentals of Neural Networks, Prentice Hall, 1994 .
- Martin T. Hagan, Howard B. Demuth, and Mark Beale. Neural Network Design, PWS Publishing Company, 1995. [Recommended].
- Mohamad H. Hassoun, Fundamentals of Artificial Neural Networks, (MIT Press, 1995).
- P.S. Churchland & T.J. Sejnowski. The Computational Brain, MIT Press, 1994 .
- R. Beale & T. Jackson. Introduction to Neural Networks, IOP Publishing, 1990.
- Robrt Callan. The Essence of Neural Networks, Prentice Hall Europe, 1999.
After completion of this course, the student shall be able to:
- Have a good knowledge of the basic ANN techniques and understanding both advantage and disadvantage of them.
- Be familiar with the most common used ANN topologies: one, two, or more hidden-layers, different activation functions, and the best training algorithms.
- Recognize the feature of problems, which neural networks can be used appropriately to solve them, and compare it with other technology methods.
- Understand and use the appropriate ANN methods and tools for specifying, designing, implementing neural network systems.
- Have a good knowledge of several types of ANN models.
- Introduction to Neural Networks and their History.
- Biological Neurons and Neural Networks. Artificial Neurons.
- Single Layer Perceptrons. 1. Networks of Artificial Neurons. 2. Learning and Generalization
- Hebbian Learning. Gradient Descent Learning.
- The Generalized Delta Rule. Practical Considerations.
- Learning in Multi-Layer Perceptrons.
- Learning with Momentum.
- Conjugate Gradient Learning.
- Bias and Variance.
- Under-Fitting and Over-Fitting.
- Improving Generalization.
- Applications of Multi-Layer Perceptrons.
- Radial Basis Function Networks: 1. Introduction. 2. Algorithms. 3. Applications. Committee Machines.
- Self Organizing Maps: 1. Fundamentals. 2. Algorithms and Applications.
- Learning Vector Quantization (LVQ).
- Hopfield ANN
- Advanced Topics.