ICS 481: Artificial Neural Networks

​

ICS 481: Artificial Neural Networks

 
Course Information
Class/Laboratory Schedule: 

Two 75 minutes lectures per week (3-0-3)

Designation: 
 Elective Course
Course Level: 
 Undergraduate
Prerequisites
Prerequisite(s) by Topic: 
  • Algorithms and Problem-Solving
  • Fundamental Data Structures
Prerequisite Courses: 
Senior Standing
Catalog Description: 

Introduction to neural computing: Real vs. artificial neurons; Threshold logic; Training a linear threshold unit, the perceptron rule; Multilayer feed-forward networks and the back propagation algorithm; The Hopfield net; Self-organizing maps; Radial basis functions; Adaptive resonance theory; Applications of Neural Networks (ANN).

Textbook(s): 

Simon Haykin, Neural Networks: A Comprehensive Foundation, 2nd Edition, Prentice Hall PTR, 1999.

Reference(s) and Other Material: 
  • C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995,
  • D. P. Mandic and J. A. Chambers, Recurrent Neural Networks for Prediction, John Wiley & Sons, 2001.
  • D.E. Rummelhart, J.L. McClelland, et al.; Parallel Distributed Processing, MIT Press, 1986.
  • F.M. Ham & I. Kostanic. Principles of Neurocomputing for Science and Engineering, McGraw Hill, 2001 .
  • J. Hertz, A. Krogh & R.G. Palmer. An Introduction to the Theory of Neural Computation, Addison Wesley, 1991 .
  • Kevin Gurney. An Introduction to Neural Networks , UCL Press, 1997 .
  • Laurene Fausett. Fundamentals of Neural Networks, Prentice Hall, 1994 .
  • Martin T. Hagan, Howard B. Demuth, and Mark Beale. Neural Network Design, PWS Publishing Company, 1995. [Recommended].
  • Mohamad H. Hassoun, Fundamentals of Artificial Neural Networks, (MIT Press, 1995).
  • P.S. Churchland & T.J. Sejnowski. The Computational Brain, MIT Press, 1994 .
  • R. Beale & T. Jackson. Introduction to Neural Networks, IOP Publishing, 1990.
  • Robrt Callan. The Essence of Neural Networks, Prentice Hall Europe, 1999.
Course Outcomes: 

After completion of this course, the student shall be able to:

  • Have a good knowledge of the basic ANN techniques and understanding both advantage and disadvantage of them.
  • Be familiar with the most common used ANN topologies: one, two, or more hidden-layers, different activation functions, and the best training algorithms.
  • Recognize the feature of problems, which neural networks can be used appropriately to solve them, and compare it with other technology methods.
  • Understand and use the appropriate ANN methods and tools for specifying, designing, implementing neural network systems.
  • Have a good knowledge of several types of ANN models.
Topics Covered: 
  • Introduction to Neural Networks and their History.
  • Biological Neurons and Neural Networks. Artificial Neurons.
  • Single Layer Perceptrons. 1. Networks of Artificial Neurons. 2. Learning and Generalization
  • Hebbian Learning. Gradient Descent Learning.
  • The Generalized Delta Rule. Practical Considerations.
  • Learning in Multi-Layer Perceptrons.
  • Back-Propagation.
  • Learning with Momentum.
  • Conjugate Gradient Learning.
  • Bias and Variance.
  • Under-Fitting and Over-Fitting.
  • Improving Generalization.
  • Applications of Multi-Layer Perceptrons.
  • Radial Basis Function Networks: 1. Introduction. 2. Algorithms. 3. Applications. Committee Machines.
  • Self Organizing Maps: 1. Fundamentals. 2. Algorithms and Applications.
  • Learning Vector Quantization (LVQ).
  • Hopfield ANN
  • Advanced Topics.​