Neural Networks for Signal Processing - I

By Prof. Shayan Srinivasa Garani | IISc Bangalore

Learners enrolled: 2203

This will be an introductory graduate level course in neural networks for signal processing. The course starts with a motivation of how the human brain is inspirational to building artificial neural networks. The neural networks are viewed as directed graphs with various network topologies towards learning tasks driven by optimization techniques. The course covers Rosenblatt’s perceptron, regression modeling, multilayer perceptron (MLP), kernel methods and radial basis functions (RBF), support vector machines (SVM), regularization theory and principal component analysis (Hebbian and kernel based). Towards the end, topics such as convolutional neural networks etc. that are based on the MLP will be touched upon. The course will have assignments that are theoretical and computer based working with actual data.
INTENDED AUDIENCE: Graduate level, Senior UG can also participate, engineers and scientists within related industry. PREREQUISITES: Basic mathematical background in probability, linear algebra, signals and systems or equivalent. INDUSTRY SUPPORT: AI based, machine learning based.

Summary

Course Status :	Completed
Course Type :	Elective
Duration :	12 weeks
Category :	Electrical, Electronics and Communications Engineering
Credit Points :	3
Level :	Postgraduate
Start Date :	29 Jul 2019
End Date :	18 Oct 2019
Exam Date :	17 Nov 2019 IST

Note: This exam date is subjected to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout

Week 1: Introduction, human brain, models of a neuron, neural communication, neural networks as directed graphs, network architectures (feed-forward, feedback etc.), knowledge representation. Week 2: Learning processes, learning tasks, Perceptron, perceptron convergence theorem, relationship between perceptron and Bayes classifiers, batch perceptron algorithm Week 3: Modeling through regression, linear and logistic regression for multiple classes. Week 4: Multilayer perceptron, batch and online learning, derivation of the back propagation algorithm, XOR problem, Role of Hessian in online learning, annealing and optimal control of learning rate Week 5: Approximations of functions, cross-validation, network pruning and complexity regularization, convolution networks, non-linear filtering Week 6: Cover’s theorem and pattern separability, the interpolation problem, RBF networks, hybrid learning procedure for RBF networks, Kernel regression and relationship to RBFs. Week 7: Support vector machines, optimal hyperplane for linear separability, optimal hyperplane for nonseparable patterns, SVM as a kernel machine,design of SVMs, XOR problem revisted, robustness considerations for regression Week 8: SVMs contd. Optimal solution of the linear regression problem, representer theorem and related discussions. Introduction to regularization theory Week 9: Hadamard’s condition for well-posedness, Tikhonov regularization, regularization networks, generalized RBF networks, estimation of regularization parameter etc Week 10: L1 regularization basics, algorithms and extensions Week 11: Principal component analysis: Hebbian based PCA, Kernel based PCA, Kernel Hebbian algorithm Week 12: Deep multi-layer perceptrons, deep autoencoders and stacked denoising auto-encoders. Thanks to the support from MathWorks, enrolled students have access to MATLAB for the duration of the course.

Books and references

1. S. Haykin, Neural Networks and Learning Machines, Pearson Press, 2009. 2. K. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012. 3. G. James, D. Witten, T. Hastie and R. Tibshirani, An Introduction to Statistical Learning, Springer, 2013. 4. Y. S. Abu-Mostafa, M. Magdon-Ismail and H. Lin, Learning from Data, AMLBook, 2012. 5. J. Nocedal and S. J. Wright, Numerical Optimization, Springer, 2006.

Instructor bio

Prof. Shayan Garani Srinivasa received his Ph.D. in Electrical and Computer Engineering from Georgia Institute of Technology – Atlanta, M.S. from the University of Florida – Gainesville and B.E. from Mysore University. Dr. Srinivasa has held senior engineering positions within Broadcom Corporation, ST Microelectronics and Western Digital. Prior to joining IISc, Prof.. Srinivasa was leading various research activities, managing and directing research and external university research programs within Western Digital. He was the chairman for signal processing for the IDEMA-ASTC and a co-chair for the overall technological committee. He is the author of a book, several journal and conference publications, holds U.S patents in the area of data storage. Dr. Srinivasa is a senior member of the IEEE, OSA and the chairman for the Photonic Detection group within the Optical Society of America.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centers.
The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).
Date and Time of Exams: 17th November 2019 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.
Registration url: Announcements will be made when the registration form is open for registrations.
The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.
Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.
Exam score = 75% of the proctored certification exam score out of 100
Final score = Average assignment score + Exam score

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75.

If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.
Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IISc Bangalore. It will be e-verifiable at nptel.ac.in/noc.
Only the e-certificate will be made available. Hard copies are being discontinued from July 2019 semester and will not be dispatched.

SWAYAM Helpline / Support