Byte-Sized Deep Learning Series: Neural Networks – Overfitting, Underfitting

Description:

In deep learning, a model can either underfit (not learn enough from the data) or overfit (memorize the data, losing the ability to generalize).
In this 90-minute session, we will dive deep into the concepts of overfitting and underfitting, with a special focus on how to address overfitting in your neural networks.
You will learn various techniques like L2 regularization, dropout, early stopping, and model checkpointing to improve your model’s ability to generalize. These methods can significantly improve model performance, especially when working with limited data or large, complex models.
We’ll provide practical, hands-on examples using Keras to apply these techniques to a real-world dataset, so you can directly see how to improve your model’s accuracy and prevent overfitting.

Duration: 90 mins

Course Code: BDT498

Learning Objectives:

After this course, you will be able to:

• Introduction to Overfitting and Underfitting
• Using L2 Regularization in Keras
• Using Dropout layer
• Early Stopping and Model check points

• Learners familiar with Keras and basic model training

• Intermediate deep learning students who have experience building neural networks. Ideal for those seeking to understand how to prevent overfitting and improve model generalization on new data

Course Outline:

1. Introduction to Overfitting and Underfitting
   • Understand what overfitting is and underfitting in neural networks
   • How to recognize overfitting and underfitting
   • Key challenges: balancing variance and bias
2. Handle Overfitting: L2 Regularization
   • Understand what is L2 Regularization?
   • Implementing L2 Regularization with Keras (keras.regularizers.l2())
   • Hands-on: Adding L2 Regularization to layers in a model
3. Handling Overfitting: Dropout Layer
   • What is a dropout layer?
   • Understanding dropout rate
   • Hands-on: Adding a dropout layer to the model
4. Handing Overfitting: Early Stopping and Model Checkpoint
   • What is early stopping?
   • Understanding monitor and patience parameters
   • What is model checkpointing?
   • Save the best weights with model checkpointing
   • Hands-on: Using both early stopping and model checkpointing during training

Training material provided: Yes (Digital format)

Hands-on Lab: Instructions will be provided to install Jupyter notebook and other required python libraries. Students can opt to use ‘Google Colaboratory’ if they do not want to install these tools