[Google_Bootcamp_Day6]

Train/dev/test set

Mismatched train/test distribution (real life)

• Make sure that dev and test sets come from the same distribution

Bias and Variance

Basic recipe for machine learning

• Check “high bias” (training set performance)
  • Try bigger network
  • Try to train longer
  • Try different nerual network architecture
• Check “high variance” (dev set performance)
  • Try to use more data
  • Try regularization
  • Try different neural network architecture

Regularization

• Logistic regression

• Neural network

How doe regularization prevent overfitting

Dropout regularization

• No dropout when making predictions at test time

Why dropout work

Downside and solution

• Downside
  • not well defined cost function so that it is hard to check downhill plot of cost function
• Solution
  • Turn off dropout and check downhill plotting
  • Then turn on dropout

Other regularization methods

• Data Augmentation

• Early Stopping

[Source] https://www.coursera.org/learn/deep-neural-network