What is “batch normalization” in deep learning?
a) A technique to standardize the inputs to each layer, improving training stability
b) A method to reduce the size of the training dataset
c) A technique to combine multiple models
d) A method to tune hyperparameters
Answer:
a) A technique to standardize the inputs to each layer, improving training stability
Explanation:
Batch normalization is a technique used in deep learning to standardize the inputs to each layer by scaling them to have a mean of zero and a standard deviation of one. This helps improve the stability and speed of the training process.
By normalizing the inputs to each layer, batch normalization reduces internal covariate shift, where the distribution of layer inputs changes during training. This leads to more stable gradients, allowing the model to converge faster and improving overall performance.
Batch normalization is commonly used in convolutional neural networks (CNNs) and other deep learning architectures to prevent issues like vanishing gradients and to help regularize the model, reducing the risk of overfitting.