Normalization

A trick in deep learning to speed up training computation.

Linear Normialization §

For each

$$f(\vec{x}{)}_i=\gamma \cdot \frac{x_i-E[\vec{x}]}{\sigma }+\beta$$

where $\gamma ,\beta$ are learnable parameters

Batch Normalization §