Matricies Differentiation

Slides

1. Scalar/Vector (result is row vector)

$$\frac{\partial y}{\partial \mathbf{x}}=\left(\frac{\partial y}{\partial x_1},\dots ,\frac{\partial y}{\partial x_n}\right)$$

2. Vector/Scalar (result is column vector)

$$\frac{\partial \mathbf{y}}{\partial x}=\left[\begin{array}{c}\frac{\partial y_1}{\partial x}\\ ⋮\\ \frac{\partial y_n}{\partial x}\end{array}\right]$$

3. Vector/Vector (result is matrix). Also called the Jacobian Matrix

$$\frac{\partial \mathbf{y}}{\partial \mathbf{x}}=\left[\begin{array}{cccc}\frac{\partial y_1}{\partial x_1}& \frac{\partial y_1}{\partial x_2}& \cdots & \frac{\partial y_1}{\partial n}\\ \frac{\partial y_2}{\partial x_1}& \frac{\partial y_2}{\partial x_2}& \cdots & \frac{\partial y_2}{\partial x_n}\\ ⋮& & \ddots & \\ \frac{\partial y_m}{\partial x_1}& \frac{\partial y_m}{\partial x_2}& \cdots & \frac{\partial y_m}{\partial x_n}\end{array}\right]$$

Tips & Tricks §

If $A\cdot b=c$ with matrix $A$, column vectors $b,c$ then:

$$\frac{\partial c}{\partial b}=A$$