Derivation of Rotation Matrix

Consider the situation where a point $p$ rotates around a normalized vector $\bar{n}$ by $\alpha$. Let $p'$ be the rotated point and $r$ be the intersection between $\bar{n}$ and the plane where $p$ is rotating. As above, $\bar{n}$ passes through $(0, 0, 0)$, Even though $\bar{n}$ can be in any direction and pass any point, this vector can be translated so that it passes through $(0, 0, 0)$ keeping the direction. So, this assumption is still valid for simplicity. Now, the following equations are derived from the dot product and cross product. $$\begin{aligned} &p' = r + q' \\ &q' = q \cos \alpha + (\bar{n} \times q) \sin \alpha \\ &r = (\bar{n} \cdot p) \bar{n}& \\ &q = p - r = p - (\bar{n} \cdot p) \bar{n} \\\\ \implies &\; \bar{n} \times q = \bar{n} \times (p - (\bar{n} \cdot p) \bar{n}) = \bar{n} \times p \\\\ \implies &\; p' = (\bar{n} \cdot p) \bar{n} + (p - (\bar{n} \cdot p) \bar{n}) \cos \alpha + (\bar{n} \times p) \sin \alpha \end{aligned}$$

Looking into the $(\bar{n} \cdot p) \bar{n}$ part from the point of view of matrix form, $$\begin{aligned} (\bar{n} \cdot p) \bar{n} &= (\bar{n}_x p_x + \bar{n}_y p_y + \bar{n}_z p_z) \left(\begin{array}{c} \bar{n}_x \\ \bar{n}_y \\ \bar{n}_z \end{array}\right) = \left(\begin{array}{c} \bar{n}_x^2 p_x + \bar{n}_x \bar{n}_y p_y + \bar{n}_x \bar{n}_z p_z \\\\ \bar{n}_x \bar{n}_y p_x + \bar{n}_y^2 p_y + \bar{n}_y \bar{n}_z p_z \\\\ \bar{n}_x \bar{n}_y p_x + \bar{n}_y \bar{n}_z p_y + \bar{n}_z^2 p_z \end{array}\right) \\\\ &= \left(\begin{array}{ccc} \bar{n}_x^2 & \bar{n}_x \bar{n}_y & \bar{n}_x \bar{n}_z \\ \bar{n}_x \bar{n}_y & \bar{n}_y^2 & \bar{n}_y \bar{n}_z \\ \bar{n}_x \bar{n}_y & \bar{n}_y \bar{n}_z & \bar{n}_z^2 \end{array}\right) \left(\begin{array}{c} p_x \\ p_y \\ p_z \end{array}\right) = \left(\begin{array}{c} \bar{n}_x \\ \bar{n}_y \\ \bar{n}_z \end{array}\right) \left(\begin{array}{ccc} \bar{n}_x & \bar{n}_y & \bar{n}_z \end{array}\right) p = (\bar{n} \times \bar{n}^t) p \end{aligned}$$

As such, the $(p - (\bar{n} \cdot p) \bar{n}) \cos \alpha$ part also can be rewritten as follows. $$\begin{aligned} (p - (\bar{n} \cdot p) \bar{n}) \cos \alpha = \cos \alpha (p - (\bar{n} \times \bar{n}^t) p) = \cos \alpha (I - \bar{n} \times \bar{n}^t) p \end{aligned}$$

The $(\bar{n} \times p) \sin \alpha$ part also can be reformed by skew-symmetric matrix. $$\begin{aligned} S = \left(\begin{array}{ccc} 0 & -\bar{n}_z & \bar{n}_y \\ \bar{n}_z & 0 & -\bar{n}_x \\ -\bar{n}_y & \bar{n}_x & 0 \end{array}\right) \implies (\bar{n} \times p) \sin \alpha = \sin \alpha S p \end{aligned}$$

Now, let $R = \bar{n} \times \bar{n}^t + \cos \alpha (I - \bar{n} \times \bar{n}^t) + \sin \alpha S$. Then $p' = Rp$. Therefore, this $R$ is the rotation matrix. $$\begin{aligned} R = \left(\begin{array}{ccc} (1 - \cos \alpha) \bar{n}_x^2 + \cos \alpha & (1 - \cos \alpha) \bar{n}_x \bar{n}_y - \bar{n}_z \sin \alpha & (1 - \cos \alpha) \bar{n}_x \bar{n}_z + \bar{n}_y \sin \alpha \\\\ (1 - \cos \alpha) \bar{n}_x \bar{n}_y + \bar{n}_z \sin \alpha & (1 - \cos \alpha) \bar{n}_y^2 + \cos \alpha & (1 - \cos \alpha) \bar{n}_y \bar{n}_z - \bar{n}_x \sin \alpha \\\\ (1 - \cos \alpha) \bar{n}_x \bar{n}_z - \bar{n}_y \sin \alpha & (1 - \cos \alpha) \bar{n}_y \bar{n}_z + \bar{n}_x \sin \alpha & (1 - \cos \alpha) \bar{n}_z^2 + \cos \alpha \end{array}\right) \end{aligned}$$

Reference

[1] 3D Graphics Programming Using OpenGL: Introduction