Hierarchy of Transformation

1. Affine and projectve transformations

AffineProjective

  • For two arbitrary ideal points which are of the form (x,y,0)t(x, y, 0)^t, these points are always on the line l=(0,0,1)tl_{\infty} = (0, 0, 1)^t since (x1,y1,0)tx(x2,y2,0)t=l(x_1, y_1, 0)^t x (x_2, y_2, 0)^t = l_{\infty}.
  • A point on ll_{\infty} is transformed to another point on ll_{\infty} by affine transformation. But it is transformed to a point on the vanishing line by projective transformation.

2. How to remove the projective distortion

  • Projective transformation causes the distortion such that parallel lines in affine space are not parallel any more because a vanishing line comes about.
  • Let the vanishing line be l=(l1,l2,l3)tl = (l_1, l_2, l_3)^t, then ll is mapped to ll_{\infty} by a matrix HH as follows:
H=(100011l1l2l3)\begin{equation} H = \left(\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 1 \\ l_1 & l_2 & l_3 \end{array}\right) \end{equation}
  • While a point is transformed by HH, a normal vector is transformed by HtH^{-t}. So Htl=lH^{-t}l = l_{\infty}.
Htl=(10l1l301l2l3001l3)(l1l2l3)=(001)=l\begin{equation} H^{-t}l = \left(\begin{array}{ccc} 1 & 0 & -\frac{l_1}{l_3} \\ 0 & 1 & -\frac{l_2}{l_3} \\ 0 & 0 & \frac{1}{l_3} \end{array}\right) \left(\begin{array}{c} l_1 \\ l_2 \\ l_3 \end{array}\right) = \left(\begin{array}{c} 0 \\ 0 \\ 1 \end{array}\right) = l_{\infty} \end{equation}
  • Therefore, HtH^{-t} transforms any line ll into ll_{\infty}, which means that it restores the projective distortion. But it still remains the affine distortion.

3. The angle between two lines

TwoLines

  • For two lines ll and mm, the angle θ\theta between them is the following:
cosθ=l1m1+l2m2(l12+22)(m12+m22)=ltCm(ltCl)(mtCm)C=(100010000):dual degenerate conic\begin{aligned} cos\theta &= \frac{l_1 m_1 + l_2 m_2}{\sqrt{(l^2_1 +ㅣ^2_2)(m^2_1 +m^2_2)}} = \frac{l^t C^{*}_{\infty}m}{\sqrt{(l^t C^{*}_{\infty}l)(m^t C^{*}_{\infty}m)}} \\ \\ C^{*}_{\infty} &= \left(\begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right) : \text{dual degenerate conic} \end{aligned}

4. How to remove the affine distortion

  • Even though the projective distortion is removed, there is still the affine distortion, which a square shape turns into a parallelogram shape. In other words, the angle between lines is not preserved.
  • Assume that two lines ll, mm, and dual degenerate conic CC^{*}_{\infty} in the original space are transformed to ll', mm' and CC'^{*}_{\infty} by affine transformation. Then l=Htll' = H^{-t}l, m=Htmm' = H^{-t}m, and C=HCHtC'^{*}_{\infty} = HC^{*}_{\infty}H^t.
  • For the angle θ\theta between ll and mm, the numerator of cosθcos\theta is ltCm=(ltH)(H1CHt)(Htm)=ltCml^t C^{*}_{\infty} m = (l'^t H)(H^{-1}C'^{*}_{\infty} H^{-t})(H^t m') = l'^t C'^{*}_{\infty} m'.
  • When ll and mm are perpendicular, ltCm=0l'^t C'^{*}_{\infty} m' = 0. It yields as follows:
ltCm=ltHCHtm=(l1,l2,l3)(Atundefined0undefined1)(I0undefined0undefined0)(A0undefinedtundefined1)(m1m2m3)=0(l1,l2,l3)(AAt0undefined0undefined0)(m1m2m3)=(l1,l2)(s11s12s21s22)(m1m2)=0\begin{aligned} l'^t C'^{*}_{\infty} m' = l'^t H C^{*}_{\infty} H^t m' = (l'_1, l'_2, l'_3) \left(\begin{array}{cc} A & \overrightarrow{t} \\ \overrightarrow{0} & 1 \end{array}\right) \left(\begin{array}{cc} I & \overrightarrow{0} \\ \overrightarrow{0} & 0 \end{array}\right) \left(\begin{array}{cc} A & \overrightarrow{0} \\ \overrightarrow{t} & 1 \end{array}\right) \left(\begin{array}{c} m'_1 \\ m'_2 \\ m'_3 \end{array}\right) = 0 \\ \\ (l'_1, l'_2, l'_3) \left(\begin{array}{cc} AA^t & \overrightarrow{0} \\ \overrightarrow{0} & 0 \end{array}\right) \left(\begin{array}{c} m'_1 \\ m'_2 \\ m'_3 \end{array}\right) = (l'_1, l'_2) \left(\begin{array}{cc} s_{11} & s_{12} \\ s_{21} & s_{22} \end{array}\right) \left(\begin{array}{c} m'_1 \\ m'_2 \end{array}\right) = 0 \end{aligned}

where (s11s12s21s22)=S=AAt\left(\begin{array}{cc} s_{11} & s_{12} \\ s_{21} & s_{22} \end{array}\right) = S = AA^t.

  • SS is symmetric, s12=s21s_{12} = s_{21}, and s22s_{22} can be set to 11 since SS is defined up to scale.
  • s11l1m1+s12(l1m2+l2m1)=0s_{11} l'_1 m'_1 + s_{12} (l'_1 m'_2 +l'_2 m'_1) = 0.
  • If we get two pairs of ll and mm which are perpendicular, then SS can be calculated.
  • Having calculated SS, HH can be esimated assuming AA is positive-definite. From S=AAtS = AA^t, the eigenvalues of AA are the positive square roots of teh eigenvalues of SS.
  • After removing the affine distortion, there still remains the similarity distortion.

5. Hierarchy of transformation in 3D is similar to that in 2D

  • As a point on ll_{\infty} is transformed to another point on ll_{\infty} by affine transformation, a point on the plane Π=(0,0,0,1)t\Pi_{\infty} = (0, 0, 0, 1)^t is also transformed to another point on Π\Pi_{\infty}.
  • As a point on ll_{\infty} is transformed to a point on the vanishing line by projective transformation, a point on Π=(0,0,0,1)t\Pi_{\infty} = (0, 0, 0, 1)^t is transformed to a point on the specific plane.

ProjectiveDistortion

  • When a cube is transformed in projective space, each a pair of parallel lines is not parallel any more and intersects at a point which is not at infinity such as v1v_1, v2v_2, and v3v_3.
  • These points v1v_1, v2v_2, and v3v_3 consist of a plane Π\Pi which does correspond to Π\Pi_{\infty} in affine space.
  • If we know this plane Π\Pi, it can be transformed to Π\Pi_{\infty} by some matrix HH, so the projective distortion can be removed.
  • Let Π=(π1,π2,π3,π4)t=(πt,π4)t\Pi = (\pi_1, \pi_2, \pi_3, \pi_4)^t = (\overline{\pi}^t, \pi_4)^t, then the following HH restores Π\Pi to Π\Pi_{\infty}:
H=(A0undefinedπtπ4),H1=(A10undefined1π4πtA11π4),Ht=(At1π4Atπ0undefined1π4)\begin{equation} H = \left(\begin{array}{cc} A & \overrightarrow{0} \\ \overline{\pi}^t & \pi_4 \end{array}\right), H^{-1} = \left(\begin{array}{cc} A^{-1} & \overrightarrow{0} \\ -\frac{1}{\pi_4} \overline{\pi}^t A^{-1} & \frac{1}{\pi_4} \end{array}\right), H^{-t} = \left(\begin{array}{cc} A^{-t} & -\frac{1}{\pi_4} A^{-t} \overline{\pi} \\ \overrightarrow{0} & \frac{1}{\pi_4} \end{array}\right) \end{equation} HtΠ=(At1π4Atπ0undefined1π4)(ππ4)=(0undefined1)=Π\begin{equation} H^{-t} \Pi = \left(\begin{array}{cc} A^{-t} & -\frac{1}{\pi_4} A^{-t} \overline{\pi} \\ \overrightarrow{0} & \frac{1}{\pi_4} \end{array}\right) \left(\begin{array}{c} \overline{\pi} \\ \pi_4 \end{array}\right) = \left(\begin{array}{c} \overrightarrow{0} \\ 1 \end{array}\right) = \Pi_{\infty} \end{equation}
  • π\overline{\pi} is the orientation of Π\Pi and Π\Pi is a homogeneous representation, so Π1π4Π\Pi \sim \frac{1}{\pi_4} \Pi.

Transforms

Reference

[1] Hartley, R. and Zisserman, A. (2003) Multiple View Geometry in Computer Vision. 2nd Edition, Cambridge University Press, Cambridge.


© 2022. All rights reserved.