Homogeneous Coordinates
Need a way to represent points at infinity -> add a ratio.
(x, y) -> (x1, x2, x3)
$$ \begin{bmatrix} x \ y \end{bmatrix} = \begin{bmatrix} \frac{x_1}{x_3} \ \frac{x_2}{x_3} \end{bmatrix}
$$
If x3 = 0, point at infinity.
Line
Representation
From
$$a_1 x + a_2 y + a_3 = 0$$
to
$$a_1(\frac{x_1}{x_3}) + a_2(\frac{x_2}{x_3}) + a_3 = 0$$
$$a_1x_1 + a_2x_2 + a_3x_3 = 0$$
which is
$$I^T \tilde{x} = 0$$ where $$I = \begin{bmatrix}a_1 &a_2 &a_3\end{bmatrix}^T$$ and $$\tilde{x} = \begin{bmatrix}x_1 &x_2 &x_3\end{bmatrix}^T$$.
Formation
Line passing through two points $$\tilde{x_1}$$, $$\tilde{x_2}$$: $$I = \tilde{x_1} \times \tilde{x_2}$$.
Point of intersection between two lines $$I_1$$, $$I_2$$: $$\tilde{x} = I_1 \times I_2$$
$$ \mathbf {a} \times \mathbf {b} =[\mathbf {a} ]{\times }\mathbf {b} ={\begin{bmatrix}\,0&!-a{3}&\,\,a{2}\\,\,a{3}&0&!-a{1}\-a{2}&\,\,a{1}&\,0\end{bmatrix}}{\begin{bmatrix}b{1}\b{2}\b{3}\end{bmatrix}}
$$
Transformation
Transformation matrix M:
$$ \tilde{X_t} = M \tilde{X}
$$
A sequence of transformations = single matrix operation:
$$ \tilde{X_t} = M_n \dots M_2 M_1 \tilde{X} = M \tilde{X}
$$
Translation
$$ \begin{bmatrix} X+T_x \ Y+T_y \ Z+T_z \ 1 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & T_x \ 0 & 1 & 0 & T_y \ 0 & 0 & 1 & T_z \ 0 & 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} X \ Y \ Z \ 1 \end{bmatrix}
$$
Scaling
$$ \begin{bmatrix} s_xX \ s_yY \ s_zZ \ 1 \end{bmatrix} = \begin{bmatrix} s_x & 0 & 0 & 0 \ 0 & s_y & 0 & 0 \ 0 & 0 & s_z & 0 \ 0 & 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} X \ Y \ Z \ 1 \end{bmatrix}
$$
Rotation
$$ \begin{bmatrix} X' \ Y' \ Z' \ 1 \end{bmatrix} = \begin{bmatrix} r{00} & r{01} & r{02} & 0 \ r{10} & r{11} & r{12} & 0 \ r{20} & r{21} & r_{22} & 0 \ 0 & 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} X \ Y \ Z \ 1 \end{bmatrix}
$$
Full Camera Model
Rigid Body Motion
Bring world coordinates $$\tilde{X}$$ to camera coordinates $$\tilde{X_c}$$.
$$ \tilde{Xc} = P_r \tilde{X} = TR \tilde{X}\ \begin{bmatrix} X_c \ Y_c \ Z_c \ 1 \end{bmatrix} = \begin{bmatrix} r{00} & r{01} & r{02} & Tx \ r{10} & r{11} & r{12} & Ty \ r{20} & r{21} & r{22} & T_z \ 0 & 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} X \ Y \ Z \ 1 \end{bmatrix}
$$
where $$P_r = \begin{bmatrix} R & T \ 0_3^T & 1 \end{bmatrix}$$ is the rigid body transformation matrix.
Perspective Projection
Project $$\tilde{X_c}$$ onto $$\tilde{x}$$ on the image plane.
$$ \tilde{x} = P_p \tilde{X_c}\ \begin{bmatrix} sx \ sy \ s \end{bmatrix} = \begin{bmatrix} f & 0 & 0 & 0 \ 0 & f & 0 & 0 \ 0 & 0 & 1 & 0 \end{bmatrix}\begin{bmatrix} X_c \ Y_c \ Z_c \ 1 \end{bmatrix}
$$
where $$P_p$$ is the perspective projection matrix.
The perspective projection of a point $$\tilde{X_c}$$ remains unchanged if either $$P_p$$ or $$\tilde{X_c}$$ is scaled by a non-zero factor.
CCD Imaging
Convert from image plane coordinates $$\tilde{x}$$ to pixel coordinates $$\tilde{w}$$.
$$ \tilde{w} = P_c \tilde{x}\ \begin{bmatrix} su \ sv \ s \end{bmatrix} = \begin{bmatrix} k_u & 0 & u_0 \ 0 & k_v & v_0 \ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} sx \ sy \ s \end{bmatrix}
$$
where $$P_c$$ is the CCD calibration matrix, $$(u_0, v_0)$$ is the principal point.
Overall
$$ \begin{aligned} \tilde{w} &= P_{ps} \tilde{X} = P_c P_p P_r \tilde{X} = K P_r \tilde{X}\ \begin{bmatrix} su \ sv \ s \end{bmatrix} &= \begin{bmatrix} k_u & 0 & u_0 \ 0 & k_v & v_0 \ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} f & 0 & 0 & 0 \ 0 & f & 0 & 0 \ 0 & 0 & 1 & 0 \end{bmatrix}\begin{bmatrix} R & T \ 0_3^T & 1 \end{bmatrix}\begin{bmatrix} X \ Y \ Z \ 1 \end{bmatrix}\ &= \begin{bmatrix} \alpha_u & 0 & u_0 \ 0 & \alpha_v & v_0 \ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} R & T \ 0_3^T & 1 \end{bmatrix}\begin{bmatrix} X \ Y \ Z \ 1 \end{bmatrix} \end{aligned}
$$
where $$P_{ps}$$ is the camera projection matrix for a perspective camera, $$K = P_c P_p$$ is the camera calibration matrix.
$$K = P_c P_p$$: intrinsic parameters of camera
$$P_r$$: extrinsic parameters of camera
$$\alpha = fk$$: image scaling factors ($$\alpha_u$$: horizontal; $$\alpha_v$$: vertical)
$$\frac{\alpha_u}{\alpha_v}$$: aspect ratio
Degrees of freedom: 10 (4 from $$K$$, 3 from $$R$$, 3 from $$T$$)
Projective Camera
A more general case for perspective camera. No need to deal with nonlinear constraints.
$$ \tilde{w} = P \tilde{X}\ P = \begin{bmatrix} p{00} & p{01} & p{02} & p{03}\ p{10} & p{11} & p{12} & p{13}\ p{20} & p{21} & p{22} & p{23}\ \end{bmatrix}
$$
Degrees of freedom: 11 (set $$p_{23}$$ to 1)
Vanishing Points
With projection matrix $$P$$, the vanishing point of a given direction $$(X_d, Y_d, Z_d)$$ is given by:
$$ \begin{aligned} \tilde{v} &= P \begin{bmatrix} X_d & Y_d & Z_d & 0 \end{bmatrix}^T\ &= \begin{bmatrix} p_0 & p_1 & p_2 & p_3 \end{bmatrix} \begin{bmatrix} X_d & Y_d & Z_d & 0 \end{bmatrix}^T \end{aligned}
$$
Vanishing points of the axes of the world coordinate system is the first three columns of projection matrix $$P$$:
$$ \tilde{v_x} = P \begin{bmatrix} 1 & 0 & 0 & 0 \end{bmatrix}^T = p_0\ \tilde{v_y} = P \begin{bmatrix} 0 & 1 & 0 & 0 \end{bmatrix}^T = p_1\ \tilde{v_z} = P \begin{bmatrix} 0 & 0 & 1 & 0 \end{bmatrix}^T = p_2
$$
Planar Scenes
Simplify the camera model with restrictive viewing conditions, e.g. $$Z = 0$$.
Rigid body transformation reduced to:
$$ \tilde{Xc} = P_r^p \tilde{X}^p\ \begin{bmatrix} X_c \ Y_c \ Z_c \ 1 \end{bmatrix} = \begin{bmatrix} r{00} & r{01} & T_x \ r{10} & r{11} & T_y \ r{20} & r_{21} & T_z \ 0 & 0 & 1 \ \end{bmatrix}\begin{bmatrix} X \ Y \ 1 \end{bmatrix}
$$
where $$X^p = (X, Y)$$ is the point on plane, $$\tilde{X^p}$$ is the homogeneous representation of it, $$P_r^p$$ is the planar rigid body transformation matrix.
Overall,
$$ \begin{aligned} \tilde{w} &= P{ps}^p \tilde{X}^p = P_c P_p P_r^p \tilde{X}^p\ \begin{bmatrix} su \ sv \ s \end{bmatrix} &= \begin{bmatrix} k_u & 0 & u_0 \ 0 & k_v & v_0 \ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} f & 0 & 0 & 0 \ 0 & f & 0 & 0 \ 0 & 0 & 1 & 0 \end{bmatrix}\begin{bmatrix} r{00} & r{01} & T_x \ r{10} & r{11} & T_y \ r{20} & r_{21} & T_z \ 0 & 0 & 1 \ \end{bmatrix}\begin{bmatrix} X \ Y \ 1 \end{bmatrix} \end{aligned}
$$
Relax the constraints on $$P^p_{ps}$$,
$$ \tilde{w} = P^p \tilde{X}^p\ P^p = \begin{bmatrix} p{00} & p{01} & p{02}\ p{10} & p{11} & p{12}\ p{20} & p{21} & p_{22}\ \end{bmatrix}
$$
where $$P^p$$ is the planar projective transformation or plane to plane projectivity.
Degree of freedoms: 8 (scale does not matter)
Linear Scenes
Similarly, set $$Y = Z = 0$$.
$$ \tilde{Xc} = P_r^l \tilde{X}^l\ \begin{bmatrix} X_c \ Y_c \ Z_c \ 1 \end{bmatrix} = \begin{bmatrix} r{00} & Tx \ r{10} & Ty \ r{20} & T_z \ 0 & 1 \ \end{bmatrix}\begin{bmatrix} X \ 1 \end{bmatrix}
$$
Overall,
$$ \begin{aligned} \tilde{w} &= P{ps}^l \tilde{X}^l = P_c P_p P_r^l \tilde{X}^l\ \begin{bmatrix} su \ sv \ s \end{bmatrix} &= \begin{bmatrix} k_u & 0 & u_0 \ 0 & k_v & v_0 \ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} f & 0 & 0 & 0 \ 0 & f & 0 & 0 \ 0 & 0 & 1 & 0 \end{bmatrix}\begin{bmatrix} r{00} & Tx \ r{10} & Ty \ r{20} & T_z \ 0 & 1 \ \end{bmatrix}\begin{bmatrix} X \ 1 \end{bmatrix} \end{aligned}
$$
Relax the constraints on $$P_r^l$$,
$$ \tilde{w} = P^l \tilde{X}^l\ P^l = \begin{bmatrix} p{00} & p{01}\ p{10} & p{11}\ p{20} & p{21}\ \end{bmatrix}
$$
Degrees of freedom: 5 (scale does not matter)