Finding Points in 3D Camera Coordinates from Points on Image Plane
When a camera is viewing an object on the flat ground, the bottom point of the object is projected on the camera image plane. The camera coordinate system here is right handed whose view vector is -axis and -axis points outwards from the screen. Let be the tilt angle of the camera, then the view vector is parallel with the ground when and intersects with a point on the ground when . Lastly, is the camera height in meter.
1. The ground plane equation
If -axis of the camera is rotated by around its -axis, its -axis will be parallel with the normal vector of the ground. Therefore, is
So is . It implies that the ground plane is the plane whose normal vector is and distance from the camera origin is . Therefore, the ground plane equation is .
is the object point which is on the ground plane, so this point is satisfied with the ground plane equation. Let be the point on the image plane and is the intrinsic matrix of the camera, then can be written by
where and are the width and height of the camera. It means , where component of is from the ground plane equation. Consequently, is as follows.
What if ?
When , is not defined because its denominator is zero. Instead, from the ground plane equation in this case. It also implies that and . Moreover, from . Note that this result is the same with as above when .
What if and ?
When and , is not defined since its all components go to infinity. But it makes sense because if the tilt angle is zero, the view vector is parallel with the ground and there is no intersection between them, or they meet at infinity.
3. Finding a point over
Until now, is always on the ground plane. But objects are not always on the ground and they can be over the ground. In this case, it is required that the height of the object from the ground be known. Let be the height of the object, then it is possible to imagine that the ground plane is going up by . It means that this point is on the new ground plane again whose distance from the camera origin is , not . Therefore, which is over the ground can be modified as follows:
4. Accuracy on the image plane
The adjacent pixel points projected on the image plane from the ground may not close enough on the ground. In other words, the points that are close on the image plane can be far from each other in the camera coordinates. and are adjacent as above, and let and be components of the 3D points on the ground which are projected on them.
Now that and are adjacent, .
This result implies that the larger , , and are, the smaller is, the less the distance between two 3D points from and is.