Glossary
1. Definitions
1.1. The Tracking System
The tracking system broadly designates the solution used for tracking the camera pose. This encompasses any system, be it optically based like Lighthouse, Optitrack or Vicon, or mechanical like a PTZ or a custom encoded rig, that gives a pose in a referential.
- Tracking Referential : This is the reference frame in which the positions and rotations are given. It corresponds to a point in space and three axes defined by the tracking system. That point is the Tracking Origin
- Tracker : This is the point in space where the pose is measured.
1.2. The Camera
The camera is a large object, but for the render engine, its position corresponds to where an equivalent pinhole model camera would produce the same image when distortion is removed.
This point does not exist physically, however its position relative to the sensor is measured during the optical calibration. This is the Entrance Pupil Distance, or Nodal Offset. The reference point on the camera for the tracking should therefore be the camera sensor, more precisely the intersection of the optical axis and the sensor.
The Tracker can't be placed at this position so the pose of the sensor center relative to the tracker has to be measured. This is the Tracker Offset, or Line-Up
1.3. The Studio
The Studio space defines the position of real objects and actors in the studio. The virtual space has to match the studio space exactly so that the virtual objects don't slide around and people on set can have an idea of their position.
- Studio Referential : This is the reference frame of the studio. It corresponds to a point in space and three axes, most likely clearly visible on set with tape or some other markers. That point is the Studio Origin
The tracking origin doesn't necessarily have to be in the same position as the studio origin. Sometimes the studio origin that makes the most sense for people on set is not practical for the tracking system. In this case the offset between the tracking origin and the studio origin need to be measured. This is the Referential Offset.
2. Tracking Data
1.1. Referential Definition
In EZlineup, all transforms follow the FreeD convention, which the supported tracking protocols also follow:
- X: Right axis, positive when camera moves toward the right
- Y: Front axis, positive when camera moves forward
-
Z: Up axis, positive when camera moves up
-
Pan: Positive when camera turns to the right (left-hand rule around Z axis)
- Tilt: Positive when camera tilts up (right-hand rule around X axis)
- Roll: Positive when camera rolls to the right (right-hand rule around Y axis)
In terms of referential, this means the front of the studio is the Y axis, the right is the X axis, and Z toward the ceiling.
1.2. FreeD protocol
In real-time applications, the most widespread protocol for raw tracking data transmission is FreeD. This is one of the protocol EZprofile uses as input. It contains the following fields:
| Parameter | Format | Precision |
|---|---|---|
| X | Integer | 24 bits |
| Y | Integer | 24 bits |
| Z | Integer | 24 bits |
| Pan | Integer | 24 bits |
| Tilt | Integer | 24 bits |
| Roll | Integer | 24 bits |
| Zoom | Integer | 24 bits |
| Focus | Integer | 24 bits |
FreeD is also the protocol most tracking systems use to send data to render engines currently. Since Zoom and Focus are encoded as 24 bit integers, the receiver can not deduce even the basic camera intrinsics like focal length and focus distance from these values directly.
This is why it is necessary to load a lens profile in EZlineup when doing a calibration using the FreeD protocol.
1.3. TCD protocol
Miraxyz develops its own protocol called TCD (Tracking Calibrated Data). There are multiple possible formats depending on the data to send. In addition to position and rotation data, the tracking system sends calibrated data (Field of View, Focus Distance, Aperture, etc...).
So using TCD the lens file is located inside the tracking system instead of being loaded at the receiving end. So a TCD stream is enough for EZlineup to get all the data it needs.