Skip to content

Troubleshooting

This page describes the different sources of error in the calibration, how to identify them, and hopefully solve them.

1. Troubleshooting Workflow Example

1.1. Check the reprojection error

Before the calibration is even completely finished, EZlineup prints to the console the average pixel reprojection error accross all images:

Here the final average error is about 7.6px.

1.2. Check the reprojections

EZlineup will try to reproject the cube using the tracking data and the offsets computed for each picture. The "Display Calibration Result" section allows you to scroll through the pictures looking at the reprojection:

One of the pictures has this reprojection:

But most others have this kind of reprojections:

Which is not absolutely perfect but much better still.

In that case, the issue was that the camera most likely moved while capturing the picture that failed. After removing that picture from the dataset, the global reprojection error falls down to 3px.

1.3. Check the visual reprojection

Let's say we want a better result than this. We need to discriminate whether the issue comes from Tracking or from the Lens Calibration. The Display Visual Pose Estimation section allows you to see the reprojection of the cube using only vision based data, that is it will try to reproject the cube using the pose estimated from the detected tags and the lens characteristics, but not use the tracking data.

The reprojections look spot on, even in the corners (where lens distortion has a greater effect)

So in that case, the lens calibration is not causing the error.

This means the tracking system is too noisy or drifts. The solution would be to check the tracking setup, or upgrade to a better tracking solution. For this test the tracking was provided by the lighthouse technology which is not very precise, so a 3px reprojection error in wide angle is to be expected. To improve on this we could switch to mechanical tracking or a high-end motion capture system.

2. Illustration with synthetic data

2.1. Perfect conditions

To isolate the different causes, we will use a synthetic dataset as an example.

Here, both the lens data and tracking data are perfectly known when creating the dataset. Constant referential and tracker offsets unknown to EZlineup were applied on the tracking.

The reprojection error is 0.4px and EZlineup finds both offset transforms exactly.

2.2. Wrong lens calibration

In this example, the focal length given is 30mm instead of the theoretical 40mm

EZlineup fails to reproject the perspective of the virtual cube on the real cube. Looking at the Visual Pose Estimation, we see that even before introducting the tracking data, the reprojection fails.

So the issue here is that the lens calibration is wrong.

2.3. Noisy tracking

In this example, we are using the exact lens characteristic, but introducing noise on the tracking data. We introduce both jitter and some drift. Both will cause issues with the line-up calibration.

Naturally, since the lens is perectly known, in all cases the Visual Pose Estimation works very well:

2.3.1 Low noise

The noise level is similar to that measured with the lighthouse technology.

The reprojection error is around 4px:

You have to look fairly closely to see the offset so in most cases that is acceptable.

2.3.2 High noise

Here the noise standard deviations are multiplied by 5, to represent a faulty tracking setup:

The reprojection error is around 33px:

If the tracking jitter is causing the problem, the line-up will be the least of your issues because the AR will look bad anyways.

If the tracking drift is causing the problem, for example with a wrong internal calibration of the encoders for mechanical or of the tracking space for optical, the tracking problem may not be obvious from AR alone because it will still look smooth. The problem is that the tracking is not spatially consistent so it is impossible to globally solve the line-up and referential problem.

3. Reprojection Error per Picture

In Advanced mode, the reprojection error for each picture is displayed next to the picture name:

This can be useful if there is suspicion of an erroneous picture inducing error on the result.