Center for Mapping

PERFORMANCE EVALUATION

The AIMS integrated positioning prototype was tested under different conditions with varying base-rover separation. In these tests the IMU data update rate was 256 Hz; however, a rate of 400 Hz was applied for data collection to guarantee no loss of data. Differential GPS observations were collected by Trimble 4000 SSE/SSI receivers.

Figure 1. Typical RMS for position estimates.

Figure 2. Typical RMS for velocity estimates.

Figure 3. Typical RMS for attitude estimates.

The examples of estimated standard deviations (RMS) for position, velocity, and orientation are plotted in Figures 1 - 3. These figures indicate that positions are resolved with estimated accuracy of ~ 1 cm for horizontal, and ~2 cm for vertical components, after ~300 s. The RMS for the orientation angles stays at the level of ~5 arcsec, after the initial calibration of about 300 s. Velocity is resolved with the estimated accuracy better than 0.001 m/s. Table 1 presents typical level of standard deviations derived from the covariance matrix of estimated positions and orientation angles, as a function of base-rover separation, obtained from different flight tests.

Estimated Error Components (RMS)	Base-rover separation			Units
	50	100	200	[km]
North coordinate	10	9	9	[mm]
East Coordinate	9	8	8	[mm]
Vertical coordinate	20	21	18	[mm]
Heading	6.0	6.3	7.5	[arcsec]
Pitch	3.0	3.7	5.1	[arcsec]
Roll	3.3	4.1	5.1	[arcsec]

Table 1. Estimated errors (RMS) in position and orientation as a function of base-rover separation.

Direct Georeferencing vs. Aerotriangulation:

The test flight over Albany, CA 1997, involved GPS/INS data collection and high-resolution aerial photographs with Zeiss RMK Top aerial camera. The aerial photographs were processed on an analytical plotter, and photo coordinates were observed with an estimated accuracy of 7m. The camera exterior orientation components were obtained from bundle block adjustment based on ground control, without GPS/INS data. The results are listed in Table 2. The GPS/INS-derived camera projection center trajectory over the test field, as well as photogrammetric camera projection center positions for the instances of image data collection, are plotted in Figure 4. Unfortunately, the quality of the imagery was not satisfactory for the high-accuracy project requirements; therefore, the results show an accuracy of position and orientation of the camera projection center at the level of 20-40 cm, and 1.4-3.5 arcmin, respectively. Due to the rather modest quality, these results did not serve as a reliable reference for the GPS/INS evaluation, where the estimated errors are significantly smaller, as presented in Figures 5 and 6.

Table 2. Photogrammetric adjustment results

Figure 4. Aircraft trajectory and camera positions

Figure 5. Estimated errors in position

Figure 6. Estimated errors in orientation

The projection center coordinates obtained from both methods were compared, and examples are presented in Table 3 below. These results show that the RMS of fit is around 14-15 cm in both east and height directions, which is consistent with the order of magnitude of the initial quality of the test range plus estimated errors of GPS/INS and photogrammetric processing.

Photo ID	East coordinate difference [m]	Height difference [m]
1011	-0.09	0.22
1012	-0.03	-0.06
1213	0.28	-0.05
1014	-0.11	0.18
4041	0.09	-0.04
4042	-0.31	-0.10
4043	-0.12	0.22
2021	0.06	-0.11
2022	0.05	-0.10
2023	0.15	-0.20
2024	-0.04	-0.01
RMS	0.15	0.14

Table 3. GPS/INS positions vs. aerotriangulation results

Another airborne test, conducted in cooperation with the University of Florida and Florida Department of Transportation, was focused on the evaluation of the georeferencing performance of AIMS Ô in the production environment, with one ultimate goal of delivering a digital elevation model  (DEM) of the transportation corridor in the Callahan area.

GPS/INS Processing Results

To  analyze  the  system’s  mapping  performance,  the  GPS  and  the  inertial  data  collected  over  the  transportation corridor were processed by the integrated filter. The resulting estimated standard deviations per coordinate for the IMU center averaged around 2 cm. The estimated standard deviations for the attitude components were at the level of about 5-8 arcsec during the portion of the flight where the image data were collected.

Aerial Triangulation Solution

A block of 18 images was aerotriangulated by the standard bundle adjustment method. The effective photo scale was 1:6,000, since a 50 mm lens was used at an average altitude of ~300 m. Control points with accuracy estimated at the 2 cm level were available from a static GPS survey.  The image measurements were collected using the Autometric  SoftPlotter  digital  photogrammetric  workstation  configured  with  the  appropriate  camera  calibration parameters,  determined  during  the  camera  calibration. The accuracy of the image coordinate measurement was at the level of 7 microns. Table 4 below summarizes the aerotriangulation results.

Exterior Orientation	Image Point Measurement RMS	Perspective Center RMS	Control Point RMS
Xo	7 microns	0.12 m	0.02 m
Yo	7 microns	0.14 m	0.02 m
Zo	N/A	0.04 m	0.02 m
wo	N/A	1.5 arcmin	N/A
jo	N/A	1.27 arcmin	N/A
ko	N/A	0.3 arcmin	N/A

Table 4. Photogrammetric adjustment results

Comparison of the Results

The boresight estimation was performed by comparing AT and GPS/INS positioning and attitude estimates.  The resulting boresight standard deviations were 0.29, 0.17 and 0.15 m for the linear displacements, and 3.7, 2.7 and 1.7 arcmin for the rotation angles, respectively (for the estimates based on the six centrally located images, containing the majority of the control points, the respective standard deviations were: 0.22, 0.08 and 0.06 m for the offsets, and 1.8, 2.4 and 0.6 arcmin for the rotation angles).  These accuracy measures were not as good as expected.  Some possible reasons were photogrammetric processing accuracy, mechanical problems with the camera body/mount, and anomalies in the image time tagging. Although the photogrammetric adjustment results were relatively good (Table 4), the photogrammetric solution was composed of internal and external constraints such as tie points and control points.  These constraints, intensified by image resolution and geometric modeling inefficiencies, implied limitations on the image measurement accuracy. For example, even though the control points were surveyed at cm-level accuracy on the ground, due to their poor signalization their image representation measuring accuracy was not even close to the otherwise sub-pixel image measurement performance.  This poor localization was subsequently propagated to the projection centers’ positioning quality. Therefore, the boresighting performance was immediately compromised and never reached the accuracy range of the GPS/INS data. Table 5 presents the differences between the bundle adjustment-determined and GPS/INS-derived exterior orientation data sets, after the boresight transformation was applied to the DOPs. Table 6 shows the average differences between nominal and manual measurements of the control point coordinates.

Image	Dx[m]	Dy[m]	Dz[m]	Dk [deg]	Df[deg]	Dw [deg]
38274	0.03	0.12	0.05	-0.014	0.025	0.023
37425	0.04	-0.02	-0.03	-0.041	0.024	-0.015
37419	0.13	0.26	-0.02	-0.010	0.023	-0.033
37026	-0.10	-0.37	-0.11	-0.024	0.022	0.020
37020	-0.09	0.03	0.00	-0.025	-0.014	-0.052
36620	0.00	-0.02	0.10	-0.015	-0.058	-0.066
Mean	0.00	0.00	-0.00	-0.021	0.004	-0.020
RMS	0.09	0.21	0.07	0.010	0.033	0.043

Table 5. The differences between the boresighted GPS/INS and AT-derived exterior orientation parameters.

Table 6. Average differences between nominal and manual measurements of control point coordinates

LAND VEHICLE-BASED TEST  

The integrated GPS/INS/CCD system was also tested in terrestrial applications. The sensor assembly was mounted on  the  top  of  a  van  for  acquiring  data  for  a  topographic  survey  of  transportation  corridors.  The  imagery  was collected along the surveyed road, and the subsequent stereo-pairs (formed by the time-offset succeeding images) were  formed  with  the  directly  acquired  orientation  parameters. The quality of the direct orientation parameters, represented by the standard deviations, is presented in Figures 7 and 8. The  resulting  GPS/INS  positioning  standard deviation stayed at the level of 1-2 cm; pitch and roll standard deviations were at ~5 arcsec level, while for heading it reached 7 arcsec. The spikes that can be observed in the position standard deviations correspond to partial or total losses of GPS lock when the vehicle was passing under the foliage or close to the buildings.

Figure 7. Attitude standard deviations

Figure 8. Position standard deviations.

Examination of the repeatability of the solution obtained for the checkpoints measured in different directly oriented stereo pairs provides an overall accuracy indication of the system. The statistics of such a comparison, based on 44 stereo pairs, is presented in Table 7, and indicate that the direct orientation parameters were indeed estimated with high quality.  Another repeatability test was performed by comparing the ground coordinates of 15 check points measured on the directly oriented stereo pairs from two different passes, as shown in Table 8. The GPS/INS/image data for those passes were collected with slightly different GPS constellation; the first pass observed six to seven satellites, whereas the second pass was able to collect GPS data from no more than five satellites. This is reflected in the differences listed in Table 8 that are slightly higher than their counterparts from Table 7.

Statistic	Easting [m]	Northing [m]	Height [m]
Mean	0.015	0.004	0.008
Median	0.006	0.003	0.006
Maximum	0.050	0.025	0.035
RMS	0.019	0.007	0.010

Table 7. Ground coordinate difference for the control points measured from different stereo pairs.

Statistic	Easting [m]	Northing [m]	Height [m]
Mean	0.015	0.014	0.044
Median	0.013	0.011	0.045
Maximum	0.050	0.034	0.130