Zhou Fang, Xinrong Wang, Wei Ji, Meng Xu, Yinan Zhang, Yan Li, Longfei Li

Abstract: The evaluation of geometric calibration accuracy of high resolution satellite images has been increasingly recognized in recent years. In order to evaluate geometric accuracy for dual-camera satellite images based on the ground control points (GCP), a rigorous geometric imaging model,which was based on the collinear equation of the probe directional angle and the optimized tri-axial attitude determination (TRIAD) algorithm, is presented. Two reliable test fields in Tianjin and Jinan (China) were utilized for geometric accuracy validation of Pakistan Remote Sensing Satellite-1. The experimental results demonstrate a certain deviation of the on-orbit calibration result from the initial design values of the calibration parameters. Therefore, on-orbit geometric calibration is necessary for optical satellite imagery. Within this research, the geometrical performances including positioning accuracy without/with GCP and band registration of the dual-camera satellite were analyzed in detail, and the results of geometric image quality are assessed and discussed. As a result, it is feasible and necessary to establish such a geometric calibration model to evaluate the geometric quality of dual-camera satellite.

Keywords: geometric calibration; image quality; dual-camera; high resolution satellite

With the rapid development of imaging technology, high resolution remote sensing images have been widely applied in disaster detection, environmental protection, and resource exploration[1].In each link of the earth observation network,high-precision geolocation accuracy is the basis for its application effectiveness and value of the spatial information services, and the accurate calibration of the geometric parameters of the remote sensing satellite imaging system is an indispensable key link. Due to the influence of space thermal and mechanical environment during satellite launch and operation, the laboratory calibration parameters of satellite imaging system have large errors and cannot be applied directly.

At present, for the on-orbit geometric calibration of optical remote sensing satellite, most of the methods used by domestic and foreign researchers need to utilize the high-precision reference data of the ground calibration test site. An excellent test site is necessary for running the ground reference targets and related instruments.Using high-accuracy ground control points(GCP), the reference orthophotos and corresponding digital elevation model (DEM), the conventional geometric calibration method can achieve required geometric positioning accuracy.In the past few decades, various calibration and validation sites have been built worldwide, e.g.,the La Crau test site in French[2], Vaihingen/Enz in German[3], Stennis in USA, Sjökulla site in Finland[4], Barrax site in Spain, Baotou, Songshan and Dunhuang site in China.

Typical methods for calibration and validation after launch involve either in-situ based measurements or model-based predictions[5]. The in-situ based measurements have been used successfully for the calibration of the SPOT,IKONOS, Rapideye, Pleiades, and GaoFen sensors[6−10]. The method requires simultaneous or near simultaneous views of the source or assumptions that the in-situ based measurements are stable in time. The model-based methods such as relying on on-orbit manoeuvrable imaging provide high-precision data with high temporal sampling. Such data are excellent for analyzing trends in the geometric calibration of a sensor, but not necessarily sufficient from a geometric positioning accuracy[11,12].

Although previous studies have made progress in on-orbit geometric calibration to a certain extent, some key scientific issues remain unsolved, and it is hard for the end user to obtain the dual-camera product due to image quality and processing time issues. In this paper, the related theory and method of dual-camera on-orbit geometric calibration are reached. Firstly, the geometric calibration procedures are introduced,we build an on-orbit geometric calibration model and study the corresponding calculation method of the calibration parameters. Secondly, through performing a reasonable optimizing and parameter selection of the dual-camera splicing algorithm, we study the corresponding calculation method of the calibration parameters. Finally, an on-orbit geometric calibration experiment and image quality are performed and verified for PRSS-1(Pakistan Remote Sensing Satellite-1)dual-camera with the reference data offered by Tianjin and Jinan calibration test sites. Our proposed calibration model and dual-camera splicing method has been proven to be stable and effective, and could significantly improve the geometric accuracy of dual-camera optical satellite imagery.

1 Geometric Calibration Procedures

1.1 Geometric Calibration Workflow

The geometric calibration objects mainly include the interior orientation parameters and the exterior orientation parameters. The former refers to the parameters of camera interior orientation(principal point coordinates, imaging sensor pixel size, pixel positions on the focal plane, and principal distance) and Lens distortions; the latter refers to relative alignments of the position and attitude measurement devices and the camera.Fig.1 is the geometric calibration workflow.

Fig.1 Geometric calibration workflow

1)Use the recording file of GCP or high-precision orthophoto image as the reference data, select homonymy points between the two images as the connection point.

2)Utilize the laboratory calibration parameters (including the elements of interior and exterior orientation) as the initial parameters of geometric calibration model.

3)Build the external calibration model and calculate the parameters.

4)Take the result of external calibration as the new input to construct the internal calibration model and calculate the parameters.

5)Take elements of interior and exterior orientation as the new calibration parameters to calculate the connection point residual, if the residual is less than the set threshold, the calibration is completed; otherwise, re-calculate the parameters.

6)Apply the dual-camera splicing algorithm,obtain the corresponding image product.

7)Verify the image quality by obtaining the accuracy reports including geometric positioning accuracy without GCP, geometric positioning accuracy with GCP, and band registration accuracy.

8)Monitor the on-orbit geometric image quality on schedule, execute the on-demand geometric calibration based on the image quality assessment results.

1.2 Mathematical Model of Geometric Calibration

Based on resection theory, the geometric calibration method calculates the systematic error of camera elements of interior and exterior orientation including the fitting coefficient of each charge coupled device (CCD) orientation angle and the install matrix from camera coordinate to satellite body coordinate. Considering the strong correlations between the interior and exterior parameters, the calibration can be divided into two steps. Calibrating the exterior parameters and the interior parameters subsequently, the result of the calibration is obtained by iterative solution.

1.2.1 Rigorous Geometric Imaging Model

Establishment of a rigorous geometric imaging model is the first step of on-orbit geometric calibration for optical satellite imagery[13,14]. Based on the collinear equation of the probe directional angle, a rigorous geometric imaging model can be constructed (Fig.2).

Fig.2 CCD probe directional angle model

The equation is described as

where (XG,YG,ZG) is the position vector of the ground target in the WGS84 coordinate system,(XS,YS,ZS) is the position vector of the satellite in the WGS84 coordinate system,uis the scale factor,PSI_Xis the angle between the vector inXOZplane of the satellite body coordinate system and the direction ofOZ,PSI_Yis the angle between the vector on theYOZplane of the satellite body coordinate system and the direction ofOZ.M1,M2, andM3are rotation matrices from the instantaneous camera’s coordinate system to the satellite’s body-fixed coordinate system, from the satellite’s body-fixed coordinate system to the J2000 coordinate system, and from the J2000 coordinate system to the WGS84 coordinate system, respectively.

1.2.2 Attitude Determination Algorithm Using Dual-Vector Measurements

Positioning accuracy error sources mainly include orbit measurement error, attitude measurement error, and error caused by stability of optical axis of camera and star tracker. Although the sensor position parameters measured by today’s orbit determination technology have certain errors, their accuracy is much higher relative to the attitude parameters, and the impact to the image positioning is almost negligible[15].TRIAD is an algorithm that finds the orientation difference between the two star tracker systems. The accuracy of each star tracker is quantified by the standard deviation of its error. Based on the original TRIAD algorithm, an optimized TRIAD algorithm was proposed[16]. The vector measured by a star tracker is assigned a standard deviation smaller than that assigned to the other. Given attitude matrixA1, with its assigned standard deviationσ1, andA2, with its assigned standard deviationσ2, we can findAˆ′, an estimate ofAthat is better than eitherA1orA2,that is,

The weights are determined by the statistics of the two star trackers that provided the vector measurements. Accurate attitude parameters can be obtained and put into the rigorous geometric imaging model to eliminate the attitude measurement error.

1.2.3 Elements of Exterior Orientation Calculation

The goal of the geometric exterior calibration is to introduce the error matrixMinto following equation, which becomes

By using a set of GCP (sample, line, latitude, longitude, height), we can get [XS,YS,ZS],[XG,YG,ZG],M3,M2,M1, so there

According to the principle of vector equality,the unit vectors at the left and right ends of following equation should be equal.

In this paper, the calibration matrix is set to a matrix consisting of the angles of rotation aroundX,YandZaxes, which is

From (7) we can get

Using the GCPs to solve the three rotation angles in (8), the error compensation matrix of the satellite can be obtained.

Taylor expansion of (8) at (φ,ω,κ)=(0,0,0),we can get

Use (9) to establish a linear system of equations, and use least squares iterative solution to get the camera’s elements of exterior orientation.

1.2.4 Elements of Interior Orientation Calculation

According to the geometrical imaging model based on the orientation angle of the probe, the interior calibration parameter model is given as

(a0,a1,a2,a3,b0,b1,b2,b3) represents the interior calibration parameters, describes the camera focal plane CCD probe angle;Srepresents the probe number;PSI_X(S) andPSI_Y(S) represents a specific probe in the camera coordinates of the two directions of the angle.

The purpose is to findPSI_XandPSI_Yin(10) for each set of GCP on the images.

For each set of GCP (sample, line, latitude,longitude, height), [XS,YS,ZS], [XG,YG,ZG],Mcan be obtained by establishing a strict imaging model after external calibration. There are

According to the principle of vector equality,the unit vectors of the vectors at both ends of(12) are also equal.

So

ForNcontrol points, the interior calibration is performed respectively to obtain the accurate ground pointing angles along the alongtrack and the across-track at theNcontrol points which are uniformly distributed. Fit theNcontrol points, and calculate the mathematical equations between each camera’s probe (column number) and its pointing angle can obtain the accurate interior orientation element at each probe of the camera.

1.2.5 Dual-Camera Splicing Algorithm

After each camera is calibrated, how to obtain dual-camera splicing product is researched.As there are two independent cameras, the installation error causes the time difference between the two cameras when imaging the same ground object in the overlapping area. This time difference leads to the relative error of attitude data used in the positioning without GCP, causing relative positioning error. In addition, the stability of on-orbit environment, the relative relations between two independent optical axes of the camera may randomly changes, these uncertain factors lead to position deviation of two cameras location. Calibrated camera installation matrix and interior elements can be used with the virtual CCD splicing technology to achieve rough splicing of the dual-camera, and the overlapping area shall be registered to realize accurate splicing.

By using gray correlation matching algorithm or cross-correlation matching algorithm[17−19],the coordinates of the homonymy points in splicing area of the two cameras are obtained, then the relevant model between the two cameras can be established according to the corresponding relationship. The least squares method is used to calculate transformation parameters between the two cameras, and optimize the re-locating process by configuring the transformation parameters automatically.

The dual-camera splicing algorithm is carried out as follows:

1)Establish the rigorous geometric positioning model of the camera CCD by using the real auxiliary data of the camera including interior elements, attitude, orbit and row time.

2)Obtain the ideal auxiliary data of the camera including real interior elements, attitude,orbit and row time, as the model parameters of the virtual CCD splicing.

3)Establish the rigorous geometric positioning model of the virtual CCD and establish the calculation model between image points and ground object points.

4)Establish the high-precision rational polynomial coefficient (RPC) model of virtual scanning scene and calculate the RPC parameters.

5)Establish the mapping relationship between the rigorous geometric positioning model of each CCD and the virtual CCD, and map through the same coordinate, so as to establish the calculation model between the image point on each CCD and the virtual CCD.

6)The images of the virtual CCD are obtained by resampling the CCD of the camera.

7)In the two camera splicing area of one scene image, 3 percent points in the across-track can be conducted uniformly to obtain multiple matching points.

8)Use camera 1 as reference to fit and calculate the distortion model parameters of camera 2 relative to camera 1;

9)Calculate the distortion quantities dxand dyof each pixel of camera 2 in theXandYdirections according to the distortion model parameters;

10)In the process of dual camera splicing,the distortion quantities dxand dyof each pixel of camera 2, is added to the re-locating process to obtain the precise splicing image.

2 Test Data Set and Results Analysis

2.1 Image and GCP Data Set

PRSS-1 was successfully launched on July 9,2018. It is a 1 228 kg satellite orbiting in a circular sun-synchronous low Earth orbit, at an altitude of 643 km. The payloads consist of two cameras, and 1 m panchromatic (PAN) and 3 m 4-channel multispectral (MSS) sensors in one camera. The width of single camera and dualcamera product are 30 km and 60 km, respectively.

The GCP data sets and PRSS-1 image were used to test the accuracies of the geometric calibration. The characteristics of each data set are described in Tab.1.

Tab. 1 The characteristics of each data set

The GCP data sets are located in Tianjin and Jinan. In Tianjin area (Fig.3b), the longitude ranges from 116°35' to 117°46', and the latitude ranges from 39°16' to 39°43', providing more than 1 000 GCPs, and the absolute geolocation accuracy of GCPs is better than 10 cm (mean square error). In Jinan area (Fig.3c), the longitude ranges from 116°59' to 118°6', and the latitude ranges from 36°24' to 36°51', providing more than 1 000 GCPs, and the absolute geolocation accuracy of GCPs is also better than 10 cm(mean square error).

Fig.3 The overall scene of the test sites

2.2 Calibration Results Analysis

2.2.1 Elements of Exterior Orientation

Tab. 2 summarizes the results of the calibrated exterior orientation parameters of each camera and the combined dual-camera. The results of the calibrated position parameter are given in WGS84 coordinate system in which the least squares estimation procedure was carried out.The orbit No. 850 was acquired in nadir, and 244 GCPs were selected for the PAN image, 239 GCPs were selected in Band 1 of MSS image, 121 GCPs were selected for each Band 2, Band 3 and Band 4 of MSS image.

Tab. 2 Elements of exterior orientation of each sensor

The camera installation angle changed to some extent during satellite on-orbit operation.These errors could significantly degrade the positioning accuracy of satellite imagery without control if on-orbit calibration for the camera installation angle is not done periodically.

2.2.2 Elements of Interior Orientation

Four CCDs are spliced for PAN sensor, and one CCD includes 8 192 pixels; and four CCDs are spliced for MSS sensor, and one CCD includes 3 072 pixels. To analyze the characteristics of the camera’s internal distortion objectively and quantitatively, differences in directional angles of the post-calibration were calculated and then plotted in a distortion curve. Fig.4 depicts the calibrated interior orientation parameter of each sensor. The accurate ground pointing angles along the along-track and the across-track were obtained.

From the distortion curve shown in Fig.4,we found that an obvious nonlinear distortion exists at along-track direction, and there is almost no internal distortion at across-track direction.Internal calibration must be done for the two cameras; otherwise, internal geometric distortion would be remarkable and the geometric quality of the images would be reduced considerably.

Fig.4 Elements of interior orientation of each sensor:(a)PAN-1;(b)PAN-2;(c)MSS-1 band-1;(d)MSS-1 band-2;(e)MSS-1 band-3;(f)MSS-1 band-4;(g)MSS-2 band-1;(h)MSS-2 band-2;(i)MSS-2 band-3;(j)MSS-2 band-4

3 Image Quality Verification and Control

3.1 Geometric Positioning Accuracy Without GCP

To evaluate the effectiveness of the calibration results, multiple PRSS-1 PAN and MSS images in the regions of Tianjin and Jinan were chosen.To verify the geometric positioning accuracy without GCP, how close the corresponding geolocated pixel in the image is to the calibrated GCPs is measured. We record these data and from that data find the circular error 90%(CE90) which is measure of the error in horizontal and vertical distances from where the satellite indicated it was pointed and where it was actually pointed when it imaged the test sites.

3.1.1 Single Camera Accuracy

Visually from Fig.5, the circle denotes the specification for the sensor in terms of CE90 which specifies that 90% of the GCPs must lie within that circle. For single camera positioning accuracy verification, the PAN geometric positioning accuracy without GCP of camera 1 and camera 2 are 25.07 m (CE90) and 24.95 m (CE90), respectively; and the MSS geometric positioning accuracy without GCP of camera 1 and camera 2 are 26.15 m (CE90) and 26.14 m (CE90), respectively.

Fig.5 Single camera positioning accuracy without GCP:(a)PAN positioning accuracy without GCP of camera 1; (b)MSS positioning accuracy without GCP of camera 1; (c)PAN positioning accuracy without GCP of camera 2; (d)MSS positioning accuracy without GCP of camera 2

3.1.2 Dual-Camera Accuracy

Visually from Fig.6, for dual-camera positioning accuracy verification, the PAN geometric positioning accuracy without GCP is 20.93 m (CE90),and the MSS geometric positioning accuracy without GCP is 24.3 m (CE90).

Fig.6 Dual-camera positioning accuracy without GCP:(a)PAN positioning accuracy without GCP; (b)MSS positioning accuracy without GCP

From the verification results of single camera and dual-camera, the positioning accuracy of the dual-camera has been improved in a certain extent. Compared with the single camera positioning, the dual-camera positioning uses more GCPs, and the compensation error matrix can obtain more auxiliary data, and the dual-camera splicing algorithm also makes an impact.

3.1.3 Accuracy Improvement Results

The results of dual-camera positioning accuracy without GCP using different attitude measurements are listed in Tab.3. For nadir, near nadir and off-nadir images, the accuracies of the dualcamera images have been greatly improved when the optimized TRIAD algorithm is applied. Quite specially, the accuracy obtains a great improvement for the off-nadir image (orbit No. 1624).The results shown systematic error is well controlled by using two vector measurements. And it indicates that the attitude measurement accuracy is significant and the key factor influencing the geometric positioning accuracy without GCP.

Tab. 3 Positioning accuracy without GCP using different attitude measurements

3.2 Geometric Positioning Accuracy with GCP

A translation model on the image space of the RPC is used for correction with three or less than three GCPs. The experimental results are listed in Fig.7. Image products include single camera and dual-camera sensors are geo-located with respect to the WGS84 reference ellipsoid, with an overall accuracy better than 5 m CE90 using not more than 3 GCPs. The correction can remove the error caused by attitude measurement, satellite position measurement and the terrain undulations, and the geometric accuracy is stabilized with a small number of control points (up to three). This practice not only improves the geometric quality of the image products greatly but also significantly reduces the in-situ cost.

Fig.7 Geometric positioning accuracy with GCP

3.3 Band Registration Accuracy

The band registration accuracy validation tests the band registration accuracy of 4 bands of MSS image and the registration accuracy between PAN and MSS images. Take one band as reference image, and other bands are test images.Considering the consistent resolution requirement, PAN image will be resampled to the resolution of MSS images. And set the PAN image as the reference band, and set the 4 bands of MSS image as the test images.

The calculation method of band registration accuracy is as follows

whereDis band registration accuracy,Nis sum of registration points,XiisXpixel coordinates of reference band,xiisXpixel coordinates of test band,YiisYpixel coordinates of reference band,yiisYpixel coordinates of test band.

The images captured by the MSS sensor of the orbit No. 850, 1 137, and 1 336 are band-registered. In the test process, the Band 3 was used as the reference band. For the orbit No. 850,Band 1 and Band 3 are matched by 236 points,Band 2 and Band 3 match 240 points, and Band 4 and Band 3 have 18 matched points. For the orbit No. 1 137, Band 1 and Band 3 are matched by 282 points, Band 2 and Band 3 match 290 points, and Band 4 and Band 3 have 12 matched points. For the orbit No. 1 336, Band 1 and Band 3 are matched by 265 points, Band 2 and Band 3 match 254 points, Band 4 and Band 3 have 31 matched points.

As shown in Tab.4, the mean error of the ErrorAbs of the Band registration of the three test data is 0.068 22 pixels (0.21 m).

The images captured by the MSS sensor and PAN sensor of the orbit No.850, 1 137, and 1 336 are band-registered. In the test process, the PAN was used as the reference band.

The images have been successfully registered with 117, 111 and 142 points, respectively. As shown in Tab.5, the mean error of the ErrorAbs of the band registration of the three test data is 0.185 340 MSS pixels(0.54 m).

Tab. 4 Registration accuracy among MSS bands

Tab. 5 Registration accuracy between PAN band and MSS bands

3.4 Periodical Calibration Activity

Maintaining system performance using traceable standards and calibration minimizes the time spent validating test results and troubleshooting instrument inconsistencies at the satellite life cycle. Calibration is typically performed at regular intervals recommended by the manufacturer.

The camera’s interior orientation elements are relatively stable, test of effectiveness of the camera’s exterior orientation elements and the calibration process of satellite on-orbit after the previous geometric calibration after 6-12 months later, can be applied to the image quality software, combined with the control precision of the statistical control positioning deviation, if the deviation exceeds the system accuracy requirements, that the interior and exterior parameters of the camera shall be calibrated.

For PRSS-1, we combine the experience and the assessment to plan the multi-temporal calibration activities. Two calibration activities were planned at January and October, 2020, respectively. Tab. 6 lists the elements of exterior orientation of each sensor at 2020. Parameter Roll was changed in a certain extent that indicates the calibration is necessary.

Tab. 6 Elements of exterior orientation of each sensor at 2020 year

Maintaining the performances is crucial to ensuring that the instruments are performing with traceable accuracy. For PRSS-1, the set thresholds for positioning accuracy without/with GCP, MSS bands registration, and PAN/MSS band registration accuracy are 50 m (CE90), 5 m(CE90), 1/3 of a MSS pixel (CE99.7) and 2 PAN pixel (CE99.7), respectively. Visually from Fig.8, the percentage represents the ratio of obtained accuracies to the set thresholds, and five times are present (For label, b means before the time. For instance, b2020-1 means the monitoring results before the first calibration in 2020).

Fig.8 Multi-temporal calibration results

The positioning accuracy without GCP is most heavily affected as time goes on. Before 2020-1 calibration, the positioning accuracy without GCP is 41.31 m (CE90). After 2020-1 calibration, the positioning accuracy without GCP is 11.82 m (CE90). Before 2020-2 calibration, the positioning accuracy without GCP is 31.48 m (CE90). After 2020-2 calibration, the positioning accuracy without GCP is 14.23 m(CE90). For other index, the fluctuation ranges are accepted because the relatively tight ranges.

4 Conclusions

Aimed at the geometric calibration of the high resolution dual-camera optical satellite, this paper has proposed an imaging geometric model that makes use of each camera’s probe and the optimized attitude determination algorithm using two star tracker measurements to obtain the accurate attitude parameters. Furthermore, for the attitude uncertainly caused by the satellite,the error matrix is used to eliminate the systematic error so as to obtain the accurate calibration parameters.

The validity and reliability of the method proposed in this paper has been tested and verified. In addition to single camera products, the dual-camera products’ image quality index including geometric positioning accuracy with and without GCP, band registration accuracy have been verified. By using the homonymy points in splicing area of the two cameras, the dual-camera splicing algorithm is stable and increases the image quality precision in a certain extent.

Admittedly, there are some shortcomings to the method. In this paper, the error of exterior orientation elements and interior orientation elements are calibrated one by one, and the errors are believed to be independent from each other.In reality however, the sources of the errors caused by the satellite motion are complicated,and the effects of errors may overlap with each other, and as a result, affect the stability of the imaging geometric model. Therefore analysis on the variation with time in the geometric calibration parameters during on-orbit operation lifecycle will be carried out in the future.