Open Access Paper
12 July 2019 On ground demonstrator of digital stabilization for high-resolution Earth observation time of delay integration imaging
S. Petit Poupart, A. Materne, C. Thiebaut, J. M. Delvit, C. Latry, B. Mithieux
Author Affiliations +
Proceedings Volume 11180, International Conference on Space Optics — ICSO 2018; 111804Q (2019) https://doi.org/10.1117/12.2536089
Event: International Conference on Space Optics - ICSO 2018, 2018, Chania, Greece
Abstract
CNES (French Space Agency) has developed a dedicated test bench to demonstrate its solution of digital stabilization for relaxing the AOCS constraints for high-resolution Earth observation Time of Delay Integration (TDI) imaging, based on a CNES patent1. TDI sensors are sensitive to high frequency attitude disturbances which may induce blurring effects when increasing TDI stages number and thus exposure time. A solution to relax the microvibration constraints (or no longer constrain the number of lines to be accumulated) is to compute, in real-time, the shift between each line and coregister them before summation. CNES solution includes a motion sensor dedicated to shift measurements, a fast realtime algorithm to compute the shift between two consecutives images delivered by the motion sensor, a multi-frame TDI sensor, and an algorithm to resample images delivered by sub-TDI devices using computed shifts, before summing them up in order to obtain the final image. The optimization study of the motion sensor and the performances of the gradientbased algorithm on these images have been already presented in a previous paper2. In this paper, we describe the dedicated test bench which contains each part of the presented chain and the obtained performances.

1.

INTRODUCTION

CNES (French Space Agency) has developed a dedicated test bench to demonstrate its solution of digital stabilization for relaxing the AOCS constraints for high-resolution Earth observation Time of Delay Integration (TDI) imaging, based on the CNES patent1. TDI sensors are sensitive to high frequency attitude disturbances which may induce blurring effects when increasing TDI stage number and thus exposure time. A solution to relax the microvibration constraints (or no longer constrain the number of lines to be accumulated) is to compute, in real-time, the shift between each line and coregister them before summation. CNES solution includes:

  • - a motion sensor dedicated to shift measurements. This sensor sees the same landscape possibly at a lower spatial resolution but at a higher frequency than the principal mission’s integration time,

  • - a fast real-time algorithm to compute the shift between two consecutives images delivered by the motion sensor: the motion computation algorithm,

  • - a multi-frame TDI sensor,

  • - and an algorithm to resample images delivered by sub-TDI devices using computed shifts, before summing them up in order to obtain the final image. Typically, a hundred TDI lines could be reached by registering and adding 10 images of sub-TDI of 10 lines each, each sub-TDI having a classical behavior.

A schematic view of the entire process is shown Figure 1.

Figure 1.

TDI CMOS sensor proposed by CNES for motion compensation.

00191_PSISDG11180_111804Q_page_3_1.jpg

The optimization study of the motion sensor and the performances of the gradient-based algorithm on these images have already been presented in a previous paper2, so they will not be detailed in this one. We invite the readers to first read this previous paper2.

In the present paper, we describe the dedicated test bench developed by CNES (see Figure 2 and 3) which contains each part of the presented chain and the obtained performances. A system allowing the generation of controlled disturbances of the line of sight has been specifically developed, consisting of a parallel faces blade oscillating along two axes. Four position sensors measuring the applied disturbance allow to compare the computed shift to the injected controlled disturbances. The motion sensor is a fast commercial CMOS camera. The TDI sensor is a multi- CCD TDI. A rotating plate topped by a mirror with a very well stabilized rotation speed scrolls a specific pattern on both detectors. The pattern contains a landscape and a slanted edge devoted to MTF (Modulation Transfer Function) assessment. Software integrates motion detection and resampling algorithms.

Figure 2.

Picture of the digital stabilization demonstration CNES test bench.

00191_PSISDG11180_111804Q_page_3_2.jpg

Figure 3.

Presentation of the digital stabilization demonstration CNES test bench. Each parts will be detailed in the next paragraphs

00191_PSISDG11180_111804Q_page_4_1.jpg

The final performances of the multi-frame registration are evaluated comparing image quality and MTF values on the resulting image with and without motion correction. The targeted MTF loss due to this motion compensation process is less than 5% which represents one third of a pixel.

Section 2 is dedicated to the description of each part of the test bench. Then, in Section 3, the obtained performances are presented.

2.

PARTS OF THE DIGITAL STABILIZATION DEMONSTRATION CNES TEST BENCH

2.1

General description

Figure 3 shows the different parts of the digital stabilization demonstration CNES test bench. There are:

  • - An integration sphere for white light,

  • - A narrow filter centered on 650 nm (+/-10 nm),

  • - A test pattern with a landscape and specifics patterns,

  • - A system allowing to generate controlled disturbances of the line of sight, consisting of a parallel face blade oscillating along two axes,

  • - A symmetrical lens, with a magnification of 1,

  • - A rotating plate topped by a mirror, ensuring that the sighting pattern on both detectors runs smoothly and regularly,

  • - Two image capture systems: a multi-frame TDI sensor (13 μm pixels) and a motion sensor (2D camera with 5.5 μm pixels),

  • - A beam splitter for simultaneous projection on the TDI sensor and the motion sensor,

  • - Software that integrates motion detection algorithm and resampling algorithm.

Each parts will be detailed in the next paragraphs

2.2

Optical line disturbance

The optical line disturbance system has been developed by Cedrat Technologies (Figure 4). It consists of a 20mm thick parallel face blade, which can oscillate along the X and Y axes with an angle of 1° 0-peak, at a frequency of 1Hz to 1kHz. The maximum disturbance of 1° 0-peak corresponds to image shift of 121 μm 0-peak on the detector, i. e. 9.3 pixels TDI of 13 μm. These specifications allow to cover the attended disturbances in flight. The amplitude of the movements applied by the disturbance is measured by four position sensors placed in a crosswise position on the disturbance. These signals can be used to determine the angle of the parallel face blade at any time.

Figure 4.

View of the Optical line disturbance system.

00191_PSISDG11180_111804Q_page_5_1.jpg

The disturbances estimated in flight on high-resolution observation satellites (Pleiades Satellite measurements) is presented in Figure 5. This template is the result of AOCS simulations combined with on flight experimental feedback. Then this template was adapted to the test bench condition (see section 2.9)

Figure 5.

Flight disturbance simulated template.

00191_PSISDG11180_111804Q_page_5_2.jpg

2.3

Rotating plate

The rotating plate allows very stable image scrolling on detectors with an instantaneous angular instability of less than 0.04[as] (i. e. 1.11e-5 degrees). It is necessary to have the best rotation stability as possible not to have an additional uncontrolled disturbance during shooting.

2.4

The test pattern

The test pattern engraved by OPTIMASK consists of a 13mm x 13mm (1000x1000 pixels TDI) landscape engraved in 4 μm pitch and 64 gray levels. Different patterns have been engraved over this landscape: 4 uniform zones of different level allowing the coverage of all the image dynamics used for the calculation of the SNR (Signal to Noise Ratio), patterns to assist in the alignment of the test pattern on the bench and a slanted edge devoted to MTF assessment. Four patterns were engraved with four landscapes of different types (urban, forest, coastal). An example of test pattern is shown in Figure 6.

Figure 6.

Example of test pattern used for on ground demonstration.

00191_PSISDG11180_111804Q_page_6_1.jpg

2.5

Lens

The custom lens of the bench was manufactured by OptoSigma. It consists of two perfectly symmetrical double Gauss pairs, optimized around the wavelength λ=650nm to ensure image quality throughout the field and limit astigmatism. Focus distance is 206mm, WFNOis equal to 10 and magnification of 1.

2.6

Motion sensor

The motion sensor is here a commercial camera which contains a 2048 x 1088 5.5μm pixels matrix. It is possible to bin (agglomerate) pixels 2 by 2, or 4 by 4, making it possible to make acquisitions with pixels of 11μm and 22μm. The integration time can be set from 24μs to 2s. For our demonstration, the camera is used in dual binning mode (22 μm pixels). The operating point tested and presented in this paper corresponds to images with a size of 80 columns per 40 lines (22μm pixels), an integration time set at 1 ms for an acquisition frequency of 830 Hz. The acquired images are coded on 8 bits.

2.7

Motion computation algorithm

To compute shifts between two consecutive images delivered by the motion sensor, an algorithm based on optical flow with Lucas-Kanade solving was selected2. This method has been selected because of its low complexity and because between two consecutive images, motion amplitude is small in terms of pixels shift. Optical flow method tends to link difference between two consecutives images (in terms of temporal evolution) with the gradient of the spatial intensity of the first one. Performances of the motion computation algorithm have been first assessed on simulated images2, and then on real images taken by the motion sensor of the test bench.

2.8

Multi-frame TDI sensor

The goal for a future satellite is to obtain the equivalent of a hundred lines TDI. This could be reached for example by registering and adding 10 images of sub-TDI of 10 lines each, each sub-TDI having a classical behavior. For this demonstration, we didn’t have such a sensor, so we have used an existing multi-frame TDI sensor and adapted the test in order to be the most representative of the target as possible. The multi-frame TDI sensor used on the bench is a multi- TDI CCD; we have chosen to use 3 sub-TDI arrays of 32 lines each on the same chip. In this sensor, the sub-TDI arrays are not contiguous but spaced of an equivalent number of 154 TDI lines (see Figure 7). It is therefore not completely representative of the multi-frame TDI targeted in this application, but allows, with some adaptations, to make a representative demonstration of the motion correction.

Figure 7.

Adaptation of disturbances to the case of spaced multi-TDI sensor.

00191_PSISDG11180_111804Q_page_7_1.jpg

The consequences of the configuration of this multi-TDI on the motion correction bench are:

  • - A need to adapt the disturbances to be applied on the line of sight to be representative of the flight conditions (see figure 8),

  • - Longer processing time to correct a single line. Indeed, it is necessary to wait for the equivalent of 340 lines between TDI1 and TDI3, instead of 96 lines in the case where the sub-TDI were quasi-contiguous (left side figure 7)

  • - Longer rotation stability is required.

Figure 8.

Adaptation of flight disturbances for on-ground demonstration.

00191_PSISDG11180_111804Q_page_7_2.jpg

2.9

Adaptation of disturbances

Considering the difference between on ground and in flight TDI conditions (pixel pitch and space between the sub-TDI devices), the flight disturbance template was adapted for the on ground demonstration as in Figure 8.

Two disturbances from the template were tested: a first with a high amplitude of 2.33 pixels TDI 0-peak at 2.8 Hz and a second with a lower amplitude of 0.23 pixels TDI 0-peak at 28Hz. These different disturbances were applied either on a single axis, X or Y, or on both axes simultaneously.

2.10

Image resampling

After having computed the satellite motion, the motion computation values are used to register images from the principal sensor, meaning the TDI images. Each sub-TDI gives an image and each image of sub-TDI has to be registered and summed up (see Figure 9). The registration values are directly computed from the motion sensor computing values. The best compromise for this step is the bicubic interpolation filter2.

Figure 9.

Principle of sub-TDI images registration and summation to generate the final image with increased Signal to Noise Ratio and MTF.

00191_PSISDG11180_111804Q_page_8_1.jpg

For demonstration, the three images of the multi-TDI are registered and summed to give the final corrected image. Finally, the performances of the multi-frame registration are evaluated comparing image quality and MTF values on this resulting image with and without motion correction.

3.

RESULTS

3.1

SNR improvement due to multiple imaging

A criterion of good bench health is the verification of the SNR improvement (√3) obtained after summation of the 3 sub- TDI images. The shooting conditions on the bench are set to approach a SNR=29 to low luminance level (L1) on the sub-TDI devices to obtain a SNR=50 to L1 on the resulting image after summation. An example is shown in Figure 10, with an operating point set to SNR=25 on the sub-TDI. After summation, the average SNR level is SNR42, which corresponds to the theoretical gain of √3 (squared sum of the SNR of the three TDI). It can be seen that after motion correction, the SNR level increases by a further +20%, which is the consequence of the resampling carried out by processing, having a smoothing effect on the final images.

Figure 10.

Evolution of the SNR after summation and correction of sub-matrices TDI.

00191_PSISDG11180_111804Q_page_8_2.jpg

3.2

Accuracy of the motion computation algorithm

The accuracy of the motion computation algorithm is evaluated by comparing the applied disturbance (from the position sensors) to the shifts (or offset) calculated by the algorithm (Figure 11).

Figure 11.

Example of accumulated offsets on 1340 TDI lines during a disturbance along the X axis.

00191_PSISDG11180_111804Q_page_9_1.jpg

By subtracting the offsets of the algorithm from the disturbance curve, it is possible to extract the error made by the algorithm. The instantaneous error between two images is first observed in order to evaluate the accuracy of the algorithm at each step. The instantaneous error is always less than 3.10-3 TDI pixels with a maximum 2σ deviation of 5.10-2 TDI pixels (which corresponds to the 1/20 of a pixel).

The observation of the cumulative error makes possible the evaluation of the maximum error that will be made between the acquisition of the first TDI sub-matrix and the third TDI sub-matrix. The computed offsets will be cumulated during the equivalent of 340 TDI lines, and we obtained an error of up to 0.195 TDI pixels. If the three sub-matrices were contiguous, the error would be cumulated during the equivalent of 96 TDI lines, giving a cumulative error of 0.06 pixels TDI.

In the case of Figure 11, the shifts are cumulated during the acquisition of the 1000 image lines of the three TDIs (equivalent to 1340 TDI lines). The curves illustrate the increase in cumulative error over time.

3.3

Modulation Transfer Function computation and improvement

The MTF is measured on slanted edge patterns shown in Figure 12. The MTF curve is extracted from the black/white transition of the pattern. It is standardized at the cut-off frequency of the TDI detector (Fc) and corresponds to the MTF of the complete optical system (detector + optics). The value at a frequency corresponding to Fc/3 is extracted to observe the impact of motion correction on image quality. We compare the MTF values on the resulting image after summation of the 3 sub-TDI with and without motion correction, with and without disturbance, which allows us to quantify the impact of the computation applied on the images.

Figure 12.

On the left, slanted black/white pattern used for FTM measurement, on the right, the MTF curve calculated from the pattern from which the value at f (0.3) is extracted.

00191_PSISDG11180_111804Q_page_9_2.jpg

The results are shown for the two cases of disturbances presented in section 2.9: large and low amplitude disturbances. In Figure 13 we can see the curves of the applied disturbances registered by the position sensors.

Figure 13.

On the left, large amplitude disturbance along X axis (2.33 pixel 0-peak at 2.8Hz). On the right, low amplitude disturbance along X axis (0.23 pixel 0-peak at 28Hz).

00191_PSISDG11180_111804Q_page_10_1.jpg

Figure 14 and Figure 15 present the MTF values measured on the TDI images before and after motion correction, for the various disturbance cases applied on the bench. The MTF value is observed in both axes of the images, where the Y axis corresponds to the direction of image scrolling (according to the TDI lines), and the X axis corresponds to the direction perpendicular to the scrolling (according to the TDI columns).

Figure 14.

MTF values at Fc/3 measured on TDI images before and after correction along the X axis.

00191_PSISDG11180_111804Q_page_10_2.jpg

Figure 15.

MTF values at Fc/3 measured on TDI images before and after correction along the Y axis.

00191_PSISDG11180_111804Q_page_10_3.jpg

In the case of a large amplitude disturbance injection (2.33 pixel TDI 0-peak at 2.8 Hz) the corrected images have a 90% MTF gain when only one axis is disturbed, and 200% when both axes are disturbed.

On the other hand, in the case of injecting a low amplitude disturbance (0.23 pixel TDI 0-peak at 28 Hz), the MTF value of the uncorrected images does not show any significant deterioration of their level. It’s like making an acquisition without disturbance. However, the image correction processing then degrades the MTF value of the final image (decrease of about 6% of the FTM). This degradation of the FTM level is caused by the resampling of the image when it is corrected at subpixel level.

3.4

Visual improvement of images

Figure 16 illustrates the visual improvement of images after motion correction. These images came from a test in which a high amplitude disturbance has been applied along the X axis. Two different areas of the test pattern are illustrated: a first one containing landscape and slanted edge pattern for the measurement of MTF and a second urban landscape area corresponding to a car park.

Figure 16.

Example of final TDI image obtained by summation of the three sub-TDI images before and after motion correction.

00191_PSISDG11180_111804Q_page_11_1.jpg

4.

CONCLUSIONS

We have developed an operational test bench which allows the demonstration of CNES solution for digital stabilization. One major difficulty was to master the applied disturbance in order to be able to compare it with the motion computation. For that, a well-mastered optical line disturbance system and a very stable rotating plate were developed. Another difficulty was the optics parts of the bench and in particular the lens, which has to be good in the field and with good MTF.

Finally, we have demonstrated the improvement due to onboard image correction with CNES methodology.

All the processing developed with this test bench allows to evaluate precisely the performances of motion correction for high-resolution Earth observation Time of Delay Integration (TDI) imaging. We are able in particular to quantify the loss or gain of MTF on computed images which is important to evaluate the final performances.

This test bench is a good complement to the simulations performed simulaneoulsy2 in testing algorithms on real images representatives in terms of noise.

REFERENCES

[1] 

A. Materne, O. Puig, P. Kubik, CNES Patent, FR2976754 (2012). Google Scholar

[2] 

C. Thiebaut, S. Petit-Poupart, J.M. Delvit, C. Latry, E. Bousquet, G. Laurent, “Real-time implementation of digital stabilization for high-resolution Earth observation imaging”,” in Proc. SPIE 10430, High-Performance Computing in Geoscience and Remote Sensing VII, 1043004 (5 October 2017), https://doi.org/10.1117/12.2278377 Google Scholar
© (2019) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
S. Petit Poupart, A. Materne, C. Thiebaut, J. M. Delvit, C. Latry, and B. Mithieux "On ground demonstrator of digital stabilization for high-resolution Earth observation time of delay integration imaging", Proc. SPIE 11180, International Conference on Space Optics — ICSO 2018, 111804Q (12 July 2019); https://doi.org/10.1117/12.2536089
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Sensors

Modulation transfer functions

Signal to noise ratio

Cameras

CCD image sensors

Image quality

Image registration

RELATED CONTENT

Onboard TDI stage estimation and calibration using SNR analysis
Proceedings of SPIE (September 29 2017)
From SPOT 5 to Pleiades HR evolution of the...
Proceedings of SPIE (November 21 2017)
Study CCD image motion for remote sensing detection
Proceedings of SPIE (January 29 2007)
SPOT4 HRVIR first in-flight image quality results
Proceedings of SPIE (December 21 1998)
Topsat: the demands placed by image quality
Proceedings of SPIE (September 24 2002)

Back to Top