2.1

FILTER DESIGN

We designed and manufactured prototype filters based on a 4x4 macro-pixel pattern. This macro-pixel is composed of narrow-band filters and panchromatic (PAN) filters. Each global filter is composed of 2048 x 2048 pixels, i.e. 512 x 512 macro-pixels. Each macro-pixel is made with 8 panchromatic pixels and 8 narrow band pixels, as shown on Figure 1.

Figure 1:

schematic of the 4x4 macro pixel

The panchromatic function acts as a density characterized by approximately 15% transmission. Narrows band are 40 nm wide, and are centered from 580 nm to 1000 nm (see Figure 2). The macro pixel is a change compared to standard SILIOS products based on a 3x3 macro-pixels, including 1 PAN pixel per macro-pixel. Our 4x4 macro-pixel enables a higher spatial resolution in PAN bands. The drawback is an increased aliasing issue on the various spectral band, as well as a more complex demosaicing procedure: for a given spectral band, we have 1 pixel over 16 in our 4x4 macro, compared to 1 over 9 in the standard 3x3 macro pixel.

Figure 2:

simulated spectral transmission of each filter and of panchromatic pixel

On top of this sample, we also designed a test sample of the same dimension, but with large areas of the same spectral band. This sample gives opportunity to characterize filters more precisely, without spectral perturbations from surrounding pixels. In other words, perturbations induced by surrounding pixels will not add a different spectral contribution.

Targeted applications for such pixelated filters includes mineralogy. This application look for spectral information toward the Near Infrared (NIR) band. That is the reason why we select spectral bands ranging from 580 nm to 1000 nm (i.e.: detector limitation in terms of detector spectral response). SILIOS characterized manufactured filters before their integration onto the CMOS detector. This step validates simulation and confirms the expected spectral responses. An example is given on Figure 2. Test sample was sent to CNES to be further characterized.

2.2

Filters characterization

Filter are manufactured using SILIOS ColorShade © technology. This technology uses microelectronic means. The filters’ pixel pitch is chosen equal to the CMV4000’s: 5.5 μm. It is not possible to characterize the pixelated filter, in our laboratories: our filter characterization test bench does not have this spatial resolution. Therefore, higher dimension test patterns has been manufactured to enable filters characterizations. This test sample is made with the same technology, except that pixelated filters are displayed in a way to form large bands of a given filter (7 spectral bands + 1 panchromatic): each area is equivalent to 120 x 1400 pixels (0.66 x 7.7 mm). Disposition is shown on Figure 3. Figure 4 presents measured transmission spectra within CNES laboratory for normal incidence and collimated beam. The test bench used for this measurement was custom-built: it allows both high spatial and spectral resolution measurements (15 μm resolution and 1 nm respectively). We observe nice transmission spectra, Lorentzian shaped, as expected by the technology used to manufacture filters. Central wavelength range from 580 nm to 940 nm as expected, with FWHM around 30 nm.

Figure 3:

schematic of test sample

Figure 4:

transmission spectra on the 7 spectral bands plus one panchromatic band. Measurements performed within CNES laboratory.

Figure 5 presents several images coming from the test bench for various wavelength generated from the broadband light source coupled to a monochromator. Images correspond to Figure 3 disposition. We can observe how spectral bands intensities increase when the incident wavelength is changing. On the first one, at 500 nm, the seventh band is the most intense one, due to the second order peak (there is no high band pass filter in this setup). With increasing wavelength, we observe changes in intensity in the various bands. To further analyze; we can calculate the uniformity of transmission as well as the spectral response dispersion.

Figure 5:

Example of spectral characterization of test sample. We can see how the setup enables highly accurate spatial and spectral characterization.: On both left and right sides, vertical red lines on spectral graph gives the selected wavelength with regards to transmission spectra. Centered pictures give a spatial outpouts of our test bench at the selected wavelength closeby. Colorscale from blue to red indicates transmission intensity as a function of spatial localization.

We observe either positive or negative peaks of transmission at filters interfaces (see Figure 6). The sign of such transient depends if we switch to a higher or a lower spectral band. This effect is not clearly understood,

Figure 6:

transmission uniformity (orange) and spectral dispersion (blue) across the test sample

For this kind of filter, we know that angular illumination may change the properties of filters. Therefore, we characterized spectral response dispersion and transmission uniformity, as previously, but with an incident light tilted by 11° with respect to normal incidence. We do not observe any significant trend due to the 11° tilt on filters properties. The centroïd wavelength does not change significantly, as shown on Figure 7, whereas only a small decrease in transmission is measured (see Figure 8).

Figure 7:

spectral response dispersion across sample, for normal collimated incident light (blue), and for collimated 11° tilted incident light (orange)

Figure 8:

transmission uniformity across sample, for normal collimated incident light (blue), and for collimated 11° tilted incident light (orange)

For small incidence angles (10/15°) this weak sensitivity to inclination show that for typical optics, with apertures higher than 4, these filters would have no degradations of performances throughout the field-of-view. For wide-angle cameras however, this remains to be characterized.

2.3

Filters assembly

The pixelated filter assembly on the CMV4000 was carried out by SILIOS. CMV4000 version procured has temporary window, easing filters integration. Concerning the CMOS detector itself, the selected version includes micro-lenses. The schematic cross section is shown on Figure 9. Critical point of this assembly is the alignment of pixelated filter onto the detector. As both have the same pixel pitch, filter pixels must be perfectly aligned in positions and angles with respect to the detector pixels.

Figure 9:

schematic cross section of our assembly: CMV-4000 detector, plus pixelated filters, plus high Band Pass Filter

The high band pass filter is fixed with kapton, giving the possibility to characterize the detector without it, but also to put it in place, especially to take images outdoor. Few assemblies were manufactured, and we characterize more deeply one of them on our internal CNES test bench for detector spectral characterization.

3.1

Transmission spectra

Our test set up is made of a broadband light source, followed by a monochromator. Monochromator output is injected within and integrating sphere. We then use a couple of objectives to image the integrating sphere’s output onto the device under test. Detector integration time is adjusted for each wavelength to stay at the detector mid dynamic. Dark noise is subtracted by post treatment, as well as spatial non-uniformity. With this setup, we can characterize the detector under monochromatic incident light. By default, incident light beam as a numerical aperture of 4.

We decide to analyze the spectral response of each of the 16 individual pixels of the 4x4 macro-pixels presented in Figure 1. As an individual instance, Figure 10 presents the mean spectral response of pixels B4 (orange plot), in superposition with B4 filter transmission spectra presented on Figure 4 and obtained on filter’s only. At detector level, we observe a broadband contribution that was not observed on filters, leading to a weaker spectral rejection. We also observe a broadening of the transmission peak, probably due to the difference in optical configuration, from collimated incident light to focused one.

Figure 10:

superposition of filter transmission spectra with spectral response of B4 pixels of the detector

Figure 11 presents transmission spectra of all 8 spectral bands. Relative maximum transmission between bands is no more similar, due to detector quantum efficiency. The broadband contribution is present for all spectral responses. It presents bumps spectrally located at the various filters central wavelengths. We estimate that this contribution is due the optical recovering between adjacent pixels, and therefore, each pixel spectral response is affected by surrounding pixels spectral responses.

Figure 11:

transmission spectra of filtered pixels B1 to B8. Each curve is the mean transmission measured on all the pixels from a given filter

We calculate the ratio of in-band spectrum with respect to full band spectra. In-band is defines as the integrated flux at half maximum of each spectra. Table 1 presents the results. The broadband contribution observed on pixelated pixels is always the main contribution to output signal. This relative contribution increases with central wavelength shifting towards near infrared.

Table 1:

in-band over full band transmission ration for all 8 spectral filters integrated on CMV4000 detector

The spectral contribution of neighboring pixels is clearly seen on panchromatic pixels spectra presented on Figure 12. Each individual PAN-pixel response presents the classic downward shape of QE in silicon, plus four spectral bumps corresponding to the central wavelengths of its four surrounding filtered pixels.

Figure 12:

transmission spectra of panchromatic pixels. Each curve is the mean transmission measured on all the pan pixels with the same position in the macro pixel

3.2

Numerical aperture influence

SILIOS put in place spectral characterization with various numerical apertures, ranging from 1.6 to collimated beam. As expected, we observe a decrease in the central wavelength with a decrease in numerical aperture, as well as a broadening of the spectral response. Decrease in central wavelength for various filters is shown on Figure 13. From collimated beam to numerical aperture of 1.6, we observe a decrease in central wavelength around 4 nm. The higher the initial central wavelength, the higher the decrease amplitude.

Figure 13:

central wavelength shifts with numerical aperture

3.3

Modulation Transfer Function

In CNES laboratory, we carried out a detector MTF characterization using broadband illumination. Used light source is a SOLIS-3C from Thorlabs. We used a Line Spread Function method to measure the MTF, using a 15 μm wide slit. The detector is placed on translation stage having an amplitude of 100 mm and a step resolution of 1 μm. MTF results for both a B2 pixels (630 nm central wavelength) and panchromatic pixel is shown on Figure 14. Dashed line represents MTF at Nyquist frequency confirming that filters do not alter the standard behavior from square CMOS pixel, which can be considered as a quasi-perfect square sampling (around 0.6 and 0.7 at Nyquist frequency)

Figure 14:

MTF measurements pixelatel pixel B2, using a monochromatic source at 635 nm.

4.1

Preprocessing, and radiometric corrections

Starting from the raw data, the first step of our post-processing aims at correcting radiometric effects. In our case, we apply the following corrections. For each image, the first step is to apply a dark and flat correction using calibration data. Defective pixels are, by definition, stable on all images. Analysis of various images gives the ability to identify them. They are then corrected using a bilinear interpolation (in both directions). An example of hot spot detection is given Figure 16. Though some residuals are visible (most will be further attenuated during denoising), most hot spots are accurately removed. These spots are likely fabrication features on the demonstration model available, and shall be minimized as much as possible for flight matrices.

Figure 16:

Example of defective pixels/hot spots correction. (Top) Raw image. (Bottom) Image corrected with detailed method.

4.2

Demosaicing and Denoising

SFA compress the spectral information into a single observation. Although beneficial for telemetry in space applications, the main complexity of the post-processing remains the demosaicing [5]. This step consists in interpolating the missing pixels of a given spectral band to recover N full resolution images, where N is the total number of available bands in the SFA (9 in this case). For space applications, multiple techniques can be considered. The most common ones either rely on single-image interpolation (see [6] for a review), or onto super-resolution approaches (either numerical [7] or based on convolutional neural networks [8]). For our studies, we retained Hamilton-Adams (HA) demosaicing [9] as a compromise between accuracy and computing speed. Panchromatic pixels, which have a finer spatial sampling, are interpolated first, and then used in combination with the aliased spectral subsets to recover the demosaiced image. Example of results are provided in Figure 17 (top panels) but spectral bands show significant Moiré and zipper artifacts.

Figure 17:

Example of demosaicing on a false color B4/B2/B1 image. Top) First approach based on HA interpolation. Bottom) Improvement using inter-band correlation and reduction of Moiré effects

The demosaicing was further improved by using inter-bands correlations. Under a constant-hue assumption [5], we can interpolate bilinearly B_i+1-B_i and B_i+2-B_i+1 planes, and obtain new estimations of each band. Through a weighted average of the bands obtained with HA and the bands interpolated under constant-hue assumption, we significantly reduce visual artifacts on the bands (see Figure 17, bottom panels).

As images from space applications are affected by acquisition conditions (noise) and compression, coupling demosaicing and denoising is an interesting option for these matrices. Non-local (NL) methods based on auto-similarity assumptions within the image were also tested, in addition to the two previous methods. As shown in Figure 18 (bottom panels), the NL Bayes [10] algorithm used here provides the best results. Further improvements in the demosaicing are under investigation, in particular using radiance based algorithms [11] when multiple images of the same scenes are available.

Figure 18:

Example of improvement of image quality. Top) Image without denoising, bottom) with denoising. Clear improvements on zipper artefacts are visible.

More details on the demosaicing techniques and its results are provided in [4].

4.3

Accounting for spectral rejection

Despite demosaicing and denoising, visual artefacts remain when observing images in false colors (i.e. by overlapping spectral subsets similarly to a RGB image). Likewise, when observing on seemingly constant scenes of the image, the pixelated motive of the matrix remains. This is related to the strong aliasing between the bands but also to possible shifts in high frequency transitions within the images, which create Moiré artefacts (see Figure 17, bottom panels). This indicates that the pixels have non-negligible spectral contamination, as also shown by the effective quantum efficiency measurements presented in §3.1. Examples of QE measurements performed at CNES are provided in Figure 19 (Panchromatic pixel #3) and Figure 20 (Filtered pixel B6). We notice that the QE of a given spectral band is well represented by a linear combination of the QE of individual pixels predicted by theoretical model (silicon transmission for the panchromatic band, Fabry-Perot resonator transmission for the deposited spectral bands). Interestingly, not only adjacent bands, but also distant bands contribute to the effective QE (see B₇ contribution in B₆). This effect is not properly understood to date, and is being characterized through dedicated measurements. A possible explanation may be related to the angle of incidence of the incident photon flux (aperture used of #f/4 i.e. an angle of approximately 10°) and the thickness of the FP, causing reflected photons to be back scattered up to several pixels in distance (see §3)

Figure 19:

Effective QE measured at CNES (blue ‘x’) for P3. The orange line is the best fit. Residuals (standard deviation) are also shown. Individual channel contributions are overlaid and used to estimate inter-band rejection.

Figure 20:

Effective QE measured at CNES (blue ‘x’) for B6. The orange line is the best fit. Residuals (standard deviation) are also shown. Individual channel contributions are overlaid and used to estimate inter-band rejection.

An attempt to correct this spectral rejection effect was performed. Three different approaches were tested (see Figure 21, more details in [4]):

Figure 21:

Two examples of spectral rejection correction in the panchromatic band. (From left to right) no crosstalk correction, blind crosstalk correction (method 3) and QE-based correction accounting for pixel distances (method 2).

Each method was tested on the image data set showing moderate improvements. Overall, the third (blind) approach provides the most satisfactory visual results, although several artefacts remain and a general smoothing of the image (which could however be recovered through subsequence deconvolution).

4.4

Visual results and practical aspects for space applications

The post-processing software detailed in the previous sections is used on our initial dataset. Representative results are provided in Figure 22, and show very promising results in interpreting data from our pixelated matrices. This pipeline is part of the CNES image-processing suite, which is currently being used for planetary rovers. The extension to SFA presented here is planned to be included, as future missions are planned to carry these types of matrices to various bodies of the solar system (e.g. the Rashid-2 rover developed by the United Arab Emirates Space Agency). When considering future space applications, several effects not included here remain however to be investigated. These include the effect of data compression on image quality or a finer understanding of the overall spectral rejection.

Figure 22:

Examples of post-processed image samples on a B3/B4/B5 false color RGB-like image. (Left) Interpolated raw image (Right) Post-processed image using our method.

5.1

Test plan

The quality assessment phase aims at verifying that the main environment constraints of space environment pose no major concern to the use of this technology. Applied test plan is presented on Figure 23. It includes a step stress on one single representative device. This step gives the green light towards a more comprehensive test plan on 8 devices.

Figure 23:

Quality assessment test plan scheme

5.2

Tests conditions

Thermal step stress phase concerns one representative device in term of assembly (material and processes). Only the filter design differs (but same substrate). The objective was to validate that there is no issue for the assembly at high and low temperatures, using a step stress approach. At each thermal step, we performed a visual inspection and a check of parametric criteria on detector. Steps applied are the following:

This sequence was passed successfully allowing to go through the remaining test plan detailed in §5.1.

Screening phase consists in a characterization step, followed by a thermal stress, and finally a visual inspection plus electro-optical characterization (see §.5.3). Step stress consists in 10 thermal cycles -55/+125°C applied on all 8 devices. This step is a standard screening test for testing the assembly on detectors [12].

Following screening, 4 devices went through a high temperature storage test. This sequence includes 4 consecutives steps of 500 h at 125°C. Then one device went to destructive physical analysis, one to purple plague test, and the two last one for additional 500 h at 150°C. At all steps, we carried out visual inspection and electro-optical tests.

In parallel to high temperature storage, 4 devices went through 500 thermal cycles from -55°C to +125°C. On top of final characterization, we carried out intermediate visual inspection and electro-optical measurements at 50 and 100 cycles. Two devices coming from the « thermal cycling » sequence were additionally submitted to 500 h at 70°C and 70% relative humidity (moisture test defined in [12]). Intermediate step for visual inspection and electro optical characterization was put in place at 240 h.

5.3

EO characterization and visual inspection results

Electro-Optical characterization include readout noise, dark noire, Dark Signal Non Uniformity (DSNU) and Pixel Response Non Uniformity (PRNU) at mid dynamic. We measured Linearity, CVF and saturation at initial stages only. All these electro-optic were carried out in a CNES internal laboratory. From a general point of view, detectors show stable performances all along the assessment sequence. We observe some changes in DSNU, which we linked to a change in offset, without impact on absolute performances.

Only one important issue was noticed. It concerns assembly of filter on the detector surface for one over 8 devices. The issue appears on one device after 1000 h of HTS at 125°C. We found out the filter unattached, exhibiting a failure in the assembly for this particular device. Figure 24 and Figure 25 present pictures coming from visual inspection on this device, at initial and final step respectively. We can see that, after high temperature storage, the filter is shifted and tilted with respect to the detection area. The integrated filter is the test sample described in §0.

Figure 24:

initial visual inspection on the failed device

Figure 25:

final visual inspection on the failed

We proceed to Destructive Physical Analysis in our internal CNES Expertise Laboratory. It does not reveal any major issue either at the beginning or at the end of the assessment. Purple Plague test based on 300°C / 1 h test was also successful: visual inspection of bondpads submitted to bondpull test before and after 300°C / 1 h does not show any difference or degradation.