Abstract

This paper discusses the problems inherent in the design of cameras for A/C landing applications and how these were solved in the Enhanced Vision System (EVS). The EVS camera was designed by OPGAL, subcontractor to Kollsman Inc. The EVS contains a custom camera, a Head Up Display (HUD), an external window mounted in the aircraft, and an electronic control box built by Kollsman Inc. The EVS passed the FAA proof of concept during August and September 2000, and is expected to pass the certification flight in the summer of 2001. The camera-related issues addressed in this paper are: Spectral band, Signal-to-Noise Ratio (SNR), signal receiving in fog conditions, and dynamic range.

A/C Landing Visibility - Background
The EVS camera is designed to provide day/night improved orientation during taxiing or flying. It allows visual landing in reduced visibility conditions, such as fog, haze, dust, smog etc. The system provides a fused, visual and near-IR picture, and displays a video image, superimposed on the pilot's Field-Of-View. The EVS video image displayed on the HUD coincides with the regular view observed by the pilot through the aircraft window. During regular landing approach, the aircraft's descent is at a 3° angle. At the Decision Height (DH), approximately 200 feet above the ground, the pilot has to recognize the runway's landing lights. If this is impossible at DH, the pilot has to execute a missed approach. The purpose of the EVS camera is to allow the pilot recognition of runway's lights at a distance of about 4,000 feet (a value obtained from the 3° descent and the 200 feet altitude). The atmospheric visibility conditions in landing applications is defined by a parameter named "Runway Visibility Range" (RVR). The RVR is the maximum distance from which runway lights are visible to the naked eye. In other words, RVR is defined as the distance at which light intensity drops to 2% of the value at zero distance. For any additional RVR distance, the runway lights signal decreases by a factor of 50. 

Technical Issues Involved in A/C Landing Visibility
Four main technical issues are involved in A/C landing visibility:
1. Spectral band
2. Signal-to-Noise Ratio (SNR)
3. Fog conditions
4. Dynamic range. 

Spectral Band
In most airports the runway lights are mounted on short poles above ground level. The bulbs are exposed to wind and rain, therefore their temperature is not very different from the background temperature. The bulb's exterior material is glass, which is opaque above 2.2 µ. This means that in bad atmospheric conditions, the only spectra available for bulb detection is in the visible or near-visible region. A very sophisticated simulation, taking into account camera performance, atmospheric transmittance and the runway bulb spectrum, yields the same conclusion.  
 

Figure 1 describes two types of bulbs used as runway lights (the two types are not the only ones in use). The question is, what should be the detection system spectral band ? In order to answer this question, it is necessary to address another issue, the issue of Signal-to-Noise Ratio (SNR).

Figure 1: Photovoltaic Detector Spectral Response to Runway Bulbs' Spectral Emission (Indium Antimonide, InSb) 

1. Run way bulb #1 Spectral response
2. Run way bulb #2 Spectral response
3. EVS detector Spectral response normalized to 1
4. EVS detector measured Spectral response 

SNR
Let assume that the difference in atmospheric transmittance between the visible and near-visible spectra is negligible. The nominal SNR required for detecting the runway lights from a distance of 1 RVR is 50:1. The nominal SNR required for detecting the runway lights from a distance of 2 RVRs is 2500:1. The nominal SNR required for detecting the runway lights from a distance of 3 RVRs is 125,000:1 

Figure 2 describes runway bulb energy fraction versus distance, assuming RVR equals 600 ft. At this point we would like to introduce the SNR definition. SNR is defined, at the output of the non-uniformity correction process, as the ratio of the average signal to the standard deviation for an ideal homogenous light source target. In order to calculate the detection of the runway light using the visible or near-visible spectra, the SNR value is crucial.
The upper limit performance in fog, assuming direct detection of the runway lights, using the visible or near-visible spectra, is given by the following simple relation: 

In order to avoid any misunderstanding, the SNR for detection of the first (greatest distance) runway lights in fog conditions, is equal to 1 or even less. Theoretically, there is always a probability greater than zero that even with a lower SNR value, runway lights would be detected. The RVR is an average value, and for very short instants the atmospheric transmittance might be higher than average. In order to design a real product, the design point for the EVS camera was selected at 2.2*RVR runway lights detection range. The SNR required for such a detection range should be greater than 5000:1.

The upper SNR limit is the square root of the amount of photoelectrons collected per frame per detector element. CCD technology collects between 50,000 to 250,000 photoelectrons per frame per detector element, sustaining an upper SNR limit of 500:1. PtSi detectors collect less than 1,000,000 photoelectrons per frame per detector element, achieving an upper SNR limit of 1000:1.

The EVS camera uses an Indium Antimonide (InSb) photo-voltaic focal plane array collecting more than 50,000,000 photoelectrons per frame per detector element. The extra amount of photoelectrons collected is required in order to overcome the other noise sources, except the quantum noise upper limit mentioned above.

At this point we are able to answer the previous question: what should the detection system spectral band be? The answer is that the system spectral band should range from 1.2 µ to 5.5 µ and is dictated by the InSb focal plane array performance.

Detection in Fog Conditions
The SNR and the spectral band are not the only two major requirements of a camera for aircraft landing. The SNR requirement is a necessary condition but not a sufficient one. We shall now address the issue of the signal collected by the camera in fog conditions. 

Figure 3: Three Different Sections Through FPA Signal in Fog Conditions

Figure 3 describes the signal collected by the FPA detector in fog conditions. The three different signals represent three arbitrary sections through the detector faceplate. As can be observed, the detector signal contains a very high contrast, low frequency signal, caused by the diffused and scattered light, and the relevant information rides on top of it. The modulation described in Figure 3 is exaggerated and it should be obvious that the real information is much smaller. The signal collected by the detector in fog conditions is a two-dimensional, high contrast, low frequency in time and in space, random wave, with an extremely low modulation on top of it. In order to detect the runway light in fog conditions, the high contrast signal caused by the diffused and scattered light has to be removed. Because a human being is capable of detecting a signal, provided its contrast is above 2%, the contrast of the runway lights have to be higher than 2% at the video output, in order to be detected.

Dynamic Range
Finally there is the issue of the camera dynamic range. The EVS camera is defined as a day/night system that should display a consistent replica of the external world while looking directly into the sun, or looking at a night sky at -60°C. The required dynamic range is therefore 10,000,000:1.  

 
Main Features of the EVS Camera

The EVS camera, is a sophisticated state-of-the-art imaging device which solves most of the problems inherent in aircraft landing applications. EVS contains the following main components:

• Focusing lens
  The lens is diffraction-limited. Its geometrical spot diameter on the detector faceplate is less than half detector pitch for the entire detector faceplate. The lens focal length is 17 mm and the lens transmittance 89%. New coating materials have been developed for this lens. The lens useful spectral band ranges from 1.2 µ to 5.5 µ.
• Detector Dewar Cooler Assembly
  The camera is based on an InSb photovoltaic staring array, cooled to 77 Kelvin. The snapshot detector contains 320 x 240 elements and its 8 readout video lines allow collection of all array information within 2.0 msec. The upper limit of the system f number (f#) is defined by the detector cold shield and is equal to 1.5. A detector cycle containing contains light collection and information readout takes about 4 msec to complete. The system usually runs at 240 frames per second, with a maximum of 300 frames per second.
• Exposure Time Control
  The highly sophisticated electronic block, implemented in hardware, controls the detector exposure time in a range of 100,000:1 in order to achieve a total dynamic range of 10,000,000:1. The detector integration time is updated 30 times per second. Integration time varies according to the amount of light collected. The integration time per frame defines the number of frames per second. For high radiation scenery, the camera runs at 300 frames per second, while for very low radiation backgrounds the frame rate will drop to 30 frames per second.
• Non-Uniformity Correction Block
  The non-uniformity correction block corrects each pixel and brings the whole picture to a uniformity dictated by the required SNR value. The non-uniformity correction process is implemented by a high-order polynomial expansion of a few variables implemented in 32-bits floating point.
• Atmospheric Transmittance Estimation
  A mathematical block uses the input raw video signal in order to estimate a relative value that describes the atmospheric transmittance. Atmospheric transmittance is usually measured by using a calibrated laser beam travelling through the atmosphere for a known distance. The EVS camera does not have a laser beam and distance to the scenery is unknown, therefore, estimating the atmospheric transmittance is extremely difficult. The EVS camera can estimate only a relatively vague value of atmospheric transmittance. The relative atmospheric transmittance is measured and estimated 30 times per second. A sophisticated time filter is used to limit the fluctuations of this value and to increase estimation accuracy.
• Special Demodulator
  The demodulation level is controlled by the atmospheric transmittance value estimated in the previous block. For high transmittance atmospheric conditions the demodulation level is very low and for low atmospheric transmittance, the demodulation level is very high. The special demodulator operates all the time, even in perfect atmospheric conditions. Running the demodulator at maximum capacity on high contrast, regular pictures, like those obtained in the laboratory for short distance scenery, will enhance the contrasts of small objects but will not deteriorate human capability to correctly interpret the images (see Figure 4). The special demodulator response is implemented in 32-bits floating point.
• Sophisticated Amplification Function
  Enables the display of very low contrast (0.01%) information.
• Compression Block
  Translates any signal collected into a limited number of gray levels at the FLIR output. 

Figure 4: Picture on the Left is the Original. Picture on the Right Was Obtained by Applying Maximum Demodulation

EVS Camera Picture Samples

We are presenting samples of pictures taken with the EVS camera. These samples have been selected randomly from thousands of pictures taken during numerous flights, at different atmospheric conditions, at night and during the day. The pictures are presented in pairs. In each pair, the picture on the left hand side represents the EVS picture displayed on the HUD and includes aircraft symbology. The picture on the right is the scenery as viewed by the copilot and has been taken by a special TV camera designed to have a resolution and sensitivity similar to the human eye. The first set of pictures shows a regular landing approach in good atmospheric conditions. In Figure 4_1, the runway can be observed in the middle of the left hand side picture from a distance in excess of 7 miles. Figure 5_x shows a landing approach performed in very bad atmospheric conditions. Those pictures belong to an approach performed in daytime. During the day, sunlight is diffused and scattered by the fog. This phenomena diminishes the camera's capability to detect the runway lights. The pictures in Figure 5_1 have been collected exactly at the DH. Therefore the estimated RVR value is about 1900 feet. In Figure 5_3, the left hand picture shows that only 4 runway lights can be detected. The distance between two runway lights is 200 feet. Taking into account EVS camera performance, the RVR is estimated at 400 feet. In this example atmospheric transmittance decreases with altitude.

Figure 5_1

Figure 4_2

Figure 4_3

Figure 4_4

Figure 6_1

Figure 5_2

Figure 5_3

References

1. General wavelength dependence of imaging through the atmosphere, by N. S. Kopeika , APPLIED OPTICS / Vol. 20,No 9/ 1 MAY 1981/ page 1532
2. Meteorological influence on aerosol extinction in the 0.2-40 -mm wavelength range, by Bertil Nilsson, APPLIED OPTICS / Vol. 18, No 20/ 15 October 1979
3. Radiometric Calibration for InSb FPA Camera, by M. Zahler , Proc. of SPIE Vol. 3110, Sep 1997
4. Test Results of a "Factory" Calibration Technique for Non-Uniformity Correction of InSb Infrared System, by Gill L. Duykers, SPIE Vol. 3063, Infrared Imaging Systems, June 1997
5. High well capacity InSb infrared arrays for the 1 to 5 mm wavelength range, by G Finger, G. Nicolini, M. Meyer and A. F. M. Moorwood, Proc. of SPIE Vol. 2475 Infrared Detectors and Instrumentation for Astronomy, June 1995
6. Image Processing for Infrared, by Dean A Scribner, Jonathon M. Shuler, Penny Warren, J. Grant Howard, Melvin R. Kruer, SPIE Vol. 4028 April 2000.
7. Digital Filters and their Applications, by V. Cappellini, A. G. Constantinides and P. Emiliani, ACADEMIC PRESS ,1978
8. Image Modeling, by ACADEMIC PRESS, 1981.