Frequency-modulated continuous-wave lidar has all-weather capabilities

STEPHEN CROUCH

Autonomous vehicles (AVs) rely on a variety of sensors, including cameras, radars, and lidars, to safely navigate. The successful deployment of AV technology will depend not only on a balance of cost and safety, but also scale. As AV rideshare services ramp, system cost will decrease and begin to support profit margins and growing revenue. Adverse environmental conditions such as rain or snow stand as limiters to AV rollout in a variety of important geographical areas. Access to these markets will be key to driving growth.

The choice of Phoenix, AZ as a rollout city for Waymo is a subtle admission that AV sensors aren’t ready for all-weather conditions. While radar performs consistently, the visibility and range of cameras and lidar can be greatly affected by precipitation. These sensors are critical to reliable navigation and localization of the vehicle. The change in camera performance is well understood through the widespread use of camera technology in advanced driver assistance and safety (ADAS) systems from vendors such as Mobileye. Lidar performance in bad weather is much less documented. Presented here is a closer look at a new type of automotive lidar sensor in the context of inclement weather.

Coherent lidar

Lidar technology can be divided into types on the basis of the detection principle. Many lidars, including pulsed time-of-flight sensors, use direct detection of back-reflected optical power to estimate distance. Coherent detection is an alternate approach to lidar that measures the back-reflected electrical field. This is achieved by optically interfering the received signal with another optical signal, often called a local oscillator. A coherent lidar is fundamentally an interferometer. Frequency modulation of the transmit and local oscillator signals support range measurements—very similar in operation to that of modern radar systems.¹

Coherent lidar, or frequency-modulated continuous-wave (FMCW) lidar, has been richly explored as a remote sensing tool, especially in the context of environmental sensing.² Products such as WindTracer from Lockheed Martin Coherent Technologies and similar products from Mitsubishi interrogate wind fields at tens of kilometers to make airports safer. Coherent detection has also supported a wide variety of long-distance imaging requirements. One inhibitor of coherent lidar has been the more complex signal processing, only recently available at commercial-friendly cost points and vehicle-friendly power budgets.

FIGURE 2. An overhead view of velocity-colored Blackmore lidar data indicates oncoming snow as volumetric noise within the sensor field of view (a); filtering of the snow using the velocity data improves the definition of a street sign and the snow-covered road surface (b).

FMCW lidar for AVs

For widespread commercial applications, such as autonomous driving, coherent lidar has only been explored in the last few years. As such sensors become more mainstream, a few key advantages stand out. Coherent detection offers unique combination of high dynamic range, high sensitivity, and total interference rejection—often competing priorities in direct-detect lidar design. High dynamic range allows the sensors to see dim objects such as tires and bright objects such as retroreflective street signs without active adjustment of detector gain. High sensitivity allows for long-range measurements without significant optical power—critical for realizing low-cost chip-scale systems that still meet range requirements. Interference rejection supports stable operation in the presence of other lidar and direct sunlight, which is often a tricky optical filter problem for direct-detect systems.

One more advantage for AVs is the sensitivity of coherent lidar to Doppler shifts. Any motion between the sensor and an object will induce a Doppler shift in the return signal. This can be interpreted as a velocity measurement concurrent with each range measurement (see Fig. 1). The extra velocity data differentiates the coherent lidar data product and enables a direct means for moving object segmentation, tracking, and identification previously reserved for radar systems.

Performance in adverse weather

The advantages noted above are critical to the performance of coherent lidar in precipitation. The high dynamic range allows the sensor to perform reliably, even when water droplets stand on the exterior of the optical window. The extra sensitivity translates to range margin. While range will certainly be affected by a heavy downpour, the reduction leaves a respectable range to operate at a reduced vehicle speed. Finally, returns from falling rain or snow often produce volumetric noise in point cloud data sets (see Fig. 2). With velocity data, the falling precipitation is easier to differentiate and filter. This advantage is shown in Figure 2b. In markets such as Seattle or Boston, where precipitation is very common, these advantages may make the difference between a research project and a viable business plan.

FIGURE 3. This steam would likely produce lidar returns—how would a vehicle know it is safe to drive over the steam?

A final environmental challenge often overlooked is a seasonal fact of life for most of the world: cold. Low temperatures convert humid vehicle exhaust or sewage vapor to visible steam. Steam is an especially tricky object for AVs to master (see Fig. 3). Computer-vision algorithms typically relying on hard edges and the persistence of shapes are frustrated by the amorphous and constantly evolving nature of steam. Further, the motion often happens at ground level. This allows for confusion between steam and small animals or children.

Lidar often does see returns from steam—one of the few objects that does not correspond to hard objects. If we assume that a fundamental goal of AVs is to avoid running into hard objects, this situation is clearly crucial to understand. The velocity data of coherent lidar offers additional information to AVs to recognize steam and operate with greater confidence (see Fig. 4).

FIGURE 4. A photograph of a smokestack above a sewer vent in downtown Detroit (a) is compared to a Blackmore lidar image of the same scene, again colorized by velocity (b); the motion of the steam is evident.

As AVs move into the mainstream, operators need every advantage in order to succeed. This will include adapting technology largely developed in warm, dry environments to broader environmental conditions. In turn, companies operating AV rideshare business models will gain access to additional markets. Coherent lidar is arriving just in time for AV teams in need of additional tools to provide improved perception in rain, snow, or just a cold morning.

REFERENCES

1. S. Crouch and Z. W. Barber, Opt. Express (2012); http://bit.ly/LidarRef1.

2. R. G. Frehlich and M. J. Kavaya, Appl. Opt. (1991); http://bit.ly/LidarRef2.

Stephen Crouch is the chief technical officer at Blackmore Sensors and Analytics, Bozeman, MT; e-mail: [email protected]; https://blackmoreinc.com.