Automatic driving is inseparable from the role of radar

In 2015, the industry was still debating whether autonomous vehicles should rely on laser radar or cameras. By 2016, things had changed significantly. After a major incident involving an autopilot system, the industry began to recognize the critical importance of laser radar. Let’s explore this topic with insights from a car electronics editor.

Lidar is an essential sensor for autonomous driving.

The diagram below illustrates the system framework of the perception layer in autonomous systems. On the left side, you can see various sensor inputs such as lidar, cameras, millimeter-wave radar, GPS, encoders, and IMU. These sensor data are fed into the perception algorithm, which processes and analyzes the information. The algorithm must distinguish between static objects and identify, classify, and track dynamic ones.

System framework of the perception layer in autonomous driving

High-precision maps are largely dependent on lidar and cameras. Once these maps are created, they are combined with GPS, IMU, and real-time environmental sensing for map matching and positioning. For path planning and motion control, the vehicle ultimately communicates via the CAN bus.

In terms of perception, lidar, cameras, and millimeter-wave radar are commonly used. Lidar helps identify, classify, and track objects, while cameras excel at classification and tracking. Millimeter-wave radar is primarily used for detecting obstacles.

Although these three sensors have overlapping capabilities, each has unique strengths that make them indispensable in different scenarios.

Millimeter-wave radar is ideal for obstacle detection, while cameras struggle with 3D object modeling but are strong in classification and tracking. Lidar, on the other hand, is widely used for positioning, obstacle detection, object classification, and tracking dynamic objects.

Automatic driving is inseparable from the role of radar

Before 2016, the development of lidar technology was relatively slow. However, today, both upstream suppliers and downstream users are pushing for cost reduction and mass production of lidar systems.

I currently categorize lidars into two types: scanning and non-scanning lidars.

1. Scanning Lidar

Mechanical rotating lidar (with transmitting and receiving lasers that rotate coaxially) is now well-established and widely used in autonomous driving. Companies like Velodyne and Ibeo are leading examples. Hybrid solid-state lidars also fall into this category.

MEMS-based scanning radars are under development. Their principle involves changing the optical path using MEMS mirrors.

Phased array lidar (OPA) is another type of scanning lidar, using multiple small antennas to scan point by point by adjusting the phase of the emitted laser.

2. Non-Scanning Lidar

Flash LiDAR emits a field of light, rather than scanning point by point. Companies like Ledder Tech in Switzerland are developing such systems.

Currently, there is significant momentum in the industry. Research and development efforts are focused on solid-state lidars, and we expect low-cost lidar solutions to emerge faster than anticipated. This is not just the effort of one company, but the entire industry working together.

A point cloud is a massive dataset that represents the spatial distribution and surface characteristics of objects in a given space.

Point cloud data can be generated by lidar or depth cameras. Lidar provides XYZ coordinates and laser reflection intensity, making it valuable for mapping and object recognition.

Application of Lidar in Autonomous Driving: Positioning

The most important use of lidar in autonomous driving is positioning. Knowing "where I am" is the first step for any self-driving vehicle to navigate effectively. Currently, RTK-GPS and visual positioning are used, but both have limitations.

RTK-GPS can be affected by signal interference, especially in urban areas with tall buildings or tunnels. Visual positioning is also environment-dependent, struggling in conditions like backlighting, rain, or snow.

Automatic driving is inseparable from the role of radar

Lidar-based positioning combines data from IMU, GPS, and encoders to predict a global position. Real-time lidar scans are matched with feature extraction from the surrounding environment.

These features include road edges, lane lines, and other elements. When compared with high-precision maps, the system achieves accurate vehicle positioning.

Second application of lidar: Obstacle detection and classification.

Lidar and vision systems are both used for obstacle detection, but they serve different purposes. Lidar doesn’t depend on lighting, offers a 360-degree view, and processes data quickly—typically within 100 milliseconds.

Lidar first identifies obstacles in space, then classifies them based on known objects. For example, cars and pedestrians are separated and tracked individually.

To simplify, the process involves removing ground points, identifying obstacles, segmenting them, and classifying each object for tracking.

Tracking begins by associating point clouds with targets. By comparing previous and current frames, the system determines if objects are the same and tracks their movement.

Current status of lidar applications

We started supplying lidar solutions in April of this year and have observed the current state of lidar usage in the industry.

First, there is a shortage of sensors. Lidar is expensive, delivery times are long, and few companies can afford widespread use. Additionally, the algorithms for processing point cloud data are still immature, limiting lidar's full potential.

For autonomous teams, besides lidar algorithms, they may also develop camera and millimeter-wave radar algorithms, along with multi-sensor fusion. This leads to scattered focus and slower progress.

Therefore, in April, we launched the Prometheus plan for lidar solutions. Our goal is to promote open sharing and accelerate the commercialization of autonomous driving.

Automatic driving is inseparable from the role of radar

The entire Prometheus module is built around what lidar can do: positioning, lane detection, roadside detection, obstacle recognition, classification, and tracking.

Whether it's a low-speed vehicle, a parking robot, or a highway car, we aim to contribute to the future of autonomous mobility.

The image above shows lidar-based lane and roadside detection. Lidar is very effective at identifying lane markings, especially when considering varying lighting conditions. We pre-process reflection intensity to help the system detect white and yellow lines more accurately.

Roadside detection uses geometric training to extract road features at heights over ten centimeters. In the image, red indicates lane lines, light colors show the road edge, and the middle is a dashed line, with solid lines on either side. This level of accuracy is promising, and with multiple frame iterations, the results will improve further.

Lidar tracking of objects involves identifying and classifying each detected object, whether it's a bicycle, truck, pedestrian, or car. After identification, the system tracks the speed and distance of surrounding vehicles, providing detailed spatial information about each obstacle.

Lidar plays a crucial role in perception, offering functions like positioning, obstacle detection, classification, tracking, and lane detection. It is a vital component in the development of autonomous vehicles.

This concludes our discussion on the role of radar in autonomous driving. If you want to learn more, stay tuned for more updates from E-engineering. We’ll provide you with complete, detailed, and up-to-date information.