Automated driving is the perfect use case for artificial intelligence (AI), because algorithms leveraging AI can handle situations that the vehicle has not been explicitly trained on. In other words, AI, unlike classical algorithms, is capable of leveraging knowledge gained from one set of situations to tackle the potentially limitless situations that an autonomous car may encounter. This is an ability known as generalization and is something not often found in classical (non-AI) algorithms.
There are many ways AI is being applied within the ADAS and automated driving space. This Tech Brief examines the use cases of AI in automotive applications and discusses how VSI has applied AI in its own AV stack.
What is AI and How Does It Work?
Often in an automotive context, the terms AI and Deep Learning are used interchangeably. Neural Networks, which are the mathematical construct upon which Deep Learning is based, are but a single form of AI that has become hugely influential in recent years. Deep Learning is the practice of creating algorithms not based on explicit instructions such as in regular (or classical) algorithms, but by an automatic process leveraging known inputs and outputs in order to train a neural network to accomplish some task. In other words, Deep Learning attempts to figure out a non-linear (or complicated) relationship between an input and its associated output. Rather than programming an algorithm with a specific set of instructions to accomplish a task, the neural network is exposed to or “trained on” large amounts of data and allowed to develop its own intuition on what relationships there may be within this data. The product of the Deep Learning process is a neural network that has learned to associate some unprocessed information that it’s taking in (such as an image, video or textual data), and outputting some valuable information derived from that input.
Within the automotive industry AI can take many forms, but it’s most widely known and hyped form is that of neural networks. This process of training neural networks is known as Deep Learning. VSI believes applications leveraging AI can perform better and oftentimes accomplish feats that shallow AI (non-neural-network based AI) and classic algorithms simply cannot do. AI alone however is often not sufficient for automotive applications, and its fusion with classical algorithms is most often both necessary and desirable.
AI Training
Before an AI module is useful to an autonomous system, it needs to be trained. In other words, it must learn the relationships (i.e., patterns) that emerge when trying to map (or transform) the input data to an output value. To train an AI, the developer must expose the neural network to large quantities of labeled data, so that it may learn the relationships between input data and its associated label, and to properly accomplish that is necessary to have a relatively large amount of labeled data.
Common Automotive Applications of AI Include:
- Perception algorithms, which attempt to parse the environment and understand its semantics. This involves applications like object identification, lane-line and free space detection, traffic sign recognition and other applications in that same vein.
- Neural networks can also be applied to capture desirable behavioral patterns. This is a less common application of Automotive AI, at least for the moment. It is generally in the context of path planning, trajectory optimization and smooth motion control. An AI is trained to associate particular environmental conditions to a particular control output. An example of this might be to teach the vehicle how to take interchanges by training it on examples of human drivers taking interchanges.
Training datasets, which are an essential component to training automotive AI, can either be sourced internally or from a third party. The form of dataset will depend entirely on what kind of neural network you are building and its intended application. Keep in mind that training data also requires labels, which are also known as ground truth labels.
Forms of Automotive AI
AI in autonomous driving has a variety of use cases and applications it’s incorporated into. For each application leveraging AI mentioned in the next section, it should be noted that AI is not always the only method of accomplishing these tasks. An example of this might be object detection; there is a long history of camera-based object detection leveraging classical computer vision. However, in recent years neural networks have shown capabilities missing in classical computer vision approaches, such as a vastly improved generalization capability to new situations. Car makers or autonomous developers may pick and choose which areas they want to use AI for, depending on their particular set of requirements.
Perception Applications
For perception applications, you often require a feed forward neural network such as a Convolution Neural Network (or CNN). This is a type of neural network that commonly takes in image inputs and makes a prediction/inference using a single camera frame, which represents a visual snapshot in time. They do not consider data from previous frames/time-steps, and this is usually not necessary. For example, a vehicle detection algorithm doesn’t care what was happening in the scene two seconds ago, its job is just to detect all the current vehicles in view. The input data for this model are images of the scenes around a vehicle. Sometimes the input data will also include information extracted from other sensors, but that is less common.
Use cases of AI for perception applications leveraging CNNs include object detection and classification, traffic sign recognition, lane detection and free-space detection (drivable area detection).
Predictive Control
Some applications in autonomous driving are time sensitive; such applications are those in which it’s important to understand information from the recent past in order to accurately infer what is currently happening. In other words, some AI applications need to consider things in a historical context to understand fully what is happening. A good example of this would be visual odometry, if a neural network is trying to figure out how fast a vehicle is going, where it was seconds ago can be an invaluable source of information. To handle this time sensitivity, these AI applications use a special neural network architecture called a Recurrent Neural Network.
A Recurrent Neural Network (RNN) is used to process data that is changing over time. This means that the input data is being periodically changed or updated. An RNN receives a sequence of values as inputs, and it outputs a value representing its inference using both information from the current timestep as well as historical information. RNNs are commonly used in speech recognition applications, but they are also used in some automotive applications like trajectory prediction, visual odometry and path planning.
Conclusion
There are many ways AI is being applied in ADAS and automated driving. Deep learning offers an intelligent solution for many autonomous driving tasks, and neural network-based algorithms can easily adapt to new situations that have not been encountered before. Deep learning-based solutions are always learning from new data, and therefore will continue to improve and become more capable.
AI has become hugely influential in making AV systems safer and smarter. Thus, the outlook for AI is favorable enough to many OEMs and Tier 1 suppliers to make major deals with AI-based technology companies.
