Automated driving is the perfect use case for artificial
intelligence (AI), because algorithms leveraging AI can handle situations that
the vehicle has not been explicitly trained on. In other words, AI, unlike
classical algorithms, is capable of leveraging knowledge gained from one set of
situations to tackle the potentially limitless situations that an autonomous
car may encounter. This is an ability known as generalization and is something
not often found in classical (non-AI) algorithms.
There are many ways AI is being applied within the ADAS and
automated driving space. This Tech Brief examines the use cases of AI in
automotive applications and discusses how VSI has applied AI in its own AV
What is AI and How Does It Work?
Often in an automotive context, the terms AI and Deep
Learning are used interchangeably. Neural Networks, which are the mathematical
construct upon which Deep Learning is based, are but a single form of AI that
has become hugely influential in recent years. Deep Learning is the practice of
creating algorithms not based on explicit instructions such as in regular (or
classical) algorithms, but by an automatic process leveraging known inputs and
outputs in order to train a neural network to accomplish some task. In other
words, Deep Learning attempts to figure out a non-linear (or complicated)
relationship between an input and its associated output. Rather than
programming an algorithm with a specific set of instructions to accomplish a
task, the neural network is exposed to or “trained on” large amounts of data
and allowed to develop its own intuition on what relationships there may be
within this data. The product of the Deep Learning process is a neural network that
has learned to associate some unprocessed information that it’s taking in (such
as an image, video or textual data), and outputting some valuable information
derived from that input.
Within the automotive industry AI can take many forms, but
it’s most widely known and hyped form is that of neural networks. This process
of training neural networks is known as Deep Learning. VSI believes
applications leveraging AI can perform better and oftentimes accomplish feats
that shallow AI (non-neural-network based AI) and classic algorithms simply
cannot do. AI alone however is often not sufficient for automotive
applications, and its fusion with classical algorithms is most often both
necessary and desirable.
Before an AI module is useful to an autonomous system, it
needs to be trained. In other words, it must learn the relationships (i.e.,
patterns) that emerge when trying to map (or transform) the input data to an
output value. To train an AI, the developer must expose the neural network to
large quantities of labeled data, so that it may learn the relationships
between input data and its associated label, and to properly accomplish that is
necessary to have a relatively large amount of labeled data.
Common Automotive Applications of AI Include:
Perception algorithms, which attempt to parse the environment and understand its semantics. This involves applications like object identification, lane-line and free space detection, traffic sign recognition and other applications in that same vein.
Neural networks can also be applied to capture desirable behavioral patterns. This is a less common application of Automotive AI, at least for the moment. It is generally in the context of path planning, trajectory optimization and smooth motion control. An AI is trained to associate particular environmental conditions to a particular control output. An example of this might be to teach the vehicle how to take interchanges by training it on examples of human drivers taking interchanges.
Training datasets, which are an essential component to
training automotive AI, can either be sourced internally or from a third party.
The form of dataset will depend entirely on what kind of neural network you are
building and its intended application. Keep in mind that training data also
requires labels, which are also known as ground truth labels.
Forms of Automotive AI
AI in autonomous driving has a variety of use cases and
applications it’s incorporated into. For each application leveraging AI
mentioned in the next section, it should be noted that AI is not always the
only method of accomplishing these tasks. An example of this might be object
detection; there is a long history of camera-based object detection leveraging
classical computer vision. However, in recent years neural networks have shown
capabilities missing in classical computer vision approaches, such as a vastly
improved generalization capability to new situations. Car makers or autonomous
developers may pick and choose which areas they want to use AI for, depending
on their particular set of requirements.
For perception applications, you
often require a feed forward neural network such as a Convolution
Neural Network (or CNN). This is a type of neural network that
commonly takes in image inputs and makes a prediction/inference using a single
camera frame, which represents a visual snapshot in time. They do not consider
data from previous frames/time-steps, and this is usually not necessary. For
example, a vehicle detection algorithm doesn’t care what was happening in the
scene two seconds ago, its job is just to detect all the current vehicles in
view. The input data for this model are images of the scenes around a vehicle.
Sometimes the input data will also include information extracted from other
sensors, but that is less common.
Use cases of AI for perception
applications leveraging CNNs include object detection and classification,
traffic sign recognition, lane detection and free-space detection (drivable
Some applications in autonomous
driving are time sensitive; such applications are those in which it’s important
to understand information from the recent past in order to accurately infer
what is currently happening. In other words, some AI applications need to
consider things in a historical context to understand fully what is happening.
A good example of this would be visual odometry, if a neural network is trying
to figure out how fast a vehicle is going, where it was seconds ago can be an
invaluable source of information. To handle this time sensitivity, these AI
applications use a special neural network architecture called a Recurrent
A Recurrent Neural Network (RNN)
is used to process data that is changing over time. This means that the input
data is being periodically changed or updated. An RNN receives a sequence of
values as inputs, and it outputs a value representing its inference using both
information from the current timestep as well as historical information. RNNs
are commonly used in speech recognition applications, but they are also used in
some automotive applications like trajectory prediction, visual odometry and
There are many ways AI is being applied in ADAS and
automated driving. Deep learning offers an intelligent solution for many
autonomous driving tasks, and neural network-based algorithms can easily adapt
to new situations that have not been encountered before. Deep learning-based
solutions are always learning from new data, and therefore will continue to
improve and become more capable.
AI has become hugely influential in making AV systems safer
and smarter. Thus, the outlook for AI is favorable enough to many OEMs and Tier
1 suppliers to make major deals with AI-based technology companies.