Open Sourcing the World's First AV Dataset for Wintry Environments
Self-driving cars promise to improve mobility, build a more efficient
transportation system and even save lives.
But while the industry has been making great progress, to date the field has
struggled to build cars and artificial intelligence models that can handle all
weather conditions.
Snow is hard to drive in—as many drivers are well aware. But wintry conditions
are especially hard for self-driving cars because of the way snow affects the
critical hardware and AI algorithms that power them.
Take a car trained to drive on the wide, sunny boulevards of Arizona or
California and place it on a snowy street like this, and things will go wrong:
Snow covering the road markings would make it hard for the car to stay in its
lane.
Accumulated snow significantly reduces the color contrast, making it harder
for the car’s system to recognize objects, like trees or parked cars covered
in snow—or even pedestrians.
Most problematically, snowfall makes it hard for a vehicle’s cameras and LiDAR
sensors to see, which significantly reduces safety:
A skilled human driver can handle the same road in all weathers—but today’s AV
models can’t generalise their experience in the same way. To do so, they need
much more data.
Until now, there has not yet been an open dataset of annotated LiDAR data and
images of driving in snowy weather that researchers can use to develop more
robust systems.
CADC: A new dataset for wintry conditions
In partnership with the University of Waterloo and the University of Toronto,
we’ve been applying our technology to this challenge.
Today, we’re open sourcing the world’s first AV dataset in wintry
conditions—the Canadian Adverse Driving Conditions dataset (CADC, or cad-see.)
Drawn from 20 km of driving in harsh, snowy conditions in southwestern Ontario
over the past two winters, CADC contains 7,000 frames of LiDAR and camera data
spread over 70 driving sequences—featuring a wide range of perception
challenges for AV models that can handle wintry conditions. Scale provided the
annotations that makes the data usable by researchers.
Researchers at the Universities of Waterloo and Toronto have also released a
detailed academic paper outlining the technical features of the data collected
and object detection labels provided, which you can view on arXiv
here.
Ultimately, this is about safety. Without robust data, self-driving models
can’t learn how to handle snowy conditions—preventing us from being able to
make AV systems a reality for the many millions of people who live in places
that receive wintry weather.
As Professor Krzyzstof Czarnecki at the University of Waterloo says, “We want
to engage the research community to generate new ideas and enable innovation.
This is how you can solve really hard problems, the problems that are just too
big for anyone to solve on their own.”
We hope this dataset will make it easier for researchers to achieve the
next-generation breakthroughs in AV systems that will take us one step closer
to self-driving vehicles that work safely in a wide range of different
environments.
As Professor Steven Waslander at the University of Toronto says, “We’re hoping
that both industry and academia go nuts with it. We want the world to be
working on driving everywhere.
“Bad weather is a condition that is going to happen.”
You can find the dataset on the
Waterloo website, with dataset
support tools available on