Scientists equipped “SnotBots” — drones using sophisticated AI programs — to learn about whales, oceans and even human health.
Like many marine species in the world’s oceans, whales face more threats today than ever before. Water pollution, noise, collisions with ships, fishing nets, hunting — plus stress from habitat and food loss due to climate change — all work against the ocean’s giants.
Studying whale health used to mean either necropsying the huge mammal or chasing down a live one and getting close enough to collect DNA, hormones, and other specimens from tissue samples or feces. It was tough and challenging work.
Enter Parley SnotBots.
“Parley Snotbot, a collaboration with Ocean Alliance and Intel, is a new and non-invasive research technology which allows us to explore our oceans in real time and open source data and knowledge,” said Cyrill Gutsch, Parley for the Oceans founder.
“Our vision is to create a global network of digital exploration tools which generate the big data we need to identify threats with new speed and precision, so we can act on them instantly.”
On a recent expedition to southeast Alaska, scientists used drones equipped with petri dishes and artificial intelligence (AI) to collect “snot” from whale blows. The snot exhaled from the whale’s lungs reveals critical data that could help preserve these great leviathans of the deep.
Along the way, researchers are also gaining insights into human health because people face many of the same sustainability threats as whales.
“We’re collecting biological data, DNA, pregnancy hormones, stress hormones and microbiomes,” said Dr. Iain Kerr, CEO of Ocean Alliance, an organization dedicated to collecting data — like toxicology, behavior, bioacoustics and genetics — on whales and other species to help monitor the health of the world’s oceans.
“It’s staggering the amount of information we’re collecting from this totally benign research tool,” said Kerr.
Aggregated data that’s collected from the project provides a priceless biological snapshot. Not only does the drone get a photo ID, it also gathers information on a whale’s location, size, social circle and familial relationships.
So far, the Parley SnotBot has been used to collect spout water from blue whales, right whales, gray whales, humpbacks, and now, on their most recent expedition in southeast Alaska, orcas, as well.
Thar She Blows 2.0
Researchers begin their work searching for whales using a boat that’s “like an aircraft carrier for drones,” according to Ted Willke, senior principal engineer and director of the Mind’s Eye Lab at Intel.
He led the team that wrote the AI programs used in data collection and analysis on the most recent Alaska expedition.
“We have very good captains,” Willke said. “They know these bays and these straits and they know where they’re likely to encounter particular types of whales. So we’ll head out on these journeys in the remote parts.”
Researchers spot the whales by their spouts, according to Willke.
“There is a nice white mist that’s created in a column and it’s very visible against the background of evergreen and sky and earth,” Willke said.
The team launches the drones off the “aircraft carrier” from a couple of miles away. They fly at just under 100 feet so the whale doesn’t know that it’s being observed. The drone is continuously shooting high-resolution video and when it gets close to the whale, it flies down and captures the whale spout — or exhaled breath condensate (EBC) — in a petri dish attached to the back of the drone.
“Think of how insane this is,” Willke said. “You are standing at the edge of the ship and this drone is coming in for a landing with all its blades running. You’re going to reach up and grab it out of the sky while you’re on the high sea and you’re both going up and down with the wake.”
Risks of severed fingers is real, he said.
Real-time Analysis Shapes the Study
The team analyzes the spout water or blow later, but reviews the video in real time.
“We wrote a program that snaps frames from the live video feed as it comes in and another one that could take segments of the video and basically buffer it for analysis instantaneously,” said Willke.
As the drone wirelessly sends the video at 60 frames per second and full resolution to the remote controller, it’s fed to MacBook Pro laptops. The image then runs through two analytic programs — one identifies whales by their tail flukes, while the other conducts volumetric analysis, assessing the whale’s fat, girth and length to see if it’s at a healthy weight.
This valuable data gives scientists information they can use immediately to direct their field studies.
“If it’s a whale of interest that’s been seen before, then it may be involved in a longitudinal story where they study the whale’s history over time,” said Willke. “We might want to emphasize tracking that particular whale and making sure we get a really good sample.”
Solving Scientific Mysteries
“What is interesting about whales is one, they’re a mammal at the top of the ocean’s food chain, and two, they’re pretty cosmopolitan, you find them all over the world,” said Kerr. “So they’re a good barometer of ocean health, or the proverbial canary in the coal mine.”
Whale health directly correlates to human health, according to Kerr, a fact that motivates people to pay attention. He said 30 years ago, the biggest threat to whales was commercial whaling. Today oceans are threatened by a flood of chemicals and other pollutants, habitat and food loss due to climate change, and a deafening rate of noise pollution.
“What happens to the whales will likely happen to humanity,” he said. “At least 50 percent of the threats that actually face whales are actually threats that face humans.”
Kerr said previous methods of examining whale health likely stressed the mammals, which skewed the data. “We had this problem — they call it the observer effect — where collecting the data actually changed the data.”
He said SnotBot is a game-changer.
“We’re living in this tsunami of data,” said Kerr, adding that the AI and machine learning algorithms Intel developed help synthesize the sea of data that researchers want but can find overwhelming.
He said SnotBot will help researchers around the world.
A lot of scientific research is conducted from big expensive vessels that many countries simply can’t afford, but drones are comparatively inexpensive.
“I think we’re at a tipping point for science where rather than having 10 of these big research vessels around the world, we’ve got 1,000 small drones bringing in all this data,” he said.
The teams will keep working to enhance SnotBot and investigate critical factors that will hopefully preserve the health of the whales.
“It’s not just all about Silicon Valley building cool Silicon Valley things. It’s how AI could actually help us save the planet and solve scientific mysteries,” Kerr said.
Editor’s note: Whale images taken using NMFS permit 18636. Thanks to Javier Turek and Ted Willke, who contributed photos to this story.