Simplifying machine learning for drone flight control

BY: GINA MANTICA

Drones can learn to fly on their own thanks to machine learning. But drones trained with current machine learning algorithms sometimes move erratically or use up a lot of energy, preventing them from traveling long distances. Kate Saenko, Co-Director of the Artificial Intelligence Research Initiative at the Hariri Institute for Computing, worked in collaboration with Renato Mancuso, Assistant Professor in Computer Science, and PhD students in the BU Computer Science department (Sid Mysore, Saenko lab, and Bassel El Mabsout, Mancuso lab) to optimize a machine learning algorithm that improves not only the maneuverability of drones, but also the amount of power that drones use during flight.

Their paper was recently accepted at the International Conference on Robotics and Automation (ICRA) 2021. The researchers optimized a machine learning algorithm to teach drones how to fly using the “Conditioning for Action Policy Smoothness” (CAPS) constraint. The team inserted CAPS into the part of the algorithm that controls how drones respond to changes in the environment. This direct algorithmic change demystifies the machine learning “black box” and allows researchers to better understand how to adjust their training parameters to ensure a smooth flight.

Saenko and colleagues used reinforcement learning, or a type of machine learning based on trial and error, to train their drones. Reinforcement learning is like learning to ride a bike, where a person tries different movements until they finally find a way to pedal in a straight line. Like people, machines can try different actions and adjust as needed until their action leads to a desired result. In the case of drones, a fundamental requirement is their ability to move smoothly over long distances, and this is the problem that CAPS seeks to address.

The researchers added CAPS to their algorithm to ensure that their machine-trained drones respond efficiently to the quick maneuvering of pilots. CAPS helps the machine choose movements that are similar to previous movements in both space and time. Choosing similar movements allows for the drone to move around smoothly, without any jerking or jittering. Sid Mysore compares CAPS to walking. “If I’ve taken a step with my right foot, then the thing I care about is if the step on my left foot is reasonable given where and when I placed down my right foot,” said Mysore.

First, the researchers trained the machine learning algorithm in a simulated environment to ensure the drone could maneuver safely through space without putting the drone’s hardware or pilot at risk. Simulated drones trained with CAPS had significantly smoother movements than those trained without CAPS.

Though promising, the researchers wanted to see how the drones would fly in real-world conditions. So, the team integrated their machine learning model into a drone’s hardware. This allows for the drone’s flight controllers to learn quickly and without human intervention. The real drone trained with CAPS moved just like the virtual drones- fluidly maneuvering through space. And, CAPS-trained drones required about 80% less power than those trained without CAPS.

By directly changing the machine learning algorithm rather than the training environment, or “black box”, the researchers better ensured the drone’s consistent maneuverability and efficiency. “It is important to understand how behaviors are controlled so we can ensure more predictable behavior,” said Mysore. He believes that sometimes the simplest methods that are able to address the root of the issue are the best for solving complex problems. “We don’t need to keep reinventing the wheel or shooting for more novelty…Sometimes research should be about methods that are effective and useful,” he said.

Interested in learning more about the transformational science happening at the Hariri Institute? Sign up for our newsletter here.