Big Meaning in the Smallest Movements

Graduate students Chris Kwan and John Magee, with their advisor, Margrit Betke, discuss and demonstrate software that adapts for users with severe motion impairments. Photos by Cydney Scott, Kalman Zabarsky and Team Hoyt View closed captions on YouTube

Born with spastic quadriplegia and cerebral palsy, 49-year-old Rick Hoyt has never been able to speak or use his hands to write. But that doesn’t mean he can’t communicate.

Hoyt (SED’93) (left), who is best known for competing in more than 1,000 races, pushed in a wheelchair by his father, Dick Hoyt, is testing assistive technology developed through a decadelong collaboration between Margrit Betke, a College of Arts & Sciences associate professor of computer science, and James Gips, a Boston College professor of computer science, with help from more than 50 students.

Camera Mouse, a tool for people with medical conditions such as cerebral palsy, spinal muscular atrophy, ALS, multiple sclerosis, and traumatic brain injury, employs a computer webcam to lock onto and track a chosen section of the user’s face—a nostril or the tip of an eyebrow, for example—and then links that person’s head movement to a cursor on the screen. Move right and the cursor goes right. If Hoyt pauses for more than one second, dwelling over a button or link, it clicks to active.

Betke and fellow researchers have adapted Camera Mouse to work with several popular programs, such as Microsoft Word. They’ve also created custom software that allows computer users with disabilities to type emails, edit photographs, create music, and fight space aliens, among other activities.

Now two graduate students (below) in Betke’s lab, Chris Kwan (CAS’11, GRS’11) and John Magee (GRS’04,’11), have crafted companion software that piggybacks on Camera Mouse’s underlying technology.While testing Camera Mouse with Hoyt, Magee noticed that Hoyt moved his head more easily in a diagonal motion—a trend Magee sees in other people with motion impairments—making it difficult for Hoyt to propel the mouse pointer in a straight line. So Magee built an adaptive mouse functions software that would translate Hoyt’s actual movement into his intended movement, sending the mouse pointer straight across the screen and helping Hoyt to more easily hit his target commands.

Magee hopes to finish developing adaptive mouse functions and release it free on CameraMouse.org.

Hoyt has also tested Camera Canvas, software custom-built by Kwan, which received the 2011 Science Day Graduate School of Arts & Sciences Dean’s Award and is already free online. The software allows Camera Mouse users to edit photos and create illustrations, performing tasks that require physically complex mouse moves with traditional software.

Kwan, who began developing the software with classmates Won-Beom Kim (CAS’08) and Igor Fedyuk (CAS’08), and further refined it through the University’s Undergraduate Research Opportunities Program (UROP), hopes Camera Canvas will give people with motor impairments an outlet to express their creativity.

Working with cocreator Isaac Paquette (CAS’11), Kwan also developed a tool within Camera Canvas that could have much broader applications. The two developers created a menu whose settings, like button size and screen placement, can be modified based on users’ abilities. Called Menu Controller, it eliminates the need for pull-down menus with small, crowded entries that can be difficult to navigate with Camera Mouse.

Menu Controller “is an important development,” Betke (right) says, “because it can be applied to any commercially available software to convert it into a piece of software that can then be used by people with severe motion disabilities.”

Since 2007, Camera Mouse has been downloaded more than 350,000 times worldwide, from as far away as Greece, South Africa, India, Northern Ireland, Pakistan, Australia, and Malaysia. Magee believes this community will continue to grow as more software is built for use with Camera Mouse.

“These projects have the possibility to be used by a small group of people,” he says. “But the change that it can make in their lives, if you do a good job, is huge.”

Robin Berghaus can be reached at berghaus@bu.edu.