Arts & Sciences Magazine

By Mandile Mpofu (COM`24)

Every year, the Chinese Language & Literature program enrolls between 600 and 700 students, all interested in integrating the language into their studies to varying degrees. With such large numbers, it is challenging for the department’s faculty to accurately place students in courses that align with their language level.

Recently, the task has proven even more challenging as the pool of students has diversified and students are entering the program with varied and complex language backgrounds.

For a decade, the department had been using a commercially-licensed proficiency test, but that became less accurate as more heritage speakers — that is, those who learn a language through informal exposure — enrolled in the program.

“It’s a broad spectrum. Some students might understand the language but cannot speak it; some can listen and speak on daily topics but cannot read or write it,” said Shutan Dong, a senior lecturer in Chinese and the Chinese language program coordinator. “There are others who can speak and listen just as well as a native speaker but don’t know any Chinese characters.”

She said the external proficiency test couldn’t distinguish between the varying linguistic competencies as well as the department would have liked. So, Dong proposed a solution to this problem: creating a new, in-house placement test that considers students’ assorted backgrounds and is tailored to Boston University’s Chinese language program curriculum.

Dong was awarded the 2022-2023 Geddes Language Center Technology Mini-Grant, which encourages faculty “to explore and report on new ways to integrate technology and language learning.”

For nine months, she collaborated with other faculty members and used the grant to develop and design a new test from scratch within Blackboard’s existing infrastructure. Her goal was to streamline and enhance the language placement process.

“We hoped to create a better alignment between the placement test and our course offerings,” Dong said. “We wanted to improve the student experience in the program and better meet their needs.”

Mark Lewis, the director of the Geddes Language Center and a lecturer of German, said he was “excited about the proposal because it showed the interest in using technology to save costs.”

Because the test was created in Blackboard, a software that BU already licenses, there were no additional charges incurred in the test’s development. By ending its commercial license with the external placement test provider, the Chinese language department saved money and now possesses a better, more accurate test.

Dong highlighted the difference between a proficiency test, which they used previously, and a placement test, explaining that a proficiency test is based on the guidelines outlined by the American Council for the Teaching of Foreign Languages (ACTFL) and does not regard the materials the students have previously studied.

Proficiency tests measure students’ language competency on various topics, rather than accumulated knowledge. While these tests are helpful in determining the student’s level based on ACTFL guidelines, Dong said the tests become increasingly inadequate for placement purposes, particularly when because of the growing number of heritage speakers

What Dong created is a placement test. It is administered entirely via Blackboard and starts with a survey that asks students about their linguistic backgrounds and prior experiences learning Mandarin and then directs them to one of three different tests based on their answers. From there, they answer questions that will determine their level.

Dong and her colleagues — Liling Huang, a senior lecturer in Chinese, Amber Navarre, a master lecturer in Chinese, and Hongyun Sun, a senior lecturer in Chinese — contributed to a question bank, each submitting questions based on their curricula and on the levels at which they teach more often. She then built on the question bank and designed the listening, speaking, and writing questions.

The test takes about 1-2 hours to complete, and students can see their placement level as soon as they complete it, a stark departure from the previous process which sometimes took up to two weeks. This allows students to know their level well before the add/drop period and to work with Dong and other instructors to decide which courses are best for them.

Dong’s prior experience in designing placement materials was useful during this project. Before coming to BU, she worked at Princeton University, where she helped redesign some of the school’s placement test items. She also led summer programs for students who wanted to study abroad in China. Thanks to that work, she is familiar with the factors that go into placing a student. She said it comes down to the curriculum, the materials, the student’s current language level and prior experiences, and the student’s motivation.

Currently, the new test places about 95 percent of students correctly, according to a pilot test Dong conducted earlier in the year. The other 5 percent are heritage speakers whose level is much harder to automatically detect and will need further faculty review. Students who enroll in Chinese language courses in the fall with be the first to officially use the placement test in its completed state.

“With the vast varieties of Chinese dialects and their varying linguistic distance to Mandarin, it’s almost impossible to have a placement test that will be 100 percent accurate,” Dong said.

She still hopes to continue to improve on the project by increasing the effectiveness of the test questions. Since the test is based on the program’s curriculum, if it changes, the test will need to change, too.

Dong said she could not have completed the project without the help of her colleagues who played a critical role in the quick turnaround time.

In addition to Huang, Navarre, and Sun, she named Senior Lecturer in Chinese Weijia Huang; Associate Professor of Chinese and Comparative Literature and Women’s, Gender and Sexuality Studies Petrus Liu; Assistant Professor of East Asian Literatures Dennis Wuerthner, and Professor of Chinese and Comparative Literature Catherine Yeh. Shawn Provencal, the assistant director of the Geddes Language Center, lent his expertise in creating Blackboard-based placement tests.

As the director of the Geddes Language Center, Lewis said he would like to see more projects that follow in the placement test’s footsteps.

“I hope that more people come up to us with a vision for a project that is collaborative in nature like this one,” Lewis said. “It embodies the spirit of teamwork we strive for.”