Boost for AI-powered sign language app that helps deaf children read

A Huawei project to help deaf children learn to read using an AI-powered sign language animation is to be expanded with a funding boost and support for additional books.

The smartphone maker is committing £500,000 to its StorySign app partnership with the European Union of the Deaf organisation, and plans to fund four new books in the summer.

Huawei launched the app on Android in December with one book, Where’s Spot?, using an avatar called Star. The avatar translates children’s books into sign language in real time when hovered over the book, with the aim of helping some of the world’s 32 million deaf children who struggle to read.

Deaf children often struggle to learn to read due to not being able to hear their parents read to them or hear a teacher repeating sentences, while there is no written form of sign language as there is no direct word-for-word translation.

“The overwhelming response to StorySign has ensured we can put our long-term vision into motion and we hope we can inspire others to get involved,” said Walter Ji, president of Huawei’s Western Europe consumer operations.

“We want to continue to explore the potential StorySign has to become a wider educational tool, revolutionising the way we teach sign language using technology and breaking down everyday barriers between communities.”

The development comes after the UK’s National Deaf Children’s Society (NDCS) charity recently warned that a “lost generation” of children are falling a grade behind hearing classmates at GCSE.

According to the charity, less than half (48%) of deaf children achieve a grade four or above in both maths and English, compared with almost three-quarters (71%) of children without hearing impairments.

Despite a slight improvement in deaf children’s grades since last year, it predicts it will take more than two decades to close the gap, resulting in a lost generation of deaf children.

“The support that the European Union of the Deaf and the National Association of the Deaf have received to date from the StorySign partnership has had a genuine impact on families across Europe, and we want to continue collaborating with Huawei to support the future development of StorySign,” said Mark Wheatley, executive director at the European Union of the Deaf.

“As a learning tool, the app not only helps deaf children learn and enjoy how to read, but also helps parents learn how to sign. It has the potential to redefine our understanding of how we most effectively teach and engage with sign language, not only for families with deaf children, but also the wider community that are keen to develop a basic knowledge of sign language.”