AI accessibility? Blind gamer puts ChatGPT to the test

AI accessibility? Blind gamer puts ChatGPT to the test

Blind eSports player Masahiro Fujimoto (L), who goes by "Mashiro", hopes that AI could help him travel alone
Blind eSports player Masahiro Fujimoto (L), who goes by "Mashiro", hopes that AI could help him travel alone. Photo: Richard A. Brooks / AFP
Source: AFP

Japanese eSports gamer Mashiro is blind and often relies on a companion to get around Tokyo -- but he hopes that artificial intelligence, hailed as a promising tool for people with disabilities, can help him travel alone.

The 26-year-old "Street Fighter" player put the latest version of AI chatbot ChatGPT to the test on his way to a stadium for a recent Para eSports meet-up.

"I can't participate in an event like this without someone to rely on," he told AFP. "Also, sometimes I just want to get around by myself without speaking to other people.

"So if I can use technology like ChatGPT to design my own special needs support, that would be great."

This year, the US firm OpenAI released GPT-4o, which understands voice, text and image commands in several languages.

Read also

Musk's Neuralink eyes more test subjects for its brain tech

This year, the US firm OpenAI released GPT-4o, which understands voice, text and image commands in several languages
This year, the US firm OpenAI released GPT-4o, which understands voice, text and image commands in several languages. Photo: Richard A. Brooks / AFP
Source: AFP

The generative gadget, along with others such as Google's Gemini, is part of a fast-growing field that experts say could make education, employment and everyday services more accessible.

Following the streets' tactile paving, Masahiro Fujimoto -- who goes by his online handle Mashiro -- used his stick adorned with a small monkey mascot to find his way from the station.

As he went, he spoke to GPT-4o like a friend, receiving its answers through an earpiece in one ear, leaving the other side free to listen out for cars.

Having asked for basic directions, he added: "In fact, I am blind, so could you give me further details for blind people?"

Mashiro (L) cheering with his friend after they arrive at a stadium using an app with ChatGPT for directions
Mashiro (L) cheering with his friend after they arrive at a stadium using an app with ChatGPT for directions. Photo: Richard A. Brooks / AFP
Source: AFP

"Of course," the bot replied. "You might notice an increase in crowd noise and the sound of activities as you get closer."

Read also

Swedish game studio channels fandom in 'Star Wars Outlaw'

The journey, 20 minutes for sighted people, took Mashiro around four times as long with several U-turns.

When it started to rain heavily, he requested help from his friend, who is partially sighted, to finish the journey.

"Arrival!" finally shouted Mashiro, who has microphthalmos and has been blind since birth, using only sound to demolish his opponents on "Street Fighter 6".

'Enormous potential'

AI can cater to specific needs better than "one-size-fits-all" assistive products and technologies, said Youngjun Cho, an associate professor in computer science at University College London (UCL).

"Its potential is enormous," said Cho, who also works at UCL's Global Disability Innovation Hub.

People with hearing loss can use AI speech-to-text transcription, while chatbots can help format a resume for someone with learning disabilities
People with hearing loss can use AI speech-to-text transcription, while chatbots can help format a resume for someone with learning disabilities. Photo: Richard A. Brooks / AFP
Source: AFP

"I envisage that this can empower many individuals and promote independence."

People with hearing loss can, for example, use AI speech-to-text transcription, while chatbots can help format a resume for someone with learning disabilities.

Some tools for visually impaired people, such as Seeing AI, Envision AI and TapTapSee, describe phone camera images.

Read also

71-year-old Ghanaian grandpa writes 2024 BECE, warms hearts: "Determined to succeed"

Danish app Be My Eyes, where real-life volunteers help via live chat, is working with OpenAI to develop a "digital visual assistant".

But Masahide Ishiki, a Japanese expert in disability and digital accessibility, warned it can be "tricky" to catch mistakes from ChatGPT, which "replies so naturally".

"The next objective (for generative AI) is to improve the accuracy of real-time visual recognition, to ultimately reach capabilities close to that of a human eye," said Ishiki, who is blind.

Mashiro (R) said ChatGPT's limited recognition of Japanese words and locations made his AI-assisted journey more challenging
Mashiro (R) said ChatGPT's limited recognition of Japanese words and locations made his AI-assisted journey more challenging. Photo: Richard A. Brooks / AFP
Source: AFP

Marc Goblot of the Tech for Disability group also cautioned that AI is trained on "very mainstream datasets" which are "not representative of the full spectrum of people's perceptions and especially the margins".

Mashiro said ChatGPT's limited recognition of Japanese words and locations made his AI-assisted journey more challenging.

Although the experiment was "a lot of fun", it would have been easier if ChatGPT was connected to a map tool, said the gamer, who travelled around Europe last year using Google Maps and help from those around him.

Read also

Trotro mate brags about his work: "I earn more than a teacher in Ghana"

He has already decided on his next travel destination: Yakushima rainforest island in southern Japan.

"I want to experience whatever happens when travelling somewhere like that," he said.

Source: AFP

Authors:
AFP avatar

AFP AFP text, photo, graphic, audio or video material shall not be published, broadcast, rewritten for broadcast or publication or redistributed directly or indirectly in any medium. AFP news material may not be stored in whole or in part in a computer or otherwise except for personal and non-commercial use. AFP will not be held liable for any delays, inaccuracies, errors or omissions in any AFP news material or in transmission or delivery of all or any part thereof or for any damages whatsoever. As a newswire service, AFP does not obtain releases from subjects, individuals, groups or entities contained in its photographs, videos, graphics or quoted in its texts. Further, no clearance is obtained from the owners of any trademarks or copyrighted materials whose marks and materials are included in AFP material. Therefore you will be solely responsible for obtaining any and all necessary releases from whatever individuals and/or entities necessary for any uses of AFP material.