EPFL students develop two apps to help the visually impaired

“We faced the difficult task of trying to make something simple,” says Blanche Duron, a Master’s student in communication systems, as she sums up her experience in the Assistive Technologies Challenge. That course-project is open to EPFL Master’s students under the MAKE initiative; it’s managed by the HackaHealth non-profit organization and supervised by three EPFL professors. The students’ goal is to develop technology that can simplify everyday tasks for people with disabilities. To ensure their apps are well-designed, the students work closely with potential users of their systems throughout the development process.

“People with disabilities who want to take part in the program contact us, and we speak with them to pinpoint exactly what difficulties they’d like to address,” says Alice Bruel, an EPFL PhD student and one of the Assistive Technologies Challenge organizers, along with Alec Chevrot and Iselin Froybu. “Then we present the project ideas to students, who work in cross-disciplinary groups to find specific solutions.” Six student projects were carried out in the spring 2023 semester (read about them here). The two for the visually impaired were carried out in association with the Swiss Federation for the Blind and Visually Impaired and could be useful for many people with a disability.

Carole Collaud is one of the individuals contacted by the Federation to work with the students. “Initiatives like this are essential for driving the development of applications and other technology that can help people with disabilities live more independently,” she says. “It’s also important to build awareness about this kind of diversity. One day, these students will be out in the workforce, potentially in jobs where they’ll be making decisions and managing projects with an impact on people with disabilities.”

Finding empty seats

Find My Seat is the app developed by one student group – Nicolas Matekalo, Aminev Tourki, Louis Gevers and Jonathan Haymoz – with Collaud’s input. With this app, visually impaired users can find empty seats on public buses, trams and more. “It was really gratifying to create something that can help people in their daily lives,” says Haymoz, a Master’s student in communication systems. “Working with Carole to find the right design for our prototype forced us to look at the technology from a non-engineering perspective.”

Find My Seat is easy to use, as you simply shake your smartphone to activate the app and to turn it off. The app uses the phone’s camera to spot empty seats and then vocally guides the user to them – further ahead, to the left or to the right – while indicating how many steps are needed to get there. Users can change the frequency with which the vocal instructions are given. “Speaking with Carole, we found that giving an instruction every two to three seconds was important,” says Gevers, a Master’s student in microengineering. “Her regular feedback was very helpful in improving our initial design.”

The app works on a standalone basis and doesn’t store any data, in order to avoid data-protection issues. In addition, the smartphone screen goes black when the camera is being used to find empty seats. “We first experimented with an approach that relies on a Bluetooth connection, but the results weren’t very accurate. And implementing code in the app proved to be really hard.” says Tourki, a Master’s student in microengineering. Matekalo, a Master’s student in computer engineering, adds: “This project gave us experience with the entire product development process: determining constraints, outlining a solution, building a prototype and testing it.”

It’s important to build awareness about this kind of diversity. One day, these students will be out in the workforce, potentially in jobs where they’ll be making decisions and managing projects with an impact on people with disabilities.

Getting drug information

Thanks to its cross-disciplinary nature, the Assistive Technologies Challenge also helps students acquire new skills. Alexander Popescu, a Master’s student in life sciences, and Mehdi Krichen, a Master’s student in microengineering, noted that they’d never worked on developing a smartphone app before. They formed part of the second group, along with Thibaut Stoltz, a Master’s student in robotics, and Duron. Their app, MedScan, lets users simply scan a box of medicine to hear the drug name, expiration date, prescribed dosage and information contained in the leaflet.

“I’m delighted to see this app being developed, because there isn’t one like it right now in Switzerland,” says Alain Barrillier, who worked with the student group. “There’s the Compendium app” – a database of drugs approved by the Swiss drug agency Swissmedic – “but it’s hard for us to use for a variety of reasons. For instance, to scan the packet of drugs you’re interested in, you have to place the barcode inside a small rectangle.”

MedScan is available in French, German and Italian. The students spent a long time developing the scanning system and the user interface, which is based on the TalkBack and VoiceOver screen readers for Android and iOS, respectively. They wanted to make sure the interface was easy to use for the visually impaired. Stoltz explains: “When we read a leaflet, for example, we scan the headings in bold until we find the information we’re looking for. But a blind person can’t do that. So we had to come up with a different way of grouping the information so that users can select exactly what section they want to hear.” Stoltz notes that small design details like that “can make a big difference, and show why it’s so important to get users involved in the design process.”

The other group members agree, and they stress that this kind of collaborative process is essential for considering different ways of thinking about and approaching a product. “A lot of software development projects fail because developers don’t take that into account,” says Duron.

Accessible apps

What’s next for the two apps? Collaud has asked EPFL’s Ethics Committee for approval to use Find My Seat, and the student group is in talks to incorporate their program into the Swiss railways’ SBB Inclusive app. The group has been in contact with Reto Allemann, the SBB manager in charge of the app, from the very start of their project.

With regard to MedScan, Barrillier would be happy to keep using it, but the students still aren’t sure what they want to do with the app. If they decide not to take it further, HackaHealth will ask them to make the code public so that one day someone else can pick up where they left off.