News details

CITI@DI Multimodal Systems researcher Sofia Cavaco with collaborators developed a new tool to help visually impaired individuals to perceive characteristics of the environment that are usually not easily acquired without vision.

The paper reporting on the system "From pixels to pitches: unveiling the world of color for the blind", by S. Cavaco, M. Mengucci, T. Henriques, N. Correia, and F. Medeiros, gets the Best Paper Award in the IEEE International Conference on Serious Games and Applications to Health, SeGAH 2013.

The system converts color information from still images or video frames into sound. The tool converts the hue, saturation and value parameters into sound parameters that influence the perception of pitch, timbre and loudness.

The tool has been experimented by visually impaired individuals, who confirmed that it can be used to give them information about the range of colors present in the images, presence or absence of light sources as well the location and shape of the objects. 

This work is part of See-Through-Sound, a research project (UTA-EXP/MAI/0025/2009) funded by the Portuguese Foundation for Science and Technology.