CAREER: Enabling Independent Access to Digital Graphical Contents for People with Visual Impairment

Project: Research project

Project Details


Digital visual information in graphical form (e.g., digital images, maps, diagrams, etc.) has become prevalent in the information era and the sighted people can easily enjoy the added value of graphical content. However, although computer users with visual impairment can independently access digital textual information with screen-reading software such as JAWS and Window-Eyes, such software cannot handle images, diagrams and other visually-represented information. On the other hand, the typical procedures for manually producing tactile graphics by sighted professionals are in general timeconsuming and labor-intensive, and hence the coverage is extremely limited. Further, there is no on-line and independent availability if the production has to be done by third-party professionals. Aiming at the ultimate and general goal of making visually-represented information independently accessible to users with visual impairment, this proposal addresses one specific and challenging problem of practical significance: how to make digital graphical content more independently accessible to students in the science, technology, engineering, and mathematics (STEM) fields. In addition to line-drawing graphics of primarily binary nature, a specific type of continuous-tone-scale image, human portraits, is also considered, for the special value that the facial images have in a humans social and emotional life. This demonstrates the potential of our approach in handling more complex visual content. The following research activities are planned to address different aspects of the general challenge: (1) A focused study on the peculiarities of haptic perception in tactile exploration of graphics; (2) Development of visual processing techniques for automated visual-to-tactile conversion; (3) Usability study for determining and developing intuitive and efficient presentation and interface schemes based on the real end-user; (4) Prototype development and evaluation; (5) Educational and outreach activities as an integrated part of the research. Preliminary results have been obtained in our initial pursuit of the above activities, including our recent experiments and evaluation with 6 users who are blind, demonstrating the feasibility and potential of the planned research and the proposed methodologies. The proposed work employs an end-user-centric approach and thus it has the potential of bringing to people with visual impairment the desired independence, privacy, coverage, and real-time availability in accessing graphical content. These are in general not achievable with current practice relying on sighted specialists. Intellectual Merit: The proposed work will advance the knowledge in the domains of visual-tactile cognition, image understanding, human-computer interaction, and prototype development, in the context of developing assistive technologies for people with visual impairment. The innovations in the proposed project contribute to addressing a real-world problem using a user-centric approach. The PI has been working towards the goal of the project for 3.5 years through feasibility study on the key components of the proposed work, in collaboration with professionals from local organizations/schools serving the blind population. Convincing preliminary results have been obtained and published in premier venues. The PIs institution provides an excellent and unique environment for performing the proposed work. Broader Impact: The proposed research has broad impacts on research, society/humanities, and education. The proposed efforts with novel research perspectives and ideas will contribute to the research fields of visual-tactile cognition, automatic understanding of visual data for visual-to-tactile conversion, and development of assistive devices for the visually impaired. The significant impact of the work on the society lies in its potential for enabling independent access to a wide range of visual content The PI currently has a funded NSF grant: 0845469 CAREER: Enabling Independent Access to Digital Graphical Content for People with Visual Impairment. This proposal seeks REU supplemental funding for two undergraduate students to work on a set of well-scoped research and development tasks that will enhance the study of the above project while being able to provide a unique research experiences for the participating undergraduate students. While the above CAREER project aims at the ultimate and general goal of making visually represented information independently accessible to users with visual impairment, the PI also has one specific and challenging problem of practical significance: how to make digital graphical content more independently accessible to students in the science, technology, engineering, and mathematics (STEM) fields. This practical problem is an excellent example where professional assistance will never come even close to leveling the playing field with regard to access to such visual information that the sighted people take for granted. To ensure the success of the project, the PI has outlined a plan for performing not only technology development but also human-centered usability study. The latter is heavily dependent of psychophysical experiments with the actual end users of the technology: the students at ASU who have visual impairment. This proposal intends to extend the scope of the above study by proposing additional research and development tasks that are appropriate for undergraduate students and that may further enhance the PIs existing objectives of the CAREER project. More specifically, I intend to recruit one Psychology senior student for working on psychophysical experiments and one Computer Science junior or senior student for working on extraction of graphics in STEM courses materials including textbooks. The specific tasks of the students will be elaborated in the below.
Effective start/end date3/1/092/28/15


  • National Science Foundation (NSF): $451,768.00


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.