Type of Project: Funded project Joint project Model project
VisualGPT: A Multimodal AI Tool for DeafDidactic Materials in Vocational Training

VisualGPT is an ongoing, BMAS-funded project developing a multimodal AI tool to transform written vocational content into accessible, DeafDidactic learning materials. Deaf learners face barriers from inaccessible written language and scarce German Sign Language (DGS) resources, limiting inclusive qualification. Currently in active development by a consortium led by Open Mind Software GmbH, VisualGPT builds a DeafDidactics-based AI pipeline that integrates specialized models for text processing, image generation, and video synthesis. This pipeline supports the creation of explanatory videos and slides that combine written language with images, animations, infographics, pictograms, and DGS-based video elements, all aligned with DeafDidactic principles and adaptable to different learner levels. Planned outcomes include VisualGPT Manager (material transformation), VisualGPT Coach (personalized AI assistant), and peer-to-peer training. All components undergo user studies with deaf learners to assess accessibility and efficacy. Developed by a fully inclusive team of deaf, hard-of-hearing, and hearing experts across partner organizations, the project aims at sustainable inclusion in vocational education and workplace training.