Modeling in 3D is inherently visual (rotating, inspecting depth, adjusting alignment), and standard tools exclude blind or low-vision users. A research team spanning multiple universities set out to change that. They created A11yShape, a system that lets blind programmers write and refine 3D models using code and AI feedback rather than relying purely on visual cues, tells Tech Xplore.
Under the hood, A11yShape combines OpenSCAD, a script-based modeling language, with GPT-4o, which can interpret both code and rendered visual inputs. Every time the user changes code, A11yShape renders views from all directions (top, bottom, left, right, front, back) and passes both the code and images to GPT-4o. The model then converts that into clear, plain-language descriptions: dimensions, positions, relationships between parts. Users can query (“How wide is the base?”) or ask for suggestions.
A11yShape links four views of the model: the raw code, the AI’s description, a hierarchical structural view, and the rendered visuals. A “cross-representation highlighting” feature ensures that selecting any part in one view highlights it across all others (code, description, and rendering). This connection gives blind users a multifaceted grasp of the model.
To test usage, researchers ran a study with four blind or low-vision programmers (none experienced in 3D modeling). After a short tutorial, they completed 12 models (skewer, robots, rocket, helicopter, etc.). All succeeded, and they rated the usability highly (System Usability Scale score ~80.6), a strong result in a domain that had been inaccessible to them.
There are challenges. Descriptions can be verbose, risking cognitive overload. Spatial judgments are tricky without tactile or depth feedback, so occasional misalignments (e.g., props hovering) appeared. The team plans to refine the AI’s brevity, add auto-complete features, and explore tactile or haptic feedback.
A11yShape doesn’t just make a technical advance; it’s a statement: the tools that define creativity and engineering should include everyone. By stitching together code, language, structure, and visuals, it opens a door for blind creators to step into 3D spaces previously off-limits.