Case study · Cal Hacks 13.0 · October 2026

TouchSpace: tactile and audio maps for blind and visually impaired people

Scan a room with a LiDAR iPhone, and TouchSpace turns it into a floor plan you can feel on a Dot Pad tactile display, hear through spatial audio, and ask questions about out loud. Built in a weekend with James Zheng and Sunghyuk Yoon.

Claude for Public Benefit Prize · $5K from Anthropic

Watch the demo → Code on GitHub → Devpost →

PythonFastAPIReact + Three.jsTypeScriptClaude APIApple RoomPlanWeb Bluetooth

Ho Lok holding the Dot Pad between teammates holding the TouchSpace web app and the iPhone scanner — The team at the Palace of Fine Arts, San Francisco. I'm in the middle, holding the Dot Pad.

The problem

Turn-by-turn directions tell you where to go next, but they never give you the shape of a building. Sighted people glance at a floor plan before visiting an unfamiliar place; blind and visually impaired people rarely get that chance. TouchSpace is a way to learn a space before you arrive, from home, through touch, sound and a first-person walkthrough.

How it works

1 · ScaniPhone LiDARApple RoomPlan captures walls, doorways, floor boundaries and furniture.

2 · ReviewLabel the mapName rooms and destinations, confirm which doorways are passable.

3 · ShareMap packageGeometry, labels and navigation data travel as one file others can download.

4 · ExploreFeel, hear, askDot Pad pins, stereo audio cues and spoken questions in the web walkthrough.

Three ways to read the same map

Touch. The web app drives a Dot Pad with a 60×40 pin display over Bluetooth. Walls rise as lines, doorway gaps stay flat, landmarks get symbols, an X marks where you stand, and a planned route appears as spaced dots. A braille line shows the current room and heading.
Sound. With headphones on, nearby walls and furniture are placed in stereo, with cues for obstacles ahead, doorway crossings and the edge of the mapped area.
Voice. Ask “Where is the restroom?” or “Take me to the elevator.” Claude works out what you mean; the answer and the route come from the recorded map, and the route is highlighted in the 3D view, the floor plan and the tactile display at once.

The design decision that mattered most

The AI is never allowed to invent a route. Claude only interprets the spoken question: which landmark, which destination, what kind of answer. Every direction is then computed by Dijkstra's algorithm over the navigation graph recorded during the scan, and every description comes from deterministic functions over the map data. If the map has no path, the app says so instead of guessing. For a tool that blind users would trust with their safety, a confident wrong answer is worse than no answer.

What I built

The FastAPI backend: 20 endpoints for map import, storage, spatial questions, routes, speech and the community catalog, with 153 automated tests.
The React web app and the walkthrough: floor plan, audio and Dot Pad panels kept together.
The map schema and floor-plan processing, including import of RoomPlan scans into a normalized map format.
Dijkstra routing over the recorded path graph, and the route editor for recording and fixing paths.
The Claude question interpreter that maps free-form speech to map operations.
The 60×40 Dot Pad preview, so the tactile output can be developed and tested without the hardware.

James built the Bluetooth Dot Pad driver. The backend, routing and the “no invented routes” rule are pieces I can walk through line by line.

20API endpoints

153automated tests

2,400pins on the Dot Pad (60×40)

3people, one weekend

What's honest to say about it

TouchSpace is a hackathon prototype. The web walkthrough is virtual practice on recorded maps; it does not track a real position or detect live obstacles, and it is not a replacement for a cane or a guide dog. The iOS scanner and the web app still use different map formats, so the full scan-to-tactile handoff is not finished, and the system has not yet been tested with blind and visually impaired users. Those are the next steps, along with richer braille labels and support for other tactile displays.

Try it

The web demo runs without an iPhone, a Dot Pad or any API keys: clone the repository, start the backend and frontend as described in the README, and choose Try Demo. The Dot Pad panel shows a pin preview on screen even with no hardware connected.