How One Computer Program Lets People Draw With Their Voice

Researchers at the University of Washington created a drawing application called VoiceDraw that allows people who cannot use a mouse or stylus to make art by speaking commands instead. The work is documented in a research paper from the Wobbrock lab.

The software translates spoken instructions — changes in pitch, how long you hold a sound, and specific words — into drawing actions. A user can tell the program to move around the canvas, pick colors, adjust how thick a line is, and draw paths. No hand or arm movement needed.

What made this project different

What sets VoiceDraw apart from other voice-controlled software of that era is how it was built. The team did not simply guess what artists would need. Instead, they worked closely with someone who already used voice as their main way to create and control things. That person was a collaborator in the design process, not just someone testing the finished product.

This matters. Assistive technology — tools designed for people with disabilities — has often been created by able-bodied engineers making assumptions. The result: software that technically works but feels awkward to use. By including a voice painter as a true partner in design, the team learned things that assumptions never would have caught: how tiring voice commands get over time, how much control you need for detailed work, how much mental effort it takes to translate what you want to draw into spoken words.

The harder problem

Most voice software of that time focused on simple tasks — typing out dictation, clicking buttons, filling forms — where success meant getting the right answer quickly and accurately. Drawing is different. When you draw, your strokes flow continuously. You often don't have a precise plan; you explore and improvise. You might want to express something rough or unfinished.

Building voice software for that kind of open, creative work is much harder than mapping speech to buttons. The interaction has to be flexible enough for an artist to think and experiment while speaking commands, not just follow a rigid script.

Broader context

The research community at University of Washington has long understood this complexity. Their work on accessibility does not treat it as an afterthought added to finished software. Instead, they see accessibility as a design challenge that, when taken seriously, makes the whole system better — a principle that has shaped how the field thinks about designing for everyone.

It is worth noting that the leap from a research prototype that works well for one motivated user to software that works reliably for thousands of people with different accents, voices, and physical abilities is steep and slow. Even today, building voice systems that work well across the full range of people who need them — not just the typical case — remains a difficult unsolved problem in assistive technology. The underlying speech recognition technology has improved greatly since this work was done, which opens the possibility of revisiting VoiceDraw's design questions with newer, more powerful tools.

For people building accessible software today, VoiceDraw offers a useful lesson. The real value is not in the specific technical details but in the process: the decision to build the design around the actual knowledge of someone who uses voice for creative work, rather than making assumptions, is the kind of careful thinking that assistive technology still needs more of.