Demo to Program
Getting specs right is the bottleneck in building software. Can a more natural way to express specs change that?
An Exploration
My premise is that it's unnatural for humans to articulate what they know in precise detail. So to create clearer specs for software, we want to ask the user to demonstrate the action across several use cases. The computer or agent observes, infers, and attempts to replicate the behaviour on new inputs. If the agent is wrong, the user corrects it. The skill is transmitted implicitly. This is how any living being learns.
Here are explorations of this approach with two different problem domains: arithmetic and spatial. In each section, the Loom videos offer more detailed discussions.
Essential Challenges in Software Engineering 🤖 - Part I (Arithmetic Spec)
I demonstrated, using before and after drawings, how a whole quantity can be represented in numerals of any base.
The agent observed, inferred, imitated, and even created an automated tool for it.
Essential Challenges in Software Engineering 🤖 - Part II (Spatial Spec)
I demonstrated, using in-process screenshots, how a kolam pattern can be drawn in increasingly larger grids.
No automated tool could be produced, but with more effort to integrate external libraries, it is possible.
Essential Challenges in Software Engineering 🤖 - Part III (Future Directions)
With the current state of AI tools, it's possible to extend example-based programming beyond current spreadsheet-based offerings. There are inherent limitations to the kinds of problems which can be solved.
Domains which can be well represented in language are better served. Domains like spatial reasoning lag because they are not easily represented in discrete symbols so far. But there is promising research in this direction as well.