Can we learn from errors? What if we're running a nuclear power plant?

There appears to be what I call a 'cognitive loop' involved in the act of (code) creation: idea -> design -> implement -> test -> idea. That is, testing a creation that reveals errors feeds back into refining the idea and therefore the subsequent steps in the repeat loop. But the process is limited by available cognitive resources and the time it takes to complete one circuit of the loop - needing too many cognitive resources or if the loop time is longer than short term memory limits significantly interferes with the ability to notice and learn from the mistakes.

Traditional waterfall models of software development have shown a serious problem in that their cognitive loop is corporate not personal, and its duration is the product cycle, so the process is incredibly sensitive to correctness in requirements and specification. And unless the project is one of automating existing well-known processes (banking, chemical plant, etc) it is fundamentally impossible to actually know in advance what the requirements or specifications really are. Plus, on a personal note, my whole reason for being in this field is the joy of exploration, of doing what by definition has not been done before, of continuous learning OTJ.

Which brings me back to the small stuff: the everyday experience of a programmer. How to balance how much code to write before syntax checking, test harness runs, operational test. These decisions are modulated by the cost of doing each part. And this cost is implicit in amongst other things the tool set available. The traditional model is edit - compile - test - edit. Which can be quite a long loop - long enough to get a cup of coffee during the compile step (maybe). I have found that (for me) an exploratory model is much more productive - continuous course correction, continuously running and demo-able code, small increments. The biggest factor has been developing a programming environment that eliminated the compile step - suddenly the cognitive loop is closed at the edit stage and course-corrected motion towards the final product becomes more like swimming than run-wait-fear jerky progress. And the environment proved also to be accessible to non-programmers; our marketing VP said "I like VNOS because it makes me feel smart", and he wrote himself a weekly alarm applet that played an mp3 at happy hour on Fridays.

Summing up: the shorter the cognitive loop the better the learning, the surer the corrections; a lesson is learned only if the mistake is visible both as what is wrong with the result and why it went wrong and how to fix it and advance. Don't choke the student.

Cognitive Daily: APS 2008: Can we learn from errors? What if we're running a nuclear power plant?

Blogged with the Flock Browser