INTELLIGENT TECHNOLOGY
AI
Retro Rabbit warns businesses of AI trap in software development
AI is transforming every aspect of the software development lifecycle, but because the technology delivers such professionallooking outputs, organisations risk falling into an‘ AI trap’ and overlooking key factors in development.
This is according to David Noakes, Software Architect at Retro Rabbit / SmarTek21, a leading African design and engineering software consultancy.
Noakes said:“ AI can improve and speed up all aspects of the software lifecycle – from writing specs to building the code to testing it. But organisations should not just blindly trust it and fall into the traps of neglecting the human connection and overlooking major domain issues.
“ An LLM-assisted document can now be produced from little more than a strong idea, a decent prompt and some context,” he said.“ AI is fluent in‘ plausible’. It’ s very good at producing answers that look convincing, but it can’ t tell you which of those answers is the correct one for your business, your customers, and your regulator. That judgement comes from people who understand the domain deeply.”
Noakes noted that the scarce, valuable skills needed today were not the ability to generate options, but rather the ability to know which option was the right one.
He said:“ For example, the specification document that’ s meant to capture what a project needs. A spec put together with AI can be genuinely hard to argue with: it’ s comprehensive, well-structured, and thoroughly considered. It may look polished, complete, and confident – all while quietly missing the very areas that matter most. It could omit all the nuances that make or break the project. In the past, a document like this was earned. It came from real domain knowledge and lengthy sessions with key stakeholders, so even when it took time to pull together, it tended to capture the most important and
It takes extensive experience and human skill to read these situations and to spot when a spec, or a whole project, has drifted from the real problem.
most nuanced parts of the problem early. The traditional spec document may have looked rough around the edges, but it covered what counted.”
He added:“ A signed-off spec captures what was said in the meeting, but the real problem usually lives in what was meant. It’ s the unwritten stuff, the human stuff: instincts, exceptions, that look of concern from a key stakeholder. That quick conversation with the quiet, sharp developer who didn’ t speak up, but filled half a notebook.”
“ It takes extensive experience and human skill to read these situations and to spot when a spec, or a whole project, has drifted from the real problem,” Noakes said.“ This matters more now because of AI, not despite it. The building phase used to be slow, and that slow pace was a safety net: sprint after sprint of development gave you time to notice something was off and correct your course. AI removes that safety net, it builds the spec fast, and it looks finished far sooner than before. For many industries that might not sound like a problem. But in highly regulated ones like insurance or banking,‘ looks right but isn’ t’ is exactly the kind of thing that turns into an expensive problem down the line.”
David Noakes, Software Architect at Retro Rabbit / SmarTek21
“ AI can be innovative, and it can unlock people to be more creative, brainstorm against it and use it as a sounding board. But it’ s important that it doesn’ t take over the process – that’ s the trap,” he warned. •
32
INTELLIGENT CIO AFRICA www. intelligentcio. com