How TuringBots Are Changing Software Development
There’s a good chance you’ve already interacted with a TuringBot. “TuringBot” is a term coined by IT research firm Forrester to describe generative AI-powered software such as Copilot or ChatGPT that helps developers code from the planning stage to the testing and deployment stage. It defines what analysts believe is a major technology trend, giving birth to a new generation of software development.
“TuringBots, essentially, generate code,” explained Mike Gualtieri, principal analyst at Forrester, during an online panel discussion held last week and now available for free online viewing. “Essentially, a TuringBot is a way to ask for run-ready code. And you can see just how transformative that can be to the developer experience.”
TuringBots are transforming all areas of the software development lifecycle, explained the panel. Diego Lo Giudice, also a Forrester principal analyst, outlined a few of the ways TuringBots are changing development:
- Prioritizing user stories on backlog based on a specific team or developer goals;
- Generating actual code, ala ChatGPT, for a user interface;
- Looking up technical documentation;
- Testing by automating thousands of visual tests over hundreds of web and mobile browser pages in seconds — testing has attracted huge investments, according to Lo Giudice;
- Automating configuration files for creating efficient DevOps pipelines;
- Simplifying teams’ collaboration and share product/project information more effectively;
- Providing data insights about quality, technical debt, or business value;
- Revisioning software by analyzing data from the application and using it to improve the code for better performance.
“When we use all these tools in a pipeline, there’s lots of data that’s being generated, there’s lots of data that we’ve created in the past and learning from all that data, we can come out with interesting insights and predictions that not only will help us improve the code, but actually take decisions on the processes, the practices, how the team is working, the deadlines, the plans — basically, TuringBots can do all of this,” Lo Giudice said. “You can see that this is much, much broader than design, prototyping, or even just coding.”
Who’s Who of TuringBots
TuringBots are not maturing at the exact same speed, Lo Giudice said. Some are still in the pre-release stage, he added. So far, the list Forrester is monitoring includes:
- Amazon has TuringBots in testing, delivery, and coding (CodeGuru, DevOps Guru, and Whisperer);
- Microsoft GitHub with its coding TuringBot Copilot, with a Copilot for Power Automate;
- IBM and RedHat with Project Wisdom;
- Tabnine with its AI assistant coder;
- Cody, a startup that does code generation;
- Unit tester TuringBots Ponicode by CircleCI and DiffBlue;
- Google with a DevOps TuringBot; and of course,
- Chat GPT
It’s a short list, but Lo Giudice also said companies that focus on application testing have been “infusing” AI into their tools for some time now.
“This list didn’t sort of appear,” Gualtieri said. “When ChatGPT was released, large language models and the use of those were not new, but it’s important to point out that even though there are a few dozen vendors doing this, it’s still very early in what they can do. It boggles the mind to think that this is really the beginning of this.”
Low Code Meets TuringBots
Low-code platforms have been using AI for some time, Lo Giudice said. AI doesn’t do away with low code; it makes it more interesting, he contended.
“Sometimes, natural language is not enough to specify the technicalities of what we want,” he said. “So the low code tools have all those technicalities as assets in the tool. So basically, we can start thinking about seriously composing easily applications with these assets.”
Some low code platforms are looking into areas where there are a lot of manual steps, such as discovery — basically, processes inside the organizations where there could be more automation and improvement, he said.
“There’s a whole lot of areas and use cases that no code tools are heading [to] with the use of AI and generative AI,” Lo Giudice said.
Using AI-Generated Code: Caveat Emptor
Often, developers are now using TuringBots to replace a search on Stack Overflow, said Gualtieri.
“What the developers now say is, instead of searching all these sites, I can prompt ChatGPT or another TuringBot, and say, write this code; and then I can say, No, write it this way, tweak it this way,” Gualtieri said. “That’s a huge productivity increase, but it’s a similar workflow that they’re using.”
A good use case for Turing bots is to improve automation, particularly in testing and prioritizing what to test, explained Lo Giudice. For instance, risk-based testing has been based mostly on intuition, but with data from TuringBots, developers can use the AI to provide more precise information on what’s really high risk and needs to be tested.
The risk that some Forrester clients have expressed is that if developers don’t find problems with the code, then they may become overly confident about future code created by a TuringBot.
“That’s when it becomes riskier because it’s based on a larger language model. And it has, again, those problems that we know about hallucination — it might generate code that is wrong. So you want to test it,” he said.
Even code from Copilot and TabNine, both of which have been in use for a while and distributed to thousands of clients, doesn’t go straight into production yet, Lo Giudice said.
“Many developers tell me [that] yes, it generates the code, but you have to tweak it, you have to adjust it, you have to make sure it’s using the latest library that you want to use,” Lo Giudice said. “They will become better and better. They improve [the] performance of developers, but you still need to go through rigorous steps of security checking, testing, and then deployment.”
