ChatGPT: Smart, but Not Smart Enough
Yes, AI can help with programming, but ChatGPT is not ready to be your programming buddy, especially regarding securing your code.
Wouldn’t it be great to have an AI pair programming friend to help you secure your code? I’d love it. But, while GitHub CoPilot can be handy — leaving aside whether it’s ethical or legal — AI’s new darling chatbot, ChatGPT, isn’t ready for programming prime-time.
Programming Is a Different Story
I’ll give you that ChatGPT is going to make life much harder for high-school English teachers. Going forward, anyone who assigns a homework paper on To Kill a Mockingbird will be much more likely to get an AI-written document than any real student thought about the literary masterpiece.
But programming, especially secure programming, that’s another story. You need to be precise, you need to take context into consideration, and, most of all, you must be right, or as right as you can be anyway.
Even OpenAI admits ChatGPT “may occasionally generate incorrect or misleading information.” When it comes to coding, I need better than that. Much better.
Sure, it can do some things right some of the time. For example, it could not only spot and suggest a fix for this simple counting bug, it could also explain the bug.
It can also do a decent job of stringing together services. Here, for example, it generates the Python code needed to input an image and then reads the text on it on the Google Cloud Platform.
You Won’t Lose Your Job
These are impressive tricks, but the more you look, and think about it, the clearer it becomes that you’re in no danger of losing your programming job… yet.
Take, for example, that moving forward, you’d better darn well be sure that all the artifact dependencies in your code are “safe.” That means I don’t want just some lines of code calling in a library. I want to see, or at least have scanned, the files listed in its Software Bill of Materials (SBOM) and Supply-Chain Levels for Software Artifacts (SLSA). Can ChatGPT do that? Nope.
Or, can ChatGPT decide on what are the correct tests to run in my continuous integration/continuous delivery (CI/CD) pipeline? I wish.
Let’s remember what ChatGPT is, shall we? It’s an OpenAI Generative Pretrained Transformer 3 (GPT-3) large language model designed expressly for chatbot applications. It was trained on a large conversational text dataset. It can “chat” with you, like a version of ELIZA crossed with IBM Watson. It’s not — I repeat, not — designed to be a programming helpmate.
Never was, never will be. I expect there will be a GPT-3 (4?) model for developers, but it’s not here yet.
Making Stuff Up
In the meantime, while ChatGPT results can look good, in its current form, it has one really troubling aspect. “It makes crap up.”
I expect that from a high-school student BSing their way through an essay question. I don’t expect it from an AI. Perhaps, I should, but I don’t. And, when it comes to code, I don’t need a program “winging it.” That’s what interns are for.
What’s worse is that ChatGPT will “confidently” give you answers that are just wrong. That’s why StackOverflow, which knows a thing or two about answering programmers’ questions correctly, has banned, for now, any ChatGPT answers.
Why? Easy. As StackOverflow management stated, “Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking or looking for correct answers.”
And, as I just pointed out, “the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting.”
So, to prevent Stack Overflow from being flooded with ChatGPT spam, they’ve blocked it for now. I expect they’ll end up banning this version, at least, for good. Eventually, AI will be able to answer. In the meantime, all too often, we’ll still see “answers” such as “This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form.”
No one ever said Stack Overflow was a friendly community.
But, in this case, they’re right. It would be all too easy to create a Distributed Denial of Service (DDoS) attack using ChatGPT answers. As a writer called Pen Magnet observed, “When friction/risk to commit something gets minimized, there is heavy, unqualified supply/input.” This means there’s a real danger of Stack Overflow’s volunteer moderators being overrun with AI-generated garbage questions and answers.
And, in the meantime, the code answers would continue to be of poor quality. Sure, sometimes, ChatGPT gets it right. And, sometimes, I’d write non-trivial C code that would compile the first time out, but that didn’t happen all that often!
Not Good Enough
No, for now, you can play with ChatGPT. Enjoy yourself. But don’t rely on it for your programming. And go ahead and use it to spot simple insecure code mistakes. For real coding and secure programming, it simply isn’t good enough.
Mind you, that will change. Sometime, before fusion power becomes commercially viable, a ChatGPT descendent will be a useful pair programming partner. But we’re not there yet.
