Ghost in the IDE: Testing Replit’s AI Helper, Ghostwriter

Recently, I’ve been taking a trip through tools that give the software developer hints, help, debugging advice or (in some cases) complete code snippets, boosted by what we are currently referring to as AI. My particular preference are hints given on the edit line, like Visual Studio already does and Copilot does even more of. Given that I like online sandboxes, and I’m looking at AI tools, there does seems one obvious tool I should investigate.

Replit is an already popular online IDE, where your projects are hosted on ‘Repls’ (the term REPL or read-eval-print loop is at the root of the name). Replit is a hosted solution, so collaboration is built in. Recently the company introduced its LLM service, Ghostwriter.

Unlike the general abilities of OpenAI, the Economist noticed that Replit was favouring using LLM models to focus just on development. They use the Databricks AI platform, which Nvidia are heavily invested in.

With no need to install anything, I went ahead and created an account with Google. I could, of course, have used GitHub for identity. Interestingly, Replit asks not just what your intended use is (personal, collaboration, etc.) but how much development you have done. The first thing I saw on the dashboard was a bounty board, which I did not expect to see. This just underlines that Replit is as much a community as a tool — so it has a handy internal marketplace.

But I’m here for the spook. I noted that I could pay for the services of Ghostwriter for $10, via “1000 Cycles”; that is, $10 buys a 1000 Cycles. At least Replit are not directly exposing their AI service costs, so if things change behind the scenes they can flex a little. And a virtual currency is transparent — it is, after all, what Fortnite does. Of course, Fortnite weighs prices carefully, so you find yourself needing just one more pack of V-bucks to buy that skin you want.

My aim is to use Ghostwriter on a ‘Repl’, so I’ll start with 1000 V-bucks Cycles and see how I do. I assume that a public Repl is free, which mirrors the Github pattern.

In a previous post, I used a small class to check out Copilot. What Copilot did was to complete my method based on its name (using common conventions) and using its understanding of the C# FlagsAttribute. All this does is indicate that an enumeration can be used as a bit field:

This technique is a compact way of storing events; it is also cheap to check whether the event has occurred. So I can represent OccurrenceA and OccurrenceC happening by bitwise ORing them together in a variable:

And with the appropriate mask, we can quickly see whether, for example, OccurrenceC happened. This should be much more efficient than using a List.

So, I started a Repl and chose C#. After generating a C# template project, the little helper showed me around, but the screen comfortably follows the normal windowed display. Also, it invited me to try Ghostwriter:

Oddly, the Run button didn’t seem to work…at first. Then I noticed that my CPU maxed out, but it did eventually manage to print out “Hello World”. Hmm. I guess I can throw V-Bucks Cycles at it! (remember this is running on Replit’s infrastructural dime.)

Unfortunately, Ghostwriter started off unresponsive. This could have been because of network issues, or shyness. It didn’t tell me. I was getting responsiveness from the CPU cycles, but not Ghostwriter. While this is inevitable if you are offloading work to another service, the platform does need to do a little more work with health.

When the chat window finally responded, I was ready to go. Unfortunately, the Ghost was still not writing any hints in real time, but it was prepared to generate code in the edit window. So I asked it to create the SetOccurrences method. )

The first attempt didn’t work.

That is:

This is wrong, but I didn’t really help much: this would have been better named “AddOccurrence”.

I rejected the hint and tried again, this time improving the name a little:

And this is correct. It is ORing the new occurrence in, not setting it directly. I still should have named it better, however.

I was less fortunate with the method to check whether an occurrence has, er, occurred. It took many rejections, but eventually, I got the right bitwise calculation answer:

This is the correct way to use the flags. It ANDs the storage variable with the occurrence to check for, treating it as a mask. Any non-zero result shows that the occurrence occurred. Again, maybe if I gave my ethereal partner a better clue in the name, perhaps HasOccurrence, it would have come to the right answer a little quicker.

I wanted to see if I could persuade it to use HasFlag, which is available in .NET 7. While it didn’t suggest this (had I looked in the project file, I would have seen that it wasn’t using this framework, so it never would suggest this) it did improve the suggestion a little:

…which is a nicer formulation. After adding some test execution lines to the main.cs file, and some console output, the project class worked as it should have:

Just one more thing; a debugging task. For the flags attribute to work, the enums must have the expected binary values. What would happen if I set OccurrenceD to 5? Ghostwriter didn’t appear to notice, and there did not appear to be a “debug” option. However, I asked it to “Explain the code” and got this excellent response while the error was in place:

This is absolutely correct, and even finds the bug during analysis — which is almost human.

Altogether, while Ghostwriter didn’t appear to perform very proactively for me, it was certainly quicker to set up than Visual Studio and Copilot. I suspect with more CPU available, and a fair wind, it could behave in a more forthright style. It takes time to get used to any pairing partner, whether corporeal or otherwise. And as Ghostwriter does its haunting closer to the Replit infrastructure, I suspect sightings will be more reliable and system health will be easier to show. In short, Replit is definitely advancing with the spirit of AI. It has much more than a ghost of a chance to succeed.

David has been a London-based professional software developer with Oracle Corp. and British Telecom, and a consultant helping teams work in a more agile fashion. He wrote a book on UI design and has been writing technical articles ever since....

Read more from David Eastman