TNS
VOXPOP
How has the recent turmoil within the OpenAI offices changed your plans to use GPT in a business process or product in 2024?
Increased uncertainty means we are more likely to evaluate alternative AI chatbots and LLMs.
0%
No change in plans, though we will keep an eye on the situation.
0%
With Sam Altman back in charge, we are more likely to go all-in with GPT and LLMs.
0%
What recent turmoil?
0%
AI / Operations

With ChatGPT, Honeycomb Users Simply Say What They’re Looking for

The conversational AI approach to debugging helps users understand the system better, something black box AIops solutions can't offer, Honeycomb CTO Charity Majors told TNS.
May 18th, 2023 9:00am by
Featued image for: With ChatGPT, Honeycomb Users Simply Say What They’re Looking for
Image by Mahesh Patel from Pixabay.

Right about now every company is trying to figure out how to get a competitive hand with ChatGPT. For observability platform provider Honeycomb, the OpenAI technology promises to make querying easier for its users.

The company has embedded into its user interface some natural language querying capabilities, called Query Assistant.

So when a system goes awry, and the operations team jumps on the Honeycomb platform to find out what’s up, they can ask questions in conversational English, rather than spending time parsing their ideas into a SQL-like query that the platform would understand.

The chat assist may even make the platform easy enough to interrogate that even non-specialists can use it to understand code behavior, promises Charity Majors, CTO of Honeycomb, in a phone conversation with The New Stack.

Prior to the Query Assistant, Honeycomb users constructed queries using a visual SQL-like language of Honeycomb’s, with the usual SQL clauses such as GROUP BY and WHERE and LIMIT, and so forth.

The Query Assistant offers a single blank box where users are urged to write their question, in the form of “Can You Show Me?” such as  “Can you show me all the slow endpoints by status code?”

Majors prefers the ChatGPT approach over the traditional use of AI in operations, often dubbed AIOps. By its conversational format, ChatGPT educates the users on how the underlying systems works, through its question-and-answer format, whereas AIops may just provide a potential answer, with no indication of how it got there.

“Anytime you’re going to rely on AI to make the decision, it’s almost impossible for humans to follow the trail of breadcrumbs and figure out why it did what it did, and then take better actions on that,” Majors said.

Tracking the State

Overall, it took Honeycomb only about six weeks to ship this feature.

Honeycomb chose ChatGPT as a chat assistant because it offered the best mix of features, Majors said. No in-house AI experts were needed: Honeycomb’s in-house software engineers were easily able to work with the technology.

The company had to do a fair amount of work to prepare materials — such as defining keywords, data types, and schemas — that ChatGPT could use to interpret the queries, given that ChatGPT can’t as of now, hold state. For a user to have an extended conversation about a problem involves collecting a bunch of information to ship over for each query.

Nonetheless, Majors was surprised in how much ChatGPT could infer additional facts based on the input. Yet, ChatGPT had no hallucinations (or false facts conjured by ChatGPT). “I just feel like with numbers, ChatGPT is a little bit better than with words,” Majors said.

BubbleUp

Query Assistant is not Honeycomb’s first user-aid. Another feature BubbleUp, which allows the user to pick a subset of a dataset where something of interest appears to be taking place, where the platform will compute all the dimensions that lie inside of that bubble. But while BubbleUp makes the inspections process easier, it still requires to user to be somewhat conversant with the system’s own domain knowledge.

Query Assistant, Majors said, “is very much for the people who aren’t quite sure where to start.”

“It jumps over that whole needing-to-know SQL, and you’re just like, ‘I just deployed something, where are the errors?’ It leaps you to something that might not be exactly what you want, but then you can tweak it,” Majors said.

Query Assistant is a free feature for Honeycomb users, currently as an experimental offering. Admins can turn it off. No customer data is shared with OpenAI or used by Honeycomb internally.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.