TNS
VOXPOP
Where are you using WebAssembly?
Wasm promises to let developers build once and run anywhere. Are you using it yet?
At work, for production apps
0%
At work, but not for production apps
0%
I don’t use WebAssembly but expect to when the technology matures
0%
I have no plans to use WebAssembly
0%
No plans and I get mad whenever I see the buzzword
0%
AI / Large Language Models

More than an OpenAI Wrapper: Perplexity Pivots to Open Source

Perplexity CEO Aravind Srinivas is a big Larry Page fan. However, he thinks he's found a way to compete not only with Google search, but with OpenAI's GPT too.
Jan 12th, 2024 9:00am by
Featued image for: More than an OpenAI Wrapper: Perplexity Pivots to Open Source
Image via a Perplexity video.   

The AI search engine Perplexity has been getting a lot of buzz recently as an alternative to ChatGPT. Unlike ChatGPT, Perplexity provides citations by default for the information it delivers. That single feature has become crucial in generative AI, given the ongoing hallucination issue with this technology. Accordingly, Perplexity has become a surprisingly strong player in a market otherwise dominated by OpenAI, Microsoft, Google and Meta.

I spoke with Perplexity co-founder and CEO Aravind Srinivas (formerly a researcher at OpenAI and DeepMind) to find out more about the product, including its recent emphasis on open source LLMs. Note that this interview was conducted before the company announced a $73.6 million Series B funding round earlier this month, which financially propels it into the big leagues.

Perplexity links

A Perplexity answer to a question about JavaScript. Each of the numbers in the response is a clickable link to the source of the information cited.

More Than a Wrapper

At its core, Perplexity is a search engine. Srinivas told me he’s “a big Larry Page fan” and right from the start, when Perplexity launched in December 2022, he wanted to take on Google’s search engine. However, at that time, Perplexity was reliant on both OpenAI’s GPT 3.5 model and Microsoft Bing. It was merely a “wrapper” of other companies’ technology, to use a popular (and somewhat derogatory) term in the AI engineering community.

But over the past year, Perplexity has evolved rapidly. It now has its own search index and has built its own LLMs based on open source models. They’ve also begun to combine their proprietary technology products. At the end of November, Perplexity announced two new “online LLMs” — LLMs combined with a search index — called pplx-7b-online and pplx-70b-online. They were built on top of the open source models mistral-7b and llama2-70b.

“We started using open source models on the day LLaMA-2 came out,” said Srinivas, referring to the July 2023 release by Meta of its second generation LLaMA model (the name is an acronym for “Large Language Model Meta AI”). They also took note when a French company called Mistral AI released an open source LLM called Mistral 7B in September. After that, Perplexity’s strategy to become more than a wrapper took shape.

“There is a healthy competition between the two,” said Srinivas, regarding Meta and Mistral. “And that benefits us because we are like, ‘Okay, we are the users of these models.’ Like, we’re going to take your chatbot, we’re going to package it into a really efficient, fast inference that we host ourselves — so that we’re not a wrapper. Then we [will] customize it and fine-tune it for our models, our product — which is summarization for search use cases — and then we will deploy it to the end user.”

Using open source models has been critical for the growth of Perplexity. Srinivas noted that “Mistral’s latest model is as powerful — if not more powerful — than GPT 3.5, which is the model we started with [at] Perplexity one year ago.”

In addition to the backend technology, Perplexity’s user interface has also evolved with the times. Its default interface is still a chatbot (similar to ChatGPT), but Perplexity now offers what it calls “Copilot search” — which “asks for details, considers your preferences, dives deeper, and then delivers pinpoint results.”

Perplexity vs. ChatGPT

As a quick demonstration of the differences between Perplexity and ChatGPT, I asked the following question to both products: “How is JavaScript used in modern web applications?”

First, a disclosure: In the lead-up to the interview, Perplexity gifted me with a year-long Pro account in order to better test its product. I was already a ChatGPT Plus user, which I pay for myself, so I was able to do an apples-to-apples comparison of the two companies’ premium offerings. Both companies charge $20 per month for their premium services. Perplexity Pro enables you to “choose your preferred AI model from GPT-4, Claude 2.1, Gemini, or Perplexity.”

In answer to my query, ChatGPT 4 replied with a ten-point list of “functionalities and benefits” — including the use of JavaScript in user interaction and real-time web applications. It was a decent summary, although fairly high-level and I spotted a couple of lines I’d want to verify.

Perplexity’s answer, which came from the default Perplexity model, was more like a short article. It was equally as good as ChatGPT, but it also included more than 20 citations. Srinivas said that its default model is based on a fine-tuned version of GPT-3.5, plus a bit of LLaMA-2 — “somehow we route the two things together.”

Perplexity citations

Perplexity citations

I tried out the same query with Perplexity’s “experimental” model, which Srinivas said was “fine tuned in-house using LLaMA-2.” The response was shorter and, I felt, not quite as thorough as the response from the default Perplexity model. But it is experimental, so your mileage may vary. And, it turns out, conciseness is one of its goals.

“The experimental model is not better than GPT-4,” Srinivas explained. “What do you get out of it? Its conciseness [and] factual accuracy without any moralizing behavior.”

What’s Next for Perplexity

As noted above, the default Perplexity model still relies on GPT 3.5 (and a dash of LLaMA-2). But the intention is to move away from that long-standing reliance on OpenAI for its base model.

“Our goal now in the next quarter or so is to move everybody completely to our Perplexity models,” Srinivas said. “And now there is a choice — we can either use LLaMA-2 as a base model or the new Mistral as the base model.”

On the search side, I asked how Perplexity’s search index currently compares to Google in terms of scale?

“We have a billion pages in the index,” he replied. “But, you know, the main point I want to make is [that] search index size is also like a large language model size — it really doesn’t matter how big an index is. What matters more is how high quality the data is; how many high quality web pages are there?”

He noted that its search rank mechanism is similar to Google’s, in that it relies on citations, but with an LLM twist. The more that Perplexity’s product cites a certain web page, the more important it becomes. To explain, Srinivas once again name-checked his hero, Larry Page.

“Similar to the insight of Larry Page, who said that the pages in the web [that are] important are those that get cited by other important pages. Except we are saying, the pages on the web [that] are important are those that get cited by a large language model, in the context of a conversational answer engine — a chatbot. And if it’s used by more and more people regularly, we get to know more and more pages on the web that are important — […] the frequency of them getting cited, whether they actually made the answer better or worse.”

Given that Google (with Bard) and Microsoft (with Bing) have already begun to use citations too in their AI chatbots, Perplexity may have a challenging year ahead. But for a freshly funded young startup, its product is already compelling and the pivot to open source LLM models seems the best way to counter those big tech players.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.