Data / Development / Sponsored / Contributed

Integrate Powerful Search Capabilities into Your Apps — without Adding Complexity

19 Apr 2021 9:47am, by

Marcus Eagan
Marcus is the senior product manager for Atlas Search at MongoDB and an open-source contributor to the Apache Lucene and Solr projects. Previously, he’s worked at Lucidworks, Ford Motor Company as well as successfully founding and selling a cybersecurity startup. In his spare time he is an angel investor and actively mentors people from traditionally underrepresented groups who are looking to break into technology and software development.

Search engines have become essential components for almost every user-facing application. For retailers, search results and recommendations directly drive revenue — since they enable users to find products fast, while offering businesses opportunities to better understand each customer and deepen engagement.

Being able to tailor experiences unique to a user’s search preferences, in order to surface more relevant content to them, is especially important for social media, digital media and e-commerce apps. For example, if I search for houseplant care videos on TikTok, I’ll be more likely to receive those types of recommendations moving forward.

Incorporating robust search engines into these and other types of apps is often non-negotiable. Fortunately, there are a number of search technologies available today — many of which are based on the industry-leading open source Apache Lucene search engine. But syncing, scaling, and maintaining your search engine alongside your operational database can add serious complexity to app development and operations.

Developer Considerations of Integrating Search With Your Operational Database

App users today want fast, easy access to information. You want to be able to find what you’re looking for quickly and easily, and you expect results to be filterable and relevant. Incorporating full-text search into your app is critical for meeting user expectations and maximizing the user experience.

Additionally, your ability to succeed as a company — no matter what industry you’re in — depends on your speed to innovate using software and data. This means that you want to empower your development teams to be as productive as possible and not waste time on undifferentiated work — such as managing a database or syncing data between systems.

Most traditional databases and query languages aren’t optimized for full-text search, so developers often use separate technologies to satisfy their speed and relevance requirements. You might choose built-in database functionality for exact match use cases, an open source search engine to power a website search bar, and a specialized search engine for an internal knowledge base portal.

Most developers stand up a separate search database alongside their operational datastore, to take advantage of search-optimized indexing and querying capabilities. But if you use a distinct search engine and database platforms, you’ll need to replicate data between the two environments and maintain consistency going forward. You’ll also need to manage and pay for the search system where the second set of data resides. If you have policies to age out data — ultimately sending it to cold storage, for example — you’ll need to age the data out of the search engine and also the database. Finally, you have to account for the learning curve and resource specialization needed to have people use and manage different technologies.

There is another option. MongoDB introduced Atlas Search to make it easier for organizations to integrate powerful full-text search capabilities into applications while minimizing backend complexity. Because Atlas Search is fully integrated with MongoDB Atlas, the global multicloud database service, there’s no need to replicate data and manage distinct data environments to add search functionality to your app. You have a single system of record. You can avoid setting up, maintaining, and scaling a separate search platform.

Atlas Search works by embedding an Apache Lucene database on each node of a MongoDB Atlas cluster, and is designed to be simple to use without any extra infrastructure setup involved. You define the field mappings and index options for the documents in a collection, then Atlas Search will index the data and make it available for relevance-based search. Once data is indexed by Atlas Search, you can build search queries using the MongoDB aggregation pipeline.

Atlas Search has dozens of search operators you can use to refine your results and craft complex search logic — like autocomplete, filters, and scoring — within a single operation.

Importantly, Atlas Search offers rich full-text search capabilities directly within your operational database, thus making it easier for developers to incorporate a robust search experience into your apps. You don’t have to pay for, set up, and maintain yet another service in your tech stack, or learn a new query language.

Realizing the Real-World Benefits of Fully Integrated Search

MongoDB Atlas Search is a relatively new offering, but over 1,000 companies are already seeing the benefits of using it to build fast, relevant, full-text search on top of their data. It’s an obvious choice for companies just starting out, who don’t want the complexity of managing multiple document stores. It’s also a great alternative for teams that already know the pain of integrating search into their apps.

Current — a leading U.S. challenger bank with over two million users — is using MongoDB Atlas with Atlas Search as part of its custom-built ledgering system, called Current Core. Atlas Search powers its popular peer-to-peer payment feature and adds search functionality directly to the data stored in Current Core’s MongoDB database, without the need for manual data replication or synchronization.

Blerp is an audio expression platform for streamers, gamers, and anyone who enjoys sharing audio memes. It has over 100,000 installations built on MongoDB Atlas with Atlas Search, which has allowed Blerp’s IT team to drastically simplify their architecture.

We’d love to learn more about the apps you’re building and how you can utilize Atlas Search and MongoDB Atlas. Learn more here.

The New Stack is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Real.

Feature image via Pixabay.

A newsletter digest of the week’s most important stories & analyses.