Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Cloud Native Ecosystem / Cloud Services

The Need for Time Series Data Is Now

Jan 9th, 2019 11:34am by
Featued image for: The Need for Time Series Data Is Now

Mark Herring, CMO, InfluxData
Mark Herring is a well-rounded silicon-valley executive with proven experience in taking complex technology and making it understandable to the broader audience. Currently, Herring is Chief Marketing Officer at InfluxData. Before InfluxData, Herring was VP of corporate marketing and developer marketing at Hortonworks, SVP of Products at Software AG, VP of Middleware, Java and MySQL Marketing at Sun Microsystems, and VP of Marketing at Forte Software.

Fueled by the massive growth of connected devices (i.e., IoT) and the rapidly increasing instrumentation requirements of next-generation software, time series technology has become more popular. Since launching InfluxDB, an open source time series platform in 2013, we have seen millions of downloads, built an expanding list of enterprise customers, and fostered a growing community that is always finding new ways to deploy and build on our platform. Most data is best understood in a time dimension, so when it comes to this space, we are really just getting started.

Amazon Web Services recently announced it will enter the market sometime next year with Timestream, a hosted time series database. We have been evangelizing the importance of high compression, super fast engines and a purpose-built stack for time series data, and Amazon seems to have followed this model pretty closely — which further validates the need for robust, established, purpose-built solutions like InfluxData. There are limited details about Timestream’s technical capabilities but based on the AWS model, there are likely several significant differences between our offerings, including:

  • Open source — InfluxData is first and foremost an open source company. We are committed to sharing ideas and information openly, collaborating on solutions and providing full transparency to drive innovation. Our products are continuously improved by an energized group of developers that help make them more reliable, secure and awesome. The power of the open source community to drive innovation is unsurpassed by any proprietary software solution.
  • Hybrid cloud and on-premise support — Companies have different and specific functionality needs, and distributing assets across multiple cloud-hosting environments is often the best option. In these instances, avoiding vendor lock-in is important as it can limit the ability to customize systems and negotiate better rates, ultimately making it very difficult, and expensive, to change cloud providers to meet ever-evolving business and technical needs. A hybrid cloud system offers the flexibility to choose services that best fit a company’s needs, whether it’s to support GDPR regulatory requirements or teams that are spread across multiple providers. And from an operations perspective, a multi-cloud system increases efficiencies and provides another layer of security to ensure there is no downtime.
  • Time-tested — Delivering a robust time series platform is not easy. We launched InfluxDB more than four years ago and InfluxEnterprise, our business offering, in 2016. Over the years, we have tweaked, redesigned and reconfigured our products to continuously improve overall functionality. Warts and bugs are inevitable stops in the natural evolution of software and we have worked out a lot of the kinks in our time- and community-tested products.

As awareness of time series technology broadens, we’ll see even more competitors enter the space, which is great because competition fuels innovation and grows the category as a whole. We are vigilant and focused on delivering an excellent platform, helping companies harvest business value from real-time analytics, and supporting the community of creative and talented developers who are discovering new ways to solve real-world problems with our platform. We will continue to push the edge with this technology — leveraging the advantages of a purpose-built system, developing new languages, aligning with new partners, supporting new platforms and new workloads, and pursuing exciting opportunities as they come.

InfluxDB is available now and we encourage developers to download and get up and running in minutes.

InfluxData is a sponsor of The New Stack, which requested this post.

Feature image via Pixabay.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.