Analysis / Top Stories

This Week on The New Stack: Apache Spark and QCon

29 Jun 2018 3:00pm, by

Hello, welcome to The New Stack Context, a podcast where we review the week’s hottest news in cloud-native technologies/at-scale application development and look ahead to topics we expect will gain more attention in coming weeks.

For this Context, we talk with two data scientists from Pivotalwho write this week about the pros and cons of using Apache Spark for running data science workloads at scale. Apache Spark is an in-memory data analytics engine that is wildly popular with data scientists because of its speed, scalability and ease-of-use. Plus, it happens to be an ideal workload to run on Kubernetes, the Pivotal team writes.

We spoke with them about the pros and cons of Apache Spark, what data science workloads they’ve been experimenting with lately at Pivotal, and how application architectures are evolving, in general, to better support data storage and processing at scale.

Then, later in the show, we’ll go over the highlights from his day at QCon developer conference in New York, including microservices debugging and how the Envoy service mesh is preparing to work with Kafka.

TNS editor-in-chief Alex Williams and TNS managing editor Joab Jackson co-hosted this episode of Context. 

LINKS

Pivotal is a sponsor of The New Stack.


A digest of the week’s most important stories & analyses.

View / Add Comments

Please stay on topic and be respectful of others. Review our Terms of Use.