Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Platform Engineering / Software Development

Prototype the Path to Keep User Experiences Front and Center

How to build a great product through iterations and constant reassessment.
Jul 11th, 2023 11:56am by
Featued image for: Prototype the Path to Keep User Experiences Front and Center
Image via Shutterstock.

I once read that product development is like inventing a new recipe. The chef has the knowledge and skills to create a new dish and adapt to the available ingredients, but the recipe isn’t expected to be perfect on the first attempt. It’s a learning process. This kind of prototyping is just as important in product development because it allows teams to learn what works and what doesn’t as they move through variations to reach the final version.

At Slack, I work with teams to ensure we’re meeting the needs of both developers building automations on our platform and the end users that interact with those automations. We recently rebuilt and re-engineered our entire platform from the ground up, an undertaking almost two years in the making that relied on relentless prototyping. Throughout the development process, we encountered speed bumps that required us to reset, reassess or just completely restart. That’s because a great product experience (or recipe) emerges through iterations and transformations. It can’t be defined upfront.

Let’s take a look at how we do it at Slack.

Improving One Thing Can Power Everything Later On

Previously, creating an app on Slack’s platform meant going through a menu of options to configure permissions, entry points and more. The process was arduous and a source of confusion for our developer community. So in 2020, we released App Manifest API to enable the creation of apps using metadata through a single interface. Now app creation can be automated, with its configuration reviewed and versioned in a source control system.

When we started building the new version of the Slack platform months later, one of our first focus areas was enabling developers to host their apps on Slack infrastructure, ensuring that customer data and apps run in a safe and compliant environment. As we prototyped different versions of this system, we quickly learned that developers would need a way to create a Slack app so its permissions and basic configuration could be defined through our new command line interface (CLI). In one of these prototypes, we used the new App Manifest API and were able to configure an app that was later deployed and hosted on Slack. This path became part of the core experience that is available today on our developer platform.

When we created the App Manifest API, we knew we needed to make the experience of creating a Slack app better, but we didn’t know that it would later be used to entirely power the new developer experience.

Focusing on the Next Hill

At Slack, we work as fast as possible to unlock the next iteration of what we’re building, so we can get more feedback to make better decisions. This option-based approach helps, particularly in the early phases of the development cycle when there’s such a high level of uncertainty on what the desirable experience should look like. Focusing on the next hill means centering the development cycle on three principles:

  • Amplify learning: Share what you’re learning from each iteration with your team, especially if your team is distributed like we are. We have public channels where team members can share their prototypes and works in progress, and we even record clips using the features we’re building so we can walk through experiences as if we’re users.
  • Decide as late as possible: Build capacity into the architecture for change to happen so you can avoid one-way-door decisions that could lock you on a specific path. Constantly ask, “What could be the cost to change our mind on this implementation if we learn something down the road?”
  • Deliver as fast as possible: The smaller the build cycle, the less time you need to get feedback to validate your path and the quicker you can learn how the product is evolving. But keep in mind that rapid development comes from the right conditions and not from pushing the team to move fast.

When we built Slack’s new platform, we defined achievable but challenging milestones that allowed us to constantly learn, decide and deliver fast. We defined an early alpha version for select customers, released it for our internal teams and then continued iterating our way through private and open betas. Two years full of discipline, mistakes, learning and sweating was worth it because we created a platform that drives customers past complexity and noise to effortless solutions and productivity. Now, users can think of something as simple as, “I want to reuse this workflow that I created before and share it with a link,” and then press one button to make it happen.

We’ll continue to tinker with the platform to clear any productivity roadblocks because there’s always room for evolution and improvement, the whole time prioritizing and keeping the user experience at the forefront.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.