Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.

It’s All About Software Development: The Fierce Rise of AWS

Nov 14th, 2016 8:17am by
Featued image for: It’s All About Software Development: The Fierce Rise of AWS
“It’s All About Software Development,” will be a series of posts, starting with this one, that explore why organizations should care about optimizing for effective software development. What does effective software development look like? How can organizations move from their (mostly) pretty ineffective software development to effective software development without magic? Read on…

Joe Emison
Bio: Joe Emison is Founder and Chief Technology Officer at BuildFax, the only provider of automated property condition information to the insurance and lending industries. Joe launched BuildFax on the cloud in 2008, and has consulted with many other companies on their own cloud launches and migrations. Joe oversees all technology and product management for BuildFax. Joe graduated with degrees in English and Mathematics from Williams College and has a law degree from Yale Law School. Find Joe on Twitter: @JoeEmison.

The coverage of the rise of Amazon Web Services (AWS) into a $10-Billion-dollar-a-year company is rarely accompanied by a coherent, comprehensive explanation of why. Why did Amazon rise so quickly with its public Infrastructure-as-a-Service (IaaS), and why is it taking so much spend from so many players (hardware, software, and services) so quickly?

The press around the rise of AWS tends to either skip over any explanation of why companies are moving to public IaaS, or tend to focus on the trees instead of the forest: things like cloud bursting (which basically no one does), or fine-grained pricing.

The rise of Amazon Web Services is about the rise of software, period. It is a natural consequence of Marc Andreessen’s observation that software is eating the world, and the technology companies that get this are the ones that are going to be the most successful.

If software is eating the world (companies are increasingly becoming software companies), then the forward progress of companies is directly related to their ability to develop software effectively.

It is a fairly simple logical path: Whether a company will be successful, and how successful it will be, relates directly to how effectively it can develop and deliver software.

Today, a huge part of successful business execution is around execution on software development. And software development is not easy to do correctly. In my experience, the vast majority of companies are developing software in ways that experts know leads to sub-optimal results, and yet most of these companies have no clue.

This relates to the lack of effective continuing education in technology, demand outstripping supply for good talent, and technological innovation outstripping most companies’ ability to leverage it.

Amazon Web Services’ public IaaS offers to software development what Software-as-a-Service (SaaS) offered to traditional business processes.

Why did Salesforce disrupt Act!? How has Google Docs disrupted a good chunk of Microsoft Office? Why do most of us prefer web- and mobile-app-based software over desktop software? Because SaaS is just better software: bugs get fixed faster, new features come out faster, and we can pay per use, instead of big capital investments up front.

Software development before Amazon Web Services sometimes meant waiting weeks or months for IT to provision development machines. Only being able to deploy updates every quarter because of manual IT and QA processes. And the business’s growth based on software being delayed because of these slow processes. The rise of AWS can be traced to tons of conversations like this:

Business: “Why will making this simple new program take us nine months?”
Software Development: “Well, the development will only take three months, but getting the machines provisioned for development, and then test, and then deployed to production adds another six.”
Business: “Isn’t there any way to make it go faster?”
Software Development: “Well, we could go around IT and buy from AWS and get all of the machines we need up and running within minutes.”
Business: “But how much will that be?”
Software Development: “A few hundred dollars per month”
Business: “Here’s my corporate card. Go.”

Today, Amazon removes so much friction from traditional software development and software deployment processes, while still providing all of the elements of what is needed to deploy software effectively for essentially every company (to the point that it is getting increasingly and problematically complex). Reducing the time to develop and deploy software translates directly into higher growth for the increasingly larger numbers of companies that are using their software to capture more revenue.

Feature image via Pixabay.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.