Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Software Development

How to End a Software Project

While starting a software project is the cool thing to do, ending things well is a sign of maturity in your approach to development.
Aug 7th, 2023 10:12am by
Featued image for: How to End a Software Project
Image via Unsplash

Whether it is closing an internal project or shuttering a public platform, there are several ways software can come to an end. While starting a software project for the first time is clearly the sexy thing to do, ending things well shows an extra degree of maturity in your approach to development. In this post, I’ll look at the areas to consider when you have a modicum of time and space to react; these may be your direct responsibilities, but more likely just in your purview.

That restaurant you went to — it is still there but now has a different owner. You noticed the menu stayed the same, but the quality seemed to dip. Some of the guests seem a bit extreme. The price structure changed. The proprietor seemed not to care but promised the establishment would have a full makeover. It changed its name to a single letter and placed a gaudy sign above the door. When you look back, you realize that something did end, but no one admitted it.

I am of course, not talking about a restaurant but a certain social media platform. To end a project improperly because you want to keep the users, or simply because you don’t know what to say, is foolish. The bond lost with customers by ruining your own products or services is much reported on at the moment. An interesting point made by Andy Piper, a long-standing developer relations luminary at Twitter, is that an end should have already happened sooner.

It is a big mistake to think that showing interest in ending things before things end makes you some type of loser. People remember downfalls, but they remember intemperate behavior with failure for far longer. And that includes the people who might otherwise work with you.

See the End, Before the End Sees You

Even in the most comfortable of gigs, try to retain the samurai (or bushido) code; always be ready to die.

The true meaning is by having a constant consciousness of death, people can achieve a state of freedom that transcends life and death.

One difference between a keen startup and a staid organization is that the former never sees or contemplates an end. And yet, this is not how the universe works. How do you combat this personally? In practice, this means as a developer being aware of decisions and actions that only make sense If Nothing Bad Happens. Don’t deny risks out of fear, but imagine how the chips will fall if things do go South. That agreement that lasts longer than your project has even existed — is that sensible? How extended are we becoming? How long will it take to undo an action? What are the assumptions made about expected user numbers? Why are we depending on the services of a company when we don’t even know how they are funded? Does this new management really understand what we are doing? This is not unnecessary pessimism, it is just accepting the full lifecycle of a project may have a shorter timeline than was first imagined.

A Good Ending

Some projects have ended with very good grace. has a lovely memorial site, almost as good in death as it was in life. While this type of act of preservation is more devotion than good business practice, it would look good on a CV. The other extreme is Google, which has so many dead projects it has its own software graveyard. But at least this sets expectations; you fail fast and move on. Your aim should just be to show pragmatism with a little empathy when an end comes.

The Why Doesn’t Matter too Much

The actual reason for closing can simply be due to success, expense, or some other resource restriction. Or it can be that the company behind the product has changed direction. Or the facility will be covered elsewhere. But the true reason for closing may not be revealed to staff or customers anyway. In fact, it may remain shrouded for years. This is not something to worry about — even in your own life you have probably been a bit economical with the truth when explaining why you stopped doing something. Organizations have share value and competitors to think about, so a reasonable fiction is usually better than prolonged uncertainty. Either way, your responsibilities as a developer remain largely the same.

Going, Going, Gone

Almost by definition, there will not be one defined “end”. Like the last Lord of the Rings film, there will be many ends. There is usually a grey period in which things are being decided, but no one is committed to anything. (I’m aware some organizations work like that permanently.) So whenever you plan, you have to caveat each point with whether you mean the left side of “end” before things cannot be reversed, or the right side of “end” when you have pointed your skies down the cliff and jumped.

Communicating intentions to users during the sunsetting period when an end is official is fraught because you can’t help but treat them as a pack of scared gazelle who will bolt at the first sign of trouble. Yet most users have been stung by disappearing services before, so where possible just stick to saying what you know. Try to offer a way that users can escape and retrieve what they need. You do have to express things forcefully, otherwise, users will default to hoping they can continue as before. Ironically, if people are used to a good service, they will find it harder to understand what is going on.

What About the Data

Sometimes you can just “give back” the data in terms of customer reports, but usually, something will be lost. A social network can give back a list of your tweets, but we all know it is the network that is of true value. In almost all cases held data should be safely disposed of, or it will become unmanaged. During the sunsetting period, try to offer the users as many escape options as you can reasonably support. This may be just an export of a comma-separated data file, or onboarding to another service entirely. A nice comprehensive blog post can at least notify users that you have thought about them. Note that services that don’t lock users in have a natural advantage here.

A List of Final Actions

The first time a project ends unexpectedly you won’t be ready, but next time maybe you will. So this or any other list of points is not for rolling out upon requirement; consider these at all times and recognize when they occur elsewhere.

  1. Work out within your project team whether employees will directly transfer to another project, or whether the closing of a project implies that the team will be disbanded. There may also be a small transition team to handle the changes. The trick here is to reduce the amount of uncertainty.
  2. Update stakeholders, and agree on a continuation plan. This may be as little as supplying a set of fixed responses to certain queries or making sure any suggested transition will be supported. At the minimum, this is an email to a small set of people about the decisions and actions below.
  3. Public communication. Decide on a message, and stick to it. Posterity will record the “end” as coinciding with that first public communication. This is when you relay those continuation plans to users. It is sensible to listen to feedback carefully and make small alterations depending on how users respond to the news of the demise. But this whole area is a well-trodden topic by itself.
  4. Transfer, archive or destroy. The project’s software assets need to be dealt with in a systematic way. This just starts with turning everything off and responding to the consequences. Some assets should be handed off, and some deleted. Do not “open source” your code unless you already have a community who are ready and willing to maintain it. Otherwise, it is no more useful than fly tipping in GitHub, and will not ultimately help any stranded users.
  5. Complete administrative tasks. There will be AWS relationships to curtail, domains or other external agreements to close off, etc. There will be some short unpleasant emails to send. And you know how Wikipedia updates things? That could be you doing that if your project is that well exposed.
  6. Hold that project post-mortem meeting. Even if it is held informally years later, lessons learned should be formally recorded. If you can, try to do it earlier so that the enclosing organization has some memory of events, and doesn’t just go down the same path. In this day and age, you may live through the end many times in a variety of podcasts and YouTube videos.

And onto the Next Thing

This is the part where I say that after an end, there is a new beginning. I don’t know how many interviewers take note of how their candidates ended projects — but they should. Even if you are impacted personally, try to do the right thing with your craft. That is, after all, what endures.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.