Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Tech Life

Challenge Industry Norms and Redefine Your Technology Roles

Mar 26th, 2019 3:00am by
Featued image for: Challenge Industry Norms and Redefine Your Technology Roles

Sharone Revah Zitzman
Sharone Zitzman is a marketing technologist and open source community builder, who likes to work with teams building products that developers love. Having built both the DevOps Israel and Cloud Native and OSS Israel communities from the ground up, she spends her time finding the places where technology and people intersect and ensuring an excellent developer experience, backed by a culture focused on engineering excellence and quality. You can find her on Twitter or Github as @shar1z.

In the technology industry, there’s a constant focus on the “talent pool” and common gripes about “talent shortage.”  This discussion goes as far back as Velocity 2013, with one of the more popular talks on this topic by Andrew Clay Shafer (former founder of Puppet, Senior Director of Technology at Pivotal today) titled “There is No Talent Shortage,” where he actually delves into whether talent is something you are born with or can be created.

The assumption is always that talent is a zero-sum game, either you have it or you don’t. However, Shafer goes on to explain that to “cultivate” actually comes from the root word “culture,” and how being “cultured” is actually an intellectual achievement. This means it is acquired, or in other words, a potential that essentially needs to be realized, and certainly can be in the right circumstances.

(Key takeaway from this: Hire people based on their potential, not their existing resume — because some people, think those that fall under the beloved buzzword “diversity,” don’t always have equal opportunities available to them to help challenge them and build the skill set that is considered the pure definition of “talent.” Organizations need to create the environments and opportunities to evolve their people into “talents.”)

In the same vein, Avishai Ish-Shalom, a local community hero to us, in one his many excellent talks, focuses on the definition of expertise and explains that real depth of knowledge takes years to cultivate, sometimes decades (where hard numbers usually estimate around 20 years). Gaining real depth and expertise in an area is something that takes time, it’s a marathon — not a sprint. And if we take an example from the world of technology, this is because technology is a constantly moving target, it’s an ongoing evolution in the way we think about and engineer systems, in the number of users and amount of data that is constantly growing, the evolution of technology stacks and architectures to support these constant changes and the way they are consumed, and even movements that influence the way we manage our organization and culture (think DevOps, agile, lean, tribes/guilds). We have to undergo these cycles of change, in order to reach maturity. Without these trials and failures, changes and challenges, there’s no growth or learning.

Another known trend (or better yet, failure), in the engineering industry, is where an excellent engineer’s growth and maturity within an organization is many times solely through management paths, eventually, great technologists are reduced to their job titles. However, not all people are born to be managers, or even want to be managers. A good article on this topic is by Nir Cohen, from the 2016 SysAdvent series. In this post, he points out that appointing technically capable people to be managers just “on a whim” or even from inertia can even have potentially destructive effects on a company.

One of the key cultural elements that organizations need to “cultivate,” is in demonstrating how knowledge experts are valued, and specifically Individual Contributors (ICs) — at least as much as managers and team leaders. This is exceptionally important, as these expert technologists mentor the team, share from their knowledge and their hard-earned experience, make managers and team leaders better at what they do — creating the culture that will ultimately make growth and learning possible. AKA individual cultivators.

Scaling Your Senior Technologists

If you are looking to scale your organization and would like to take your senior technologists to their next phase of growth, you need to be able to identify the engineers best suited for leadership and those who are meant to be pure technologists. To do so, ask yourself two very important questions:

  1. Is management the growth path this engineer wants? This is probably the most obvious but many times the least validated aspect when building senior technical staff.
  2. Are there other growth paths in this organization for those who do not want to take on management roles? (From a remuneration and fulfillment perspective.)

Let’s Talk About #1.

If the organization has identified engineers who are looking into leadership roles it is the job of the organization to ensure that they are properly mentored and trained, and make it a priority to provide them with the tools this new management role will require. It should be as important to a company to engineer its culture as much as its systems and applications.

Management is in the large part managing people, and when there’s time left, managing technology and processes. Therefore, if characteristics of empathy, empowerment, communication, time management, or other critical skills required to be a successful manager, need to be strengthened, it is the company’s responsibility to provide forums that enrich soft skills alongside professional hard skills. Remember that not all excellent engineers will make good managers without the proper cultivation.

Many times it’s quite the opposite, and alienating other excellent engineers by promoting unfit managers is a critical mistake many organizations make. It’s also important to enable transition phases into leadership roles, enabling junior managers to start small and slowly grow teams. Each additional person on a team is a world of management unto themselves and it takes time to learn how to navigate between managing several people.

Throwing previously inexperienced managers into positions where they manage large teams, and are out of their element, will set them (and their team) up for failure. A really great book you want to read if you are looking to make a move to a management position: “Leading Snowflakes: The New Engineering Manager’s Handbook,” by Oren Ellenbogen — a book all managers should read, not just engineering managers.

Let’s talk about #2.

Many times, one of the biggest factors that influence whether a great engineer will choose to go down a management path, is ironically a financial decision or a job title or label — thanks to the world of title inflation, and not necessarily one of passion or drive to mentor and lead a team. This shouldn’t even be on the table.

The decision to take on a leadership role based on an improved salary or title is a fundamental error, and should be neutralized by compensating technology experts well and demonstrating their seniority, whether they choose to be managers or not. Neither a title nor a salary should be dependent on whether you have direct reports or not.  It should be based on the value you bring to the organization and your contribution to cultivating organizational knowledge and expertise. Netflix is famous for their mantra on culture: “your company values are measured by who gets rewarded, promoted, or let go,” and this is something we strive to apply at AppsFlyer, as well.

Empowering Managers and Individual Contributors Equally

Eventually, people stay with people, and leave people. An employee’s retention is prolonged when they feel inspired by the work they do daily, and how much they admire their leadership.

Inspiring technology leaders, whether managers or ICs, attract talent.

If we are able to build a culture that enables all types of technologists freedom to discover their ideal growth path (i.e. in the context of purpose, autonomy, mastery, if you’re not familiar with this concept, this is a good place to start), not only will we maintain our senior technologists in our organization, we will also empower our next generation of technologists to grow in a safe environment. This mindset is what will make it possible to attract the right talent who are engaged and empowered by the company’s core values.

Managers that challenge and mentor their teams, will ultimately inspire them to continue their growth path within the organization — whether as a manager or an individual contributor, understanding that they are both equally valuable and viable roles within the organization.

Note to employees: One of the most important things to keep in mind when you are debating which path to choose for yourself, like all things in life, a career path does not have to be linear. If you are an IC engineer looking to transition into a management role, and discover you’ve moved too far away from the technology and want to go back to being an IC again — this too should be a career path enabled by your organization. When we think about the amount of time we will be investing in work over the course of our career, we should choose a place that will make it a priority to cultivate our expertise and talent, and not find ourselves having wasted important formative years in stagnation.

Remember: How you define success in your organizations directly maps to the talent you will be able to retain within your organization.

Pro tip: Challenge this definition constantly, and learn from the success of others, by benchmarking yourselves against organizations you value in the industry.

Pivotal and Puppet are sponsors of The New Stack.

Feature image by truthseeker08 from Pixabay.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Velocity, The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.