Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Edge Computing / Software Development

As Blockchain Hype Fades, Developers Give Ethereum a Serious Look

Businesses are finding new uses for blockchain, including supply chain management and smart contracts. For developers, this means job opportunities.
Jun 22nd, 2021 9:44am by
Featued image for: As Blockchain Hype Fades, Developers Give Ethereum a Serious Look

Photo by Chuttersnap on Unsplash

A couple of years ago, proponents touted the blockchain, the underlying technology behind Bitcoin and other cryptocurrencies, as a solution to nearly every business problem. Investment money flooded in. But many of those early proofs of concept fizzled out and the hype died down.

Now, however, the noise about blockchain’s superpowers is fading, according to analysts like Gartner. With the hype settling down, companies are starting to look at blockchain with a more realistic, practical-minded approach — and starting to zero in on those applications where the technology’s advantages make sense.

“2021 is the year enterprise blockchain begins its long climb out of the ‘trough of disillusionment,'” wrote Avivah Litan, a Gartner analyst, in a report released in January.

Blockchain, businesses are finding, has a number of practical uses, including supply chain management, financial instrument auctions, and smart contracts, which are self-executing contracts written in code. For developers, this means opportunity in the form of growing demand for people with rare skills.

Among the signs that industry investment in blockchain is slowly starting to pay off:

  • Although only 5% of blockchain projects moved into production in 2019, that percentage nearly tripled in 2020, to 14%, according to Gartner.
  • This year, Gartner expects production use cases to continue growing by double-digit percentages, wrote Litan, in the January Gartner report.
  • Companies are expected to spend nearly $7 billion on blockchain technology globally this year, up nearly 50% from last year, according to an April report by IDC, a global market intelligence firm.
  • Business investment in blockchain technology will rise to nearly $19 billion by 2024, IDC projects. If that occurs, it would mean a five-year compound annual growth rate of 48%, making it one of the fastest-growing areas of technology.

Use Cases for Business

The chaos generated by COVID-19 is one reason why blockchain is getting a second look from enterprises.

“The pandemic highlighted the need for more resilient, more transparent supply chains, health-care delivery, financial services, and so much more, and enterprises around the world have been investing in blockchain to provide that resiliency and transparency,” said James Wester, IDC’s research director for worldwide blockchain strategies, in the company’s report.

There’s now real interest and investment by corporations, financial institutions and governments, he said. “This investment will have major implications in a very short time on everything from retail to financial services to capital markets.”

The leading use case for the blockchain this year and in the near future is cross-border payments and settlements, Wester said. The next is using blockchain to track origin and authenticity of products through the value chain.

But building with blockchain technology isn’t easy for many enterprises, which may present an obstacle to more rapid adoption. Setting up a custom-built blockchain solution requires technical overhead and expertise, said John Carey, managing director in the technology practice at AArete, a global technology consulting firm.

“Quite frankly, databases are getting cheaper and solutions around security are evolving and becoming more trustworthy,” he said. “And you’ve got billions invested in existing systems that you can’t easily transform overnight.”

AArete has built projects using the Ethereum blockchain technology, he added. “We’re always looking at it. But the challenge always with things like blockchain is that it’s a tool looking for a solution, and that’s where it’s going to be, for a while, anyway.”

Career Prospects for Developers

With investment in blockchain projects on the rise, there’s a benefit for developers in learning how to program in Ethereum and related blockchain technology said AArete’s Carey: “Because there’s a shortage of capability.”

Of course, tech skills in short supply and high demand tend to be handsomely compensated:

  • As of May, the average annual salary of a U.S.-based  blockchain developer was $107,000, according to Glassdoor.
  • Meanwhile, ZipRecruiter puts the average salary at $154,550.
  • Hired reports a range of average pay by market, from at $169,653 in the San Francisco Bay Area, down $133,750 in Atlanta.
  • Having blockchain skills brings an average pay premium of 18%, according to a new survey by the analyst firm Foote Partners.

Banking is the most prominent sector to employ developers with blockchain skills. According to IDC, the banking industry accounts for nearly 30% of all global blockchain spending this year. The next most promising industry is manufacturing, followed by professional services, retail and insurance.

Blockchain development requires a lot of specialized skills — everything from user interface development to back end data structures to encryption, which are also applicable to the non-blockchain project.

Other key skills for blockchain development include experience working with open source projects, understanding of peer-to-peer networks, and proficiency in programming languages such as C++ and Javascript, according to the Blockchain Council.

But there are some specialized skills as well. One of the most valued skills, according to the tech jobs website Hired, is that of Ethereum smart contract development. This requires knowing how to code smart contracts in Ethereum’s native programming language, Solidity.

Solidity, the prerequisite language for being an Ethereum developer, “has a very welcoming community and it is like learning any programming language, you would just need to invest and dedicate some time,” said Victor David, CEO at Epik, a digital content company.

People can also learn it on their own, he added. “I know some really elite blockchain developers who are self-taught.”

What Is Ethereum?

Ethereum is the starting point of all innovation in the blockchain space, said David.

“We started on Ethereum in 2018 and will always use it, but we are blockchain agnostic, which means we also use other blockchains,” he said. “Ethereum is not perfect, but most tech is not perfect — and it is by far the most dominant and will continue to be the most dominant blockchain for a long while.”

Ethereum is the blockchain used to power the Ether cryptocurrency, which is the second-largest cryptocurrency today by market capitalization, according to CoinMarketCap.

The Ethereum network became the first to settle $1 trillion in transactions in one calendar year in 2020, according to CBInsights.

But Ethereum is used for a lot more than just virtual money and NFTs, the non-fungible tokens that are suddenly all the rage in collector circles. It’s the foundational technology for smart contracts, for example, and for tracing the flow of goods and services through a supply chain.

Companies also use the technology to set up their own, private blockchains for internal use, or to share with partners.

Learning Ethereum on the Job

The pool of Ethereum developers is so small that some businesses are helping developers learn on the job.

That’s what’s happening at UST, a global consulting company focusing on digital transformation. UST has more than 26,000 employees in more than 25 countries, serving mostly Fortune 500 and Global 2000 clients.

The company has built many projects in Ethereum over the years, including traceability solutions, certification platforms, loyalty programs, and smart contract auctions, said Daniel Field, global head of blockchain at UST.

UST started out with two blockchain developers in 2016, and the team has since grown to 56 people. Usually, the company hires people fresh out of college, he said, and trains them. “There aren’t a lot of people with experience and training,” Field said.

And when it does hire people with experience developing in blockchain, UST isn’t picky about what specific kinds of experience and expertise they have.

“Blockchain skills are quite transferable from one blockchain to another,” he said. “So if someone has experience in Solidity, we’ll know that you know what you’re doing and can pick up other languages.”

Field sees a continuing need for Ethereum skills but recommends that developers combine this with experience using other blockchains as well since Ethereum has some drawbacks. For example, it has some technical weaknesses that are addressed by some of the newer blockchain technologies.

In addition, he said, because blockchain technology is evolving so quickly, there’s no formalized set of standards.

Building for Transparency

One recent blockchain project that Field’s team worked on was a mortgage auction system for a bank in Spain. The episode shows both the promise of blockchain and the current limitations in a business setting.

After the global financial crisis, there were a lot of bad mortgages and the government conducted a restructuring program to incentivize the purchase of those bad debts. The bank that Field’s team worked with wound up with a portfolio of mortgages; some of the debts were lost causes, but others were still being paid off by the property owners.

The bank wanted to sell these mortgages to institutional investors, but those investors were wary of buying in bulk — maybe, they thought, the bank would only sell off the bad ones.

So the bank decided to build a blockchain-based auction process that would include all the relevant information about the performance of each particular mortgage, including the collateral and the payment history.

“The institutional investors wanted to have some transparency,” Field said.

Since the blockchain is an immutable ledger, the mortgages could then be resold again, and again, with the new buyers also having confidence in what they were buying. And with Ethereum’s smart-contract capability, he said, the auctions could be automated.

“It’s very open and transparent, with no back-handed favors,” Field said. “You were reducing friction in the market and the institutional investors would be more likely to acquire the debt.”

The bank decided to go down this path in 2017, and the proof of concept was built in 2019, with contracts valued in the tens of millions of Euros processed through the system.

But, after the conclusion of the pilot phase, this particular Ethereum project went the way of many others —it involved too much work on behalf of the bank and its IT teams, and existing legal structures were already in place to handle both the transparency and the accountability investors demanded.

For example, each of the major participating institutional investors had to set up their own node on this blockchain, and the participating institutions needed to coordinate with the bank about how the nodes were set up and working, how to keep all the software updated and aligned, agree on how the smart auctions would be conducted and recording, and match all the data to the blockchain records.

“They came to the conclusion that there were other ways it could be done,” he said.

Plus, since it was a small group of investors, there wasn’t a need for the kind of trustless system — in which all participating parties can reach a consensus about canonical truth — that the blockchain allows.

Pilot Projects

One project that did move beyond the pilot stage was a blockchain set up for shipping and customs data, tracking shipping manifests and other documents and making it easier to change things on the fly.

It replaced a cumbersome paper-based system, Field said. “If there’s a storm and the ship docks in a different port, you have to change all the paperwork,” he said. “You end up with ridiculous situations where you have lawyers flying around the world with documents to have them signed.”

The blockchain provides a chain of custody for the documents, he said.

There were other projects that UST worked on that were potentially good ideas, Field said, but they weren’t aligned with a business case.

For example, one project involved using a blockchain to validate people’s college degrees and certificates. “Nobody wanted to pay for it,” he said. Universities didn’t care about building it, and the people who did care — like employers and employment agencies — didn’t have the leverage to get the system going.

These types of failed projects have made a lot of people skeptical about the blockchain, Field said.

“But it’s actually made my job easier,” he said, because clients are less likely to have over-inflated ideas of what blockchain can do. “I don’t have to sort people’s expectations as much.”

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.