Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Networking / Operations

EU Analyst: The End of the Internet Is Near

Fracturing DNS! Incompatible Internet Protocols! Filters and walled gardens galore! The internet is being torn asunder by meddling governments and greedy corporations, a EU researcher warns.
Feb 2nd, 2023 3:00am by
Featued image for: EU Analyst: The End of the Internet Is Near

The internet as we know it may no longer be a thing, warns a European Union-funded researcher. If it continues to fray, our favorite “network of networks” will just go back to being a bunch of networks again. And it will be the fault of us all.

“The idea of an open and global internet is progressively deteriorating and the internet itself is changing,” writes Konstantinos Komaitis, author of the report, “Internet Fragmentation: Why It Matters for Europe” posted Tuesday by the EU Cyber Diplomacy Initiative.

In short, the global and open nature of the internet is being impacted by larger geo-political forces, perhaps beyond everyone’s control. “Internet fragmentation must be seen both as a driver and as a reflection of an international order that is increasingly growing fragmented,” Komaitis concluded.

The vision for the internet has always been one of end-to-end communications, where one end device on the internet can exchange packets with any other end device, regardless of what network either one of them was on. And, by nature, the internet was meant to be open, with no central governing authority, allowing everyone in the world to join, for the benefit of all, rich or poor.

In practice, these technical and ideological goals may have played out inconsistently (NAT… cough), but the internet has managed to keep on keeping on for a remarkably long time for such a minimally-managed effort.

Yet, this may not always be the case, Komaitis foretells.

He notes the internet is besieged from all sides by potential fragmentation: from commercial pressures, technical changes and government interference. Komaitis highlighted a few culprits:

  • DNS: The Domain Name System is the index that holds everything together, mapping domain names to IP numbers. The Internet Corporation for Assigned Names and Numbers (ICANN) manages this work on a global scale, but there’s nothing to stop another party from setting up an alternative root server. A few have tried: The International Telecommunications Union’s Digital Object Architecture (DOA) as well as Europe’s Network and Information Systems both set out to challenge the global DNS.
  • Stalled IPv4 to IPv6 translation: The effort to move the internet from the limited IPv4 addressing scheme to the much larger IPv6 address pool has been going on for well over two decades now, with only limited success thus far. “Even though there is a steady increase in the adoption of IPv6 addresses, there is still a long way to go,” Komaitis for writes. He notes that “Just 32 economies” have IPv6 adoption rates above the global average of 30%. Without full IPv6 adoption, he argues, the internet will continue to e fragmented, with no assurance of end-to-end connectivity across those using one version or the other.
  • Internet content blocking: Governments have take an increasing interest in curating the internet for its own citizens, using tools such as DNS filtering, IP blocking, distributed denial of service (DDoS) attacks and search result removals. The most prominent example is China, which runs “a sophisticated filtering system that can control which content users are exposed to,” Komaitis wrote.
  • Breakdown of peering agreements: The internet is the result of a set of bilateral peering agreements, which allow very small internet service providers to share the address space with global conglomerates. Increasingly, however, the large telcos are prioritizing their own traffic at the expense of smaller players. The European Union is looking at ways to restructure these agreements,  though South Korea tried this, and the results ended up just confusing and burdening the market, Komaitis wrote.

Other mitigating factors that Komaitis discussed include wall gardens, data localization practices (i.e. GDPR) and ongoing governmental interest/interference in open standards bodies.

What does all this mean for the European Union, which funded this overview? The Union has already pledged to offer everyone online access by 2030, as well as to thwart any commercial of government attempts to throttle or prioritize internet traffic. It has also made a pledge, with the U.S. and other governments to ensure the Internet ” is “open, global and interoperable.”

So the EU needs to make the choice of whether or not to back its pledges.

“Moving forward, Europe must make a choice as to what sort of internet it wants: an open, global, interoperable internet or one that is fragmented and limited in choice?” Komaitis wrote.

The EU Cyber Diplomacy Initiative is “an EU-funded project focused on policy support, research, outreach and capacity building in the field of cyber diplomacy,” according to the project’s site.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.