What GDPR, Schrems 2 and the End of Privacy Shield Means to Your Kubernetes-Based Service
In a July 16, 2020 ruling dubbed “Schrems 2,” the European Court of Justice ruled that the EU-U.S. “Privacy Shield” agreement does not provide adequate protection of personally identifiable information under the General Data Protection Regulation (GDPR).
What’s the Problem?
In essence, the Schrems 2 ruling means that U.S.-owned cloud providers such as Google, Amazon Web Services (AWS), and Microsoft Azure cannot be used to store data about European citizens without violating the GDPR. You know, the regulation that famously can fine violators 20 million Euros or 4% of annual revenue, whichever is larger.
Merely force-locating data to an EU-based region in these clouds is not sufficient, because the problem is not geographical in nature. The problem is actually that U.S.-owned cloud vendors are subject to, for example, the CLOUD Act and National Security Letters. Both of these can be used to force cloud vendors to hand out customer data to the U.S. government, even if the servers happen to reside on foreign soil.
We have yet to find out. Will new legislation come in and fix things in a new iteration of Privacy Shield? It happened before: Schrems 2 is called that because the first time Schrems pressed the issue, the previous agreement called “Safe Harbor” was invalidated.
For the public sector in EU countries, the answer to the current situation seems crystal clear: the use of EU-based cloud services is not a question, it’s a mandate.
For private companies based in either the US or EU, but doing business with EU citizens, there is more hesitation. Perhaps they hope that these things will sort themselves out. A small but interesting study carried out by Fieldfisher found that even if all respondents relied on data processors based outside of EEA/UK, only 12% intended to reduce their use of such processors (30% were undecided and the remaining 58% did not intend to reduce). 58% did intend to ask their processors to offer protection under European Standard Contractual Clauses, which could replace previous reliance on Privacy Shield.
Let’s be crystal clear: legally speaking, riding out the storm to see what happens is a very risky position to take. Facebook, the original defendants of Schrems 2, has been ordered to cease the transfer of data about EU citizens to the US. As this would have a huge impact on their service, it is no wonder they are scrambling to fight the decision. It is such a big issue that they are threatening to cease offering their services in the EU region entirely. If “wait and see” was a valid strategy, it should be rather obvious that they would have done that instead.
However, companies that have adopted Kubernetes and other cloud native technologies can actually draw a sigh of relief. Because by building services atop of cloud vendor-neutral technology, adopting a multicloud strategy is more feasible than ever!
So while this new ruling does make us all have to either be aware of, or downright worry about, jurisdictions and the ramifications it may have for our business, the most obvious and future-proof technical solution is to start using European cloud providers for all data that pertains to our EU citizen users. At the very least, investigating your options at this time is a prudent move by your CTO, CISO, and cloud architects.
The catch is a rather big one. Surveying the field of European cloud vendors shows that Infrastructure-as-a-Service (IaaS) offerings are abundant and of high quality. No question about that — you can host your data in old Cold War bunkers or even that Church in Barcelona. However, the vast ecosystem of services like the ones offered by AWS has yet to be matched by EU-based providers (although, in fairness, the other hyperscalers also do not offer as many as AWS does).
But Seriously, Now What?
Coming from the U.S.-based hyperscalers, we are used to not only infrastructure-as-a-service, but also great platform and software services offered on top of it. Managed databases, message queues, log storage services, you name it. Will we have to build or manage these ourselves if we choose EU-based cloud providers?
Thankfully, no. In addition to mature databases such as MariaDB and PostgreSQL within the cloud native community, we also have CNCF-backed projects that offer a staggering amount of smart software to make up for the as-a-Service offerings that we have come to love and depend upon. We have everything from service meshes to storage solutions to distributed databases to streaming messaging services. Just check out the CNCF landscape!
True, these services are not all API-compatible with what (for example) AWS offers. But rather than standardizing on AWS-specific services to power our applications, we could (and should) rely on vendor-neutral CNCF-backed projects instead. Here are some examples:
|AWS service||CNCF alternative|
|Key-value store||AWS Redis||TiKV|
|Logging||CloudWatch Logs||Elasticsearch with Fluentd|
|Serverless runtime||Lambda||OpenFaaS / OpenWhisk|
|Service mesh||App mesh||Linkerd / Istio|
However, addressing self-management of these services: obviously, not all of us want to manage platform services ourselves. Taking on more system administration tasks is hardly what increases our agility and ability to deliver great features to our customers. But I believe we won’t have to manage everything ourselves. As the dust is settling after Schrems 2, there will for sure be a market for hosted cloud native services in the EU.
Right now the case may be that EU-based cloud providers do not offer as full a service offering as we are used to from the likes of AWS. But there are companies that will manage platform services for you on top of EU-based cloud infrastructure. By using them, you can maintain agility and, as a bonus, be less technologically locked into the offerings of a single U.S.-based cloud vendor. And perhaps that is good from a technological perspective anyway. Because vendor lock-in was never a desired end result, rather it was one we begrudgingly accepted in order to not have to manage platform services on our own.
Kubernetes as Enabler For GDPR Compliance
A key component in being able to efficiently orchestrate cloud native software is Kubernetes. And its own value proposition is highly relevant: that Kubernetes helps us move our workloads from one Kubernetes cluster to another. Workloads deploy just the same, and Kubernetes’ abstractions make it possible to not have to worry about implementation details. Our developers or DevOps engineers can simply request that a Deployment of Pods have a service do load balancing for it, expose it via an Ingress, and have TLS certificates managed by cert-manager. Which cloud infrastructure all this runs on is less of an issue than it may have been some five-plus years ago, in the bad old days before Kubernetes.
So is Kubernetes out of the box all you need for GDPR compliance? No, of course not, for two reasons. First, GDPR mandates that the system should be designed to secure personal data. Kubernetes is not secure, neither by default, nor by itself. For example, without the proper PodSecurityPolicies or roles in place, an attacker who gained access to a pod can create a privileged pod, take over the host and move laterally within the Kubernetes cluster. Furthermore, Kubernetes, being a laser-focused project on container orchestration, does not offer solutions for intrusion detection or vulnerability management by itself. Instead, it relies on external components for that (for this, see the CNCF project Falco).
Second, GDPR mandates data minimization. Our experience shows that personal data may leak into backups or logs, which means that retention and restoration need to be performed carefully to avoid accidentally revealing or restoring data that was ordered to be erased, via a GDPR right-to-be-forgotten request.
But does Kubernetes provide us with a key piece of the puzzle toward deploying an EU-specific subset of our services in European clouds? Most definitely. And as this need becomes ever larger, the interest for Kubernetes Cluster Federation (KubeFed) will just keep growing.
For cloud native software, the Schrems 2 ruling may have been a blessing in disguise. Because the more we have to move away from the hyperscalers, the more we will truly value getting a great storage service from Rook (for instance). And for us as cloud customers, it is a blessing also, because we may have to reconsider the comfortable default dependence on services offered by the hyperscalers. Perhaps it took a ruling in the EU Court of Justice for us all to figure that out.
So I say that the time is nigh for us all to develop and use truly cloud native software, not services native to a particular vendor. Are you with me?
To learn more about Kubernetes and other cloud native technologies, consider coming to KubeCon + CloudNativeCon North America 2020, Nov. 17-20, virtually.