The Network Impact of the Global COVID-19 Pandemic
With so many countries in lockdown and so many people working (and learning) from home, online usage has risen significantly but so far, the internet is holding up well.
Internet traffic is generally 25% to 30% higher than usual, and what we do online is also changing. Internet usage often increases in a typical month; for Akamai that’s usually 3% growth, in the last month it’s been 30%. In March 2019 their peak traffic was 82Tbps; this March it was 167Tbps and the sustained daily traffic rate is higher than last year’s peak for March. Internet exchanges in Amsterdam, Frankfurt and London saw 10-20% increases in traffic around March 9. The exchange in Milan had a 40% increase the day Italy was quarantined.
Disturbingly, attacks are up, too: Akamai has seen a 20% increase in daily DDoS attacks between February and March and Nokia measured a 40% increase. Internet traffic growth that Cloudflare tracks varies by city; it’s only up 11% in Berlin and 22% in London between early January and late March (and 17% up for the UK as whole), but it’s grown by 40% in New York and 48% in San Francisco and Silicon Valley. In fact, across the globe Cloudflare Chief Technology Officer John Graham-Cumming says the increase is anywhere from 20% to 70%.
You can also see the change in where people are connecting from; usage is up in residential areas but visibly down in downtown San Francisco, downtown San Jose and especially the Cupertino and Mountain View neighborhoods where Apple and Google have their campuses. Usage at universities is down to what Graham-Cumming says you’d see “on a Sunday afternoon during a long vacation during the summer.” Overall, U.S. internet usage is up from January, and still rising.
What We’re Doing Online
Usage of the Windows Virtual Desktop service has tripled, Azure usage increased by several million cores in March and meetings in Teams are going up at an exponential rate: from 560 million minutes on March 12 and 900 million on March 16, to 2.7 billion on March 31. Google Meet has 2 billion minutes of usage a day and Skype has 40 million daily users (up by 40% from February to March) with Skype to Skype calling minutes up 220%.
Perhaps looking for a more personal connection, twice as many people are turning on video in Teams now and video calls went up by over 1,000% in March. Azure Stream meetings are getting larger: Microsoft had to raise the limit from 10,000 to 100,000 users in one meeting. In the U.S. alone. All the video conferencing is changing internet traffic patterns. Upstream traffic is up by 30% in the U.S. in March, according to Nokia; in some parts of Europe the average per consumer upstream has gone from a peak of 1.1Mbps to 1.7Mbps since the beginning of March (and it stays at those levels for long periods of the day.)
Gaming traffic is up, but according to Nokia it’s eclipsed by people just browsing the web; web traffic is generating more traffic than anything except streaming video. And Akamai’s traffic suggests that not only are existing video streaming subscribers watching significantly more but that new subscriptions are rising, too.
Verizon saw a 20% increase in web traffic in a week as U.S. users started staying home in March and it’s still 27% higher than usual; VPN usage is up 49% with employees working from home, video is up by 36% and online gaming by 115%. ThousandEyes monitors both cloud and ISP connectivity and says VPN usage is up five-fold globally.
Vodafone saw data usage rise by 30% in the UK even before the government ordered a lockdown, and by 50% in some EU countries. Globally we downloaded 30% more mobile games in March compared to the previous year, according to App Annie, especially multiplayer games like Words With Friends, but people are also downloading more shopping apps, more health and fitness apps, more food delivery apps, more apps for managing your money — and more video conferencing apps.
We’re visiting different types of web sites and searching for different things, Graham-Cumming told us: entertainment, jobs, fitness and financial planning are up, sports, travel and weddings are down. “At the beginning of the lockdowns, we see a big increase around parenting resources, educational resources, particularly for younger kids, as parents figure how do I deal with my kids, and then they figure it out and it goes down.”
“Things that are down dramatically are anything to do with pro sports: basketball, baseball, soccer, even football in the US — dramatic drops in all those because there are no games happening. Other things that have dropped quite a lot are hunting for new apartments, buying and selling cars, weddings.” Interest in information about pregnancy is up, however.
That matches Bank of America figures on changes in spending: grocery sales and online electronics are up (laptop PCs are in high demand but supply has been limited), travel, clothing and restaurant spending is down.
E-commerce traffic is naturally high, said Akamai president Rick McConnell. “I talked to one of our larger customers the other day, who said that while the stores are closed their traffic was running as though it were cyber Monday here in the United States. The traffic is running very, very hot, very high, because it’s the only way their customers are able to buy from them right now.”
As Akamai CEO Tom Leighton puts it, “None of these use cases are new but they’re operating at a much larger scale and if they weren’t critical before, they are now.”
Mobile phone calls are up too: the 800 million daily calls on Verizon are twice what the network sees on Mother’s Day (its busiest Sunday of the year), mainly to call into conference numbers. Smartphone use is up generally, but so is laptop and desktop use in all age groups: 72% of millennials in a recent Global Web Index study said they were using their phones more, but 42% said they were using their laptops more and 31% their desktop PCs.
And because people are at home, more mobile devices are connecting over Wi-Fi, Graham-Cumming notes.
All the World, All the Time
It’s not just the amount of time we’re spending on the internet and the different services that we’re using that have changed; the time of day is different, too. According to Akamai Executive Vice President Adam Karon the previous peak time was between 5 p.m. and 11 p.m., and now it’s from 8 a.m. to 11 p.m. globally. “Pretty much the online downtime is when you’re sleeping.”
Cloudflare sees a distinct morning peak in traffic in areas under lockdown as well as the usual evening peak. Vodafone peak mobile usage in the UK used to be 6 p.m. to 8 p.m., now it’s noon til 9 p.m. Also in the U.K., BT saw its broadband daytime data traffic go up by 60%, although that’s still only half the average evening peak usage. U.S. cable subscribers are online more during working hours: usage in March was up 40% between 9 a.m. and 5 p.m. in the week after the CDC declared COVID-19 to be a pandemic, according to OpenVault.
Internet use is showing a clear pattern that corresponds to news like that as well as to lockdown orders, and that seems to be consistent between different countries.
Akamai compared traffic in the first four countries to lockdown (China, Italy, Korea and Japan) to global Internet traffic growth. At the beginning of the year, perhaps because of the Chinese new year celebrations, traffic growth was slightly below the rest of the world’s but when lockdowns started, traffic growth was 30% faster than other countries — until other countries also went into lockdown and their traffic growth caught up. In the first two weeks of February, weekly downloads of apps from the Apple App Store in China were up 40% from the previous year according to App Annie.
The same pattern shows up in other countries. When the Dutch government announced that non-essential businesses would close and when the French president announced lockdown, Graham-Cumming said, internet traffic dipped while people watched and listened to the news.
Immediately after lockdown, the weekday peak internet traffic Nokia measures with its Deepfield service goes up by 45-50% and weekend evening peak traffic by 20-40% but then stabilizes at the 20-30% level. That’s the same thing Akamai sees, said Akamai’s Amanda Goedde: There were repeated peaks for the different announcements of progressive closures and then lockdown in France and New York, with dips in the traffic right after.
“People take time to watch the news and the videos and adapt to their new work from home environment. There’s always a spike in traffic as isolation protocols are enacted, and then once that’s enacted and people get themselves ready for a new normal, and they [settle] into what they’re needing for their home office, their home-schooling, and home entertainment, the increase levels out as protocols start to take effect and people get used to being at home all the time,” Goedde said.
The Internet Is Holding Up
The internet is designed to scale, but usually, there are peaks. Now Goedde said, “it’s like a major gaming event or a bank holiday every day.”
There’s a lot of work going on to keep things stable, but the internet is holding up says Graham-Cumming. “The internet is a network of networks and so the overall internet is great. We’ve seen very big increases in traffic around the world and it’s hard to think of another network, like the electricity or water networks, that could scale to that increase in demand so well.”
Akamai’s own network performance is holding up for both network and mobile traffic.
Partly that’s because telcos, CDNs, cloud and other network providers are ramping up infrastructure; they’re classed as essential services so installations and maintenance can still go ahead. “People are discouraged from doing non-essential work but we are having no problems getting into any of our locations worldwide,” Graham-Cumming told us.
Microsoft, for example, expects to have increased Teams capacity by 60 times over seven weeks, starting from the end of February.
There’s also been some proactive reduction in loads. To prioritize educational and health usage, Microsoft has put some limits in place for free trials and tweaked some cloud service features (thumbnails for files stored in OneDrive will still show up in File Explorer for photos but not for videos, for example, and game downloads are getting delivered overnight).
Microsoft is also putting early test releases of optional non-security updates to Windows on hold from May, but that’s to let IT teams prioritize security updates rather than to save bandwidth.
Streaming video services like Amazon, Disney+, Netflix and YouTube have been reducing bitrates to limit the bandwidth they use. But although bitrates are still down from their usual quality, Nokia notes that they’ve gone up by 11% since Netflix first reduced stream quality as networks continue to cope well with the extra demand.
In fact, says Graham-Cumming, that reduction has mainly been in Europe; “In the U.S. we’re not seeing this, especially not on the large networks like Comcast. In the UK, the major networks feel fine about their architecture.” Again, the congestion isn’t in the internet backbone or the large network providers but, potentially, in the last mile into the home where some ISPs may not have enough capacity.
To avoid problems, he says there’s been a lot of industry cooperation between the networks and the people publishing videos and big software downloads like games. “What the networks are really trying to avoid is multiple things happening at the same time. They already see a lot of traffic when popular games come out with a new version, so they’re saying ‘let’s not do two of those at once: let’s not stream all those Netflix videos and get two games updates and a new operating system all coming out at eight o’clock in the morning’.”
While public clouds have had low numbers of outages, ISP outages have been going up over the last eight weeks, ThousandEyes director of product marketing Archana Kesavan told us — but that now seems to be leveling off.
In fact, she suggests, some of the outages were so short they may actually have been ISPs preparing for traffic to go up. “They were probably working through some traffic engineering optimization on the back end, that maybe resulted in packets being dropped or the rate dropping. It’s not necessarily because there is a heavy congestion on their network or they’re overwhelmed and their infrastructure can’t handle the surge of traffic.”
Communications and video conferencing services have had some outages, mainly in the U.S.; “they are definitely more strained by the amount of traffic. The internet backbone has the capacity to handle all this traffic, but a particular app may not have the capacity to handle a surge of usage.”
Internet download speeds have gone down in some areas according to Fastly: down by 34% in Italy over a month and by 20% in France on the day of the national lockdown. Internet infrastructure in Spain, New York and California has shown much less impact: 5.5% and 1.2% respectively. Again, that’s down to the quality of last-mile infrastructure into the home.
And if you’re using mobile broadband exclusively (something that the mobile carriers have been pushing in recent years), you may find that network congestion and low bandwidth can lock you out of some streaming services like On24 notes CCS Insight vice president for enterprise research Nicholas McQuire. Broadband and LTE still aren’t universally available, even in the US. He’s keen to see more providers look at how well online meeting and video tools work with suboptimal bandwidth. That’s going to be important not just during the pandemic, but for giving tools to first-line workers down the line. “We need to think about not just reliability in terms of the application on the cloud, but also in terms of connectivity.”
The New Normal?
Looking to the future, it’s hard to predict whether internet traffic will go back down when lockdowns and shelter-in-place orders lift. Microsoft and Akamai both think at least some of the new behavior will stick.
“I think we’re going to go through a step function here, where normal traffic across the globe grows anywhere between 30% and 40% per year across the board — and this year it’s going to grow significantly more than that — and that step function will then stay in place and grow again at its normal organic pace,” McConnell suggested.
Early figures from China suggest that may be true. Even with the lockdown lifting there, Teams usage is still growing in China with twice as many new Teams users each day at the end of March as there were in January. Video usage might fall back down as social gatherings go back to being in person and business users don’t feel the same need for human contact that has them turning on, but online collaboration and remote working will continue.
As a popular joke on Twitter suggested, COVID-19 is driving digital transformation far faster than many CEOs and CTOs had been able to manage, Kesevan agrees.
“COVID-19 has certainly accelerated the digital transformation efforts of many enterprises; it’s been a forcing function for them to go ahead and do it, rather than proceed gradually. Something like SD-WAN rollout might not be top of mind but anything around employees or remote workers or staff usage, that’s definitely picking up.”
It’s not just VPN usage that’s up; companies are also adopting cloud access gateway tools from vendors like Akamai, Cloudflare and Microsoft that give employees secure remote access to applications that usually live behind the firewall. When you’re setting up remote access for 30,000 employees to dozens of applications over the weekend, you have to use cloud tools rather than shipping gateway appliances to data centers.
Companies who can switch to fully remote work are driving a 25% increase in traffic on Akamai’s app gateway platform. But organizations that are limited by their ability to scale their VPN infrastructure are having to get creative; some are splitting employees into two groups and giving them VPN access in alternate weeks, Akamai global vice president Lelah Manz said (which she politely called “suboptimal”).
But working from home in a time of pandemic isn’t the same as regular remote work. McQuire says that after the initial adjustments for working at home have been made, companies will need to start paying more attention to the employee experience at home.
While Microsoft Research has found that developers at the company were “more productive” in March 2020 than in the same month in 2018 or 2019, with company-wide pull requests staying at the same level as the organization switched to working from home, engineers in the Office team were also working longer days. Some of that is time saved on commuting; some of it might be forgetting to stop for lunch without the familiar office routine to remind you, and working under stress without breaks isn’t sustainable.
Even for a company that has implemented “zero-trust” access and usually treats its offices as if they were WiFi hotspots and every employee connection as if it was remote the way Akamai does, there are culture and process issues to get over. Employees can’t call the help desk and get IT to come down and help them with a problem, new employees can’t come into the office to get onboarded and collect a laptop. Retailers will likely keep their new e-commerce sites and businesses will keep their VPN alternatives, but children and students will go back to school and some people will go back to work.
But as Graham-Cumming points out, some employees may ask to continue working remotely. “If they like the flexibility to work from home they may say ‘guess what we can do it when it was a crisis, we can probably do it now’.”