Tencent: Serving a Billion Users with OpenStack
OpenStack Summit Sydney brought us many unique use cases that continue to prove that OpenStack is mature, stable and continues to grow.
Tencent, for example, one of China’s three leading tech companies alongside Baidu and Alibaba (together they are known as BAT), serves more than one billion active users with its private OpenStack cloud. It’s an incredible use case that made Tencent the winner of this year’s OpenStack Superuser Award. The company was also recently accepted as a Gold member of the OpenStack Foundation.
Tencent Holdings Limited is the parent company of QQ and WeChat. Along with these two flagship products, Tencent also serves governments with e-governance systems, manages various public services including public transportation, taxes, social insurance, health care, and more. Tencent also offers one of the biggest online gaming platforms. They have music, movies, a web browser, e-commerce and more.
We met and interviewed Bowyer Liu, the chief architect at Tencent, to understand how they are using OpenStack.
- Bowyer Liu, chief architect at Tencent.
A Brief History of Tencent
In 1998, Tencent was founded by Ma Huateng, Zhang Zhidong, Xu Chenye, Chen Yidan and Zeng Liqing. Their first product was QQ instant messaging service that was launched in 1999. Over time QQ added more services including online games, music, shopping, microblogging, movies, and WeChat, the voice chat platform.
In 2011, Tencent announced Weixin, also known as WeChat. As the company was expanding its internet-based services, it needed a cloud strategy. With the launch of WeChat, Tencent also announced its Open Platform Strategy. To keep up with arch-rivals Alibaba and Baidu, Tencent set up a Cloud Computing Center and e-Commerce Foundation. After working on its cloud internally for two years, Tencent announced the public availability of Tencent Cloud in 2013.
Transitioning to OpenStack
Tencent Cloud, powered by their own infrastructure called TStack, was still running on the traditional infrastructure with more than 6,000 servers, 80,000 desktop terminals, 110,000 mobile terminals and 1,000 internal applications. The company had more than six data centers and more than 70 workplaces around the globe.
Tencent was facing the same scaling, optimizing and managing challenges that one would expect from traditional cloud infrastructure. TStack was not a cloud management platform which could support heterogeneous virtualization. “Many resources couldn’t be managed and fully utilized, including heterogeneous virtual machines, thousands of physical servers and many third-party storage devices,” the TStack team said in an interview with Superuser.
OpenStack was a relatively new project when Tencent launched its cloud platform in 2011; it was merely a few months old. But the skyrocketing growth and adoption of OpenStack caught the attention of Tencent, just like everyone else in the industry. To overcome the challenges of TStack, Tencent placed its bet on OpenStack as the infrastructure layer for TStack in 2014.
Liu said, “We chose OpenStack to build our own public cloud because it’s a very strong platform and it’s open enough to help us build the private cloud we needed.”
TStack has more than 80,000 cores and over 360 TB of memory that’s managed by more than 6,000 nodes. Out of these, there are 2,000 bare-metal nodes managed by Ironic. They are running more than 12,000 virtual machines (VMs), covering 17 clusters, spread across seven data centers in four regions. TStack is managing over 300 services that include QQ and WeChat.
According to the TStack team, TStack has been deployed to 14 clusters in seven data centers in four regions of Shanghai, Chengdu, Tianjin and Shanwei. It manages 10,000+ OSes, hosts internal IT systems, functional department business systems and most of the development and testing systems within Tencent. TStack has been running in production for four years with over 99.99 percent availability.
The TStack Stack
TStack is currently based on the Kilo release of OpenStack. Tencent is evaluating an upgrade to the latest release. In addition to using core OpenStack components like Keystone, Nova, Neutron, Cinder, Glance, Ironic, Heat, Swift, Manila and Horizon, TStack is also using other open source projects like CoreOS, Docker, Kubernetes, KVM, HAProxy, Keepalived, Clear Container, Rabbitmq, MariaDB, Nginx, Ansible, Jenkins, Git, ELK, Zabbix, Grafana, InfluxDB, Tempest and Rally.
With OpenStack, TStack transformed into an extensible and highly available cloud computing management platform. It was able to offer service interfaces for managing computing resources, storage, network, image, authentication and measurement and it’s compatible with heterogeneous virtualization, servers, storage devices and network devices. It’s also suitable for distributed computing and storage.
Not only did it allow better management of resources, moving to OpenStack saved Tencent 30 percent in server costs and over 55 percent in operations and management costs.
At Your Service
While WeChat and QQ are the flagship services powered by OpenStack, TStack also provides email services to more than 40,000 employees. It uses TStack to offer a CI/CD environment to its 20,000+ developers.
OpenStack-powered TStack runs more than 300 internal IT services, including OA authentication, WeChat gateway, RTX, mail system, video surveillance, internal security, function management and ERP. TStack also manages development and testing services of various products in Tencent, e.g. WeChat, QQ, browser, games and more.
There was more to come.
In 2015, the visionary co-founder and CEO of Tencent, Ma Huateng announced the Internet Plus strategy that was also included in the Government Work Report by the Chinese Prime Minister Li Keqiang.
The idea behind Internet Plus was to increase the usage of the internet and other modern IT technologies — such as cloud computing, big data/machine learning and IoT — in more traditional Chinese industries.
OpenStack was at the heart of the Internet Plus strategy.
Tencent is also offering OpenStack-based services to external organizations. It’s now being used by many provinces and cities in China. The recent BRICS Summit was powered by TStack. And Tencent is now using TStack to offer NVIDIA’s deep learning platform for AI development to developers.
There is only one word to describe what Tencent is doing with OpenStack: incredible.
OpenStack is a sponsor of The New Stack.
