How to Use InfluxDB with Its Python Client on Kubernetes
InfluxData sponsored this post.
In this tutorial, we will discuss InfluxDB and its Python client. We will deploy InfluxDB inside a Kubernetes cluster and then use the InfluxDB Python client to send data to InfluxDB.
What you will learn?
- How to deploy InfluxDB to a Kubernetes cluster (quickstart way).
- How to use the InfluxDB Python client.
Prerequisites
- A Kubernetes cluster you control. We’ll take advantage of Civo’s super-fast managed K3s service to experiment with this quickly. If you don’t yet have an account, sign up for the beta now to take advantage of quick deployment times and $70 free credit per month. Alternatively, you could also use any other Kubernetes cluster.
- Install and set up kubectl, and have the kubeconfig file for your cluster downloaded.
Make sure you can connect to your Kubernetes cluster by running:
|
1 2 3 4 5 |
kubectl get nodes NAME STATUS ROLES AGE VERSION kube-master-18e1 Ready master 8h v1.18.6+k3s1 kube-node-4e70 Ready <none> 8h v1.18.6+k3s1 kube-node-d58a Ready <none> 8h v1.18.6+k3s1 |
You should see the names of the nodes in your cluster displayed.
Getting Up and Running with InfluxDB
Clone the repository: https://github.com/saiyam1814/pyconf.git
|
1 2 3 4 5 6 7 |
git clone https://github.com/saiyam1814/pyconf.git cd pyconf/deploy # Apply the influx.yaml file kubectl create -f influx.yaml namespace/influxdb created statefulset.apps/influxdb created service/influxdb created |
The above script creates the Namespace, StatefulSet and service for InfluxDB version 2.0.1.
Now we will create the ingress, and for that, we will modify the ing.yaml file and input the DNS name of the created cluster. You can get the DNS name from the dashboard.
In the host section of ing.yaml, point to influx.{DNS NAME}.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: influxdb namespace: influxdb spec: rules: - host: influxdb.360dfac7-2adc-4e60-bdcb-0b7a283ac3c8.k8s.civo.com http: paths: - backend: serviceName: influxdb servicePort: 8086 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
kubectl create -f ing.yml ingress.extensions/influxdb created kubectl get all -n influxdb NAME READY STATUS RESTARTS AGE pod/influxdb-0 1/1 Running 0 6m45s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/influxdb ClusterIP 192.168.145.255 <none> 8086/TCP 6m45s NAME READY AGE statefulset.apps/influxdb 1/1 6m46s kubectl get ing -n influxdb 'NAME CLASS HOSTS ADDRESS PORTS AGE influxdb <none> influxdb.360dfac7-2adc-4e60-bdcb-0b7a283ac3c8.k8s.civo.com 91.211.153.65 80 55s |
Navigate to the HOST ADDRESS:
Now, in the Advanced section from the Token, select and copy the generated token — as that would be required for connecting to InfluxDB via the Python client.
Next Steps
Create Kubernetes Secret from the Token
|
1 2 |
kubectl create secret generic influx --from-literal=token="qWpu90WO31r02miedaP7-BXG9hmrtfBCPgncqu3PsU-PzZZ3jrg7eC9RE2yZzLurhNlk8tr_maKYE3zpk1GJ2A==" secret/influx created |
Create Config Map for the Ingress HOST Address
|
1 2 |
kubectl create configmap host --from-literal=host="influxdb.360dfac7-2adc-4e60-bdcb-0b7a283ac3c8.k8s.civo.com" configmap/host created |
Create the Daemonset from the Deploy Folder
|
1 2 3 4 5 6 7 |
kubectl create -f ds.yaml daemonset.apps/pyconf-demo created kubectl get pods NAME READY STATUS RESTARTS AGE pyconf-demo-dq5c9 1/1 Running 0 40m pyconf-demo-lbxww 1/1 Running 0 40m pyconf-demo-q65ps 1/1 Running 0 86s |
Viewing from InfluxDB UI
Understanding the Connection and Writing Points to InfluxDB via Its Python Client
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
from influxdb_client import InfluxDBClient, Point, WritePrecision from datetime import datetime from influxdb_client.client.write_api import SYNCHRONOUS import schedule import time import socket import os def influx(): print("starting") org = "demo" bucket = "demo" client = InfluxDBClient(url="{}".format(os.environ.get('host')), token="{}".format(os.environ.get('token'))) meminfo = dict((i.split()[0].rstrip(':'),int(i.split()[1])) for i in open('/proc/meminfo').readlines()) freemem = meminfo['MemFree'] / 1024 /1024 write_api = client.write_api(write_options=SYNCHRONOUS) point = Point("free_mem").tag("host", socket.gethostname()).field("free_memory_Gb", freemem ).time(datetime.utcnow(), WritePrecision.NS) write_api.write(bucket, org, point) schedule.every(5).seconds.do(influx) while 1: schedule.run_pending() time.sleep(1) |
- Import influxdb-client.
- Org: Corresponds to the org where data has to be pushed.
- Bucket: Corresponds to the bucket where data has to be pushed.
- Client: The connection created by proving the host and the token.
- Freemem: To get free memory of the host in Gb.
- client.write_api for writing to InfluxDB.
- Point: The actual point that will be creating measurement free_mem, tag host and field free_mem_Gb.
- write_api.write(bucket,org,point)for the insertion to the Db.
- Last section makes it run every 5 seconds.
Use Cases
You might have certain scripts that would capture some data, or do some level of processing and then create the data, which can then be sent via the Python client to InfluxDB (for this particular use case). There are a lot of client libraries that are supported and can be used to send the data.
Summary
We have seen how to deploy InfluxDB on an existing K3s Kubernetes cluster. Use the Python client to write data to InfluxDB, and then view the data from InfluxDB UI.
Feature image via Pixabay.