Join us as we briefly discuss Prometheus and Grafana and how these fantastic open-source software can help elevate your DevOps processes. Additionally, we’ll cover a section exclusively for the techies - installing Prometheus & Grafana on a GKE Cluster.

Prometheus: Brief History & Overview

In its inception, the idea behind Prometheus was to monitor dynamic environments like Kubernetes or docker swarm. Prometheus aims to solve the ever-increasing data complexity of DevOps.

Typically, when you’re dealing with multiple servers and 100s of disparate processes, manual monitoring is cumbersome. Prometheus offers a viable solution: Automation.

With Prometheus, SREs and DevOps engineers can configure automatic monitoring of various critical alerts like backend status, auth-service status, crash errors, etc. Moreover, Prometheus also helps with issue prevention. For instance, Prometheus alerts maintainers when a particular memory usage spikes - and consequently, SREs can take proactive measures to prevent downtime.

Unveiling Grafana

Grafana is a scintillating data visualization tool that queries Prometheus and presents valuable information. Moreover, Grafana serves as a single pane of glass (SPOG) for displaying different kinds of data. Grafana’s highlights include the following:

• Interactive graphs, charts, and maps
• Custom dashboards
• Alerting rules to query multiple data sources
• User access management

Dashboard Templating

Dashboard templating makes Grafana dashboards a lot more dynamic and interactive. For instance, you can create custom Dashboard template variables and redefine legend names or row titles.

Broadly, there are three types of Template variables:

Query:

The Query template generates a dynamic list of variables from your data source. You can use wildcard entries like prod.servers.* to fetch all possible values in your wildcard entry.

Interval:

The interval type helps present data around time ranges (e.g., 1 minute, 1 hour, 1 day).

Custom:

The custom option is about creating user-defined template variables on the Grafana dashboard.

Image Source

Prometheus and Grafana are Kube’s equivalents of Romeo and Juliet. They are a match made in cloud computing heaven!

Kubestate Metrics: In a Nutshell

Kube-state-metrics (KSM) comprise a simple service that fetches metrics from k8s deployments. These metrics range from health & resource usage to internal object states.

KSM - at its core, is a service that calls the Kubernetes API to generate metrics regarding Kubernetes object states. The API exposes critical parameters like:

• Pod lifecycle status
• Node status
• Available deployment replicas

Image Source

The critical difference between Kubestate metrics and traditional metrics servers (pre-installed on cloud clusters) is that KSM provides data in a Prometheus format. As a result, SREs and DevOps functions can leverage customizable metric visualization and reporting tools.

Prometheus on Google Cloud

Grafana is an open-source tool that essentially adds a reporting layer to the Prometheus DB.

However, since Grafana is OSS, certain cloud providers might impose restrictions via a vendor lock-in.

The Managed Prometheus service on GCP allows you to leverage open-source tools and bypass such vendor lock-in situations.

As a result, you continue to scale your K8 deployments on Google Cloud - while harnessing the full potential of OSS like Grafana for out-of-the-box reporting and dashboard capabilities.

Installation of OSS version of Prometheus and Grafana

This one’s for all the SREs and Kube platform engineers! You guys rock! Here’s how you can deploy Prometheus and Grafana on your GKE cluster:

Installing Prometheus

We will walk you through the installation of Prometheus Operator on a K8s cluster. Prometheus Operator essentially simplifies the Prometheus deployment and configuration.

Without any further ado, here are the installation steps:

1. Install Prometheus Operator

   • Use the bundle.YAML file to install the Prometheus Operator
   • Key in the following command:
   • kubectl create -f
     https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml

2. Configure RBAC Privileges

   • Create a directory to store K8s manifests.
   • Use these commands
   • mkdir operator_k8s
   • cd operator_k8s
   • Create a YAML file named prom_rbac.yaml. Paste in the Kubernetes manifest

3. Deploy Prometheus

   • Create a YAML file named Prometheus.YAML
   • Paste in the manifest from the official documentation
   • Define certain key critical parameters like name, label, and memory
   • Save and close the file.
   • Deploy it to your GKE cluster by using
   • kubectl apply -f
     prometheus.monitoring.coreos.com/prometheus created

4. Creating a Prometheus Service

   • Create another manifest file called prom_svc.yaml and paste in the definitions
   • Save and close the file.
   • Now, create the service by:
   • kubectl apply -f prom_svc.yaml
     service/Prometheus created

5. Creating a Prometheus Service Monitor

   • Create a new manifest file called prometheus_servicemonitor.yaml and paste in the ServiceMonitor resource definition
   • Define key parameters like Service Monitor, label, scrape interval.
   • Save and close the file.
   • Deploy it to your GKE cluster by using
   • kubectl apply -f prometheus_servicemonitor.yaml
     servicemonitor.monitoring.coreos.com/prometheus-self created

Installing Grafana on Kubernetes

Here’s how you can deploy Grafana on your GKE cluster:

1. Create a Grafana Kubernetes manifest

   • Create a manifest file named grafana.yaml and paste in the definitions.

2. Send the manifest to your Kubernetes API server

   • Key in the following command:
   • kubectl apply -f grafana.YAML
   • Verify by executing this command:
   • kubectl port-forward service/grafana 3000:3000
   • Access localhost:3000 from your browser. You will land on a Grafana login page.
   • Use admin as the username & password for login.

Check out the official Grafana documentation for a more comprehensive tutorial.

Advantages of Prometheus & Grafana on GKE Cluster

Prometheus and Grafana are popular open-source software with a reasonably straightforward deployment and configuration setup.

Prometheus offers out-of-the-box data monitoring from various sources like Datadog, influxDB, MySQL, etc. Grafana is the ideal tool for visualizing all the data scraped by Prometheus. When these two tools work together, you can collect valuable insights.



Image Source

Prometheus integrates seamlessly with Kubernetes by automatically fetching metrics from replicas as services scale up.

On a side note, here are four benefits of monitoring your Kubernetes using Prometheus + Grafana:

1. Real-time alerts

   You can detect errors early and reduce your restoration time with proactive monitoring.

2. Efficient workload management

   Monitoring your k8s cluster can unveil pressing concerns like resource contention or unstable pod distribution. Straightforward tweaks like anti-affinities and setting affinities can improve performance.

3. Better cost visibility

   Kubernetes can drive up your bills in no time. Real-time monitoring of your clusters will help you keep an eye on rising costs.

4. Seamless Troubleshooting

   Constant monitoring makes it easy to quickly access logs and conduct an RCA for any issues on your Kubernetes cluster. As a result, you encounter lesser downtimes.

Wrapping Up

The bottom line is that both Prometheus and Grafana are game-changing additions to your k8s cluster. As your cloud services scale, it becomes critical to monitor key parameters about the health of your cloud containers. Prometheus does an excellent job of scraping valuable data metrics - while Grafana is a fantastic presentation tool that displays your metrics in various formats.

Our developers behind this guide hope can be easily followed - though if you or your team need any assistance in this implementation or anything else GCP related, get in touch for a free technical consultation today.