Epsagon's integration to Kubernetes clusters provides:

• Performance metrics, insights, and alerts across your whole cluster.
• Detailed mappings of clusters, nodes, pods, containers, and deployments to verify their health.
• Containers' logs.

📘

Integrate multiple Kubernetes clusters

With Epsagon, you can integrate as many Kubernetes clusters that you want. Just follow the deployment process on each cluster that you wish to integrate.

Introduction

In this guide, we will go over how to view your cluster resources with their metrics in Epsagon's Kubernetes Explorer. To achieve this we will use the Epsagon cluster agent to send all the cluster resources, and a Prometheus agent to send metrics about the cluster resources.

Prometheus is a standalone monitoring system. It is an open-source project, maintained independently of any company.

Prometheus collects metrics from monitored targets by scraping metrics HTTP endpoints on these targets. Since Prometheus also exposes data in the same manner about itself, it can also scrape and monitor its own health.

Setup

Installing Epsagon's agent

1. Install Helm.
2. Install the Epsagon cluster agent using helm to send all the cluster resources and then view them in Epsagon Explorer:
   1. Replace <EPSAGON_TOKEN> with Epsagon Token.
   2. Choose a <CLUSTER_NAME> for your cluster, the chosen name will be displayed in Epsagon. Then, run:

helm repo add epsagon https://helm.epsagon.com
helm install <RELEASE_NAME>
    --set epsagonToken=<EPSAGON_TOKEN> --set clusterName=<CLUSTER_NAME> epsagon/cluster-agent

Go to Epsagon's Kubernetes Explorer, you'd now be able to see your k8s resources.

Sending Metrics to Epsagon

Choose your Prometheus:

• Prometheus installed using helm chart
• Prometheus operator
• No Prometheus installed
  
  
  
  

Prometheus Helm chart

Prometheus has an option to send metrics to a remote endpoint, this option is called remote write. Let's configure the remote write to send metrics into epsagon.

In the values.yaml we configured before, let's configure the following lines, make sure the configured <CLUSTER_NAME> matches the cluster name used when installing Epsagon's agent (displayed in Epsagon's Kubernetes Explorer).

server:
  remoteWrite:
  - url: https://collector.epsagon.com/ingestion?<EPSAGON_TOKEN>
    basic_auth:
      username: <EPSAGON_TOKEN>
    write_relabel_configs:
      - target_label: cluster_name
        replacement: <CLUSTER_NAME>

Make sure to also install kube-state-metrics, in order to fully utilize our kubernetes dashboards:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install [RELEASE_NAME] bitnami/kube-state-metrics

You're done! you should now see your metrics in Epsagon's Kubernetes Dashboard and Explorer.

Using remote write increases memory usage for Prometheus up to ~25% and depends on the shape of data. If you are experiencing issues with too high memory consumption try to lower  max_samples_per_send and capacity params (keep in mind that these two params are tightly connected). Read more about tuning remote write for Prometheus here.

Prometheus operator

For prometheus operator to be able to send basic auth requests we need to create a secret with the username. The secret should be created inside the same namespace that promethues is installed. You need to use replace <EPSAGON_TOKEN> with Epsagon Token, and keep the password value empty.

kubectl create secret generic epsagon-secret --from-literal=username=<EPSAGON_TOKEN> --from-literal=password=

In the manifest that contains the prometheus operator configurations, we will add the following remote write configuration, which contains the url with your <EPSAGON_TOKEN> and the <CLUSTER_NAME>, which will identify from what cluster the resources are coming from. Make sure the configured <CLUSTER_NAME> matches the cluster name used when installing Epsagon's agent (displayed in Epsagon's Kubernetes Explorer).

remoteWrite:
  - url: 'https://collector.epsagon.com/ingestion?<EPSAGON_TOKEN>'
    basicAuth:
      username:
        name: epsagon-secret
        key: username
      password:
        name: epsagon-secret
        key: password
    writeRelabelConfigs:
      - targetLabel: cluster_name
        replacement: <CLUSTER_NAME>

Make sure to also install kube-state-metrics, in order to fully utilize our kubernetes dashboards:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install [RELEASE_NAME] bitnami/kube-state-metrics

You're done! you should now see your metrics in Epsagon's Kubernetes Dashboard and Explorer.

Using remote write increases memory usage for Prometheus up to ~25% and depends on the shape of data. If you are experiencing issues with too high memory consumption try to lower  max_samples_per_send and capacity params (keep in mind that these two params are tightly connected). Read more about tuning remote write for Prometheus here.

No Prometheus installed

1. Let's install Prometheus agent with Helm charts. Add the repo as follows:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

2. Then install the prometheus helm chart:

helm install [RELEASE_NAME] prometheus-community/prometheus --set serviceAccounts.alertmanager.create=false --set serviceAccounts.nodeExporter.create=false --set serviceAccounts.pushgateway.create=false --set alertmanager.enabled=false --set nodeExporter.enabled=false --set pushgateway.enabled=false --set server.persistentVolume.size=10Gi

3. Now start sending your metrics to Epsagon.

FAQ

How strong should be the compute for Prometheus?

It mainly depends on the number of metrics that Prometheus will scrape, but the minimum would be 4GB of RAM and 2 CPU cores (the equivalent of t3a.medium in AWS).
Its recommended to start with 10 GB of storage.

How do I backup/restore?

No need to backup prometheus, the data is stored in Epsagon

How do I monitor my Prometheus?

You can configure prometheus to monitor itself, see more info here.
Once configured you will be able to watch those metrics at Epsagon as well.

How do I send metrics from Kubernetes? Do I need to install something on every node? container?

No, prometheus has scrape configuration which contains all the options needed to scrape metrics inside kubernetes. More info here

More FAQ's can be found here.

Correlate traces and logs

Epsagon automatically correlates traces and logs, so that you can view the logs written in the context of a specific span, without having to manually search in your logs or manually inject span IDs in your logs.

To enable trace to log correlation

1. Trace your containers. Log correlation is supported for Node.js, Python, and Java.
2. Send your logs to AWS CloudWatch by setting up FluentD as a DeamonSet. Make sure that the configmap is named cluster-info, and is in the namespace amazon-cloudwatch.

To view logs of a trace, open a trace, select a node in the graph, and click Show Logs.