Logging and monitoring#
Rackspace Kubernetes-as-a-Service (KaaS) provides state-of-the-art monitoring and logging tools to deliver the best user experience out-of-the-box. Proactive monitoring helps you detect problems early and act fast to ensure successful cloud infrastructure operation.
Log in to the logging and monitoring dashboards by using the URLs and credentials listed in Access the Rackspace KaaS dashboards. The provided credentials grant you read and write access to the Kibana and Grafana dashboards.
The Managed Services credentials for your cluster are stored in
the installer/clusters/<cluster-name>/generated/auth/credentials.sh
file.
Logging#
Elasticsearch, Fluentd, and Kibana, or EFK stack, is an industry
standard for collecting, storing, and visualizing Kubernetes logs.
Fluentd collects the Kubernetes containers logs and transmits them to
Elasticsearch. Elasticsearch stores the logs and exposes them through a
RESTful API. Kibana provides a graphical representation of the collected
logs along with filtering and search capabilities. All components of
EFK are deployed as Kubernetes Pods in the rackspace-system
namespace.
Using Kibana#
You can access Kibana by using your Managed Services credentials and the URL
listed in Access the Rackspace KaaS dashboards. To view and visualize the logs
You need to configure index patterns. By default, Kibana suggests using the
logstash-*
index pattern. You can use the default setting or
configure additional patterns as needed.
To configure Kibana, complete the following steps:
Log in to the Kibana UI using the URL and credentials provided in Access the Rackspace KaaS dashboards.
Go to DevTools > Console.
List available indices:
GET _cat/indices
Alternatively, you can use
curl -XGET "http://<path-to-Kibana-UI>_cat/indices"
.Example of system response:
green open .monitoring-es-6-2018.06.21 woJ90oNBTY2IdrN8edLh2w 1 1 182422 648 344.5mb 172.2mb green open .monitoring-es-6-2018.06.22 Br6Ioc8PRKu1ySQAt-d-cA 1 1 121440 432 229mb 114.8mb green open .watcher-history-3-2018.06.20 sopRUohGTIWtPRGKf-VGEA 1 1 5760 0 9.6mb 4.8mb green open .monitoring-kibana-6-2018.06.22 FE6508QfTyWDgc-WoQr_bg 1 1 5737 0 4.5mb 2.2mb green open .triggered_watches eca1efrIQN2E1XLiujeXEw 1 1 0 0 4.9mb 3.3mb green open .kibana eKGb8L2XRvCoOcUWBsDA1A 1 1 3 0 90kb 45kb green open .monitoring-es-6-2018.06.20 wH20lwaHT2ikRlwTCS1J_Q 1 1 182573 594 339.9mb 170mb green open logstash-2018.06.20 iuaiT6abSsmYSFTN7laLoA 5 1 188593 0 264.8mb 132.4mb green open .watches V1r07Q0wQ1WpHAsPsyNm3Q 1 1 4 0 30.7kb 19.3kb green open logstash-2018.06.22 O9me9g3BSE6H-0jtKU5oYA 5 1 121590 0 162.1mb 81mb green open .monitoring-kibana-6-2018.06.20 qZrbShReQka1Eufxb6iQ9A 1 1 8630 0 6.9mb 3.4mb green open logstash-2018.06.21 jsk2AKCPRfSwkFgg3_TGkQ 5 1 181909 0 251.4mb 125.8mb green open .monitoring-kibana-6-2018.06.21 Se2Qfs8WQtGTXlAzMd5Q-A 1 1 8632 0 6.7mb 3.3mb green open .watcher-history-3-2018.06.22 ENhLUYkmRlOpxOx2HavA6w 1 1 3828 0 6.3mb 3.1mb green open .watcher-history-3-2018.06.21 GQZ2-GvaSIq_dA7VTidHdA 1 1 5760 0 9.5mb 4.7mb
Click Management > Index Pattern > Create Index Pattern.
Configure the default
logstash
pattern or a custom pattern as needed.Click Create.
In the left navigation panel, click Discover to view the logs.
Example:
The EFK stack provides rich capabilities to configure logging. For more information, see Kibana documentation, Fluentd documentation, and Elasticsearch documentation.
Monitoring#
While logging is important, the information provided in logs is incomplete without monitoring data. The Rackspace KaaS monitoring solution is based on the popular Prometheus and Grafana bundle. Prometheus is a powerful monitoring and alerting tool for distributed systems that assists cloud administrators in watching the health of not only the Kubernetes nodes but the workloads that run on the Kubernetes cluster as well. Data visualization powered by Grafana provides insights into the cluster health through a web-based graphical user interface.
Rackspace KaaS deploys two instances of Prometheus in each Kubernetes cluster. One instance is for Kubernetes applications monitoring and is for the Rackspace customers. The other instance monitors the Kubernetes cluster itself and is for Rackspace operators use. Customers can view the Prometheus internal dashboard but cannot modify any existing settings or add new settings. Both internal and customer-facing Prometheus instances are preconfigured as data resources in Grafana.
Using Grafana#
Rackspace KaaS provides preconfigured dashboards for the cluster resource monitoring including resource utilization analytics for Pods, Deployments, Kubernetes nodes, etcd nodes, as well as other useful information that might help you with cluster capacity planning.
When you create a new Pod in your Kubernetes cluster, Grafana automatically adds it to the preconfigured Pods monitoring dashboard. Therefore, you can monitor Pod resource utilization through the Pod dashboard. If you need to set up additional metrics to track the health of your Kubernetes applications, you can do so by creating or importing custom Grafana dashboards and using them with the customer-facing Prometheus instance.
To use Grafana, complete the following steps:
Log in to the Grafana UI using the URL and credentials provided in Access the Rackspace KaaS dashboards.
Click Home.
Select a dashboard to display. For example, Pods.
You can select a namespace and Pod to display.
For more information about using Grafana and Prometheus, see Grafana documentation and Prometheus documentation.