Overview#
Kubernetes® is a software tool that helps you better orchestrate and manage your cloud, as well as significantly simplifies cloud operations and lowers the cost of cloud computing expenses. While running a simple Kubernetes cluster for testing purposes is a relatively easy task, configuring a fully operational production Kubernetes cluster is challenging and requires specific expertise in container management, networking, storage, security, and other aspects of a production cloud. Many organizations decide to outsource this work to cloud providers who specifically focus on complex cloud systems.
The Rackspace KaaS solution enables you to run Kubernetes workloads on top of Rackspace Private Cloud powered by OpenStack (RPCO). You can request that Rackspace Kubernetes-as-a-Service be installed on an existing or new RPCO environment starting with RPCO v14.
KaaS Control Panel#
The KaaS Control Panel is an integral part of the KaaS experience for administrators and regular users alike. The control panel allows users to create tokens, which are used to authenticate with the Kubernetes API and some managed services, to easily access and authenticate with managed services, and to perform other administrative actions on a cluster.
Authentication#
Authentication with the KaaS Control Panel is made with the username and password of any OpenStack user. The control panel looks up the provided credentials with OpenStack and allows the user to access the control panel if the credentials are valid.
Note
Any OpenStack user can log in to the control panel; no role memberships or other whitelists are necessary. However, an OpenStack users’ role memberships affect what they can do in the control panel and on a Kubernetes cluster.
Note
Authentication with the control panel is separate from authentication with a Kubernetes cluster. Control panel authentication uses OpenStack credentials, while Kubernetes authentication uses a token that is generated in the control panel. While the authentication mechanisms are different, users’ identifying information still comes from OpenStack in each case.
Authorization#
The KaaS Control Panel only distinguishes users as belonging to one of two groups: Cluster administrators and regular users. This determination is made based on the control panel’s active cluster (see Cluster Selection), and whether a user has membership in one of that cluster’s designated admin roles. When you provision a new cluster, a single “cluster admin” role is created in OpenStack, and a single OpenStack user is assigned to that role. This user can perform administrative actions in both the control panel and the cluster itself. Later, if other users are added to the same “cluster admin” role, they will inherit the same administrative capabilites.
Note
While members of the “cluster admin” role are the only OpenStack users who can administer a cluster from the time it is provisioned, any other users can later be granted similar capabilites using Kubernetes’ native RBAC tools.
When the control panel determines that a user is an administrator for the active cluster, it displays an Administration menu in the main navigation menu with links to pages for various administrative actions.
Cluster Selection#
Attention
In OpenStack environments with a single Kubernetes cluster, cluster selection does not apply. The environment’s one and only cluster is implicitly selected as the control panel’s active cluster.
At the top of the control panel’s main navigation menu is a blue dropdown showing the name of the active cluster. Any cluster-specific actions in the control panel are executed against the active cluster. To change the active cluster, click or tap the dropdown and select the name of the desired cluster from the list.
Caution
If you are seeing unexpected data in the control panel after changing the active cluster, refreshing the page in your browser will resolve any issues.
Managed services#
In addition to the basic Kubernetes functionality, your Kubernetes cluster comes with managed services that provide extra features.
Warning
The rackspace-system
and kube-system
namespaces in Kubernetes
are used for managed services. Do not modify any resources
in these namespaces.
Rackspace KaaS provides the following managed services:
- Monitoring
- Any production cloud requires performance and uptime monitoring to enable cloud administrators to execute steps to address issues. The Managed Kubernetes solution leverages such tools as Prometheus and Grafana® integrated with the internal Monitoring as a Service (MaaS) system to enable Rackspace operators to track the health of your cloud. Rackspace KaaS deploys two instances of Prometheus. One is for internal use and the other is for monitoring Kubernetes applications. For more information, see Logging and monitoring.
- Logging
- Implemented by using such tools as Elasticsearch™, Fluentd, and Kibana, the logging managed service provides real-time data analytics and system resource utilization statistics of your cloud.
- Private Docker® image registry
- In addition to public Docker image registry, you can store and manage your own Docker images in a private registry implemented with VMware Harbor.
- Networking
- Rackspace KaaS uses Flannel for communication between the Kubernetes Pods. Flannel enables advanced networking features, such as network policies and overlay networking. You can define network policies as required for your cloud or request Rackspace Fanatical Support® to define them for you.
- Storage
- Rackspace KaaS uses a highly-available Ceph storage cluster provisioned through the OpenStack Block Storage (cinder) driver for Kubernetes PersistentVolumes (PV). Ceph provides persistent block storage for Kubernetes users and managed services internal data.
Ingress controller#
Kubernetes supports the concept of virtual hosting by using Ingress resources. The Ingress controller provided with your cluster enables Ingress resources for your cluster. The Ingress controller is highly available (HA) with two replicas. You can read more about Ingress resources in the official Kubernetes documentation.
Sites available through the Ingress controller are routed to by subdomain, such
as https://kibana.<cluster-name>.<domain-name>
for Kibana.
Authentication and authorization#
Rackspace manages a Kubernetes Authentication service with which users do not interact directly, but which plays an integral role in granting access to a Kubernetes cluster. This service is responsible for managing users’ tokens and providing Kubernetes with identifying information about users, which the Kubernetes Authentication service derives from the OpenStack Identity service (keystone).
Users access Kubernetes clusters by providing a token that they generate in the Rackspace KaaS Control Panel at https://control-panel.<openstack-env-domain-name>/login.
The Kubernetes Authentication service stores only hashes of tokens, so that the actual value of each token is visible only when it is created. If a token is misplaced or no longer used, it must be deleted. You can use the same token to authenticate with any Kubernetes cluster in the environment. Although each token is valid for one year from the time of creation, you can delete a token any time before it expires. Each user can create an unlimited number of tokens.
Users can access Kubernetes clusters by providing a token either by
invoking kubectl --token
or by adding a token to the kubeconfig
file.
Upon receiving a request with a token, the Kubernetes API passes the token to
the Kubernetes Authentication service. If the Kubernetes Authentication service
determines that the token is valid, it queries keystone to find information
about the corresponding user and returns that information to the Kubernetes
API. The Authentication service provides the following information
to Kubernetes:
- The user’s OpenStack username
- The user’s OpenStack ID
- The IDs of any OpenStack roles to which the user is assigned
Cluster roles#
The Kubernetes Authentication service enables access to Kubernetes clusters to
users who provide valid tokens. However, to authorize a user to perform a
specific set of operations, you must create a Kubernetes
ClusterRoleBindings
.
The subject
of the role bindings must reference such information
as user and role IDs collected from the OpenStack Identity service.
Your Kubernetes cluster is pre-configured with a ClusterRoleBinding
that
grants cluster-wide administrative access to all users with a specific
Keystone role. For example, if you created the
kubernetes-admins
role with the ID
f6120fe6406a473682c3b25cdea4510a
, your Kubernetes cluster has the
following ClusterRoleBinding
:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: "rackspace:default-cluster-admin"
subjects:
- kind: Group
apiGroup: rbac.authorization.k8s.io
name: f6120fe6406a473682c3b25cdea4510a
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
Note
Note the difference in the Kubernetes and keystone terminology. A Kubernetes group is referred to as role in Keystone terminology. Do not confuse Keystone’s concept of groups with the Kubernetes concept of groups.
You might want to create additional Keystone roles and role bindings to provision and control Kubernetes access for users without administrative privileges.
Disaster recovery#
Disaster recovery is an integral part of any robust production system. In a Kuberentes environment, you have multiple master, worker, and etcd nodes. Each of these servers runs multiple stateful and stateless components. Stateless components that run on Kubernetes master and worker nodes restore themselves automatically by using the KaaS automation tools. However, stateful components, such as etcd that stores a cluster's data and persistent volumes that store persistent data for your applications, are backed up by Heptio Ark.
Rackspace KaaS uses Heptio Ark to automatically create snapshots of persistent volumes and back up the cluster data that can be used later to recover from a cluster failure.
High availability#
Your Kubernetes cluster is configured to be highly available. The Kubernetes components are replicated behind a load balancer and distributed on multiple compute resources.
Resource limits#
By default, your Kubernetes cluster has three worker nodes. The default configuration of your cluster node includes:
- Kubernetes worker nodes:
- vCPU: 4
- RAM: 8 GB
- Local storage: 40 GB of local storage
- Private Docker registry:
- Database: 10 GB
Note
The private Docker registry database stores Docker images metadata. The actual Docker images are stored in the object storage system, such as Ceph RGW, OpenStack Swift, or other compatible object store deployed as part of your cloud.
If you need to resize your cluster, contact your Rackspace representative and request additional resources.
Monitoring#
To aid in Rackspace's monitoring of the cluster health anonymoys access to the healthz endpoint of the kubernetes API server is enabled. Users are free to integrate that with their own (external) monitoring solution.