Overview#

Kubernetes® is a software tool that helps you better orchestrate and manage your cloud, as well as significantly simplifies cloud operations and lowers the cost of cloud computing expenses. While running a simple Kubernetes cluster for testing purposes is a relatively easy task, configuring a fully operational production Kubernetes cluster is challenging and requires specific expertise in container management, networking, storage, security, and other aspects of a production cloud. Many organizations decide to outsource this work to cloud providers who specifically focus on complex cloud systems.

The Rackspace KaaS solution enables you to run Kubernetes workloads on top of Rackspace Private Cloud powered by OpenStack (RPCO). You can request that Rackspace Kubernetes-as-a-Service be installed on an existing or new RPCO environment starting with RPCO v14.

KaaS Control Panel#

The KaaS Control Panel is an integral part of the KaaS experience for administrators and regular users alike. The control panel allows users to create tokens, which are used to authenticate with the Kubernetes API and some managed services, to easily access and authenticate with managed services, and to perform other administrative actions on a cluster.

Authentication#

Authentication with the KaaS Control Panel is made with the username and password of any OpenStack user. The control panel looks up the provided credentials with OpenStack and allows the user to access the control panel if the credentials are valid.

Note

Any OpenStack user can log in to the control panel; no role memberships or other whitelists are necessary. However, an OpenStack users’ role memberships affect what they can do in the control panel and on a Kubernetes cluster.

Note

Authentication with the control panel is separate from authentication with a Kubernetes cluster. Control panel authentication uses OpenStack credentials, while Kubernetes authentication uses a token that is generated in the control panel. While the authentication mechanisms are different, users’ identifying information still comes from OpenStack in each case.

Authorization#

The KaaS Control Panel only distinguishes users as belonging to one of two groups: Cluster administrators and regular users. This determination is made based on the control panel’s active cluster (see Cluster Selection), and whether a user has membership in one of that cluster’s designated admin roles. When you provision a new cluster, a single “cluster admin” role is created in OpenStack, and a single OpenStack user is assigned to that role. This user can perform administrative actions in both the control panel and the cluster itself. Later, if other users are added to the same “cluster admin” role, they will inherit the same administrative capabilites.

Note

While members of the “cluster admin” role are the only OpenStack users who can administer a cluster from the time it is provisioned, any other users can later be granted similar capabilites using Kubernetes’ native RBAC tools.

When the control panel determines that a user is an administrator for the active cluster, it displays an Administration menu in the main navigation menu with links to pages for various administrative actions.

Cluster Selection#

Attention

In OpenStack environments with a single Kubernetes cluster, cluster selection does not apply. The environment’s one and only cluster is implicitly selected as the control panel’s active cluster.

At the top of the control panel’s main navigation menu is a blue dropdown showing the name of the active cluster. Any cluster-specific actions in the control panel are executed against the active cluster. To change the active cluster, click or tap the dropdown and select the name of the desired cluster from the list.

Caution

If you are seeing unexpected data in the control panel after changing the active cluster, refreshing the page in your browser will resolve any issues.

Navigation#

Along the left side of the control panel UI is the main navigation menu. This menu is divided into three sections, My Cluster, Administration, and Settings.

The My Cluster section links to pages that allow you to perform typical, non-administrative tasks on your cluster. It also contains links to each of the Managed services running on the cluster. This section might have the Cluster Selection dropdown in lieu of a header if the environment has more than one Kubernetes cluster.

The Administration section is only available to administrators of the active cluster. It enables users to manage RBAC and security settings on their cluster, including managing Pod Security Policies, namespaces, and role bindings.

The Settings section is available to all users and enables each user to manage their tokens.

Cluster Overview#

The Cluster Overview, or dashboard page, is the first page a user sees when they log in to the control panel. It includes a Cluster Info tile that shows each node in the active cluster, along with its status. Also in the Cluster Info tile is a Get Kubeconfig button, which will generate a new token named generated for kubeconfig on the user’s behalf, and then insert that token into a valid kubeconfig file for the active cluster. A user can then save the contents of that generated file and configure kubectl to access the cluster

Pod Security Policies#

Note

This page of the control panel is only available to cluster administrators.

The Pod Security Policies page enables administrators to manage Pod Security Policies, which define the security-sensitive privileges pods only the cluster can use. The list of pod security policies displays summary information about each one, as well as red buttons for each to remove them from the cluster permanently. The Create Pod Security Policy button displays a form for configuring the various options available in a pod security policy. See the Kubernetes Policy Reference for detailed descriptions about each of these options.

When you create a new pod security policy, the control panel also creates a cluster role named psp:<name-of-podsecuritypolicy>. This role enables users or service accounts to use the pod security policy, but an administrator must separately create a role binding or cluster role binding to bind specific users to the psp:<name-of-podsecuritypolicy> role.

Namespaces#

Note

This page of the control panel is only available to cluster administrators.

The Namespaces page enables users to manage Kubernetes namespaces and the rolebindings for each one. Clicking on a namespace displays details about the namespace itself, and a list of “access rules” (“role bindings” in Kubernetes parlance) for the namespace. Existing access rules can only be deleted, not edited. Clicking the Add Access Rule button shows a small form for defining a new access rule. Users can give the new access rule a name, select an OpenStack role from the Applies To dropdown list, and select a Kubernetes role or cluster role from the Role dropdown list.

Tokens#

The Tokens page enables to create and delete tokens, which are random strings used to authenticate with the Kubernetes API and some of the managed services. All tokens expire after a certain amount of time, after which they are no longer usable for authentication. Tokens can also be deleted if they are compromised or the user wishes to revoke the access granted by that token. Creating a token will fail if the Expires in field is set past the allowed maximum, which is typically 365 days. After you create a new token, its value is displayed only once. You cannot view the token at any later time.

Managed services#

In addition to the basic Kubernetes functionality, your Kubernetes cluster comes with managed services that provide extra features.

Warning

The rackspace-system and kube-system namespaces in Kubernetes are used for managed services. Do not modify any resources in these namespaces.

Rackspace KaaS provides the following managed services:

Monitoring: Any production cloud requires performance and uptime monitoring to enable cloud administrators to execute steps to address issues. The Managed Kubernetes solution leverages such tools as Prometheus and Grafana® integrated with the internal Monitoring as a Service (MaaS) system to enable Rackspace operators to track the health of your cloud. Rackspace KaaS deploys two instances of Prometheus. One is for internal use and the other is for monitoring Kubernetes applications. For more information, see Logging and monitoring.
Logging: Implemented by using such tools as Elasticsearch™, Fluentd, and Kibana, the logging managed service provides real-time data analytics and system resource utilization statistics of your cloud.
Private Docker® image registry: In addition to public Docker image registry, you can store and manage your own Docker images in a private registry implemented with VMware Harbor.
Networking: Rackspace KaaS uses Flannel for communication between the Kubernetes Pods. Flannel enables advanced networking features, such as network policies and overlay networking. You can define network policies as required for your cloud or request Rackspace Fanatical Support® to define them for you.
Storage: Rackspace KaaS uses a highly-available Ceph storage cluster provisioned through the OpenStack Block Storage (cinder) driver for Kubernetes PersistentVolumes (PV). Ceph provides persistent block storage for Kubernetes users and managed services internal data.

Ingress controller#

Kubernetes supports the concept of virtual hosting by using Ingress resources. The Ingress controller provided with your cluster enables Ingress resources for your cluster. The Ingress controller is highly available (HA) with two replicas. You can read more about Ingress resources in the official Kubernetes documentation.

Sites available through the Ingress controller are routed to by subdomain, such as https://kibana.<cluster-name>.<domain-name> for Kibana.

Authentication and authorization#

Rackspace manages a Kubernetes Authentication service with which users do not interact directly, but which plays an integral role in granting access to a Kubernetes cluster. This service is responsible for managing users’ tokens and providing Kubernetes with identifying information about users, which the Kubernetes Authentication service derives from the OpenStack Identity service (keystone).

Users access Kubernetes clusters by providing a token that they generate in the Rackspace KaaS Control Panel at https://control-panel.<openstack-env-domain-name>/login.

The Kubernetes Authentication service stores only hashes of tokens, so that the actual value of each token is visible only when it is created. If a token is misplaced or no longer used, it must be deleted. You can use the same token to authenticate with any Kubernetes cluster in the environment. Although each token is valid for one year from the time of creation, you can delete a token any time before it expires. Each user can create an unlimited number of tokens.

Users can access Kubernetes clusters by providing a token either by invoking kubectl --token or by adding a token to the kubeconfig file. Upon receiving a request with a token, the Kubernetes API passes the token to the Kubernetes Authentication service. If the Kubernetes Authentication service determines that the token is valid, it queries keystone to find information about the corresponding user and returns that information to the Kubernetes API. The Authentication service provides the following information to Kubernetes:

The user’s OpenStack username
The user’s OpenStack ID
The IDs of any OpenStack roles to which the user is assigned

Cluster roles#

The Kubernetes Authentication service enables access to Kubernetes clusters to users who provide valid tokens. However, to authorize a user to perform a specific set of operations, you must create a Kubernetes ClusterRoleBindings. The subject of the role bindings must reference such information as user and role IDs collected from the OpenStack Identity service.

Your Kubernetes cluster is pre-configured with a ClusterRoleBinding that grants cluster-wide administrative access to all users with a specific Keystone role. For example, if you created the kubernetes-admins role with the ID f6120fe6406a473682c3b25cdea4510a, your Kubernetes cluster has the following ClusterRoleBinding:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: "rackspace:default-cluster-admin"
subjects:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: f6120fe6406a473682c3b25cdea4510a
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

Note

Note the difference in the Kubernetes and keystone terminology. A Kubernetes group is referred to as role in Keystone terminology. Do not confuse Keystone’s concept of groups with the Kubernetes concept of groups.

You might want to create additional Keystone roles and role bindings to provision and control Kubernetes access for users without administrative privileges.

Disaster recovery#

Disaster recovery is an integral part of any robust production system. In a Kuberentes environment, you have multiple master, worker, and etcd nodes. Each of these servers runs multiple stateful and stateless components. Stateless components that run on Kubernetes master and worker nodes restore themselves automatically by using the KaaS automation tools. However, stateful components, such as etcd that stores a cluster's data and persistent volumes that store persistent data for your applications, are backed up by Heptio Ark.

Rackspace KaaS uses Heptio Ark to automatically create snapshots of persistent volumes and back up the cluster data that can be used later to recover from a cluster failure.

High availability#

Your Kubernetes cluster is configured to be highly available. The Kubernetes components are replicated behind a load balancer and distributed on multiple compute resources.

Resource limits#

By default, your Kubernetes cluster has three worker nodes. The default configuration of your cluster node includes:

Kubernetes worker nodes:

vCPU: 4
RAM: 8 GB
Local storage: 40 GB of local storage

Private Docker registry:

Database: 10 GB

Note

The private Docker registry database stores Docker images metadata. The actual Docker images are stored in the object storage system, such as Ceph RGW, OpenStack Swift, or other compatible object store deployed as part of your cloud.

If you need to resize your cluster, contact your Rackspace representative and request additional resources.

Monitoring#

To aid in Rackspace's monitoring of the cluster health anonymoys access to the healthz endpoint of the kubernetes API server is enabled. Users are free to integrate that with their own (external) monitoring solution.