Skip to main content
Version: v1.10

Migration From Kubefed

Karmada is developed in continuation of Kubernetes Federation v1 and Federation v2(aka Kubefed). Karmada inherited a lot of concepts from these two versions. For example:

  • Resource template: Karmada uses Kubernetes Native API definition for federated resource template, to make it easy to integrate with existing tools that already adopt Kubernetes.
  • Propagation Policy: Karmada offers a standalone Propagation(placement) Policy API to define multi-cluster scheduling and spreading requirements.
  • Override Policy: Karmada provides a standalone Override Policy API for specializing cluster relevant configuration automation.

Most of the features in Kubefed have been reformed in Karmada, so Karmada would be the natural successor.

Generally speaking, migrating from Kubefed to Karmada would be pretty easy. This document outlines the basic migration path for Kubefed users. Note: This document is a work in progress, any feedback would be welcome.

Cluster Registration

Kubefed provides join and unjoin commands in kubefedctl command line tool, Karmada also implemented the two commands in karmadactl.

Refer to Kubefed Cluster Registration, and Karmada Cluster Registration for more details.

Joining Clusters

Assume you use the kubefedctl tool to join cluster as follows:

kubefedctl join cluster1 --cluster-context cluster1 --host-cluster-context cluster1

Now with Karmada, you can use karmadactl tool to do the same thing:

karmadactl join cluster1 --cluster-context cluster1 --karmada-context karmada

The behavior behind the join command is similar between Kubefed and Karmada. For Kubefed, it will create a KubeFedCluster, object and Karmada will create a Cluster object to describe the joined cluster.

Checking status of joined clusters

Assume you use the kubefedctl tool to check the status of the joined clusters as follows:

$ kubectl -n kube-federation-system get kubefedclusters

NAME AGE READY KUBERNETES-VERSION
cluster1 1m True v1.21.2
cluster2 1m True v1.22.0

Now with Karmada, you can use karmadactl tool to do the same thing:

$ kubectl get clusters

NAME VERSION MODE READY AGE
member1 v1.20.7 Push True 66s

Kubefed manages clusters with Push mode, however Karmada supports both Push and Pull modes. Refer to Overview of cluster mode for more details.

Unjoining clusters

Assume you use the kubefedctl tool to unjoin as follows:

kubefedctl unjoin cluster2 --cluster-context cluster2 --host-cluster-context cluster1

Now with Karmada, you can use karmadactl tool to do the same thing:

karmadactl unjoin cluster2 --cluster-context cluster2 --karmada-context karmada

The behavior behind the unjoin command is similar between Kubefed and Karmada, they both remove the cluster from the control plane by removing the cluster object.

Propagating workload to clusters

Assume you are going to propagate a workload (Deployment) to both clusters named cluster1 and cluster2, you might have to deploy following yaml to Kubefed:

apiVersion: types.kubefed.io/v1beta1
kind: FederatedDeployment
metadata:
name: test-deployment
namespace: test-namespace
spec:
template:
metadata:
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
placement:
clusters:
- name: cluster2
- name: cluster1
overrides:
- clusterName: cluster2
clusterOverrides:
- path: "/spec/replicas"
value: 5
- path: "/spec/template/spec/containers/0/image"
value: "nginx:1.17.0-alpine"
- path: "/metadata/annotations"
op: "add"
value:
foo: bar
- path: "/metadata/annotations/foo"
op: "remove"

Now with Karmada, the yaml could be split into 3 yamls, one for each of the template, placement and overrides.

In Karmada, the template doesn't need to embed into Federated CRD, it just the same as Kubernetes native declaration:

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx

For the placement part, Karmada provides PropagationPolicy API to hold the placement rules:

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: nginx-propagation
spec:
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginx
placement:
clusterAffinity:
clusterNames:
- cluster1
- cluster2

The PropagationPolicy defines the rules of which resources(resourceSelectors) should be propagated to where (placement). See Resource Propagating for more details.

For the override part, Karmada provides OverridePolicy API to hold the rules for differentiation:

apiVersion: policy.karmada.io/v1alpha1
kind: OverridePolicy
metadata:
name: example-override
namespace: default
spec:
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginx
overrideRules:
- targetCluster:
clusterNames:
- cluster2
overriders:
plaintext:
- path: "/spec/replicas"
operator: replace
value: 5
- path: "/metadata/annotations"
operator: add
value:
foo: bar
- path: "/metadata/annotations/foo"
operator: remove
imageOverrider:
- component: Tag
operator: replace
value: 1.17.0-alpine

The OverridePolicy defines the rules of which resources(resourceSelectors) should be overwritten when propagating to where(targetCluster).

In addition to Kubefed, Karmada offers various alternatives to declare the override rules, see Overriders for more details.

FAQ

Will Karmada provide tools to smooth the migration?

We don't have the plan yet, as we reached some Kubefed users and found that they're usually not using vanilla Kubefed but the forked version, they extended Kubefed a lot to meet their requirements. So, it might be pretty hard to maintain a common tool to satisfy most users.

We are also looking forward to more feedback now, please feel free to reach us, and we are glad to support you finish the migration.