Kubernetes add-ons for clusterAPI and n…

Projectsveltos is a controller that that extends the functi…
projectsveltos
01/16/2023

Description

Install, configure, upgrade, delete Kubernetes add-ons on ClusterAPI powered clusters and not. Support for Helm charts. Cluster classification based on run-time state. Configuration drift detection and recovery.

  • projectsveltos

    Hi all

    I have been using ClusterAPI for an internal project at work. One problem I saw when creating tens of clusters per day with ClusterAPI, was that all cluster addons had to be manually installed/upgraded (there are solutions like Flux, which is a solid one, but I did not like the amount of user interaction that was still required). I wanted something capable of automatically discovering clusters and understanding their state.

    So I started working on an open source project called projectsveltos https://github.com/projectsveltos. I use golang and the project consists in a set of Kubernetes controllers, some deployed in the management cluster others in the managed cluster.

    The main idea of Sveltos is to allow users to programmatically list which addons need to be deployed where (where is expressed using a ClusterSelector which selects all ClusterAPI powered clusters with labels matching the selector).

    Helm values can be passed directly. Or Sveltos can be instructed to fetch those from the management cluster.

    Having worked in the past in a different startup doing policy distribution in a distributed system, I know visibility is important. So sveltos provides user with DryRun mode as well where user can see the effect of a change before committing it https://github.com/projectsveltos/sveltosctl#display-outcome-of-clusterprofiles-in-dryrun-mode

    But I was still not happy. Main reason, users still had to go and manage cluster labels. I wanted clusters' labels to change as cluster runtime state was changing. So I can express my intent and then forget about it.

    So I introduced a second concept in Sveltos. Classifier CRD (https://github.com/projectsveltos/demos/blob/main/classifier/classifier.gif) which allows users to classify clusters based on cluster runtime state (currently kubernetes version and/or resources deployed, but I am working on adding more).

    Doing so I can easily now say:

    • any cluster running Kubernetes version v1.25.2 needs this version of calico as CNI
    • as clusters are upgraded to Kubernetes version v1.25.2 the right labels are added on cluster so that right version of calico is deployed.

    Current feature list:

    1. Kubernetes addon distribution across multiple clusters (ClusterAPI powered clusters and not, like GKE for instance);

    2. Configuration drift detection: when sveltos detects a configuration drift, it re-syncs the cluster state back to the state described in the management cluster;

    3. Templates instantiated reading values from management cluster;

    4. Dry run to preview effect of a change;

    5. Kubernetes cluster classification and automatic label management based on cluster runtime states;

    6. Snapshot and Rollback.

    01/16/2023