Skip to main content

Deploy a Custom Pack

Custom add-on packs allow you to deploy Kubernetes applications in clusters and reuse them in multiple deployments. This ensures uniformity across your clusters. The primary use cases for creating custom packs are:

  • Aggregated configuration and application dependencies simplify deployment and consumption.

  • Open-source contributors can add new Kubernetes applications to a custom add-on pack for the community.

  • Enterprises can add proprietary Kubernetes applications to a custom add-on pack.

In this tutorial, you will create a custom add-on pack to package a sample Kubernetes application, Hello Universe, and deploy the application to a cluster. You will learn to create the pack in two ways - using manifest files and Helm charts.

After defining the custom pack, you will set up a new registry server or leverage an existing Open Container Initiative (OCI) registry. Then, you will publish the pack to the registry and configure the registry server in Palette. Lastly, you will create a cluster profile that contains your custom pack and apply the profile to a cluster using either Palette or Terraform.

The following diagram illustrates the sequential steps required to successfully complete this tutorial.

Architecture Diagram of the Deploy a Custom Pack Tutorial

Prerequisites​

To complete this tutorial, ensure you have the following prerequisites in place:

  • A Palette account.
  • Tenant admin access to Palette for the purpose of adding a new registry server.
  • An Amazon Web Services (AWS) account added to your Palette project settings. Refer to the Add an AWS Account to Palette guide for instructions.
  • An SSH key available in the region where you plan to deploy the cluster.
  • Docker or Podman installed on your local machine to start the tutorial container.
  • Basic knowledge of containers and Kubernetes manifest file attributes. Refer to the Docker Get Started guide and the Learn Kubernetes Basics tutorial to start learning.

If you choose to use an OCI registry, you will need the following item.

If you opt for an ECR OCI registry, you will require the following.

  • An AWS Identity and Access Management (IAM) user with the following permissions.

    • ecr:CreateRepository
    • ecr:InitiateLayerUpload
    • ecr:CompleteLayerUpload
    • ecr:InitiateLayerUpload
    • ecr:PutImage
    • ecr:UploadLayerPart
    • ecr:BatchCheckLayerAvailability
    • ecr:ListImages
    • ecr:DescribeImages
    • ecr:BatchDeleteImage
    • ecr:DeleteRepository
warning

AWS expenses are associated with this tutorial. An estimated hourly cost is provided by Palette during the cluster creation step. After completing the tutorial, make sure to delete the infrastructure to avoid additional charges.

Set Up the Tutorial Environment​

In this tutorial, you will work in a container pre-configured with the necessary tools. Alternatively, you can choose to follow along with the tutorial in any linux/amd64 or x86_64 environment by installing the required tools and cloning the GitHub repository that contains the tutorial files. To initialize the tutorial container, follow the steps described below.

Start Docker Desktop on your local machine and ensure that the Docker daemon is available by issuing a command to list the currently active containers.

docker ps

Use the following command to download the ghcr.io/spectrocloud/tutorials:1.1.2 image to your local machine. This Docker image includes the necessary tools.

docker pull ghcr.io/spectrocloud/tutorials:1.1.2

Next, start the container and open a bash session into it.

docker run --name tutorialContainer --publish 7000:5000 --interactive --tty ghcr.io/spectrocloud/tutorials:1.1.2 bash

If the port 7000 on your local machine is unavailable, you can use any other port of your choice.

warning

Do not exit the container until the tutorial is complete. Otherwise, you may lose your progress.

Build a Pack​

Building a custom pack involves defining specific files. As outlined in the Adding Add-on Packs guide, there are two ways to define a custom pack: using manifest files or Helm charts. The file structure differs for manifest-based packs and Helm chart-based packs. Below is the reference file structure for each.

.
├── pack.json           # Mandatory.
├── values.yaml         # Mandatory.
├── manifests           # Mandatory.
    ├── manifest-1.yaml
    ├── manifest-2.yaml
│   └── manifest-3.yaml
├── logo.png            # Mandatory.
└── README.md           # Optional.

For your convenience, we provide you with the manifest-based pack files for the Hello Universe application. These files are located in the packs/hello-universe-pack folder.

Navigate to the packs/hello-universe-pack directory and list its files.

cd /packs/hello-universe-pack && ls -ll

Ensure you have the following files in the current directory.

.
├── pack.json           # Mandatory.
├── values.yaml         # Mandatory.
├── manifests           # Mandatory.
│   └── hello-universe.yaml
├── logo.png            # Mandatory.
└── README.md           # Optional.

Pack File Structure​

Review each of the following five files in the hello-universe-pack folder.

  • pack.json - This file contains the pack metadata such as addonType, cloudTypes, and the kubeManifests array. The array consists of a list of manifest files: layer, name, and version. Refer to the JSON Schema for a list of attributes and respective data types. The schema validation happens when you push a pack to the registry.

    {
      "addonType": "app services",
      "cloudTypes": ["all"],
      "displayName": "Hello Universe",
      "kubeManifests": ["manifests/hello-universe.yaml"],
      "layer": "addon",
      "name": "hellouniverse",
      "version": "1.0.0"
    }

  • values.yaml - This file contains configurable parameters you can define while adding the current pack to a cluster profile. In the values.yaml file for this tutorial, the pack/namespace attribute specifies the namespace on the target cluster to deploy the pack. If the values.yaml specifies a namespace value, then Palette first checks to confirm if the namespace has been created. If so, Palette uses the existing namespace. If the namespace has not yet been created, Palette creates a new one using the value specified in the YAML file.

    Keep in mind that if the values.yaml does not specify a namespace value, Palette will deploy the application to the default namespace.

    The manifests section exposes the configurable parameters for each manifest file listed in the manifests directory. For example, in the sample code snippet below, the hello-universe attribute exposes the registry, repository, and tag parameters.

    pack:
      namespace: "hello-universe"
    manifests:
      hello-universe:
        registry: ghcr.io
        repository: spectrocloud/hello-universe
        tag: 1.0.12

    Optionally, you can define presets, which are pack configuration values predefined in a file called presets.yaml within the pack. Once defined, the Presets field becomes visible in both the Clusters and Profile sections of the Palette UI. Users can select any preset from the available pack presets, and upon selection, the predefined values of the chosen preset are applied to the pack. Refer to Pack Presets for details and examples of how to define presets.

    The example below shows the parameters you can configure in the values.yaml file for the hello-universe manifest during the creation of the cluster profile.

Screenshot of the configurable parameters in the values.yaml file.

  • manifests - This directory contains the manifest files for your Kubernetes application. This tutorial has only one manifest, hello-universe.yaml. Note that the values.yaml file has a corresponding manifests/hello-universe element with the same name as the YAML file.

  • logo.png - This file contains a logo that displays when you create a cluster profile.

  • README.md - This file may contain the pack description, purpose, authors, and other relevant information. The README in the current example introduces the application used in the pack.

After completing the review of all files in the pack directory, the next step is to set up a registry server, publish the pack to the registry, and configure the registry in Palette.

Set Up the Registry Server​

You can set up a registry server using either the Spectro registry or an OCI-compliant registry. Palette supports all OCI-compliant registries, and you can refer to the Spectro Cloud OCI Registry resource for more information.

The tutorial environment already includes the Spectro registry service and other necessary tools. For OCI registries, as per the Prerequisites section, ensure you have an active OCI registry. Two types of OCI authentication are available: Amazon (ECR) and Basic. To learn more about Amazon ECR, consult the What is ECR user guide.

For Basic OCI Authentication, this tutorial uses a Harbor registry as an example. However, you have the flexibility to opt for the OCI registry of your choice. Learn how to set up a Harbor registry server using the Harbor Installation and Configuration guide.

The following sections will guide you through starting the registry server, authenticating, pushing your custom add-on pack, and, finally, configuring the registry server in Palette. Select the tab below corresponding to the registry type you want to use.

Start the Registry Server​

Start the registry server by issuing the following command from the tutorial container bash session initialized in the Set Up the Tutorial Environment section.

registry serve /etc/spectro/config.yml > /var/log/registry.log 2>&1 &

The registry server starts in HTTP mode. If you want to deploy an HTTPS registry server, refer to the Add a Custom Registry guide.

Next, make the registry server accessible to the public using Ngrok reverse proxy so that you can configure it later in Palette. Execute the command below to expose the registry server listening on port 5000 via an HTTP tunnel.

ngrok http 5000 --log-level debug

This command reserves the current bash session and displays the status of each HTTP request made to the Ngrok server. The image below shows the registry server successfully exposed via Ngrok.

Screenshot of registry server exposed via ngrok

Check if the registry server is accessible from outside the tutorial container by visiting the /health endpoint. Open your browser and go to https://Your-URL-Here/health, replacing the base URL with the Ngrok URL output. You should get a {"status":"UP"} response.

Log in to the Registry Server​

Once the /health endpoint of the registry server displays an UP status, proceed to the authentication step. In a new terminal window, start another bash session in the tutorial container.

docker exec -it tutorialContainer bash

Log in to the registry server using the Ngrok public URL assigned to you. Issue the following command, replacing the URL with your Ngrok URL. The --insecure flag indicates that the connection to the Spectro registry will be made without verifying the TLS certificate. The command below uses these credentials to log in to the registry server: {username: admin, password: admin}.

spectro registry login  --insecure --default --username admin --password admin \
58ec-174-119-143-38.ngrok-free.app
warning

Do not include the "https://" or "http://" prefixes in the Ngrok URL. Using either of these prefixes will result in an authorization issue.

You will receive a Login Succeeded response upon successful login.

# Output condensed for readability
WARNING! Your password will be stored unencrypted in /root/.spectro/config.json.
Login Succeeded

Push the Pack to the Registry Server​

Once you are logged in, push the pack to the registry server using the following command.

spectro pack push /packs/hello-universe-pack/

To confirm that the pack is now in the registry, use the ls command. This command lists all packs available in the registry.

spectro pack ls

Check if the pushed pack is listed, as shown in the image below.

Screenshot of spectro pack ls

For assistance with Spectro CLI commands, refer to the Spectro CLI Commands guide.

Configure the Registry Server in Palette​

After pushing the pack to the registry server, follow the next steps to log in to Palette and add the registry server to it.

Log in to Palette and switch to the Tenant Admin view.

Screenshot of Palette tenant settings.

Navigate to the Tenant Settings > Registries > Pack Registries section and click on Add New Pack Registry. Palette will open a pop-up window prompting you for the required fields to configure a custom pack registry.

A screenshot highlighting the fields to configure a custom pack registry.

Provide the pack registry name, endpoint, and user credentials in the pop-up window. For consistency, we suggest using the registry name spectro-pack-registry. Use your Ngrok URL as the pack registry endpoint. Ensure to add "https://" as the prefix in the pack registry endpoint. Set both the username and password as admin.

In the TLS Configuration section, select the Insecure Skip TLS Verify checkbox. This tutorial does not establish a secure HTTPS connection between Palette and your pack registry server. Therefore, you can skip the TLS verification. Instead, this tutorial uses an unencrypted HTTP connection. However, in a production environment, you can upload your certificate in the TLS Configuration section if you need Palette to establish a secure HTTPS connection while communicating with the pack registry server.

Click on Validate to ensure the provided URL and credentials are correct, then click on Confirm to finish the registry server configuration.

Screenshot of registry server edit option in Palette tenant settings.

Palette automatically syncs the registry server. However, you can sync it manually by clicking the three-dot Menu next to the registry server name and selecting Sync.

Screenshot of registry server sync in Palette

Create a Cluster Profile and Deploy a Cluster​

Once you have configured the registry, create a cluster profile and apply the profile to a cluster in Palette. This tutorial provides two workflows from which you can choose: Palette User Interface (UI) or Terraform.

Create a Cluster Profile​

Switch to the Default project scope for creating a cluster profile.

Screenshot of the Palette Default scope.

Next, select the Profiles section in the left Menu to create a cluster profile that combines the core infrastructure and add-on layers. Click on the Add Cluster Profile button.

Basic Information​

Complete the wizard using the values provided below.

FieldValue
Namepack-tutorial-profile
Version1.0.0
DescriptionCluster profile as part of the Deploy a Custom Pack tutorial.
TypeFull
Tagsspectro-cloud-education, app:hello-universe, terraform_managed:false

Click on Next to continue.

Cloud Type​

In the Cloud Type section, select AWS as the infrastructure provider, and click on Next at the bottom to proceed to the next section.

info

If you choose a different cloud provider, note that the options for core infrastructure layers, as outlined in the Profile Layers section below, will differ from those presented in this tutorial.

Profile Layers​

In the Profile Layers section, add the following core infrastructure layers.

Pack TypeRegistryPack NamePack Version
OSPublic RepoUbuntuLTS__22.4.x
KubernetesPublic RepoPalette eXtended Kubernetes1.28.x
NetworkPublic RepoCalico3.26.x
StoragePublic RepoAmazon EBS CSI1.22.x

As you add each layer, click on the Next layer button. After adding the Storage layer, click on the Confirm button to complete the core infrastructure stack. Palette displays the newly created infrastructure profile as a layered diagram. You can select any layer to make further edits or change the version if desired.

Next, proceed to include the add-on layers. Click on the Add New Pack button.

Add the Spectro Proxy pack to enable a reverse proxy to connect to the cluster's API. Adding this pack is optional, but it will help connect your local machine to the cluster's API for debugging. Refer to the Spectro Proxy guide for more details.

Pack TypeRegistryPack NamePack Version
AuthenticationPublic RepoSpectro Proxy1.4.x

Click on the Confirm & Create button to finish adding the Spectro Proxy pack.

Now, click on the Kubernetes layer and add the following certificate Subject Alternative Name (SAN) value under the apiServer parameter section to configure the Spectro Proxy pack.

certSANs:
  - "cluster-{{ .spectro.system.cluster.uid }}.{{ .spectro.system.reverseproxy.server }}"

Screenshot of the certificate Subject Alternative Name.

Finally, add the Hello Universe pack. Click on Add New Pack and select the registry you created earlier in this tutorial. The table below specifies the registry name corresponding to the registry type. For example, if you created an ECR registry, select ecr-registry.

Pack TypeRegistry TypeRegistry NamePack NamePack Version
App ServicesSpectro Registryspectro-pack-registryHello Universe1.0.x
App ServicesECRecr-registryHello Universe1.0.x
App ServicesBasicharbor-registryHello Universe1.0.x

Click on the Confirm & Create button to finish adding the Hello Universe pack.

If there are no errors or compatibility issues, Palette displays the cluster profile. Verify the layers you added, and click Next.

Screenshot of the Profile Layers success.

Review​

Review the cluster layers and click on Finish Configuration to complete the creation of the cluster profile.

Create a Cluster​

Navigate to the Profiles page and select the recently created cluster profile. Click on the Deploy button to start the deployment of a new cluster.

Basic Information​

For the first section, Basic information, use the following values.

FieldValue
Cluster namepack-tutorial-cluster
DescriptionCluster as part of the Deploy a Custom Pack tutorial.
Tagsspectro-cloud-education, app:hello-universe, terraform_managed:false
Cloud AccountSelect the cloud you have registered in Palette for AWS

Click Next to proceed.

Parameters​

The Parameters section allows you to change the profile configurations. For example, by clicking on the Hello Universe 1.0.x layer, you can configure the registry, repository, and tag parameters defined in the values.yaml file.

Screenshot of the Cluster layers.

Keep the default values and click Next.

Cluster config​

In the Cluster config section, make sure to uncheck the Static Placement field. If it is checked, the Static Placement will deploy the cluster within an existing Virtual Private Cloud (VPC), and you will need to provide the Amazon Resource Names (ARNs) for the existing subnets, roles, and other resources. This tutorial will use dynamic placement, allowing Palette to create a new VPC along with all necessary resources for the cluster.

In the Region field, choose the region of your preference. This tutorial will deploy the cluster in the us-east-1 region. For the SSH Key Pair Name field, select the SSH key pair name available in the chosen region. Ensure that you have already created an SSH key in the AWS region where you plan to deploy the cluster.

Click Next to continue.

Nodes config​

In the Nodes config section, provide the details for the control plane and worker pools. For this tutorial, you can use the following minimal configuration:

FieldValue for the control-plane-poolValue for the worker-pool
Node pool namecontrol-plane-poolworker-pool
Number of nodes in the pool11
Allow worker capabilityCheckedNot applicable
Enable AutoscalerNot applicableNo
Rolling updateNot applicableExpand First. Launch a new node first, then shut down the old one.

Keep the Cloud Configuration the same for both control plane and worker pools.

FieldValue
Instance TypeGeneral purpose m4.xlarge A minimum allocation of four CPU cores is required for the control plane node.
Availability zonesChoose any one availability zone. This tutorial uses the us-east-1a availability zone.
Disk size60 GiB

Click Next to continue.

Settings​

The Settings section displays options for OS patching, scheduled scans, scheduled backups, and cluster role binding. Use the default values, and click on the Validate button.

Review​

Review all configurations in this section. The Review page displays the cluster name, tags, cloud account name, node pools, layers, and an estimated hourly cost. If everything is correct, click on the Finish Configuration button to complete the cluster deployment, which may take up to 20 minutes.

While the deployment is in progress, Palette displays the cluster status as Provisioning. While you wait for the cluster to finish deploying, you can explore the various tabs on the cluster page, such as Overview, Workloads, and Events.

Validate​

In Palette, navigate to the left Main Menu and select Clusters. Next, select your cluster to display the cluster's Overview page and monitor the provisioning progress.

Screenshot of the cluster health.

Once the cluster status displays Running and Healthy, you can access the application through the exposed service URL along with the displayed port number. For the Hello Universe application, port 8080 is exposed. Click on the Services URL to access the application.

warning

We recommend waiting for one to three minutes before clicking on the service URL. This allows DNS to properly resolve the public load balancer URL, preventing the browser from caching an unresolved DNS request.

Screenshot of the successful accessing the application using the load balancer URL.

You can also look at real-time metrics, such as CPU and memory consumption, in the cluster's Overview tab in Palette.

Screenshot of the cluster metrics.

By using your custom pack, you have successfully deployed the Hello Universe application to the cluster.

Cleanup​

The following steps will guide you in cleaning up your environment. Follow the Palette-specific steps if you used Palette to deploy the cluster. Alternatively, use Terraform commands to delete the cluster if you used Terraform for deployment.

Delete the Cluster and Cluster Profile​

Navigate to the Cluster section in Palette's left Main Menu and access the details page for the pack-tutorial-cluster. To delete the cluster, click on the Settings button to expand the drop-down Menu, and select the Delete Cluster option. Palette will prompt you to enter the cluster name and confirm the deletion.

Screenshot of deleting the cluster in Palette.

The cluster status will display Deleting, and the deletion may take up to 10 minutes.

info

If a cluster remains in the delete phase for over 15 minutes, it becomes eligible for force deletion. Navigate to the cluster's details page and click on Settings. Select Force Delete Cluster. Palette automatically removes clusters that are stuck in the deletion phase for over 24 hours.

After deleting the cluster, proceed to delete the cluster profile. In the left Main Menu, click on Profiles and select the profile you want to delete. Next, click on the Delete option in the three-dot Menu.

Screenshot of deleting the profile in Palette.

Wait for the resources to complete cleanup and ensure they are successfully deleted.

Delete the Registry Server​

After deleting the cluster and cluster profile, navigate to the Tenant Settings > Registries > Pack Registries section in Palette to remove the registry configuration.

Screenshot of registry server delete in Palette

Now, delete the registry server. If you used the Spectro registry, stop the registry server by closing the tutorial container bash session that serves the Ngrok reverse proxy server. If you used the ECR registry, you must first remove the pack from the repository before deleting it.

Execute the following command to delete the pack from your ECR repository.

aws ecr batch-delete-image --repository-name $REGISTRY_NAME/spectro-packs/archive/$NAME --image-ids imageDigest=$(aws ecr describe-images --repository-name $REGISTRY_NAME/spectro-packs/archive/$NAME --region $AWS_DEFAULT_REGION --query 'imageDetails[0].imageDigest' --output text)

The snippet below displays the output of the aws ecr batch-delete-image command, confirming the deletion of the Hello Universe pack.

{
    "imageIds": [
        {
            "imageDigest": "sha256:<YourImageSha>",
            "imageTag": "1.0.0"
        }
    ],
    "failures": []
}

Next, proceed to delete the repositories.

aws ecr delete-repository --repository-name $REGISTRY_NAME/spectro-packs/archive
aws ecr delete-repository --repository-name $REGISTRY_NAME/spectro-packs/archive/$NAME

The output should provide information regarding the deleted repositories.

{
    "repository": {
        "repositoryArn": "arn:aws:ecr:us-east-1:<YourAccountId>:repository/spectro-oci-registry/spectro-packs/archive",
        "registryId": "<YourRegistryId>",
        "repositoryName": "spectro-oci-registry/spectro-packs/archive",
        "repositoryUri": "<YourAccountId>.dkr.ecr.us-east-1.amazonaws.com/spectro-oci-registry/spectro-packs/archive",
        "createdAt": "2023-11-28T16:23:20+00:00",
        "imageTagMutability": "MUTABLE"
    }

    "repository": {
        "repositoryArn": "arn:aws:ecr:us-east-1:<YourAccountId>:repository/spectro-oci-registry/spectro-packs/archive/hellouniverse",
        "registryId": "<YourRegistryId>",
        "repositoryName": "spectro-oci-registry/spectro-packs/archive/hellouniverse",
        "repositoryUri": "<YourAccountId>.dkr.ecr.us-east-1.amazonaws.com/spectro-oci-registry/spectro-packs/archive/hellouniverse",
        "createdAt": "2023-11-28T16:23:29+00:00",
        "imageTagMutability": "MUTABLE"
    }
}

Last, if you used a Basic registry, such as Harbor, make sure to delete your Harbor registry server.

At this point, you can close all the bash sessions. To remove the container and the image from your local machine, issue the following commands.

docker container rm --force tutorialContainer
docker image rm --force ghcr.io/spectrocloud/tutorials:1.1.2

Wrap-Up​

In this tutorial, you learned how to create a custom pack using manifest files. You packaged up an application in a custom pack that you pushed to a registry server and added to Palette.

Next, you created a cluster profile that included all the core infrastructure layers, such as the OS, Kubernetes distribution, and more. Additionally, you added your custom pack to the cluster profile, enabling your application to be deployed to a Kubernetes cluster.

Packs are the building blocks of cluster profiles, allowing you to customize your Kubernetes clusters. Palette enables you to use different packs to create multiple cluster profiles, each for specific purposes. As a result, you can ensure all Kubernetes deployments contain all the required dependencies and applications without developing complicated deployment scripts. All you need to do is maintain the cluster profiles.

To learn more about packs in Palette, we encourage you to check out the reference resources below.