This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Tasks

1: Set Up Tools
2: Create a Project

3: Developer Guide

This section of the Datum documentation provides pages that guide you through specific tasks. Each task page focuses on a single objective, usually presented as a concise sequence of steps.

1 - Set Up Tools

Set up tools to work with Datum.

The Datum control plane is a collection of multiple projects developed with Kubernetes control plane technology, most of which can be installed into native Kubernetes clusters.

As a result, you will leverage common Kubernetes tooling such as kubectl to interact with Datum.

Install Tools

datumctl

Install datumctl with the Homebrew package manager:

brew install datum-cloud/tap/datumctl

kubectl

Refer to the official Kubernetes documentation for installation instructions, making sure to skip the Verify kubectl configuration section in the guide you choose.

Later in this guide, you will configure a kubeconfig file as required to interact with Datum via kubectl.

Create API Credentials

Sign in to Datum at https://cloud.datum.net
Create an API token by navigating to User Settings > API Tokens > Create a new token. Save this token in your password manager or preferred method of storage.

Configure Tools

Authentication

Configure datumctl authentication by activating the API token created in the previous section. Run the following command and enter your API token at the prompt:

datumctl auth activate-api-token

Add a kubeconfig context for your organization

Obtain your organization’s resource ID with datumctl by listing organizations that your user has access to:

datumctl organizations list

The output is similar to:

DISPLAY NAME           RESOURCE ID
Personal Organization  pp4zn7tiw5be3beygm2d6mbcfe

Create a kubeconfig context to access your organization’s resources by copying the the RESOURCE ID value and executing following command, replacing RESOURCE_ID with the value:

datumctl auth update-kubeconfig --organization RESOURCE_ID

The output is similar to:

Successfully updated kubeconfig at getting-started.kubeconfig

Verify kubectl configuration

Check that kubectl is properly configured by getting authorized user info:

kubectl auth whoami

The output is similar to:

ATTRIBUTE                                                VALUE
Username                                                 datum@example.com
Groups                                                   [system:authenticated]
Extra: authentication.datum.net/datum-organization-uid   [pp4zn7tiw5be3beygm2d6mbcfe]
Extra: authentication.kubernetes.io/credential-id        [JTI=01jgsr1m8fpb9cn0yrh05taa5v]

What’s next

Create a Project

2 - Create a Project

Create a Project via kubectl

Before you begin

This tutorial assumes you have already registered an account and have installed and configured the necessary tools to interact with Datum.

Confirm your kubeconfig context

Ensure that your kubectl tool is configured to use the correct context to interact with your organization by running the following command:

kubectl config current-context

The output is similar to:

datum-organization-pp4zn7tiw5be3beygm2d6mbcfe

Create a project

Write the following project manifest to intro-project.yaml, replacing RESOURCE_ID with your organization’s resource id.

Note that generateName is used here, which will result in a name with the prefix of intro-project- and a random suffix.

apiVersion: resourcemanager.datumapis.com/v1alpha
kind: Project
metadata:
  generateName: intro-project-
spec:

Create the project

kubectl create -f intro-project.yaml

The output is similar to:

project.resourcemanager.datumapis.com/intro-project-zj6wx created

Copy the generated project name, in this example it is intro-project-zj6wx.

Wait for the project’s control plane to become ready, which can take up to two minutes. Exit the command once the control plane status is Ready.

kubectl get projects -w

The output is similar to:

NAME                   AGE   CONTROL PLANE STATUS
intro-project-zj6wx   2s    APIServerProvisioning
intro-project-zj6wx   22s   ControllerManagerProvisioning
intro-project-zj6wx   43s   NetworkServicesOperatorProvisioning
intro-project-zj6wx   64s   WorkloadOperatorProvisioning
intro-project-zj6wx   106s   InfraProviderGCPProvisioning
intro-project-zj6wx   2m3s   Ready

Add a kubeconfig context for your project

Create a kubeconfig context to access your project’s resources by executing following command, replacing PROJECT_NAME with your project’s name:

datumctl auth update-kubeconfig --project PROJECT_NAME

Confirm that the control plane is accessible:

kubectl explain locations.spec

GROUP:      networking.datumapis.com
KIND:       Location
VERSION:    v1alpha

FIELD: spec <Object>


DESCRIPTION:
    LocationSpec defines the desired state of Location.

... continued

What’s next

Set up a Datum managed Location backed by GCP

3 - Developer Guide

This guide helps you get started developing Datum components.

Summary

This guide provides step-by-step instructions for setting up a development environment to install and run the Datum Cloud operators. It is targeted toward a technical audience familiar with Kubernetes, kubebuilder, and controller-runtime.

By following this guide, you will:

Install and configure necessary development tools.
Set up a kind cluster for access to a Kubernetes control plane.
Install and run the Workload Operator, Network Services Operator, and Infra Provider GCP components.
Configure and use Config Connector for managing GCP resources.
Register a Location and create a sample Datum Workload.

Prerequisites

Ensure the following are installed and properly configured:

Troubleshooting

If errors such as Command 'make' not found are encountered, reference the following guides for installing required build tools:

Control Plane Setup

Create Kind Cluster

Create a kind cluster for development:

kind create cluster --name datum

Install Third Party Operators

cert-manager

Install cert-manager:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml

Ensure that cert-manager pods are running and ready:

kubectl wait -n cert-manager --for=condition=Ready pod --all

The output is similar to:

pod/cert-manager-b6fd485d9-2s78z condition met
pod/cert-manager-cainjector-dcc5966bc-ntbw4 condition met
pod/cert-manager-webhook-dfb76c7bd-vxgb8 condition met

Refer to the cert-manager installation guide for more details.

GCP Config Connector

GCP Config Connector is used to manage Google Cloud resources directly from Kubernetes. The infra-provider-gcp application integrates with GCP Config Connector to create and maintain resources in GCP based on Kubernetes custom resources.

Tip

The service account creation instructions in the installation guide result in granting significantly more access to the GCP project than necessary. It is recommended to only bind the following roles to the service account:

roles/compute.admin
roles/container.admin
roles/secretmanager.admin
roles/iam.serviceAccountAdmin
roles/iam.serviceAccountUser

Follow the installation guide, making sure to retain the service account credential saved to key.json, as this will be required later by infra-provider-gcp. The target Kubernetes cluster will be the kind cluster created in this guide.

Note

The section “Specifying where to create your resources” can be skipped.

Datum Operator Installation

Clone the following repositories into the same parent folder for ease of use:

Note

The make commands can take some time to execute for the first time.

Workload Operator

In a separate terminal, navigate to the cloned workload-operator repository:
```
cd /path/to/workload-operator
```
Install CRDs:
```
make install
```
Start the operator:
```
make run
```

Network Services Operator

In a separate terminal, navigate to the cloned network-services-operator repository:
```
cd /path/to/network-services-operator
```
Install CRDs:
```
make install
```
Start the operator:
```
make run
```

Infra Provider GCP

In a separate terminal, navigate to the cloned infra-provider-gcp repository:
```
cd /path/to/infra-provider-gcp
```
Create an upstream.kubeconfig file pointing to the datum kind cluster. This extra kubeconfig file is required due to the operator’s need to orchestrate resources between multiple control planes. For development purposes, these can be the same endpoints.
```
kind export kubeconfig --name datum --kubeconfig upstream.kubeconfig
```
Start the operator after ensuring that the GOOGLE_APPLICATION_CREDENTIALS environment variable is set to the path for the key saved while installing GCP Config Connector.
```
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json
```
```
make run
```

Create Datum Resources

Register a Self Managed Location

Before creating a workload, a Location must be registered.

Use the following example manifest to create a location which Datum’s control plane will be responsible for managing, replacing GCP_PROJECT_ID with your GCP project id:

apiVersion: networking.datumapis.com/v1alpha
kind: Location
metadata:
  name: my-gcp-us-south1-a
spec:
  locationClassName: self-managed
  topology:
    topology.datum.net/city-code: DFW
  provider:
    gcp:
      projectId: GCP_PROJECT_ID
      region: us-south1
      zone: us-south1-a

Replace topology.datum.net/city-code’s value (DFW) with the desired city code for your workloads.
Update the gcp provider settings to reflect your GCP project ID, desired region, and zone.

Apply the manifest:

kubectl apply -f <path-to-location-manifest>

List Locations:

kubectl get locations

NAME                 AGE
my-gcp-us-south1-a   5s

Create a Network

Before creating a workload, a Network must be created. You can use the following manifest to do this:

Note

In the future, a default network may automatically be created in a namespace.

apiVersion: networking.datumapis.com/v1alpha
kind: Network
metadata:
  name: default
spec:
  ipam:
    mode: Auto

Apply the manifest:

kubectl apply -f <path-to-network-manifest>

List Networks:

kubectl get networks

NAME      AGE
default   5s

Create a Workload

Caution

These actions will result in billable resources being created in the GCP project for the target location. Destroy any resources which are not needed to avoid unnecessary costs.

Create a manifest for a sandbox based workload, for example:

apiVersion: compute.datumapis.com/v1alpha
kind: Workload
metadata:
  name: my-container-workload
spec:
  template:
    spec:
      runtime:
        resources:
          instanceType: datumcloud/d1-standard-2
        sandbox:
          containers:
            - name: httpbin
              image: mccutchen/go-httpbin
              ports:
                - name: http
                  port: 8080
      networkInterfaces:
        - network:
            name: default
          networkPolicy:
            ingress:
              - ports:
                - port: 8080
                from:
                  - ipBlock:
                      cidr: 0.0.0.0/0
  placements:
    - name: us
      cityCodes: ['DFW']
      scaleSettings:
        minReplicas: 1

Apply the manifest:

kubectl apply -f <path-to-workload-manifest>

Check the state of the workload

kubectl get workloads

The output is similar to:

NAME                    AGE   AVAILABLE   REASON
my-container-workload   9s    False       NoAvailablePlacements

The REASON field will be updated as the system progresses with attempting to satisfy the workload’s intent.

Check Workload Deployments

A Workload will result in one or more WorkloadDeployments being created, one for each unique CityCode per placement.

kubectl get workloaddeployments

The output is similar to:

NAME                           AGE   LOCATION NAMESPCE   LOCATION NAME        AVAILABLE   REASON
my-container-workload-us-dfw   58s   default             my-gcp-us-south1-a   False       LocationAssigned

Similar to workloads, the REASON field will be updated as the system progresses with attempting to satisfy the workload’s intent. In this case, the infra-provider-gcp operator is responsible for these actions.

Check Instances

kubectl -n default get instances -o wide

The output is similar to:

NAME                             AGE   AVAILABLE   REASON              NETWORK IP   EXTERNAL IP
my-container-workload-us-dfw-0   24s   True        InstanceIsRunning   10.128.0.2   34.174.154.114

Confirm that the go-httpbin application is running:

curl -s http://34.174.154.114:8080/uuid

{
  "uuid": "8244205b-403e-4472-8b91-728245e99029"
}

Delete the workload

Delete the workload when testing is complete:

kubectl delete workload my-container-workload