Download Talos Linux v1.13.x

Go to https://factory.talos.dev/
Select Cloud Server > select V1.13.x > Next > select Nocloud > Next
Select amd64, disable SecureBoot > Next
Add System Extensions: qemu-guest-agent and util-linux-tools > Next
Select auto Bootloader > Next > Download ISO

Setup Proxmox Vm for Talos

Remove Pre-Enroll keys

Prepare networking

Add DHCP reservation and firewall to ensure the IP of the Control Plane and Work Nodes is fixed

Set context

Prepare Management Machine – Ubuntu Core 26

Create an account at https://login.ubuntu.com/
Go to My account > SSH key > Import your own ssh public key
Browse to Browse to https://cdimage.ubuntu.com/ubuntu-core/26/stable/current/ to identify latest version
In Proxmox GUI: create a new VM with basic hardware: 1 socket, 2 cores, 2gb memory, no disk
In Proxmox CLI:

# Download Ubuntu Core 26 image
wget https://cdimage.ubuntu.com/ubuntu-core/26/stable/current/ubuntu-core-26-amd64.img.xz

# Extract image
unxz ubuntu-core-26-amd64.img.xz

# Import extracted image to VM
qm importdisk <VM-ID> ubuntu-core-26-amd64.img local-lvm
In Proxmox GUI: select <VM-ID>, select the imported Unused disk > Edit > Check Discard, SSD emulation > Add
Adjust boot order and start VM
Configure your network, login with your Ubuntu account

Install Required Management Tools in Ubuntu Core

Logon to Ubuntu Core IP address using ssh key

ssh username@ubuntu_core_ip

Download kubectl, talosctl, kustomize, cilium, helm, argocd, yq and copy to Ubuntu Core

# from Windows terminal - ensure your ssh key id_ed25519 are at ./ssh
scp -r .downloadedbin* username@ubuntu_core_ip:/home/username/bin/

# from Ubuntu terminal - make these files executable
chmod +x ~/bin/*

# Add ~/bin to your active environment path
echo 'export PATH="$HOME/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Configure the cluster control plane

Start talos Control Plane and Workers

Set up environment variables

# Setup some environment variables
export CLUSTER_NAME=cluster_name
export CONTROL_PLANE_IP=192.168.0.1

# Find out what disks are on the server
talosctl get disks --insecure --nodes "$CONTROL_PLANE_IP"
export DISK_NAME=sda # name of your k8s disk

Create cluster patch file for Celium setup later

# cluster-patch.yaml
cluster:
  network:
    cni:
      name: none
  proxy: # Disable kube-proxy, Cilium will replace it too
    disabled: true

Generate talos configuration for new cluster

talosctl gen config $CLUSTER_NAME "https://$CONTROL_PLANE_IP:6443" --install-disk "/dev/$DISK_NAME" --config-patch @cluster-patch.yaml

The following files are created:
- controlplane.yaml
- worker.yaml
- talosconfig

# set environment variable
export TALOSCONFIG="$(pwd)/talosconfig"

Create Control Plane patch file

# controlplane-1-patch.yaml
apiVersion: v1alpha1
kind: HostnameConfig
auto: off # A method to automatically generate a hostname for the machine.

# A static hostname to set for the machine.
hostname: controlplane.example.com

Apply talos configuration to our new cluster

# Apply the configuration
talosctl apply-config --insecure 
  --nodes $CONTROL_PLANE_IP 
  --file controlplane.yaml 
  --insecure 
  --config-patch @controlplane-1-patch.yaml

# Wait for talos control plane pull image, install, bootup and request for `talosctl bootstrap`

Bootstrap etcd in the cluster

# Bootstrap etcd in the cluster
talosctl --talosconfig ./talosconfig config endpoint "$CONTROL_PLANE_IP"
talosctl bootstrap --nodes "$CONTROL_PLANE_IP" --talosconfig=./talosconfig

Create the Kubeconfig

# Verify the Context
talosctl config contexts

#--- BEGIN SET CONTEXT (only if context is not set)
# If no context set
talosctl kubeconfig $CLUSTER_NAME-kubeconfig --nodes $CONTROL_PLANE_IP --talosconfig=./talosconfig
export KUBECONFIG="$(pwd)/$CLUSTER_NAME-kubeconfig"

# Merge local file into the global config
talosctl config merge ./talosconfig
#---  END SET CONTEXT

# Set default endpoint and node in talosconfig
talosctl config endpoint $CONTROL_PLANE_IP

# Set the default node (optional, saves time for future commands)
talosctl config node $CONTROL_PLANE_IP

# Generate the Kubeconfig at $HOME/.kube/config
talosctl kubeconfig -n $CONTROL_PLANE_IP

# Check Dashboard
talosctl --talosconfig ./talosconfig dashboard --nodes $CONTROL_PLANE_IP
# Check health
talosctl health --endpoints $CONTROL_PLANE_IP --nodes $CONTROL_PLANE_IP

Install Cilium

Install by Helm

Install the CRDs

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_grpcroutes.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml

Confirm the gateway classes are present

kubectl get crd gatewayclasses.gateway.networking.k8s.io gateways.gateway.networking.k8s.io httproutes.gateway.networking.k8s.io

Setup the Cilium helm repo

# Add repo
helm repo add cilium https://helm.cilium.io/
helm repo update

# List available versions
helm search repo cilium/cilium --versions | head -10

# Install Cilium
helm install cilium cilium/cilium --version 1.19.4 --namespace kube-system

# Update config 
helm upgrade cilium cilium/cilium --version 1.19.4 
  --namespace kube-system 
  --set ipam.mode=kubernetes 
  --set kubeProxyReplacement=true 
  --set securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" 
  --set securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" 
  --set cgroup.autoMount.enabled=false 
  --set cgroup.hostRoot=/sys/fs/cgroup 
  --set k8sServiceHost=$CONTROL_PLANE_IP 
  --set k8sServicePort=6443 
  --set=l2announcements.enabled=true 
  --set=externalIPs.enabled=true 
  --set gatewayAPI.enabled=true 
  --set=devices=e+ 
  --set=operator.replicas=1

Install Cilium by Kustomize helmCharts and values.yaml

base/kustomize.yaml

# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources: []

helmCharts:
  - name: cilium
    releaseName: cilium
    version: 1.19.4
    repo: https://helm.cilium.io/
    namespace: kube-system
    valuesFile: cilium-values.yaml
    includeCRDs: true

base/cilium-values.yaml

# cilium-values.yaml - Production Cilium values for Helm installation
# IPAM configuration
ipam:
  mode: kubernetes

# Enable kube-proxy replacement for better performance
kubeProxyReplacement: true
k8sServiceHost: <CONTROL_PLANE_IP>  # control plane IP address
k8sServicePort: 6443

securityContext:
  capabilities:
    ciliumAgent:
      - CHOWN
      - KILL
      - NET_ADMIN
      - NET_RAW
      - IPC_LOCK
      - SYS_ADMIN
      - SYS_RESOURCE
      - DAC_OVERRIDE
      - FOWNER
      - SETGID
      - SETUID
    cleanCiliumState:
      - NET_ADMIN
      - SYS_ADMIN
      - SYS_RESOURCE

cgroup:
  autoMount:
    enabled: false
  hostRoot: /sys/fs/cgroup

l2announcements:
  enabled: true

externalIPs:
  enabled: true

gatewayAPI:
  enabled: true

devices: "e+"

operator:
  replicas: 1

# Enable Hubble for observability
hubble:
  enabled: true
  relay:
    enabled: true
    replicas: 2
  ui:
    enabled: true
    replicas: 1
  metrics:
    enabled:
      - dns
      - drop
      - tcp
      - flow
      - port-distribution
      - icmp
      - httpV2

# Enable eBPF masquerading
bpf:
  masquerade: true

# Resource limits for Cilium agents
resources:
  requests:
    cpu: 200m
    memory: 1Gi
  limits:
    cpu: 1000m
    memory: 2Gi

# Enable Prometheus metrics
#prometheus:
#  enabled: true
#  port: 9962
#  serviceMonitor:
#    enabled: true
#    labels:
#      release: kube-prometheus-stack

./cilium/overlays/prod/kustomization.yaml

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: kube-system

resources:
  - ../../base

# Build with kustomize
kustomize build ./cilium/overlays/prod --enable-helm | kubectl apply -f -

Add a worker node

Create another VM

Add it to the cluster

Create a patch file with the node name you want

# worker-1-patch.yaml
apiVersion: v1alpha1
kind: HostnameConfig
auto: off # A method to automatically generate a hostname for the machine.

# # A static hostname to set for the machine.
# hostname: controlplane1
hostname: worker1.example.com

When the worker node gets to Maintenance

# Set environment variable for worker1
export WORKER1_IP=192.168.0.51
# Apply configuration
talosctl apply-config --insecure --nodes "$WORKER1_IP" --file worker.yaml --config-patch @worker-1-patch.yaml

Create tls secret

sudo kubectl --kubeconfig=$HOME/.kube/config create secret tls wildcard-example-tls 
  --cert=/etc/ssl/private/example.com.crt 
  --key=/etc/ssl/private/example.com.key 
  -n default

Shut it down

# Nodes name
kubectl get nodes
kubectl drain <node1> --ignore-daemonsets --delete-emptydir-data
talosctl shutdown --nodes $WORKER1_IP
kubectl drain <controlplane_name> --ignore-daemonsets --delete-emptydir-data
talosctl shutdown --nodes $CONTROL_PLANE_IP

References

https://unixorn.github.io/post/homelab/k8s/01-talos-with-cilium-cni-on-proxmox

Install and Setup Talos Linux on Proxmox