Getting started

Instruction

Control plane vs worker nodes

Control Plane:

The control plane is installed on your master node
Can be both a control plane node and a worker node
It houses the API server, scheduler, and controller manager settings

Worker Nodes:

This is where the kubelet and kube-proxy are installed
You can use the kubeadm join command to join workers to the master node to form the cluster

First Test

New Pod

shell-demo.yaml

apiVersion: v1
kind: Pod
metadata:
  name: shell-demo
spec:
  volumes:
  - name: shared-data
    emptyDir: {}
  containers:
  - name: nginx
    image: nginx
    volumeMounts:
    - name: shared-data
      mountPath: /usr/share/nginx/html
  hostNetwork: true
  dnsPolicy: Default

Create a Pod

kubectl apply -f https://k8s.io/examples/application/shell-demo.yaml

Verify that the container is running

kubectl get pod shell-demo

# Get a shell to the running container
kubectl exec -it shell-demo -- /bin/bash

New Deployment

nginx-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Create a Deployment

kubectl apply -f https://k8s.io/examples/controllers/nginx-deployment.yaml

# Without a yaml file
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
EOF

Verify

kubectl get deployments
kubectl get pods --show-labels

Concepts

YAML File

~~myapp.yaml:~~

apiVersion: v1
kind: Pod
metadata:
  name: memory-demo
  namespace: mem-example
spec:
  containers:
  - name: memory-demo-ctr
    image: polinux/stress
    resources:
      requests:
        memory: "100Mi"
      limits:
        memory: "200Mi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "150M", "--vm-hang", "1"]

# Create a pod by applying a yaml file
kubectl apply -f myapp.yaml

# 建立一個部署用的初始 YAML 格式內容
kubectl create deploy nginx-deployment --image=nginx --dry-run=client -o yaml

~~Get the complete manifest of the deployed Kubernetes object.~~

kubectl get deploy nginx -n nginx-demo -o yaml
kubectl get svc <service-name> -n <name-space> -o yaml
kubectl get deployment <deployment-name> -n <name-space> -o yaml

Kubectl Configuration

kubectl version
kubectl cluster-info
kubectl config view
kubectl config view --minify --raw

Namespace & Node

~~Namespace 命名規則：名稱會用於內部的 subdomain，開頭與結尾必須字母數字，不可超過 253 個字元，只能包含小寫字母數字及 hyphen, dot 符號。~~

# Create a namespace mem-example
kubectl create namespace mem-example
kubectl get ns

# Check the nodes
kubectl get nodes
kubectl describe node

Pod

# Create a pod memory-demo
kubectl apply -f myapp.yaml

# Check the pods
kubectl get pods
kubectl get all -A
kubectl get pod memory-demo --namespace=mem-example
kubectl get pod memory-demo --output=yaml --namespace=mem-example
kubectl top pod memory-demo --namespace=mem-example
kubectl describe pod memory-demo --namespace=mem-example
# Get a list of pods and the node they run on
kubectl get po -o=custom-columns=NODE:.spec.nodeName,NAME:.metadata.name
# Get the name of the containers of a running pod
kubectl get pod MYPOD -o 'jsonpath={.spec.containers[*].name}'

# Delete a pod
kubectl delete pod memory-demo --namespace=mem-example

# Run Shell in the Pod
kubectl exec -it <pod-name> -n <namespace> -- bash

# Attach the pod
kubectl attach <pod-name> -n <namespace> -i

# Force to delete a pod
kubectl delete pod <pod-name> --force --grace-period=0

# Copy a file from a pod to a local file
kubectl cp <namespace>/<pod>:<file_path> <local_file_path>

Deployment

kubectl get deployments
kubectl rollout status deployment/nginx-deployment
kubectl describe deployment <deployment-name>

# Update a new image
kubectl set image deployment/nginx-deployment nginx=nginx:sometag

# Scale a deployment
kubectl scale deployment deployment --replicas=X

~~Delete a deployment~~

~~NOTE: 刪除 deployment 後，底層的其他相關資源，例如 container, PersistentVolumes, image, kubernetes secrets 仍然存在，必須手動移除。~~

# Delete the specified deployment
kubectl delete deployment <deployment name> --namespace <namespace name>

# Delete all deployments for the specified namespace
kubectl delete deployment --all --namespace=<namespace name>

# Using yaml file to delete the deployment
kubectl delete -f blog-deployment-1.yaml -f blog-deployment-2.yaml

# Using label selector
kubectl get deployment -l app=my-app
kubectl get all --selector app=[app-label]

kubectl delete deployment -l app=my-app
kubectl delete all --selector app=[app-label]

# Force-delete a hung deployment
kubectl delete deployment <deployment-name> --grace-period=0 --force
kubectl delete replicaset -l app=<label>
kubectl delete pods -l app=<label>

~~Deployment with health-check~~

~~initialDelaySeconds:~~

~~設定當 service 剛啟動時，要延遲幾秒再開始做 health check~~

~~periodSeconds: 代表每隔幾秒訪問一次，預設值為 10秒~~ ~~timeoutSeconds: 連線愈時~~ ~~successThreshold: 可以設置訪問幾次就代表目前 service 還正常運行~~ ~~failureThreshold: 連線失敗的次數~~

spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-deployment
...
    spec:
      containers:
      - name: webapp
        image: zxcvbnius/docker-demo
        ports:
        - name: webapp-port
          containerPort: 3000
        livenessProbe:
          httpGet:
            path: /
            port: webapp-port
          initialDelaySeconds: 15
          periodSeconds: 15
          timeoutSeconds: 30  
          successThreshold: 1
          failureThreshold: 3

Secret

~~變數的值以 yaml 格式輸出時，會以 base64 編碼顯示。~~

# Create a secret as environment variables
kubectl create secret generic [secret-name] \
  --from-literal=APP_API_KEY='your-api-key' \
  --from-literal=EXTERNAL_SERVICE_API_KEY='your-external-key' \
  -n [your-namespace]

~~Pull Docker Images from Private Docker Registries~~

# Create
# Usage: kubectl create secret docker-registry [secret-name] \
#  --docker-server:[address] \
#  --docker-username=[username] \
#  --docker-password=[password] \
#  -n [namespace]

# For Docker Hub Registry
kubectl create secret docker-registry dockerhub-pull-secret \
  --docker-server=docker.io \
  --docker-username=yourusername \
  --docker-password='Your-Personal-Key' \
  -n my-devops-prod

~~Check the secret~~

kubectl get secret [secret-name] -n [namespace] -o yaml

# get the value of a secret (if you have the base64 command available)
kubectl -n mynamespace get secret MYSECRET -o 'jsonpath={.data.DB_PASSWORD}' | base64 -d

~~字串轉 base64~~

# String to Base64
echo -n 'ThisIsSecretStrings' | base64

# Base64 to String
echo -n 'VGhpc0lzU2VjcmV0U3RyaW5ncw==' | base64 -d

~~secret.yaml~~

apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
type: Opaque
data:
  db-password: cm9vdHBhc3M=

~~deployment.yaml~~

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: mysql-server
...
    spec:
      containers:
      - name: mysql
        image: mysql:5.7
        args:
          - "--ignore-db-dir=lost+found"
        ports:
        - name: mysql-port
          containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: db-password
        volumeMounts:
...

PVC

~~PersistentVolumeClaim~~

kubectl get pvc -n <namespace>

kubectl delete pvc <pvc-name> -n <namespace>

~~掛載 node 端的本地磁碟~~

apiVersion: v1
kind: PersistentVolume
metadata:
  name: ollama-pv
  labels:
    type: local
spec:
  storageClassName: localvol-ollama
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/k8svol/ollama"

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ollama-pvc
  namespace: ollama
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
  storageClassName: localvol-ollama

Label

~~常用 Labels：~~

~~app: myapp~~ ~~release: stable, release: qa~~ ~~env: dev, env: prod~~ ~~tier: backend, tier: frontend~~ ~~depart: enginerring, depart: marketing~~

# 動態新增標籤
kubectl label pods <pod-name> <label-key>=<label-value>

# Show labels
kubectl get pods  --show-labels -n <namespace>

# Remove an existed label 'app=XXXX'
kubectl label pods <pod-name> app-

~~Annotations: 其他註解資訊~~

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  labels:
    app: webserver
    tier: backend   
  annotations:
    version: latest
    release_date: 2017/12/28
    contact: zxcvbnius@gmail.com
spec:
  containers:
...

Service

~~nodePort range: 3000 ~ 32767~~

# Check service
kubectl get svc -n <name-space>

# Create a service to expose the port of a pod
kubectl expose pod <pod-name> --port=<pod-port> --name=<service-name> -n <name-space>

# Create s service to expose a random port of a pod/deploy
kubectl expose pod <pod-name> --type=NodePort --name=<service-name> -n <name-space>
kubectl export deploy <deploy-name> --type=NodePort --name=<service-name> -n <name-space>

# Forward a port from a service to a local port
kubectl port-forward svc/<service-name> <local-port>:<remote-port>

ConfigMap

~~一個 ConfigMap 物件可以存入整個 configuration~~
~~像是 webserver config file, Nginx config file~~ ~~無需修改 container 程式碼，可以替換不同環境的 Config~~ ~~統一存放所有的 configuration~~ ConfigMap v.s Secret: 我們可以將機密的資料存在 Secret 中，且 Secret 會將這些值經過 Base64 加密，機密的資料像是 API 或是 database 的密碼；而將非機密但屬於部署面的資料放在 ConfigMap，好比資料庫的 port number 或是 Redis 的 config file。

# Create from a file
# kubectl create configmap <configmap-name> --from-file=/path/to/file
kubectl create configmap redis-config --from-file=my-redis.conf
kubectl create configmap nginx-conf --from-file=./my-nginx.conf

kubectl get configmap
kubectl describe configmap <configmap-name>

# Create with a command
# Usgae: kubectl create configmap <configmap-name> --from-literal=<key>=<value>
kubectl create configmap mysql-host --from-literal=ip=127.0.0.1

~~Apply to pod~~

apiVersion: v1
kind: Pod
metadata:
  name: apiserver
  labels:
    app: webserver
    tier: backend
spec:
  containers:
  - name: nodejs-app
    image: zxcvbnius/docker-demo
    ports:
    - containerPort: 3000
  - name: nginx
    image: nginx:1.13
    ports:
    - containerPort: 80
    volumeMounts:
    - name: nginx-conf-volume
      mountPath: /etc/nginx/conf.d
  volumes:
  - name: nginx-conf-volume
    configMap:
      name: nginx-conf
      items:
      - key: my-nginx.conf
        path: my-nginx.conf

ResourceQuota

~~對特定 namespace 限制 CPU/RAM 使用~~

apiVersion: v1
kind: ResourceQuota
metadata:
  name: quota-cpu-mem
spec:
  hard:
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi

~~限制物件使用，例如 secrets, services~~

apiVersion: v1
kind: ResourceQuota
metadata:
  name: object-quotas
  namespace: hellospace
spec:
  hard:
    services: "2"
    services.loadbalancers: "1"
    secrets: "1"
    configmaps: "1"
    replicationcontrollers: "10"

~~Apply to the namespace~~

kubectl apply -f quota-cpu-mem.yaml -n <namespace>

~~Check the resourcequota~~

kubectl get resourcequotas -n <namespace>

kubectl describe resourcequotas <resourcequotas-name> -n <namespace>

Node

# List nodes
kubectl get nodes
kubectl get nodes,pods,deploy -o wide

# Node 維護
kubectl drain <node-name>
kubectl drain <node-name> --force

# Node 恢復
kubectl uncordon <node-name>

Logs

kubectl logs -f deployment/<deployment-name> -n <name-space>
kubectl logs -f pod/<pod-name> -n <name-space>
kubectl logs -l app=xyz

環境變數

kubectl get deployment your-app-name -n your-namespace-prod -o yaml | grep EXTERNAL_SERVICE_URL

kubectl set env <resource>/<resource-name> --list

Networking

Inbound Rules for K3s Nodes

Protocol	Port	Source	Destination	Description
TCP	2379-2380	Servers	Servers	Required only for HA with embedded etcd
TCP	6443	Agents	Servers	K3s supervisor and Kubernetes API Server
UDP	8472	All nodes	All nodes	Required only for Flannel VXLAN
TCP	10250	All nodes	All nodes	Kubelet metrics
UDP	51820	All nodes	All nodes	Required only for Flannel Wireguard with IPv4
UDP	51821	All nodes	All nodes	Required only for Flannel Wireguard with IPv6
TCP	5001	All nodes	All nodes	Required only for embedded distributed registry (Spegel)
TCP	6443	All nodes	All nodes	Required only for embedded distributed registry (Spegel)

Typically, all outbound traffic is allowed.

Network access to other pods

Different Namespace: http://<service-name>.<namespace>:<port>
Same Namespace: http://<service-name>:<port>

Network access within the same pod

http://localhost:<port>
每個 container 有不一樣的 port