Deploy 4PointX IAoT Suite On K8S¶

Login to K8S Master VM and go to 4px Linux User:

sudo su - 4px

Modify the containerd.service configuration to allow Kubernetes to access the Docker registry.¶

Containerd Changes to connect with 4PX Image Repository

Open containerd Configuration file:
sudo vi /etc/containerd/config.toml

Edit/replace the configuration to access the insecure registry registry.4pointx.com:5000:*Depreciating Soon*

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = ""

  [plugins."io.containerd.grpc.v1.cri".registry.auths]

  [plugins."io.containerd.grpc.v1.cri".registry.configs]
    [plugins."io.containerd.grpc.v1.cri".registry.configs."registry.4pointx.com:5000".tls]
      insecure_skip_verify = true
    [plugins."io.containerd.grpc.v1.cri".registry.configs."localhost:5000".tls]
      insecure_skip_verify = true
  [plugins."io.containerd.grpc.v1.cri".registry.headers]

  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.4pointx.com:5000"]
      endpoint = ["https://registry.4pointx.com:5000"]

Restart containerd
sudo systemctl restart containerd
Repeat the above steps on all Kubernetes nodes.

Configuring Worker Nodes For OpenSearch.¶

Run the following command on each worker node:Ref Link: OS Installation Important Settings
Open the /etc/sysctl.conf file in your preferred text editor with superuser privileges. For example, using vi
sudo vi /etc/sysctl.conf
Add the following line to the file to set vm.max_map_count to 262144:
vm.max_map_count=262144
Save the file and reload the sysctl configuration to apply the changes:
sudo sysctl -p

Preparing SSL Certificate.¶

Before deploying 4pointX, ensure there is an SSL certificate. If not, create it using the following command, or rename .crt to __4pointx_com.crt and .key to server.key:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -subj "/C=IN/ST=KN/L=BN/O=EV/OU=4PX/CN=*.4pointx.com" -keyout server.key -out __4pointx_com.crt

Create 4px Namespace in the K8S Cluster.¶

Create the namespace from the K8S Master Node (VM):

kubectl create namespace 4px

Verify the namespace creation:

kubectl get namespace

OSD Secret.¶

Create OSD secret using the below command where __4pointx_com.crt & server.key is present.

kubectl create secret generic osd --from-file=__4pointx_com.crt --from-file=server.key -n 4px

Docker Registry secret¶

kubectl create secret docker-registry 4px-registry-creds  --docker-server=registry.4pointx.com:5000  --docker-username=""  --docker-password="" -n 4px

Deploying 4PX Setup¶

Create an Object Definition File 4px_setup.yaml
Example content for 4px_setup.yaml:Note: Change the Version and size of PV for the respective module.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-setup
  namespace: 4px
spec:
  replicas: 1
  selector:
    matchLabels:
      app: k8s-setup
  template:
    metadata:
      labels:
        app: k8s-setup
    spec:
      serviceAccount: k8s-setup
      serviceAccountName: k8s-setup
      containers:
      - name: setup-k8s
        image: registry.4pointx.com:5000/4pointx/setup:2017
        imagePullPolicy: Always
        env:
        - name: SUBDOMAIN
          value: "test1"
        - name: KI_HOST
          value: "osd-opensearch-dashboards"
        - name: REGISTRY_URL
          value: "registry.4pointx.com:5000"
        - name: VERSION
          value: "test"
        - name: KAFKA_BROKER_DATA_VOLUME_SIZE
          value: "4Gi"
        - name: KAFKA_BROKER_LOGS_VOLUME_SIZE
          value: "4Gi"
        - name: KAFKA_BROKER_REPLICAS
          value: "2"
        #The number of nodeports should be equal to the number of replicas.
        #The values should be separated by comma and in "".
        - name: KAFKA_BROKER_NODEPORT
          value: "32005, 32006"
        - name: KAFKA_BROKER_NODE_SERVICE_ADDRESS
          # Initiate value from kafka-broker-0 to kafka-broker-(numberOfKafkaBrokerReplicas-1).
          # Example value: "kafka-broker-0.kafka-broker-headless.4px.svc.cluster.local:9094, kafka-broker-1.kafka-broker-headless.4px.svc.cluster.local:9094"
          value: "kafka-broker-0.kafka-broker-headless.4px.svc.cluster.local:9094, kafka-broker-1.kafka-broker-headless.4px.svc.cluster.local:9094"
        - name: KAFKA_BROKER_HOST_NAME
          value: "34.125.120.4"
        - name: KAFKA_ZOOKEEPER_VOLUME_SIZE
          value: "4Gi"
        - name: KAFKA_ZOOKEEPER_REPLICAS
          value: "1"
        - name: OPENSEARCH_VOLUME_SIZE
          value: "35Gi"
        - name: OPENSEARCH_REPLICAS
          value: "2"
        - name: OPENSEARCH_HEAP_SIZE_MB
          value: "4096"
        - name: STORAGE_CLASS
          value: "longhorn"
        command: ["/bin/bash"]
        args:
        - "-c"
        - "/app/run.sh"
      imagePullSecrets:
      - name: 4px-registry-creds
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: k8s-setup
  namespace: 4px
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: k8s-setup-role
  namespace: 4px
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: k8s-setup-role-binding
  namespace: 4px
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: k8s-setup-role
subjects:
- kind: ServiceAccount
  name: k8s-setup
  namespace: 4px
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: k8s-setup-cluster-role
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: k8s-setup-cluster-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: k8s-setup-cluster-role
subjects:
- kind: ServiceAccount
  name: k8s-setup
  namespace: 4px
---
apiVersion: v1
kind: Service
metadata:
  name: k8s-setup
  namespace: 4px
spec:
  selector:
    app: k8s-setup
  type: NodePort
  ports:
    - protocol: TCP
      port: 80
      nodePort: 32700
      targetPort: 8090

Commands:

Create the 4px_setup.yaml file and paste the above content on the K8S Master Node.

Apply the 4px_setup.yaml file using kubectl:.
kubectl apply -f 4px_setup.yaml
Check the status of all modules.
kubectl get pods -n 4px

Note: By default, there will be 2 Node Kafka-broker and Opensearch, 1 Node Zookeeper and Opensearch-Dashboard. To scale up and scale down Refer To: the Resize 4PX Modules Page under the Administration Tab.

Testing the 4PX OSD¶

Test 4px OpenSearch Dashboards is deployed and running.

Run the following command on the K8S Master Node to check if it's accessible:

curl -vk https://localhost:32000