Ingress services on IBM Cloud

TCP ports (tcp-ports)

Access an app via a non-standard TCP port.

Description
Use this annotation for an app that runs a TCP streams workload.

The ALB operates in pass-through mode and forwards traffic to back-end apps. SSL termination is not supported in this case. The TLS connection is not terminated and passes through untouched.

The below was commented by Tong

If this is indeed the case, then this should be the ideal case for Fabric node setup. It should be desirable for the traffic being untouched between the client and the Fabric nodes.

Here is another section about TLS using ALB on IBM Cloud.

Step 4: Select TLS termination

After you map your custom domain, choose whether to use TLS termination.

The ALB load balances HTTP network traffic to the apps in your cluster. To also load balance incoming HTTPS connections, you can configure the ALB to decrypt the network traffic and forward the decrypted request to the apps that are exposed in your cluster.

SSL services support (ssl-services)

Allow HTTPS requests and encrypt traffic to your upstream apps.

Description
When your Ingress resource configuration has a TLS section, the Ingress ALB can handle HTTPS-secured URL requests to your app. By default, the ALB terminates the TLS termination and decrypts the request before using the HTTP protocol to forward the traffic to your apps. If you have apps that require the HTTPS protocol and need traffic to be encrypted, use the ssl-services annotation. With the ssl-services annotation, the ALB terminates the external TLS connection, then creates a new SSL connection between the ALB and the app pod. Traffic is re-encrypted before it is sent to the upstream pods.
If your back-end app can handle TLS and you want to add additional security, you can add one-way or mutual authentication by providing a certificate that is contained in a secret.

Ingress Nginx network settings

The following network settings are for a machine which hosts nginx.

sysctl -e -w fs.file-max=6000000;  
sysctl -e -w fs.nr_open=10000000;  
sysctl -e -w net.core.rmem_max=16777216;  
sysctl -e -w net.core.wmem_max=16777216;  
sysctl -e -w net.core.rmem_default=12582912;  
sysctl -e -w net.core.wmem_default=12582912;  
sysctl -e -w net.core.optmem_max=25165824;  
sysctl -e -w net.core.netdev_max_backlog=262144;  
sysctl -e -w net.core.somaxconn=32768;  
sysctl -e -w net.core.rps_sock_flow_entries=32768;  
sysctl -e -w net.ipv4.ip_local_port_range="1025 65535";  
sysctl -e -w net.ipv4.tcp_rmem="8192 262144 16777216";  
sysctl -e -w net.ipv4.tcp_wmem="8192 262144 16777216";  
sysctl -e -w net.ipv4.udp_rmem_min=16384;  
sysctl -e -w net.ipv4.udp_wmem_min=16384;  
sysctl -e -w net.ipv4.ip_no_pmtu_disc=0;  
sysctl -e -w net.ipv4.route.flush=1;  
sysctl -e -w net.ipv4.tcp_dsack=1;  
sysctl -e -w net.ipv4.tcp_sack=1;  
sysctl -e -w net.ipv4.tcp_fack=1;  
sysctl -e -w net.ipv4.tcp_max_tw_buckets=1440000;  
sysctl -e -w net.ipv4.tcp_tw_recycle=0;  
sysctl -e -w net.ipv4.tcp_tw_reuse=1;  
sysctl -e -w net.ipv4.tcp_frto=0;  
sysctl -e -w net.ipv4.tcp_syncookies=1;  
sysctl -e -w net.ipv4.tcp_max_syn_backlog=32768;  
sysctl -e -w net.ipv4.tcp_synack_retries=2;  
sysctl -e -w net.ipv4.tcp_syn_retries=3;  
sysctl -e -w net.ipv4.tcp_fin_timeout=5;  
sysctl -e -w net.ipv4.tcp_retries2=5;  
sysctl -e -w net.ipv4.tcp_no_metrics_save=1;  
sysctl -e -w net.ipv4.tcp_moderate_rcvbuf=1;  
sysctl -e -w net.ipv4.tcp_timestamps=1;  
sysctl -e -w net.ipv4.tcp_keepalive_time=300;  
sysctl -e -w net.ipv4.tcp_keepalive_intvl=30;  
sysctl -e -w net.ipv4.tcp_keepalive_probes=6;  
sysctl -e -w net.ipv4.tcp_slow_start_after_idle=0;  
sysctl -e -w net.ipv4.tcp_window_scaling=1;  
sysctl -e -w net.ipv4.tcp_low_latency=1;  
sysctl -e -w net.ipv4.tcp_max_orphans=262144;  
sysctl -e -w net.nf_conntrack_max=9145728;  
sysctl -e -w net.netfilter.nf_conntrack_max=9145728;  
sysctl -e -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=10;  
sysctl -e -w net.netfilter.nf_conntrack_tcp_timeout_fin_wait=10;  
sysctl -e -w net.netfilter.nf_conntrack_tcp_timeout_close_wait=30;  
sysctl -e -w net.netfilter.nf_conntrack_tcp_loose=1;  
sysctl -e -w net.ipv4.tcp_rfc1337=1;

Create k8s load balancer with nginx backend

Use the following yaml file to create a load balancer service on k8s. This load

balancer service will be backed by the deployment which only consists replicate

set with 2 containers using nginx.

---
apiVersion: v1
kind: Service
metadata:
  name: my-service8080
spec:
  selector:
    app: my-nginx
  ports:
    - port: 8080
      targetPort: 80
  type: LoadBalancer
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      app: my-nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80

The above picked a port 8080 to access the service, the application actually runs on port 80. In many
other applications, the port may be other numbers but the service port can be always specified to maintain a consistent endpoint for other applications.

Once the service and the deployment were created successfully, you can use the following methods to get url to hit the service.

*

   kubectl get svc my-service8080

The above should return information which contains external-ip with ports, the combination of the two will make up the url to hit the service.

**

   kubectl get svc my-service8080 -o yaml

This will return a yaml file, showing port and the hostname which will also make up the url to hit the service.

How to get kubeconfig file from a k8s cluster running on IBMCloud

Once your k8s cluster is up running, follow the below steps to get kubeconfig file.


1. install ibmcloud CLI tools if you do not already have it
   curl -sL https://ibm.biz/idt-installer | bash

2. install ibmcloud CLI ks plugin
   ibmcloud plugin install kubernetes-service

3. login to the account
   ibmcloud login -a cloud.ibm.com -r us-south --sso
   -u <id@us.ibm.com> -p <password> -g Default

this step will ask you to get a one-time passcode by providing a url you can copy and paste the url to a browser to get one-time passcode to proceed. Once you provide the passcode, it will ask you to select an account to continue.

4. retrieve the kubeconfig file, the kubeconfig file should be saved at the current directory.
   ibmcloud ks cluster config --cluster <cluster-id>

Note:

* To check what plugins are available for CLI, run the following command
ibmcloud plugin repo-plugins -r 'IBM Cloud'

** The kubeconfig should normally reference a cert .pem file, when move this kubeconfig file to another environment, the referenced pem file should also be moved.