what is happening when istio gets installed?

After using istioctl install command to install istio, then run the uninstall command to remove istio from the cluster, many resources will be removed, that also means, the install process created these resources during the install

When use this command to remove istio,

istioctl x uninstall --purge

The following things will happen:

  Removed IstioOperator:istio-system:installed-state.
  Removed HorizontalPodAutoscaler:istio-system:istio-ingressgateway.
  Removed HorizontalPodAutoscaler:istio-system:istiod.
  Removed PodDisruptionBudget:istio-system:istio-ingressgateway.
  Removed PodDisruptionBudget:istio-system:istiod.
  Removed Deployment:istio-system:istio-ingressgateway.
  Removed Deployment:istio-system:istiod.
  Removed Service:istio-system:istio-ingressgateway.
  Removed Service:istio-system:istiod.
  Removed ConfigMap:istio-system:istio.
  Removed ConfigMap:istio-system:istio-sidecar-injector.
  Removed Pod:istio-system:istio-ingressgateway-6968d58d88-9dq7k.
  Removed Pod:istio-system:istiod-74d4864d8d-psjs8.
  Removed ServiceAccount:istio-system:istio-ingressgateway-service-account.
  Removed ServiceAccount:istio-system:istio-reader-service-account.
  Removed ServiceAccount:istio-system:istiod-service-account.
  Removed RoleBinding:istio-system:istio-ingressgateway-sds.
  Removed RoleBinding:istio-system:istiod-istio-system.
  Removed Role:istio-system:istio-ingressgateway-sds.
  Removed Role:istio-system:istiod-istio-system.
  Removed EnvoyFilter:istio-system:metadata-exchange-1.10.
  Removed EnvoyFilter:istio-system:metadata-exchange-1.9.
  Removed EnvoyFilter:istio-system:stats-filter-1.10.
  Removed EnvoyFilter:istio-system:stats-filter-1.9.
  Removed EnvoyFilter:istio-system:tcp-metadata-exchange-1.10.
  Removed EnvoyFilter:istio-system:tcp-metadata-exchange-1.9.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.10.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.9.
  Removed MutatingWebhookConfiguration::istio-sidecar-injector.
  Removed ValidatingWebhookConfiguration::istiod-istio-system.
  Removed ClusterRole::istio-reader-istio-system.
  Removed ClusterRole::istiod-istio-system.
  Removed ClusterRoleBinding::istio-reader-istio-system.
  Removed ClusterRoleBinding::istiod-istio-system.
  Removed CustomResourceDefinition::authorizationpolicies.security.istio.io.
  Removed CustomResourceDefinition::destinationrules.networking.istio.io.
  Removed CustomResourceDefinition::envoyfilters.networking.istio.io.
  Removed CustomResourceDefinition::gateways.networking.istio.io.
  Removed CustomResourceDefinition::istiooperators.install.istio.io.
  Removed CustomResourceDefinition::peerauthentications.security.istio.io.
  Removed CustomResourceDefinition::requestauthentications.security.istio.io.
  Removed CustomResourceDefinition::serviceentries.networking.istio.io.
  Removed CustomResourceDefinition::sidecars.networking.istio.io.
  Removed CustomResourceDefinition::telemetries.telemetry.istio.io.
  Removed CustomResourceDefinition::virtualservices.networking.istio.io.
  Removed CustomResourceDefinition::workloadentries.networking.istio.io.
  Removed CustomResourceDefinition::workloadgroups.networking.istio.io.

After istioctl operator init, there are these things created. The list is created when using the uninstall --purge.

  Removed Deployment:istio-operator:istio-operator.
  Removed Service:istio-operator:istio-operator.
  Removed ServiceAccount:istio-operator:istio-operator.
  Removed ClusterRole::istio-operator.
  Removed ClusterRoleBinding::istio-operator.
  Removed CustomResourceDefinition::istiooperators.install.istio.io.

After do the following:

kubectl apply -f - <<EOF

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-operator
  name: example-istiocontrolplane
spec:
  profile: default
EOF

If remove everything, these are the things will be removed.

  Removed IstioOperator:istio-system:example-istiocontrolplane.
  Removed HorizontalPodAutoscaler:istio-system:istio-ingressgateway.
  Removed HorizontalPodAutoscaler:istio-system:istiod.
  Removed PodDisruptionBudget:istio-system:istio-ingressgateway.
  Removed PodDisruptionBudget:istio-system:istiod.
  Removed Deployment:istio-operator:istio-operator.
  Removed Deployment:istio-system:istio-ingressgateway.
  Removed Deployment:istio-system:istiod.
  Removed Service:istio-operator:istio-operator.
  Removed Service:istio-system:istio-ingressgateway.
  Removed Service:istio-system:istiod.
  Removed ConfigMap:istio-system:istio.
  Removed ConfigMap:istio-system:istio-sidecar-injector.
  Removed Pod:istio-system:istio-ingressgateway-6968d58d88-wcmvt.
  Removed Pod:istio-system:istiod-84cb7c8f48-7q6rx.
  Removed ServiceAccount:istio-operator:istio-operator.
  Removed ServiceAccount:istio-system:istio-ingressgateway-service-account.
  Removed ServiceAccount:istio-system:istio-reader-service-account.
  Removed ServiceAccount:istio-system:istiod-service-account.
  Removed RoleBinding:istio-system:istio-ingressgateway-sds.
  Removed RoleBinding:istio-system:istiod-istio-system.
  Removed Role:istio-system:istio-ingressgateway-sds.
  Removed Role:istio-system:istiod-istio-system.
  Removed EnvoyFilter:istio-system:metadata-exchange-1.10.
  Removed EnvoyFilter:istio-system:metadata-exchange-1.9.
  Removed EnvoyFilter:istio-system:stats-filter-1.10.
  Removed EnvoyFilter:istio-system:stats-filter-1.9.
  Removed EnvoyFilter:istio-system:tcp-metadata-exchange-1.10.
  Removed EnvoyFilter:istio-system:tcp-metadata-exchange-1.9.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.10.
  Removed EnvoyFilter:istio-system:tcp-stats-filter-1.9.
  Removed MutatingWebhookConfiguration::istio-sidecar-injector.
  Removed ValidatingWebhookConfiguration::istiod-istio-system.
  Removed ClusterRole::istio-operator.
  Removed ClusterRole::istio-reader-istio-system.
  Removed ClusterRole::istiod-istio-system.
  Removed ClusterRoleBinding::istio-operator.
  Removed ClusterRoleBinding::istio-reader-istio-system.
  Removed ClusterRoleBinding::istiod-istio-system.
  Removed CustomResourceDefinition::authorizationpolicies.security.istio.io.
  Removed CustomResourceDefinition::destinationrules.networking.istio.io.
  Removed CustomResourceDefinition::envoyfilters.networking.istio.io.
  Removed CustomResourceDefinition::gateways.networking.istio.io.
  Removed CustomResourceDefinition::istiooperators.install.istio.io.
  Removed CustomResourceDefinition::peerauthentications.security.istio.io.
  Removed CustomResourceDefinition::requestauthentications.security.istio.io.
  Removed CustomResourceDefinition::serviceentries.networking.istio.io.
  Removed CustomResourceDefinition::sidecars.networking.istio.io.
  Removed CustomResourceDefinition::telemetries.telemetry.istio.io.
  Removed CustomResourceDefinition::virtualservices.networking.istio.io.
  Removed CustomResourceDefinition::workloadentries.networking.istio.io.
  Removed CustomResourceDefinition::workloadgroups.networking.istio.io. 

In operator case, the deployment of istio-operator uses image docker.io/istio/operator:1.10.2 which deploys into namespace istio-operatorby default. it will only create the operator crd, only when control plan gets created by using the following command:

kubectl apply -f - <<EOF
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-system
  name: example-istiocontrolplane
spec:
  profile: default
EOF

Then all other crds will be created. The deployment of istiod will use docker.io/istio/pilot:1.10.2, istiod (or pilot) is now only watching the cluster, does the certificate and configuration. It is not doing what istio operator does which is to accept crd then convert them to various k8s resources. If just use istioctl, then there is no operator to interpret these requests, istioctl will convert all the request and create k8s resources, vs in operator case, it is the operator takes the request and create k8s resources.

Regardless using istioctl or istio operator or helm, the istiod will have to be deployed

Use protoc to generate go code using protobuf

This example is based on istio.io/api project to generate all the go code based on provided proto files in various sub directories.

 

protoc --proto_path=$(pwd) --go_out=$(pwd) --go_opt=paths=source_relative networking/**/*.proto security/**/*.proto type/**/*.proto analysis/**/*.proto authentication/**/*.proto meta/**/*.proto telemetry/**/*.proto

Rebase with upstream master branch

1. Make sure that the upstream repo is added to your configuration.

2. Do a git fetch

git fetch upstream

3. Do rebase

git rebase --interactive upstream/master

4. Push to remote

git push -f 

Install tekton and its dashboard on IBM Cloud

1. Install tekton 0.21.0

kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.21.0/release.yaml

2. Install latest tekton dashboard

kubectl apply --filename https://storage.googleapis.com/tekton-releases/dashboard/latest/openshift-tekton-dashboard-release.yaml --validate=false

3. Expose the dashboard UI

kubectl patch svc tekton-dashboard -n tekton-pipelines -p '{"spec": {"type": "LoadBalancer"}}'

4. Access the dashboard by using the external IP (hostname) and port 9097

kubectl get svc tekton-dashboard -n tekton-pipelines

The external IP or hostname may take a bit of time to be available.

 

For tectoncd/pipeline development, to use ko, set the docker repo env to be the full path like below

export KO_DOCKER_REPO=registry.hub.docker.com/email4tong

 The short name no longer works.

Expose TCP traffic examples

After istio is installed, follow these steps:

0. Label the default namespace for istio sidecar injection

kubectl label namespace default istio-injection=enabled --overwrite

1. Patch istio-ingressgateway service so that the new port is supported.

Create a file named patch-service.yaml with the following content:

spec:
  ports:
  - name: tcp-31400
    protocol: TCP
    port: 31400
    targetPort: 31400

Run the following command

kubectl -n istio-system patch service istio-ingressgateway --patch "$(cat patch-service.yaml)"  

2. Create deployment, service, gateway and virtual service

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-deployment
spec:
  selector:
    matchLabels:
      greeting: hello
      department: world
  replicas: 1
  template:
    metadata:
      labels:
        greeting: hello
        department: world
    spec:
      containers:
      - name: hello
        image: "email4tong/pathecho:latest"
        imagePullPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world
spec:
  selector:
    greeting: hello
    department: world
  ports:
  - protocol: TCP
    port: 7000
    targetPort: 8080
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: tcp-echo-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 31400
      name: tcp
      protocol: TCP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: tcp-echo
spec:
  hosts:
  - "*"
  gateways:
  - tcp-echo-gateway
  tcp:
  - match:
    - port: 31400
    route:
    - destination:
        host: hello-world.default.svc.cluster.local
        port:
          number: 7000

3. Now use the istio-ingressgateway service external endpoint (IP or hostname) and port 31400 to access the service. In above example, it is a simple http echo, so use curl to test is fine. If the actual service is not http but using any other tcp protocols, then you cannot use curl to test

Istio tcp ingress traffic

 To allow tcp traffic on a specific port using Istio, one will have to patch the istio-ingressgateway service to add the new port if the port is not 80 or 443. Then one will have to create a gateway, a virtual service, and also make sure that the deployments have the sidecar injected automatically. So to summarize:

1. Patch istio-ingressgateway to add new tcp port

2. Create gateway resource

3. Create virtual service resource

4. Make sure that the actual pod is injected with sidecar.

It is very important that the tcp port to be added before the gateway and virtual service resource were created, otherwise, it wont work.

k8s ambassador and tls cert management

 With brand new google k8s, do the followings:

1. Set up kubeconfig by using the google connect command found from google cloud console, something similar to this.

gcloud container clusters get-credentials tongedge --zone us-east1-b --project odin-network-301700

2. Run the following command to deploy Ambassador

kubectl apply -f https://www.getambassador.io/yaml/aes-crds.yaml &&
kubectl wait --for condition=established --timeout=90s crd -lproduct=aes&&
kubectl apply -f https://www.getambassador.io/yaml/aes.yaml &&
kubectl -n ambassador wait --for condition=available --timeout=90s deploy -lproduct=aes

Now, get the external ip address

 kubectl get -n ambassador service ambassador -o
 "go-template={{range .status.loadBalancer.ingress}}{{or .ip .hostname}}{{end}}"

3. Deploy kubernetes cert-manager

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.1.0/cert-manager.yaml 

Verify cert manager is working correctly

kubectl get pods --namespace cert-manager

k8s cert manager reference link https://cert-manager.io/docs/installation/kubernetes/ 

Now create a mapping and a service to make sure cert-manager created ingress can meet the http01 chanllenge.

---
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
  name: acme-challenge-mapping
spec:
  prefix: /.well-known/acme-challenge/
  rewrite: ""
  service: acme-challenge-service
---
apiVersion: v1
kind: Service
metadata:
  name: acme-challenge-service
spec:
  ports:
  - port: 80
    targetPort: 8089
  selector:
    acme.cert-manager.io/http01-solver: "true"

The ambassador external ip now can be used to configure your dns entry via whatever the dns providers. The end result will be your domain name can be resolved into the ambassador external IP address.
To request a certificate, one has to set up an issuer first, using the follow yaml file to accomplish that.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    preferredChain: "ISRG Root X1"
    privateKeySecretRef:
      name: letencrypt-secret
    solvers:
    - http01:
        ingress:
          class: nginx
      selector: {}

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: tonglitls
  namespace: default
spec:
  dnsNames:
    - tongli.myddns.me
  secretName: tonglitls-secret
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt