go tool pprof -http=:8888 localhost:8080/debug/pprof/heapOne can use a specific IP address to allow access from outside of the machine which is running the tool
go tool pprof -http=192.168.56.32:8888 localhost:8080/debug/pprof/heap2. When running in the server env. -no_browser option probably will be nice to avoid the warning messages from the process.
go tool pprof -no_browser -http=192.168.56.32:8888 localhost:8080/debug/pprof/heap
Assume that you've setup your istio project in VSCode, and also run go mod vendor,
you can do the following to debug step-by-step in VSCode.
1. Create a kind k8s cluster
2. Create an Istio integration k8s cluster topology file named single.json, like this.
[
{
"kind": "Kubernetes",
"clusterName": "istio-testing",
"network": "istio-testing",
"meta": {
"kubeconfig": "/home/ubuntu/.kube/config"
}
}
]
Notice that the kubeconfig field, the value should be the kube config file
3. Now in VSCode, make sure that you have the following in your settings.
"go.buildTags": "integ",
"go.testFlags": ["-args", "--istio.test.kube.topology=/home/ubuntu/test/single.json", "--istio.test.skipVM"],
Now if you nagivate to an integration test go file in VSCode, you should be able to click on the codelens `debug test` to start debugging your code.
=================================================
For multiple cluster integration tests, the following few items will need to be taken care of:
1. Create multiple clusters using this script with a topology json file
2. To use the code that you as a developer just built (such as istioctl or docker images such as pilot and proxyv2), you will need to make sure that these images get preloaded into the clusters created in the above step #1. If you use the scripts described in that step, then the newly built images should be loaded onto the cluster automatically.
3. In each test setup (most likely in TestMain method), you will need to setup how the tag and hub should be so that the process will use these images correctly. Otherwise, the process will use the public images not the ones that you just built.
To do this, you most likely will need to do the followings:
a. Create a new method like this
b. Then call that method in the TestMain method's Setup call like this. Notice one of the Setup method call uses the method enableMCSServiceDiscovery which got defined above.
4. Make sure that VSCode settings file, contains the go.buildTags, go.testFlags settings like the followings
5. Once these above steps done, you can simply click on the debug test button (the codelens) above the MainTest method.
====================================================
If want to run the test locally (not via vscode codelens), you can do the following:
1. Found a PR from istio project on github which run successfully, there should be many integration tests, such as the following:
Save the above content into a file such as topology.json
2. Then you should be able to find a command like the following:
3. Change the parameters of the above command to fit your own env. Pay special attention to parameters like istio.test.tag, istio.test.hub, making changes based on your own build. In the above command, I built istio images locally and tagged them like public istio images, and preloaded into the k8s clusters, so that everything is ready to go.
4. The parameter ./tests/integration/pilot/... indicates what tests will be run, that must be a directory from the source tree. It will normally contain multiple TestMain methods, each TestMain method is considered as test suite. When it starts, you should see something like the following:
that should give you some clear indication what test suite it is running. If there is any error, you can find that test suite in the source code and start debugging using VSCode. Notice that normally one integration test may contain many test suites, that is, as stated above, many TestMain methods in that directory or sub directory.
Run the following command to show which folder uses spaces exceeded G bits
du -h ~ 2>/dev/null | grep '[0-9\.]\+G'
Envoy can use a set of APIs to update configurations, without any downtime or restart. Envoy only needs a simple bootstrap configuration file, which directs configurations to the proper discovery service API. Other settings are dynamically configured. Envoy's dynamic configuration APIs are called xDS services and they include:
You can use one or more xDS services for configuration. Envoy's xDS APIs are designed for eventual consistency, and proper configurations are eventually converged. For example, Envoy may eventually use a new route to retrieve RDS updates, and this route may forward traffic to clusters that have not yet been updated in the CDS. As a result, the routing process may produce routing errors until the CDS is updated. Envoy introduces the ADS to solve this problem. Istio also implements the ADS, which can be used to modify proxy configurations.
The definition of service mesh
A service mesh is a distributed application
infrastructure that is responsible for handling network traffic on
behalf of the application in a transparent, out of process manner.
Data Plane and Control Plane
The service proxies form the "data plane" through which all traffic is handled and observed. The data plane is responsible for establishing, securing, and controlling the traffic through the mesh. The management components that instruct the data plane how to behave is known as the "control plane". The control plane is the brains of the mesh and exposes an API for operators to manipulate the network behaviors. Together, the data plane and the control plane provide important capabilities necessary in any cloud-native architecture such as:
With a service proxy next to each application instance, applications no
longer need to have language-specific resilience libraries for circuit
breaking, timeouts, retries, service discovery, load balancing, et. al.
Moreover, the service proxy also handles metric collection, distributed
tracing, and log collection.
To get all the pods from a namespace,
curl -k --cacert ca.crt -H "Authorization: Bearer <The token>" https://172.19.0.3:6443/api/v1/namespaces/metallb-system/pods
Where the IP address and port should be the k8s api server IP and port, then the url should follow the naming convention which should be always
/api/<version>/namespaces/<namespace>/<resourcetype>
in the example above, the version is v1, namespace is metallb-system and we are trying to get all the pods.
Use --cacert to indicate an ca certificate file and use -k to allow insecure server connections when use ssl.
When a job is configured like the following:
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
action: replace
target_label: __scheme__
regex: (.+) This is hard to figure out what it is saying. it turns out that this job basicallycreate a new url based on the formula of ${__scheme__}://${__address__}/${__metrics_path__}
which in many cases __scheme__, __address__, __metrics_path__ are all usethe default value, it makes things even more confusing.for example scheme is normally http if it is not specified__metrics_path__ normally defaults to /metrics__address__ normally defaults to the object IP address.
So for pod type, we are looking at http://${POD_IP}:8080/metrics by defaultSo, it is up to the person who configure this job to make up the partof the url by using various actions. For example, in the above configurationtarget_label: __metrics_path__ actually uses the pod annotation'sprometheus.io/path value if the pod has such annotation. if a pod does nothave such annotation, then the value of __metrics_path__ obviously willbe an empty string, which most likely wont produce a valid url for prometheus to retrieve any metrics.For target_label __scheme__ in the above example, the action is to replace,so the scheme will be basically whatever the annotation's prometheus.io/schemeindicates. where __address__ will be made up by two parts which was made up by theregular expression using __address__ and pod annotation prometheus.io/portif that pod indeed has that annotation. The default __address__ is the pod IPaddress if nothing get changed.