which layer does tls belong in networking?

Transport Layer Security (TLS) primarily operates between the Transport Layer and the Application Layer in the networking stack. While it is often associated with the Transport Layer due to its role in securing data transmission, it technically sits above the Transport Layer and below the Application Layer

Placement in Models:

• OSI Model: TLS is generally considered to operate at the Presentation Layer (Layer 6), as it handles encryption and decryption of data.
• TCP/IP Model: TLS is seen as part of the Application Layer, since the TCP/IP model has fewer layers and combines some of the OSI layers.

Functionality:

• Encryption: Ensures that data is encrypted before transmission and decrypted upon receipt.
• Authentication: Verifies the identities of the communicating parties using certificates.
• Integrity: Ensures that the data has not been tampered with during transit.

This positioning allows TLS to provide secure communication for various application protocols like HTTP (resulting in HTTPS), SMTP, and IMAP.

 

OSI 7 layers 

Layers: 7 layers (Physical, Data Link, Network, Transport, Session, Presentation, Application).

 

TCP/IP layers:

Layers: 4 layers (Network Interface, Internet, Transport, Application).

 

Since OSI model is not really used in real world application, focus on TCP/IP model is better in programming.

 

Ubuntu docker access to other users

 

 

 Also make sure that docker.sock file is accessible by others.

sudo chmod 666 /var/run/docker.sock

 

How to remove custom resource stuck on delete

 To delete the custom resource which stuck in delete state , follow these steps:

1. Edit the Object : kubectl edit customresource/name.
2. Remove finalizer parameter.
3. Delete the object : kubectl delete customresource/name.

Show what code were covered or not using go tool cover command

 Once go test produces cover.out , you can use the following command to launch browser to show the coverage.

go tool cover -html=cover.out

To git restore all the unstaged files

 Sometimes, you may purposely delete some of the files for testing, then you will need to restore these files, it will take a lot of key strokes if you do file by file, the below command can restore all the unstaged files in git

git restore -- $(git ls-files -m)

KubeBuilder

Use kubebuilder to start a new project:

 

     kubebuilder init --domain my.domain --repo my.domain/guestbook

This step creates make file, dockerfile etc.

Once a project gets created, you normally run the following command to add API

     kubebuilder create api --group webapp --version v1 --kind CronJob 

Then you normally would edit the files in api/v1 _types.go files to add your own struct

basically data structure for your api. and make changes to the controller.go in the

controllers directory to implement your business logic.

Then you normally would need to run:

    make manifests to generate crds, roles, role bindings etc.

    make generate to generate code in zz_generated.deepcopy.go to capture changes that you

make to apis, that is, changes made in _types.go file will need to be reflected in

zz_generated.deepcopy.go file.

Deal with a local helm chart repo

 

# Start up a helm client

kubectl run -n pcloud helmtest --image=alpine/helm:3.12.0 --command -- sleep 3000

# add the repo locally named acc, given that the repo sits at http://acc-helm-repo:8080

helm repo add acc http://acc-helm-repo:8080

# search the repo and list all the charts from the repo acc.

helm search repo acc

# now inspect a particular chart, in this case, in repo named acc and chart named netapp-monitoring-operator

helm inspect all acc/netapp-monitoring-operator

Process of working with un-merged branches

 1. local check out the branch, for example

     git checkout -b the-dirty-branch

2. pull the remote branch to the local

    git pull the-dirty-branch

3. you can do the same thing for other branches if more branches are needed

4. then switch to the integration (or main) branch

5. cherry-pick from the dirty branch or rebase from the dirty branch

6. do git reset --soft to maintain unchanged release tag

K8S CSI related commands

 

kg csidrivers

kg storageclass

kg volumesnapshotclass

To list volume snapshot and it content

kg volumesnapshot -n test01

kg volumesnapshotcontent -n test01

Volume snapshot class uses driver.

storage class uses provisioner

volume snapshot class uses driver.

So driver and provisioner should be same thing?

How to support multi-arch docker image build on mac

 Simply do the following command:

docker buildx create --use

 

Then run command 

docker buildx ls

You should see amd64 included like the following.


linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6





Some document says turn on the experienmental flag on which is not needed.


k8s backup products

1. https://portworx.com/

    You don’t have to use Portworx storage to use Portworx Backup. Backup and recover Kubernetes applications using Amazon EBS, Google Persistent Disk and Azure Block storage directly via CSI.

2.  https://velero.io/  used to be called Heptio Ark

    uses object storage

3. https://stash.run/

    Stores backup data in AWS S3, Minio, Rook, GCS, Azure, OpenStack Swift, Backblaze B2 and Rest Server

4. https://trilio.io/

5. https://metallic.io/   very weak.

6. https://www.kasten.io/

7. https://www.rubrik.com/

8. https://storware.eu/

 

 

Use k8s rbac for authorization

K8S rbac is solid implementation of rbac. Use it welll can avoid reinvent the wheel for an application.

Use a namespace per user to scope user resources

Use user to represent an actual user. Then use role and role binding to control permissions.

Expose services from k8s clusters running on MacBook

Since Mac won’t route to docker network, it is a big problem when you run multiple clusters on Mac when you need to expose services outside of the k8s clusters, but with Nginx proxy, one can simply use the docker port map to achieve this. 

The idea is to do the following

1. Write a script which  will query every cluster and find all load balancer services

2. With all known load balancer services, create Nginx configuration file to proxy request

3. Once the Nginx configuration file is created, lunch Nginx on the docker network used by k8s clusters and map all the ports used.

4. Based on the nature of the services, either http, https or stream can be used.

The drawback is that the port map certainly will change, and each time a service is added or removed, the Nginx container will need to be restarted with new port map and configuration.

For test purposes, this solution should work.

Allow application to accept incoming network connection

 When k8s kind trying to use a local non loopback ip for api server address, mac firewall will ask if the incoming network connection should be allowed. A screen like this will be popped up,

The right way to avoid this is to change the macbook firewall rule to allow this kind action.

how to practise music instrument

 The following process uses all free available tools to prepare a piece of music so that one can practice.

1. Generate music XML file.

Use musescore to generate a piece of music and save as music XML file (either compressed or non compressed)

2. Create a free soundslice account and upload the generate music xml file to your account. Doing this step from a computer is a lot easier.

3. Soundslice is a web based application, so there is no mobile app for mobile devices such as android or iOS tablets or phones, so you can go to www.soundslice.com via a browser, then use the share button to add the bookmark onto you homescreen, that will work just like a native app.

4. You can pick any uploaded the piece (slice) to play, you can even mute the sound if you like, so that the app simply read through your music, and you can simply follow it to practice.

Note:

One can also use the tool to convert to music xml file by doing the following:

1. Simply screen capture an image.

2. Convert that image into a pdf file using image viewer

3. Use musescore pdf to music xml file tool to convert to musescore file

4. Edit/correct convertion issues, then export to music xml file.

Machine learning terms

 Graph:

Graphs are data structures to describe relationships and interactions between entities in complex systems. In general, a graph contains a collection of entities called nodes and another collection of interactions between a pair of nodes called edges

Shape:

shape (Tensor)

The number of elements in each dimension of a tensor. The shape is represented as a list of integers. For example, the following two-dimensional tensor has a shape of [3,4]:

[[5, 7, 6, 4],
 [2, 9, 4, 8],
 [3, 6, 5, 1]]

TensorFlow uses row-major (C-style) format to represent the order of dimensions, which is why the shape in TensorFlow is [3,4] rather than [4,3]. In other words, in a two-dimensional TensorFlow Tensor, the shape is [number of rows, number of columns].

gradient

The vector of partial derivatives with respect to all of the independent variables. In machine learning, the gradient is the vector of partial derivatives of the model function. The gradient points in the direction of steepest ascent.

Allow ssh to access a remote linux system

 in the machine where you like to access the remote system, generate a ssh key and certificate file, normally id_rsa and id_rsa.pub file. Then copy the content of id_rsa.pub file to the remote system ~/.ssh/authorized_keys file, then you should be able to access the system.

Istio ambient mesh ztunnel implementations

 

# The results come from ztunnel pod

# iptables -S -t nat

-P PREROUTING ACCEPT

-P INPUT ACCEPT

-P OUTPUT ACCEPT

-P POSTROUTING ACCEPT

-A PREROUTING -j LOG --log-prefix "nat pre [ztunnel-ntkqj] "

-A INPUT -j LOG --log-prefix "nat inp [ztunnel-ntkqj] "

-A OUTPUT -j LOG --log-prefix "nat out [ztunnel-ntkqj] "

-A OUTPUT -p tcp -m tcp --dport 15088 -j REDIRECT --to-ports 15008

-A POSTROUTING -j LOG --log-prefix "nat post [ztunnel-ntkqj] "

# iptables -S -t mangle

-P PREROUTING ACCEPT

-P INPUT ACCEPT

-P FORWARD ACCEPT

-P OUTPUT ACCEPT

-P POSTROUTING ACCEPT

-A PREROUTING -j LOG --log-prefix "mangle pre [ztunnel-ntkqj] "

-A PREROUTING -i pistioin -p tcp -m tcp --dport 15008 -j TPROXY --on-port 15008 --on-ip 127.0.0.1 --tproxy-mark 0x400/0xfff

-A PREROUTING -i pistioout -p tcp -j TPROXY --on-port 15001 --on-ip 127.0.0.1 --tproxy-mark 0x400/0xfff

-A PREROUTING -i pistioin -p tcp -j TPROXY --on-port 15006 --on-ip 127.0.0.1 --tproxy-mark 0x400/0xfff

-A PREROUTING ! -d 10.30.0.5/32 -i eth0 -p tcp -j MARK --set-xmark 0x4d3/0xfff

-A INPUT -j LOG --log-prefix "mangle inp [ztunnel-ntkqj] "

-A FORWARD -j LOG --log-prefix "mangle fw [ztunnel-ntkqj] "

-A OUTPUT -j LOG --log-prefix "mangle out [ztunnel-ntkqj] "

-A POSTROUTING -j LOG --log-prefix "mangle post [ztunnel-ntkqj] "

===

-A PREROUTING -i pistioin -p tcp -m tcp --dport 15008 -j TPROXY --on-port 15008 --on-ip 127.0.0.1 --tproxy-mark 0x400/0xfff

Note: take every tcp packet targeting port 15008, deliver to 127.0.0.1:15008 and mark packet with 0x400/0xfff

port 15008 is Istio HBONE mTLS tunnel port

-A PREROUTING -i pistioout -p tcp -j TPROXY --on-port 15001 --on-ip 127.0.0.1 --tproxy-mark 0x400/0xfff

Note: take every tcp packet, then deliver them to 127.0.0.1:15001 and also mark packet with 0x400/0xfff

port 15001 is envoy outbound port

-A PREROUTING -i pistioin -p tcp -j TPROXY --on-port 15006 --on-ip 127.0.0.1 --tproxy-mark 0x400/0xfff

Note: take every tcp packet, then deliver to 127.0.0.1:15006 and mark packet with 0x400/0xfff

port 15006 is envoy inbound port

FIB vs RIB

 

The forwarding information base (FIB) is the actual information that a routing/switching device uses to choose the interface that a given packet will use for egress. For example, the FIB might be programmed such that a packet bound to a destination in 192.168.1.0/24 should be sent out of physical port ethernet1/2. There may actually be multiple FIB's on a device for unicast forwarding vs multicast RPF checking, different protocols (ip vs mpls vs ipv6) but the basic function is the same - selection criteria (usually destination) mapping to output interface/encapsulation. Individual FIB's may also be partitioned to achieve concurrent independent forwarding tables (i.e. vrf's).

Each FIB is programmed by one or more routing information bases (RIB). The RIB is a selection of routing information learned via static definition or a dynamic routing protocol. The algorithms used within various RIB's will vary - so, for example, the means by which BGP or OSPF determines potential best paths vary quite a bit. The means by which multiple RIB's are programmed into a common (set) of FIB's in a box will vary by implementation but this is where concepts like administrative distance are used (e.g. identical paths are learned via eBGP and OSPF, the eBGP is usually preferred for FIB injection). Again, RIB's may also be potentially partitioned to allow for multiple vrf's, etc.

Allow two docker networks to communicate with each other

In some cases, it is useful to have containers running on two different docker bridge networks to communicate with each other. The easist thing is to remove the docker created isolation rules so that containers running on different bridged docker networks wont have their packets dropped by iptable rules. One other way is to add forward rules so that their packet will be accepted. Here is an example, assume there are two bridged docker networks b1 172.19.0.0/16 and b2 172.20.0.0/16. By default the containers running on these two separate networks are isolated (on purpose). With the following two Iptable rules, containers can communicate with each other. 

iptables -I FORWARD -s 172.19.0.0/16 -d 172.20.0.0/16 -j ACCEPT

iptables -I FORWARD -d 172.20.0.0/16 -s 172.19.0.0/16 -j ACCEPT