Using cello ansible agent to setup fabric network

docker run -it --rm -v ~/test/vars:/opt/agent/vars hyperledger/cello-ansible-agent ansible-playbook -e "mode=apply env=fabricspec deploy_type=k8s" setupfabric.yml

Script to start prometheus and grafana using docker

#!/bin/bash

# Start prometheus server
docker run -d -p 9090:9090 --restart unless-stopped \
  -v /home/ubuntu/graph/prometheus:/etc/prometheus \
  -v /home/ubuntu/graph/prometheus:/prometheus \
  --user $(id -u) --name prometheus prom/prometheus \
  --config.file=/etc/prometheus/prometheus.yml

# Start prometheus node exporter which collects node resource metrics
docker run -d -p 9100:9100 --restart unless-stopped \
  --user $(id -u) --name node-exporter prom/node-exporter

# Start grafana
docker run -d -p 3000:3000 --restart unless-stopped \
  -v /home/ubuntu/graph/grafana:/var/lib/grafana \
  --user $(id -u) --name grafana grafana/grafana


Assume that directories exists

    /home/ubuntu/graph/prometheus
    /home/ubuntu/graph/grafana

#Using graphite seems to be the better option since it allows posting data to it.
#Here is how to start it.

docker run -d   --name graphite   -p 8080:80   -p 2003:2003   sitespeedio/graphite:1.1.3

then access the web UI at 8080.

Run the following command to post some random data.

while true; do echo "fabric.tong.diceroll;nn=$((RANDOM%30));john=100 $((RANDOM % 100)) -1" | nc -q0 192.168.56.32 2003;sleep 3; done

The above command actually post metric named fabric.tong.diceroll with two tags, one is named nn, one is named john.

VirtuaBox host only network stops working

Once in awhile, virtualbox host only network will stop working. The host will not be able to ping the guest system. Even if the removal and recreate the host only network won't help. What it really helped me was to completely shutdown my macbook machine. do a cold restart, then network started working.

The route table of the host will look like this if the routing is working.

Use this command to show the current routing related to the VB host-only network
netstat -nr | grep 192.168

192.168.56         link#17            UC              3        0 vboxnet
192.168.56.1       a:0:27:0:0:0       UHLWIi          1        6     lo0
192.168.56.3       8:0:27:15:45:66    UHLWI           0        5 vboxnet   1171
192.168.56.255     ff:ff:ff:ff:ff:ff  UHLWbI          0        3 vboxnet

When it was not working, the netstat -nr shows no entries like the above.

Regardless the host network is working or not, from the host, you won't be able to ping the gateway of the host network, for example, ping 192.168.56.1 will fail.

If not wantting cold restart the machine, do the following.

sudo route delete 192.168.56.0/24

Then delete host only network and recreate host only network, then restart VM, it will work.

For some reasons, the mac host may not show any route to network 192.168.56.0/24. when that happens, use the following procedure to get it back.

1. Delete all host-only networks from virtualbox
2. Completely shut down the mac host, then reboot the host
3. Recreate the host-only network.

At this point, mac host should show at least the following (using netstat -nr command):

192.168.56         link#17            UC              2        0 vboxnet

After you start a VM which uses host-only network, you should see the following:

192.168.56         link#17            UC              2        0 vboxnet
192.168.56.255     ff:ff:ff:ff:ff:ff  UHLWbI          0        1 vboxnet

If you actually ping the guest VM with IP 192.168.56.32, and the ping is successful, then you should see the following:

192.168.56         link#17            UC              2        0 vboxnet
192.168.56.32      8:0:27:8f:d9:5e    UHLWIi          1        3 vboxnet   1195
192.168.56.255     ff:ff:ff:ff:ff:ff  UHLWbI          0        1 vboxnet

Fabric sdk-go usage

The flow to use go-sdk to create a new channel.

1. Get configuration from a configuration file using config.FromFile(path_to_config_file), this returns a ConfigProvider type.
2. Use ConfigProvider function and fabsdk.Option to create a sdk.
        sdk := fabsdk.New(configOpt, sdkOpts...)
3.  Use sdk.Context to create a client context.

clientContext := sdk.Context(fabsdk.WithUser(orgAdmin),

fabsdk.WithOrg(ordererOrgName))

4. Use resmgmt to create a new res management client

resMgmtClient, err := resmgmt.New(clientContext)

5. Use the resMgmtClient, orgName, and the sdk context to create a mspClient

mspClient, err := mspclient.New(sdk.Context(),

mspclient.WithOrg(orgName))

6. Get AdminIdentity using the orgAdmin string.

adminIdentity, err := mspClient.GetSigningIdentity(orgAdmin)

7. Use resmgmt.SaveChannelRequest.

    req := resmgmt.SaveChannelRequest{ChannelID: channelID,

             ChannelConfigPath: integration.GetChannelConfigPath(channelID + ".tx"),

             SigningIdentities: []msp.SigningIdentity{adminIdentity}}

8. Save the channel

    txID, err := resMgmtClient.SaveChannel(req,

resmgmt.WithRetry(retry.DefaultResMgmtOpts),

resmgmt.WithOrdererEndpoint("orderer.example.com"))

9. Make sure things went well.    

    require.Nil(t, err, "error should be nil")

    require.NotEmpty(t, txID, "transaction ID should be populated")

 

health vital info collection

we could create a block chain system which allow users to send data to a blockchain which de-identifies the user specific information such as name, location, etc. for example,

1. name or id will be replaced with something to prevent being used to identify a person. the ID may be replaced with something which can be used to correlate data set but can not be used to identify a user.
2. age can be mapped to an age group
3. location, optional
4. sex, optional
5. race, optional

The information posted onto the network if get confirmed, will receive points. transaction gets posted to the network once a day or every 6 hours. each transaction should contain a number of data points, for example, the vitals will be measured every 10 minutes, if post is done every 6 hours, then each transaction should contain 36 data points.

The data can be used to track population in a particular areas during a disaster. data can be correlated to some other statistics to predict the population.

Hyperledger Fabric operation

Fabric operation sequence

1. Create channel
2. Peer join channel
3. Install Chaincode
4. Instantiate Chaincode

Create channel

1. an orderer endpoint and its ca certificate
2. channel name
3. channel transaction file
4. tls flag
5. timeout value
6. peer command, set as environment variable
7. peer information

Example:

peer channel create -o {{ cliorderer.name }}:7050 -c firstchannel
  -f /etc/hyperledger/allorgs/keyfiles/firstchannel.tx
  --tls true --timeout $TIMEOUT
  --cafile msp/tlscacerts/tlsca.{{ cliorderer.org }}-cert.pem

Instantiate chaincode (Per channel)

1. an orderer endpoint and its ca certificate
2. channel name
3. chaincode name and chaincode must be placed at the right location
4. tls flag
5. timeout value
6. version number
7. argument
8. peer information, set as environment variable
9. endorsement policy

peer chaincode instantiate -o {{ cliorderer.name }}:7050 --tls true
  --cafile msp/tlscacerts/tlsca.{{ cliorderer.org }}-cert.pem
  -C firstchannel -n firstchaincode -v 1.0
  -c '{"Args":["init","a", "100", "b","200"]}'
  -P "AND ('{{ orgmembers }}.member')"

Join channel (Per peer)

1. channel block
8. peer information, set as environment variable

peer channel join -b firstchannel.block

Install chaincode (Per peer)

1. chaincode name and version
2. chaincode path in GOPATH src directory, for example: $GOPATH/src/chaincode
3. peer information, set as environment variable

peer chaincode install -n firstchaincode -v 1.0 -p chaincode

k8s issues

1. pods dns search and communication
2. persistent volume capabilities
     so many different drivers, very confusing.
     policies are very confusing, tied way too much to the underlying storage. Recycle, Retain, what are the differences and what do they exactly mean.
3. pod allocation policies
4. Start using k8s is not a walk in the park, gke(google), aks(Azure), CKS (Cloud Kubernetes Service) (IBM) provide a fairly easy process to start a k8s cluster, however, eks(Amazon) requires a user to create a role to authorize a cluster creation, the process of creating that role is not very obvious. EKS asks to add worker node which is a separate step.
5. Getting the kubeconfig file is where things really going dramatically different.
     1. IBM offers a link to download
     2. Google and Microsoft require their own client tools to get i, glcoud and az
     3. Amazon offers an instruction and copy/paste to accomplish that.
6. To create a persistent volume claim, this is very different.
7. Dashboard access, from easy to somewhat so weird approaches, for example, IBM provides a simple link to the standard k8s dashboard, Azure provides a command to start a proxy server, then user can access that proxy server for the dashboard, I think this is a rather strange way of offering the dashboard access.
8. Length of the provisioning is long.
9. docker in docker issue. end point unknown, using daemon set to create dind container for endpoint.

alibaba kubernetes service.

a PV can only be used by one PVC. and the policy has to be set to Recycle for NFS persistent volume.