Enable OpenStack sending log to a remote rsyslog server

On OpenStack side:

Security Rules, open the following ports: These rules are required when all your nodes running in a OpenStack cloud.

   9200 (ElasticSearch)
   9300 (ElasticSearch transport between nodes)
   9400 (for syslog) to collect via logstash
   5601 (Kibana)
   22, 80 and icmp

Config rsyslog to remote logging:
create a file in /etc/rsyslog.d, named it like 60-openstack.conf
put the following content in the that file

local6.*               @10.0.50.9:9400

NOTE: that the ip address must be the remote rsyslog server IP. When working with ElasticSearch, that IP should be the IP address of logstash server. The port 9400 should match up with the UDP port sets in /etc/logstash/conf.d/logstach.conf file. Using port lower than 1024 will require special permission. After making these changes, restart the service like this:

service rsyslog restart

Config OpenStack component to use the new log facility:

Change component file such as nova.conf, neutron.conf to use syslog like the following:

[DEFAULT]
debug = False
use_syslog = True
syslog_log_facility = LOG_LOCAL6

You can use LOG_LOCAL0 to LOG_LOCAL7 as long as the facility points to the remote logging server.

After making these changes, restart the components



This procedure is using ElasticSearch logstash as a rsyslog server. When there is no particular filter setup, you still will be able to use kibana to chart log data.  Follow these steps to produce a pie chart:

1. Use the logstash-* index
2. Click on Visualize button at the top of the kibana screen
3. Click on Pie Chart
4. Select from new search
5. Select split slices
6. Select terms from the aggregation  drop down box
7. Select syslog_program.raw from the field drop down box, leave others alone
8. Click on the run button at the options bar, a chart should be displayed

How to install docker client working

apt-get update
apt-get -y install docker.io
ln -sf /usr/bin/docker.io /usr/local/bin/docker 

OpenStack wiered errors

This is the post I will use to record some of the strange errors I have encounted.
    1. When image file gets moved to a different location, glance will fail to provide the image for a VM, but the instance somewhere will get multiple fixed IP addresses. This is a very strange problem. After going to the database, corrected the image_location value field records, restart glance services will resolve this error.
   2. There is also a problem when instance_path is not writable by libvirt-qemu user, for some reason, each VM will get two private IPs. Make sure that the libvirt-qemu user can write to instance_path which is defined in nova.conf file.
     

Setup Ansible to use openstack cloud module

Follow these steps to install ansible and shade to use openstack cloud modules.

sudo apt-add-repository ppa:ansible/ansible -y
sudo apt-get update && sudo apt-get install ansible -y 

sudo apt-get install python-dev python-pip libssl-dev libffi-dev -y
sudo pip install six shade --upgrade