Warning :Remote branch XXX.. not found in upstream origin, using HEAD instead when cloning xen source

When you clone git repository for xen 4.3, you would get the following message:
"Remote branch RELEASE-4.3.0 not found in upstream origin, using HEAD instead"

This is caused because of git version, prior version to 1.7.10 don't support checking out tag.

# My git version 
$ git --version
git version 1.7.9.5

# Download Xen source
$ git clone -b RELEASE-4.3.0 git://xenbits.xen.org/xen.git
Cloning into 'xen'...
remote: Counting objects: 272877, done.
remote: Compressing objects: 100% (55538/55538), done.
remote: Total 272877 (delta 214914), reused 270399 (delta 212700)
Receiving objects: 100% (272877/272877), 53.61 MiB | 1.75 MiB/s, done.
Resolving deltas: 100% (214914/214914), done.

warning: Remote branch RELEASE-4.3.0 not found in upstream origin, using HEAD instead

Simply, you are able to check out additionally.

$ cd xen
$ git checkout RELEASE-4.3.0
Note: checking out 'RELEASE-4.3.0'.

Or, you can re-install the newer version of git.

# Remove the old git
$ sudo apt-get remove git

# Download git
$ wget https://git-core.googlecode.com/files/git-1.8.1.2.tar.gz
$ tar -xzvf ./git-1.8.1.2.tar.gz
$ cd git-1.8.1.2
$ make prefix=/usr/local all
$ sudo make prefix=/usr/local install
$ git --version
git version 1.8.1.2

# Download Xen source 
$ git clone -b RELEASE-4.3.0 git://xenbits.xen.org/xen.git
Cloning into 'xen'...
remote: Counting objects: 272877, done.
remote: Compressing objects: 100% (55538/55538), done.
remote: Total 272877 (delta 214914), reused 270399 (delta 212700)
Receiving objects: 100% (272877/272877), 53.61 MiB | 4.01 MiB/s, done.
Resolving deltas: 100% (214914/214914), done.
Note: checking out 'f8cc9c2b713b1739b1d3d324716547fa639dce86'.

References:
1. http://stackoverflow.com/questions/17216067/git-clone-b-tag-results-in-warning-remote-branch-not-found-in-upstream-origi/17216068#17216068

Note things in dealing with Nagios+Ndo2DB and properties in host configuration

I'd like to point out two things related nagios.
As we know, we are able to save status data of host and service into relational database using ndo2db module and I described how to setup at a post.

When people have nagios + nod2db monitoring system in their environments, they may find that data is duplicated in the table "nagios_hosts" and what makes this happen.

For example, this is result of the query of host definitions

mysql> SELECT host_id, alias, address, config_type FROM `nagios_hosts`;
+----------+-----------+----------------+---------------+
| host_id  | alias     |  address       |  config_type  |
+----------+-----------+----------------+---------------+
|10963     |test03     | 192.168.20.183 |            1  |                                 
|10962     |cent-test  | 192.168.20.175 |            1  |                                 
|10961     |FDCMOCDB01 | 192.168.20.172 |            1  |                                 
|10960     |test03     | 192.168.20.183 |            0  |                                 
|10959     |cent-test  | 192.168.20.175 |            0  |                                 
|10958     |FDCMOCDB01 | 192.168.20.172 |            0  |   
+----------+-----------+----------------+---------------+

Like this, every host is duplicated except for config_type.

Ndo2db can either get host definition from the original config file (It is usually located in /etc/nagios/objects/hosts.cfg) or retained config file(/var/log/nagios/status.dat)

As default, config_output_options is 3 which means get both data from config file and retained config file.

$ cat /etc/nagios/ndomod.cfg | grep config*
...
# CONFIG OUTPUT OPTION
# This option determines what types of configuration data the NDO NEB module will dump from Nagios.
# Values can be OR'ed together.
# Values:
#         0 = Don't dump any configuration information
#         1 = Dump only original config (from config files)
#         2 = Dump config only after retained information has been restored
#         3 = Dump both original and retained configuration
config_output_options=3

I changed this value to 2, I'd like to get data only from the retained file.

$ cat /etc/nagios/ndomod.cfg
...
config_output_options=2

Then, Restarting Services

$ service ndo2db restart
$ service nagios restart

mysql> SELECT host_id, alias, address, config_type FROM `nagios_hosts`;
+----------+-----------+----------------+---------------+
| host_id  | alias     |  address       |  config_type  |
+----------+-----------+----------------+---------------+
|10972     |test03     | 192.168.20.183 |            1  |                                 
|10971     |cent-test  | 192.168.20.175 |            1  |                                 
|10970     |FDCMOCDB01 | 192.168.20.172 |            1  |                                 
+----------+-----------+----------------+---------------+

Next topic, I think it is more important point.
I usually edit config files in the directory "/etc/nagios/objects/" when need to change some properties, active_checks_enabled for instance.
But I managed to know that nagios doesn't pick up some properties from config files. Rather, nagios relies retention files.

It is said that the information from nagios manual

"It is important to point out that several directives in host, service, and contact definitions may not be picked up by Nagios when you change them in your configuration files. Object directives that can exhibit this behavior are marked with an asterisk (*). The reason for this behavior is due to the fact that Nagios chooses to honor values stored in the state retention file over values found in the config files, assuming you have state retention enabled on a program-wide basis and the value of the directive is changed during runtime with an external command."

And these are directives for host definition
...
active_checks_enabled *: ....
passive_checks_enabled *: ....
...

The problem is that I cannot edit the retention status file which is read-only file and only editable by nagios. So, how can I do to change unchangeable properties by ending in config file in programming manner not using nagios web.

Naiogs provides cmd.cgi interface and it can run like the below:

$ curl -d "cmd_mod=2&cmd_typ=48&host=FDCMOCDB01" "http://localhost/nagios/cgi-bin/cmd.cgi" -u "nagiosadmin:*******"
<html>
<head>
<link rel="shortcut icon" href="/nagios/images/favicon.ico" type="image/ico">
<title>
External Command Interface
</title>
<LINK REL='stylesheet' TYPE='text/css' HREF='/nagios/stylesheets/common.css'>
<LINK REL='stylesheet' TYPE='text/css' HREF='/nagios/stylesheets/cmd.css'>
</head>
<body CLASS='cmd'>

<!-- Produced by Nagios (http://www.nagios.org).  Copyright (c) 1999-2007 Ethan Galstad. -->
<table border=0 width=100%>
<tr>
<td align=left valign=top width=33%>
<TABLE CLASS='infoBox' BORDER=1 CELLSPACING=0 CELLPADDING=0>
<TR><TD CLASS='infoBox'>
<DIV CLASS='infoBoxTitle'>External Command Interface</DIV>
Last Updated: Tue Jul 9 16:04:13 KST 2013<BR>
Nagios® Core™ 3.3.1 - <A HREF='http://www.nagios.org' TARGET='_new' CLASS='homepageURL'>www.nagios.org</A><BR>
Logged in as <i>nagiosadmin</i><BR>
</TD></TR>
</TABLE>
</td>
<td align=center valign=top width=33%>
</td>
<td align=right valign=bottom width=33%>
</td>
</tr>
</table>
<P><DIV CLASS='infoMessage'>Your command request was successfully submitted to Nagios for processing.<BR><BR>
Note: It may take a while before the command is actually processed.<BR><BR>
<A HREF='javascript:window.history.go(-2)'>Done</A></DIV></P>
<!-- Produced by Nagios (http://www.nagios.org).  Copyright (c) 1999-2007 Ethan Galstad. -->
</body>
</html>

This is how to run cmd.cgi with curl in the shell environment

$ curl -d cmd_typ=${cmd} \
      -d cmd_mod=2 \
      -d host=${host} \
      -d service=${svc} \
      -d btnSubmit=Commit \
      -s \
      -u 'user:pass \
      "http://nagios.server/nagios/cgi-bin/cmd.cgi"

As parameter "cmd_typ", constant values are defined in the header file "common.h"

...
#define CMD_ENABLE_HOST_CHECK 47
#define CMD_DISABLE_HOST_CHECK 48
...

If you implement this in java, you are able to execute it by using httpclient class (package org.apache.commons.httpclient) like my case.

References:
1. http://eggsonbread.com/2011/03/18/disable-enable-nagios-notifications-via-command-line-curl/
2. https://github.com/ageric/nagios/blob/cc67733e82721546d9d04180c4441cdb41a6f9a2/include/common.h
3. http://forums.cacti.net/post-133568.html

Xen 4.3 RC1 installation

I'd like to share how to install Xen 4.3 RC (Release Candidate) as part of a testing on Xen test day. I freshly installed Ubuntu 12.04 on my physical machine.

Xen source is able to download from git repository or tarball.

# download xen source from git repository
$ mkdir ~/xen-4.3
$ cd ~/xen-4.3
$ git clone -b 4.3.0-rc1 git://xenbits.xen.org/xen.git
Cloning into 'xen'...
remote: Counting objects: 271307, done.
remote: Compressing objects: 100% (59143/59143), done.
remote: Total 271307 (delta 213525), reused 263420 (delta 207531)
Receiving objects: 100% (271307/271307), 53.31 MiB | 748 KiB/s, done.
Resolving deltas: 100% (213525/213525), done.
Note: checking out '4de97462d34f7b74c748ab67600fe2386131b778'.

If you not familiar with git, you can download tarball source

$ wget http://bits.xensource.com/oss-xen/release/4.3.0-rc1/xen-4.3.0-rc1.tar.gz
--2013-05-08 17:55:22--  http://bits.xensource.com/oss-xen/release/4.3.0-rc1/xen-4.3.0-rc1.tar.gz
Resolving bits.xensource.com (bits.xensource.com)... 63.80.4.58, 63.80.4.218
Connecting to bits.xensource.com (bits.xensource.com)|63.80.4.58|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16375588 (16M) [application/x-gzip]
Saving to: `xen-4.3.0-rc1.tar.gz'

34% [=================>                                   ] 5,670,832    468K/s  eta 25s

Installing build tools and related modules.

$ sudo apt-get install build-essential

$ sudo apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif 
$ sudo apt-get install texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial
$ sudo apt-get install make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg-dev
$ sudo apt-get install iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils libyajl-dev gettext markdown
$ sudo apt-get install libpixman-1-0 libpixman-1-dev 
$ sudo apt-get install libc6-dev*

Confuging the environment for installing.

$ sudo ./configure
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
Will build the following subsystems:
  xen
  kernels
  tools
  stubdom
  docs
configure: creating ./config.status
config.status: creating ./config/Toplevel.mk
=== configuring in tools (/downloads/xen-4.3.0-rc1/tools)
configure: running /bin/bash ./configure --disable-option-checking '--prefix=/usr/local'  --cache-file=/dev/null --srcdir=.
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
...
checking for fig2dev... /usr/bin/fig2dev
checking for pod2man... /usr/bin/pod2man
checking for pod2html... /usr/bin/pod2html
checking for pod2text... /usr/bin/pod2text
checking for markdown... /usr/bin/markdown
configure: creating ./config.status
config.status: creating ../config/Docs.mk

To build all subsystems, use "dist" and then run "make install" to install xen.

################
# Build Xen
################
$ sudo make dist
make -C xen install
make[1]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -f Rules.mk _install
make[2]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -C tools
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
[ -d figlet ] && make -C figlet
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
gcc -o figlet figlet.c
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make symbols
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
gcc -Wall -Werror -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -Wdeclaration-after-statement -o symbols symbols.c
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[3]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make -f /home/yeonki/xen-4.3/xen/xen/Rules.mk include/xen/compile.h
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -C tools
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
[ -d figlet ] && make -C figlet
make[5]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make[5]: `figlet' is up to date.
make[5]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make symbols
make[5]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[5]: `symbols' is up to date.
make[5]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
 __  __            _  _    _____                    _        _     _
 \ \/ /___ _ __   | || |  |___ /    _   _ _ __  ___| |_ __ _| |__ | | ___
  \  // _ \ '_ \  | || |_   |_ \ __| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
  /  \  __/ | | | |__   _| ___) |__| |_| | | | \__ \ || (_| | |_) | |  __/
 /_/\_\___|_| |_|    |_|(_)____/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
...
make[1]: Leaving directory `/home/yeonki/xen-4.3/xen/docs'
install -d -m0755 -p /home/yeonki/xen-4.3/xen/dist/
install -m0644 -p ./COPYING /home/yeonki/xen-4.3/xen/dist
install -m0644 -p ./README /home/yeonki/xen-4.3/xen/dist
install -m0755 -p ./install.sh /home/yeonki/xen-4.3/xen/dist

################
# Install Xen
################
$ sudo make install
make -C xen install
make[1]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -f Rules.mk _install
make[2]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -C tools
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
[ -d figlet ] && make -C figlet
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make[4]: `figlet' is up to date.
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make symbols
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[4]: `symbols' is up to date.
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[3]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make -f /home/yeonki/xen-4.3/xen/xen/Rules.mk include/xen/compile.h
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
 __  __            _  _    _____                    _        _     _
 \ \/ /___ _ __   | || |  |___ /    _   _ _ __  ___| |_ __ _| |__ | | ___
  \  // _ \ '_ \  | || |_   |_ \ __| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
  /  \  __/ | | | |__   _| ___) |__| |_| | | | \__ \ || (_| | |_) | |  __/
 /_/\_\___|_| |_|    |_|(_)____/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
...
set -x; make -C figs
+ make -C figs
make[2]: Entering directory `/home/yeonki/xen-4.3/xen/docs/figs'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/yeonki/xen-4.3/xen/docs/figs'
rm -rf //usr/local/share/doc/xen
install -d -m0755 -p //usr/local/share/doc/xen
install -d -m0755 -p //usr/local/share/man
cp -R man1 //usr/local/share/man
cp -R man5 //usr/local/share/man
[ ! -d html ] || cp -R html //usr/local/share/doc/xen
make[1]: Leaving directory `/home/yeonki/xen-4.3/xen/docs'

Xen is successfully installed. The next is to check whether xen boot files has created in /boot/ directory.

ls -al /boot/xen*
lrwxrwxrwx 1 root root       19 Jun  5 15:50 /boot/xen-4.3.gz -> xen-4.3-unstable.gz
-rw-r--r-- 1 root root   821559 Jun  5 15:50 /boot/xen-4.3-unstable.gz
lrwxrwxrwx 1 root root       19 Jun  5 15:50 /boot/xen-4.gz -> xen-4.3-unstable.gz
lrwxrwxrwx 1 root root       19 Jun  5 15:50 /boot/xen.gz -> xen-4.3-unstable.gz
-rw-r--r-- 1 root root 14012010 Jun  5 15:50 /boot/xen-syms-4.3-unstable

# Execute update-grub
$ sudo update-grub
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
dpkg: warning: version 'syms-4.3-unstable' has bad syntax: version number does not start with digit
dpkg: warning: version '/boot/xen.gz' has bad syntax: version number does not start with digit
dpkg: warning: version 'syms-4.3-unstable' has bad syntax: version number does not start with digit
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
dpkg: warning: version 'syms-4.3-unstable' has bad syntax: version number does not start with digit
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found memtest86+ image: /memtest86+.bin
done

# When run "update-grub", an error was occurred. 
# To fix this, remove xen-syms-4.3-unstable and xen.gz
$ sudo rm /boot/xen-syms-4.3-unstable
$ sudo rm /boot/xen.gz

# Re-run update-grub
$ sudo update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found memtest86+ image: /memtest86+.bin
done

#################################
# Register xen moduels as service
#################################
$ sudo update-rc.d xencommons defaults 19 18
update-rc.d: warning: xencommons start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
Adding system startup for /etc/init.d/xencommons ...
   /etc/rc0.d/K18xencommons -> ../init.d/xencommons
   /etc/rc1.d/K18xencommons -> ../init.d/xencommons
   /etc/rc6.d/K18xencommons -> ../init.d/xencommons
   /etc/rc2.d/S19xencommons -> ../init.d/xencommons
   /etc/rc3.d/S19xencommons -> ../init.d/xencommons
   /etc/rc4.d/S19xencommons -> ../init.d/xencommons
   /etc/rc5.d/S19xencommons -> ../init.d/xencommons

$ sudo update-rc.d xendomains defaults 21 20
update-rc.d: warning: xendomains start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
Adding system startup for /etc/init.d/xendomains ...
   /etc/rc0.d/K20xendomains -> ../init.d/xendomains
   /etc/rc1.d/K20xendomains -> ../init.d/xendomains
   /etc/rc6.d/K20xendomains -> ../init.d/xendomains
   /etc/rc2.d/S21xendomains -> ../init.d/xendomains
   /etc/rc3.d/S21xendomains -> ../init.d/xendomains
   /etc/rc4.d/S21xendomains -> ../init.d/xendomains
   /etc/rc5.d/S21xendomains -> ../init.d/xendomains

$ sudo update-rc.d xen-watchdog defaults 22 23
update-rc.d: warning: xen-watchdog start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
Adding system startup for /etc/init.d/xen-watchdog ...
   /etc/rc0.d/K23xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc1.d/K23xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc6.d/K23xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc2.d/S22xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc3.d/S22xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc4.d/S22xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc5.d/S22xen-watchdog -> ../init.d/xen-watchdog

#################################
# rebooting boot with xen
#################################
$ cat /boot/grub/grub.cfg | grep Xen
submenu "Xen 4" {
menuentry 'Ubuntu GNU/Linux, with Xen 4 and Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4 ...'
menuentry 'Ubuntu GNU/Linux, with Xen 4 and Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4 ...'
submenu "Xen 4.3" {
menuentry 'Ubuntu GNU/Linux, with Xen 4.3 and Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3 ...'
menuentry 'Ubuntu GNU/Linux, with Xen 4.3 and Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3 ...'
submenu "Xen 4.3-unstable" {
menuentry 'Ubuntu GNU/Linux, with Xen 4.3-unstable and Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3-unstable ...'
menuentry 'Ubuntu GNU/Linux, with Xen 4.3-unstable and Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3-unstable ...'


$ sudo vi /etc/default/grub
...
GRUB_DEFAULT='Ubuntu GNU/Linux, with Xen 4.3 and Linux 3.2.0-23-generic'
..

$ sudo update-grub
$ sudo boot

References:
1. http://wiki.xen.org/wiki/Archived/Xen_4.3_RC1_test_instructions
2. http://wiki.xen.org/wiki/Compiling_Xen_From_Source
3. http://locatrix.com/build-and-install-xen-4-2-1-from-source-on-ubuntu-12-10/
4. http://blog.heidt.biz/blog/2012/09/17/setting-up-ubuntu-12-dot-04-precise-pangolin-as-xen-dom0/
5. http://lists.xen.org/archives/html/xen-users/2013-05/msg00136.html

Xen is moving foward

Although Xen has not been included as default hypervisor in RHEL and has been losing some supporters. Xen is still attractive hypervisor. Xen is trying to support platforms in many ways. 

There some highlighted projects: 

• Mirage OS: a library OS that supports the most common web protocols and enables the development of sealed Xen Project appliances that can be run without a guest operating system on any Xen Project based cloud - led by Cambridge University
• Support for ARMv7 and ARMv8 based servers - led by Citrix
• Improvements to Nested Virtualization: Virtualzation of guest machines - led by Intel
• Significant extensions to the Xen Project Security Modules and Flask - led by the National Security Agency
• A new virtualization mode called PVH (PV on HVM: improving performance by support PV, remove emulated hardware pocessing), which promises to combine the best aspects of hardware virtualization and paravirtualization - led by Oracle
• Significant performance and scalability improvements - led by SUSE

References:

1. Xen became Linux Foundation Collaborative Project: http://blog.xen.org/index.php/2013/04/15/xen-is-now-a-linux-foundation-collaborative-project/

2. Xen Proejct home: http://www.xenproject.org/

Openstack: How to solve pkg_resources.DistributionNotFound when executing glance

I installed and configured glance to try openstack grizzly. I followed the instruction from Grizzly install manual.
When I tried to create an image, I encountered the error and this was the error: 

$ sudo glance image-create --name="tty-linux-kernel" --disk-format=aki --container-format=aki < ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
Traceback (most recent call last):
  File "/usr/bin/glance", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2711, in <module>
    parse_requirements(__requires__), Environment()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 584, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: requests>=0.8.8

Glance has dependencies on other modules, for example, python-swiftclient, python-keystoneclient and so on.
Even, python-swiftclient which installed here has dependencies on its own.
In order to get a clue to solve the error, I printed the command "glance"

$ cat /usr/bin/glance
#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'python-glanceclient==0.9.0','console_scripts','glance'
__requires__ = 'python-glanceclient==0.9.0'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
    sys.exit(
        load_entry_point('python-glanceclient==0.9.0', 'console_scripts', 'glance')()
    )

It seemed that it was something wrong with modules related to "python-glanceclient". Next, I looked at /usr/lib/python2.7/dist-packages/python_glanceclient-0.9.0.egg-info/requires.txt file.
requires.txt fie is used to write a list of packages to install.

$ vi /usr/lib/python2.7/dist-packages/python_glanceclient-0.9.0.egg-info/requires.txt
prettytable&gh;=0.6,<0.8
python-keystoneclient>=0.2.3
pyOpenSSL
warlock>=0.7.0,<2

But, there was NO module named "requests" in here, so I tried to do other files which were indicated here. I opened the next file, requires.txt for keystoneclient.

$ vi /usr/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.egg-info/requires.txt
iso8601>=0.1.4
prettytable
requests>=0.8.8
simplejson
oslo.config>=1.1.0

I finally found "request" in here and I checked the version of my request module to compare the version number.

$ sudo dpkg -l | grep request
ii  python-requests                      0.8.2-1                                  elegant and simple HTTP library for Python, built for human beings

My python-requests module was 0.8.2. I changed it into:

$ vi /usr/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.egg-info/requires.txt
iso8601>=0.1.4
prettytable
requests>=0.8.2
simplejson
oslo.config>=1.1.0

I was able to solve my problem by doing this. However, I got another problem with this.

$ sudo glance image-create --name="tty-linux-kernel" --disk-format=aki --container-format=aki --file=ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
Authorization Failed: request() got an unexpected keyword argument 'verify'

This was caused because I had old-dated "python-requests", I needed to install 0.8.8 or higher to work "verify". I found how to solve from the site: https://answers.launchpad.net/keystone/+question/218143

"this duo to you OS has a low version of python-requests by defalut,now python-keystoneclient require a python-requests>0.8.8,<1.0 this bug has been fixed to master. you can first git pull the lastest python-keystoneclient code. then run pip install -r tools/pip-requires."

I downloaded new version of it from https://pypi.python.org/pypi/requests/1.2.0

$ sudo tar -xzvf ./requests-1.2.0.tar.gz 
$ cd requests-1.2.0
$ sudo ./setup.py build

Finally it worked.

References: 
1. http://pedrojimenez.github.io/blog/2012/12/12/ubuntu-python-keystoneclient-dependency-error/
2. https://answers.launchpad.net/keystone/+question/218143
3. http://www.pip-installer.org/en/latest/requirements.html
   

VMware: Lable of VirtualDevice only accept alphabetical characters, If not, got InvalidRequest error

vSphere API threw InvalidRequest error while I was trying to add a networkadapter of a VM.

com.vmware.vim25.InvalidRequest
at com.vmware.vim25.mo.VirtualMachine.reconfigVM_Task...

My code was following:

...
...
// Network Connection name
String[] network_name =  {"VM Network"};

// Get the existing network devices of the VM
List<VirtualDevice> listNetworkDevice = new ArrayList<VirtualDevice>();

for (VirtualDevice vDevice: vm.getConfig().getHardware().getDevice()) {
  if (vDevice.getDeviceInfo().getLabel().toLowerCase().indexOf("network") >= 0 || vDevice.getDeviceInfo().getLabel().indexOf("네트워크") >= 0)
    listNetworkDevice.add(vDevice);

}

// Get the label of network device 
String listNicLabel[] = listNetworkDevice.get(listNetworkDevice.size() - 1).getDeviceInfo().getLabel().split(" ");

int seqOfNIC = 0;
try {
  if( listNicLabel != null && listNicLabel.length >= 2)
    seqOfNIC = Integer.parseInt(listNicLabel[2]);                                                   
} catch (Exception e) {
    listNicLabel = new String[] { "Network", "Adapter" };
}

// Unique key that distinguish the device from other devices in the same VM. Here, it is just a temporary value
int nicKeyVal = 0;

for (int i = 0; i < customServerInfo.getNetworkBridgeName().length; i++) {                                 
             
  VirtualEthernetCard nic = new  VirtualVmxnet3();
  VirtualEthernetCardNetworkBackingInfo nicBacking = new VirtualEthernetCardNetworkBackingInfo();
  nicBacking.setDeviceName(customServerInfo.getNetworkBridgeName()[i]); // Network Connection
                
  String nicLabel = listNicLabel[0] + " " + listNicLabel[1] + " " + (++seqOfNIC);   
  
  Description info = new Description();
  info.setLabel(nicLabel);
  info.setSummary(customServerInfo.getNetworkBridgeName()[i]); 
  nic.setDeviceInfo(info);
                                  
  // type: "generated", "manual", "assigned" by VC
  nic.setAddressType("generated");
  nic.setBacking(nicBacking);
  nic.setKey(-1 * (4000 + (nicKeyVal++)));      
    
  VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
  deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.add);
  deviceConfigSpec.setDevice(nic);
    
  listDeviceConfigSpec.add(deviceConfigSpec);
            
}

if (listDeviceConfigSpec.size() > 0) {
  vmSpec.setDeviceChange(listDeviceConfigSpec.toArray(new VirtualDeviceConfigSpec[listDeviceConfigSpec.size()]));

  Task task = vm.reconfigVM_Task(vmSpec);
  ...
  ...
}

In my above coding, I got the last label from the existing NetworkAdapter in line 21. (My VM already had two networks) and replaced the last sequence number with new one in line 35.
e.g.)
<<The last seq and name>>    <<The new seq and name>>
Network Adapter 2        =>      Network Adapter 3

However, Labels of vDevices were displayed in the language which you chose one when you installed vCenter. I installed it with Korean language, so I actually got the result of labels, for example, 네트워크 어댑터 1, 네트워크 어댑터 2 (They both mean "Network Adapter").

In Label property of Description class in vSphere API, it doesn't accept characters other than Alphabetical(and number).
So, You should change it with alphabetical characters for Label property. I changed this like:

String nicLabel = "Network Adapter" + " " + (++seqOfNIC);

Description info = new Description();
info.setLabel(nicLabel);
...

When you get InvalidRequest error for VM Creation / Reconfig job and you try to debug it, it can be one of good solutions that you comment all of setter methods except for mandatory and then uncomment one setter method per testing, after that you can find where it is wrong in your code. It seems not to have easy solution about solvoing InvalidRequest error.


References:
1. http://www.jarvana.com/jarvana/view/org/jvnet/hudson/vijava/2120100824/vijava-2120100824-sources.jar!/com/vmware/vim25/mo/samples/vm/CreateVM.java

Book review: Nagios Core Administration Cookbook

I've involved in a cloud development project. I use Nagios core for monitoring servers and devices in this project. As we know, Nagios is one of famous open sourced monitoring tools. I needed references in dealing with Nagios. While I was reading this, it remind me of basic concept of Nagios again. 



There are some sections in every chapter. "How to do it" , "How it works" and "There's more". it describes how we start a certain feature of Nagios according to the section in "How to do it" and then explains its mechanism with easy way
in "How it works" and finally there is additional contents related to the chapter in "There's more"

There is impressive paragraph in the book that nagios is close to MONITORING FRAMWORK rather than monitoring tool. It is perfect that describes nagios characteristic because it has already provide ecosystem, for example, lots of plugins support . And it is important to know where to find if need specific plugin that doesn't meet requirements. Author instructed that how to find a plugin that the administrator needs and where. 

People who are unfamiliar with Nagios, in my opinion, they may be confused that Nagios has SOFT and HARD state in response to server or host failing. In chapter 3, author tries to explain the difference between two by giving an understandable example. 

At last, in chapter 11, he gives an explanation about add-on projects like NRPE, NSClient++, NDOUtils and SNMP trap. If you need to extend the feature of Nagios to provide better-fitting solution on your own production, this will very helpful to you.

Book link: http://www.packtpub.com/nagios-core-administration-cookbook/book