Warning :Remote branch XXX.. not found in upstream origin, using HEAD instead when cloning xen source

When you clone git repository for xen 4.3, you would get the following message:
"Remote branch RELEASE-4.3.0 not found in upstream origin, using HEAD instead"

This is caused because of git version, prior version to 1.7.10 don't support checking out tag.

# My git version 
$ git --version
git version 1.7.9.5

# Download Xen source
$ git clone -b RELEASE-4.3.0 git://xenbits.xen.org/xen.git
Cloning into 'xen'...
remote: Counting objects: 272877, done.
remote: Compressing objects: 100% (55538/55538), done.
remote: Total 272877 (delta 214914), reused 270399 (delta 212700)
Receiving objects: 100% (272877/272877), 53.61 MiB | 1.75 MiB/s, done.
Resolving deltas: 100% (214914/214914), done.

warning: Remote branch RELEASE-4.3.0 not found in upstream origin, using HEAD instead

Simply, you are able to check out additionally.

$ cd xen
$ git checkout RELEASE-4.3.0
Note: checking out 'RELEASE-4.3.0'.

Or, you can re-install the newer version of git.

# Remove the old git
$ sudo apt-get remove git

# Download git
$ wget https://git-core.googlecode.com/files/git-1.8.1.2.tar.gz
$ tar -xzvf ./git-1.8.1.2.tar.gz
$ cd git-1.8.1.2
$ make prefix=/usr/local all
$ sudo make prefix=/usr/local install
$ git --version
git version 1.8.1.2

# Download Xen source 
$ git clone -b RELEASE-4.3.0 git://xenbits.xen.org/xen.git
Cloning into 'xen'...
remote: Counting objects: 272877, done.
remote: Compressing objects: 100% (55538/55538), done.
remote: Total 272877 (delta 214914), reused 270399 (delta 212700)
Receiving objects: 100% (272877/272877), 53.61 MiB | 4.01 MiB/s, done.
Resolving deltas: 100% (214914/214914), done.
Note: checking out 'f8cc9c2b713b1739b1d3d324716547fa639dce86'.

References:
1. http://stackoverflow.com/questions/17216067/git-clone-b-tag-results-in-warning-remote-branch-not-found-in-upstream-origi/17216068#17216068

Xen 4.3 RC1 installation

I'd like to share how to install Xen 4.3 RC (Release Candidate) as part of a testing on Xen test day. I freshly installed Ubuntu 12.04 on my physical machine.

Xen source is able to download from git repository or tarball.

# download xen source from git repository
$ mkdir ~/xen-4.3
$ cd ~/xen-4.3
$ git clone -b 4.3.0-rc1 git://xenbits.xen.org/xen.git
Cloning into 'xen'...
remote: Counting objects: 271307, done.
remote: Compressing objects: 100% (59143/59143), done.
remote: Total 271307 (delta 213525), reused 263420 (delta 207531)
Receiving objects: 100% (271307/271307), 53.31 MiB | 748 KiB/s, done.
Resolving deltas: 100% (213525/213525), done.
Note: checking out '4de97462d34f7b74c748ab67600fe2386131b778'.

If you not familiar with git, you can download tarball source

$ wget http://bits.xensource.com/oss-xen/release/4.3.0-rc1/xen-4.3.0-rc1.tar.gz
--2013-05-08 17:55:22--  http://bits.xensource.com/oss-xen/release/4.3.0-rc1/xen-4.3.0-rc1.tar.gz
Resolving bits.xensource.com (bits.xensource.com)... 63.80.4.58, 63.80.4.218
Connecting to bits.xensource.com (bits.xensource.com)|63.80.4.58|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16375588 (16M) [application/x-gzip]
Saving to: `xen-4.3.0-rc1.tar.gz'

34% [=================>                                   ] 5,670,832    468K/s  eta 25s

Installing build tools and related modules.

$ sudo apt-get install build-essential

$ sudo apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif 
$ sudo apt-get install texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial
$ sudo apt-get install make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg-dev
$ sudo apt-get install iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils libyajl-dev gettext markdown
$ sudo apt-get install libpixman-1-0 libpixman-1-dev 
$ sudo apt-get install libc6-dev*

Confuging the environment for installing.

$ sudo ./configure
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
Will build the following subsystems:
  xen
  kernels
  tools
  stubdom
  docs
configure: creating ./config.status
config.status: creating ./config/Toplevel.mk
=== configuring in tools (/downloads/xen-4.3.0-rc1/tools)
configure: running /bin/bash ./configure --disable-option-checking '--prefix=/usr/local'  --cache-file=/dev/null --srcdir=.
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
...
checking for fig2dev... /usr/bin/fig2dev
checking for pod2man... /usr/bin/pod2man
checking for pod2html... /usr/bin/pod2html
checking for pod2text... /usr/bin/pod2text
checking for markdown... /usr/bin/markdown
configure: creating ./config.status
config.status: creating ../config/Docs.mk

To build all subsystems, use "dist" and then run "make install" to install xen.

################
# Build Xen
################
$ sudo make dist
make -C xen install
make[1]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -f Rules.mk _install
make[2]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -C tools
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
[ -d figlet ] && make -C figlet
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
gcc -o figlet figlet.c
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make symbols
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
gcc -Wall -Werror -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -Wdeclaration-after-statement -o symbols symbols.c
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[3]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make -f /home/yeonki/xen-4.3/xen/xen/Rules.mk include/xen/compile.h
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -C tools
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
[ -d figlet ] && make -C figlet
make[5]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make[5]: `figlet' is up to date.
make[5]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make symbols
make[5]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[5]: `symbols' is up to date.
make[5]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
 __  __            _  _    _____                    _        _     _
 \ \/ /___ _ __   | || |  |___ /    _   _ _ __  ___| |_ __ _| |__ | | ___
  \  // _ \ '_ \  | || |_   |_ \ __| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
  /  \  __/ | | | |__   _| ___) |__| |_| | | | \__ \ || (_| | |_) | |  __/
 /_/\_\___|_| |_|    |_|(_)____/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
...
make[1]: Leaving directory `/home/yeonki/xen-4.3/xen/docs'
install -d -m0755 -p /home/yeonki/xen-4.3/xen/dist/
install -m0644 -p ./COPYING /home/yeonki/xen-4.3/xen/dist
install -m0644 -p ./README /home/yeonki/xen-4.3/xen/dist
install -m0755 -p ./install.sh /home/yeonki/xen-4.3/xen/dist

################
# Install Xen
################
$ sudo make install
make -C xen install
make[1]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -f Rules.mk _install
make[2]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
make -C tools
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
[ -d figlet ] && make -C figlet
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make[4]: `figlet' is up to date.
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools/figlet'
make symbols
make[4]: Entering directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[4]: `symbols' is up to date.
make[4]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make[3]: Leaving directory `/home/yeonki/xen-4.3/xen/xen/tools'
make -f /home/yeonki/xen-4.3/xen/xen/Rules.mk include/xen/compile.h
make[3]: Entering directory `/home/yeonki/xen-4.3/xen/xen'
 __  __            _  _    _____                    _        _     _
 \ \/ /___ _ __   | || |  |___ /    _   _ _ __  ___| |_ __ _| |__ | | ___
  \  // _ \ '_ \  | || |_   |_ \ __| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
  /  \  __/ | | | |__   _| ___) |__| |_| | | | \__ \ || (_| | |_) | |  __/
 /_/\_\___|_| |_|    |_|(_)____/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
...
set -x; make -C figs
+ make -C figs
make[2]: Entering directory `/home/yeonki/xen-4.3/xen/docs/figs'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/home/yeonki/xen-4.3/xen/docs/figs'
rm -rf //usr/local/share/doc/xen
install -d -m0755 -p //usr/local/share/doc/xen
install -d -m0755 -p //usr/local/share/man
cp -R man1 //usr/local/share/man
cp -R man5 //usr/local/share/man
[ ! -d html ] || cp -R html //usr/local/share/doc/xen
make[1]: Leaving directory `/home/yeonki/xen-4.3/xen/docs'

Xen is successfully installed. The next is to check whether xen boot files has created in /boot/ directory.

ls -al /boot/xen*
lrwxrwxrwx 1 root root       19 Jun  5 15:50 /boot/xen-4.3.gz -> xen-4.3-unstable.gz
-rw-r--r-- 1 root root   821559 Jun  5 15:50 /boot/xen-4.3-unstable.gz
lrwxrwxrwx 1 root root       19 Jun  5 15:50 /boot/xen-4.gz -> xen-4.3-unstable.gz
lrwxrwxrwx 1 root root       19 Jun  5 15:50 /boot/xen.gz -> xen-4.3-unstable.gz
-rw-r--r-- 1 root root 14012010 Jun  5 15:50 /boot/xen-syms-4.3-unstable

# Execute update-grub
$ sudo update-grub
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
dpkg: warning: version 'syms-4.3-unstable' has bad syntax: version number does not start with digit
dpkg: warning: version '/boot/xen.gz' has bad syntax: version number does not start with digit
dpkg: warning: version 'syms-4.3-unstable' has bad syntax: version number does not start with digit
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
dpkg: warning: version 'syms-4.3-unstable' has bad syntax: version number does not start with digit
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found memtest86+ image: /memtest86+.bin
done

# When run "update-grub", an error was occurred. 
# To fix this, remove xen-syms-4.3-unstable and xen.gz
$ sudo rm /boot/xen-syms-4.3-unstable
$ sudo rm /boot/xen.gz

# Re-run update-grub
$ sudo update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found linux image: /boot/vmlinuz-3.2.0-23-generic
Found initrd image: /boot/initrd.img-3.2.0-23-generic
Found memtest86+ image: /memtest86+.bin
done

#################################
# Register xen moduels as service
#################################
$ sudo update-rc.d xencommons defaults 19 18
update-rc.d: warning: xencommons start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
Adding system startup for /etc/init.d/xencommons ...
   /etc/rc0.d/K18xencommons -> ../init.d/xencommons
   /etc/rc1.d/K18xencommons -> ../init.d/xencommons
   /etc/rc6.d/K18xencommons -> ../init.d/xencommons
   /etc/rc2.d/S19xencommons -> ../init.d/xencommons
   /etc/rc3.d/S19xencommons -> ../init.d/xencommons
   /etc/rc4.d/S19xencommons -> ../init.d/xencommons
   /etc/rc5.d/S19xencommons -> ../init.d/xencommons

$ sudo update-rc.d xendomains defaults 21 20
update-rc.d: warning: xendomains start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
Adding system startup for /etc/init.d/xendomains ...
   /etc/rc0.d/K20xendomains -> ../init.d/xendomains
   /etc/rc1.d/K20xendomains -> ../init.d/xendomains
   /etc/rc6.d/K20xendomains -> ../init.d/xendomains
   /etc/rc2.d/S21xendomains -> ../init.d/xendomains
   /etc/rc3.d/S21xendomains -> ../init.d/xendomains
   /etc/rc4.d/S21xendomains -> ../init.d/xendomains
   /etc/rc5.d/S21xendomains -> ../init.d/xendomains

$ sudo update-rc.d xen-watchdog defaults 22 23
update-rc.d: warning: xen-watchdog start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (2 3 5)
Adding system startup for /etc/init.d/xen-watchdog ...
   /etc/rc0.d/K23xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc1.d/K23xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc6.d/K23xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc2.d/S22xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc3.d/S22xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc4.d/S22xen-watchdog -> ../init.d/xen-watchdog
   /etc/rc5.d/S22xen-watchdog -> ../init.d/xen-watchdog

#################################
# rebooting boot with xen
#################################
$ cat /boot/grub/grub.cfg | grep Xen
submenu "Xen 4" {
menuentry 'Ubuntu GNU/Linux, with Xen 4 and Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4 ...'
menuentry 'Ubuntu GNU/Linux, with Xen 4 and Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4 ...'
submenu "Xen 4.3" {
menuentry 'Ubuntu GNU/Linux, with Xen 4.3 and Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3 ...'
menuentry 'Ubuntu GNU/Linux, with Xen 4.3 and Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3 ...'
submenu "Xen 4.3-unstable" {
menuentry 'Ubuntu GNU/Linux, with Xen 4.3-unstable and Linux 3.2.0-23-generic' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3-unstable ...'
menuentry 'Ubuntu GNU/Linux, with Xen 4.3-unstable and Linux 3.2.0-23-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os --class xen {
        echo    'Loading Xen 4.3-unstable ...'


$ sudo vi /etc/default/grub
...
GRUB_DEFAULT='Ubuntu GNU/Linux, with Xen 4.3 and Linux 3.2.0-23-generic'
..

$ sudo update-grub
$ sudo boot

References:
1. http://wiki.xen.org/wiki/Archived/Xen_4.3_RC1_test_instructions
2. http://wiki.xen.org/wiki/Compiling_Xen_From_Source
3. http://locatrix.com/build-and-install-xen-4-2-1-from-source-on-ubuntu-12-10/
4. http://blog.heidt.biz/blog/2012/09/17/setting-up-ubuntu-12-dot-04-precise-pangolin-as-xen-dom0/
5. http://lists.xen.org/archives/html/xen-users/2013-05/msg00136.html

Xen is moving foward

Although Xen has not been included as default hypervisor in RHEL and has been losing some supporters. Xen is still attractive hypervisor. Xen is trying to support platforms in many ways. 

There some highlighted projects: 

• Mirage OS: a library OS that supports the most common web protocols and enables the development of sealed Xen Project appliances that can be run without a guest operating system on any Xen Project based cloud - led by Cambridge University
• Support for ARMv7 and ARMv8 based servers - led by Citrix
• Improvements to Nested Virtualization: Virtualzation of guest machines - led by Intel
• Significant extensions to the Xen Project Security Modules and Flask - led by the National Security Agency
• A new virtualization mode called PVH (PV on HVM: improving performance by support PV, remove emulated hardware pocessing), which promises to combine the best aspects of hardware virtualization and paravirtualization - led by Oracle
• Significant performance and scalability improvements - led by SUSE

References:

1. Xen became Linux Foundation Collaborative Project: http://blog.xen.org/index.php/2013/04/15/xen-is-now-a-linux-foundation-collaborative-project/

2. Xen Proejct home: http://www.xenproject.org/

How to change HVM's CD on Xen

I was trying to change CD of the HVM during online on Xen, but I couldn't. I'd got an the following error when I tried to click disconnect button on the pane "virtual hardware details" of the virtual machine manager.

Error changing VM configuration: XML error: failed to parse domain description
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/details.py", line 2323, in _change_config_helper
self.vm.redefine_cached()
File "/usr/share/virt-manager/virtManager/domain.py", line 326, in redefine_cached
self._redefine_xml(xml)
File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 147, in _redefine_xml
return self._redefine_helper(origxml, newxml)
File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 137, in _redefine_helper
self._define(newxml)
File "/usr/share/virt-manager/virtManager/domain.py", line 762, in _define
self.conn.define_domain(newxml)
File "/usr/share/virt-manager/virtManager/connection.py", line 805, in define_domain
return self.vmm.defineXML(xml)
File "/usr/lib64/python2.6/site-packages/libvirt.py", line 2097, in defineXML
if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)
libvirtError: XML error: failed to parse domain description

Instead, I did shutdown of the VM normally and then change the path of CD in configuration file. As I know, VM management generally supports changing CDs without re-boot. I found the answer from the site. 

I tried to change CD of the VM named "vcenter".
$ xm list
Name                               ID   Mem VCPUs      State   Time(s)
Domain-0                         0    3500      6         r-----   2564788.5
......
vcenter                           16    3012     2         -b----   490.1

# Lists the virtual block devices for the VM.
$ xm block-list vcenter
Vdev  BE handle state evt-ch ring-ref BE-path
768      0    0     1       -1       -1       /local/domain/0/backend/vbd/16/768
5632    0    0     1       -1       -1       /local/domain/0/backend/vbd/16/5632

* If I run command with --long, it is able to see more detailed information:
$ xm block-list vcenter --long
(768
    ((backend-id 0)
        (virtual-device 768)
        (device-type disk)
        (state 1)
        (backend /local/domain/0/backend/vbd/16/768)
    )
)
(5632
    ((backend-id 0)
        (virtual-device 5632)
        (device-type cdrom)
        (state 1)
        (backend /local/domain/0/backend/vbd/16/5632)
    )
)

# Check the first device, but it is main virtual hard drive.
xenstore-read /local/domain/0/backend/vbd/16/768/params
/var/lib/libvirt/images/vcenter.img

# Check the next device, it is the media which I try to change. 
$ xenstore-read /local/domain/0/backend/vbd/16/5632/params
/isos/vwware-5-vcenter.iso

# Run the command "xenstore-write" 
$ xenstore-write /local/domain/0/backend/vbd/16/5632/params /isos/vmware-5.1-vcenter.iso

# Confirm it, when returns to OS, CD has been changed. 
$ xenstore-read /local/domain/0/backend/vbd/16/5632/params
/isos/vmware-5.1-vcenter.iso


References: 
1. http://www.novell.com/support/kb/doc.php?id=3172795
2. http://www.virtuatopia.com/index.php/Adding_Disk,_CDROM_and_DVD_Devices_to_a_Running_Xen_domainU_Guest_System

How to resolve time drift of Windows HVM on Xen

I have two windows HVMs(Hardware Assisted VM). Both are running into VMs on a host which is installed Xen 4.0 and CentOS 6.2.

Whenever I log into those servers, I find that times are faster than host's and these time gaps are getting bigger.

Usually, commercial Hypervisors such as XenServer, VMware and Hyper-V keep time synchronization between the host and the guest through installing a program called "guest tool".

However, In the case, they not provided any tool in Xen (not XenServer)
When I was stuck in the same situation on Linux,  I resolved time drift in the case of Linux Operating system by setting NTP. 
So, It would recommend that NTP is best solutions for Windows HVM, too.

"For Windows guests, time is initially driven from the control domain clock. However, XenServer also stores for each Windows VM a time offset. This represents the difference between the control domain time and the guest, and is persisted for each VM. We recommend running a reliable NTP service in the control domain to set a global time accurately across all VMs in the host, and not one NTP service in each individual Windows VM."

This is a hint from the online document.

To enable NTP on Windows, it is able to change it by configuring the Windows Time Service. Time service is registered as Windows Service automatically and it is also provide the way of changing in command prompt.   

By running command prompt, At first, Check current time source of the computer.

C:\>w32tm /query /status
Leap Indicator: 0(no warning)
Stratum: 1 (primary reference - syncd by radio clock)
Precision: -6 (15.625ms per tick)
Root Delay: 0.0000000s
Root Dispersion: 10.0000000s
ReferenceId: 0x4C4F434C (source name:  "LOCL")
Last Successful Sync Time: 10/4/2012 4:13:13 AM
Source: Free-running System Clock
Poll Interval: 6 (64s)

Then, It adds a ntp server to the list. 

C:\>w32tm /config /manualpeerlist:kr.pool.ntp.org /syncfromflags:manual /reliable:yes /update
The command completed successfully.

Verify it.
C:\>w32tm /query /status
Leap Indicator: 0(no warning)
Stratum: 3 (secondary reference - syncd by (S)NTP)
Precision: -6 (15.625ms per tick)
Root Delay: 0.0370178s
Root Dispersion: 0.8297591s
ReferenceId: 0xD3278804 (source IP:  211.39.136.4)
Last Successful Sync Time: 10/4/2012 11:02:09 AM
Source: kr.pool.ntp.org
Poll Interval: 7 (128s)

Loot at a member server of the active directory, you don't have to change it again. Any member servers of the AD, they are synchronizing the time with AD.   

C:\>w32tm /query /status
Leap Indicator: 0(no warning)
Stratum: 4 (secondary reference - syncd by (S)NTP)
Precision: -6 (15.625ms per tick)
Root Delay: 0.0682526s
Root Dispersion: 7.8910279s
ReferenceId: 0xC0A814A8 (source IP:  192.168.20.168) <= IP address of AD
Last Successful Sync Time: 10/4/2012  11:06:54 AM
Source: ad
Poll Interval: 10 (1024s)

If you can't see the same time with AD, you can restart the "Windows Time Service" in services panel.

References: 

1. http://technet.microsoft.com/en-us/library/cc731191(v=ws.10).aspx
2. http://www.thirdtier.net/2011/02/how-to-set-an-ntp-time-server-in-windows-2008-sbs-2008-and-sbs-2011/

When starting Ubuntu domU on xen, an error raised: Boot loader didn't return any data

A few days ago, I upgraded a server's OS and hypervisor. I had a Ubuntu 12.04 VM (PVM) on the old system and I migrated the VM to new system, of course, this was running well on old system.

                         [old]                         [new]

OS:               CentOS 5.6       =>      CentOS 6.2 

Hypervisor:     Xen 3.0.3         =>      Xen 4.0.1

But, It raised an error when I started the VM on new one.     

Error: Boot loader didn't return any data!

/var/log/xen/xend.log was: 

[2012-04-27 18:13:32 6622] DEBUG (XendDomainInfo:3053) XendDomainInfo.destroy: domid=24

[2012-04-27 18:13:32 6622] DEBUG (XendDomainInfo:2416) No device model

[2012-04-27 18:13:32 6622] DEBUG (XendDomainInfo:2418) Releasing devices

[2012-04-27 18:13:32 6622] ERROR (XendDomainInfo:106) Domain construction failed

Traceback (most recent call last):

  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 104, in create vm.start()

  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 469, in start XendTask.log_progress(31, 60, self._initDomain)

  File "/usr/lib64/python2.6/site-packages/xen/xend/XendTask.py", line 209, in log_progress retval = func(*args, **kwds)

  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 2820, in _initDomain self._configureBootloader()

  File "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py", line 3266, in _configureBootloaderbootloader_args, kernel, ramdisk, args)

  File "/usr/lib64/python2.6/site-packages/xen/xend/XendBootloader.py", line 215, in bootloader raise VmError, msg VmError: Boot loader didn't return any data!

I was googling and the search result recommended that firstly, excute the command "/usr/bin/pygrub with the VM image and secondly, boot as the py-grub.    


Run pygrub command to check whether the vm image boot normally 
$ /usr/bin/pygrub ./vm01.img

Using to parse /boot/grub/grub.cfg
WARNING:root:Unknown directive load_video
WARNING:root:Unknown directive terminal_output
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive else
WARNING:root:Unknown directive export
WARNING:root:Unknown image directive recordfail
Traceback (most recent call last):
  File "/usr/bin/pygrub", line 713, in
    chosencfg = run_grub(file, entry, fs, incfg["args"])
  File "/usr/bin/pygrub", line 548, in run_grub
    g = Grub(file, fs)
  File "/usr/bin/pygrub", line 204, in __init__
    self.read_config(file, fs)
  File "/usr/bin/pygrub", line 412, in read_config
    self.cf.parse(buf)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 400, in parse
    self.add_image(Grub2Image(title, img))
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 316, in __init__
    _GrubImage.__init__(self, title, lines)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 85, in __init__
    self.reset(lines)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 101, in reset
    self._parse(lines)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 96, in _parse
    map(self.set_from_line, lines)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 326, in set_from_line
    setattr(self, self.commands[com], arg.strip())
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 104, in set_root
    self._root = GrubDiskPart(val)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 55, in __init__
    (self.disk, self.part) = str.split(",", 2)
  File "/usr/lib64/python2.6/site-packages/grub/GrubConf.py", line 80, in set_part
    self._part = int(val)
ValueError: invalid literal for int() with base 10: 'msdos1'


I found something was wrong related to keyword "msdos1", the next step is check the grub.cfg file.



Boot the VM using pv-grub to check which is wrong in the grub.cfg
# Modify config file to boot using pv-grub
$ vi ./vm01.cfg


# Comment out default bootloader
# bootloader="/usr/bin/pygrub"



# Add Pvgrub lines (Ubuntu uses grub.cfg)
kernel="/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra="(hd0,0)/grub/grub.cfg"




$ xm create vm01.cfg -c
    GNU GRUB  version 0.97  (1048576K lower / 0K upper memory)
       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]
grubdom> cat (hd0,0)/grub/grub.cfg
......
set root='(hd0,msdos1)'
......
......


CentOS didn't recognize the keyword "msdos1" which used in the Ubuntu. ( I also created a Ubuntu 12.04 VM on Ubuntu dom0, it was booted successfully) Looking at the /boot/grub/menu.list in CentOS, it defined as "root (hd0,0)"  
I changed the value of the property "root", before change, I needed to mount the VM image.



Mount VM image and change values
$ fdisk -ul ./vm01.img

You must set cylinders.
You can do this from the extra functions menu.

Disk ./vm01.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b0b36

      Device          Boot      Start         End             Blocks     Id  System
./vm01.img1        *        2048      499711          248832   83  Linux
Partition 1 does not end on cylinder boundary.
./vm01.img2                501758   104855551    52176897    5  Extended
Partition 2 has different physical/logical endings: phys=(1023, 254, 63) logical=(6526, 243, 53)
./vm02.img5               501760   104855551    52176896   8e  Linux LVM

# offset: 2048 * 512 = 1048576 

$ losetup -o 1048576 /dev/loop1 ./vm01.img

$ mkdir /mnt/tmp

$ mount /dev/loop1 /mnt/tmp/

$ ls /mnt/tmp

abi-3.2.0-24-generic     initrd.img-3.2.0-24-generic  memtest86+_multiboot.bin

config-3.2.0-24-generic  lost+found                   System.map-3.2.0-24-generic

grub                     memtest86+.bin               vmlinuz-3.2.0-24-generic

$ cp /mnt/tmp/grub/grub.cfg /mnt/tmp/grub/grub.cfg.orig

$ vi /mnt/tmp/grub/grub.cfg

Replace set root='(hd0,msdos1)'   with  set root='(hd0,0)'

Once update the values, needs to umount.
$ umount /mnt/tmp
$ losetup -d /dev/loop1


Try creating the VM again and the booting was fine.
$ xm create vm01.cfg
$ xm list 

Name                          ID  Mem   VCPUs      State   
Domain-0                     0   2987     6          r-----     
vm01                          93  1024     1          r-----     

............

References: 

1. Fix pygrub booting of debian squeeze: http://www.magicspace.eu/linux/fix-pygrub-booting-of-debian-squeeze/

2. Mount KVM/Xen virtual disk image outside guest OS: http://blog.leenix.co.uk/2010/07/howto-mount-kvmxen-virtual-disk-image.html

3. PVGrub Howto: http://wiki.xen.org/wiki/PVGrub_HowTo

Creating Windows HVM on various Xens

This post is describing how I had worked to create Windows VMs (HVM type) on xen hypervisor. I needed to make test environment for VMware 5. vCenter server needs to be installed on Windows machine and needs to join to Windows Active Directory. In this environment, at least two windows machines was needed so I decided to install these in the VM. 


I've already had three physical machines. (a Redhat 5.6 host which xen enabled, a XCP + Ubuntu 12.04 host and a WMware ESXi host)
At first, I tried to create those VMs on XCP. 


1. XCP 1.5 beta on Ubuntu 12.04
I created the first VM via xe command, it was installed without an error. 
However, It was not able to connect to the network.
Click this link to know how it didn't work. https://plus.google.com/103057145276112976207/posts/UrGA7KqVG7m
When I tried it again by XenCenter 5.6. It didn't work, too.


Linux VMs were okay (PV and HVM) on this host, they had no problem for connection.
Next, I tried to do on Redhat 5.6.


2. Redhat 5.6
This machine was kernel-xen-2.6.18-308.4.1.el5 and xen-3.0.3-135.el5


1) Got an error when I created a vm via xm command, /var/log/xen/xend-debug was:  
Traceback (most recent call last):
  File "/usr/lib64/python2.4/logging/handlers.py", line 71, in emit
    if self.shouldRollover(record):
  File "/usr/lib64/python2.4/logging/handlers.py", line 149, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib64/python2.4/logging/__init__.py", line 617, in format
    return fmt.format(record)
  File "/usr/lib64/python2.4/logging/__init__.py", line 405, in format
    record.message = record.getMessage()
  File "/usr/lib64/python2.4/logging/__init__.py", line 276, in getMessage
    msg = msg % self.args
TypeError: int argument required


It's a known bug.
https://bugzilla.redhat.com/show_bug.cgi?id=279581


2) Couldn't finish installation process via virt-manager
It stopped at this screen and it didn't go to the next stage. 



It seemed that xen 3.0 too out-dated and made my mind to upgrade xen version. 


3. Upgrade to xen 3.3.x or 4.x on Redhat 5.6
I downloaded from http://www.gitco.de/linux/x86_64/centos/5/ and install newer xen 3.3.x or 4.x.
After install newer version, xen kernel was the same with before update.
xen only was changed to new one.
But, it didn't nominally booted and caused continuous reboot.


4. Install CentOS 6.2 and xen 4.0.1
I installed CentOS 6.2 and xen 4.0.1 on the machine that used be redhat 5.6. 
As you know, KVM has been the only default hypervisor since CentOS 6.x. so xen users has to install xen manually. 
gitco.de site provies xen RPM and related others.


# Install xen
$ cd /etc/yum.repos.d
$ wget http://www.gitco.de/linux/x86_64/centos/6/gitco-centos6-x86_64.repo
$ yum install xen

# check kernel version before install xen-kernel

$ uname -a

Linux mcloud.******* 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6

19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux

# Install xen aware kernel

$ yum install kernel kernel-devel kernel-headers kernel-firmware

# check if xen was normally installed.

$ ls -al /boot/xen*

-rw-r--r--. 1 root root   677326 Nov 27  2010 /boot/xen-4.0.1.gz

lrwxrwxrwx. 1 root root       12 Apr 26 17:23 /boot/xen.gz -> xen-4.0.1.gz

-rw-r--r--. 1 root root 12091421 Nov 27  2010 /boot/xen-syms-4.0.1

# Modify boot file

$ vi /boot/grub/grub.conf

...

default=0

...

title CentOS (2.6.32.26-174.1.xendom0.el6.x86_64)

       root (hd0,0)

       kernel /xen.gz

       module /vmlinuz-2.6.32.26-174.1.xendom0.el6.x86_64 ro

root=/dev/mapper/....

       module /initramfs-2.6.32.26-174.1.xendom0.el6.x86_64.img

title CentOS (2.6.32-220.el6.x86_64)

       root (hd0,0)

       kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/mapper/...

       initrd /initramfs-2.6.32-220.el6.x86_64.img

$ reboot

Note: After reboot, it was causing continuous reboot !!!. If you have the same problem with this, you should build kernel for xen. For me, I picked the way that downloaded its rpm source and built it.

Here's my way of building the rpm source.

# Remove packages that I previously installed 

$ yum remove kernel-firmware kernel-headers kernel-devel kernel

# Get the kernel-xen rpm source

$ mkdir /root/src

$ cd /root/src

$ wget http://www.gitco.de/linux/x86_64/centos/6/src/kernel-2.6.32.26-174.1.xendom0.el6.src.rpm

# Install rpm-build tool

$ yum install rpm-build

$ rpm -i ./*.src.rpm

.....

warning: user rpmbuild does not exist - using root

.....

=> Ignore these warnings

$ cd /root/rpmbuild/SPECS/

$ rpmbuild -bb kernel.spec

error: Failed build dependencies:

       gcc >= 3.4.2 is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

       redhat-rpm-config is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

       xmlto is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

       asciidoc is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

       elfutils-libelf-devel is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

       zlib-devel is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

       binutils-devel is needed by kernel-2.6.32.26-174.1.xendom0.el6.x86_64

# Meet required dependencies

$ yum install gcc redhat-rpm-config xmlto asciidoc elfutils-libelf-devel zlib-devel binutils-devel

# It took about 15 minutes for rpmbuild

$ rpmbuild -bb kernel.spec

$ cd ../RPMS/x86_64/

$ ls -al

total 265816

-rw-r--r--. 1 root root  19851580 Apr 26 19:56

kernel-2.6.32.26-174.1.xendom0.el6.x86_64.rpm

-rw-r--r--. 1 root root 212588868 Apr 26 19:57

kernel-debuginfo-2.6.32.26-174.1.xendom0.el6.x86_64.rpm

-rw-r--r--. 1 root root  32409920 Apr 26 19:56

kernel-debuginfo-common-x86_64-2.6.32.26-174.1.xendom0.el6.x86_64.rpm

-rw-r--r--. 1 root root   6543740 Apr 26 19:56

kernel-devel-2.6.32.26-174.1.xendom0.el6.x86_64.rpm

-rw-r--r--. 1 root root    791828 Apr 26 19:56

kernel-headers-2.6.32.26-174.1.xendom0.el6.x86_64.rpm

$ chmod u+x *

# needs to one more dependancy 

$ yum install kernel-firmware

$ rpm -i --force *

ldconfig: /etc/ld.so.conf.d/kernel-2.6.32.26-174.1.xendom0.el6.x86_64.conf:6:

hwcap index 1 already defined as nosegneg

$ rpm -qa kernel

kernel-2.6.32-220.el6.x86_64

kernel-2.6.32.26-174.1.xendom0.el6.x86_64

# Modify boot file

$ vi /boot/grub/grub.conf

...

default=0

...

title CentOS (2.6.32.26-174.1.xendom0.el6.x86_64)

       root (hd0,0)

       kernel /xen.gz

       module /vmlinuz-2.6.32.26-174.1.xendom0.el6.x86_64 ro

root=/dev/mapper/....

       module /initramfs-2.6.32.26-174.1.xendom0.el6.x86_64.img

title CentOS (2.6.32-220.el6.x86_64)

       root (hd0,0)

       kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/mapper/...

       initrd /initramfs-2.6.32-220.el6.x86_64.img

$ reboot

$ xm list ( It also worked by excute "xl list"

Name                                        ID   Mem VCPUs      State   Time(s)

Domain-0                                     0 14534     8        r--     35.7

This time, I was finally done to create the windows VM. 
After create, I found some less important problems.
- VNC of virt-manager sometimes didn't work. ( I used another RD client.)
- For windows vm, different values between current memory and max memory may cause continuous reboot. ( Once I set the same value with them, It worked.)

References:

- Install xen 4.0 on CentOS 6.0 (Korean language): http://guni.tistory.com/328

XCP: Comparison between XCP and XenServer

Functionality comparison between XCP and XenServer is following. This content  comes from http://wiki.xen.org/wiki/XCP/XenServer_Feature_Matrix

Features	Xen Cloud Platform	XenServer Free	XenServer Advanced	XenServer Enterprise	XenServer Platinum
Cost/Licensing	Free/Open Source (Multiple Licenses)	Free/Citrix EULA	Paid/Citrix EULA
XenServer hypervisor	X	X	X	X	X
IntelliCache	X	X	X	X	X
Resilient distributed management architecture	X	X	X	X	X
VM disk snapshot and revert	X	X	X	X	X
XenCenter management	X	X	X	X	X
Conversion tools	X	X	X	X	X
XenMotion® live Migration	X	X	X	X	X
Heterogeneous pools	X		X	X	X
Dynamic Memory Control	X		X	X	X
Performance alerting and reporting	X		X	X	X
Distributed virtual switching management tool			X	X	X
High availability			X	X	X
Automated VM protection and recovery	X		X	X	X
Host power management	X			X	X
Live memory snapshot and revert	X			X	X
Role-based administration	X			X	X
Dynamic workload balancing				X	X
Provisioning services (virtual)				X	X
StorageLink				X	X
Web self-service with delegated admin	X			X	X
Site recovery	X				X
Lab manager with self-service portal					X
Provisioning services (physical)					X

Openstack: 1. XCP install & configuration

To test openstack compute (Nova) with xen based machine, it needs XCP (Xen Cloud  Planform) or Xenserver. Openstack compute supports KVM, XenServer/XCP and ESXi hypervisor. I chose XCP because it needs no license. 


XCP is open source version of XenServer which has the same toolstack with XerServer. As a Cronos project, it is able to use XCP on Ubuntu 12.04 called Precise. (For Fedora and Centos users, they should wait until May. XCP team is porting xen on Fedora and CentOS, It will be finished by May)


I prepared the following servers for the test.
- 1 Virtual Machine: Openstack compute, installed Ubuntu.
- 1 Physical Machine : Openstack Controller, Ubuntu 12.04 beta (XCP 1.5 beta)




When creating virtual machine on XCP, it has to create a storage repository first. XCP uses SR (Storage Repository) to store virtual machine images, ISO files and templates. SR supports for IDE, SATA, SCSI and SAS drives locally connected, and iSCSI, NFS, SAS and Fibre Channel remotely connected.  


I have known storage repository concept since Oracle VM. I used NFS as a shared resository. NFS also can be used in XCP. However, I only have one physical machine this time. There are two physical hard disks on the server. One is for operating system and the other is for storage of XCP.     


Set the environment of XCP 
As a result of Cronos project, it is able to install xcp using agt-get command on Ubuntu 12.04.

1) Set up software repositories
$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:ubuntu-xen-org/xcp-unstable

2) Workaround xcp-networkd missing file
$ mkdir /etc/xcp
$ echo "bridge" >> /etc/xcp/network.conf

3) Install xapi: This step installs XCP's xapi and all its dependencies, including the Xen hypervisor.
$ apt-get pciutils
$ apt-get update
$ apt-get install xcp-xapi
$ apt-get install xcp-xe

4) Workaround VM's not going to power-state: halted after shutdown

$ vi /usr/lib/xcp/scripts/vif

remove)    =>   remove|offline)
    if [ "${TYPE}" = "vif" ] ;then
        call_hook_script $DOMID "${ACTION}"
        # Unclear whether this is necessary, since netback also does it:
        logger -t script-vif "${dev}: removing ${HOTPLUG_STATUS}"
        xenstore-rm "${HOTPLUG_STATUS}"

5) Workaround XAPI conflicts with XEND
$ sudo sed -i -e 's/xend_start$/#xend_start/' -e 's/xend_stop$/#xend_stop/' /etc/init.d/xend
$ sudo update-rc.d xendomains disable

6) Workaround qemu keymap location preventing vncterm from starting
$ sudo mkdir /usr/share/qemu
$ sudo ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps

7) Make xen the default grub entry.

$ sed -i 's/GRUB_DEFAULT=.\+/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub
$ update-grub

Create local storage repository
If you want to create a VM before create a SR, you meet the following error: 
Error: No SR specified and Pool default SR is null


So, I make a partition to map storage repository 

$ pvcreate /dev/sda3

Physical volume "/dev/sda3" successfully created

$ pvdisplay

----- Physical volmue -------

 PV Name               /dev/sda2

 VG Name               VolGroup00

 PV Size                 11.72 GB / not usable 1.68 MB

 Allocatable            yes (but full)

 PE Size (KByte)      32768

 Total PE                375

 Free PE                 0

 Allocated PE          375

 PV UUID                AN--------------

----- Physical volmue -------

 PV Name               /dev/sda3

 VG Name               VolGroup00

 PV Size                 8.17 GB / not usable 17.80 MB

 Allocatable            yes (but full)

 PE Size (KByte)      32768

 Total PE                261

 Free PE                 261

 Allocated PE          0

 PV UUID                Tj--------------

$ xe sr-create type=lvm content-type=user device-config:device=/dev/disk/by-id/scsi-SATA_ST3320620AS_5QF7QZZL name-label=”local storage”
After run this command, an error was occurred.
The SR could not be connected because the driver was not recognised.
driver: lvm
Set EXT if you have an unused block device. I got an idea of how to solve from here. 


Tried again after changed type from lvm to ext 
$ xe sr-create type=ext content-type=user device-config:device=/dev/disk/by-id/scsi-SATA_ST3320620AS_5QF7QZZL name-label=”local storage”


# Create a vm
$ net_uuid=$(xe network-list bridge=xenbr0 --minimal)
$ vm=$(xe vm-install new-name-label="centos-test" template="CentOS 5 (64-bit)" sr-name-label="local stroage")

$ xe vif-create vm-uuid=$vm network-uuid=$net_uuid mac=random device=0
$ xe vm-param-set uuid=$vm other-config:install-repository=http://ftp.daum.net/centos/5/os/x86_64
$ xe vm-param-set uuid=$vm other-config:disable_pv_vnc=1
$ xe vm-start uuid=$vm


# Connecting to xenconsole
$ dom_id=$(xe vm-list uuid=$vm params=dom-id --minimal)
$ /usr/lib/xen-4.1/bin/xenconsole ${dom_id}


Type ctrl + ] if you want to close xencosole


References: 
1. XCP toolstack on a Debian: http://wiki.xen.org/wiki/XCP_toolstack_on_a_Debian-based_distribution 
2. Hypervisor support matrix: http://wiki.openstack.org/HypervisorSupportMatrix 
3. CentOS 6 VM (64 bit) automated installation on XCP: http://grantmcwilliams.com/item/563-centos6-on-xcp