2012년 3월 26일 월요일

Openstack: 1. XCP install & configuration

To test openstack compute (Nova) with xen based machine, it needs XCP (Xen Cloud  Planform) or Xenserver. Openstack compute supports KVM, XenServer/XCP and ESXi hypervisor. I chose XCP because it needs no license. 


XCP is open source version of XenServer which has the same toolstack with XerServer. As a Cronos project, it is able to use XCP on Ubuntu 12.04 called Precise. (For Fedora and Centos users, they should wait until May. XCP team is porting xen on Fedora and CentOS, It will be finished by May)


I prepared the following servers for the test.
- 1 Virtual Machine: Openstack compute, installed Ubuntu.
- 1 Physical Machine : Openstack Controller, Ubuntu 12.04 beta (XCP 1.5 beta)




When creating virtual machine on XCP, it has to create a storage repository first. XCP uses SR (Storage Repository) to store virtual machine images, ISO files and templates. SR supports for IDE, SATA, SCSI and SAS drives locally connected, and iSCSI, NFS, SAS and Fibre Channel remotely connected.  


I have known storage repository concept since Oracle VM. I used NFS as a shared resository. NFS also can be used in XCP. However, I only have one physical machine this time. There are two physical hard disks on the server. One is for operating system and the other is for storage of XCP.     


Set the environment of XCP 
As a result of Cronos project, it is able to install xcp using agt-get command on Ubuntu 12.04.

1) Set up software repositories
$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:ubuntu-xen-org/xcp-unstable

2) Workaround xcp-networkd missing file
$ mkdir /etc/xcp
$ echo "bridge" >> /etc/xcp/network.conf

3) Install xapi: This step installs XCP's xapi and all its dependencies, including the Xen hypervisor.
$ apt-get pciutils
$ apt-get update
$ apt-get install xcp-xapi
$ apt-get install xcp-xe

4) Workaround VM's not going to power-state: halted after shutdown
$ vi /usr/lib/xcp/scripts/vif

remove)    =>   remove|offline)
    if [ "${TYPE}" = "vif" ] ;then
        call_hook_script $DOMID "${ACTION}"
        # Unclear whether this is necessary, since netback also does it:
        logger -t script-vif "${dev}: removing ${HOTPLUG_STATUS}"
        xenstore-rm "${HOTPLUG_STATUS}"

5) Workaround XAPI conflicts with XEND
$ sudo sed -i -e 's/xend_start$/#xend_start/' -e 's/xend_stop$/#xend_stop/' /etc/init.d/xend
$ sudo update-rc.d xendomains disable

6) Workaround qemu keymap location preventing vncterm from starting
$ sudo mkdir /usr/share/qemu
$ sudo ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps

7) 
Make xen the default grub entry.
$ sed -i 's/GRUB_DEFAULT=.\+/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub
$ update-grub

Create local storage repository
If you want to create a VM before create a SR, you meet the following error: 
Error: No SR specified and Pool default SR is null


So, I make a partition to map storage repository 
$ pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created


$ pvdisplay
----- Physical volmue -------
 PV Name               /dev/sda2
 VG Name               VolGroup00
 PV Size                 11.72 GB / not usable 1.68 MB
 Allocatable            yes (but full)
 PE Size (KByte)      32768
 Total PE                375
 Free PE                 0
 Allocated PE          375
 PV UUID                AN--------------

----- Physical volmue -------
 PV Name               /dev/sda3
 VG Name               VolGroup00
 PV Size                 8.17 GB / not usable 17.80 MB
 Allocatable            yes (but full)
 PE Size (KByte)      32768
 Total PE                261
 Free PE                 261
 Allocated PE          0
 PV UUID                Tj--------------


$ xe sr-create type=lvm content-type=user device-config:device=/dev/disk/by-id/scsi-SATA_ST3320620AS_5QF7QZZL name-label=”local storage”
After run this command, an error was occurred.
The SR could not be connected because the driver was not recognised.
driver: lvm
Set EXT if you have an unused block device. I got an idea of how to solve from here


Tried again after changed type from lvm to ext 
$ xe sr-create type=ext content-type=user device-config:device=/dev/disk/by-id/scsi-SATA_ST3320620AS_5QF7QZZL name-label=”local storage”


# Create a vm
$ net_uuid=$(xe network-list bridge=xenbr0 --minimal)
$ vm=$(xe vm-install new-name-label="centos-test" template="CentOS 5 (64-bit)" sr-name-label="local stroage")

$ xe vif-create vm-uuid=$vm network-uuid=$net_uuid mac=random device=0
$ xe vm-param-set uuid=$vm other-config:install-repository=http://ftp.daum.net/centos/5/os/x86_64
$ xe vm-param-set uuid=$vm other-config:disable_pv_vnc=1
$ xe vm-start uuid=$vm


# Connecting to xenconsole
$ dom_id=$(xe vm-list uuid=$vm params=dom-id --minimal)
$ /usr/lib/xen-4.1/bin/xenconsole ${dom_id}


Type ctrl + ] if you want to close xencosole


References: 
1. XCP toolstack on a Debian: http://wiki.xen.org/wiki/XCP_toolstack_on_a_Debian-based_distribution 
2. Hypervisor support matrix: http://wiki.openstack.org/HypervisorSupportMatrix 
3. CentOS 6 VM (64 bit) automated installation on XCP: http://grantmcwilliams.com/item/563-centos6-on-xcp

댓글 19개:

  1. First time wen i starting vm > xe vm-start uuid=$vm

    I m getting :
    Cannot plug VIF
    VIF: 904b1ac9-04f5-7455-3ef2-cb95d1ba4a0f

    After sometime if i try to start again , i m getting ,

    Error code: Traceback (most recent call last):
    Error parameters: File "/usr/lib/xcp/lib/pygrub.xcp", line 811, in , fs = fsimage.open(file, part_offs[0], bootfsoptions), IOError: [Errno 95] Operation not supported,

    답글삭제
  2. When i'm shutting down a VM I can't start it after that.
    I always geht

    Traceback (most recent call last): - File "/usr/lib/xcp/lib/pygrub.xcp", line 811, in - fs = fsimage.open(file, part_offs[0], bootfsoptions) - IOError: [Errno 95] Operation not supported

    Anyone got this error too on Ubuntu 12.04?

    답글삭제
  3. Thanks for the comprehensive tutorial, it gave me many insights. I've also used the instructions located at:

    http://wiki.openstack.org/XenServer/Install/XcpXapiOnPrecise
    (because I'm using Ubuntu Server 12.04, not CentOS)

    Everything went fine. The hint to leave a partition was useful. Instead of taking an entire disk, I just spared a partition for later use with Xen (in my case /dev/sda3).

    답글삭제
  4. Hi Yeonki,

    I am Momo and I am doing my internship. So, my topic is to do an cloud Platform with XCP and Openstack. Your tutorial is very comprehensive and useful but everything don't be clear for me.

    In your first Picture, I don't understand why you put XCP in the Host 2 and XEN in the host number 1.
    Can you help me please. For information, i only have one physical server to do my test.

    Thanks

    답글삭제
  5. Hi, Momo.

    The number of server was optional. I just wanted to separate controller and compute at that time. You said that you have just one physical machine. So, you can configure the physical machine for compute, and make a virtual machine in the physical machine and then use it for controller.

    Regards,
    Yeonki.

    답글삭제
  6. thanks you very much for your answer Yeonki,

    but, i have other questions.
    So, For the moment, i begin my internship by installing XCP-XAP. I want to create a VM and understand how it's works. But, I can't create a local storage. When i do pvcreate /dev/sda3, i have this result:
    Device /dev/sda3 not found (or ignored by filtering).

    The result of fdisk -l /dev/sda is:
    /dev/sda1 * 2048 471853055 235925504 83 Linux
    /dev/sda2 471855102 488396799 8270849 5 Étendue
    /dev/sda5 471855104 488396799 8270848 82 partition d'échange Linux / Solaris

    So, i don't understand why???

    Thank you
    Regards
    Momo

    답글삭제
  7. 작성자가 댓글을 삭제했습니다.

    답글삭제
  8. Hi, Momo
    Sorry to reply late.
    Do you still have this problem?

    Regards,
    Yeonki.

    답글삭제
  9. Hi,
    I thank you to answer to my request.

    Yes. In fact, I understood that I must create a partition with free space name /dev/sda3.
    But, the result of fdisk -l /dev/sda is:
    The result of fdisk -l /dev/sda is:
    /dev/sda1 * 2048 499711 248832 83 Linux
    /dev/sda2 501758 488396799 243947521 5 extended
    /dev/sda5 501760 488396799 243947520 8e Linux LVM

    I don't know how i can create /dev/sda3 ??

    Thank you
    Regards
    Momo


    답글삭제
  10. Hi, Momo.

    Although I gave an example of creating partition /dev/sda3/, you only need a unused block device for making local storage (Any device name is okay). So, you don't have to have partition named /dev/sda3/. It is able to changeable based on your environment.

    Could you visit to this site: http://wiki.xen.org/wiki/Using_XCP_-_preparing_the_toolstack. I was given the answer when I was having the problem related to local storage.

    Regards,
    Yeonki

    답글삭제
  11. Hi,

    It's work, thank you.

    I just have a last problem. When I start my VM (xe vm-start uuid=5b50*********), it's running during 2mn, after it turns off (power-state (R0): halted)?? I don't understand why. I googled, but nothing.

    Regards
    Momo

    답글삭제
  12. Hi Yeonki,

    I managed to understand XCP. Thank you for all. Your tutorial was very useful for me.

    So, I only have one physical server. I install on this Server XCP (DOM0) and I create 3 VM. In the first VM, I installed Ubuntu 12.10 and I download devstack:
    git clone git://github.com/openstack-dev/devstack.git

    After that, I started the install with this command:
    cd devstack; ./stack.sh

    So, I can use the dashboard (horizon) with the IP adress of my VM.

    But, I have a problem. Indeed, I can't create a new instance. I do "launch instance", with the image "cirros-0.3.1-x86_64-uec". When I click "launch", and I see this new instance with:
    Status: "Build"
    Task: "Scheduling"
    Power state: "No state"
    And, it's never change, So I don't know why ???

    Can I control the other VM that I create with XCP command. How ??


    I also install Openstack in other VM. I want to know, if I can also control VMs with openstack. How??

    I googled, but I don't found response.

    Thank you.

    Regards
    Momo

    답글삭제
  13. The reason LVM as a SR type isn't working under Ubuntu 12.04 LTS can be found here: http://www.gossamer-threads.com/lists/xen/api/215973 You can check here for a solution: http://unix.stackexchange.com/questions/60227/xen-storage-repository-has-option-type-lvm-missing-xcp/62406#62406

    It's a matter of putting the right 'drivers' there for xcp-xapi

    답글삭제
  14. Thanks for the information.
    I'll check this.

    답글삭제
  15. Hi Yeonki
    Am new to openstack, need to install openstack in centos and add Xen as host. I have a CentOS VM and Xen Server separately. Can you please point me to tutorial for step by step installation and configuration.

    Thanks
    SriHarish

    답글삭제
  16. Hi, SriHarish

    I'd like to highly recommend that you should deploy Openstack on Ubuntu + KVM combination if you're new to it. Because there are useful references of over the internet for Ubuntu + KVM.
    Is there any possible for you to change the current environment?

    Regards,
    Yeonki

    답글삭제
  17. 블로그 관리자가 댓글을 삭제했습니다.

    답글삭제
  18. Great post thanks for sharing this.I was looking this kind of info.
    Openstack Training

    답글삭제