2013년 4월 27일 토요일

Xen is moving foward

Although Xen has not been included as default hypervisor in RHEL and has been losing some supporters. Xen is still attractive hypervisor. Xen is trying to support platforms in many ways. 

There some highlighted projects: 
  • Mirage OS: a library OS that supports the most common web protocols and enables the development of sealed Xen Project appliances that can be run without a guest operating system on any Xen Project based cloud - led by Cambridge University
  • Support for ARMv7 and ARMv8 based servers - led by Citrix
  • Improvements to Nested Virtualization: Virtualzation of guest machines - led by Intel
  • Significant extensions to the Xen Project Security Modules and Flask - led by the National Security Agency
  • A new virtualization mode called PVH (PV on HVM: improving performance by support PV, remove emulated hardware pocessing), which promises to combine the best aspects of hardware virtualization and paravirtualization - led by Oracle
  • Significant performance and scalability improvements - led by SUSE

References:
2. Xen Proejct home: http://www.xenproject.org/

2013년 4월 25일 목요일

Openstack: How to solve pkg_resources.DistributionNotFound when executing glance


I installed and configured glance to try openstack grizzly. I followed the instruction from Grizzly install manual.
When I tried to create an image, I encountered the error and this was the error: 

$ sudo glance image-create --name="tty-linux-kernel" --disk-format=aki --container-format=aki < ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
Traceback (most recent call last):
  File "/usr/bin/glance", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2711, in <module>
    parse_requirements(__requires__), Environment()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 584, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: requests>=0.8.8

Glance has dependencies on other modules, for example, python-swiftclient, python-keystoneclient and so on.
Even, python-swiftclient which installed here has dependencies on its own.
In order to get a clue to solve the error, I printed the command "glance"

$ cat /usr/bin/glance
#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'python-glanceclient==0.9.0','console_scripts','glance'
__requires__ = 'python-glanceclient==0.9.0'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
    sys.exit(
        load_entry_point('python-glanceclient==0.9.0', 'console_scripts', 'glance')()
    )

It seemed that it was something wrong with modules related to "python-glanceclient". Next, I looked at /usr/lib/python2.7/dist-packages/python_glanceclient-0.9.0.egg-info/requires.txt file.
requires.txt fie is used to write a list of packages to install.

$ vi /usr/lib/python2.7/dist-packages/python_glanceclient-0.9.0.egg-info/requires.txt
prettytable&gh;=0.6,<0.8
python-keystoneclient>=0.2.3
pyOpenSSL
warlock>=0.7.0,<2

But, there was NO module named "requests" in here, so I tried to do other files which were indicated here. I opened the next file, requires.txt for keystoneclient.

$ vi /usr/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.egg-info/requires.txt
iso8601>=0.1.4
prettytable
requests>=0.8.8
simplejson
oslo.config>=1.1.0

I finally found "request" in here and I checked the version of my request module to compare the version number.

$ sudo dpkg -l | grep request
ii  python-requests                      0.8.2-1                                  elegant and simple HTTP library for Python, built for human beings

My python-requests module was 0.8.2. I changed it into:

$ vi /usr/lib/python2.7/dist-packages/python_keystoneclient-0.2.3.egg-info/requires.txt
iso8601>=0.1.4
prettytable
requests>=0.8.2
simplejson
oslo.config>=1.1.0

I was able to solve my problem by doing this. However, I got another problem with this.

$ sudo glance image-create --name="tty-linux-kernel" --disk-format=aki --container-format=aki --file=ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
Authorization Failed: request() got an unexpected keyword argument 'verify'

This was caused because I had old-dated "python-requests", I needed to install 0.8.8 or higher to work "verify". I found how to solve from the site: https://answers.launchpad.net/keystone/+question/218143

"this duo to you OS has a low version of python-requests by defalut,now python-keystoneclient require a python-requests>0.8.8,<1.0 this bug has been fixed to master. you can first git pull the lastest python-keystoneclient code. then run pip install -r tools/pip-requires."
I downloaded new version of it from https://pypi.python.org/pypi/requests/1.2.0
$ sudo tar -xzvf ./requests-1.2.0.tar.gz 
$ cd requests-1.2.0
$ sudo ./setup.py build

Finally it worked.

References: 
1. http://pedrojimenez.github.io/blog/2012/12/12/ubuntu-python-keystoneclient-dependency-error/
2. https://answers.launchpad.net/keystone/+question/218143
3. http://www.pip-installer.org/en/latest/requirements.html
   

2013년 3월 6일 수요일

VMware: Lable of VirtualDevice only accept alphabetical characters, If not, got InvalidRequest error


vSphere API threw InvalidRequest error while I was trying to add a networkadapter of a VM.

com.vmware.vim25.InvalidRequest
at com.vmware.vim25.mo.VirtualMachine.reconfigVM_Task...

My code was following:
...
...
// Network Connection name
String[] network_name =  {"VM Network"};

// Get the existing network devices of the VM
List<VirtualDevice> listNetworkDevice = new ArrayList<VirtualDevice>();

for (VirtualDevice vDevice: vm.getConfig().getHardware().getDevice()) {
  if (vDevice.getDeviceInfo().getLabel().toLowerCase().indexOf("network") >= 0 || vDevice.getDeviceInfo().getLabel().indexOf("네트워크") >= 0)
    listNetworkDevice.add(vDevice);

}

// Get the label of network device 
String listNicLabel[] = listNetworkDevice.get(listNetworkDevice.size() - 1).getDeviceInfo().getLabel().split(" ");

int seqOfNIC = 0;
try {
  if( listNicLabel != null && listNicLabel.length >= 2)
    seqOfNIC = Integer.parseInt(listNicLabel[2]);                                                   
} catch (Exception e) {
    listNicLabel = new String[] { "Network", "Adapter" };
}

// Unique key that distinguish the device from other devices in the same VM. Here, it is just a temporary value
int nicKeyVal = 0;

for (int i = 0; i < customServerInfo.getNetworkBridgeName().length; i++) {                                 
             
  VirtualEthernetCard nic = new  VirtualVmxnet3();
  VirtualEthernetCardNetworkBackingInfo nicBacking = new VirtualEthernetCardNetworkBackingInfo();
  nicBacking.setDeviceName(customServerInfo.getNetworkBridgeName()[i]); // Network Connection
                
  String nicLabel = listNicLabel[0] + " " + listNicLabel[1] + " " + (++seqOfNIC);   
  
  Description info = new Description();
  info.setLabel(nicLabel);
  info.setSummary(customServerInfo.getNetworkBridgeName()[i]); 
  nic.setDeviceInfo(info);
                                  
  // type: "generated", "manual", "assigned" by VC
  nic.setAddressType("generated");
  nic.setBacking(nicBacking);
  nic.setKey(-1 * (4000 + (nicKeyVal++)));      
    
  VirtualDeviceConfigSpec deviceConfigSpec = new VirtualDeviceConfigSpec();
  deviceConfigSpec.setOperation(VirtualDeviceConfigSpecOperation.add);
  deviceConfigSpec.setDevice(nic);
    
  listDeviceConfigSpec.add(deviceConfigSpec);
            
}

if (listDeviceConfigSpec.size() > 0) {
  vmSpec.setDeviceChange(listDeviceConfigSpec.toArray(new VirtualDeviceConfigSpec[listDeviceConfigSpec.size()]));

  Task task = vm.reconfigVM_Task(vmSpec);
  ...
  ...
}

In my above coding, I got the last label from the existing NetworkAdapter in line 21. (My VM already had two networks) and replaced the last sequence number with new one in line 35.
e.g.)
<<The last seq and name>>    <<The new seq and name>>
Network Adapter 2        =>      Network Adapter 3

However, Labels of vDevices were displayed in the language which you chose one when you installed vCenter. I installed it with Korean language, so I actually got the result of labels, for example, 네트워크 어댑터 1, 네트워크 어댑터 2 (They both mean "Network Adapter").

In Label property of Description class in vSphere API, it doesn't accept characters other than Alphabetical(and number).
So, You should change it with alphabetical characters for Label property. I changed this like:
String nicLabel = "Network Adapter" + " " + (++seqOfNIC);

Description info = new Description();
info.setLabel(nicLabel);
...
When you get InvalidRequest error for VM Creation / Reconfig job and you try to debug it, it can be one of good solutions that you comment all of setter methods except for mandatory and then uncomment one setter method per testing, after that you can find where it is wrong in your code. It seems not to have easy solution about solvoing InvalidRequest error.


References:
1. http://www.jarvana.com/jarvana/view/org/jvnet/hudson/vijava/2120100824/vijava-2120100824-sources.jar!/com/vmware/vim25/mo/samples/vm/CreateVM.java

2013년 2월 26일 화요일

Book review: Nagios Core Administration Cookbook

I've involved in a cloud development project. I use Nagios core for monitoring servers and devices in this project. As we know, Nagios is one of famous open sourced monitoring tools. I needed references in dealing with Nagios. While I was reading this, it remind me of basic concept of Nagios again. 



There are some sections in every chapter. "How to do it" , "How it works" and "There's more". it describes how we start a certain feature of Nagios according to the section in "How to do it" and then explains its mechanism with easy way
in "How it works" and finally there is additional contents related to the chapter in "There's more"

There is impressive paragraph in the book that nagios is close to MONITORING FRAMWORK rather than monitoring tool. It is perfect that describes nagios characteristic because it has already provide ecosystem, for example, lots of plugins support . And it is important to know where to find if need specific plugin that doesn't meet requirements. Author instructed that how to find a plugin that the administrator needs and where. 

People who are unfamiliar with Nagios, in my opinion, they may be confused that Nagios has SOFT and HARD state in response to server or host failing. In chapter 3, author tries to explain the difference between two by giving an understandable example. 

At last, in chapter 11, he gives an explanation about add-on projects like NRPE, NSClient++, NDOUtils and SNMP trap. If you need to extend the feature of Nagios to provide better-fitting solution on your own production, this will very helpful to you.

Book link: http://www.packtpub.com/nagios-core-administration-cookbook/book

2013년 2월 24일 일요일

Nagios NDO2DB: Unicode problem

I had encountered a problem related NDO2DB. I configured some of services' name  in Korean character. It shows well in Nagios web, but these services' name could't be recognized in Mysql tables which Ndo2db saves into becuase Ndo2db  add-on processes data without considering uni-code data. It was not solved even though I changed character set to UTF-8 in the tables.     

One of possible way to deal with is to change convert column which have to show by UTF8 in SQL query. 

The solutions is as following: 
CONVERT(CAST(column_name) as BINARY) USING UTF8) 
"This is a similar approach to our SELECT CONVERT(CAST(city as BINARY) USING utf8) trick above, where we basically hide the column’s actual data from MySQL by masking it as BINARY temporarily. 
If you simply force the column to UTF-8 without the BINARY conversion, MySQL does a data-changing conversion of your “latin1″ characters into “UTF-8″ and you end up with improperly converted data. Converting the column to BINARY first forces MySQL to not realize the data was in UTF-8 in the first place."
I referenced this from this site: http://nicj.net/mysql-converting-an-incorrect-latin1-column-to-utf8/


2013년 1월 24일 목요일

Nagios: How to change datetime format of snmp trap

After I installed and Configured SNMPTT, saving snmpt trap data into table worked well.
Additionally, I needed to change the date format so that I could filter them easily by date and time.

By default, date format should be "Sun Dec 16 15:23:47 2012", but I want it to look like "2012-12-16 15:23:47"

When look at /etc/snmp/snmptt.ini, there are some properties related to datetime format.
$ vi /etc/snmp/snmptt.ini
...
# The date time format in strftime() format for the date/time when logging to 
# standard output, snmtt log files (log_file) and the unknown log file
# (unknown_trap_log_file). Defaults to localtime(). For SQL, see
# date_time_format_sql.
# Example:  %a %b %e %Y %H:%M:%S
# date_time_format = 
...
...
# The date time format for the traptime column in SQL, Defaults to 
# localtime(). When a date/time field is used in SQL, this should be 
# changed to follow a standard that is supported by the SQL server.
# Example:  For a MySQL DATETIME, use %Y-%m-%d %H:%M:%S.
# date_time_format_sql = 
...

As we can know from the above description, snmptt uses strftime() format.
The first format definition (date_time_format) is for log files. Generally, /var/log/snmptt/snmptt.log and /var/log/snmptt/snmpttunknown.log. The second definition (date_time_format_sql) is for traptime column of snmptt table in MySQL.

Before change the format in snmptt.ini, you can validate the format like the following:
$ mydate=`perl -e 'use POSIX; print strftime "%Y-%m-%d %T", localtime time;'`
$ echo $mydate
2013-01-22 10:19:45

Then uncomment those lines and change the formats in snmptt.ini
$ vi /etc/snmp/snmptt.ini
...
date_time_format = %Y-%m-%d %T
...
date_time_format_sql = %Y-%m-%d %T

# It needs to restart
$ service snmptt restart

Please visit the site to see various date time formats in strftime
http://my.safaribooksonline.com/book/operating-systems-and-server-administration/bash/0596526784/reference-lists/date_and_time_string_formatting_with_strftime

References:
1. http://stackoverflow.com/questions/13093709/how-to-use-shell-variables-in-perl-command-call-in-a-bash-shell-script

2013년 1월 17일 목요일

VMware vCenter Error: Database connection has failed.


I got an error "Database connection has failed" during vCenter 5.1 SSO. 

The error message was: 


VMware vCenter Sing Sign On service keeps data into a relational database. I installed MS SQL Server 2008 R2 express, which is provided as bundle database of vCenter. (It additionally needs to install SQL Server Management Studio GUI console)

After I was checking %temp%\vm-sso.javaLib.log, logs were follows: 

[2013-01-15 19:17:06,599] INFO ... - IP stack is fine
[2013-01-15 19:17:06,599] INFO ... - IP Stack works
[2013-01-15 19:18:46,392] INFO ... - Validate database connection
[2013-01-15 19:18:46,392] INFO ... - DB information jdbcUrl:jdbc:sqlserver://;serverName=vcenter;port=1433;databaseName=RSA type: Mssql
[2013-01-15 19:18:46,392] INFO ... - Initialize values of user DBA user RSA_DBA , other user RSA_USER
[2013-01-15 19:18:46,392] INFO ... - DB information jdbcUrl:jdbc:sqlserver://;serverName=vcenter;port=1433;databaseName=RSA type: Mssql userName:RSA_DBA
[2013-01-15 19:19:16,621]ERROR .... - Failed to established connection :com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host vcenter, port 1433 has failed. Error: "Connection refused: connect. Verify the connection properties, check that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port, and that no firewall is blocking TCP connections to the port."
[2013-01-15 19:19:16,621] INFO .... - could not close JDBC connection

The resolution was simpler than I expended. I found the answer from this site: http://vhomelab.com/2012/09/21/vmware-single-signon-service-inventory-service-vcenter-5-1-installation/
This post of the site has been well written about every step of installation of vCenter 5.1.  

Go to the manu "Microsoft SQL Server 2008 R2 > Configuration tools > SQL Configuration Manager" and should configure to set TCP Port property to 1433


Before I found the site which gave me an resolution, I re-installed SQL server , and then created DB and users manually, however you won't need to do these useless tasks if you already set tcp port. 

Related Posts Plugin for WordPress, Blogger...