Thursday, January 29, 2009

Building Enterprise OpenSim Grid on 2nd Birthday



Building enterprise quality virtual environment services out of OpenSim is a complex operation but today you can do it all with open source software. OpenSim still enjoys alpha status and many of the core components are under active development on daily basis. Following the svn head and figuring out stable revisions to baseline your system to is challenging task and not suitable for faint hearted.


First things first: I named my grid "Bubble Cloud" according to the website where we have been drafting out next generation virtual reality protocol MXP with Ben Lindquist and Ryan McDougall :


http://www.bubblecloud.org/


Building


I went for the following architecture:




I reserved one virtual machine (Psi) for shared grid services called UGAIM according to User, Grid, Asset, Inventory and Messaging. Another virtual server went to monitoring use (Omega). Last but not least I configured 2 virtual machines for simulators which run regions (Alpha and Beta).


All virtual machines have 2 Giga bytes memory, 1 Xeon Core and 20 Giga bytes of raid disk.


OS selection was a tough one as .NET is rumored to be very good choice for running OpenSim servers. In the end I went for open source and thus latest stable Ubuntu was the clear choice as it is most used OS for running OpenSim servers.


Database was no brainer as MySQL is the most used database in OpenSim Ubuntu environments. I installed local MySQL instances to all virtual machines to enable fast database access and to save in resource costs.


Mono version was another hard question but I thought to go the easy way and use the standard mono version provided by the Ubuntu release (1.9.1) all though version 2 has many optimizations available. It might be that I need to upgrade soon to newer mono.


For easy administration I was happy to acquire linux init scripts from Sean Dague's Linux Service Helpers project for both ugaim and simulator:


http://forge.opensimulator.org/gf/project/linuxservice/


By copying these scripts to /etc/init.d and linking them to run levels you can have your OpenSim services start and stop automatically with your server.


Monitoring


To be able to measure the resource utilization, receive alerts, diagnose problems and track service availability I chose Zabbix server to be installed to Omega virtual machine and agents to other servers. Zabbix is excellent open source monitoring solution:


http://www.zabbix.com/


Configuring basic zabbix Linux monitoring is easy and can be done in two hours. Monitoring OpenSim servers and regions took a bit more work. I did this with Web Monitoring feature by hooking separate scenarios, triggers and IT services to each component. As a result everyone can follow availability figures of Bubble Cloud by bookmarking this url:


http://monitor.bubblecloud.org/srv_status.php




It is also possible to follow things like processor loads, memory usage, network traffic and disk space:





Cost


Running Enterprise OpenSim Grid is not free but you do not need mega corporation to finance it. One needs to pay for Virtual Machine rents or alternatively hardware costs and server hotel rent.


Renting these virtual machines will cost anything between 100 - 500 us dollars per virtual machine depending where you acquire them and if you pay by use or constant rate.


If you decide to build a server yourself (2 x Quad Core Xeon, 32G memory, raid 1-0 disks...) and take it to good server hotel with 100 Mbps or better internet connections you have to prepare for 5000 us dollar up front cost and 100 hundred dollar monthly cost. With this setup you can get 8 heavy duty virtual machines with 1 Xeon Core and 4G memory. If you divide the cost for period of 3 years you get approximately 30 dollars monthly cost which is not bad at all. The downside is the up front investment, knowledge requirement and time needed.


If you have the knowledge you can build the environment yourself but if you need to hire someone the time required for building this kind of setup is at minimum 1 week for the hardware and 1 week for the software.


Thanks


I want to thank all the nice people in freenode irc channels #opensim, #osgrid and #zabbix for their invaluable effort put in building this software and the help they gave to me in my learning process.

3 comments:

Anonymous said...

How about using Amazon Elastic Computing Cloud(EC2) infrastructure?
Rightnow AMIs for standalone opensim are available. If there is enough interest, cooking up an AMI for UGAIM is possible.
Check out the standalone opensim AMI at
http://simondemand.ec29.com/index.html
It has a GUI based launcher, so that the EC2 API details are sugar coated for the Sim admin.

Tommi S. E. Laukkanen said...

EC2 is very interesting concept and probably cost effective too if you do not have many users. I am concerned about the cost as OpenSim is both memory and CPU heavy application. When you have several users in your sim in the same 24/7 you may end up paying the full value of EC2 virtual machine.

Anonymous said...

Can anyone recommend the robust Managed Service tool for a small IT service company like mine? Does anyone use Kaseya.com or GFI.com? How do they compare to these guys I found recently: N-able N-central remote pc access
? What is your best take in cost vs performance among those three? I need a good advice please... Thanks in advance!