building virtual appliances on ubuntu

Posted on Nov 14, 2011

building virtual appliances on ubuntu

Ubuntu to me is still the best OS to use for virtual appliances. It may not be for the reasons that you’re thinking about so here’s my short list.

it’s wildly successful

to me this means lots of updates and support for packages both from the vendor (canonical) as well as the community.

support available from ONE install

What other distribution has one install for the community AND commercial support? Yea. I can’t think of one either. This allows users to choose how much support they want/need from one virtual appliance/installation.

small minimal virtual machine footprint

The ovf will fit neatly on a 512M usb drive when complete.

why another virtual appliance tutorial?

I know that building virtual appliances is old hat for many of you, but I keep getting the question about how to build virtual appliances for distribution. In a recent scenario at my work, they were looking for a splunk virtual appliance that members of the consulting group could quickly leave on site. I thought this would be a good walkthrough. Let me know if you have any questions.

let’s get started!

First things first we need to download a copy of ubuntu server. Once you’ve got your iso downloaded, go ahead and build out the base VM that you’re going to use. For these types of deployments, I go ahead and configure the appliance inside a vApp as we are starting to deploy more of these on vCloud deployments.

build your vApp and VM

In vSphere, right click on your cluster or host and select New vApp.
Name your vApp, select next.
If you need/want to isolate resources specifically for this vApp you can do so on the Resource Allocation screen. I’ll go ahead and reserve 1G of Memory cause I’m greedy like that. Select next.
Select Finish and find your vApp in the VM list.
Now we need to add the VM to the vApp.
Right click on the vApp and select New Virtual Machine.
Name your virtual machine and select Next (this demo will be for spunk so I’ll use the ever so cunning name splunk. Select Next.
select which datastore you want this VM to temporarily live, select next.
select the Guest Operating System for this VM, in this case, select the Linux radio button, and select Ubuntu Linux (64-bit) in the Version dropdown window. Select Next. (you did download the 64bit version right?!)
Set the size of your virtual appliance, I’ll crank this down to 10G, as the splunk data directories will be remote NFS mounted in most cases. I’m a huge fan of thin provisioning. check it if it’s appropriate for your workload. select Next.
Select Finish.

install ubuntu

mount the iso to your vm, and start ‘er up. Once you’ve selected your language, on the initial install screen…
select f4
select Install a minimal virtual machine [1]
select enter (this takes you back to the main screen)
select Install Ubuntu Server (should be checked by default) Select Next
select your language again (don’t ask) Hit Enter
select your location. hit Enter
select No for keyboard layout
select your Keyboard language (English (US) is default)
select your Keyboard layout (English (US) is default)
The system will load additional components as needed
Network configuration. I SERIOUSLY recommend DHCP for just about everything. It’s 2011 folks, DNS and DHCP works.
give your appliance a hostname (splunkappliance in my case) Select next.
it’ll start up the clock and check your time zone. change it if needed or select Yes.
For partitioning, I select Guided – use entire disk and set up LVM to make it easier to expand the install as needed down the road if required. Select this and hit enter
default vm will have one disk listed, select it and hit enter.
Select Yes to write the changes to disk, and hit enter
If using LVM you’ll be prompted with the maximum amount of space in the volume group. Select Continue and hit enter.
Select Yes to write the changes to disk and hit enter
Enter your default full username, select continue.
Enter your default user name, select continue.
Enter your default password, select continue.
You’ll be prompted with “How do you want to manage upgrades on this system?” I select install security updates automatically so I don’t have to worry about being blamed for something that is WAY outside my control 6 months from now. It’ll still happen, but I’ll sleep easier. Select which of the first two you want, and hit enter.
Choose software to install. The only thing I select here is OpenSSH server. that’s it. select it and select continue.
This will start the package install, and it should go relatively fast
select Yes to install the GRUB boot loader to the master boot record.
Select Continue to finish your installation
unmount the iso from vSphere client
The above will install a really base system for your virtual appliance. I go ahead and install openssh-server on the vm so that it doesn’t need unproxied internet access during it’s first boot to get it installed. As I’m regenerating the ssh keys anyway, I don’t worry about it too much. This VM should boot REALLY really fast.

/etc/issue

/etc/issue.net is the file that handles what is displayed before a user logs in. like this:

that’s pretty boring right? what we want for our appliance is to have something awesome like this:

To get this we’ll need to do a couple of things. we’ll want to create a new script that is run at bootup every time. This will generate the /etc/issue.net for us after we get our ip address. Here’s the contents of a generic one I used for this demo. Adjust as needed.

#! /bin/bash
#
# issue-reset set ipaddr in /etc/issue
#
# chkconfig: 2345 90 60
# description: this will echo the ip address to /etc/issue for display
#
# Source function library.
#. /etc/init.d/functions
RETVAL=0
#
# See how we were called.
#
prog=”issue-reset”
start() {
#
#change /etc/issue to display ip address
#
echo “AWESOMETACULAR Splunk Appliance” > /etc/issue
echo “Powered by Ubuntu Linux and caffeine” >> /etc/issue
echo ” ” >> /etc/issue
netcount=$(/sbin/ifconfig | grep -o “inet”)
if [[ $netcount > 1 ]]
then
echo “To manage this virtual appliance please use a web ” >> /etc/issue
echo “browser on a different system and navigate to ” >> /etc/issue
echo ” ” >> /etc/issue
/sbin/ifconfig | grep “inet addr” | grep -v “127.0.0.1″ | awk ‘{ print $2 }’ | awk -F: ‘{ print “http://”$2″:8000″ }’>> /etc/issue
echo ” ” >> /etc/issue
else
echo “This appliance does not have networking configured. Please log in to configure networking” >> /etc/issue
fi
echo ” ” >> /etc/issue
cp /etc/issue /etc/issue.net
# RETVAL=$?
echo
[ $RETVAL -eq 0 ]
return $RETVAL
}
case “$1″ in
start)
start
;;
*)
echo $”Usage: $0 {start}”
exit 1
esac

exit $?

Once you’ve downloaded this, we’re going to move it to the /etc/init.d directory and rename it banner:
mv ./banner.txt /etc/init.d/banner
change the owner to root, and make it executable.
sudo chmod +x /etc/init.d/banner && sudo chown root.root /etc/init.d banner
add /etc/init.d/banner to be run at startup:
sudo update-rc.d banner defaults

that’s it! You’ll want to edit that file to make it reflect what you want it to say, but it’s pretty self explanatory.

firstboot startup script

There are a couple of ways to handle first boot only startup scripts. The easiest way I’ve found is to edit the rc.local file to run a script, and when complete, clean up after itself. Here’s what my firstboot script looks like:

#This script is run the first time a user logs in
echo “Your Splunk appliance is almost finished being deployed”

INSTALL=”apt-get install -y”
REMOVE=”apt-get remove -y”
SPLUNK_HOME=”/opt/splunk”
apt-get update

# email
$INSTALL exim4-daemon-heavy

# expire the user account passwd
passwd -e defaultuser

# new ssh key generation
rm /etc/ssh/ssh_host*key*
dpkg-reconfigure -f noninteractive -p critical openssh-server

# splunk
dpkg -i /root/splunk-4.2.4-110225-linux-2.6-amd64.deb

#start splunk server for the first time
$SPLUNK_HOME/bin/splunk start –accept-license

#configure splunk to autostart at boot
$SPLUNK_HOME/bin/splunk enable boot-start

# clean up
sed -i ‘s_/root/firstboot.sh_exit 0_’ /etc/rc.local

echo “firstboot complete”

You’ll want create a file with that as the content, save it, and put it somewhere where it can be run during boot. For ease of this demo, we’re going to put it in our home folder. Don’t Judge, this is just a demo.

sudo mv ./firstboot.txt /root/firstboot.sh && sudo chmod +x /root/firstboot.sh && sudo chown root.root /root/firstboot.sh

Now we’re going to setup this script to be run during first boot. using whatever editor you want edit /etc/rc.local and remove the exit 0 line and substitute it with the following line:

/root/firstboot.sh

Let’s walk through firstboot.sh and take a look at what we’re doing. In our example, we’re creating a splunk virtual appliance, so from the top:
we’re updating our repositories
we’re installing an MTA for splunk to use.
we’re expiring the password for our default user account. Change this to reflect whatever username you selected.
we’re deleting and regenerating ssh keys
we’re installing splunk. For this example, you’ll need to get the splunk .deb package downloaded and put somewhere accessible. It’s free, go grab it now.
Now we’re going to do some startup specific commands for splunk, first we’re going to set splunk to start for the first time and accept the license.
Next we’re going to go ahead and configure splunk to start every time the server starts.
after that’s complete, we’re going to clean up /etc/rc.local to not run this script again.That’s it for our startup script. pretty straightforward.

Test your appliance
power off the VM and create a snapshot.
Power the vm on and test the startup, the install of our splunk package, and the firstboot configuration steps.
If everything works, go ahead and roll back our changes and create your ovf below
if it doesn’t power back on and dig in

add some info to your appliance

If you’re planning on posting updates to this, or you want to provide the default username and password information directly in the ovf file, you’ll want to edit some of the details of your vApp.
Right-click on your vApp and select Edit Settings.
On the options tab, select Advanced.
Enter your appliance name, website, and any other information that you want to provide.
export to ovf
From the vSphere client, select the vApp, and in the File menu select Export, Export OVF Template.
pick a directory where you can store your appliance
export!
test appliance

one last test of your appliance and you’re ready for deployments. This last check for me usually provides a sanity check for the finished appliance. to deploy from vSphere client just:
select file
deploy OVF template
browse to the directory and select the OVF file
the only thing you should have to do other than picking a datastore, is selecting a network
Go get a cup of coffee while the deployment occurs
done!
conclusion

This walkthrough for a splunk appliance is pretty simple but if done correctly virtual appliances deployments can shave hours off of simple traditional application installations for your team. Getting more advanced versions of this, you could get fancy and add additional layers like puppet autoconfigs to further automate package deployments. Let me know what you think and if this worked for you!

[1] this is what used to be the ubuntu JeOS install.