This is a read-only archived version of wiki.centos.org

( This is just a copy/paste from the centos-devel mailing list post, to be used as a starting point. [KB] )

http://www.karan.org/blog/index.php/2010/03/30/autmated-install-testing-for-centos has something to start with, it is very basic but it's something. There have been a few posts about that on the mailing list as well. But I'll give you enough with which to get started.

The tests themselves can be of two types :

a) installer based ( where one would write a kickstart that sets up the machine and does its tests or sets up the tests in %post ). Kickstarts are well, kickstarts. Look here for some examples if you need to : https://nazar.karan.org/cgit/bluecain/tree/

b) post-install scripts. These are the ones where one would do functional tests. Each of these tests is a single script which must - at this time - be self contained. And the only real concern for the testing system is what the exit code was ( 0 = pass, 1 = fail ). They can be written in any language you like. Here is a very simple example:

echo -n 'Test that all updates can be applied to this machine cleanly'
yum -d0 -y upgrade > /dev/null 2>&1
if [ $? -eq 0 ]; then
  echo ': PASS'
else
  echo ': Fail'
  exit 1
fi

here is another which does something similar :

echo -n 'Test that all 32 rpms can be removed'
# only run this test on x86_64 machines!
is64=$(uname -m|grep x86_64)
if [ "$?" -ne '0' ]; then
  echo ' Skip'
  exit 1
fi
yum -d0 -y erase *.i?86 > /dev/null 2>&1
if [ $? -eq 0 ]; then
  echo ' PASS'
else
  echo ' Fail'
  exit 1
fi

# note, this does not imply the machine is still usable after
# the remove! need to test that independently

Things that you *do* need to be concerned about :

- kickstarts should make no assumptions about storage ( since we run them on xen and kvm hosts ). If you need specific names and labels, use lvm.

- Getting the tests into the machine post install is not something you need to worry about, thats done by the test harness.

- Write a test for specific functionality to make sure that it works

- Also consider writing tests for things that should fail, eg. 1 must never be equal to 2. or with iptables blocking :22 there should be no way to connect to that port etc.

- ideally, tests will be in filenames that reflect the package or the nature of the test they are hoping to work against. eg: yum_CanBlindUpgrade.sh and yum_CanRemove32bit.sh

- if you are not writing a kickstart to cover the scope of tests, then you need to assume packages being tested are not installed. eg. php tests should start with a 'yum -d0 install php'

- test real functionality first, then work your way down if you want to. Eg:

echo -n 'Test to see if dns works'
echo 'nameserver 8.8.8.8'  > /etc/resolv.conf
# its important we dont hit a dns record with a wildcard like centos.org
ping -c 1 www.google.com > /dev/null 2>&1
if [ $? -eq 0 ]; then
  echo ' PASS'
else
  echo ' Fail'
  exit 1
fi

# implied results:
# - network works
# - default route is really honoured
# - atleast one network link on the machine is working
# - ipv4 is functional

So in this case, its testing if DNS is functional ( something a user would care about ) and you get some free test wins ( network works, route etc ).

Questions? Then ask away. Maybe a few people can get together and even start working on a set of wiki pages for this stuff!

The tests are all inside a git repo, but for the first few that you write - how about posting them here to the list and we can take it from there.