Software performance analysis packages such as PerfSuite need to be able to access hardware performance counters. Unfortunately, the standard linux kernels do not provide this functionality by default. These instructions describe how to install PerfSuite on a CMU SCS facilitized Redhat 9 machine. They are probably useful for any Redhat 9 setups (just ignore the afs installation steps). Update (5/30/2007) I just installed onto a Fedora Core 6 installation. There is no need to install a different version of gcc and tDom/tcllib can be installed using Add/Remove Programs. You will need to disable SELinux in the Security Level and Firewall control panel. The kernel source is at download.fedora.redhat.com/pub/fedora/linux/core/6/source/SRPMS After installing the RPM edit /usr/src/redhat/SPECS/kernel-2.6.spec to remove both references to the utrace patch10 (which removes the more basic ptrace support that perfctr uses) and the s390-kprobes patch500 which seems to depend on the utrace patch. After make mrproper, you need to edit the Makefile to remove the EXTRAFLAG -prep prior to applying the perfctr patch. You need to apply a small patch by hand to kernel/ptrace.c: replace the one call to may_attach with a call to __ptrace_may_attach. After applying the perfctr patch you need to manually remove all the includes of linux/config.h from $KERN/drivers/perfctr and remove the reference to i_blksize from virtual.c (these are both guarded by a check for a 2.6.19 kernel). Finally, you can just run make etc. These instructions are current as of 1/23/2007 1. Download the necessary software. PAPI (Performance Application Programming Interface) Includes source code for PerfCtr kernel module http://icl.cs.utk.edu/projects/papi/downloads/papi-3.5.0.tar.gz Kernel source Get the 2.4.22-1.2199.nptl source rpm from ftp://download.fedora.redhat.com/pub/fedora/linux/core/updates/1/SRPMS/kernel-2.4.22-1.2199.nptl.src.rpm In theory it may be possible to build from a CMU SCS kernel (the source is on afs) but I haven't figured out exactly how to do that yet. It is a little worrisome that the current SCS kernel is more recent (2.4.25) than this kernel, but when I tried using a vanilla 2.4.30 kernel (the most recent vanilla kernel with a perfctr patch) the result was disastrous. gcc32 RPM This is necessary to build this particular kernel. ftp://download.fedora.redhat.com/pub/fedora/linux/core/1/i386/os/Fedora/RPMS/gcc32-3.2.3-6.i386.rpm PerfSuite You need to get the 0.6.2 alpha release. http://perfsuite.ncsa.uiuc.edu/download/perfsuite-0.6.2a6.tar.gz PerfSuite needs tDOM http://www.tdom.org/files/tDOM-0.8.0.tar.gz If you want AFS get the kernel module source from openafs.org. My facilitized redhat 9 machine is running afs version 1.2.10 (run /usr/local/sbin/rxdebug localhost 7001 -version to get your afs version) so: http://www.openafs.org/dl/openafs/1.2.10/redhat-9.0/openafs-kernel-source-1.2.10-rh9.0.1.i386.rpm I'm assuming all the above software was downloaded into the same directory, $DIR. 2. Unpack everything. gtar xvfz papi-3.5.0.tar.gz etc. 3. Install some rpms (as root) OpenAFS kernel module source rpm -i openafs-kernel-source-1.2.10-rh9.0.1.i386.rpm gcc32 rpm rpm -i gcc32-3.2.3-6.i386.rpm (note that passing -ivh will segfault) 3. Install the kernel source and prepare the build (as root) rpm -ivh kernel-2.4.22-1.2199.nptl.src.rpm cd /usr/src/redhat/SPECS rpmbuild -bp kernel-2.4.spec (Note that it is possible and preferable, if not as easy, to not do this as root). 4. Patch and configure the kernel. (as root) cd /usr/src/redhat/BUILD/kernel-2.4.22/linux-2.4.22 make mrproper edit the Makefile so that EXTRAVERSION=-1.2199.nptl $DIR/papi-3.5.0/src/perfctr-2.6.x/update_kernel make menuconfig Under Processor type and features ->Performance-monitoring counters support Turn on (space) along with all sub options If Symmetric multi-processing support isn't on you must select Local APIC support on uniprocessors In order to get my USB mouse to work I had to turn on HID mouse support in the USB settings panel. make dep vmlinux modules ...wait patiently... make modules_install make install (this will modify /boot/grub/menu.lst for you) reboot (cross fingers) (you may want to do step 5 before rebooting, especially if you don't have a local root account to log into). 5. Build and install AFS kernel module (as root) cd /usr/src/ (change the linux symlink to point to the kernel we just built) rm linux ln -s redhat/BUILD/kernel-2.4.22/linux-2.4.22/ linux cd /usr/src/openafs-kernel-1.2.10/src ./configure make cp MODLOAD-2.4.22-1.2199.nptl-SP/libafs-2.4.22-1.2199.nptl.o /usr/local/lib/openafs cd /usr/local/lib/openafs /usr/local/etc/afsmodname -f SymTable.local -g libafs-2.4.22-1.2199.nptl.o reboot 6. Install perfctr device file and libraries (as root) mknod /dev/perfctr c 10 182 chmod 644 /dev/perfctr add the line alias char-major-10-182 perfctr to /etc/modules.conf cd $DIR/papi-3.5.0/src/perfctr-2.6.x/ make make PREFIX=/usr0/local install 7. Install PAPI cd $DIR/papi-3.5.0/src ./configure -- prefix=/usr0/local --with-tls=no make /usr0/local/lib needs to be in your LD_LIBRARY_PATH make fulltest (sanity check) make install 8. Install PerfSuite First install tDOM cd $DIR/tDOM-0.8.0/unix ../configure --prefiux=/usr0/local make make install You will need to add the tDOM library directory to your TCLLIBPATH: setenv TCLLIBPATH "/usr0/local/lib/tdom0.8.0" cd $DIR/perfsuite-0.6.2a6 ./configure --with-papi=/usr0/local --prefix=/usr0/local (you must include -with-papi even if it's the same as prefix) If you do not have a shared object version of libbfd on your system (locate libbfd.so), add --disable-binutils make make install Make sure LD_LIBRARY_PATH includes /usr0/local/lib. It should now work. For example: /usr0/local/bin/psrun gzip input creates gzip...xml which can be viewed with /usr0/local/bin/psprocess gzip...xml Note that the accuracy of the counters decreases if they are multiplexed, requiring a longer running benchmark. However, if a single counter configuration file is created using /usr0/local/bin/psconfig then you can get very stable numbers for that counter even with a short running program.