Tim's Bits and Pieces: Linux

Sting like a bee

TimP — Fri, 17 Jun 2011 23:03:54 +0200

Speedup of Trac Static Content

TimP — Wed, 25 Aug 2010 23:03:54 +0200

At work I recently spent some time working on getting our hosted Trac to run faster. I found one simple little trick that gave up to a 20% speed improvement, so I thought I'd share it here.

The basic idea is that Trac has a lot of overhead due to executing a pile of Python code for each requested file including static content like the logos, background images, CSS, etc. If you tell Apache just to skip the Python code for the static content you can save a lot of time and processing power. All the default static content is under "/python/library/path/site-packages/trac/htdocs" which Trac servers up as "/chrome/common/". Basically a request for http://server.com/trac/chrome/common/logo.png will get a file from somewhere like /usr/lib/python2.5/site-packages/trac/htdocs/logo.png on the harddisk. To skip Trac for this we want to add something like the following to the Apache configuration (AliasMatch is part of mod_alias):

 	AliasMatch ^/(.+)/chrome/common(.*) /usr/lib/python2.5/site-packages/trac/htdocs$2

This matches any request for a path that matches /something/chrome/common/something_else, and then sends back the file located at /usr/lib/python2.5/site-packages/trac/htdocs/something_else; preventing the Trac Python code from running. Since there is about a dozen images, CSS files, and JavaScript files for each Trac page the time saved by this very quickly adds up.

Slackware 64 Bit

TimP — Wed, 20 May 2009 23:03:54 +0200

Slackware now has an official 64-bit port, so far it's only available for people who run -current, but be available as an alternative architecture for the next release (Slackware 13). Whoo!

Awk and Trac.ini

TimP — Tue, 05 May 2009 23:03:53 +0200

You'll notice I'm posting late again, for which I blame C.S. Lewis, and since he's dead he can't defend himself from this ~~unfounded accusation~~ completely serious and non-trivial explanation for my tardiness.

As for actually post, I work as a support guy for a Subversion hosting company. We also host Trac, so part of my job is occasionally setting up custom ticket fields for customers. This involves editing the trac.ini, a simple text file. This is fairly easy when you just have to modify one Trac instance, but every now and again I have to modify a couple of dozen files. I have tried this using sed before, but this is a prime example of the the "When all you've got is a hammer, every problem is a nail" fallacy. I'd been sort of interested in learning more about awk and this seemed like a prime opportunity to "learn by doing", and now I'll reinforce with "learning by teaching". So with out further ado here's my short script interspersed with an explanation of how it works, and a complete copy of the code at the bottom.

BEGIN { i = 0 }

The BEGIN section is a special section that is run once before processing the file, it can be used for setting variables, as I have done here. The i variable, a more descriptive name would have been good, stores the state of the script.

{
if ( i == 0 ) { print }

The { starts the main awk script, which is run once for each line in the file being processed. This first line simply prints out the input line being examined if i equals zero, which it does in the initial state of the program.

    else
    {
        if ( i == 1 ) {
            print "# Ticket Custom Section\n"

            i = 2 }

The next section is the else clause for the preceding if clause, which executes it's own if-else pair. The if clause executes if i equals one, which it does if we have just passed the "[ticket-custom]" line in the trac.ini file. This section prints out the required custom ticket definitions and changes i to two, which will prevent anything from being written until it gets to the end of the custom ticket section, see below. You need to change the print statement to print the the required ticket custom section of course; it might be more readable to use a series of individual statements rather than just one.

else { if ( $0 ~ /\[.*\]/ ) { print ; i = 0 } }
}

This next line is executed if i doesn't equal zero or one, meaning that we are currently in the custom field section. It's a single short if statement that checks if the line being processed ($0) matches (~) a regular expression, in this case /\[.*\]/. This will match any line that contains a "[" followed by a series of characters, followed by a "]"; basically a trac.ini section head. So if it finds the start of a new section in the file it will print it out, and then set i back to zero, which means it will just print out the rest of the lines in the file.

if ( $0 == "[ticket-custom]" ) { i = 1 }
}

Then here's the final line which is a simple if that is run for each line in the file and checks if the line is the start of the custom field section ([ticket-custom]), and if so sets i to one; the result of which is show above.

Since the code is probably slightly unclear unless you can see the whole thing here it is:

BEGIN { i = 0 }
{
    if ( i == 0 ) { print }
    else
    {
        if ( i == 1 ) {
            print "# Ticket Custom Section\n"

            i = 2 }
        else { if ( $0 ~ /\[.*\]/ ) { print ; i = 0 } }
    }

    if ( $0 == "[ticket-custom]" ) { i = 1 }
}

Countdown in a Shell Script

TimP — Tue, 07 Apr 2009 23:03:53 +0200

Tuesday's still the weekend right? ;-)

Anyhow here's a little code snippet for inserting a count down into a shell script. Simply put this before whatever you want to delay, and then make sure that the first argument you pass to the script is the number of seconds to wait. An example of the sort of thing you might want to use this for is taking screenshots.

#!/bin/bash
for i in $(seq $1 -1 1); do
echo $i
sleep 1
done
echo 0
$2

For example if you run it like this: ./countdown.sh 5 "echo Blast Off" you'll get the following output:

$ ./countdown.sh 5 "echo Blast Off"
5
4
3
2
1
0
Blast Off
$

Battery State Monitor Script

TimP — Sat, 04 Apr 2009 23:03:53 +0200

Oops, completely missed posting last weekend, mostly just because I'm too easily distracted. Such is life, hopefully I'll do two or three posts this weekend to even it up.

This is a simple script I wrote to notify me whenever my battery is running low on power. Basically to use it you just create a new shell script with the following in it and then run it using bash ~/bin/battery_monitor.sh & in your .xinitrc script. This should then check your battery status every minute and if it's below ten minutes remaining it will produce a warning message, after generating a warning it will wait five minutes before checking again to make sure it doesn't get really annoying.

#!/bin/bash
# Simple shell script that runs as a daemon to check the status of a Laptops
# battery and produce an error message every five minutes if it get's to
# less than 10 minutes of charge remaining and is not plugged in.

# Copyright (c) 2009, Timothy Pollard
#
# Redistribution and use with or without modification, are permitted provided
# that the above Copyright notification remains complete.

# The file that contains the state of the Battery
PROC_FILE=/proc/acpi/battery/BAT1/state

while true; do
    # Get the charging state from the PROC_FILE, this should be one of "charged",
    # "charging", or "discharging"
    CHARGING_STATE=$(cat $PROC_FILE | grep "^charging state:" | sed -e "s/charging state: *//g")

    if [ $CHARGING_STATE == "discharging" ]; then
        # Get the current rate and remaining capacity
        CURRENT_RATE=$(cat $PROC_FILE | grep "^present rate:" | sed -e "s/present rate: *//g" -e "s/ mW//g")
        REMAINING_CAPACITY=$(cat $PROC_FILE | grep "^remaining capacity:" | sed -e "s/remaining capacity: *//g" -e "s/ mWh//g")

        # Remaining battery life in minutes
        REMAINING_TIME=$(echo "scale=4; $REMAINING_CAPACITY/$CURRENT_RATE*60" | bc | sed -e "s/\..*//g")

        if [ $REMAINING_TIME -lt 10 ]; then
            xmessage -center "There are currently only $REMAINING_TIME minutes of battery life remaining."

            # Sleep for 4 minutes (4*60=240), this combined with the default
            # sleep of 60 seconds assures that you are only alerted once every 5
            # minutes
            sleep 240
        fi
    fi

    # Delay the next run for 60 seconds
    sleep 60
done

Some tips on speeding up Slackware 12.2 Boot

TimP — Sat, 14 Feb 2009 23:03:54 +0200

My Slackware laptop had decently fast boot times (a little over a minute), but I'm impatient and decided to see if I could speed it up some more. Here's the changes I made and the results down the bottom. This is on a Dell Vostro 1510, with 4GB of ram and a 2.1GHz Core 2 Duo. Obviously the results will vary depending on your hardware.

Lilo and the Kernel
/etc/rc.d/rc.M
/etc/rc.d/rc.inet1
The Results
References

Lilo and the Kernel

Probably the easiest piece of low hanging fruit is Lilo's "compact" option. This gets rid of the "Loading Linux................" pause straight after the Lilo prompt.

# Start LILO global section
boot = /dev/sda
compact

Another popular step that can provide not inconsiderable speed advantages is to compile yourself a custom kernel.

/etc/rc.d/rc.M

There are quite a few options in rc.M that you can either run in the background (add & to the line) to speed up the boot times, and there's even a few that you can just comment out and ignore.

Starting from the top and going down we have the optical drive symlinks, simply add an "&" to the end of the "/bin/sh /lib/udev/rc.optical-symlinks" line in the following code segment:

# If we're using udev, make /dev/cdrom and any other optical drive symlinks
# if some udev rule hasn't made them already:
if grep -wq sysfs /proc/mounts && grep -wq tmpfs /proc/filesystems; then
  if ! grep -wq nohotplug /proc/cmdline ; then
    if [ -x /lib/udev/rc.optical-symlinks -a -x /etc/rc.d/rc.udev ]; then
      /bin/sh /lib/udev/rc.optical-symlinks &
    fi
  fi
fi

After that there's fc-cache, gtk-update-icon-cache, and ldconfig; these can be run manually whenever you install a new Font, icon set, or library, so I simply commented out these lines:

## Update the X font indexes:
#if [ -x /usr/bin/fc-cache ]; then
# echo "Updating X font indexes: /usr/bin/fc-cache -f &"
# /usr/bin/fc-cache -f
#fi

## Update any existing icon cache files:
#if find /usr/share/icons 2> /dev/null | grep -q icon-theme.cache ; then
#  for theme_dir in /usr/share/icons/* ; do
#    if [ -r ${theme_dir}/icon-theme.cache ]; then
#      echo "Updating icon-theme.cache in ${theme_dir}..."
#      /usr/bin/gtk-update-icon-cache -t -f ${theme_dir} 1> /dev/null 2> /dev/null
#    fi
#  done
#  # This would be a large file and probably shouldn't be there.
#  if [ -r /usr/share/icons/icon-theme.cache ]; then
#    echo "Deleting icon-theme.cache in /usr/share/icons..."
#    /usr/bin/gtk-update-icon-cache -t -f /usr/share/icons 1> /dev/null 2> /dev/null
#    rm -f /usr/share/icons/icon-theme.cache
#  fi
#fi

## Update all the shared library links:
## Can do this manually after installing new libraries
#if [ -x /sbin/ldconfig ]; then
# echo "Updating shared library links: /sbin/ldconfig"
# /sbin/ldconfig
#fi

Then there's another stack of services that I start running in the background by adding ampersands:

# Start networking daemons:
if [ -x /etc/rc.d/rc.inet2 ]; then
. /etc/rc.d/rc.inet2 &
fi

# Start the Network Time Protocol daemon:
if [ -x /etc/rc.d/rc.ntpd ]; then
sh /etc/rc.d/rc.ntpd start &
fi

# SCIM and other GTK+ input methods like this file kept updated:
if [ -x /usr/bin/gtk-query-immodules-2.0 ]; then
echo "Updating gtk.immodules: gtk-query-immodules-2.0 > /etc/gtk-2.0/gtk.immodules"
/usr/bin/gtk-query-immodules-2.0 > /etc/gtk-2.0/gtk.immodules 2> /dev/null &
fi

# Start the print spooling system.  This will usually be LPRng (lpd) or CUPS.
if [ -x /etc/rc.d/rc.cups ]; then
  # Start CUPS:
  /etc/rc.d/rc.cups start &
elif [ -x /etc/rc.d/rc.lprng ]; then
  # Start LPRng (lpd):
  . /etc/rc.d/rc.lprng start &
fi

# Start netatalk. (a file/print server for Macs using Appletalk)
if [ -x /etc/rc.d/rc.atalk ]; then
/etc/rc.d/rc.atalk &
fi

# Start the sendmail daemon:
if [ -x /etc/rc.d/rc.sendmail ]; then
. /etc/rc.d/rc.sendmail start &
fi

# Load ALSA (sound) defaults:
if [ -x /etc/rc.d/rc.alsa ]; then
. /etc/rc.d/rc.alsa &
fi

/etc/rc.d/rc.inet1

Finally I recommend decreasing the DHCP timeout used in rc.inet1; the default is 30 seconds (it used to be 60), I decreased mine further down to 15 seconds:

# 30 seconds should be a reasonable default DHCP timeout. 60 was too much. :-)
echo "/etc/rc.d/rc.inet1: /sbin/dhcpcd -d -t ${DHCP_TIMEOUT[$i]:-15} ${DHCP_OPTIONS} ${1}" | $LOGGER
/sbin/dhcpcd -d -t ${DHCP_TIMEOUT[$i]:-15} ${DHCP_OPTIONS} ${1}

The Results

Boot time from Lilo to login prompt was approximately one minute thirty before I performed these changes, and that decreased to twenty-five seconds. Once you include the time from first power on till Lilo it comes to about 35-40 seconds boot time, plus another five seconds to login and start X. The biggest speed increase seems to be from using compact in Lilo and the custom kernel.

References

There where a couple other resources on the Internet that got me started on this, and pointed out several of the ideas that I used: The section on boot speed in the Slackware tips article on f'(x) and an alt.os.linux.slackware thread on boot time.

Stuff

TimP — Fri, 12 Dec 2008 23:03:54 +0200

A few things of note:

I missed a post last week, I'll try and get an extra post up on Sunday or Monday to make up for it.
I had one of my pictures posted on InsectPOD
Slackware 12.2 was released a couple of days ago.
Today's Word of the Day is pollard, cool. :-)

Random Backgrounds Script

TimP — Wed, 06 Aug 2008 23:03:54 +0200

I recently wrote this simple script to allow me to set a random image as my background:

#!/bin/bash
# Selects a random jpeg file in the ~/backgrounds/ directory (or it's
# sub-directories) and makes it the background image.
file_list=$(find ~/backgrounds/ -name "*.jpg")
num_files=$(echo $file_list | wc -w)
ran_line_number=$(($RANDOM%$num_files+1))
selected_file=$(echo $file_list | tr " " "\n" | sed -n ${ran_line_number}p)
fbsetbg $selected_file

On most light-weight window managers, like Blackbox, simply copy any images you want to use as backgrounds into ~/backgrounds/ and you're good to go. Some other Window Managers/Desktop environments look after the desktop background themselves so you would have to disable that in their configuration first. If you wanted to run this every half hour or something you could put it in your crontab, or else you can just set it up so it runs when you start your window manager.

-- Edit 09/05/2009 --

I've modified the code very slightly, to fix a bug which either didn't exist when run with the older version of bash/find I was have been using when I wrote this. I've also cleaned up the code a little.

Installing Kazehakase 0.5.1 on Slackware 12

TimP — Wed, 30 Jan 2008 23:03:54 +0200

I recently decided to install Kazehakase 0.5.1 on my computer. As with any software installation of Slackware my first step was to check for a SlackBuild on SlackBuilds.org. There is one but it is for version 0.4.7. OK, that's often not a problem, you just change the $VERSION variable to the version you actually want. So I tried that, the SlackBuild printed out the usual range of progress reports, etc, then gave me this:

gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src -I../../src/bookmarks -I../../src/libegg/pixbufthumbnail -DGTK_DISABLE_DEPRECATED=1 -DGDK_DISABLE_DEPRECATED=1 -DG_LOG_DOMAIN=\"Kazehakase-Utils\" -DG_DISABLE_DEPRECATED=1 -O2 -march=i486 -mtune=i686 -Wall -Wmissing-declarations -Wmissing-prototypes -Wpointer-arith -Wcast-align -MT glib-utils.lo -MD -MP -MF .deps/glib-utils.Tpo -c glib-utils.c -fPIC -DPIC -o .libs/glib-utils.o
In file included from glib-utils.c:27:
glib-utils.h:24:18: error: glib.h: No such file or directory
glib-utils.h:25:21: error: gtk/gtk.h: No such file or directory
In file included from glib-utils.c:27:
glib-utils.h:32: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'G_END_DECLS'
make[5]: *** [glib-utils.lo] Error 1
make[5]: Leaving directory `/tmp/SBo/kazehakase-0.5.1/src/utils'
make[4]: *** [all] Error 2
make[4]: Leaving directory `/tmp/SBo/kazehakase-0.5.1/src/utils'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/tmp/SBo/kazehakase-0.5.1/src'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/tmp/SBo/kazehakase-0.5.1/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/tmp/SBo/kazehakase-0.5.1'
make: *** [all] Error 2

What the? It can't find the libraries? Isn't that what configure is supposed to do? Oh, well I guess I'll just modify the SlackBuild to pass the locations of the libraries required. It can't find some others as well? Darn, I guess I'll have to add them as well.

Fortunately before I went through adding a dozen different libraries locations to the SlackBuild, I noticed to following way up in the configure output:

checking for GTK+ - version >= 2.12.0... no
*** Could not run GTK+ test program, checking why...
*** The test program failed to compile or link. See the file config.log for the
*** exact error that occured. This usually means GTK+ is incorrectly installed.

Wait so one of the required libraries is not present (or in my case the version was too low), but configure happily lets my continue with the build process anyhow? It should just return an error and quit.

Fortunately, this is not going to stop me, I'll simply upgrade GTK, so I go to the Slackware FTP site find the GTK+2 official SlackBuild and change the $VERSION to 2.12.5 and compile the source from ftp.gtk.org. There now I should be able to compile Kazehakase, wait no I need to upgrade glib2 and pango before I can upgrade GTK+, so I upgrade them to 2.14.5 and 1.18.4 respectively and now I can compile GTK+ 2.12.5, and then finally compile Kazehakase 0.5.1.

Aaah, the joy of a job well done.