Implementing locks for bash scripts

In the last couple of days, I’ve been working on our CI system: builbo (the BUILd BOt ). Builbo’s job is to build an ipa or apk from every branch we have in our repositories, and make it available for developers and QA engineers. In order to take full advantage of our CI server, we needed to be able to invoke multiple instances of the build scripts in parallel. That meant we needed resource locking.

There seems to be a couple of techniques for implementing locks in bash scripts, as discussed in this stackexchange question. However, the solutions discussed there do not put these techniques into a reusable form.

So I’ve written a small library that can be included in bash scripts to create and maintain multiple locks. It uses the technique that uses the no-clobber option of the shell. It’s tested with bash 3.2 on OSX, but should work on linux as well.

# **************************************************
# Lock management functions and variables

# This is where you want locks to be stored.
DATA_DIR=.

ACQUIRED_LOCKS=''

# internal method, don't call directly
is_acquired_lock () [[ ACQUIRED_LOCKS =~ (^| )$1( |$) ]]

# internal method, don't call directly
add_to_acquired_locks () {
    ACQUIRED_LOCKS="$ACQUIRED_LOCKS $1"
}

remove_from_acquired_locks () {
    ACQUIRED_LOCKS=${ACQUIRED_LOCKS/$1}
    ACQUIRED_LOCKS=${ACQUIRED_LOCKS/#. }
    ACQUIRED_LOCKS=${ACQUIRED_LOCKS/%. }
}

remove_all_acquired_locks () {
    local lock_file_name=''
    for lock_file_name in $ACQUIRED_LOCKS
    do
        remove_from_acquired_locks "$lock_file_name"
    done
}

# This takes a single parameter: name of lock file. It should not include a path.
acquire_lock () {
    local lock_file_name=$1
    local lock_file="$DATA_DIR/$lock_file_name"
    local my_pid=$$

    # Check if we already have the lock and avoid deadlocks
    is_acquired_lock $lock_file_name && return

    # Check if the lock owner is still alive: clean up lock file if it's not
    # builtin test does not do lazy eval, so we do nested ifs
    if [ -e "$lock_file" ]
    then
        if [ $( ps $( head -n 1 "$lock_file" ) > /dev/null ; echo $? ) -ne 0 ]
        then
            echo "Removing stale lock $lock_file for pid `head -n 1 "$lock_file"`"
            rm -f "$lock_file" > /dev/null
        fi
    fi

    # Setting noclobber and trying to redirect to lock file, 
    # then adding to the list of acquired locks as soon as possible
    until ( set -C; echo "$my_pid" > "$lock_file" && add_to_acquired_locks "$lock_file_name" ) 2> /dev/null
    do
        sleep 5
    done
    set +C
}

# This takes a single parameter: name of lock file. It should not include a path.
# It releases the lock only if the lock is this process's.
release_lock () {
    local lock_file_name=$1
    local lock_file="$DATA_DIR/$lock_file_name"
    # check that it's our lock, then remove
    is_acquired_lock $lock_file_name && [ $$ -eq $(head -n 1 "$lock_file") ] && \
            rm "$lock_file" && remove_from_acquired_locks "$lock_file_name"
}

# End of lock management functions and variables
# **************************************************

Note that these functions also detect and remove stale locks, by writing the pid of the shell that has the lock in the lock file.

If you find this useful, or have any ideas about improvements, please do let me know.