Professional backup with Git

Backup is a very-very important thing in IT. The loss of code-snippets, patch-sets, configurations or documents is very painful for programmers, system administrators and secrataries.

There are many options for this, among others hand (manually) copying. There are many applications responsible for backup on Linux, Mac or Windows. Many people use rsync as lacyc3 wrote a good post about it (in Hungarian).

What happens if the target is a remote machine?

Of course, it’s a good idea because when your local machine fails you can restore all data. For example I use git.

Why? How?

Take a very simple case in which there are two servers. One (www) is a webserver while another (bck) is just a backup machine.

www:

• Nginx
• PHP
• Ruby
• Rails
• Postgresql
• bind9
• UnrealIRCd
• sshd

bck: There is nothing on it (just for backup).

What should be saved from www to bck:

• /etc
• /srv/www
• /srv/UnrealIRCd
• /srv/scripts

I will not elaborate on the basic operation and usage of git. This is a version control system which can be stored both remote and local level.

First step we login in to the www and create a backup user and group (security). This is important because of the permissions. Backup user is separated from other users (to do it as root is really ugly). Let’s create this the user on bck. If this is done you can create a certification (backup@bck). The certification is needed for passwordless authentication.

(bck) $ sudo su backup -c /bin/bash
(bck) $ mkdir ~/.ssh
(bck) $ cd ~/.ssh
(bck) $ ssh-keygen -t dsa

This will create two files: id_dsa, id_dsa.pub

id_dsa: Private key (keep it secret… it’s PRIVATE)

id_dsa.pub: Public key

You need to copy the public key (id_dsa.pub) onto the www server into the .ssh directory of the backup user and rename this file to authorized_keys.

Now delete the password of our backup user. Set up the SSH daemon to accept the certifications as login method (eg: /etc/sshd_config):

PubkeyAuthentication yes
  AuthorizedKeysFile .ssh/authorized_keys

If all true then we can login into www from bck without password. (I use all of my users with certs and disable password login on my servers).

Ok. Then we need a Git repository on www. Go to the directory and initialize the git repository with the git init commend. Then add all files (what we want) with git add filename command or use git add . command to add all files. We do it as root… now…

(www) $ cd /etc
(www) $ git init
(www) $ git add .
(www) $ git commit -m 'Initial commit'
(www) $ chown -R backup.backup .git
(www) $ chmod -R o= .git

(www) $ cd /srv/www
(www) $ git init
(www) $ git add .
(www) $ git commit -m 'Initial commit'
(www) $ chown -R backup.backup .git
(www) $ chmod -R o= .git

(www) $ cd /srv/UnrealIRCd
(www) $ git init
(www) $ git add .
(www) $ git commit -m 'Initial commit'
(www) $ chown -R backup.backup .git
(www) $ chmod -R o= .git

(www) $ cd /srv/scripts
(www) $ git init
(www) $ git add .
(www) $ git commit -m 'Initial commit'
(www) $ chown -R backup.backup .git
(www) $ chmod -R o= .git

Yes. We did it. The first part. We need a clone on bck.

(bck) $ sudo su backup -c /bin/bash
(bck) $ cd -
(bck) $ mkdir www-backup
(bck) $ git clone backup@www:/etc etc
(bck) $ git clone backup@www:/srv/www www
(bck) $ git clone backup@www:/srv/UnrealIRCd UnreadlIRCd
(bck) $ git clone backup@www:/srv/scripts scripts

Now we have a copy of all of our important datas. We need a script that commits daily changes. Create a file on www (eg: /srv/scripts/backup.zsh):

#!/usr/bin/env zsh

DIRS=("/etc", "/srv/www", "/srv/scripts", "/srv/UnrealIRCd")

for d in $DIRS
do
  cd "${d}"
  git add -A
  gi commit -m "Daily commit"
done

Add this script to the crontab:

(www) $ chmod +x /srv/scripts/backup.zsh
(www) $ crontab -e

now add this line:

0 3 * * * /srv/scripts/backup.zsh > /dev/null

Run daily commit every day at 3am. Then we need a script on bck to fetch changes. Example we put this script into /srv/scripts/backup-pull.zsh on www (it will be fetched on pull on bck):

#!/usr/bin/env zsh

cd ~/www-backup
for d in *
do
  cd ${d}
  git pull
  cd ..
done

Create a crontab entry on bck (home of backup user: /var/backup):

0 5 * * * /var/backup/www-backup/scripts/backup-pull.zsh > /dev/null

You can merge the two scripts if you want:

#!/usr/bin/env zsh

ssh backup@www /srv/scripts/backup.zsh

cd ~/www-backup
for d in *
do
  cd $d
  git pull
  cd ..
done

In this case www doesn’t need a crontab entry.

If you want a separated commit (eg: /etc after add_user) then:

(www) $ cd /etc
(www) $ git add -A
(www) $ git commit -m 'New user: yitsushi'

Why I like this?

It is very easy to recover an older version of any file because it’s a version control system. we can commit and push in locally (eg: laptop) so we don’t need a secondary server. Also check out the operation and use of git because is has so many pranks. For example if we configure the mail server but don’t want to fail the present system then we can create a branch and when we are tired just commit changes in the new branch and checkout the master (original). When we finish changes then we can merge it into the origin. Largely risk-free way to test =).