How To: Backup with SSH and Rsync

Posted on by
1

PuTTY logoMy websites (hosted with EvoHosting) are incrementally backed up to a server running on Atlantis – my VMware ESXi host – for the dual purposes of offline development and data resilience. I’m fairly sure that there are numerous guides out there already to accomplish this, but hopefully this will be useful for some.

For ease of explanation, I will skip any network configuration, though if anyone would like detailed configuration on NAT, port forwarding and dynamic DNS, please by all means leave a comment or email me.

SSH Key Configuration

The first thing to do is to configure SSH key authentication, so that your source server can login to your backup server without being prompted for a password. Run the following command on your source server:

ssh-keygen -t rsa

Copy this key into your authorized_keys file (usually in /home/$USER/.ssh) on your backup server and set permissions to 700 so that nobody else can read or edit the file – these permissions are explicitly required for most SSH installations, so double-check that they are correct.

Rsync Configuration

After looking around for help on rsync for a while, this is the rsync command I have come up with:

rsync --delete  -ae 'ssh -p999' /home/craig/public_html/ backup.network.local:/home/craig/public_html/

The “delete” option tells rsync to delete files on the backup that aren’t in the source, and the -p999 option tells SSH to use port 999 for the transfer – I used this as I use a non-standard port for SSH for security, however if you run SSH on its standard port (22), you can remove the -p option.

MySQL Backups

Although Rsync will copy my public_html directory, it won’t backup my MySQL databases. To do this, I used the mysqldump utility. Unfortunately my file permissions wouldn’t allow me to use the more efficient mysqlhotcopy utility, so I have had to make do with a slightly crude full backup, which is gzipped to keep the file size minimal:

mysqldump --all-databases | gzip > ~/databases.sql.gz

This full backup is then imported back into the backup server’s MySQL databases.

Automation via Cron

All that was left to do after this was to schedule the backups automatically via each server’s cron scheduler. The MySQL database imports take place 45 minutes after the Rsync copy to give the backup time to complete.

Where needed, I have written simple Bash scripts to simplify the process and to remove the backup SQL scripts after transfers and imports.

~~

Hopefully that has been educational, please feel free to comment if you have questions/suggestions :)

Multi-booting Windows and Linux using GRUB

Posted on by
2

After my last post on multi-booting different operating systems on the same PC – which was written close to 18 months ago now – I have gained even more experience in setting up multi-boot systems and would like to take the opportunity to revise my previous post and supersede it with this guide.

Partitioning

My only advice with partitioning is to set sensible partition sizes. Most Linux distributions will sit and work happily on 10GB of drive space, though I usually give 20GB to leave some wiggle-room. Don’t forget to incorporate a swap partition for your Linux distribution, the general “rule of thumb” is to use the same amount of space as you have physical memory, so if you have 2GB of RAM, use a 2GB swap partition.

As you will be installing Windows first, you can go ahead and create the partitions for Windows and leave the rest of the space blank.

Installing the Operating Systems

The major difference to the last guide I posted is that this time I will be using GRUB – the Linux bootloader – to choose between operating systems when the machine boots. To do this, first install Windows as normal using your Windows installation media (CD or DVD).

When you have your Windows installation up-and-running, reboot and install Linux. I will be using Fedora as the Linux distribution in this guide as it’s the distribution I am most familiar with, but these instructions should be generic enough to apply to most distributions (Ubuntu, SuSE, Debian, etc).

Proceed with the Linux installation as normal, but look for the screen where the installer configures the bootloader. This could be hidden in an “Advanced” menu somewhere in some distributions, but on Fedora it is the last screen before the installation gets under way.

Your Windows installation should already be listed as “Other”. You can rename this if you want, and you may also be able to change the default option, depending on the options you have within your distribution’s installer. When you have the bootloader configured, move on and install Linux as normal.

When the machine reboots, you should be provided with a screen similar to this:

Congratulations, you now have a working dual-boot system!

Further Configuring GRUB

You can change quite a few options and customise your GRUB screen even further by editing GRUB’s “menu.lst” file. This is found in the /boot/grub folder, and you will have to use root (or sudo) priviledges to edit it as it is a system file.

A lot of in-depth information is available on the GNU GRUB Manual, however here is a quick run-down of a couple of the most useful options:

default – Sets the default option to use. The list is zero-based, so the top option is 0 and increases as you move down the list.
timeout – The number of seconds before GRUB automatically boots using the default option. Set this to 0 to disable the timeout.
hiddenmenu – This tells GRUB to hide the menu and just display a generic “press any key” message. Put a # before this line to comment it out if you want to view the menu by default.

Hopefully the above guide has been useful, please feel free to leave comments if you have any questions or suggestions!

Multi-Booting: A Revised Guide – XP/Vista/Linux

Posted on by
4

After writing my last quad-boot entry, I’ve gained quite a bit of experience in troubleshooting and sorting out multi-boot installs so I thought I’d re-write my old quad-boot blog to be slightly more like an informative guide rather than a story, and also downgrading it to just cover tri-booting, as I haven’t gotten around to replacing Apple’s OS X Leopard with Sun’s OpenSolaris yet.

Summary

This guide will use Fedora 8 as the Linux distribution, live CDs are available in both Gnome and KDE flavours for those of you who want to try it before actually touching your hard drive’s partition layout. I would recommend installing from the Fedora Install DVD as from experience, the live CD misses out several packages – make and gcc among others – by default that make life a lot easier after installation.

The other operating systems I’ll be dealing with are Windows XP Media Center Edition and Windows Vista Business.

Step One – Partitioning

My desktop PC’s hard drive layout consists of one 200Gb drive for the operating systems and one 320Gb NTFS drive as a central data area, but I’ll assume that you’re dealing with a single drive. Partition whatever space you have into three sections – four if you want a central data area. Although Windows is a lot more space-hungry than Linux, I gave each system an equal amount of space.

You can use whatever tools are available to you to create your partition layout. Both XP and Vista partitions should be NTFS and it’s best to either leave the Linux partition and format this using the install disc. If you’re creating a central data area to use between operating systems, use the FAT32 file system as while Linux can read/write to NTFS, it’s not proven with large file sizes – I use NTFS for my 320Gb data drive purely because FAT32 cannot cope with that size without splitting into separate partitions – something I wanted to avoid.

Step Two – Installing Fedora

Not much instruction is needed at this point. The Fedora Installation Guide is an excellent tutorial through the process. You can install GRUB – the Linux bootloader – to the drive to get the system installed, booting and stable, although booting to Linux will be configured via a GUI utility in Vista later on.

The general guide for creating a swap partition is to use double the amount of RAM you have in your system. My PC has 2Gb of RAM, so I partitioned off 4Gb and set this to be my swap space.

Step Three – Installing Windows

I’ll deal with both Windows systems in the same step here because they’re both relatively similar. You’ll have to do some partition trickery before installation so that Windows assigns the correct drive letters and installs each version in ignorance of anything else on the system.

You can do this via pretty much any partition manager, but I used GParted as included on the Fedora and Ubuntu LiveCDs – or you can use the dedicated 50Mb GParted LiveCD. Select the XP partition, right-click on it and select Manage Flags. Put a tick next to the “hidden” flag and close the window.

Now simply place the XP installation CD in the drive, fire up the installer and install as normal. Just remember to make sure you’re installing to the right partition on your drive. When you’ve installed and can boot to XP, reboot to GParted and swap the “hidden” flags. You can then install Vista as normal.

Step Four – Pulling It All Together

You should now have three working operating systems on your hard drive, but you can only boot to Vista. Head over and download the free EasyBCD program from NeoSmart and install it on Vista. You can then use that to create the necessary boot menu options for each system. EasyBCD includes GRUB and installs it on your Vista partition, so you don’t need GRUB on the Linux partition. Clever eh?

Done!

After you’ve created the boot menu options, reboot into each system once to test. You should now have a working tri-boot system – congratulations! If anything goes wrong or if you’ve got any questions, by all means e-mail me and I’ll do my best to help. If you’re using Fedora as your Linux distribution, head over to the Fedora forums and register.

Hopefully this is more of an informative guide rather than a narrative guide and you’ve found it helpful. If you have (or even if you haven’t), feel free to leave a comment!

Cheers
Craig