How To: Backup with SSH and Rsync

Posted on by
2

PuTTY logoMy websites (hosted with EvoHosting) are incrementally backed up to a server running on Atlantis – my VMware ESXi host – for the dual purposes of offline development and data resilience. I’m fairly sure that there are numerous guides out there already to accomplish this, but hopefully this will be useful for some.

For ease of explanation, I will skip any network configuration, though if anyone would like detailed configuration on NAT, port forwarding and dynamic DNS, please by all means leave a comment or email me.

SSH Key Configuration

The first thing to do is to configure SSH key authentication, so that your source server can login to your backup server without being prompted for a password. Run the following command on your source server:

ssh-keygen -t rsa

Copy this key into your authorized_keys file (usually in /home/$USER/.ssh) on your backup server and set permissions to 700 so that nobody else can read or edit the file – these permissions are explicitly required for most SSH installations, so double-check that they are correct.

Rsync Configuration

After looking around for help on rsync for a while, this is the rsync command I have come up with:

rsync --delete  -ae 'ssh -p999' /home/craig/public_html/ backup.network.local:/home/craig/public_html/

The “delete” option tells rsync to delete files on the backup that aren’t in the source, and the -p999 option tells SSH to use port 999 for the transfer – I used this as I use a non-standard port for SSH for security, however if you run SSH on its standard port (22), you can remove the -p option.

MySQL Backups

Although Rsync will copy my public_html directory, it won’t backup my MySQL databases. To do this, I used the mysqldump utility. Unfortunately my file permissions wouldn’t allow me to use the more efficient mysqlhotcopy utility, so I have had to make do with a slightly crude full backup, which is gzipped to keep the file size minimal:

mysqldump --all-databases | gzip > ~/databases.sql.gz

This full backup is then imported back into the backup server’s MySQL databases.

Automation via Cron

All that was left to do after this was to schedule the backups automatically via each server’s cron scheduler. The MySQL database imports take place 45 minutes after the Rsync copy to give the backup time to complete.

Where needed, I have written simple Bash scripts to simplify the process and to remove the backup SQL scripts after transfers and imports.

~~

Hopefully that has been educational, please feel free to comment if you have questions/suggestions :)

Multi-booting Windows and Linux using GRUB

Posted on by
2

After my last post on multi-booting different operating systems on the same PC – which was written close to 18 months ago now – I have gained even more experience in setting up multi-boot systems and would like to take the opportunity to revise my previous post and supersede it with this guide.

Partitioning

My only advice with partitioning is to set sensible partition sizes. Most Linux distributions will sit and work happily on 10GB of drive space, though I usually give 20GB to leave some wiggle-room. Don’t forget to incorporate a swap partition for your Linux distribution, the general “rule of thumb” is to use the same amount of space as you have physical memory, so if you have 2GB of RAM, use a 2GB swap partition.

As you will be installing Windows first, you can go ahead and create the partitions for Windows and leave the rest of the space blank.

Installing the Operating Systems

The major difference to the last guide I posted is that this time I will be using GRUB – the Linux bootloader – to choose between operating systems when the machine boots. To do this, first install Windows as normal using your Windows installation media (CD or DVD).

When you have your Windows installation up-and-running, reboot and install Linux. I will be using Fedora as the Linux distribution in this guide as it’s the distribution I am most familiar with, but these instructions should be generic enough to apply to most distributions (Ubuntu, SuSE, Debian, etc).

Proceed with the Linux installation as normal, but look for the screen where the installer configures the bootloader. This could be hidden in an “Advanced” menu somewhere in some distributions, but on Fedora it is the last screen before the installation gets under way.

Your Windows installation should already be listed as “Other”. You can rename this if you want, and you may also be able to change the default option, depending on the options you have within your distribution’s installer. When you have the bootloader configured, move on and install Linux as normal.

When the machine reboots, you should be provided with a screen similar to this:

Congratulations, you now have a working dual-boot system!

Further Configuring GRUB

You can change quite a few options and customise your GRUB screen even further by editing GRUB’s “menu.lst” file. This is found in the /boot/grub folder, and you will have to use root (or sudo) priviledges to edit it as it is a system file.

A lot of in-depth information is available on the GNU GRUB Manual, however here is a quick run-down of a couple of the most useful options:

default – Sets the default option to use. The list is zero-based, so the top option is 0 and increases as you move down the list.
timeout – The number of seconds before GRUB automatically boots using the default option. Set this to 0 to disable the timeout.
hiddenmenu – This tells GRUB to hide the menu and just display a generic “press any key” message. Put a # before this line to comment it out if you want to view the menu by default.

Hopefully the above guide has been useful, please feel free to leave comments if you have any questions or suggestions!

2008 – The Year So Far

Posted on by
1

So, what’s been happening in the last two months?

Linux – Transition Complete

I’ve now finally got Fedora 8 up and running on my desktop PC as well as my laptop so Linux is my main operating system – with Windows being used from time to time for various web design tasks (the recent redesign of my university webspace being one of them) and I have to admit that despite my stance as a self-confessed “Windows user” a year ago, I’m now really enjoying the Linux experience. Everything is so streamlined, but it does take some getting used to and playing around with to get it the way I want it (some of the difficulties and shortcomings being detailed in my transitioning blog) – but I’m generally happy to play around and try different things so maybe this suits me better than others.

SonicaCMS goes live!

SonicaCMS – my plain-text PHP content management system – has now been released as an alpha version (meaning for very early testing and evaluation purposes). It’s been a work-in-progress for quite a while and with a general lull in university work I’ve decided to go public and make the first version available, so if you’d like to download it and try it out, by al means head over to www.sonicadesign.com/scms – I’d be very grateful for any bugs/feedback.

University - New term

As far as university goes, I’m awaiting the results of my first semester exams on Thursday 28th February – some more eagerly than others though. I’ve also actually given a lecture on web development to first years – a somewhat daunting experience but turned out well in the end once I’d gotten over the nerves and into the flow of things. The lecture mainly covered bits and pieces that I’d picked up over the last year and a half of doing websites on a semi-professional freelance basis as well as some search engine optimisation (SEO) material. By the end I actually started enjoying it, so I could be making a return to the lectern at some point in the not-too-distant future…

Along with the new term comes new modules – one of which I’m actually (shock, horror) enjoying: CS35910, aka Internet Services Administration. It generally covers a lot of the things I’ll need to know when I’m out in industry, and it’s material I’m genuinely interested in which always makes a difference. I’ve yet to be given any assignments this semester, but I’m sure they’ll start rolling in soon, as there’s only four weeks left before term ends for Easter.

Liverpool FC - Benitez to go?

Away from computers and all things geek, Liverpool FC are having a bit of a rough period recently, with several high-profile spats between manager Rafael Benitez and the club’s American owners George Gillett and Tom Hicks. After being knocked out of the Carling Cup by Chelsea and the FA Cup today by Barnsley, as well as being effectively ruled out of the Premier League title race by a string of sub-standard performances against sides we should have been beating, we have to yet again concentrate all efforts on the Champions League, which despite all my optimism, isn’t looking too promising either seeing as we’re up against the Italian Serie A champions Inter Milan in the last 16.

I have a feeling that if we don’t progress against Milan, then Benitez’s job will definitely be on the line – more than it ever has done in the past – and, to be honest, I wouldn’t mind seeing him go. Obviously he brought Liverpool the Champions League in Istanbul in 2005 (a night which I will never forget and get goosebumps every time I watch the replays on YouTube) as well as the FA Cup in 2006 in Cardiff and he has secured one of the most important signings in recent LFC history in Fernando Torres, but something tells me that Benitez’s time has come. Maybe I’m being over-critical, but time will tell I think.

We Will Rock You 2008: A New Dawn?

Over Christmas I boosted my show-count to 51, seeing the new cast that took over in September for the first time – and what a cast. You can read detailed reviews about each show on my WWRY website forum, but the shows on 17th December and 10th January certainly gave me the impression that the show is in extremely capable hands, although I haven’t yet seen Sabrina Aloueche in the Scaramouche role – being treated to Sarah French-Ellis on both visits – but I have it on extremely good authority that she is every bit as good as the rest of the cast. My advice – if you’re anywhere near London and are even remotely interested in musical theatre or rock music (or both), book and see this show – you will not regret it.

To go with the new line-up of cast members, an all-new souvenir brochure has bee released, and I have been lucky enough to have been posted a copy by a friend. All fifty-four scans are online on the WWRY-London.co.uk Gallery, I’ve also been busy making my first new banners and wallpapers for over a year, so please check them out! Also in a completely shameless plug, please also feel free to join up on the WWRY-London.co.uk forums.

OK, maybe this entry has been slightly longer than I originally intended, but with two month’s having passed, I thought it was about time for an update.

Multi-Booting: A Revised Guide – XP/Vista/Linux

Posted on by
4

After writing my last quad-boot entry, I’ve gained quite a bit of experience in troubleshooting and sorting out multi-boot installs so I thought I’d re-write my old quad-boot blog to be slightly more like an informative guide rather than a story, and also downgrading it to just cover tri-booting, as I haven’t gotten around to replacing Apple’s OS X Leopard with Sun’s OpenSolaris yet.

Summary

This guide will use Fedora 8 as the Linux distribution, live CDs are available in both Gnome and KDE flavours for those of you who want to try it before actually touching your hard drive’s partition layout. I would recommend installing from the Fedora Install DVD as from experience, the live CD misses out several packages – make and gcc among others – by default that make life a lot easier after installation.

The other operating systems I’ll be dealing with are Windows XP Media Center Edition and Windows Vista Business.

Step One – Partitioning

My desktop PC’s hard drive layout consists of one 200Gb drive for the operating systems and one 320Gb NTFS drive as a central data area, but I’ll assume that you’re dealing with a single drive. Partition whatever space you have into three sections – four if you want a central data area. Although Windows is a lot more space-hungry than Linux, I gave each system an equal amount of space.

You can use whatever tools are available to you to create your partition layout. Both XP and Vista partitions should be NTFS and it’s best to either leave the Linux partition and format this using the install disc. If you’re creating a central data area to use between operating systems, use the FAT32 file system as while Linux can read/write to NTFS, it’s not proven with large file sizes – I use NTFS for my 320Gb data drive purely because FAT32 cannot cope with that size without splitting into separate partitions – something I wanted to avoid.

Step Two – Installing Fedora

Not much instruction is needed at this point. The Fedora Installation Guide is an excellent tutorial through the process. You can install GRUB – the Linux bootloader – to the drive to get the system installed, booting and stable, although booting to Linux will be configured via a GUI utility in Vista later on.

The general guide for creating a swap partition is to use double the amount of RAM you have in your system. My PC has 2Gb of RAM, so I partitioned off 4Gb and set this to be my swap space.

Step Three – Installing Windows

I’ll deal with both Windows systems in the same step here because they’re both relatively similar. You’ll have to do some partition trickery before installation so that Windows assigns the correct drive letters and installs each version in ignorance of anything else on the system.

You can do this via pretty much any partition manager, but I used GParted as included on the Fedora and Ubuntu LiveCDs – or you can use the dedicated 50Mb GParted LiveCD. Select the XP partition, right-click on it and select Manage Flags. Put a tick next to the “hidden” flag and close the window.

Now simply place the XP installation CD in the drive, fire up the installer and install as normal. Just remember to make sure you’re installing to the right partition on your drive. When you’ve installed and can boot to XP, reboot to GParted and swap the “hidden” flags. You can then install Vista as normal.

Step Four – Pulling It All Together

You should now have three working operating systems on your hard drive, but you can only boot to Vista. Head over and download the free EasyBCD program from NeoSmart and install it on Vista. You can then use that to create the necessary boot menu options for each system. EasyBCD includes GRUB and installs it on your Vista partition, so you don’t need GRUB on the Linux partition. Clever eh?

Done!

After you’ve created the boot menu options, reboot into each system once to test. You should now have a working tri-boot system – congratulations! If anything goes wrong or if you’ve got any questions, by all means e-mail me and I’ll do my best to help. If you’re using Fedora as your Linux distribution, head over to the Fedora forums and register.

Hopefully this is more of an informative guide rather than a narrative guide and you’ve found it helpful. If you have (or even if you haven’t), feel free to leave a comment!

Cheers
Craig

Switching to Linux from Windows – Tips from experience

Posted on by
4

In the last few months, I have made the pretty massive jump from being completely Windows-based to being heavily reliant on Linux – the free, open-source alternative operating system. A vast amount of the help and tips I found while in the transition period were found on message forums and blogs, so maybe this will help prospective Linux users decide to switch.

If it ain’t broke, don’t fix it
Don’t switch to Linux “just because”. I had a pretty solid reason to start exploring, but if Windows works for you, then don’t feel pressurised to try Linux. It won’t change your world. It may make your computing experience slightly less stressful, but like 99% of software systems, Linux has bugs – and many of them.

If you’re curious and would like to try it out, get your hands on a LiveCD – there are various Linux distros that can be run from a CD so you can try them before you install, the most popular of which is Ubuntu.

Get used to the command line
Linux has indeed moved on dramatically in recent years to make itself presentable to the (I hate to use the word) “average” user – the person who just wants to surf the web and write letters. That said, while Windows users can often sit in the blissful ignorance that there is such a thing as a command line environment, it’s still very much present in most Linux installations.

Over the six months or so I’ve been running Linux on my laptop, I could not have lived without the command line. Those who have little or no experience with it will either learn or sink. The flip-side of this is that once you have learned a couple of commands, you’ll feel a lot more confident and eventually you will build up experience.

Google is your friend
I also can’t count the number of times I’ve turned to Google with a specific problem and it’s been solved in a matter of seconds with a simple config file edit or command-line instruction. Especially with well-known distros like Ubuntu, support is very easy to find.

Ubuntu is not the only distro out there
Many users refer to Ubuntu in the same was as they refer to Windows – as a complete system. This is completely untrue. Ubuntu is a fairly new entry into the Linux arena and, some say, has only made it to the big-time because of some serious financial clout from a very generous multi-millionaire.

The key to finding a good distro is to research and experiment. You wouldn’t buy a brand new car on just taking a look through a marque list, so why do the same with a computer OS? Ubuntu, for example, didn’t like my laptop’s Intel GMA950 display card, refusing to give me the native 1280×800 resolution, although it detected my Intel 3945ABG wireless card perfectly. Fedora was the exact opposite, but I decided I could work with no wireless for the time being until I found a solution. If in doubt, check the distro’s Hardware Compatibility Lists.

Don’t expect it to work 100% out-of-the-box…
As with the hardware example above, Linux isn’t a “one-size-fits-all” magic system that many Linux fans make out. Personally, my Fedora installation has taken many hours of tweaking and playing with to get it the way I want it – installing media codecs, finding Linux-equivalent software for various tasks, configuring the OS to run the way I want it to.

…But don’t be afraid to play
I haven’t worked with any other distro, but don’t be afraid to get your hands dirty and mess around. If something isn’t how you like it, there’s more than likely a way to alter it. For example, my desktop (screenshot below) has been heavily modified from the standard GNOME desktop layout – the default desktop environment for Fedora and Ubuntu. I’ve made it look the way I want it to look – like Windows – because that’s what I’m used to and I want to use the software because of the functionality, not the look.

Hopefully I’ve given some useful info that’s helped people to decide if they want to switch. I’m quite happily using both Linux and Windows.

My Software
Linux Distro – Fedora 8
Visual Effects Engine – Compiz-Fusion (used to be Beryl on Fedora Core 7)
Window Decorator – Emerald with Crystal-ICE theme

Music – Amarok
Video – VLC & mPlayer
DVDs – Xine
MSN – aMSN (Alvaro’s Messenger)
Email – Mozilla Thunderbird
Web – Mozilla Firefox
FTP – gFTP (standalone) & FireFTP (Firefox extension)
Office suite – OpenOffice
BitTorrent – Azureus
Graphics – Adobe Fireworks CS3 (running under Wine)
Text Editor – Gedit