[LeapList] Re: Backing up entire disk <- dd, tar, mkisofs, cpio (afio), etc...

[Bryan J. Smith]

Jesse Goerz wrote:
> I have a server with a 20 GB hard drive and was considering
> backing up my workstation (two drives, 4GB and 1.5GB) onto the
> server.  What I would like to do is set up a "mirror" of my
> workstation on the server so that it looks just like my
> workstation but is updated manually (i.e.  when I want it to so
> I can get back to a known state).  What are some of the tools
> for this and what's the best approach?

UNIX has a "universal tool" called "dd".  It can be used to make a
file to/from any device (remember, in UNIX, devices are just special
files).  This includes floppies, CDs and other removable disks as
well as fixed disks.  dd's basic syntax is:

   dd if=(input file/device) of=(output file/device) [bs=(block
size)] [count=(number of blocks)]

Want to make an ISO9660 image of our CD?
  dd if=/dev/cdrom of=/mycd.iso

Floppy?
  dd if=/dev/fd0 of=/myfloppy.img

For the most part, you can ignore the optional "of" and "bs" options
(and there are another dozen or so options you'll rarely use).  For
example, if I wanted to make a replica of my C: drive (that is
/dev/hda1 in Linux) on a server drive I had mounted:

  dd if=/dev/hda1 of=/server_mount/hda1.dd

Or whole hard drive:

  dd if=/dev/hda of=/server_mount/hda.dd

[ * Note: again, /server_mount is a NFS or SMB mount on a server ]

If I want to compress the output as it is written, you can actually
pipe it to GZip (or BZip2) by removing the "of" parameter (since it
is optional -- if removed, "stdout"** is used):

  dd if=/dev/hda1 | gzip > /server_mount/hda1.dd.gz
  dd if=/dev/hda1 | bzip2 > /server_mount/hda1.dd.bz2

[ ** Side discussion:  stdout, stdin and stderr are reserved file
handles in UNIX (and MS-DOS 2.x+ which adopted UNIX-like file
handling, although quite limited in their implementation).  The are
used to redirect normal input, output and error messages.  E.g., if
you type "ls" the command assumes you want to display on "stdout". 
You could then do "ls > filename.txt" ( > = redirect output from
stdout to file/device) to write to a file.  Now if you want to sort
the output, you could use stdin with: "sort < filename.txt >
filename_sorted.txt" ( < = redirect file/device as input to stdin). 
We could also use a "pipe" to filter stdout through a program with: 
"ls | sort > filename.txt" ( | = pipe to) which first sorts output,
then redirects to a file.  So, by default, dd uses stdin/stdout and
you can override with filenames by using the "if/of" parameters. 
More examples follow. ]

Now when you want to reverse the process (i.e. restore), you simply
reverse the parameters:

  dd if=/server_mount/hda1.dd of=/dev/hda1
  dd if=/server_mount/hda.dd of=/dev/hda

Likewise, if you compressed:

  gzip -cd /server_mount/hda1.dd.gz | dd of=/dev/hda1
  gzip -cd /server_mount/hda.dd.gz | dd of=/dev/hda

[ *** "gzip -cd" can be replaced with "gunzip -c".  The "d" option
tells gzip to decompress, same as the "gunzip" name.  Since gzip
(and bzip2) defauls to useing the filname-.gz (-.bz2 for bzip2) by
default, you must use "-c" to send output to stdout, which can be
piped to "dd" in our example). ]

> I've seen some stuff hinting that rsync can do this but I'm
> not quite sure how to go about it.  Has anyone done it?

Rsync is another way.  I'm assuming you want to backup a Windows
drive though?

> Would it be easier just to tar the entire workstation onto
> the server?

Yes and no.  "dd" is powerful, but you can run into trouble if the
hard drive you restore to is NOT the same size/geometry when you
created the image.  "tar" is more flexible and dynamic.  Tar also
have excellent data integrity checking (with a _real_ tar program
and NOT simplistic implementations like WinZip) and can become
corrupted and still let you access the data past the corrupted
portion (quite UNlike PKZip).  Of course if you compress the tarball
(.tar file) with GZip or BZip2, you remove that "robustness" of Tar.

If you are worried about corruption, but still want to compress, you
should compress the files _before_ you put them into the tarball.  I
have a backup script that creates an ISO9660+Joliet/RockRidge CD
file (ready-to-burn) that will let you fit ~1-2GB of data on a CD. 
You can get it from here:
  
http://lists.leap-cf.org/pipermail/leaplist/2000-December/008604.html
http://lists.leap-cf.org/pipermail/leaplist/2000-December/008611.html

[ The first post is version 1.2 and the second is a diff (i.e.
"difference file" to patch to 1.4 (instructions included). ]

Another backup archive format is "cpio".  "cpio" is also very
recoverable like Tar, until you compressed it.  I bring up cpio
because there is a backup program known as "afio" that compresses
files before putting in a cpio archive.  You can use it to create
backup files on disk/server, tape, etc...  I highly recommend you
check out "afio" if you're interested in long-term backups.

> I'm using Debian and currently using dpkg --get-selections
> saved on a file on the server but this requires a reinstall
> of the base system before I reset all the packages with dpkg
> --set-selections.

Huh?  I'm not a Debian expert, but I know many Debian users who run
for years without "reinstalling".  Are you sure about this?

(Not to mention quite a bit of us RedHat users)

> reconfiguring after that, can this be avoided by backing
> up /etc?)  Anyone using Debian got a better way?

Backing up /etc is always good, let alone the whole system.  Again,
I don't think you need to re-install here.

-- TheBS

-- 
Bryan "TheBS" Smith, Engineer                  CONTACT INFO
***********************************************************
 Chat: thebs413 @ AOL/MSN/Yahoo (see http://Everybuddy.com)
Email: mailto:thebs@smithconcepts.com,thebs@theseus.com