From: Andrzej Kukuła (no email)
Date: Thu Jul 14 2005 - 05:38:52 EDT
On 7/14/05, Michael Nguyen <> wrote:
> From: "cv" <>
>
> Hi Curtis.
>
> > I was just wondering how long is long and why that matters (unless we're
> > talking about half a day). I also use an rsync script that I have
> > crontab-bed to run after midnight. It may take a couple of hours to back
> up
> > each mail server, but then, what do I care? I'm asleep.
>
> We're talking more than half a day. What seems to take the longest is
> calculating the file lists as there are just tons and tons and tons of tiny
> files that are often moved.
> The backup of the mail servers started at 00:18 AM and it's currently 15:32
> PM here. The mail servers are about 70% done. Size-wise, we're talking
> about 3 servers totalling less than 500GB. The actual size doesn't seem to
> be the problem, but rather the time it takes to create the list of files to
> be transferred.
>
> We have servers that have much more data, but take much less time to backup.
Disable atime updates for the filesystem -- Maildirs are designed as
so they don't need atime to work. This will save much time when
scanning.
If you use rsync, disable any checksumming features, partial updates
and compression. DO NOT run rsync over SSH as this involves crypto
calculations what slows down the entire thing. Run unencrypted rsync
--daemon instead.
If you use some commercial backup software, disable its size
estimation features, checksumming, compression and transmission
security.
At the lower level, maybe change filesystem to one that performs
better with many small files.
Regards,
Andrzej Kukula
|
|
|