Re: fsync() takes about 0.06 second ---makes Cyrus deliver slow

From: Jeremy Rumpf (no email)
Date: Tue Sep 10 2002 - 12:19:06 EDT


Hmmm, this is an interesting thread.

Using ext3 or another journaling system might be a good idea if your mail
store size is large, ie +500G. In the event of a crash, fsck time would be
tremendous.

One cool thing about ext3 is that you have good control over its journaling
semantics. I'm assuming your using the default mode of "ordered", which is a
good all around choice for maintaining metadata and data consistency.

When using ext3 in its default "ordered" mode, metadata is logged, while data
is synced before the metadata is flushed. This may be a little redundant
since cyrus is exclusively flushing data with fsync (I'm outta my league a
bit here).

I would be really curious for you to run your tests with ext3 running in
"writeback" journaling mode. You can change modes by modifying your fstab and
remounting the file system:

LABEL=/var /var ext3 defaults 1 2

to

LABEL=/var /var ext3 defaults,data=writeback 1 2

Some good articles on understanding ext3 are:

http://www.linuxplanet.com/linuxplanet/reports/4136/1/
http://www-106.ibm.com/developerworks/linux/library/l-fs7/

Also, you previously stated that your using a RAID subsystem to store data on.
Ask your system admin, or perhaps you are the admin :), if the cache on the
controller is set for "writeback" mode vs "writethrough" mode. Having a
battery backed cache on the RAID controller + writeback operation could speed
up the fsync calls since data will be synchronously flushed to the RAID
controller's cache vs physically being flushed to the disks. This would only
be possible when running the controller's cache in writeback mode.

Some reading on this is at:

http://www.storageadmin.com/Articles/Index.cfm?ArticleID=5398

Hope this helps,
Jeremy

On Tuesday 10 September 2002 10:31 am, Su Li wrote:
> Daniel,
>
> My Linux is red hat 7.2-1 with kernel 2.4.7-10. The file system is ext3.
> Hard disk it RAID5.
>
> Jure,
> Thanks for your configuration. I 'll give a try and post my result to the
> new group.
>
> David,
> I tried remove suppression from /etc/cyrus.conf. The suppression is still
> there. Maybe I should try remove it from the code.
>
>
> -----
> Besides, I did some calculation on the time of the system calls on a
> new-email. 3 fcync takes about 0.15 seconds. Each mail overall takes 0.65
> seconds. I see the number of locks and deliver process can go up to
> thousands. Maybe some thing else is also taking big time, say the
> communication between deliver and lmtpd?
>
> Anyway, I'll keep trying. Thanks for every body's help. If anybody got some
> idea, please let me know.
>
>
> Su
>
>








Hosted Email Solutions

Invaluement Anti-Spam DNSBLs



Powered By FreeBSD   Powered By FreeBSD