Re: Berkeley DB upgrade - 2nd try

• Previous message: Martin G.H. Minkler: "Re: sieve doesn't work [auf Viren qberpr=FCft=5D?="
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Hi everyone,

Another follow-up to my monologue thread - I've had a few helpful
replies off-list though, and comments (on or off list) are very much
appreciated.

> I've started building cyrus-imapd-2.2.13 - the Cyrus homepage still
> lists 2.2.12 as the 'current' version, but is apparently outdated a bit.
> In another posting to this list, Andrew Morgan hinted that "Cyrus
> v2.2.12 and older will not work with Berkeley DB 4.3+ without a patch".
> With 2.2.13, at least I get a meaningful error out of Berkeley when I
> start Cyrus:
>
> DBERROR db4: Skipping log file /var/imap/db/log.0000000001: historic log
> version 7
> DBERROR db4: /var/imap/db/log.0000000002: log file open failed: No such
> file or directory
> DBERROR db4: PANIC: No such file or directory
>
> Running db_recover from Berkeley4.4.20 gave me essentially the same errors.
>
> In the end, I figured it apparently doesn't care about log.00000001, so
> I moved it aside. This time, it also stopped looking for log.00000002
> and recovered succesfully. And now Cyrus works again!
>
> Before I even want to consider doing this kind of surgery on any of my
> production Cyrus servers, I'd like to know what would have been in
> log.00000001 - what risks do I run when removing it?

The above procedure is not the correct way to go about, I've found.
Cyrus does work in the sense that I can open my existing mail
(mailboxes.db is skiplist), but sending a new message into the system
caused LMTP to complain about the state of the deliver.db, and one cpu
becomes fully loaded with kernel tasks, probably lots of syscalls.

So what would the correct procedure be? I'm upgrading from Berkely
4.1.25 to 4.4.20. I've looked at the Berkeley documentation, and found
this on upgrading:

> Otherwise, if the application has a Berkeley DB transactional
> environment and the databases need upgrading, the application may
> be installed in the field using the following steps:

Given that there are 'log' files in the DB environment, I'm assuming
this is indeed a transactional Berkeley DB. Because the database format
changed between 4.1.25 and 4.4.20, the above case applies. The procedure
then would be:

1. Shut down the old version of the application.
2 Run recovery on the database environment using the DB_ENV->open method
or the db_recover utility.
3. Remove any Berkeley DB environment using the DB_ENV->remove method or
an appropriate system utility.
4. Archive the database environment for catastrophic recovery. See
Archival procedures for more information.
5. Recompile and install the new version of the application.
6. Upgrade the application's databases. See Upgrading databases for more
information.
7. Archive the database for catastrophic recovery again (using different
media than before, of course). Note: This archival is not strictly
necessary. However, if you have to perform catastrophic recovery after
restarting the application, that recovery must be done based on the last
archive you have made. If you make this second archive, you can use it
as the basis of that catastrophic recovery. If you do not make this
second archive, you have to use the archive you made in step 4 as the
basis of your recovery, and you have to do a full upgrade on it before
you can apply log files created after the upgrade to it.
8. Force a checkpoint using the DB_ENV->txn_checkpoint method or the
db_checkpoint utility.
9. Restart the application.

(source: http://www.sleepycat.com/docs/ref/upgrade/process.html)

Quite an involved procedure. Some people have suggested dropping
Berkeley alltogether, but I feel it's worth the effort to actually gain
some understanding of this complex environment, and for the added
performance. So next order of business is to downgrade my test
environment, then re-upgrade it and test the procedure above.

Question: 'Remove any Berkeley DB environment using the DB_ENV->remove
method or an appropriate system utility'. I guess 'rm -rf
/var/imap/db/log* /var/imap/db/_*' would be 'appropriate'? (this way, I
don't delete the skipstamp file).

Regards, Paul Boven.

----
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

• Previous message: Martin G.H. Minkler: "Re: sieve doesn't work [auf Viren qberpr=FCft=5D?="
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]