Next message: Martin Konold: "Re: Coverity scans of Cyrus SASL and IMAPd"
Grrr, bad cut-and-paste. Correct patches attached.
Quoting John Capo ():
> These patches handle the message deleted case and the folder rename
> case properly. Should handle all cases of messages and folders
> disappearing while working on a folder. May handle deleting a user
> while working on that user.
>
> John Capo
>
>
>
> Quoting John Capo ():
> > Pages in the middle of the night the last few days prompted me to
> > look into this long standing problem.
> >
> > upload_message_work() returns IMAP_IOERROR when it can't map the
> > message being uploaded due to a race between reading the index and
> > actually doing the upload work. upload_message_work() returns with
> > these items in the stream to the server.
> >
> > PARSED msgid uid internaldate sent-date last-updated modseq flags
> >
> > The server is expecting the header_size to arrive next but most
> > anything can arrive depending on what client work is left to do.
> >
> > Partially sanitized and shortened log entries for a case where a
> > folder was renamed with other operations on the same mailbox and
> > other mailboxes in the work queue.
> >
> > 03:25:01 sync_client: DELSUB gersham_domain1^com user.gersham_domain1^com.mova
> > 03:25:01 sync_client: MAILBOX user.gersham_domain1^com
> > 03:25:01 sync_client: MAILBOX user.gersham_domain1^com.mova
> > 03:25:01 sync_client: MAILBOX user.gersham_domain1^com.something.mova
> > 03:25:01 sync_client: MAILBOX user.1040001_1051001-domain2^com
> > 03:25:01 sync_client: MAILBOX user.gersham_domain1^com.something
> > 03:25:01 sync_client: MAILBOX user.gersham_domain1^com.Trash
> > 03:25:01 sync_client: MAILBOX user.rismi_domain3^co^id
> > 03:25:01 imap: Rename: gersham_domain1^com something/mova => mova
> > 03:25:01 sync_client: IOERROR: opening message file 594 of user.gersham_domain1^com.something.mova: No such file or directory
> > 03:25:01 sync_client: Promoting: MAILBOX user.gersham_domain1^com.mova -> USER gersham_domain1^com
> > 03:25:01 imap: Deleted mailbox user.gersham_domain1^com.something.mova
> > 03:25:03 sync_client: MAILBOX user.gersham_domain1^com.something.mova
> > 03:25:03 sync_client: MAILBOX user.1040001_1051001-domain2^com
> > 03:25:03 sync_client: MAILBOX user.gersham_domain1^com.something
> > 03:25:03 sync_client: MAILBOX user.gersham_domain1^com.Trash
> > 03:25:03 sync_client: MAILBOX user.rismi_domain3^co^id
> > 03:25:03 sync_client: MAILBOXES received BAD response: Syntax error in Append at item 3: Invalid flags
> > 03:25:03 sync_client: Promoting: MAILBOX user.gersham_domain1^com.something.mova -> USER gersham_domain1^com
> > 03:25:07 sync_client: MAILBOX user.1040001_1051001-domain2^com
> > 03:25:07 sync_client: MAILBOX user.gersham_domain1^com.something
> > 03:25:07 sync_client: MAILBOX user.gersham_domain1^com.Trash
> > 03:25:07 sync_client: MAILBOX user.rismi_domain3^co^id
> > 03:25:07 sync_client: USER gersham_domain1^com
> > 03:25:07 sync_client: UPLOAD received BAD response: Syntax error in Append at item 3: Unknown Reserved message
> > 03:25:09 sync_client: USER gersham_domain1^com
> > 03:25:09 sync_client: UPLOAD received BAD response: Syntax error in Append at item 3: Unknown Reserved message
> > 03:25:13 sync_client: USER gersham_domain1^com
> > 03:25:13 sync_client: UPLOAD received BAD response: Syntax error in Append at item 1: Unknown Reserved message
> > 03:25:13 sync_client: Error in do_sync(): bailing out!
> >
> > The failure message depends on how much work is in the queue.
> >
> > sync_client: MAILBOXES: Invalid unsolicited response type 1 from server: (null)
> > sync_client: Discarding: 104056 010400463ca07b23db000000 ()
> >
> > A few to many thousands of those depending on the mailbox size.
> >
> > sync_client: MAILBOXES received BAD response: Syntax error in Append at item 3: Invalid flags
> >
> > Locking the mailbox would fix it the race but it may stay locked
> > for longer that one would want.
> >
> > I tinkered with sending a token to the server signaling it to cleanup
> > the message_list and the upload_list and wait for a new command but
> > doing that right is rather involved.
> >
> > A really ugly hack is to and fake up a message and send that to
> > keep the stream in sync. The fake message is deleted from the
> > replica on the next MAILBOX command.
> >
> > if (r) {
> > char *message = "Message vanished from primary, probably deleted\r\n";
> >
> > syslog(LOG_ERR, "IOERROR: opening message file %lu of %s: %m",
> > record->uid, mailbox->name);
> >
> > prot_printf(toserver, " 0 0 0 {0+}\r\n");
> > prot_printf(toserver, "{%lu+}\r\n", strlen(message));
> > prot_write(toserver, message, strlen(message));
> > mailbox_unmap_message(mailbox, record->uid, &msg_base, &msg_size);
> > sequence++; /* This variable is not used */
> > return (0);
> > }
> >
> > Monitoring and restarting the sync_client always works but that's
> > ugly too.
> >
> > John Capo
> >
> >
> >
> diff -ru cyrus-imapd-2.3.7/imap/sync_client.c cyrus-imapd-2.3.7-abort-upload/imap/sync_client.c
> --- cyrus-imapd-2.3.7/imap/sync_client.c Sun May 13 11:56:58 2007
> +++ cyrus-imapd-2.3.7-abort-upload/imap/sync_client.c Sun May 13 12:06:27 2007
> @@ -1179,6 +1179,9 @@
> if (r) {
> syslog(LOG_ERR, "IOERROR: opening message file %lu of %s: %m",
> record->uid, mailbox->name);
> + prot_printf(toserver, " %lu ", 0xdeadbeef);
> + prot_flush(toserver);
> + sync_msgid_remove(msgid_onserver, &record->uuid);
> return(IMAP_IOERROR);
> }
>
> diff -ru cyrus-imapd-2.3.7/imap/sync_server.c cyrus-imapd-2.3.7-abort-upload/imap/sync_server.c
> --- cyrus-imapd-2.3.7/imap/sync_server.c Sun May 13 11:56:58 2007
> +++ cyrus-imapd-2.3.7-abort-upload/imap/sync_server.c Sun May 13 12:05:41 2007
> @@ -1780,6 +1780,8 @@
> upload_list = sync_upload_list_create(new_last_uid, mailbox->flagname);
>
> do {
> + unsigned long hdr_size;
> +
> err = NULL;
> item = sync_upload_list_add(upload_list);
>
> @@ -1870,19 +1872,28 @@
> goto parse_err;
> }
>
> - message = sync_message_add(message_list, &item->uuid);
> -
> /* Parse Message (header size, content lines, cache, message body */
> if ((c = getastring(sync_in, sync_out, &arg)) != ' ') {
> err = "Invalid Header Size";
> goto parse_err;
> }
> - message->hdr_size = sync_atoul(arg.s);
>
> if ((c = getastring(sync_in, sync_out, &arg)) != ' ') {
> err = "Invalid Content Lines";
> goto parse_err;
> }
> +
> + if ((hdr_size = sync_atoul(arg.s)) == 0xdeadbeef)
> + {
> + /* Make sure cache data flushed to disk before we commit */
> + sync_message_fsync(message_list);
> + sync_message_list_cache_flush(message_list);
> + sync_upload_list_free(&upload_list);
> + syslog(LOG_ERR, "IOERROR: UPLOAD ABORT");
> + return;
> + }
> + message = sync_message_add(message_list, &item->uuid);
> + message->hdr_size = sync_atoul(arg.s);
> message->content_lines = sync_atoul(arg.s);
>
> if ((r=sync_getcache(sync_in, sync_out, message_list, message)) != 0) {
> diff -ru cyrus-imapd-2.3.7/imap/sync_support.c cyrus-imapd-2.3.7-abort-upload/imap/sync_support.c
> --- cyrus-imapd-2.3.7/imap/sync_support.c Sun May 13 11:56:59 2007
> +++ cyrus-imapd-2.3.7-abort-upload/imap/sync_support.c Sun May 13 12:03:25 2007
> @@ -544,6 +544,25 @@
> return(NULL);
> }
>
> +struct sync_msgid *sync_msgid_remove(struct sync_msgid_list *l,
> + struct message_uuid *uuid)
> +{
> + int offset = message_uuid_hash(uuid, l->hash_size);
> + struct sync_msgid *msgid;
> +
> + if (message_uuid_isnull(uuid))
> + return(NULL);
> +
> + for (msgid = l->hash[offset] ; msgid ; msgid = msgid->hash_next) {
> + if (message_uuid_compare(&msgid->uuid, uuid))
> + {
> + msgid->uuid.value[0] = 0;
> + return;
> + }
> + }
> + return(NULL);
> +}
> +
> /* ====================================================================== */
>
> struct sync_folder_list *sync_folder_list_create(void)
> diff -ru cyrus-imapd-2.3.7/imap/sync_support.h cyrus-imapd-2.3.7-abort-upload/imap/sync_support.h
> --- cyrus-imapd-2.3.7/imap/sync_support.h Sun May 13 11:56:59 2007
> +++ cyrus-imapd-2.3.7-abort-upload/imap/sync_support.h Sun May 13 12:11:21 2007
> @@ -143,6 +143,9 @@
> struct sync_msgid *sync_msgid_add(struct sync_msgid_list *list,
> struct message_uuid *uuid);
>
> +struct sync_msgid *sync_msgid_remove(struct sync_msgid_list *list,
> + struct message_uuid *uuid);
> +
> struct sync_msgid *sync_msgid_lookup(struct sync_msgid_list *list,
> struct message_uuid *uuid);
>
