« MapKit Framework for … | Home | MBS Xojo Plugins, ver… »

CURL change for email download via IMAP


Last year there was a little change in the CURL library for handling email downloads via IMAP. We used to download the list of email, get the index in the email box and download the email by passing the index with UID parameter in the URL. The first email in the mailbox is 1. If you delete email with ID 1, than the second email moves down to become the new first email.

Now since CURL version 7.62.0 from October, the default fetch is using the IMAP UID command. That means the IMAP server gives every email in the mailbox an unique ID which counts up for every new email coming in. The UID does not change if emails are moved or deleted, so we can address emails on the server much better. Now the new parameter is named UID and the old UID parameter is renamed MAILINDEX.

As you may guess, the old and new UID are not the name values, so existing applications with newer plugin fail to load emails. You can fix your script by changing UID to MAILINDEX. MBS FileMaker Plugin in version 9.0 for FileMaker and MBS Xojo CURL Plugin version 19.0 for Xojo have a mitigation to switch to MAILINDEX if the query via UID failed.

For newer projects we recommend to do the lookup of the emails in the mailbox with UID FETCH and get the emails with unique IDs. Next we recommend to use BODY.PEEK instead of just BODY to avoid emails being marked as read automatically. When switching to BODY.PEEK you can remove workarounds which removed the seen flag. And you can include options like INTERNALDATE for the date of the email and/or RFC822.SIZE for the email size:

UID FETCH 1:* (FLAGS INTERNALDATE RFC822.SIZE BODY.PEEK[HEADER.FIELDS (Message-Id DATE FROM SUBJECT TO SENDER REPLY-TO CC BCC)])

As you see we include a few headers fields in this sample query.
The result looks like this:

* 1 FETCH (UID 103 FLAGS (\Seen $junk) INTERNALDATE "06-Apr-2018 19:42:11 +0200" RFC822.SIZE 2091 BODY[HEADER.FIELDS (MESSAGE-ID DATE FROM SUBJECT TO SENDER REPLY-TO CC BCC)] {214}
From: Christian Schmitz
Subject: Hello World
Message-Id: <6F3B718F-8427-4EFB-B291-106A8ABFC1AA@monkeybreadsoftware.de>
Date: Fri, 6 Apr 2018 19:42:05 +0200
To: test@macsw.de

)

Now you can parse that easily. But for our FileMaker plugin we made a new function CURL.GetResultAsEMailList to parse the list and return as JSON. The JSON looks like this:

[{
    "Index":   1,
    "UID":   103,
    "Size":   2091,
    "InternalDate":   "06-Apr-2018 19:42:11 +0200",
    "URL":   "imap://imap.macsw.de/INBOX;UID=103",
    "Flags":   ["Seen"],
    "Subject":   "Hello World",
    "Message-Id":   "<6F3B718F-8427-4EFB-B291-106A8ABFC1AA@monkeybreadsoftware.de>",
    "Addresses":   [{
        "Type":   "from",
        "Email":   "testing@monkeybreadsoftware.de",
        "Name":   "Christian Schmitz"
      }, {
        "Type":   "to",
        "Email":   "test@macsw.de",
        "Name":   ""
      }],
    "Date":   "06.04.2018 09:42:05"
  }]

This JSON contains all the parsed values. We convert the date to the timestamp for FileMaker into local time zone. The subject lines and names are decoded if needed. For each email we include the URL to the email for easier download. Please try the new functions for UID fetch and let us know how you like them.
08 01 19 - 17:04