Automatic invitation processing

Note: while the feature described in this post lives on, the technical details in this post no longer describe how it works.

---

Today we’re pleased to announce the launch of a much requested new feature at FastMail: automatically updating your calendar with incoming invitations and responses (also known as iMIP messages).

By default, we now do the following:

• If a new invitation arrives from one of your contacts, add it to your default calendar and deliver the invitation to your inbox.
• If an update arrives to any event already in your calendar, update that event with the changes in the email and deliver the email to your inbox.

In the FastMail settings you can now configure to automatically file these emails rather than leaving them in your Inbox if you prefer, or turn off automatically updating your calendar. You can read about the new configuration options in our help documentation.

If you access your FastMail calendar via a CalDAV client, for example Apple’s iPhone or Mac Calendar app, new invitations will now appear in the client and you will be able to RSVP directly, without having to log in to your email.

We have also made all messages related to the same event be included in a single mail conversation in the mail interface, to make it easier to manage.

Technical details

For those who want to know more about how this is implemented, read on!

There are three components to the handling of iMIP request emails at FastMail.

Stage 1: Detection

Detection is run in the lmtpprox mail filter which runs between accepting email and delivering it to your mail store. This is the stage where we can modify the email. If there is an attached calendar event (content type of “text/calendar” or “application/ics”, or a filename ending with “.ics”) then we extract the calendar event and parse it to find the METHOD and UID fields.

If there is no METHOD then it’s a calendar event, but not an iMIP invitation, so we ignore it. If there is both a METHOD and a UID, we look up the UID in your calendars.

Regardless of whether it exists, we add two headers: X-ME-Cal-Method (with the method as the value, converted to lower case) and X-ME-Cal-UID (with the UID as the value). If the event was found, we add a third header X-ME-Cal-Exists with the calendarId of the calendar in which the UID already exists.

We also add an X-ME-Message-Id derived from the UID, which is used to link the messages together into a single conversation in the mail interface.

If there are any errors parsing the attachment, then we continue and deliver it as if there were no attached event.

Stage 2: Filtering

Filtering is run in your sieve script on the IMAP server. If you go to Settings => Rules => Edit custom sieve code, you can see the specific rules calculated from the Calendar Preferences values in the generated part of your sieve script.

We have extended the notify sieve extension to add two new targets, addcal and updatecal, which work as you might guess from the names. The addcal target also takes an :options key value list, for which the option calendarId is defined - and takes the ID of a calendar to store the new event into. If not provided, the default calendar is used.

Stage 3: Updating the calendar

Our notify pipeline goes through a non-blocking server which does immediate push notifications, and then to a background daemon which handles long running tasks. The notifyd daemon does the heavy lifting, parsing the email again (using a library with the same code used earlier by lmtpprox) and checking again for the existing record in your calendar.

The code for adding or updating from iMIP was extracted from the web interface and generalised as well, so the notifyd does exactly what would have happened if you had pressed Add to Calendar or Update Calendar in the web interface.

RSVPs

We do not automatically send an RSVP to invitations you receive. You can accept or decline invitations in our web interface, from either the invitation email or the event in your calendar. Alternatively, if you are using a CalDAV client you can let the organiser know whether you can attend directly from the client. Either way, our server will send out iMIP emails in response to the organiser of the event.

Blogging from the CalConnect conference in Hong Kong.