Totara Learn Open Discussions

Integrating Totara with company email - what can replace SMTP Auth?

 
Ian Wallbridge
Integrating Totara with company email - what can replace SMTP Auth?
door Ian Wallbridge - Tuesday, 13 July 2021, 01:19 AM
 

Hi all,

We run an instance of Totara Learn, and send notification emails through our corporate Outlook system, using what Microsoft call Office 365 Client Submission, or SMTP Auth as it's called in Totara config.

On the whole this has worked well. It gives us a real mailbox that we can check in case of dispute over sent emails, although - given that seminar notifications are send with calendar invites - it would be really good if a straightforward 'decline' would interact with Totara to cancel attendance on the seminar. That's a whole other story however...

What I want to ask everyone about is the recent decision by Microsoft to start phasing out Basic Authentication, which will have an impact on SMTP Auth. I have found an article that suggests this can be upgraded to use OAuth 2.0 (link below) but this seems to relate to user login rather than to platform outgoing emails.

Announcing OAuth 2.0 support for IMAP and SMTP AUTH protocols in Exchange Online - Microsoft Tech Community

My question is: what mechanisms do other organisations use for outgoing emails, specifically with Office 365 / Outlook systems, and - if you currently use SMTP Auth - are you intending to replace this with a different method?

All comments gracefully received!

Thanks,

Ian

   


Craig Eves
Re: Integrating Totara with company email - what can replace SMTP Auth?
door Craig Eves (Totara Support) - Thursday, 15 July 2021, 21:07 PM
Groep Totara

Hi Ian

We are aware of the phasing out of Basic authentication and are evaluating alternatives - I have highlighted this post to the development team looking at this for comment.

Regards



Fabian Derschatta
Re: Integrating Totara with company email - what can replace SMTP Auth?
door Fabian Derschatta - Tuesday, 20 July 2021, 16:51 PM
Groep Totara

Hi Ian,

as Craig mentioned our Product team is looking into this. Ideally we can add support for OAuth2 as a replacement for SMTP AUTH but this is still subject to technical research on our end. 

If you have access to the public tracker you find the related ticket here: https://totara.community/local/publictracker/issuedetail.php?key=TL-31706

Kind regards,
Fabian Derschatta

Totara Learning
FABIAN DERSCHATTA
Engineering Manager 
T: +64 4 385 8399Timezone: New Zealand
totaralearning.com  LinkedInTwitterInstagram

00_Totara_Banner.jpg

Totara Learning Solutions Limited is globally headquartered at Level 2, 186 Willis Street. Te Aro, Wellington 6011, New Zealand. Company number: 3107649.

Simon Coggins
Re: Integrating Totara with company email - what can replace SMTP Auth?
door Simon Coggins - Wednesday, 21 July 2021, 19:02 PM
Groep Totara

Hi Ian,

I have been looking into this issue as we've been seeking to understand the impact of Microsoft's policy change on outgoing mail. While I don't have a definite answer quite yet I'm happy to share what I've found and our general plan to tackle this.

As you mentioned, most communications from Microsoft regarding this change seem to be focused on how individual users connect to the mail server - e.g. when an individual sets up their desktop email client. In that scenario, where the emails are from a specific user OAuth2 makes sense as an authentication mechanism as the user is present to authenticate and share access permissions interactively.

In the case of Totara we are connecting to the mail server as an application rather than as an individual user. The best Microsoft documentation I've been able to find on this topic is this page:

https://docs.microsoft.com/en-us/Exchange/mail-flow-best-practices/how-to-set-up-a-multifunction-device-or-application-to-send-email-using-microsoft-365-or-office-365

which provides some options for configuring such a setup. They describe three ways of setting it up:

1. SMTP AUTH

This sounds like what you are doing from what I can tell. You create a real user account with a generic email address that all system mail comes from. When configuring SMTP to you point to the mail server and provide the user's credentials. Messages are then sent from the specified user (you need to be careful to make sure the system 'from' address matches the account email and that they are both using the right domain).

I'm currently unclear on if this mechanism is going to break when basic authentication is shut down. On one hand it does seem to be the very thing they are describing (SMTP AUTH), but on the other hand there is information from a Microsoft rep here suggesting this specifically won't be affected.

I am planning to do some testing to turn off SMTP AUTH via configuration and see if it prevents this working. However it's hard for me to be sure that will exactly replicate the change Microsoft are planning until they take effect.

If you have access to Microsoft support as part of your Exchange server you could try asking them for a specific response to this question.

2. Direct send

I don't think this option is appropriate for this case as it only supports messaging to internal exchange users (not external users).

3. Configure a connector

This method is a bit more complex to setup but does seem like it has more likelihood of continuing to work after this change comes into effect. Essentially you connect the Totara site to Exchange via a connector which seems to replace basic authentication with an IP address whitelist. So as long as your Totara site is using a fixed IP address your exchange server can accept and send messages without authentication from Totara.

I haven't been able to test this yet to confirm it works as it requires changes to the company DNS records to create a MX record.

Separate from the above there is a fourth method which Microsoft seems to be recommending these days:

4. Send via Graph API

Instructions for this one can be found here:

https://developer.microsoft.com/en-us/outlook/blogs/announcing-oauth-2-0-support-for-imap-smtp-client-protocols-in-exchange-online/

https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth

and they do explicitly state in the first link that this is the best approach for application level configuration.

This method does require using OAuth2 for Authentication, so would need changes to our SMTP configuration and mailer library to support it. However if we were to implement those changes this would seem like it might be the most future proof/recommended approach.

I am going to look into how much work that would be as part of my investigation into this issue, after which it will be a product prioritisation decision as to whether we proceed.


In general it would be really helpful to get more information from any Totara partners who are using Microsoft Exchange, providing more details on how they currently have it configured (is it one of the options above or something else?). That will allow me to make sure we can continue to support those methods as much as possible after these changes take effect. If you'd rather not post details here, please open a support ticket or email me directly at simon.coggins@totaralearning.com.


Simon Coggins
Re: Integrating Totara with company email - what can replace SMTP Auth?
door Simon Coggins - Thursday, 22 July 2021, 18:53 PM
Groep Totara

Further investigation has indicated that Microsoft have backtracked on their plans to disable SMTP AUTH entirely. Their last two updates indicate they will not turn it off if it is in use, and they intend to give at least 12 months notice before they would do that.

04 Feb 2021

https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-february-2021-update/ba-p/2111904

“The first change is that until further notice, we will not be disabling Basic Auth for any protocols that your tenant is using. When we resume this program, we will provide a minimum of twelve months notice before we block the use of Basic Auth on any protocol being used in your tenant.“

16 Jun 2021

https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-june-2021-update/ba-p/2454827

“Up until the point at which we start to disable Basic Auth for protocols which are in-use – we are still planning on doing that and will have news on that later this year)”

 

Therefore any emails received from Microsoft about this should only be for MS tenants that are not currently using this feature, and we don't expect a disruption to Totara email sending in the short term.

We have described the results of our investigation on an improvement ticket TL-31706. We are still putting this change forward as an improvement to be prioritised for the next quarter (which would allow us to support OAuth2 for email sending/receiving), but at this stage don't envisage needing a short-term fix.