Office 365 “A local loop was detected”

Yesterday I encountered a problem with an Office 365 hybrid environment where mail suddenly began looping back and forth between the on premise environment and office 365 for all remote mail users. No changes had been made to the environment.

Mail was transferred successfully to Office 365 using the correct connector, but office 365 was then passing the mail back to on premise. This resulted in a mail loop and users sending e-mail to office 365 accounts receiving an NDR with the following:

servername.local #<servername.local #5.4.6 smtp;554 5.4.6 Hop count exceeded - possible mail loop>

Following a support call with Microsoft lasting around 4 hours, it turns out an internal change has been made to the way Microsoft deal with wildcard certificates. By changing the Office 365 inbound connector to use the SubjectAlternativeName of the wildcard certificate rather than the subject, our issue was resolved:

Before

PS C:\> Get-InboundConnector "Inbound" | fl Id,Tls*
Id                       : Inbound 2
TlsSenderCertificateName : <I>CN=COMODO RSA Organization Validation Secure Server CA, O=COMODO CA Limited, L=Salford,
S=Greater Manchester, C=GB<S>CN=*.domain.co.uk, OU=PremiumSSL Wildcard,
O=Organisation, STREET=Road Name, L=Location,
S=County, PostalCode=Postal Code, C=GB

After

PS C:\> Get-InboundConnector "Inbound" | fl Id,Tls*
Id                       : Inbound 2
TlsSenderCertificateName : *.domain.co.uk

The subject of the certificate had been automatically used by the hybrid configuration wizard and been working for at least the past three months.

Updated 3rd November

Microsoft have now provided the following update, though no such incident appears in the Office 365 portal (for me at least).

Current Status: Engineers have confirmed with some customers that the workaround resolves the issue. Currently, engineers are developing and testing a long-term fix for the code defect, which is expected to take an extended period of time to complete. User Impact: Users with mailboxes hosted on-premises are receiving an error message when attempting to send email to Office 365-hosted users. As a workaround, administrators can enable IP-based inbound on-premises connectors in Office 365 to successfully send email. Customer Impact: Your organization is affected by this event. Impact is specific to a subset of your users. Engineers have received a few isolated customer reports of this issue. Incident Start Time: Monday, November 2, 2015, at 8:53 AM UTC Preliminary Root Cause: A code defect caused an issue with a certificate-based connector. Next Update by: Wednesday, November 4, 2015, at 8:00 PM UTC

Log-off terminal services session remotely

There are times when you want to quickly log-off a number of terminal services sessions without having to log on to the server itself, perhaps because of the following error:

The terminal server has exceeded the maximum number of allowed connections.

Microsoft provide two useful command-line tools to view and terminate sessions, qwinsta (Query WINdoes STAtion) and rwinsta (Reset WINdows STAtion).
Continue reading Log-off terminal services session remotely

Outlook error 0x8004011d during Send/receive

When using Microsoft Outlook (2007 or 2010) to automatically initiate a dial-up connection you may receive error 0x8004011d during the Send/Receive process. This problem occurs under Windows 7 when all network adapters are disconnected. Windows XP and earlier are not affected.

Outlook Connection Manager, which controls the outlook state (Disconnected, Connected or Offline), believes there is no connectivity to the Exchange server even once a dial-up connection is made. Eventually error 0x8004011d is reported.

The workaround is to ensure there is another network adapter connected, even if it has no connectivity to Exchange or other networks. This could be a Windows loopback adapter or an RJ45 loopback jack plugged into a network adapter. Alternatively manually initiating the dial-up connection before launching Microsoft Outlook will bypass the problem.

This bug has been acknowledged by Microsoft but given that it only effects dial-up connections and it’s not a security or data-loss bug it has been decided that it won’t be fixed. Outlook 2013 removed support for automatically initiating a dual-up connection so this error no longer applies there.