Office 365 “A local loop was detected”

Yesterday I encountered a problem with an Office 365 hybrid environment where mail suddenly began looping back and forth between the on premise environment and office 365 for all remote mail users. No changes had been made to the environment.

Mail was transferred successfully to Office 365 using the correct connector, but office 365 was then passing the mail back to on premise. This resulted in a mail loop and users sending e-mail to office 365 accounts receiving an NDR with the following:

servername.local #<servername.local #5.4.6 smtp;554 5.4.6 Hop count exceeded - possible mail loop>

Following a support call with Microsoft lasting around 4 hours, it turns out an internal change has been made to the way Microsoft deal with wildcard certificates. By changing the Office 365 inbound connector to use the SubjectAlternativeName of the wildcard certificate rather than the subject, our issue was resolved:

Before

PS C:\> Get-InboundConnector "Inbound" | fl Id,Tls*
Id                       : Inbound 2
TlsSenderCertificateName : <I>CN=COMODO RSA Organization Validation Secure Server CA, O=COMODO CA Limited, L=Salford,
S=Greater Manchester, C=GB<S>CN=*.domain.co.uk, OU=PremiumSSL Wildcard,
O=Organisation, STREET=Road Name, L=Location,
S=County, PostalCode=Postal Code, C=GB

After

PS C:\> Get-InboundConnector "Inbound" | fl Id,Tls*
Id                       : Inbound 2
TlsSenderCertificateName : *.domain.co.uk

The subject of the certificate had been automatically used by the hybrid configuration wizard and been working for at least the past three months.

Updated 3rd November

Microsoft have now provided the following update, though no such incident appears in the Office 365 portal (for me at least).

Current Status: Engineers have confirmed with some customers that the workaround resolves the issue. Currently, engineers are developing and testing a long-term fix for the code defect, which is expected to take an extended period of time to complete. User Impact: Users with mailboxes hosted on-premises are receiving an error message when attempting to send email to Office 365-hosted users. As a workaround, administrators can enable IP-based inbound on-premises connectors in Office 365 to successfully send email. Customer Impact: Your organization is affected by this event. Impact is specific to a subset of your users. Engineers have received a few isolated customer reports of this issue. Incident Start Time: Monday, November 2, 2015, at 8:53 AM UTC Preliminary Root Cause: A code defect caused an issue with a certificate-based connector. Next Update by: Wednesday, November 4, 2015, at 8:00 PM UTC

OpenSource PHP Class for working with Hikvision cameras

Earlier today I released an open-source (GPL 2) PHP class for interacting with Hikvision IP Cameras, you can find the code on GitHub.

The PHP Class works with Hikvisions IP Camera index files which get created whenever video footage is stored on SD, SMB/CIFS or NFS. It’s needed since Hikvision stores multiple recordings (known as segments) inside a number of MP4 (H.264 – MPEG4-AVC) files.

This will allow you to read the index file (usually named “index00.bin”) and extract information about the files that have been created along with the recording segments they contain. You can then extract the video and thumbnails you’re interested in.

A sample application is included that allows you to search recordings and extract footage.

Getting started with DN42

A week or two ago I became aware of DN42, a private network run to teach people how to use BGP. DN42 users connect to each other using site-site VPNs and then use BGP to exchange routing information. As someone who learns best from hands-on activity I simply couldn’t resist.

This blog post will discuss getting connected to the DN42 network using a Cisco router, be it physical or in a virtualisation solution such as GNS3/VIRL. At a high level there are three main steps:

  1. Create a number of “objects” in order to allocate a network address that you advertise in BGP;
  2. Configure your router so it can access the internet;
  3. Locate a suitable network to establish a VPN with and then form a BGP adjacency;

I’ll try and cover off the various DN42 specifics, but do not plan on covering basic router configuration tasks.
Continue reading Getting started with DN42