What not to do when Connecting Azure DevOps to AzureAD
Background
Visual Studio Team Services…. ummm I mean Azure DevOps (…long live VSTS) is a really amazing offer from Microsoft. It is a great place for Developers and Operations people to work together, store code, run CI/CD pipelines, and keep track of projects. It is also free for the first 5 basic users (more if you are a MSDN Enterprise/Professional License), hosts private repositories, allows pipelines in azure, etc. I started using AzDO about a year ago when I was tired of maintaining my own GitLab environment, and our Lead Dev was tired of TFS. We also had both suffered from a lack of documentation and consistent code tracking. We both agreed it would be better if we shared a version control system and not repeated the missteps of our predecessors.
If you haven’t used Azure DevOps I suggest you check it out.
When we started using DevOps, we had not fully migrated to Azure AD as an organization. At the time the only way we could use our MSDN subscription or DevOps was by using a Microsoft account that we personally had setup. Some of us had opted for email accounts we already had (@outlook.com) and others to mimic organization ids (@contoso.com). This worked fine, but as we started on-boarding more Devs, we thought it might be best to pivot to using our Azure AD accounts. This decision was to make the user life-cycle a little easier to manage (…”oh hey X still has access to our code.”), plus we wanted to integrate notifications into some of our MS Teams.
So this week we went to under take the change of adding our Azure DevOps Organization to Azure AD, and it did not go so well. Don’t do what we did..
Methodology
Connect Azure DevOps to an AzureAD Tenant used by my organization.
- Setup a Microsoft account in your DevOps Organization
- Connect the Tenant
- Test Logins
Setup a Microsoft Account
Before I started down this path of connecting these two services I briefly read Microsoft’s Documentation on the process (Connecting & Disconnecting). I would urge you to read both closely before doing this change. Also you will need to alert your users of the change and possibly switch their MSDN license to use their AzureAD account.
The basic outline for the Azure DevOps change as I understood it was:
- Get a Microsoft Account (MSA)
- Make it a Project Collection Administrator*
- Make sure it is not the same name as an existing AzureAD identity
This is going to be easy…
As I said earlier, we were already using MSA accounts in our Azure DevOps organization and my account already had the rights (Step 1 + 2 are done!). The one part I was concerned about was that I had named my MSA the same as my friendly email alias in our AzureAD tenant; however, this did not turn out to be an issue since my UserPrincipalName is different than my Alias so I could add both to the DevOps organization without any issues.
- Looking back, I should have created a new MSA account specifically for this change. It would have made me pay more attention to the article. The MSA account I use already had the proper permissions in Azure DevOps, but it was not a Guest in our Azure AD tenant, which was a part of Step 2. Interestingly, if you use @contoso.com for your MSA it doesn’t allow you to add that user to your AzureAD as a guest. So you would need to use some other external domain for this MSA.
By default all MSAs create their own AzureAD Directory (usually something like .onmicrosoft.com, and once you add them to your directory they receive that as an additional directory the account can use. The one I was using had its own default Directory, but it also had Microsoft’s Directory. I believe the reason this happened is either an interactive lab I did back at Ignite, or when I did a lesson on Microsoft.com/learn for Azure. (They allow you to spin up resources to provide interactive learning)
To figure out which Directory your account has: login to portal.azure.com click on your name in the right corner click “switch directory”
Connect the Tenant
We began our move earlier in the week by switching all of our MSDN users to their AzureAD account. This worked for most users, and they were able to login to MSDN using their AzureAD account (although it took almost a day for a few users) and DevOps using both their MSA and AzureAD accounts. They could use both in DevOps because we had added the new account to DevOps in preparation of the change to make sure they got the same permissions.
The last user who was having trouble kept getting a 401 for their AzureAD account. The error kept saying not found in Directory. We tried a little bit of this and that, but ultimately we decided let’s just flip it on.
If we connect it to our Directory then he will be able to login. What is the worst that could happen…
I logged in with my MSA to Portal.Azure.com and navigated to the Azure DevOps Organization blade and clicked “Connect AAD.” Asked me one more time if I was sure I wanted to do this, and away we went.
It took a few minutes, but then it told me that it had connected the Service to Directory 72f988bf-86f1-41af-91ab-2d7cd011db47….
Wait that isn’t our directory. I connected it to the wrong one!
Test Logins
By this point in the day, after the feeling of despair set in, things weren’t looking so awesome. We immediately found out that the one person we were trying to fix could not login (because it was the wrong Tenant). Neither could anyone else in our DevOps organization. Neither their MSA or AzureAD accounts worked except for the account with which I had bound everything. (We later found out that all our existing PAT and SSH keys worked, even though we could not login. This ended up being a saving grace as our Development team was able to pull down the critical repos they were working on).
We had read the disconnect instructions, and tried to reverse the change, but in order to do this you need to have a Microsoft account as the owner, and it needs to be a Global Admin . Fun fact: I am not (and may never be at this point) a Global Admin in Microsoft’s Tenant. So we sent a support message to Microsoft. It took a few days of tense waiting (mostly because of our support tier), but we eventually had them switch the directory identity to our organization. This resolved everyone’s login issues and got us back to the desired state.
aww snapdizzle. I’m in. What is this magical world!" – An Unnamed Dev after 2 days without DevOps
Conclusion
Read the Manual. Closely. I usually read a lot and plan a lot before making a change like this, but by missing one important detail (MSA must be a guest of the joining Tenant) it caused days of chaos for our devs. Make a new MSA when joining to your Organization. You can always throw it away after the switch is made. Also make sure you’re paying for the right level of support. There were several times where we almost switched to Primer (and we may still do this in the future)