For months, I had been fighting a recurring problem with our OKTA Active Directory integration service account. Multiple times per week – sometimes multiple times per day – the account would lock out, breaking OKTA’s ability to synchronize user information from Active Directory. Each lockout meant manual intervention: unlock the account, reset the password in both OKTA and Active Directory, wait for synchronization to resume, then hope it would stay running for more than a few hours. The interruptions were disrupting user provisioning, preventing new hires from getting access on their first day, and generally making our identity management system unreliable.
The standard troubleshooting steps weren’t revealing anything obvious. The account wasn’t being used for interactive logins, which ruled out someone accidentally trying to log in with the wrong password. The lockout logs didn’t show failed authentication attempts from unexpected locations. OKTA support suggested it might be a clock synchronization issue between our Domain Controllers and the OKTA cloud service, but fixing time synchronization didn’t help. I spent hours combing through event logs trying to find the pattern, but lockouts seemed almost random.
Finally, on March 29th, I decided to do a complete audit of the OKTA accounts groups and permissions, for the THIRD TIME. I schedule time with the OKTA support tech and finally that’s when we found it. The OKTA Service account was simultaneously a member of the Domain Admins AND the Service Account OU.
We had NOT FOUND THIS BEFORE because it was hidden in nested groups. In this case the OKTA Service Account was nested 3 levels deep as a child of the Domain Admin group. This caused a conflict with the inherent limitations of The Service Account group in Windows that is designed intentionally to limit access.
This was NOT even caught byour scannung tools such as CrowdStrike and Tennable. When I noticed that I notified the Security Team so they could keep a close eye on things moving forward and hopefully inform the vendor about what their tool missed.
The solution was to create a new “Service Admins” security group that granted the necessary Active Directory permissions without the baggage of Domain Admins membership.
I removed the OKTA service account from Domain Admins, added it to the new Service Admins group, and adjusted the OU structure so the OKTA service account inherited appropriate policies without conflicts.
The lockouts stopped immediately.