How to Test Microsoft Entra Conditional Access Policies Safely with the What If Tool

Conditional Access (CA) policies are one of the strongest defences in Microsoft Entra ID , but they can also be dangerous when misconfigured. A single mistake can lock out every user, including your global administrators.

That’s why the What-If Tool exists. It allows you to test Conditional Access policies safely before enforcing them, helping you understand how each rule behaves without affecting users in production.

Never miss an article and subscribe, and don’t forget to check out my YouTube channel, Control Alt Delete Tech Bits

In this guide, we’ll cover how to use the What-If Tool properly, how to test safely with report-only mode, and why you should still keep break-glass accounts in place, just in case.

What are Conditional Access Policies and the what if tool?

Conditional Access is the gatekeeper of Microsoft Entra ID. It decides who can sign in, under what conditions, and from which devices or locations. Every authentication request that hits your tenant passes through these rules.

When configured correctly, Conditional Access becomes one of your strongest defences. It can:

• Block risky sign-ins automatically, based on signals like unfamiliar locations or high sign-in risk scores.
• Enforce Multi-Factor Authentication (MFA) only when needed, reducing unnecessary prompts while keeping accounts secure.
• Ensure only compliant or hybrid-joined devices can access sensitive apps.
• Apply context-aware controls, such as allowing browser access but blocking downloads on unmanaged devices.

In short, it stops attackers before they reach your data, and it does it silently, behind the scenes, using real-time risk signals.

But when Conditional Access is misconfigured, the same power that protects you can become your biggest problem. A single overly broad rule can deny access to every account, including your global administrators. This is why so many admins have learned the hard way that testing before enforcing isn’t optional; it’s essential.

Testing each policy first allows you to spot unexpected behaviour before it reaches production. For example:

• A rule meant for staff accidentally blocks external examiners.
• A device-based restriction stops teachers using shared classroom PCs.
• An “All Users” policy accidentally includes your break-glass accounts.

These are avoidable mistakes, and that’s where the What-If Tool makes the difference.

The What-If Tool gives you visibility and control. It simulates a complete sign-in journey based on parameters you define, such as user identity, device type, location, and target application. In seconds, it tells you which Conditional Access policies would apply, which ones would be skipped, and why.

That means you can see how your rules behave before enforcing them, removing guesswork from one of the most sensitive parts of Entra ID configuration. For environments like schools, trusts, or multi-site organisations, this is the difference between a smooth rollout and a full-blown service outage.

Start in Report-Only Mode

Before touching the What-If Tool, always build and test policies in Report-Only Mode. This lets you see how rules would behave without enforcing them.

1. Sign in to https://entra.microsoft.com.
2. Navigate to Entra ID > Conditional Access > Policies.
3. Select + New Policy and give it a clear name, for example Require MFA for Admins (Report-Only).
4. Under Assignments, choose a small pilot group of users.
5. Under Access controls, select Grant access > Require multi-factor authentication.
6. At the top, switch Report-Only to On and save.

You can now use the What-If Tool to see how that policy would behave for real-world scenarios.

Using the What-If Tool Step by Step

1. In the Entra Admin Centre, go to Protection > Conditional Access > What-If.
2. Choose the user you want to test.
3. Select a cloud app (for example, Microsoft 365 Exchange Online).
4. Pick a device platform and location (for instance, Windows from Head Office).
5. Click What If.

The results page lists every Conditional Access policy in your tenant and whether it applies, does not apply, or is report-only.

You can click into each policy to understand why it applied, invaluable when troubleshooting access issues.

Interpreting Results

The What-If results tell you which policies would apply and why for a given user, app, device, location, and risk level. Use them to spot issues before you flip a policy to On.

Unexpected matches

Symptom: A policy applies to users, apps, or locations you didn’t intend.

Typical causes

• Scope too broad: All users or All cloud apps used during early testing and never narrowed.
• Group drift: The pilot or exclusion group contains nested groups you didn’t account for.
• Named location mismatch: Office IP range changed; your trusted range is out of date.
• Device platform guess: Browser user agents can be ambiguous; platform scoping may catch more than expected.

How to fix

• Replace All users with a named pilot group.
• Check group membership and nesting; prefer static pilot groups for first deployments.
• Re-verify Named locations; use exact static IPs, not broad ranges.
• If you must target platforms, add device state (e.g., require compliant) to tighten scope.

Overlapping conditions

Symptom: Two or more policies hit the same sign-in with different outcomes (e.g., one requires MFA, another requires compliant device).

What to know

• Conditional Access isn’t ordered; all matching policies evaluate, and controls combine. If any one policy blocks, access is blocked. If multiple grant controls apply, all grant controls must be satisfied (MFA and compliant device, etc.).

Typical causes

• A broad “baseline” policy overlaps a tighter “department” policy.
• Legacy “catch-all” policies still enabled after adding new templates.
• Duplicate policies created by different admins with similar names.

How to fix

• Consolidate into layered policy sets:
• Org-wide baseline (e.g., block legacy auth).
• Admin protection (MFA, phishing-resistant methods).
• User/app specific (e.g., require compliant device for Exchange/SharePoint).
• Standardise naming (e.g., Security – Admins – Require MFA) and document intent in the description.
• Use the What-If tool on key personas (admin, teacher, student, guest) to confirm the final, combined effect is what you expect.

Gaps in coverage

Symptom: A policy you rely on doesn’t apply to certain users, devices, or apps.

Typical causes

• New SaaS apps or service principals not included in scope.
• Guests and external collaborators excluded by mistake.
• Mobile or browser access path not targeted (e.g., “All cloud apps” replaced with a subset).
• Device compliance required before devices are enrolled or evaluated.

How to fix

• Expand Cloud apps scope or switch back to All cloud apps and add explicit exclusions if needed.
• Add a dedicated Guest/External users policy (require MFA, restrict download).
• Roll out Intune enrolment and compliance policies before enforcing “Require compliant device”.
• Re-run What-If using guest and service principal identities to validate coverage.

Reading the details (so you know why)

In the What-If results, click a policy to see Conditions matched and Controls required. Cross-check these against your intent:

• Users/Groups: Is the user included through a nested group?
• Cloud apps: Is the target app listed explicitly or via All cloud apps?
• Conditions: Locations, platforms, client apps (modern vs legacy) match expectations?
• Grant controls: MFA, compliant device, approved client app — is that the combo you intend?
• Session controls: If you’re using “Use app enforced restrictions” or “Sign-in frequency”, confirm they appear.

Break-Glass Accounts: Your Safety Net

Even with careful testing, mistakes happen. That’s why you should always have break-glass accounts, emergency Global Admin accounts excluded from Conditional Access.

Best practice for break-glass accounts

• Maintain two accounts (primary and backup).
• Exclude them from all Conditional Access and Identity Protection policies.
• Use long, unique passwords stored securely offline (for example, in a safe).
• Disable MFA and passwordless methods for them — they must always work.
• Turn on sign-in alerts so you know if they’re ever used.
• Test quarterly to confirm access still works.

Think of break-glass accounts as your insurance policy: you hope never to need them, but you’ll be relieved they exist if something goes wrong. Checkout this article about break glass accounts

The What-If Tool is one of those Entra ID features that most admins overlook until they’ve been locked out once , then it becomes their favourite button.

It turns Conditional Access from a risky experiment into a predictable science. You can test new rules, model real-world sign-ins, and confirm that your policies do exactly what you expect.

Combine it with report-only mode, proper naming conventions, and tested break-glass accounts, and you’ll never fear a Conditional Access change again.

Security improves. User experience stays smooth. And you, as the admin, stay firmly in control.

Feel free to buy me a coffee to keep this website up and running