Microsoft Entra ID E2E Testing¶

End-to-end testing guide for Microsoft Entra ID (Azure AD) SSO integration with ContextForge.

This guide walks you through setting up Azure resources and running fully automated E2E tests that validate group-based role assignment for platform_administrator.

Overview¶

The E2E tests in tests/e2e/test_entra_id_integration.py are fully self-contained:

• Create test users and groups in Azure AD before tests
• Execute SSO role mapping tests against real Azure infrastructure
• Delete all created resources after tests complete

This ensures repeatable, isolated test runs without manual cleanup.

Prerequisites¶

Azure Subscription¶

You need access to an Azure tenant where you can:

• Create App Registrations
• Grant admin consent for API permissions
• Create users and groups (or have a service principal that can)

Local Development Environment¶

# Clone and set up the project
git clone <repository-url>
cd hoax-id
make venv install-dev

Step 1: Create an Azure App Registration¶

1.1 Navigate to Azure Portal¶

1. Go to Azure Portal
2. Navigate to Microsoft Entra ID > App registrations
3. Click + New registration

1.2 Register the Application¶

Click Register.

1.3 Note the Application Details¶

After registration, note these values from the Overview page:

# You'll need these later
AZURE_CLIENT_ID=<Application (client) ID>
AZURE_TENANT_ID=<Directory (tenant) ID>

Step 2: Create a Client Secret¶

1. In your App Registration, go to Certificates & secrets
2. Click + New client secret
3. Add a description: E2E Test Secret
4. Choose expiration: 12 months (or as appropriate)
5. Click Add

Important: Copy the secret value immediately - it won't be shown again!

AZURE_CLIENT_SECRET=<secret-value>

Step 3: Configure API Permissions¶

The service principal needs permissions to create/delete users and groups.

3.1 Add Required Permissions¶

1. Go to API permissions
2. Click + Add a permission
3. Select Microsoft Graph
4. Choose Application permissions (not Delegated)
5. Add these permissions:

3.2 Grant Admin Consent¶

1. Click Grant admin consent for [Your Organization]
2. Confirm by clicking Yes

You should see green checkmarks next to all permissions indicating consent is granted.

Step 4: Enable Public Client Flows (Optional)¶

If you want to test ROPC (Resource Owner Password Credentials) token acquisition:

1. Go to Authentication
2. Scroll to Advanced settings
3. Set Allow public client flows to Yes
4. Click Save

Note: ROPC is considered less secure and may be disabled by your organization's security policies. The E2E tests will skip ROPC tests gracefully if this is not enabled.

Step 5: Determine Your Test Domain¶

Test users need a User Principal Name (UPN) in your tenant's domain.

Find Your Domain¶

1. Go to Microsoft Entra ID > Overview
2. Look for Primary domain (e.g., yourcompany.onmicrosoft.com)

Or check Custom domain names for verified domains.

TEST_ENTRA_DOMAIN=yourcompany.onmicrosoft.com

Step 6: Choose a Test User Password¶

The password must meet Azure AD complexity requirements:

• Minimum 8 characters
• At least 3 of: uppercase, lowercase, number, special character
• Cannot contain the user's name

# Example: Strong password for test users
TEST_ENTRA_USER_PASSWORD='ContextForge2024!Test'

Step 7: Configure Environment Variables¶

Create a .env.test file or export variables directly:

# Azure Service Principal (for Graph API access)
export AZURE_CLIENT_ID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
export AZURE_CLIENT_SECRET="your-client-secret-value"
export AZURE_TENANT_ID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

# Test Configuration
export TEST_ENTRA_USER_PASSWORD="ContextForge2024!Test"
export TEST_ENTRA_DOMAIN="yourcompany.onmicrosoft.com"

Environment Variable Reference¶

Step 8: Run the E2E Tests¶

Run All Entra ID Tests¶

# Source your environment variables
source .env.test

# Run the tests
uv run pytest tests/e2e/test_entra_id_integration.py -v

Run Specific Test Classes¶

# Test role mapping only
uv run pytest tests/e2e/test_entra_id_integration.py::TestEntraIDRoleMapping -v

# Test HTTP endpoints
uv run pytest tests/e2e/test_entra_id_integration.py::TestEntraIDEndToEndHTTP -v

# Test admin role retention behavior
uv run pytest tests/e2e/test_entra_id_integration.py::TestEntraIDAdminRoleRetention -v

# Test multiple admin groups
uv run pytest tests/e2e/test_entra_id_integration.py::TestEntraIDMultipleAdminGroups -v

# Test token validation
uv run pytest tests/e2e/test_entra_id_integration.py::TestEntraIDTokenValidation -v

Run with Debug Output¶

uv run pytest tests/e2e/test_entra_id_integration.py -v -s --log-cli-level=INFO

Test Structure¶

The test suite includes these test classes:

Test Scenarios - Detailed Reference¶

Complete Test List¶

Behavior Summary by Scenario¶

What Each Test Validates¶

What Gets Created in Azure¶

During test execution, the following resources are created and then deleted:

Groups¶

Users¶

All resources are automatically cleaned up after tests complete (even on test failure).

Resource Lifecycle¶

Test Start
    │
    ├── Create ContextForge-TestAdmins-{uuid}
    ├── Create ContextForge-TestAdmins2-{uuid}
    ├── Create ContextForge-TestUsers-{uuid}
    │       │
    │       ├── Wait for Azure AD replication
    │       │
    ├── Create cftest-admin-{uuid} → Add to TestAdmins
    ├── Create cftest-regular-{uuid} → Add to TestUsers
    │       │
    │       ├── Wait for membership replication
    │       │
    ├── Run all 15 tests
    │       │
    │       ├── Tests use real Azure group IDs
    │       ├── Tests acquire real ROPC tokens (if enabled)
    │       │
Test End (success or failure)
    │
    ├── Delete cftest-admin-{uuid}
    ├── Delete cftest-regular-{uuid}
    ├── Delete ContextForge-TestAdmins-{uuid}
    ├── Delete ContextForge-TestAdmins2-{uuid}
    └── Delete ContextForge-TestUsers-{uuid}

Troubleshooting¶

Tests Skip with "Azure credentials not configured"¶

Ensure all required environment variables are set:

echo $AZURE_CLIENT_ID
echo $AZURE_CLIENT_SECRET
echo $AZURE_TENANT_ID
echo $TEST_ENTRA_USER_PASSWORD
echo $TEST_ENTRA_DOMAIN

"Insufficient privileges" Error¶

The service principal lacks required permissions. Verify:

1. Correct permissions are added (Application, not Delegated)
2. Admin consent is granted
3. The secret hasn't expired

"Invalid client secret" Error¶

• Check the secret hasn't expired
• Verify you copied the Value, not the Secret ID
• Create a new secret if needed

User Creation Fails with Password Policy Error¶

Your password doesn't meet Azure AD requirements:

# Good password examples
TEST_ENTRA_USER_PASSWORD='ContextForge2024!Test'
TEST_ENTRA_USER_PASSWORD='E2e-Testing-Password-123'

ROPC Tests Skip¶

If ROPC tests are skipped, your tenant may have ROPC disabled. This is common in enterprise environments for security reasons. The other tests will still run.

CI/CD Integration¶

GitHub Actions Example¶

Add these secrets to your repository:

• AZURE_CLIENT_ID
• AZURE_CLIENT_SECRET
• AZURE_TENANT_ID
• TEST_ENTRA_USER_PASSWORD
• TEST_ENTRA_DOMAIN

# .github/workflows/test.yml
jobs:
  entra-e2e:
    runs-on: ubuntu-latest
    # Only run if secrets are configured
    if: ${{ vars.AZURE_CLIENT_ID != '' }}
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          pip install uv
          uv sync

      - name: Run Entra ID E2E Tests
        env:
          AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
          AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
          AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
          TEST_ENTRA_USER_PASSWORD: ${{ secrets.TEST_ENTRA_USER_PASSWORD }}
          TEST_ENTRA_DOMAIN: ${{ secrets.TEST_ENTRA_DOMAIN }}
        run: |
          uv run pytest tests/e2e/test_entra_id_integration.py -v

Security Considerations¶

Principle of Least Privilege¶

The service principal has broad permissions (User.ReadWrite.All, Group.ReadWrite.All). Consider:

• Using a dedicated test tenant
• Limiting the service principal's scope via Conditional Access
• Rotating secrets regularly

Test Isolation¶

Each test run creates uniquely-named resources with UUIDs, so multiple concurrent test runs won't conflict.

Cleanup Guarantee¶

The test fixtures use pytest finalizers that run even on test failure, ensuring resources are always cleaned up.

Summary¶

Related Documentation¶

• Microsoft Graph API - Users
• Microsoft Graph API - Groups
• Azure AD App Registration
• ContextForge SSO Configuration