Azure DevOps Test Plans: Quality Assurance at Scale

Azure Test Plans provides manual and exploratory testing capabilities. Test cases, test suites, and traceability—quality built into your workflow.

Test Plan Hierarchy

Test Plan (Release 2.0)
├── Test Suite (User Authentication)
│   ├── Test Case: Login with valid credentials
│   ├── Test Case: Login with invalid password
│   └── Test Case: Password reset flow
├── Test Suite (Shopping Cart)
│   ├── Test Case: Add item to cart
│   ├── Test Case: Remove item from cart
│   └── Test Case: Apply discount code
└── Test Suite (Checkout)
    ├── Test Case: Complete purchase
    └── Test Case: Cancel order

Creating Test Plans

# Create test plan
az devops invoke \
    --area test \
    --resource plans \
    --route-parameters project=MyProject \
    --http-method POST \
    --in-file test-plan.json

# test-plan.json
{
    "name": "Release 2.0 Test Plan",
    "area": {
        "name": "MyProject\\Release 2.0"
    },
    "iteration": "MyProject\\Sprint 10"
}

Test Cases

**Test Case: Login with valid credentials**

**Preconditions:**
- User account exists in system
- User is on login page

**Steps:**
| # | Action | Expected Result |
|---|--------|-----------------|
| 1 | Enter valid username | Username field populated |
| 2 | Enter valid password | Password field shows dots |
| 3 | Click Login button | Redirected to dashboard |
| 4 | Verify user name displayed | Name shown in header |

**Shared Parameters:**
@username, @password (from data set)

Parameterized Tests

<!-- Shared parameter data -->
<ParameterizedData>
  <DataTable>
    <Row>
      <username>user1@test.com</username>
      <password>Pass123!</password>
      <role>Admin</role>
    </Row>
    <Row>
      <username>user2@test.com</username>
      <password>Pass456!</password>
      <role>User</role>
    </Row>
  </DataTable>
</ParameterizedData>

Test Execution

Web Runner

1. Open Test Plans
2. Select test suite
3. Click “Run”
4. Execute steps, mark pass/fail
5. Add comments and attachments
6. Submit results

Test Runner Extension

// Azure DevOps REST API
const testRun = await client.createTestRun({
    name: "Automated Run",
    plan: { id: planId },
    pointIds: [pointId1, pointId2]
});

// Update results
await client.updateTestResults(testRunId, [{
    id: resultId,
    state: "Completed",
    outcome: "Passed"
}]);

Exploratory Testing

Test & Feedback Extension
├── Capture screenshots
├── Record screen
├── Create bugs inline
├── Add annotations
└── Capture user actions

Linked Work Items

Test Case ←→ User Story
    ↓
Test Run
    ↓
Bug (on failure)
    ↓
Linked to failing test

Test Analytics

// Query test results
TestResultsDaily
| where Date >= ago(30d)
| summarize
    TotalTests = sum(TotalCount),
    PassedTests = sum(ResultPassedCount),
    FailedTests = sum(ResultFailedCount)
    by TestPlanId, Date
| extend PassRate = round(100.0 * PassedTests / TotalTests, 2)
| order by Date desc

REST API Examples

from azure.devops.connection import Connection
from msrest.authentication import BasicAuthentication

credentials = BasicAuthentication('', personal_access_token)
connection = Connection(base_url=organization_url, creds=credentials)

test_client = connection.clients.get_test_client()

# Get test plans
plans = test_client.get_plans(project)

# Get test results
results = test_client.get_test_results(project, run_id)

# Create test run
run = test_client.create_test_run(
    test_run={
        'name': 'API Test Run',
        'plan': {'id': plan_id},
        'isAutomated': True
    },
    project=project
)

Integration with Pipelines

# Run tests and publish results
- task: VSTest@2
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: '**\*Tests*.dll'
    testPlan: 123
    testSuite: 456
    testConfiguration: 789

Azure Test Plans: structured testing for quality delivery.