AI Workplace Quality: handouts

Choose Print / Save as PDF, then select Save as PDF in your browser. The learning copy hides answers.

AI Workplace Quality

Original practice materials · Version 2.0.0 · Synthetic business data

1. Extract a usable brief

Turn a source into explicit constraints without inventing details.

Read each source ID and list supported values.
Separate the task goal from the operational constraints.
Use the requested JSON field types.

Extract the pilot requirements

Return start_date (string), agents (number), budget_usd (number), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return start_date (string), agents (number), budget_usd (number), source_ids (array).",
  "flawed_output": "The pilot starts next month for 20 agents with unlimited budget."
}

Reference answer

{
  "start_date": "2026-11-02",
  "agents": 12,
  "budget_usd": 2400,
  "source_ids": [
    "S1"
  ]
}

S1 supplies every requested value. Use USD as a number, not a formatted string.

Capture the operating constraints

Return owner (string), duration_days (number), human_approval (boolean), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return owner (string), duration_days (number), human_approval (boolean), source_ids (array).",
  "flawed_output": "Riley will run the 30-day pilot with automatic sending."
}

Reference answer

{
  "owner": "Morgan",
  "duration_days": 14,
  "human_approval": true,
  "source_ids": [
    "S1",
    "S2"
  ]
}

S1 gives the duration; S2 gives the owner and approval requirement.

Extension: A clear brief helps a model, but final outputs still need checking.

2. Check claims against evidence

Separate unsupported and contradicted claims.

Compare each claim with a named source.
A contradicted claim has opposing evidence. An unsupported claim has no measurement.
Keep source IDs valid and avoid treating plausible statements as facts.

Repair an inflated accuracy claim

Return correct_tickets, total_tickets (numbers), accuracy_percent (number), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return correct_tickets, total_tickets (numbers), accuracy_percent (number), source_ids (array).",
  "flawed_output": "The pilot achieved 95% accuracy across 100 tickets."
}

Reference answer

{
  "correct_tickets": 72,
  "total_tickets": 80,
  "accuracy_percent": 90,
  "source_ids": [
    "S3"
  ]
}

Compute 72 ÷ 80 × 100 = 90. S3 is the fixed evidence.

Mark an unsupported satisfaction claim

Return claim (string), verdict (string), measured (boolean), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return claim (string), verdict (string), measured (boolean), source_ids (array).",
  "flawed_output": "Customer satisfaction improved significantly."
}

Reference answer

{
  "claim": "Customer satisfaction improved",
  "verdict": "unsupported",
  "measured": false,
  "source_ids": [
    "S3"
  ]
}

The memo explicitly says satisfaction was not measured. Unknown evidence is not a positive result.

Extension: A structural score does not certify the quality of arbitrary prose.

3. Produce a reliable structure

Use a precise schema and stable units.

The output contract names each field and its type.
Numeric values must not contain display units.
Use null for unknown numeric measurements, not zero.

Build a results JSON object

Return tickets, median_minutes (numbers), satisfaction_percent (null), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return tickets, median_minutes (numbers), satisfaction_percent (null), source_ids (array).",
  "flawed_output": "{\"tickets\":\"80 tickets\",\"median_minutes\":\"6m\",\"satisfaction_percent\":0}"
}

Reference answer

{
  "tickets": 80,
  "median_minutes": 6,
  "satisfaction_percent": null,
  "source_ids": [
    "S3"
  ]
}

Zero would assert a measured value. Use null for missing satisfaction data.

Make the budget calculation explicit

Return budget_usd, agents, usd_per_agent (numbers), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return budget_usd, agents, usd_per_agent (numbers), source_ids (array).",
  "flawed_output": "Budget per agent is about 250 USD."
}

Reference answer

{
  "budget_usd": 2400,
  "agents": 12,
  "usd_per_agent": 200,
  "source_ids": [
    "S1"
  ]
}

2400 ÷ 12 = 200 USD per agent; retain the source of the inputs.

Extension: Schema validity and factual correctness are checked separately in the feedback.

4. Review before handoff

Check a result for evidence, constraints and an explicit limitation.

Check numerical claims with arithmetic.
Check prohibited actions against the review policy.
Make unknown measurements visible in the final artifact.

Review the reply workflow

Return allow_personal_data (boolean), allow_unapproved_sending (boolean), owner (string), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return allow_personal_data (boolean), allow_unapproved_sending (boolean), owner (string), source_ids (array).",
  "flawed_output": "Upload customer records and automatically send replies."
}

Reference answer

{
  "allow_personal_data": false,
  "allow_unapproved_sending": false,
  "owner": "Morgan",
  "source_ids": [
    "S2"
  ]
}

S2 prohibits personal data and unapproved customer replies.

Write a bounded evidence summary

Return accuracy_percent (number), median_minutes (number), satisfaction_status (string), source_ids (array).
Use the fixed source IDs. Array order follows the contract.
Do not invent unsupported values. Missing evidence must remain unknown.

{
  "sources": [
    {
      "id": "S1",
      "title": "Pilot brief",
      "text": "The fictional Northstar pilot starts 2026-11-02. It covers 12 support agents. The pilot lasts 14 calendar days. Budget is 2400 USD."
    },
    {
      "id": "S2",
      "title": "Review policy",
      "text": "Morgan owns the review. No customer personal data may enter the pilot. No automation may send customer replies without human approval."
    },
    {
      "id": "S3",
      "title": "Results memo",
      "text": "The test processed 80 synthetic tickets. 72 had the correct category. Median handling time was 6 minutes. Customer satisfaction was not measured."
    }
  ],
  "brief": "Return accuracy_percent (number), median_minutes (number), satisfaction_status (string), source_ids (array).",
  "flawed_output": "The system improved accuracy and satisfaction and cut time by half."
}

Reference answer

{
  "accuracy_percent": 90,
  "median_minutes": 6,
  "satisfaction_status": "not measured",
  "source_ids": [
    "S3"
  ]
}

Use the measured results without inventing satisfaction or causal improvements.

Extension: Use the checklist to self-review free-form prose; this checker evaluates the fixed evidence fields only.

Final project

Deliver a grounded pilot brief

Return exactly start_date (string), total_tickets, labelled_tickets, correct_tickets, accuracy_percent, median_minutes (numbers), satisfaction_percent (null), human_approval (boolean), accepted_ticket_ids, excluded_ticket_ids and source_ids (arrays).
Use ticket_id as the business key. Compare actual, predicted and minutes. Collapse identical measurements; keep the first source record as evidence. Exclude all measurements for any ticket whose fields conflict.
Sort accepted_ticket_ids and excluded_ticket_ids ascending. source_ids starts with BRIEF and POLICY, followed by the first evidence source for each accepted ticket in accepted_ticket_ids order.
total_tickets includes accepted unlabelled tickets. Accuracy uses labelled accepted tickets only; round 100 × correct / labelled to two decimal places. Do not use source record count as the denominator.
Median uses known minutes among accepted tickets, sorted numerically; average the two middle values for an even count. Unknown minutes remain excluded. Satisfaction is null; retain the human-approval policy.

Use the complete project source data in the course download. Every output check must pass.

{
  "start_date": "2026-11-02",
  "total_tickets": 56,
  "labelled_tickets": 51,
  "correct_tickets": 44,
  "accuracy_percent": 86.27,
  "median_minutes": 7,
  "satisfaction_percent": null,
  "human_approval": true,
  "accepted_ticket_ids": [
    "T100",
    "T101",
    "T102",
    "T103",
    "T104",
    "T105",
    "T107",
    "T108",
    "T109",
    "T110",
    "T111",
    "T112",
    "T113",
    "T114",
    "T115",
    "T116",
    "T117",
    "T118",
    "T119",
    "T120",
    "T121",
    "T122",
    "T123",
    "T124",
    "T125",
    "T126",
    "T127",
    "T128",
    "T129",
    "T130",
    "T131",
    "T132",
    "T133",
    "T134",
    "T135",
    "T136",
    "T137",
    "T138",
    "T139",
    "T140",
    "T141",
    "T142",
    "T143",
    "T144",
    "T145",
    "T146",
    "T147",
    "T148",
    "T149",
    "T150",
    "T151",
    "T152",
    "T153",
    "T160",
    "T161",
    "T162"
  ],
  "excluded_ticket_ids": [
    "T106"
  ],
  "source_ids": [
    "BRIEF",
    "POLICY",
    "E100",
    "E101",
    "E102",
    "E103",
    "E104",
    "E105",
    "E107",
    "E108",
    "E109",
    "E110",
    "E111",
    "E112",
    "E113",
    "E114",
    "E115",
    "E116",
    "E117",
    "E118",
    "E119",
    "E120",
    "E121",
    "E122",
    "E123",
    "E124",
    "E125",
    "E126",
    "E127",
    "E128",
    "E129",
    "E130",
    "E131",
    "E132",
    "E133",
    "E134",
    "E135",
    "E136",
    "E137",
    "E138",
    "E139",
    "E140",
    "E141",
    "E142",
    "E143",
    "E144",
    "E145",
    "E146",
    "E147",
    "E148",
    "E149",
    "E150",
    "E151",
    "E152",
    "E153",
    "E157",
    "E158",
    "E159"
  ]
}

Related public objectives: https://learn.microsoft.com/en-us/training/modules/build-effective-generative-ai-solutions-organization/.

Independent original materials. CC BY 4.0.