Intent boundary: A pilot tests operating fit on a defined plan and configuration. It is not a substitute for security, legal, accounting, or contract review.
repeatable evidence for choose, reject, or investigate
The process succeeds only when the team can explain the result, recover from exceptions, and retain evidence.
The control sequence
Freeze the test brief
Record product, plan, trial restrictions, devices, country, currency, tax settings, team roles, integrations, and every vendor-assisted configuration.
Intake and estimate
Create a new lead, repeat customer, emergency call, multi-property account, quote with option, deposit request, and follow-up task.
Dispatch under pressure
Schedule normal work, insert two urgent calls, move one job, cancel another, change the assigned technician, and notify customers.
Complete work in the field
Run mobile check-in, notes, photos, forms, price-book edits, signature, time, material, equipment, and offline or weak-signal behavior.
Invoice, pay, and reconcile
Create partial and final invoices, take two payment types, issue a refund, add a tip or fee where relevant, and reconcile the accounting result.
Run recurring and automated work
Test a recurrence, reminder, estimate follow-up, review request, missed-call response, permission boundary, and human override.
Stress support and recovery
Submit the same support question, create a duplicate, break a mapping, restore a reschedule, inspect the audit trail, and export core data.
Decide with evidence
Score observed outcomes, normalize three-year cost, list unresolved risks, name implementation ownership, and choose advance, reject, or extend.
The locked scorecard
| Dimension | Weight | Evidence required | Reject signal |
|---|---|---|---|
| Field usability | 20% | Timed mobile completion by intended role | Critical task cannot be completed reliably |
| Workflow depth | 20% | Baseline and vertical scenarios | Required exception needs shadow process |
| Admin efficiency | 15% | Dispatch, correction, and reporting time | Duplicate entry becomes routine |
| Integration + exit | 15% | Accounting exception and export evidence | Core data cannot be reconciled or retrieved |
| Cost clarity + value | 15% | Written steady-state quote and conservative return | Required cost remains unquoted |
| Support + reliability | 10% | Same support case and recovery test | Operational issue lacks a credible path |
| AI safeguards | 5% | Disclosure, override, permissions, and error handling | Automation acts without acceptable control |
Failure modes to expose
- Letting the vendor choose only happy-path scenarios
- Testing on a plan the business will not buy
- Using only office users and skipping actual field roles
- Changing the scorecard after seeing the product
- Counting promised automation as observed savings
- Ending without testing export, correction, support, and accounting
Worksheet questions
- 01
Which five workflows must pass for the product to remain eligible?
- 02
Which two failures are immediate rejection conditions?
- 03
Who records evidence and who has final decision authority?
- 04
What assistance did the vendor provide that ordinary users will not receive?
- 05
What test data must be removed when the pilot ends?
- 06
Which unresolved issue needs a second week instead of optimistic scoring?