Philippines staffing research · Published October 22, 2026

Checking Whether a VA Work Sample Scores the Right Thing

A work sample can be realistic and still unreliable. Test the task, the rubric, and reviewer agreement before using it to hire.

On this page

Methodology

Bounded method review with 4 authoritative sources, 6 analysis sections, and 0 service performance claims.

Key Stats

4: authoritative sources
6: analysis sections
0: service performance claims

Key Takeaways

Name the construct
Test the task with current staff
Measure reviewer agreement

Name the construct

Write down what the sample is meant to measure. Accuracy, source verification, customer writing, prioritization, and boundary awareness are different constructs. A single task can touch several, but the rubric should identify the primary one. If the role needs accuracy and speed, decide how a fast but wrong answer is scored. Check that the task actually requires the construct. A spreadsheet with no discrepancies does not test verification. A customer email with no ambiguity does not test judgment. Add one deliberate missing fact or conflicting source, and observe whether the candidate stops or guesses.

Test the task with current staff

Ask two experienced team members to complete the sample under the same conditions. Compare their outputs and their reasoning. If they disagree about what good looks like, the task or the instructions are ambiguous. Fix that before using the sample on candidates. Check the time limit. A task that takes an experienced person twice the allotted time is not a fair sample. Check the tools and the data. A candidate should not be penalized for lacking a license or a device that the role does not require. Offer an equivalent format where needed.

Measure reviewer agreement

Give two reviewers the same submissions and the rubric, and ask them to score independently. Compare exact agreement and agreement within one point. Discuss every disagreement and record whether the cause was the rubric, the evidence, or the reviewer's product knowledge. A rubric with vague words such as professional or proactive will produce low agreement. Replace them with observable criteria: the reply states the next action, cites the source, avoids an unsupported promise, and stops when approval is missing. Keep the initial scores so calibration does not erase the disagreement.

Evidence and reporting

Keep the scored submissions, the rubric version, the reviewer notes, and the calibration record. Report the score distribution and the agreement result, not only the selected candidate's score. If the sample is used across roles, separate the results by role and by task. A sample that works for an inbox role may not work for a reporting role. State what the sample does not measure, such as long-term reliability, learning speed, or fit with a particular manager. Those require the trial and the review cadence. The hiring decision should combine the sample with the interview evidence and the reference check, not replace them.

Limitations

A small hiring round cannot establish statistical validity. The sample measures performance under observation, which may differ from daily work. The rubric reflects the current standard, which may change. A candidate who performs well on a narrow task may still need support in the full lane. The protocol also cannot eliminate bias in interviewing or evaluation. It can make the evidence more visible and the criteria more consistent. Use it as one input, not as a complete hiring decision.

Decision use

If the sample is valid for the construct, keep the task and rubric stable across candidates. If it is not, revise the task or use the sample as a structured interview aid rather than a decisive score. Record what was learned from the first ninety days and update the sample for the next role. Do not reuse a candidate's work as production output without an agreement. Pay for substantive work that produces value. The hiring process is part of the employer's reputation. Keep the sample instructions and the scoring notes for the next hiring round, and review them with the hiring manager before the role opens. A sample that was fair in one market may create an unnecessary barrier in another if the instructions assume a tool, a currency, or a local convention that the job does not require.

Sources checked October 22, 2026. Method protocol only; no provider performance claim.

Sources

U.S. GAO, Assessing Data Reliability: Public guidance supporting the protocol.
International Labour Organization, Data: Public guidance supporting the protocol.
W3C Web Content Accessibility Guidelines 2.2: Public guidance supporting the protocol.
World Bank, Philippines Overview: Public guidance supporting the protocol.

FAQs

Plan the next step

Translate the method into a bounded role brief with named review points.

Review virtual assistant services

Read the daily blog guides · Explore service workflows · Plan your staffing routine

Use the method to define the population, evidence, and reviewer before testing.