Testing evidence completeness in inbox triage without a convenience sample

Question and scope

The protocol does not estimate an industry benchmark; it produces a local description that business owners hiring overseas can review. State how duplicate requests are identified and whether they are merged, linked, or counted separately. If a definition changes during the review, keep the earlier classification and apply the new rule only to later cases. Record the exclusions in advance so a difficult case cannot be removed after the result is known. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract. State the observation window, the sites or teams included, and the exclusions before collecting the first case. The population should include cases handled by every owner and channel in scope, not only the queue with the cleanest records. Define the start event, the stop event, the pause rule, and the treatment of a missing meeting note before collecting data. The protocol should be short enough for an operator to follow and specific enough for a reviewer to repeat. At Overseas Virtual Assistant, the inbox triage review should end with a dated note and a named owner for the next check.

Population and unit

Record how duplicate requests were matched, merged, or kept separate, because that choice changes the denominator. A denominator that changes during the window should be reported separately for each period. Write the rule for a missing meeting note before the first case so the treatment is consistent. Keep a list of requests closed as duplicates and show which case survived. If the evidence is sensitive, record the approved reference and access path rather than copying the content. Record how duplicate requests were matched, merged, or kept separate, because that choice changes the denominator. A denominator that changes during the window should be reported separately for each period. The definition should make it possible to classify a case without asking the person who did the work. A case that never received an owner still belongs in the population; do not drop it because it is difficult to classify. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

Definitions and measures

Name the owner who can declare a case complete, and record that role in the data dictionary. Define allowed values for status, reason, owner, and evidence state before collection. Record the person who approved the definition and the date it became effective. If two reviewers disagree, preserve both classifications until the definition is clarified. Record this decision in the study log with the case key, the review date, and the person who approved the classification. Measure elapsed time from the first recorded request to the verified destination update, and report waiting intervals separately. Record the source system, the record identifier, the extraction time, and the reviewer for every extracted value. Before collection, test the definitions on a missing trial scorecard, a reopened case, and a handoff. If two reviewers disagree, preserve both classifications until the definition is clarified. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

Data dictionary and evidence

Use a pseudonymous case key and keep the identity crosswalk in an approved system with restricted access. Name the required fields for each article type and mark conditional fields in the dictionary. Store sensitive evidence in the approved repository and keep only the reference in the analysis file. Define who may change a coded value and how the change is logged. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it. Each case needs the source, the observation time, the field supported, the owner, and the reviewer. Keep the identity crosswalk in the approved system and restrict it to the review owner. Keep the original evidence reference next to the coded value so the reviewer can verify the classification. Archive the dictionary with the study so a later reviewer can reproduce the categories. At Overseas Virtual Assistant, the reviewer should be able to explain this step to business owner from the case record alone. If the source is a document, record the page or field; if the source is a system, record the record identifier and extraction time. Use a controlled list for exception reasons; do not let free text become the analysis category. Store sensitive evidence in the approved repository and keep only the reference in the analysis file. Pilot the dictionary with two reviewers and compare their classifications before the main sample. Use the same note in the case file and the study extract; do not create a second version in a personal document.

Sampling and collection

Sample across the full observation window, including the days when the queue was busiest and the days when coverage changed. Keep the collection window and the number of collectors in the report; both affect how the result should be interpreted. Record the eligible count and every exclusion before the sample is drawn. Keep a table of planned, eligible, selected, and reviewed counts for each stratum. When a case needs an exception, record the reason, the owner, and the date before continuing the review. Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Use the same collection form for every case and keep the original record linked to the extracted fields. If a stratum has fewer cases than planned, report the smaller count instead of replacing it with a convenience case. Record the sampling frame as an export with a date, filter, and row count. Keep the measurement note short enough to read with the case and specific enough to support the reported access closure. Use consecutive inbox triage cases where possible, then stratify by owner, channel, and risk class. Keep the collection window and the number of collectors in the report; both affect how the result should be interpreted. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. Use a random start and a fixed interval when the frame is ordered, and record both. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.

Analysis and reporting

If the sample is small, report the instability and the cases that drive the result instead of a precise-looking percentage. If two measures disagree, report both and explain what each one counts. Report the number of cases that changed status after the first extraction and how the change was handled. When a measure is based on a small sample, state the case count and avoid ranking owners by the result. Link the observation to the exact field in the task board and to the inbox triage note that supports it. Keep case-level detail available for review, because a summary chart without records cannot support a decision. If two measures disagree, report both and explain what each one counts. Show the unresolved cases by owner, age, and dependency instead of reporting a single completion percentage. Report the number of cases that drive a small-sample result and the direction of the effect. Do not let a summary written later replace the evidence that was available at the first decision.

Limitations

Report what remains unknown and what evidence would be needed to answer the next question. The record may be incomplete or delayed, so the method cannot infer work that never entered the queue. Report the limits of the shared inbox data, including fields that were unavailable or changed during the window. State that the result describes the observed sample, not the whole industry or every team in the service. If the case moved between owners, include both owner periods in the review note and name the handoff time. Small samples are sensitive to one case; report the instability instead of presenting a percentage as a stable rate. Report the unknown population and the evidence that would be needed to describe it. Record the response or access problems that prevented a case from being reviewed. If the reviewer performed the work, use a second reviewer for a subsample and report the disagreement. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.

Review and next action

Choose one next action: fix the source, simplify the meeting note, change the queue, adjust access, or pause the work until the owner decides. Report what moved, what stayed the same, and what the study still cannot explain. Keep the old process note until the new process has been tested; do not erase the baseline. Keep the prior procedure available so the next reviewer can compare the old and new states. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract. Name the owner who will read the result and set a review date before the sample is collected. A quality review should end with a testable next step, not a general instruction to improve. Report the action result even if the change did not work; a failed test is still evidence. Report a failed action with the same detail as a successful one so the team can learn from it. At Overseas Virtual Assistant, the inbox triage review should end with a dated note and a named owner for the next check.

What is measured?

Whether the operating record supports a reproducible review.

Who decides?

The named client owner.

Main limit?

A small sample cannot establish causation.