What you'll need
- A draft assessment assembled in the builder (sections, pages, blocks)
- Rubrics or answer keys attached to every scored task
- A phone or narrow browser window for the mobile pass
- 1–2 colleagues available for a short pilot run
Step 1: Open the candidate preview from the builder header
Start every QA pass from "Preview as candidate" in the builder header. The preview renders the assessment exactly as a candidate will see it — the same pages, blocks, instructions, and timers — so anything you catch here is a real problem fixed before it costs you a response.
Resist the urge to skim your own content. You wrote these tasks, so your brain will fill in gaps a candidate cannot. Read every instruction as if you have never seen the role, and answer every question honestly rather than clicking through. The point is to experience the assessment, not to tour it.
Step 2: Walk every page on desktop, answering for real
Go page by page and actually complete each task: type the text answers, fill the table inputs, upload a file, record the audio or video if the assessment asks for it. Interactive item types fail in ways a visual skim never reveals — an upload that rejects the obvious file format, a code task with a confusing starter state, a recording block a candidate might miss.
Pay special attention to side-by-side stimulus layouts. Confirm the stimulus and the question are visible together, that long documents scroll independently, and that nothing forces the candidate to memorize the stimulus before answering. A layout problem here quietly changes what the task measures — from the skill you designed for to working memory and patience.
- Every instruction is unambiguous to someone outside your team
- Uploads, recordings, and code blocks accept a realistic response
- Stimulus and question are readable together on one screen
- Required fields and navigation behave as expected on every page
Step 3: Repeat the walkthrough on a phone
Run the same preview on a phone or a narrow browser window. Candidates do not always sit at a desk, and layouts that look fine on a wide monitor can bury a stimulus, truncate a table, or push the submit button below an endless scroll on a small screen.
If a task genuinely requires a desktop — a code environment, a long side-by-side document review — say so explicitly in the assessment instructions rather than letting candidates discover it mid-attempt. A sentence of expectation-setting up front is far cheaper than a frustrated, half-finished response.
Step 4: Time a realistic dry run against your limits
Complete the assessment at a realistic working pace and compare your elapsed time to the total and per-page limits you set in the builder. Then add margin: you already know the material, and a candidate reading everything for the first time will be meaningfully slower than you were.
Check per-page limits individually, not just the total. One underestimated page — usually the one with the richest work sample — can wreck an otherwise well-timed assessment, because time pressure on a single task punishes careful candidates the most. If a page felt rushed even to you, extend it before anyone else sees it.
Step 5: Verify every rubric and answer key against its task
Open each scored task and confirm its scoring setup matches the task as actually written — not the task as you drafted it three edits ago. For auto-scored items, check that the answer key marks the correct options and that point values are what you intend. Remember that a task with no answer key auto-scores as null and waits for manual review; it does not silently score zero, but it will sit unscored until someone handles it.
For rubric-scored tasks, reread each criterion against the task prompt and confirm a strong response to this prompt could actually demonstrate every criterion. Note that rubrics freeze into a snapshot when attached, so this is the moment to fix wording — later edits in your library will not reach the published assessment.
Step 6: Clear the publish checklist
The publish checklist is the platform's final gate: it blocks Publish until required setup is complete, flagging things like missing scoring configuration or incomplete section setup. Work through every item it raises rather than looking for the fastest path to a green button.
Treat the checklist as a floor, not a ceiling. It verifies structural completeness; it cannot judge whether your instructions are clear or your timing is humane. That is what the previous steps were for. When the checklist is clear and your manual passes are done, you are ready to pilot.
Step 7: Run a pilot with one or two internal people
Before any candidate sees the assessment, have one or two colleagues take it end to end under real conditions — the actual delivery link, the actual time limits, and the security settings you plan to use, so the consent and preflight experience gets tested too. Choose people who did not help build it; fresh eyes find what yours cannot.
Debrief immediately while the experience is fresh. Then score their responses against your rubrics as a final check that the rubric language works on real output, not just in theory. Fix what the pilot surfaces, re-run the preview on anything you changed, and only then send invitations.
- Where did instructions feel ambiguous or surprising?
- Which page felt most time-pressured?
- Did the preflight and security prompts feel proportional to the stakes?
- Could the pilot's responses be scored cleanly against the rubric?
Pro tips
- QA after every meaningful edit, not just once — a one-line change to a task can invalidate an answer key.
- Keep a shared QA checklist per assessment so a second person can verify the pass instead of trusting that it happened.
- Preview with the same security settings you will deliver with, so you experience the consent and preflight gate yourself.
- Budget the timing for a first-time reader, not for the author — you will always be the fastest person to ever take this assessment.
- If the pilot surfaces a rubric wording problem, fix it before publishing; attached rubrics are frozen snapshots.