Skip to main content
Safe Online Exam testing is layered because the critical path crosses server code, Canvas, and a native Safe Exam Browser client. Automated checks protect application contracts; a full Canvas and SEB acceptance run validates the integration boundaries that cannot be fully reproduced in a unit test. This page covers every verification command, explains what each gate protects, and walks through the complete acceptance sequence for administrator, instructor, and student roles.

​
Automated Verification Gates

​
Installing Dependencies

Before running any gate, install dependencies using the enforced supply-chain procedure:
Run the dependency-policy check first because a contributor can change .npmrc, package.json, or package-lock.json. The --ignore-scripts flag keeps lifecycle scripts disabled even if repository configuration changes. install:trusted verifies the complete lockfile, then rebuilds and executes only esbuild. The policy enforces npm 11.19.0, exact direct versions, the public registry, SHA-512 integrity metadata, and rejects packages published less than three days ago. Run the network-backed supply-chain gate before a release or when dependencies change:
This verifies npm registry signatures and available provenance attestations, then rejects production vulnerabilities.

​
Individual Gates

Run individual gates when iterating on a specific area:

​
Full Local Verification

1

Standard non-browser gate

Run npm run verify to check release metadata consistency, then execute type checking, linting, Prettier verification, Vitest coverage tests, and the production build.
2

PostgreSQL integration gate

Run npm run verify:postgres to start PostgreSQL 17, apply migrations in isolated schemas, and execute repository atomicity and concurrency tests.
3

Browser E2E gate

Run the Playwright smoke suite to confirm public routes, detector availability, and React app-shell rendering.
The deploy pipeline runs both npm run verify and npm run verify:postgres before promotion. GitHub Actions also runs the Compose smoke test for every pull request and push to main.
If Chromium is not installed for E2E tests, add it before retrying:

​
Compose Smoke Test

Verify the production topology end-to-end, including image build, migration/readiness gates, public health and JWKS probes, and named-volume persistence:
CI builds the image once through Buildx and supplies it via COMPOSE_SMOKE_IMAGE with COMPOSE_SMOKE_SKIP_BUILD=true to avoid rebuilding. The script restarts the app and confirms the PostgreSQL volume retained a written row.

​
Local Production-Build Smoke Run

Use this after npm run build to diagnose the deployed runtime shape:
Then verify the key public endpoints:
This mode cannot validate Canvas OAuth, PostgreSQL, certificate decryption, or real SEB runtime behavior. Use it only to confirm the server starts and public routes respond correctly.

​
Test Coverage by Layer

The table below maps every test layer to the trust boundary it protects. A high coverage percentage alone is insufficient — the critical requirement is that each test exercises the same boundary it claims to protect: signed LTI data for identity, server proof for access-code release, and Canvas-authored completion for exit behavior.

​
Playwright E2E Scope

npm run test:e2e builds the application and starts it locally with USE_IN_MEMORY_STORE=true, TOOL_URL=http://localhost:8080, and test-only LTI and Canvas API values. It runs Chromium in desktop and mobile projects and confirms:
  • public health, JWKS, and LTI metadata responses;
  • detector availability on both supported Canvas assessment URL patterns;
  • selected React app-shell behavior; and
  • error-free page rendering on desktop and mobile.
It does not contact Canvas or start a native SEB client.

​
Canvas and SEB Acceptance Sequence

Run the full acceptance sequence after any deployment that affects authentication, Canvas interaction, settings, configuration generation, the detector, certificate material, or exit behavior. Use separate administrator, instructor, and student accounts against a stable Canvas sandbox with dedicated synthetic accounts only.
The complete three-role acceptance sequence — administrator, instructor, and student in a real SEB client — is required before any deployment is considered production-ready. Automated gates do not replace this sequence.
1

Administrator acceptance

  1. Confirm /health, /lti/config, JWKS, and detector endpoints on the deployed URL.
  2. Confirm Canvas stores the current LTI client ID, deployment ID, login URL, target link URI, and JWKS URL from /lti/config.
  3. Confirm the API OAuth Developer Key has the complete scope set described in the Canvas setup guide.
  4. Open Safe Online Exam Admin from root-account navigation and complete its separate OAuth authorization.
  5. Confirm only Canvas-authorized root-account courses appear. Create and assign a school tool preset; refresh one test course; reveal a password and verify it disappears automatically; rotate the course exit password; reset one assessment exit password; rotate its code; toggle SEB; and verify secret-free activity records.
  6. Repeat the launch as an instructor, sub-account administrator, and student and confirm the root-account dashboard is denied to each.
  7. Confirm the active Canvas theme loads the detector asset on a Classic Quiz /take page and a New Quiz assignment route.
  8. Confirm the client certificate profile is installed on an approved test device and the active public-key hash matches the service.
2

Instructor acceptance

  1. Launch the tool from a Canvas course and complete or re-run Canvas OAuth.
  2. Refresh the course. Confirm it finds the intended published Classic Quiz and New Quiz data.
  3. Configure a valid effective exit password, optional start password, selected course tools, and one quiz-only tool. Confirm the quiz-only tool does not appear in another assessment or in course defaults.
  4. Save one instructor-owned course tool, use Duplicate to courses, and select a second active course. Confirm the target receives the full launch/resource policy without replacing existing tools.
  5. Enable one Classic Quiz and one New Quiz. Confirm Canvas requires an access code and the management response does not reveal it.
  6. Change one protected setting, confirm the previous configuration becomes unsuitable for proof, and download a fresh configuration.
  7. Disable each assessment and confirm Canvas access-code protection is removed only through the intended action.
3

Student acceptance — normal browser

  1. Confirm /lti/config reports the correct course-navigation visibility for the LTI_COURSE_NAVIGATION_VISIBLE_TO_STUDENTS setting.
  2. Before acknowledging readiness, confirm the SEB-required page shows Setup check (recommended) beside Return to course and Open Safe Exam Browser but does not open the setup dialog automatically.
  3. Open each enabled assessment. Confirm the ordinary browser presents the protected launch/download flow rather than an access-code value.
  4. After the handoff opens SEB, quit SEB and select Return to course. Confirm it opens the Canvas course rather than the SEB-required page.
  5. Confirm the detector’s launch UI is available on the Canvas assessment route and approved tools are not exposed before a valid SEB proof.
  6. Confirm assessments without SEB enabled retain their normal Canvas access-code flow and never show the SEB-required prompt.
4

Student acceptance — Safe Exam Browser client

  1. Start from an SEB client with no usable Canvas browser session, open a fresh configuration, and verify it reaches Canvas through the generated session URL.
  2. Confirm the encrypted configuration opens only on a client with the managed private identity.
  3. For Classic Quiz and New Quiz, verify Config Key proof succeeds, the correct access-code prompt is filled once, and the assessment becomes available.
  4. Confirm disabled tools are unavailable and each selected tool opens only its intended launch/resources.
  5. Cancel a final Canvas confirmation. Verify no submission exit begins.
  6. Submit successfully. Verify Classic Quiz exit waits for Canvas’s completed-submission result and New Quiz exit waits for its authoritative result UI.
  7. Verify the validated quit action works after completion and native early quit follows the configured password policy.
The setup check at /seb/check is a device-readiness aid available to students before an exam. It confirms the correct private identity is installed and that the public-key hash matches the active service. Run it first when validating a new client or certificate profile, and direct students to it when troubleshooting SEB configuration decryption failures.

​
Commit Testbed Acceptance

For per-commit testing, maintain a stable self-hosted Canvas sandbox with one root administrator, one instructor, and one student account. Keep one published course named Safe Online Exam Commit Testbed with:
  • a published two-question Classic Quiz named Classic Quiz - Baseline;
  • a published two-question New Quiz named New Quiz - Baseline (when New Quizzes is enabled);
  • Safe Online Exam enabled for both assessments with a non-production exit password; and
  • the account-level LTI installation, API Developer Key, and detector theme loader pointed at the stable custom tool origin.
Record the /api/testbed/status response with each test result so that a failure maps to one commit, diff, image digest, build, and revision.

​
Regression Checklist

Before merge or release, confirm all applicable items are true:
  • npm run verify, npm run verify:postgres, and the relevant Compose/Playwright checks pass.
  • Public LTI, JWKS, health, detector, and Canvas OAuth callback URLs are unchanged, or Canvas configuration has been updated intentionally.
  • A settings change has tests showing that stale configuration and proof behavior is rejected.
  • Instructor and learner roles cannot reach each other’s privileged endpoints or UI paths.
  • Classic Quiz and New Quiz are both exercised when changing common assessment, configuration, or detector code.
  • The current certificate, configuration encryption, Config Key proof, student session handoff, start password, exit password, and URL-tool policy work together on supported client devices.
  • The browser console is clean on desktop and mobile app-shell routes.
  • The release uses the intended immutable image digest, runtime service account, PostgreSQL database, numbered secret versions, and public service URL.

​
Test Data and Safety

Use isolated Canvas courses and non-sensitive test accounts only. Never paste actual access codes, OAuth tokens, session URLs, private keys, .p12 files, or password values into test fixtures, snapshots, shell history, or issue reports. Clear temporary client configuration files and test assessment data according to the environment’s retention policy.