> ## Documentation Index
> Fetch the complete documentation index at: https://docs.safeonlineexam.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing and Acceptance for Safe Online Exam Deployments

> Test gates, local verification commands, and the Canvas and SEB role-based acceptance sequence required before Safe Online Exam goes to production.

Safe Online Exam testing is layered because the critical path crosses server code, Canvas, and a native Safe Exam Browser client. Automated checks protect application contracts; a full Canvas and SEB acceptance run validates the integration boundaries that cannot be fully reproduced in a unit test. This page covers every verification command, explains what each gate protects, and walks through the complete acceptance sequence for administrator, instructor, and student roles.

## Automated Verification Gates

### Installing Dependencies

Before running any gate, install dependencies using the enforced supply-chain procedure:

```bash theme={null}
npm run verify:dependency-policy
npm ci --ignore-scripts
npm run install:trusted
```

Run the dependency-policy check first because a contributor can change `.npmrc`, `package.json`, or `package-lock.json`. The `--ignore-scripts` flag keeps lifecycle scripts disabled even if repository configuration changes. `install:trusted` verifies the complete lockfile, then rebuilds and executes only `esbuild`. The policy enforces npm 11.19.0, exact direct versions, the public registry, SHA-512 integrity metadata, and rejects packages published less than three days ago.

Run the network-backed supply-chain gate before a release or when dependencies change:

```bash theme={null}
npm run verify:supply-chain
```

This verifies npm registry signatures and available provenance attestations, then rejects production vulnerabilities.

### Individual Gates

Run individual gates when iterating on a specific area:

```bash theme={null}
npm run typecheck
npm run lint
npm run format:check
npm run test:coverage
npm run build
```

### Full Local Verification

<Steps>
  <Step title="Standard non-browser gate">
    Run `npm run verify` to check release metadata consistency, then execute type checking, linting, Prettier verification, Vitest coverage tests, and the production build.

    ```bash theme={null}
    npm run verify
    ```
  </Step>

  <Step title="PostgreSQL integration gate">
    Run `npm run verify:postgres` to start PostgreSQL 17, apply migrations in isolated schemas, and execute repository atomicity and concurrency tests.

    ```bash theme={null}
    npm run verify:postgres
    ```
  </Step>

  <Step title="Browser E2E gate">
    Run the Playwright smoke suite to confirm public routes, detector availability, and React app-shell rendering.

    ```bash theme={null}
    npm run test:e2e
    ```
  </Step>
</Steps>

<Note>
  The deploy pipeline runs both `npm run verify` and `npm run verify:postgres` before promotion. GitHub Actions also runs the Compose smoke test for every pull request and push to `main`.
</Note>

If Chromium is not installed for E2E tests, add it before retrying:

```bash theme={null}
npm exec --offline -- playwright install chromium
```

### Compose Smoke Test

Verify the production topology end-to-end, including image build, migration/readiness gates, public health and JWKS probes, and named-volume persistence:

```bash theme={null}
bash scripts/compose-smoke.sh
```

CI builds the image once through Buildx and supplies it via `COMPOSE_SMOKE_IMAGE` with `COMPOSE_SMOKE_SKIP_BUILD=true` to avoid rebuilding. The script restarts the app and confirms the PostgreSQL volume retained a written row.

### Local Production-Build Smoke Run

Use this after `npm run build` to diagnose the deployed runtime shape:

```bash theme={null}
HOST=127.0.0.1 \
USE_IN_MEMORY_STORE=true \
TOOL_URL=http://localhost:8080 \
LTI_CLIENT_ID=test-client \
CANVAS_API_CLIENT_ID=test \
CANVAS_API_CLIENT_SECRET=test \
npm start
```

Then verify the key public endpoints:

```bash theme={null}
curl -fsS http://127.0.0.1:8080/health
curl -fsS http://127.0.0.1:8080/.well-known/jwks.json
curl -fsS http://127.0.0.1:8080/lti/config
curl -fsS http://127.0.0.1:8080/js/canvas-seb-detector.js | head
```

<Note>
  This mode cannot validate Canvas OAuth, PostgreSQL, certificate decryption, or real SEB runtime behavior. Use it only to confirm the server starts and public routes respond correctly.
</Note>

## Test Coverage by Layer

The table below maps every test layer to the trust boundary it protects. A high coverage percentage alone is insufficient — the critical requirement is that each test exercises the same boundary it claims to protect: signed LTI data for identity, server proof for access-code release, and Canvas-authored completion for exit behavior.

| Layer                                  | Location                                                    | What it protects                                                                                                                                |
| -------------------------------------- | ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| Shared domain policy                   | `test/server/models.test.ts` and related                    | Content IDs, roles, URL-rule/tool normalization, course defaults, and password-state behavior                                                   |
| Configuration and deployment artifacts | `app-config.test.ts`, `deployment-hardening-static.test.ts` | Runtime validation, public/private artifact handling, image-digest deployment, and documented deployment invariants                             |
| LTI and OAuth                          | LTI/OAuth controller and service tests                      | Signed-claim validation, browser binding, state replay prevention, role routing, OAuth state binding, cross-site rejection, and token ownership |
| Account administration                 | `admin-dashboard.test.ts` and action-token tests            | Root-account/course binding, administrator request integrity, recovery actions, course connection, and durable bulk rollout                     |
| Persistence and concurrency            | Repository/session/assessment and PostgreSQL tests          | Atomic claims, one-time consumption, cleanup, session storage, distributed locks, and Canvas/database consistency                               |
| SEB configuration and proof            | `seb-*.test.ts`                                             | Plist generation, encryption, Config Key validation, proof redemption, handoff records, exit grants, and password rules                         |
| Detector                               | `canvas-seb-detector-script.test.ts`, static-asset tests    | Loading, Canvas route handling, access-code flow, approved tools, completion detection, and exit behavior                                       |
| Browser app shell                      | `test/e2e/app-shell.spec.ts`                                | Server startup, public metadata routes, lazy React routes, compressed assets, content-hashed caching, and browser console errors                |

### Playwright E2E Scope

`npm run test:e2e` builds the application and starts it locally with `USE_IN_MEMORY_STORE=true`, `TOOL_URL=http://localhost:8080`, and test-only LTI and Canvas API values. It runs Chromium in desktop and mobile projects and confirms:

* public health, JWKS, and LTI metadata responses;
* detector availability on both supported Canvas assessment URL patterns;
* selected React app-shell behavior; and
* error-free page rendering on desktop and mobile.

It does **not** contact Canvas or start a native SEB client.

## Canvas and SEB Acceptance Sequence

Run the full acceptance sequence after any deployment that affects authentication, Canvas interaction, settings, configuration generation, the detector, certificate material, or exit behavior. Use separate administrator, instructor, and student accounts against a stable Canvas sandbox with dedicated synthetic accounts only.

<Note>
  The complete three-role acceptance sequence — administrator, instructor, and student in a real SEB client — is required before any deployment is considered production-ready. Automated gates do not replace this sequence.
</Note>

<Steps>
  <Step title="Administrator acceptance">
    1. Confirm `/health`, `/lti/config`, JWKS, and detector endpoints on the deployed URL.
    2. Confirm Canvas stores the current LTI client ID, deployment ID, login URL, target link URI, and JWKS URL from `/lti/config`.
    3. Confirm the API OAuth Developer Key has the complete scope set described in the [Canvas setup guide](deployment/canvas-setup).
    4. Open **Safe Online Exam Admin** from root-account navigation and complete its separate OAuth authorization.
    5. Confirm only Canvas-authorized root-account courses appear. Create and assign a school tool preset; refresh one test course; reveal a password and verify it disappears automatically; rotate the course exit password; reset one assessment exit password; rotate its code; toggle SEB; and verify secret-free activity records.
    6. Repeat the launch as an instructor, sub-account administrator, and student and confirm the root-account dashboard is denied to each.
    7. Confirm the active Canvas theme loads the detector asset on a Classic Quiz `/take` page and a New Quiz assignment route.
    8. Confirm the client certificate profile is installed on an approved test device and the active public-key hash matches the service.
  </Step>

  <Step title="Instructor acceptance">
    1. Launch the tool from a Canvas course and complete or re-run Canvas OAuth.
    2. Refresh the course. Confirm it finds the intended published Classic Quiz and New Quiz data.
    3. Configure a valid effective exit password, optional start password, selected course tools, and one quiz-only tool. Confirm the quiz-only tool does not appear in another assessment or in course defaults.
    4. Save one instructor-owned course tool, use **Duplicate to courses**, and select a second active course. Confirm the target receives the full launch/resource policy without replacing existing tools.
    5. Enable one Classic Quiz and one New Quiz. Confirm Canvas requires an access code and the management response does not reveal it.
    6. Change one protected setting, confirm the previous configuration becomes unsuitable for proof, and download a fresh configuration.
    7. Disable each assessment and confirm Canvas access-code protection is removed only through the intended action.
  </Step>

  <Step title="Student acceptance — normal browser">
    1. Confirm `/lti/config` reports the correct course-navigation visibility for the `LTI_COURSE_NAVIGATION_VISIBLE_TO_STUDENTS` setting.
    2. Before acknowledging readiness, confirm the SEB-required page shows **Setup check (recommended)** beside **Return to course** and **Open Safe Exam Browser** but does not open the setup dialog automatically.
    3. Open each enabled assessment. Confirm the ordinary browser presents the protected launch/download flow rather than an access-code value.
    4. After the handoff opens SEB, quit SEB and select **Return to course**. Confirm it opens the Canvas course rather than the SEB-required page.
    5. Confirm the detector's launch UI is available on the Canvas assessment route and approved tools are not exposed before a valid SEB proof.
    6. Confirm assessments without SEB enabled retain their normal Canvas access-code flow and never show the SEB-required prompt.
  </Step>

  <Step title="Student acceptance — Safe Exam Browser client">
    1. Start from an SEB client with no usable Canvas browser session, open a fresh configuration, and verify it reaches Canvas through the generated session URL.
    2. Confirm the encrypted configuration opens only on a client with the managed private identity.
    3. For Classic Quiz and New Quiz, verify Config Key proof succeeds, the correct access-code prompt is filled once, and the assessment becomes available.
    4. Confirm disabled tools are unavailable and each selected tool opens only its intended launch/resources.
    5. Cancel a final Canvas confirmation. Verify no submission exit begins.
    6. Submit successfully. Verify Classic Quiz exit waits for Canvas's completed-submission result and New Quiz exit waits for its authoritative result UI.
    7. Verify the validated quit action works after completion and native early quit follows the configured password policy.
  </Step>
</Steps>

<Tip>
  The setup check at `/seb/check` is a device-readiness aid available to students before an exam. It confirms the correct private identity is installed and that the public-key hash matches the active service. Run it first when validating a new client or certificate profile, and direct students to it when troubleshooting SEB configuration decryption failures.
</Tip>

## Commit Testbed Acceptance

For per-commit testing, maintain a stable self-hosted Canvas sandbox with one root administrator, one instructor, and one student account. Keep one published course named `Safe Online Exam Commit Testbed` with:

* a published two-question Classic Quiz named `Classic Quiz - Baseline`;
* a published two-question New Quiz named `New Quiz - Baseline` (when New Quizzes is enabled);
* Safe Online Exam enabled for both assessments with a non-production exit password; and
* the account-level LTI installation, API Developer Key, and detector theme loader pointed at the stable custom tool origin.

Record the `/api/testbed/status` response with each test result so that a failure maps to one commit, diff, image digest, build, and revision.

## Regression Checklist

Before merge or release, confirm all applicable items are true:

<Accordion title="View full regression checklist">
  * `npm run verify`, `npm run verify:postgres`, and the relevant Compose/Playwright checks pass.
  * Public LTI, JWKS, health, detector, and Canvas OAuth callback URLs are unchanged, or Canvas configuration has been updated intentionally.
  * A settings change has tests showing that stale configuration and proof behavior is rejected.
  * Instructor and learner roles cannot reach each other's privileged endpoints or UI paths.
  * Classic Quiz and New Quiz are both exercised when changing common assessment, configuration, or detector code.
  * The current certificate, configuration encryption, Config Key proof, student session handoff, start password, exit password, and URL-tool policy work together on supported client devices.
  * The browser console is clean on desktop and mobile app-shell routes.
  * The release uses the intended immutable image digest, runtime service account, PostgreSQL database, numbered secret versions, and public service URL.
</Accordion>

## Test Data and Safety

Use isolated Canvas courses and non-sensitive test accounts only. Never paste actual access codes, OAuth tokens, session URLs, private keys, `.p12` files, or password values into test fixtures, snapshots, shell history, or issue reports. Clear temporary client configuration files and test assessment data according to the environment's retention policy.