Docs/Testing & Quality/End-to-End Suites

End-to-End Suites

Everything under web/e2e/: the Playwright business-case suite, the smaller TypeScript specs, three Playwright configurations, and the @mvp-acceptance slice that the (currently manual) GitHub workflow runs. The suite header states the convention: a real running core + sidecar + Postgres + Keycloak + Redis stack, no mocking anywhere. Setup detail lives in web/e2e/README.md.

Three Playwright configurations

ConfigTest dirBehaviour as configured
web/playwright.config.js (the business-case suite)./e2e/specsSequential: fullyParallel: false, workers: 1, retries: 0, test timeout 120 s, expect timeout 15 s, action/navigation timeout 15 s. Headed with a 1000 ms slowMo by default; set E2E_HEADLESS=true and E2E_SLOWMO_MS=0 for a fast run. Base URL from E2E_BASE_URL (default http://localhost:3000). Sequential on purpose: shared tenant state and the Redis-backed signup rate limiter.
web/playwright.staging.config.ts./e2eTargets https://app.staging.example.com, 1 retry, 4 workers, longer action timeout (15 s), no web server.
Staging config target

The staging config hard-codes a staging hostname (app.staging.example.com in these docs). Point it at your own staging domain, confirm the target before running it, and never describe a run against it as production-verified.

Business-case suite (web/e2e/specs/, 22 files)

SpecCovers (per the suite README)
00 full-platform-showcaseOne slow narrated walkthrough, signup through channels, for a human to watch (npm run e2e:showcase)
01 signup-and-loginSignup form, duplicate org/email rejection, Keycloak login, logout
02 providers-agents-chatProvider activation (and invalid-key rejection), agent creation, streaming chat
03 mcp-registryRegister / enable / disable an MCP tool server
04 knowledge-collections-and-multimodalCollections, .md/PDF/DOCX/image/audio/video ingestion, search, async job polling
05 agent-access-controlPer-agent tool and knowledge allow-lists
06 admin-users-and-rolesCreate teammate, assign roles, forced password change
07 approvals-and-auditHITL approve/reject, per-action policy override, audit log
08 channels-whatsappTwilio credential activation (and rejection), default agent routing
09 multi-tenant-isolationTwo tenants, zero cross-visibility, UI and direct API probe
10 api-securityWebhook signature rejection, unauthenticated / under-privileged 401/403
11 workflow-designerCanvas build/save/validate, free-tier 3-workflow cap, triggers and a synchronous run (API-level)
12 quick-taskDelivery-number gate, example chips, task history
13 user-created-toolsSandboxed SpEL tool create/test/toggle/delete
14 issues-portalAdmin-only gate, status tabs, failed run recorded as an issue and resolved
15 agent-templatesAudience filtering; a template pre-fills the create form
16 google-integrationAdmin-only gate, disconnected state, OAuth callback parameters
17 social-channelsMessenger/Instagram/Slack/Discord/X validation and X callback handling
18 subscription-tiers5-agent free-tier cap and upgrade dialog; POST /api/tenant/upgrade gate
19 web-admin-platform-appSeparate web-admin app: a tenant-admin without platform-admin sees “Access restricted”
20 telegram-and-email-channelsFabricated bot token / unreachable IMAP host rejected by each channel's own check
21 agent-paymentsNot in the README table; its header says it covers the /admin/payments mandate console up to the grant wizard review step, deliberately without completing a grant, and that it had not yet been run when written

Also under web/e2e/ (run by the TypeScript config): axon-redesign.spec.ts, admin-panel.spec.ts, whatsapp-integration.spec.ts, core-api.spec.ts, signup-onboarding.spec.ts, one spec for an optional external integration, and edge-faq-eval.spec.ts, which collects the report from the /eval/edge-faq page (the on-device FAQ eval, run inside the browser).

Design conventions worth knowing

cd web
npm run e2e            # headless, all specs
npm run e2e:ui         # interactive UI mode
npm run e2e:report     # open the last HTML report
npm run e2e:showcase   # headed full-platform walkthrough

@mvp-acceptance slice and the e2e workflow

Six journeys are tagged in the test(...) title, per the README (six passing-tagged tests; workflow-designer contributes two): signup and login (01), agent-template pre-fill (15), knowledge .md upload and search (04), workflow create as draft and synchronous run (11, two tests), and the web-admin access-restricted negative path (19). A tagged FREE-tier chat journey in spec 02 is recorded as test.fixme, not passing.

flowchart LR A["workflow_dispatch e2e.yml"] --> B["boot-stack composite action: compose + ci overlay, health gate"] B --> C["npm install + playwright install chromium"] C --> D["timeout 900 npx playwright test --grep @mvp-acceptance"] D -->|failure| E["upload traces + service logs"] D --> F["compose down -v"]
Not documented in the repository

Current pass rates, the count of individual test cases, the date of the last full run, and whether the last full run was green are not recorded in the files read.

Evidence & references

Related: Test Strategy · Quality Gates · Docker Compose Stack