16. Testing Strategy
Current Test Coverage
Test Framework
| Tool |
Purpose |
| Vitest |
Test runner (integration and unit tests) |
| better-sqlite3 |
In-memory database for test isolation |
Existing Tests
| Test File |
Type |
Coverage |
server/__tests__/integration/scheduler-api.test.js |
Integration |
Scheduler CRUD, enable/disable, execution |
server/__tests__/integration/execution-log-api.test.js |
Integration |
Execution log listing, filtering |
server/__tests__/integration/tasks-api.test.js |
Integration |
Tasks API (external), CRUD, run |
server/__tests__/integration/meeting-api.test.js |
Integration |
Meeting API (external), start/stop |
server/database/__tests__/schedulerDb.test.js |
Unit |
Scheduler database operations |
server/services/__tests__/scheduler.test.js |
Unit |
Scheduler service logic |
server/services/__tests__/schedulerLogger.test.js |
Unit |
Scheduler logging |
server/services/elo/__tests__/promptMaterializer.test.js |
Unit |
ELO prompt materialization |
src/utils/__tests__/workflowParser.test.js |
Unit |
Workflow markdown parsing |
src/utils/__tests__/workflowSerializer.test.js |
Unit |
Workflow markdown serialization |
src/reducers/__tests__/projectReducer.test.js |
Unit |
Project state reducer |
Test Helpers
server/__tests__/integration/helpers.js provides:
- JWT token generation for test users
- HTTP request helpers with auth headers
- Test database setup/teardown
Test Configuration
# Run integration tests
npm run test:integration # Uses --env-file=.env.test
# Run specific test suites
npm run test:convert # Document conversion tests
npm run test:bug-report # Bug reporting smoke test
npm run test:mcp # MCP integration tests
Validation Methods
Build Validation
| Check |
Command |
Blocking? |
| ESLint (server only) |
npm run lint |
Yes (CI blocks) |
| Frontend build |
npx vite build |
Yes |
| TypeScript |
N/A (no TypeScript) |
N/A |
Note: ESLint only covers server/ directory. Frontend is validated via successful Vite build.
Known Build Artifacts
- CSS warning about
{ at line 1901 is pre-existing and not a problem
Critical Flows That Must Be Tested
P0 -- Must Not Break
| Flow |
Current Coverage |
| User registration (first user = admin) |
Not tested |
| User login / JWT generation |
Not tested |
| Chat message → Claude CLI → streaming response |
Not tested |
| File upload and download |
Not tested |
| Scheduled prompt execution |
Integration tests exist |
P1 -- Should Be Tested
| Flow |
Current Coverage |
| Password reset flow |
Not tested |
| Skill CRUD and execution |
Not tested |
| Admin user management |
Not tested |
| AI provider configuration |
Not tested |
| Onboarding flow |
Not tested |
| Cloud drive OAuth + mount |
Not tested |
| Git operations |
Not tested |
P2 -- Nice to Have
| Flow |
Current Coverage |
| Meeting transcription |
Integration tests exist |
| MCP service registration |
MCP tests exist |
| Analytics report generation |
Not tested |
| Output style management |
Not tested |
| Workflow editor parsing |
Unit tests exist |
CI Integration
GitHub Actions Pipeline
PR opened → ci.yml:
1. npm install
2. npm run lint (server)
3. npx vite build (frontend)
4. npm run test:integration (optional)
5. CodeQL analysis (security)
6. Semgrep scan (SAST)
7. OSV scan (dependencies)
Security Scanning in CI
| Scanner |
Focus |
Blocking? |
| CodeQL |
Code patterns, injection, auth issues |
Advisory |
| Semgrep |
SAST rules, OWASP patterns |
Advisory |
| OSV |
Known dependency vulnerabilities |
Advisory |
| Container scan |
Docker image vulnerabilities |
Advisory |
Test Accounts
Local Development
- Username:
lindsay / Password: password
Docker Development
- First user registered becomes admin
- No pre-seeded test accounts
Gaps and Recommendations
Major Gaps
- No auth tests -- Registration, login, JWT validation, admin checks untested
- No chat/CLI tests -- Core product functionality has zero test coverage
- No file operation tests -- Upload, download, path traversal prevention untested
- No E2E tests -- No Playwright or Cypress tests for full user flows
- No frontend component tests -- 80+ React components with no unit tests (except reducers)
- No performance tests -- No load testing, no benchmarks
- No accessibility tests -- No automated a11y checks
Recommended Priority
- Integration tests for auth flow (registration, login, protected routes)
- Integration tests for file operations (upload, safePath validation)
- E2E tests for chat → response → tool call flow
- E2E tests for onboarding flow
- Performance baseline for WebSocket streaming
