Test branch: Add --offline flag to codegen command

[jlevy]

⚠️ TEST BRANCH - NOT FOR MERGE

This PR is for review and testing purposes only. It includes modifications to enable direct GitHub installation:

npm install github:jlevy/convex-js#offline-flag

or

npx --yes github:jlevy/convex-js#offline-flag codegen --offline

---

Component Type Preservation

Problem Solved: Offline codegen now preserves existing component types from previously generated api.(d.)ts files and never downgrades them to AnyComponents. If no prior component types exist, offline emits the stub with a warning; otherwise it reuses the preserved types, keeping offline runs idempotent for component users.

How it works:

1. Before regenerating, reads existing _generated/api.ts or _generated/api.d.ts.
2. Uses the TypeScript Compiler API to extract export declare const components: {...}.
3. Regenerates api and internal exports from local files.
4. Re-injects preserved component types; uses AnyComponents only when no prior types exist.
5. Post-pass guard ensures stubs are replaced with preserved types if a template ever emitted AnyComponents.

Implementation Highlights:

• Preservation logic checks both .ts and .d.ts outputs and post-fixes stubs when preserved types exist.
• Idempotent offline runs: once component types exist from an online run, offline will not drop them.
• No new dependencies.

New files:

• src/cli/lib/componentTypePreservation.ts - TypeScript AST extraction utilities
• src/cli/lib/componentTypePreservation.test.ts - 30 unit tests (edge cases covered)
• src/cli/lib/__tests__/offlineComponentPreservation.test.ts - Additional guard for stub replacement

---

Changes Required for Upstream PR

To create a mergeable PR for get-convex/convex-js, the following must be removed/reverted:

1. Remove dist/ folder

The entire dist/ directory was force-added (bypassing .gitignore) to enable GitHub installation without build toolchain.

2. Revert package.json changes

  "bin": {
-    "convex": "bin/main.js",
-    "convex-dev": "bin/main-dev"
+    "convex": "bin/main-dev",
+    "convex-bundled": "bin/main.js"
  },
  "scripts": {
-    "prepare:local": "npm run build",
+    "prepare": "npm run build",

3. Remove docs/ folder

• docs/impl-offline-codegen.md - implementation tracking
• docs/spec-offline-codegen.md - feature specification

4. Separate or include unrelated fix

src/cli/lib/mcp/tools/index.ts - Removes unused ToolSchema import. This is a valid cleanup that could be included in upstream or submitted as a separate PR.

---

Core Feature Files (for upstream)

File	Change
src/cli/codegen.ts	Add --offline flag to command definition
src/cli/lib/codegen.ts	Add offline: boolean, component preservation logic (handles .ts and .d.ts), and offline warning/fallback + post-pass stub replacement
src/cli/lib/components.ts	Route to local codegen when --offline is set
src/cli/codegen_templates/api.ts	Add includeComponentsStub and preservedComponentTypes options
src/cli/lib/componentTypePreservation.ts	NEW - TypeScript AST extraction utilities
src/cli/lib/componentTypePreservation.test.ts	NEW - 30 unit tests
src/cli/lib/offlineCodegen.test.ts	NEW - 13 unit tests for offline mode logic
src/cli/lib/__tests__/offlineComponentPreservation.test.ts	NEW - guard to ensure stubs are replaced when preserved types exist

---

Problem

npx convex codegen currently requires a backend connection and authentication to generate TypeScript types, making it unsuitable for:

• CI/CD pipelines that need type checking without deploy keys (fixes Unable to run npx convex codegen on Vercel without CONVEX_DEPLOY_KEY get-convex/convex-js#81)
• AI agent workflows that need offline, no-auth code generation
• Offline development scenarios (planes, trains, poor connectivity)

Solution

Add an --offline flag to npx convex codegen that generates types purely from local files without requiring backend communication or authentication.

npx convex codegen --offline

Key insight

Offline mode provides identical type safety to backend mode for standard applications. The CLI already has a dynamic type generation system that uses TypeScript's type inference to provide full type safety from local files:

• DataModelFromSchemaDefinition<typeof schema> infers all table types from local schema
• ApiFromModules<{...}> infers all function signatures from local code

The backend connection was only needed for validation, not for type safety.

Trade-offs in Offline Mode

Feature	Backend Mode	Offline Mode
Function types	✅ Full	✅ Full (identical)
Data model types	✅ Full	✅ Full (identical)
Component types	✅ Full	✅ Preserved from previous run; warns and falls back to AnyComponents when none available
Schema validation	✅ At codegen	⚠️ At deploy
Auth required	✅ Yes	❌ No
Network required	✅ Yes	❌ No

Testing

All 430 tests pass (44 new tests added):

npm test
# ✓ 430 passed, 17 skipped

New test files:

• componentTypePreservation.test.ts - 30 tests for AST extraction (including edge cases)
• offlineCodegen.test.ts - 13 tests for offline mode behavior
• __tests__/offlineComponentPreservation.test.ts - stub replacement guard

How to Test This Branch

npm install github:jlevy/convex-js#offline-flag
npx convex codegen --offline
npx tsc --noEmit  # Verify types work

Example CI/CD Usage

Before (requires secrets):

env:
  CONVEX_DEPLOY_KEY: ${{ secrets.CONVEX_DEPLOY_KEY }}
steps:
  - run: npx convex codegen
  - run: tsc --noEmit

After (no secrets needed):

steps:
  - run: npx convex codegen --offline
  - run: tsc --noEmit  # Full type safety!

Related Issues

• Fixes Unable to run npx convex codegen on Vercel without CONVEX_DEPLOY_KEY get-convex/convex-js#81 - Unable to run codegen on Vercel without CONVEX_DEPLOY_KEY
• Related to Convex dev local offline error get-convex/convex-js#73 - Offline development error

Contributor Agreement

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.