Rewrite graph-cli in Rust #6282
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Move dev command logic to commands/dev.rs module - Add subcommand structure with stubs for all graph-cli commands: codegen, build, deploy, init, add, create, remove, auth, publish, test, clean - Add version output showing graph-cli compatibility version (0.98.1) - Preserve all existing dev command functionality and options This is Phase 1 of the gnd CLI expansion to become a drop-in replacement for the TypeScript graph-cli. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the `gnd clean` command to remove build artifacts and generated files. - Removes `generated/` and `build/` directories by default - Supports custom paths via --codegen-dir and --build-dir flags - Includes unit tests for both success and missing directory cases - Matches graph-cli clean command behavior Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the `gnd auth` command to save deploy keys for Graph Node authentication. - Stores keys in ~/.graph-cli.json (compatible with TS graph-cli) - Supports custom node URLs via --node flag - Defaults to Subgraph Studio URL - Validates Studio deploy key format (32 hex chars) - Includes unit tests for key storage and retrieval Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add a JSON-RPC client for communicating with Graph Node's admin API. This client is used by the create and remove commands to register and unregister subgraph names. Features: - HTTP/HTTPS support with protocol validation - Optional access token authentication via Bearer header - User-Agent header with gnd version - 120 second timeout for long operations - create_subgraph and remove_subgraph methods Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add commands to register and unregister subgraph names with a Graph Node. Both commands: - Use GraphNodeClient for JSON-RPC communication - Support --node/-g flag for Graph Node URL (required) - Support --access-token flag for authentication - Automatically read deploy key from ~/.graph-cli.json if no token provided - Include unit tests for CLI argument parsing Usage: gnd create <name> --node <url> [--access-token <token>] gnd remove <name> --node <url> [--access-token <token>] Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the output module with spinner functionality matching the TypeScript graph-cli output format (gluegun/ora style). This provides: - Spinner struct for progress indicators with colored output - with_spinner() helper for wrapping long-running operations - SpinnerResult for operations that can warn or fail - Checkmark/cross/warning symbols matching TS CLI output The module uses indicatif and console crates for terminal handling. Co-Authored-By: Claude Opus 4.5 <[email protected]>
This commit adds the foundation for code generation functionality: - codegen/typescript.rs: AST builders for generating TypeScript/AssemblyScript code (classes, methods, types, imports) - codegen/types.rs: Type conversion utilities between GraphQL, Ethereum ABI, and AssemblyScript types - codegen/schema.rs: SchemaCodeGenerator that generates entity classes from GraphQL schemas, matching the TS CLI output format The schema code generator supports: - Entity classes with constructor, save(), load(), loadInBlock() - Field getters and setters with proper type conversions - Nullable field handling with proper null checks - Entity reference fields (stored as string IDs) - Derived fields with loader classes - Multiple ID field types (String, Bytes, Int8) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implements codegen/abi.rs that generates AssemblyScript bindings from Ethereum contract ABIs: - Event classes with typed parameters and getters - Call classes for function calls with inputs/outputs - Smart contract class with typed call methods - Tuple handling for nested struct types - Support for indexed event parameters - Reserved word escaping for AssemblyScript The generated code matches the format of the TypeScript graph-cli. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implements codegen/template.rs that generates AssemblyScript classes for subgraph data source templates: - Template class extending DataSourceTemplate - Static create() method for creating new data sources - Static createWithContext() method with context parameter - Support for Ethereum (Address param) and file (cid param) templates Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implements formatter.rs that shells out to Prettier to format generated TypeScript/AssemblyScript code: - format_typescript() for strict formatting with error handling - try_format_typescript() for graceful fallback when prettier unavailable - Tries npx, pnpx, and global prettier installations - is_prettier_available() utility to check for prettier Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implements the `gnd codegen` command that generates AssemblyScript types from a subgraph manifest: - Parses subgraph.yaml to extract schema, ABIs, and templates - Generates schema.ts from GraphQL schema using SchemaCodeGenerator - Generates <ABI>.ts for each ABI using AbiCodeGenerator - Generates templates.ts for data source templates - Formats output with Prettier when available - Adds --output-dir, --skip-migrations, --watch, --ipfs flags Also adds: - Step/StepResult output utilities for consistent CLI formatting - serde_yaml dependency for manifest parsing Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implements the --watch/-w flag for the codegen command that: - Watches the manifest, schema, and ABI files for changes - Automatically regenerates types when files are modified - Uses debouncing to batch rapid file changes - Shows informative messages about detected changes Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implements the migrations framework matching the TypeScript graph-cli behavior. Migrations automatically update subgraph manifests to newer versions based on the installed graph-ts dependency. Spec version migrations: - 0.0.1 -> 0.0.2: Bump for top-level templates - 0.0.2/0.0.3 -> 0.0.4: Bump for feature management API version migrations (require specific graph-ts versions): - 0.0.1 -> 0.0.2: Requires graph-ts > 0.5.1 - 0.0.2 -> 0.0.3: Requires graph-ts > 0.12.0 - 0.0.3 -> 0.0.4: Requires graph-ts > 0.17.0 - 0.0.4 -> 0.0.5: Requires graph-ts >= 0.22.0 - 0.0.5 -> 0.0.6: Requires graph-ts >= 0.24.0 The codegen command now runs migrations by default, with a --skip-migrations flag to bypass them. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Adds the build command which compiles AssemblyScript mappings to WebAssembly using the asc compiler. The implementation includes: Compiler module (compiler/asc.rs): - Wrapper around the asc command-line tool - Detection of graph-ts package and node_modules directories - Compilation with flags matching graph-cli behavior Build command (commands/build.rs): - Compiles data source and template mappings - Copies schema and ABI files to build directory - Generates output manifest with updated paths - Watch mode for automatic rebuilding on file changes - Support for both wasm and wast output formats Key features: - Caching of compiled files by content hash (SHA1) - Migration support (runs migrations unless --skip-migrations) - Proper error handling and progress output Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the deploy command that deploys a subgraph to a Graph Node via JSON-RPC. The command: - Takes subgraph name, manifest path, node URL, IPFS URL, version label - Supports --ipfs-hash for pre-uploaded manifests - Calls the subgraph_deploy JSON-RPC method - Displays playground and queries URLs on success Also extend GraphNodeClient with deploy_subgraph method that sends the subgraph_deploy JSON-RPC request with name, ipfs_hash, version_label, and debug_fork parameters. Note: IPFS upload during build is not yet implemented; users must provide --ipfs-hash when deploying pre-built subgraphs. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add IPFS upload functionality to the build command using the Kubo RPC
API. When --ipfs is provided, the build command now:
- Creates an IpfsClient using the Kubo HTTP API
- Uploads schema, ABI, and WASM mapping files to IPFS
- Updates the manifest with IPFS links (using {"/": "/ipfs/<hash>"} format)
- Uploads the updated manifest and returns its hash
The deploy command now uses this functionality to automatically build
and upload subgraphs before deploying, eliminating the need for users
to manually provide --ipfs-hash.
Changes:
- Add services/ipfs.rs with IpfsClient for Kubo RPC API
- Update build command to return BuildResult with optional IPFS hash
- Add upload_to_ipfs() and helpers for manifest transformation
- Update deploy command to use build's IPFS upload
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the test command that runs Matchstick tests for subgraphs. The command supports both binary and Docker execution modes: Binary mode (default): - Looks for Matchstick in node_modules/.bin or PATH - Supports -c/--coverage for coverage reporting - Supports -r/--recompile for force recompilation Docker mode (-d/--docker): - Runs tests in a Docker container - Automatically generates Dockerfile if missing - Builds matchstick image if needed Flags match graph-cli test command for compatibility. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the init command that creates new subgraph projects. Currently supports --from-example mode which: - Clones the graphprotocol/example-subgraph repository - Removes existing .git directory and optionally initializes a fresh repo - Installs dependencies via pnpm or npm (unless --skip-install) Also defines flags for future --from-contract and --from-subgraph modes which will require additional infrastructure (Etherscan API, IPFS fetch). Flags match graph-cli init command for compatibility. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the add command that adds new data sources to existing subgraphs. Currently provides helpful error message explaining the requirements for full implementation: - ABI fetching from Etherscan/Sourcify - Schema generation for contract events - Mapping stub generation - Manifest modification Validates contract address format and provides workaround instructions for manual addition of data sources. Flags match graph-cli add command for compatibility. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add the publish command that publishes subgraphs to The Graph's decentralized network. Currently provides helpful error message explaining the requirements for full implementation: - Integration with The Graph Network subgraph - Wallet connection for signing transactions - GRT token handling for indexer rewards Provides workaround instructions via Subgraph Studio UI. Flags match graph-cli publish command for compatibility. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add config/networks module for parsing and applying networks.json configuration files, which allow deploying the same subgraph to different networks with network-specific contract addresses and start blocks. Key features: - Load and parse networks.json configuration files - Apply network config to manifest YAML (addresses, start blocks) - Initialize networks.json from existing manifest - Update networks.json with new data source entries Integrate into build command: - Add --network flag to apply network config before build - Check network file exists when --network is specified - Update manifest in place with network-specific values Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add services/contract module that provides functionality to fetch verified contract ABIs and metadata from block explorer APIs (Etherscan, Blockscout) and Sourcify. Key features: - NetworksRegistry for loading/parsing The Graph Networks Registry - ContractService for fetching ABIs, contract names, and start blocks - Support for Etherscan-compatible APIs (etherscan, blockscout) - Support for Sourcify API as primary source - Automatic network lookup by ID or alias - Environment variable substitution in API URLs - Fallback URLs for common networks This enables: - `gnd init --from-contract` mode - `gnd add` with automatic ABI fetching Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add scaffold module that generates all necessary files for a new subgraph: - subgraph.yaml manifest with data source configuration - schema.graphql with entity types based on contract events - AssemblyScript mapping handlers for events - package.json, tsconfig.json, .gitignore Implement init_from_contract mode that: - Fetches contract ABI from Etherscan/Sourcify (via ContractService) - Supports --abi flag to provide local ABI file - Extracts events from ABI for entity/handler generation - Supports --index-events to generate event-indexing handlers - Optionally installs npm dependencies - Optionally initializes git repository This enables the primary workflow: `gnd init --from-contract <address>` Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implement the full `gnd add` command that adds a new data source to an existing subgraph: Features: - Fetches contract ABI from Etherscan/Sourcify if not provided - Adds ABI file to abis/ directory - Generates entity types and appends to schema.graphql - Creates mapping file with event handlers - Updates subgraph.yaml with new data source entry - Supports --merge-entities to skip existing entities - Auto-detects network from existing manifest Usage: gnd add 0x1234... --abi MyContract.json gnd add 0x1234... --network goerli gnd add 0x1234... --contract-name MyToken --start-block 12345678 Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Remove unused imports (HashMap, Serialize, slog::info) - Add #[allow(dead_code)] to utility functions and types that are defined for future use but not yet integrated - Fix BuildResult struct warning Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add unit tests for helper functions in the add command: - solidity_to_graphql type conversion - sanitize_field_name for GraphQL field naming - to_kebab_case for file naming - generate_event_entity for GraphQL entity generation - generate_event_handler for mapping handler generation - generate_mapping for full mapping file content - Error handling for invalid addresses and missing manifests Co-Authored-By: Claude Opus 4.5 <[email protected]>
When no --node URL is provided, the deploy command now defaults to Subgraph Studio (https://api.studio.thegraph.com/deploy/), matching the behavior of the TypeScript graph-cli. This simplifies the common workflow of deploying to Subgraph Studio without requiring users to remember the URL. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add tests for: - format_network function with full_name and without - SourceType display implementations - SimpleNetworkCompleter autocomplete functionality - Exact ID matches - Alias matches - Empty input returning all networks - Non-matching input returning empty results Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add tests for: - All Protocol enum variants' Display implementation - from_subgraph mode in should_run_interactive - InitOpt default values verification Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add test to verify that single-level array fields like [String!]! are correctly generated with Array<string> type. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Test that the ABI codegen correctly disambiguates: - Events with the same name but different inputs (Transfer, Transfer1, Transfer2) - Functions with the same name but different inputs (getSomething, getSomething1) This verifies the disambiguation logic matches the TS CLI behavior. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add support for nested array types (matrices) like [[String]], [[Int]], [[BigInt]], etc. in schema code generation: - types.rs: Add matrix type mappings for asc_type_for_value(), value_to_asc(), and value_from_asc() functions. These now correctly generate Array<Array<T>> types and use toXxxMatrix()/fromXxxMatrix() methods. - typescript.rs: Extend ArrayType to support any TypeExpr as inner type (not just NamedType), enabling nested array types. Add ArrayType::with_depth() helper for creating arrays with arbitrary nesting. - schema.rs: Replace boolean is_list with list_depth counter to track nesting levels. Update value_type_from_field() to wrap base type with correct number of brackets. Update type_from_field() to create properly nested Array<Array<T>> types. Add comprehensive tests for: - Matrix type conversions in types.rs - Nested array field generation in schema.rs - list_depth() function correctness - value_type_from_field() with nested arrays Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add comprehensive tests to verify tuple/struct handling in ABI code generation: - test_tuple_types_in_functions: Verifies struct classes are generated for both function inputs and outputs, with proper ethereum.Tuple extension and component accessors - test_tuple_types_in_events: Verifies struct classes are generated for tuple parameters in events - test_nested_tuple_types: Verifies nested tuples (struct containing struct) generate multiple struct classes with proper hierarchy - test_tuple_array_types: Verifies tuple[] returns Array<StructType> These tests cover important edge cases identified in the TS CLI test suite and ensure tuple/struct handling matches expected behavior. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add a `completions` subcommand that generates shell completion scripts for bash, elvish, fish, powershell, and zsh. Users can install completions by adding the output to their shell configuration. Example usage: gnd completions bash >> ~/.bashrc gnd completions zsh > ~/.zfunc/_gnd gnd completions fish > ~/.config/fish/completions/gnd.fish This uses the clap_complete crate which generates completions directly from the clap CLI definition, ensuring they stay in sync with the actual commands and options. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add additional tests to cover array-related edge cases in ABI code generation: - test_array_types_in_events: Verifies array parameters in events (address[] and uint256[]) generate correct Array<Address> and Array<BigInt> return types - test_matrix_types_in_functions: Verifies 2D array types (uint256[][]) generate correct Array<Array<BigInt>> return types These tests complement the existing tuple array tests and ensure comprehensive coverage of array-based ABI patterns. Co-Authored-By: Claude Opus 4.5 <[email protected]>
lutter
force-pushed
the
lutter/gnd-cli
branch
from
3b2843b to
c60b8e3
Compare
Add integration tests that verify gnd codegen produces compatible output to graph-cli codegen using golden test fixtures from the graph-cli repo. Changes: - Add walkdir and similar dev-dependencies for test infrastructure - Create codegen_verification.rs with tests for 8 fixtures - Fix ABI preprocessing to use correct default param names - Fix typescript class method ordering to match graph-cli output Known differences that are normalized in tests: - Int8 import (gnd always includes it) - Trailing commas (gnd uses them) - 2D array accessors (gnd correctly uses toStringMatrix) Skipped fixtures with documented reasons: - derived-from-with-interface: derived field loaders not implemented - invalid-graphql-schema: gnd generates schema.ts for invalid schemas - no-network-names: template ABI types not in subdirectories Co-Authored-By: Claude Opus 4.5 <[email protected]>
Fix 3 previously skipped fixtures so all 11 pass verification:
- derived-from-with-interface: Generate derived field loaders by calling
generate_derived_loaders() and combining with entity classes
- no-network-names: Generate ABI types in templates/<Name>/ subdirectories
by parsing template ABIs from manifest and generating them alongside
the templates.ts file
- invalid-graphql-schema: Add schema validation to reject non-nullable
lists with nullable members (e.g., [Something]! is invalid)
Additional ABI codegen fixes:
- Include return types in function signatures for call/tryCall
- Generate getValue{N}() getters for unnamed output parameters
- Filter call functions to exclude view/pure (matching graph-cli)
- Sort functions alphabetically for deterministic output
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Embed fixture data directly in the gnd crate instead of requiring graph-cli to be installed. This removes the GRAPH_CLI_PATH environment variable dependency and makes tests runnable without external setup. Added regenerate.sh script for updating fixtures when needed. Co-Authored-By: Claude Opus 4.5 <[email protected]>
The main() function uses #[tokio::main] which creates a tokio runtime, but init, add, and build commands were creating their own runtimes and calling block_on(), causing "Cannot start a runtime from within a runtime" errors. Changes: - Convert run_init, run_add, run_build to async functions - Convert helper functions (run_interactive, init_from_contract, get_contract_info, build_subgraph, watch_and_build) to async - Replace std::sync::mpsc with tokio::sync::mpsc in file watcher - Replace std::thread::sleep with tokio::time::sleep - Update main.rs to await the async command handlers - Convert affected tests to #[tokio::test] Co-Authored-By: Claude Opus 4.5 <[email protected]>
The publish command now: - Builds the subgraph and uploads to IPFS (reusing build command) - Opens browser to cli.thegraph.com/publish with query params - Supports --ipfs-hash to skip build and use existing hash - Supports --subgraph-id and --api-key for updating existing subgraphs - Supports --protocol-network (arbitrum-one, arbitrum-sepolia) - Prompts user before opening browser This matches graph-cli's approach of delegating wallet/GRT handling to the web UI rather than implementing it in the CLI. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add support for creating a subgraph scaffold from an existing deployed subgraph. This fetches the manifest and schema from IPFS, validates the network matches, finds immutable entities, and generates a scaffold with grafting instructions. Changes: - Add IPFS cat/fetch_manifest/fetch_schema methods to IpfsClient - Add manifest parsing utilities (extract_schema_cid, extract_network, get_min_start_block) - Add GraphQL schema parsing to find immutable entities - Implement full init_from_subgraph() async function - Add 11 unit tests for parsing functions Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add comprehensive README.md with command reference for all gnd commands - Add MIGRATING_FROM_GRAPH_CLI.md guide for users transitioning from graph-cli - Document all flags, examples, and common workflows - Mark documentation items as complete in the plan Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Update overall progress to ~98% (only manual testing remaining) - Mark Phase 12 (Testing & Polish) as complete - Add documentation file references to status summary - Document test porting analysis (11/12 success fixtures, NEAR out of scope) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add test infrastructure to verify gnd works as a drop-in replacement for graph-cli: - tests/tests/gnd_cli_tests.rs: Integration tests that run with GRAPH_CLI pointing to gnd binary, testing codegen/create/deploy workflow via block-handlers, value-roundtrip, and int8 tests - gnd/tests/cli_commands.rs: Standalone command tests for init, add, codegen, build, and clean commands that don't require Graph Node - justfile: Add test-gnd-cli and test-gnd-commands targets - Update plan/spec docs with Phase 13: CLI Integration Tests Co-Authored-By: Claude Opus 4.5 <[email protected]>
lutter
force-pushed
the
lutter/gnd-cli
branch
from
0f39307 to
021d897
Compare
The gnd codegen command uses Prettier to format generated TypeScript. Without Node.js/pnpm installed, unit tests fail because the generated code doesn't match the Prettier-formatted fixtures. Co-Authored-By: Claude Opus 4.5 <[email protected]>
gnd's compilation output is sensitive to asc version changes, so we now enforce that asc version 0.19.23 is installed. The version check runs before compilation and provides helpful error messages with installation instructions. For cases where bypassing the check is necessary, users can use either: - --skip-asc-version-check flag - GND_SKIP_ASC_VERSION_CHECK=1 environment variable Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is all Claude-generated code, I haven't even looked at it or tried it. It's mostly here for informational purposes