diff --git a/doc/developer/generated/adapter-types/dyncfgs.md b/doc/developer/generated/adapter-types/dyncfgs.md index 32feb2d4f67c9..df369938b0972 100644 --- a/doc/developer/generated/adapter-types/dyncfgs.md +++ b/doc/developer/generated/adapter-types/dyncfgs.md @@ -1,10 +1,10 @@ --- source: src/adapter-types/src/dyncfgs.rs -revision: b1e123e786 +revision: 1505e0c3f0 --- # mz-adapter-types::dyncfgs Declares all dynamic configuration flags owned by the adapter layer as `mz_dyncfg::Config` constants. -Covers session gating (`ALLOW_USER_SESSIONS`), zero-downtime deployment parameters (`WITH_0DT_*`, `ENABLE_0DT_*`), feature flags for expression caching, multi-replica sources, statement lifecycle logging, introspection subscribes, plan insights optimization thresholds, continual task builtins, password authentication, OIDC settings (`OIDC_ISSUER`, `OIDC_AUDIENCE`, `OIDC_AUTHENTICATION_CLAIM`), console OIDC configuration (`CONSOLE_OIDC_CLIENT_ID`, `CONSOLE_OIDC_SCOPES`), MCP endpoint toggles (`ENABLE_MCP_AGENT`, `ENABLE_MCP_AGENT_QUERY_TOOL`, `ENABLE_MCP_DEVELOPER`), persist fast-path ordering, S3 Tables region checks, and the user ID pool batch size. +Covers session gating (`ALLOW_USER_SESSIONS`), zero-downtime deployment parameters (`WITH_0DT_*`, `ENABLE_0DT_*`), feature flags for expression caching, multi-replica sources, statement lifecycle logging, introspection subscribes, plan insights optimization thresholds, continual task builtins, password authentication, OIDC settings (`OIDC_ISSUER`, `OIDC_AUDIENCE`, `OIDC_AUTHENTICATION_CLAIM`), console OIDC configuration (`CONSOLE_OIDC_CLIENT_ID`, `CONSOLE_OIDC_SCOPES`), MCP endpoint toggles (`ENABLE_MCP_AGENT`, `ENABLE_MCP_AGENT_QUERY_TOOL`, `ENABLE_MCP_DEVELOPER`) and response size limit (`MCP_MAX_RESPONSE_SIZE`), persist fast-path ordering, S3 Tables region checks, and the user ID pool batch size. `all_dyncfgs` registers every config constant into a `ConfigSet` for use during bootstrap. diff --git a/doc/developer/generated/adapter/catalog/_module.md b/doc/developer/generated/adapter/catalog/_module.md index fe0646c15ed9e..6b48a125d229e 100644 --- a/doc/developer/generated/adapter/catalog/_module.md +++ b/doc/developer/generated/adapter/catalog/_module.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/catalog.rs -revision: 0b6718c2a4 +revision: bce428d203 --- # adapter::catalog diff --git a/doc/developer/generated/adapter/catalog/apply.md b/doc/developer/generated/adapter/catalog/apply.md index 2fa1ec07e3ef1..0217b4c163c83 100644 --- a/doc/developer/generated/adapter/catalog/apply.md +++ b/doc/developer/generated/adapter/catalog/apply.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/catalog/apply.rs -revision: f75acad2d2 +revision: ae3360a4c1 --- # adapter::catalog::apply diff --git a/doc/developer/generated/adapter/catalog/open.md b/doc/developer/generated/adapter/catalog/open.md index 50bc625012b5e..48045f0ffd7f0 100644 --- a/doc/developer/generated/adapter/catalog/open.md +++ b/doc/developer/generated/adapter/catalog/open.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/catalog/open.rs -revision: 210c00eb3d +revision: 00cc513fa5 --- # adapter::catalog::open diff --git a/doc/developer/generated/adapter/catalog/open/_module.md b/doc/developer/generated/adapter/catalog/open/_module.md index a6c5e0f00b268..34cb8ae985e5d 100644 --- a/doc/developer/generated/adapter/catalog/open/_module.md +++ b/doc/developer/generated/adapter/catalog/open/_module.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/catalog/open.rs -revision: 210c00eb3d +revision: 00cc513fa5 --- # adapter::catalog::open diff --git a/doc/developer/generated/adapter/catalog/transact.md b/doc/developer/generated/adapter/catalog/transact.md index c3469f9393d77..4fe7ba6028487 100644 --- a/doc/developer/generated/adapter/catalog/transact.md +++ b/doc/developer/generated/adapter/catalog/transact.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/catalog/transact.rs -revision: 5680493e7d +revision: 00cc513fa5 --- # adapter::catalog::transact diff --git a/doc/developer/generated/adapter/command.md b/doc/developer/generated/adapter/command.md index 1199f484f91a1..0cc6b0a38cdff 100644 --- a/doc/developer/generated/adapter/command.md +++ b/doc/developer/generated/adapter/command.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/command.rs -revision: f7c755e1ed +revision: bce428d203 --- # adapter::command diff --git a/doc/developer/generated/adapter/coord/_module.md b/doc/developer/generated/adapter/coord/_module.md index dc239784cef09..5e770301881fc 100644 --- a/doc/developer/generated/adapter/coord/_module.md +++ b/doc/developer/generated/adapter/coord/_module.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord.rs -revision: f7c755e1ed +revision: bce428d203 --- # adapter::coord @@ -9,4 +9,4 @@ The core coordinator: `coord.rs` defines the `Coordinator` struct (the central s The coordinator owns the `Catalog`, active compute sinks, pending peeks, read-policy manager, timeline oracles, and all inter-subsystem handles (controller, storage collections, secrets, orchestrator). The file also defines `IdPool`, a pre-allocated pool of user `GlobalId` integers that amortizes per-DDL persist writes by reserving batches of IDs at once; the pool is owned by the coordinator and access is serialized through its single-threaded event loop. Child modules partition the coordinator's responsibilities: `command_handler` handles external `Command` messages; `message_handler` handles internal async `Message` responses; `sequencer` executes SQL plans; `appends` manages table and builtin-table writes; `catalog_implications` derives and applies downstream effects from catalog state changes; `ddl` wraps catalog transactions; `peek` and `read_policy` manage query execution and compaction; `timestamp_selection` and `timeline` handle temporal reasoning; `catalog_serving` serves catalog snapshots; and supporting modules cover cluster scheduling, introspection routing, consistency checking, index management, and statement logging. -Bootstrap handles derived builtin storage collections (builtin MVs and CTs) separately: after registering input-less collections, it bumps their sinces based on transitive dependency frontiers to satisfy as-of selection invariants. +Bootstrap handles derived builtin storage collections (builtin MVs and CTs) separately: after registering input-less collections in dependency order, it bumps their sinces based on transitive dependency frontiers to satisfy as-of selection invariants. diff --git a/doc/developer/generated/adapter/coord/catalog_implications.md b/doc/developer/generated/adapter/coord/catalog_implications.md index 1e154b00dfb0b..86b9dd5ef839b 100644 --- a/doc/developer/generated/adapter/coord/catalog_implications.md +++ b/doc/developer/generated/adapter/coord/catalog_implications.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/catalog_implications.rs -revision: 62f6c5cc15 +revision: 00cc513fa5 --- # adapter::coord::catalog_implications diff --git a/doc/developer/generated/adapter/coord/ddl.md b/doc/developer/generated/adapter/coord/ddl.md index 021db083f6aaf..dfd7630e3ce76 100644 --- a/doc/developer/generated/adapter/coord/ddl.md +++ b/doc/developer/generated/adapter/coord/ddl.md @@ -1,11 +1,11 @@ --- source: src/adapter/src/coord/ddl.rs -revision: 0b6718c2a4 +revision: 57ddfadb9c --- # adapter::coord::ddl Implements `Coordinator::catalog_transact` and related DDL helpers that combine a `catalog::transact` call with the downstream `apply_catalog_implications` step, publishing telemetry events and propagating errors back to the client. -Also provides `drop_compute_sinks`, `retire_execution`, and other coordinator-level housekeeping triggered by DDL. +Also provides `drop_compute_sinks` and other coordinator-level housekeeping triggered by DDL. `catalog_transact_with_side_effects` supports DDL operations that need to run async side-effects (e.g. dropping persist shards) after the catalog transaction commits. The `Op::InjectAuditEvents` variant is recognized as a no-op for downstream implication processing. diff --git a/doc/developer/generated/adapter/coord/in_memory_oracle.md b/doc/developer/generated/adapter/coord/in_memory_oracle.md index 0385d35b7e451..17fe2dfecc1ac 100644 --- a/doc/developer/generated/adapter/coord/in_memory_oracle.md +++ b/doc/developer/generated/adapter/coord/in_memory_oracle.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/in_memory_oracle.rs -revision: 8ed0d4135b +revision: bce428d203 --- # adapter::coord::in_memory_oracle diff --git a/doc/developer/generated/adapter/coord/peek.md b/doc/developer/generated/adapter/coord/peek.md index fabb332e524f4..1df2b68cfe979 100644 --- a/doc/developer/generated/adapter/coord/peek.md +++ b/doc/developer/generated/adapter/coord/peek.md @@ -1,10 +1,10 @@ --- source: src/adapter/src/coord/peek.rs -revision: 5680493e7d +revision: bce428d203 --- # adapter::coord::peek Defines the data structures and execution logic for coordinator-side peeks (SELECT queries). -`PeekPlan` discriminates between fast-path (`FastPathPlan`) and slow-path (`PeekDataflowPlan`) execution; `FastPathPlan` covers constant results, direct arrangement reads, and persist fast-path reads. +`PeekDataflowPlan` describes a slow-path peek that requires a dataflow; `FastPathPlan` covers constant results, direct arrangement reads, and persist fast-path reads. `implement_peek_plan` ships the peek to the compute layer or evaluates it immediately, returning a `PeekResponseUnary` stream; `PeekResponseUnary` carries rows, errors, or a cancellation signal back to the client. diff --git a/doc/developer/generated/adapter/coord/read_policy.md b/doc/developer/generated/adapter/coord/read_policy.md index f7a17ce23999d..c0ea3d683d97e 100644 --- a/doc/developer/generated/adapter/coord/read_policy.md +++ b/doc/developer/generated/adapter/coord/read_policy.md @@ -1,9 +1,9 @@ --- source: src/adapter/src/coord/read_policy.rs -revision: d7c2126b4a +revision: fcc110b5fe --- # adapter::coord::read_policy -Manages compaction read policies for collections: `ReadHolds` is an RAII handle that reserves a timestamp lower bound for a set of collections, preventing their `since` from advancing past the held timestamp; releasing the handle allows compaction to proceed. -`ReadPolicyManager` tracks all active read holds per collection and computes the effective `since` that must be maintained; it is consulted by the coordinator when advancing compaction frontiers. +Manages compaction read policies for collections: `ReadHolds` is an RAII handle that bundles a set of `ReadHold` tokens (one per storage or compute collection), preventing their `since` from advancing past the held timestamp; dropping the handle relinquishes the read capabilities. +`Coordinator` methods in this module (`initialize_read_policies`, `update_storage_read_policies`, `update_compute_read_policies`, `acquire_read_holds`) install and update read policies on the storage and compute controllers. diff --git a/doc/developer/generated/adapter/coord/sequencer.md b/doc/developer/generated/adapter/coord/sequencer.md index 36a514f41b103..9cbf997aba95a 100644 --- a/doc/developer/generated/adapter/coord/sequencer.md +++ b/doc/developer/generated/adapter/coord/sequencer.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer.rs -revision: 1d15c92c08 +revision: 00cc513fa5 --- # adapter::coord::sequencer diff --git a/doc/developer/generated/adapter/coord/sequencer/_module.md b/doc/developer/generated/adapter/coord/sequencer/_module.md index a859664f5c146..fbf7d8dbcd5d8 100644 --- a/doc/developer/generated/adapter/coord/sequencer/_module.md +++ b/doc/developer/generated/adapter/coord/sequencer/_module.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer.rs -revision: 1d15c92c08 +revision: 00cc513fa5 --- # adapter::coord::sequencer diff --git a/doc/developer/generated/adapter/coord/sequencer/inner.md b/doc/developer/generated/adapter/coord/sequencer/inner.md index 27b45dee6c74c..0bc89448011f6 100644 --- a/doc/developer/generated/adapter/coord/sequencer/inner.md +++ b/doc/developer/generated/adapter/coord/sequencer/inner.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer/inner.rs -revision: 4218b69078 +revision: bce428d203 --- # adapter::coord::sequencer::inner diff --git a/doc/developer/generated/adapter/coord/sequencer/inner/_module.md b/doc/developer/generated/adapter/coord/sequencer/inner/_module.md index 3f9247d4371a5..390bd7c7b545b 100644 --- a/doc/developer/generated/adapter/coord/sequencer/inner/_module.md +++ b/doc/developer/generated/adapter/coord/sequencer/inner/_module.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer/inner.rs -revision: 4218b69078 +revision: bce428d203 --- # adapter::coord::sequencer::inner diff --git a/doc/developer/generated/adapter/coord/sequencer/inner/create_materialized_view.md b/doc/developer/generated/adapter/coord/sequencer/inner/create_materialized_view.md index 6bb2aa3d188e1..acf39e9f66a23 100644 --- a/doc/developer/generated/adapter/coord/sequencer/inner/create_materialized_view.md +++ b/doc/developer/generated/adapter/coord/sequencer/inner/create_materialized_view.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer/inner/create_materialized_view.rs -revision: 5b9fb22e87 +revision: fcc110b5fe --- # adapter::coord::sequencer::inner::create_materialized_view diff --git a/doc/developer/generated/adapter/coord/sequencer/inner/explain_timestamp.md b/doc/developer/generated/adapter/coord/sequencer/inner/explain_timestamp.md index 66a9ed12bcbd6..c7981e518e36e 100644 --- a/doc/developer/generated/adapter/coord/sequencer/inner/explain_timestamp.md +++ b/doc/developer/generated/adapter/coord/sequencer/inner/explain_timestamp.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer/inner/explain_timestamp.rs -revision: e816eeb545 +revision: bce428d203 --- # adapter::coord::sequencer::inner::explain_timestamp diff --git a/doc/developer/generated/adapter/coord/sequencer/inner/peek.md b/doc/developer/generated/adapter/coord/sequencer/inner/peek.md index 60b3d3e82fef8..4e6ffa78030d1 100644 --- a/doc/developer/generated/adapter/coord/sequencer/inner/peek.md +++ b/doc/developer/generated/adapter/coord/sequencer/inner/peek.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer/inner/peek.rs -revision: 4267863081 +revision: bce428d203 --- # adapter::coord::sequencer::inner::peek diff --git a/doc/developer/generated/adapter/coord/sequencer/inner/subscribe.md b/doc/developer/generated/adapter/coord/sequencer/inner/subscribe.md index 86fae9fb59270..84cdf2d62b1c1 100644 --- a/doc/developer/generated/adapter/coord/sequencer/inner/subscribe.md +++ b/doc/developer/generated/adapter/coord/sequencer/inner/subscribe.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/sequencer/inner/subscribe.rs -revision: 2982634c0d +revision: fcc110b5fe --- # adapter::coord::sequencer::inner::subscribe diff --git a/doc/developer/generated/adapter/coord/sql.md b/doc/developer/generated/adapter/coord/sql.md index eb17cf96e36bb..2a8659e8bb1e4 100644 --- a/doc/developer/generated/adapter/coord/sql.md +++ b/doc/developer/generated/adapter/coord/sql.md @@ -1,9 +1,9 @@ --- source: src/adapter/src/coord/sql.rs -revision: 7340287c14 +revision: bce428d203 --- # adapter::coord::sql -Implements the coordinator's SQL-layer entry points: `parse`, `plan`, and the `prepare`/`execute` steps of the extended query protocol. -`plan` resolves names against the catalog, invokes the SQL planner, and returns a `Plan`; `prepare` additionally describes the statement's parameter and result types for the pgwire Describe flow. +Implements coordinator utility methods for the SQL layer: `plan_statement` resolves names against the catalog and invokes the SQL planner; `declare` and `describe` support cursor declaration and the pgwire Describe flow; `verify_prepared_statement` and `verify_portal` re-check prepared statement and portal validity after catalog changes. +Also contains `clear_transaction`, `clear_connection`, `add_active_compute_sink`, and `remove_active_compute_sink` for per-connection and per-sink bookkeeping. diff --git a/doc/developer/generated/adapter/coord/timeline.md b/doc/developer/generated/adapter/coord/timeline.md index 2d39270b989ca..9a7354d5111e9 100644 --- a/doc/developer/generated/adapter/coord/timeline.md +++ b/doc/developer/generated/adapter/coord/timeline.md @@ -1,10 +1,10 @@ --- source: src/adapter/src/coord/timeline.rs -revision: 4a1aeff959 +revision: fcc110b5fe --- # adapter::coord::timeline Manages per-timeline timestamp oracles and the coordinator's timeline bookkeeping. -`TimelineState` holds the `TimestampOracle` for each timeline and the oracle's current read/write timestamps; `timedomain_for` computes the union of timelines that a query's input collections belong to; the coordinator creates and caches oracle instances here. -`TimelineContext` (re-exported from the catalog) describes whether a collection is timeline-dependent or timestamp-independent. +`TimelineState` holds the `TimestampOracle` for each timeline and a set of `ReadHolds` that keep those read timestamps valid; the coordinator creates and caches oracle instances here. +`TimelineContext` describes whether a collection is timeline-dependent (`TimelineDependent`), timestamp-dependent (`TimestampDependent`), or timestamp-independent (`TimestampIndependent`). diff --git a/doc/developer/generated/adapter/coord/timestamp_selection.md b/doc/developer/generated/adapter/coord/timestamp_selection.md index 9db2ae6c5d2d0..13415257ce2e8 100644 --- a/doc/developer/generated/adapter/coord/timestamp_selection.md +++ b/doc/developer/generated/adapter/coord/timestamp_selection.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/timestamp_selection.rs -revision: b0ce85a355 +revision: bce428d203 --- # adapter::coord::timestamp_selection diff --git a/doc/developer/generated/adapter/coord/validity.md b/doc/developer/generated/adapter/coord/validity.md index cd2d666b8982a..90c501e2c6b76 100644 --- a/doc/developer/generated/adapter/coord/validity.md +++ b/doc/developer/generated/adapter/coord/validity.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/coord/validity.rs -revision: aa7a1afd31 +revision: bce428d203 --- # adapter::coord::validity diff --git a/doc/developer/generated/adapter/error.md b/doc/developer/generated/adapter/error.md index 64c588891c506..1299272d64588 100644 --- a/doc/developer/generated/adapter/error.md +++ b/doc/developer/generated/adapter/error.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/error.rs -revision: 892cf626bc +revision: 00cc513fa5 --- # adapter::error diff --git a/doc/developer/generated/adapter/explain/insights.md b/doc/developer/generated/adapter/explain/insights.md index 589d3a5d09bfa..dc8e49a6ee1b3 100644 --- a/doc/developer/generated/adapter/explain/insights.md +++ b/doc/developer/generated/adapter/explain/insights.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/explain/insights.rs -revision: 2a6ac3ab4c +revision: bce428d203 --- # adapter::explain::insights diff --git a/doc/developer/generated/adapter/frontend_peek.md b/doc/developer/generated/adapter/frontend_peek.md index 6b307103683aa..aefbbdc133058 100644 --- a/doc/developer/generated/adapter/frontend_peek.md +++ b/doc/developer/generated/adapter/frontend_peek.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/frontend_peek.rs -revision: 2982634c0d +revision: bce428d203 --- # adapter::frontend_peek diff --git a/doc/developer/generated/adapter/notice.md b/doc/developer/generated/adapter/notice.md index e01614731f0b4..63d1df1131486 100644 --- a/doc/developer/generated/adapter/notice.md +++ b/doc/developer/generated/adapter/notice.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/notice.rs -revision: 4267863081 +revision: bce428d203 --- # adapter::notice diff --git a/doc/developer/generated/adapter/optimize/copy_to.md b/doc/developer/generated/adapter/optimize/copy_to.md index 17e1d0be4bd93..e2abac0283e56 100644 --- a/doc/developer/generated/adapter/optimize/copy_to.md +++ b/doc/developer/generated/adapter/optimize/copy_to.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/optimize/copy_to.rs -revision: c0e930f9df +revision: bce428d203 --- # adapter::optimize::copy_to diff --git a/doc/developer/generated/adapter/optimize/peek.md b/doc/developer/generated/adapter/optimize/peek.md index 0c7225e4b4a1f..6a766dd286dbd 100644 --- a/doc/developer/generated/adapter/optimize/peek.md +++ b/doc/developer/generated/adapter/optimize/peek.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/optimize/peek.rs -revision: c0e930f9df +revision: bce428d203 --- # adapter::optimize::peek diff --git a/doc/developer/generated/adapter/peek_client.md b/doc/developer/generated/adapter/peek_client.md index 41cc24d0ea537..d0b1dfdf2d3be 100644 --- a/doc/developer/generated/adapter/peek_client.md +++ b/doc/developer/generated/adapter/peek_client.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/peek_client.rs -revision: 5680493e7d +revision: fcc110b5fe --- # adapter::peek_client diff --git a/doc/developer/generated/adapter/session.md b/doc/developer/generated/adapter/session.md index 130e29926cc6b..e5ed5d70572ae 100644 --- a/doc/developer/generated/adapter/session.md +++ b/doc/developer/generated/adapter/session.md @@ -1,12 +1,12 @@ --- source: src/adapter/src/session.rs -revision: aa7a1afd31 +revision: bce428d203 --- # adapter::session -Defines `Session`, the per-connection state object held by the coordinator for the duration of a client connection. +Defines `Session`, the per-connection state object held by the coordinator for the duration of a client connection. A `Session` tracks the connection ID and UUID, the current transaction state (`TransactionStatus`), open cursors, prepared statements with their logging metadata, portal bindings, session variables, role metadata, and lifecycle timestamps used for statement logging. `SessionConfig` includes an `authenticator_kind: AuthenticatorKind` field that records which authenticator was used for the connection. -`Transaction` holds the current transaction's operations (`TransactionOps`), timestamp context, and read holds; `TransactionStatus` is the state machine covering `Default`, `Started`, `InTransaction`, `InTransactionImplicit`, and `Failed` states. +`Transaction` holds the current transaction's operations (`TransactionOps`), a plan context, and optional write-lock guards; `TransactionStatus` is the state machine covering `Default`, `Started`, `InTransaction`, `InTransactionImplicit`, and `Failed` states. The module also defines `RowBatchStream` (the channel type for streaming subscribe results to pgwire) and `PreparedStatement` (a parsed, described statement with optional logging info). diff --git a/doc/developer/generated/adapter/util.md b/doc/developer/generated/adapter/util.md index 432c0f35fa599..83de33238fb24 100644 --- a/doc/developer/generated/adapter/util.md +++ b/doc/developer/generated/adapter/util.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/util.rs -revision: 5680493e7d +revision: 00cc513fa5 --- # adapter::util diff --git a/doc/developer/generated/adapter/webhook.md b/doc/developer/generated/adapter/webhook.md index 6662a95197ed2..96183cc6080dc 100644 --- a/doc/developer/generated/adapter/webhook.md +++ b/doc/developer/generated/adapter/webhook.md @@ -1,6 +1,6 @@ --- source: src/adapter/src/webhook.rs -revision: 5f6e25ff20 +revision: 00cc513fa5 --- # adapter::webhook diff --git a/doc/developer/generated/catalog-debug/_crate.md b/doc/developer/generated/catalog-debug/_crate.md index dee8bf5e4ad84..e1cddf8b0630c 100644 --- a/doc/developer/generated/catalog-debug/_crate.md +++ b/doc/developer/generated/catalog-debug/_crate.md @@ -1,6 +1,6 @@ --- source: src/catalog-debug/src/main.rs -revision: 7ee7a08339 +revision: 00cc513fa5 --- # mz-catalog-debug diff --git a/doc/developer/generated/catalog/builtin/_module.md b/doc/developer/generated/catalog/builtin/_module.md index c8cf27f1bb599..a3a0c0f8871a7 100644 --- a/doc/developer/generated/catalog/builtin/_module.md +++ b/doc/developer/generated/catalog/builtin/_module.md @@ -1,6 +1,6 @@ --- source: src/catalog/src/builtin.rs -revision: bf2293aa49 +revision: ae3360a4c1 --- # catalog::builtin diff --git a/doc/developer/generated/catalog/durable/error.md b/doc/developer/generated/catalog/durable/error.md index 662c8b86939f5..35ffdd431b882 100644 --- a/doc/developer/generated/catalog/durable/error.md +++ b/doc/developer/generated/catalog/durable/error.md @@ -1,11 +1,11 @@ --- source: src/catalog/src/durable/error.rs -revision: 8be2c88c0f +revision: 00cc513fa5 --- # catalog::durable::error Defines the error hierarchy for durable catalog operations. `CatalogError` wraps either a `SqlCatalogError` (logical catalog errors) or a `DurableCatalogError` (storage-level errors). -`DurableCatalogError` covers fencing (`FenceError`), incompatible data versions, persist upper mismatches, uninitialized catalog, and general internal errors. -`FenceError` distinguishes being fenced by a higher epoch from being fenced by a newer deploy generation. +`DurableCatalogError` covers fencing (`FenceError`), incompatible data versions (`IncompatibleDataVersion`, `IncompatiblePersistVersion`), uninitialized catalog, not-writable catalog, protobuf deserialization errors, duplicate key, uniqueness violations, storage errors, and internal errors. The method `can_recover_with_write_mode` reports whether the error can be recovered by reopening the catalog in writable mode. +`FenceError` has three variants: `DeployGeneration` (fenced by a newer deployment generation), `Epoch` (fenced by a newer epoch), and `MigrationUpper` (fenced during 0dt builtin table migration). The enum is ordered from most to least informative. diff --git a/doc/developer/generated/catalog/durable/transaction.md b/doc/developer/generated/catalog/durable/transaction.md index f8b86da88b684..3961163a606b6 100644 --- a/doc/developer/generated/catalog/durable/transaction.md +++ b/doc/developer/generated/catalog/durable/transaction.md @@ -1,6 +1,6 @@ --- source: src/catalog/src/durable/transaction.rs -revision: 210c00eb3d +revision: 00cc513fa5 --- # catalog::durable::transaction diff --git a/doc/developer/generated/cluster/client.md b/doc/developer/generated/cluster/client.md index e5b9107800c75..d1b6ff0aaccc0 100644 --- a/doc/developer/generated/cluster/client.md +++ b/doc/developer/generated/cluster/client.md @@ -1,6 +1,6 @@ --- source: src/cluster/src/client.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # client diff --git a/doc/developer/generated/cluster/communication.md b/doc/developer/generated/cluster/communication.md index 34c3bd7956012..6dc3129f51901 100644 --- a/doc/developer/generated/cluster/communication.md +++ b/doc/developer/generated/cluster/communication.md @@ -1,6 +1,6 @@ --- source: src/cluster/src/communication.rs -revision: 8cd889e259 +revision: b0fa98e931 --- # communication diff --git a/doc/developer/generated/clusterd/_crate.md b/doc/developer/generated/clusterd/_crate.md index aa8176f52f3c4..13e9d1d4ed5b3 100644 --- a/doc/developer/generated/clusterd/_crate.md +++ b/doc/developer/generated/clusterd/_crate.md @@ -1,6 +1,6 @@ --- source: src/clusterd/src/lib.rs -revision: 8cd889e259 +revision: 086ebe05a3 --- # clusterd diff --git a/doc/developer/generated/compute-client/as_of_selection.md b/doc/developer/generated/compute-client/as_of_selection.md index 3f6591cfdf185..16b7e7e543fb9 100644 --- a/doc/developer/generated/compute-client/as_of_selection.md +++ b/doc/developer/generated/compute-client/as_of_selection.md @@ -1,6 +1,6 @@ --- source: src/compute-client/src/as_of_selection.rs -revision: 52af3ba2a1 +revision: 00cc513fa5 --- # mz-compute-client::as_of_selection diff --git a/doc/developer/generated/compute-client/controller/_module.md b/doc/developer/generated/compute-client/controller/_module.md index 87f60bdad34ec..c6888b88ce9c0 100644 --- a/doc/developer/generated/compute-client/controller/_module.md +++ b/doc/developer/generated/compute-client/controller/_module.md @@ -1,6 +1,6 @@ --- source: src/compute-client/src/controller.rs -revision: 1e050fdd07 +revision: 16feb87fcb --- # mz-compute-client::controller diff --git a/doc/developer/generated/compute-client/controller/instance.md b/doc/developer/generated/compute-client/controller/instance.md index ff1f904a9f34a..ba9ff767426f3 100644 --- a/doc/developer/generated/compute-client/controller/instance.md +++ b/doc/developer/generated/compute-client/controller/instance.md @@ -1,6 +1,6 @@ --- source: src/compute-client/src/controller/instance.rs -revision: 2982634c0d +revision: 831ebabe2a --- # mz-compute-client::controller::instance diff --git a/doc/developer/generated/compute-client/controller/instance_client.md b/doc/developer/generated/compute-client/controller/instance_client.md index bfc9c8e70b009..a4a42e1506061 100644 --- a/doc/developer/generated/compute-client/controller/instance_client.md +++ b/doc/developer/generated/compute-client/controller/instance_client.md @@ -1,11 +1,11 @@ --- source: src/compute-client/src/controller/instance_client.rs -revision: 9ae71814e5 +revision: f4f99cbc37 --- # mz-compute-client::controller::instance_client Provides `InstanceClient`, the public interface through which external callers (e.g., the adapter) communicate with a compute instance task over channels. -Commands are sent as boxed closures (`Command`) over an `mpsc` channel; `call` is fire-and-forget while `call_sync` awaits a `oneshot` response. Read hold changes are propagated via a separate `read_holds::ChangeTx` sender. +Commands are sent as boxed closures (`Command = Box`) over an `mpsc` channel; `call` is fire-and-forget while `call_sync` awaits a `oneshot` response. Read hold changes are propagated via a separate `read_holds::ChangeTx` sender. `spawn` creates the `Instance` task and returns a client handle. Exposes operations for peek sequencing, acquiring read holds, and querying collection write frontiers; `pub(super)` methods serve the `ComputeController`'s internal use. Also defines `PeekError`, `AcquireReadHoldsError`, and `InstanceShutDown` specific to this client boundary. diff --git a/doc/developer/generated/compute-client/controller/introspection.md b/doc/developer/generated/compute-client/controller/introspection.md index 3d2f973cfa1be..d3f306c54cbf3 100644 --- a/doc/developer/generated/compute-client/controller/introspection.md +++ b/doc/developer/generated/compute-client/controller/introspection.md @@ -1,6 +1,6 @@ --- source: src/compute-client/src/controller/introspection.rs -revision: 7eb0cd86f7 +revision: 00cc513fa5 --- # mz-compute-client::controller::introspection diff --git a/doc/developer/generated/compute-client/controller/replica.md b/doc/developer/generated/compute-client/controller/replica.md index 80d300c9aea75..f8ee4842e96d2 100644 --- a/doc/developer/generated/compute-client/controller/replica.md +++ b/doc/developer/generated/compute-client/controller/replica.md @@ -1,6 +1,6 @@ --- source: src/compute-client/src/controller/replica.rs -revision: c58b2ebb27 +revision: f4f99cbc37 --- # mz-compute-client::controller::replica diff --git a/doc/developer/generated/compute-client/controller/sequential_hydration.md b/doc/developer/generated/compute-client/controller/sequential_hydration.md index 321e5ec3dcdf8..c2eb7bfd61044 100644 --- a/doc/developer/generated/compute-client/controller/sequential_hydration.md +++ b/doc/developer/generated/compute-client/controller/sequential_hydration.md @@ -1,6 +1,6 @@ --- source: src/compute-client/src/controller/sequential_hydration.rs -revision: e757b4d11b +revision: f4f99cbc37 --- # mz-compute-client::controller::sequential_hydration diff --git a/doc/developer/generated/compute-client/metrics.md b/doc/developer/generated/compute-client/metrics.md index 299e8a19a3100..e6ed41b0927fd 100644 --- a/doc/developer/generated/compute-client/metrics.md +++ b/doc/developer/generated/compute-client/metrics.md @@ -1,10 +1,10 @@ --- source: src/compute-client/src/metrics.rs -revision: 19608a8469 +revision: f4f99cbc37 --- # mz-compute-client::metrics Defines Prometheus metrics for the compute controller, organized into three layers: `ComputeControllerMetrics` (registry-level, shared across all instances), `InstanceMetrics` (per compute instance), and `ReplicaMetrics` (per replica). -`CommandMetrics` and `ResponseMetrics` provide keyed metric sets for each command and response variant, respectively, and `PeekMetrics` tracks peek outcomes by result type. -`HistoryMetrics` tracks the size of the command history, and `ReplicaCollectionMetrics` holds wallclock-lag metrics for individual collections. +`CommandMetrics` provides a keyed metric set for each `ComputeCommand` variant (including `Hello`, `CreateInstance`, `CreateDataflow`, `Schedule`, `AllowCompaction`, `Peek`, `CancelPeek`, `InitializationComplete`, `UpdateConfiguration`, and `AllowWrites`). `ResponseMetrics` (private) provides keyed metrics per `ComputeResponse` variant. `PeekMetrics` tracks peek outcomes by result type (rows, rows_stashed, error, canceled). +`HistoryMetrics` tracks command and dataflow counts in the command history. `ReplicaCollectionMetrics` holds wallclock-lag metrics for individual non-transient collections. diff --git a/doc/developer/generated/compute-client/protocol/history.md b/doc/developer/generated/compute-client/protocol/history.md index 3249d5a69bda4..d71ed372d9634 100644 --- a/doc/developer/generated/compute-client/protocol/history.md +++ b/doc/developer/generated/compute-client/protocol/history.md @@ -1,10 +1,11 @@ --- source: src/compute-client/src/protocol/history.rs -revision: c0a1c584d1 +revision: f4f99cbc37 --- # mz-compute-client::protocol::history Implements `ComputeCommandHistory`, a compacted log of `ComputeCommand`s used to replay state to a newly connected replica. -The history automatically reduces itself by eliminating superseded commands (e.g., compacting `AllowCompaction` commands and dropping dataflows whose compaction frontier is empty) whenever the command sequence doubles in size. -Tracks command and dataflow counts via `HistoryMetrics`. +The history automatically reduces itself by eliminating superseded commands (e.g., compacting `AllowCompaction` commands, dropping dataflows whose compaction frontier is empty, deduplicating configuration updates, and retaining only live peeks) whenever the command sequence doubles in size. +Reduction also handles `Schedule` commands (retaining only those for live dataflows), `AllowWrites` commands (retaining only those for live dataflows), and `InitializationComplete`. +Tracks command and dataflow counts via `HistoryMetrics`. Also provides `discard_peeks` to drop all peek commands and `update_source_uppers` to refresh storage import upper frontiers before connecting a new replica. diff --git a/doc/developer/generated/compute-client/protocol/response.md b/doc/developer/generated/compute-client/protocol/response.md index 9984c4c3d47b9..c3b252761e3ff 100644 --- a/doc/developer/generated/compute-client/protocol/response.md +++ b/doc/developer/generated/compute-client/protocol/response.md @@ -1,11 +1,11 @@ --- source: src/compute-client/src/protocol/response.rs -revision: 2982634c0d +revision: f4f99cbc37 --- # mz-compute-client::protocol::response Defines `ComputeResponse`, the enum of all responses sent from replicas to the compute controller. Variants cover collection frontier advancement (`Frontiers`), one-shot peek results (`PeekResponse`), streaming subscribe batches (`SubscribeResponse`), COPY TO completion (`CopyToResponse`), and status updates (`Status`). -Also defines `FrontiersResponse`, `PeekResponse`, `StashedPeekResponse`, `SubscribeResponse`, and `SubscribeBatch`. -`SubscribeBatch::updates` holds a `Vec>` rather than a flat vec of `(T, Row, Diff)` tuples; each `UpdateCollection` is sorted by time (then by the subscribe's order-by), and multiple collections may be combined when aggregating results across workers. +Also defines `FrontiersResponse` (with `write_frontier`, `input_frontier`, and `output_frontier` optional fields), the `PeekResponse` enum (variants: `Rows`, `Stashed`, `Error`, `Canceled`), `StashedPeekResponse` (for large peek results persisted to a Persist shard), `SubscribeResponse`, and `SubscribeBatch`. +`SubscribeBatch::updates` holds a `Vec` rather than a flat vec of `(Timestamp, Row, Diff)` tuples; each `UpdateCollection` is sorted by time (then by the subscribe's order-by), and multiple collections may be combined when aggregating results across workers. diff --git a/doc/developer/generated/compute-client/service.md b/doc/developer/generated/compute-client/service.md index 79a5a979d02f2..006340ce903c8 100644 --- a/doc/developer/generated/compute-client/service.md +++ b/doc/developer/generated/compute-client/service.md @@ -1,11 +1,11 @@ --- source: src/compute-client/src/service.rs -revision: 2982634c0d +revision: f4f99cbc37 --- # mz-compute-client::service Defines `ComputeClient`, the trait alias for a `GenericClient` over compute commands and responses, and `PartitionedComputeState`, which merges responses from multiple partitioned workers into a single unified stream. -`PartitionedComputeState` tracks frontier state per collection, merges peek and copy-to responses once all shards have replied, and sequences subscribe batches by holding updates until their timestamps are complete. -Stashed subscribe updates are stored as `Vec>`; when the frontier advances, each collection is split at the frontier boundary and the prefix is shipped while the remainder is retained for future batches. -It implements the `PartitionedState` trait and is instantiated at both the controller–cluster boundary and within each cluster process to dispatch across timely workers. +`PartitionedComputeState` tracks frontier state per collection (via `TrackedFrontiers`, which maintains `MutableAntichain`s plus per-shard antichains for write, input, and output frontiers), merges peek and copy-to responses once all shards have replied, and sequences subscribe batches by holding updates until their timestamps are complete. +Stashed subscribe updates are stored as `Vec`; when the frontier advances, each collection is split at the frontier boundary and the prefix is shipped while the remainder is retained for future batches. +It implements the `PartitionedState` trait and is instantiated at both the controller–cluster boundary and within each cluster process to dispatch across timely workers. `Hello` and `UpdateConfiguration` commands are forwarded to all shards; all other commands go only to the first shard. diff --git a/doc/developer/generated/compute/arrangement/manager.md b/doc/developer/generated/compute/arrangement/manager.md index ece79ed0cbb41..a0381224ed357 100644 --- a/doc/developer/generated/compute/arrangement/manager.md +++ b/doc/developer/generated/compute/arrangement/manager.md @@ -1,6 +1,6 @@ --- source: src/compute/src/arrangement/manager.rs -revision: 844ad57e4b +revision: b0fa98e931 --- # mz-compute::arrangement::manager diff --git a/doc/developer/generated/compute/command_channel.md b/doc/developer/generated/compute/command_channel.md index 2b1221edcebc0..e665169d75a79 100644 --- a/doc/developer/generated/compute/command_channel.md +++ b/doc/developer/generated/compute/command_channel.md @@ -1,6 +1,6 @@ --- source: src/compute/src/command_channel.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-compute::command_channel diff --git a/doc/developer/generated/compute/compute_state/_module.md b/doc/developer/generated/compute/compute_state/_module.md index dca7aec5086a0..8d160481363c6 100644 --- a/doc/developer/generated/compute/compute_state/_module.md +++ b/doc/developer/generated/compute/compute_state/_module.md @@ -1,6 +1,6 @@ --- source: src/compute/src/compute_state.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-compute::compute_state diff --git a/doc/developer/generated/compute/compute_state/peek_result_iterator.md b/doc/developer/generated/compute/compute_state/peek_result_iterator.md index f507ff6475473..726ceaef60aa5 100644 --- a/doc/developer/generated/compute/compute_state/peek_result_iterator.md +++ b/doc/developer/generated/compute/compute_state/peek_result_iterator.md @@ -1,6 +1,6 @@ --- source: src/compute/src/compute_state/peek_result_iterator.rs -revision: 6f6a40feb7 +revision: b0fa98e931 --- # mz-compute::compute_state::peek_result_iterator diff --git a/doc/developer/generated/compute/extensions/arrange.md b/doc/developer/generated/compute/extensions/arrange.md index c6215063b51f5..d471350f358c3 100644 --- a/doc/developer/generated/compute/extensions/arrange.md +++ b/doc/developer/generated/compute/extensions/arrange.md @@ -1,6 +1,6 @@ --- source: src/compute/src/extensions/arrange.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-compute::extensions::arrange diff --git a/doc/developer/generated/compute/extensions/reduce.md b/doc/developer/generated/compute/extensions/reduce.md index 389eb98af5866..21371d435a293 100644 --- a/doc/developer/generated/compute/extensions/reduce.md +++ b/doc/developer/generated/compute/extensions/reduce.md @@ -1,6 +1,6 @@ --- source: src/compute/src/extensions/reduce.rs -revision: d76e608a67 +revision: b0fa98e931 --- # mz-compute::extensions::reduce diff --git a/doc/developer/generated/compute/extensions/temporal_bucket.md b/doc/developer/generated/compute/extensions/temporal_bucket.md index e6cfb3fd227cd..832f46543f197 100644 --- a/doc/developer/generated/compute/extensions/temporal_bucket.md +++ b/doc/developer/generated/compute/extensions/temporal_bucket.md @@ -1,6 +1,6 @@ --- source: src/compute/src/extensions/temporal_bucket.rs -revision: f498b6e141 +revision: b0fa98e931 --- # mz-compute::extensions::temporal_bucket diff --git a/doc/developer/generated/compute/logging/_module.md b/doc/developer/generated/compute/logging/_module.md index 2f919eeb32fd7..9abfb37481e6a 100644 --- a/doc/developer/generated/compute/logging/_module.md +++ b/doc/developer/generated/compute/logging/_module.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging.rs -revision: f94584ddef +revision: b0fa98e931 --- # mz-compute::logging diff --git a/doc/developer/generated/compute/logging/compute.md b/doc/developer/generated/compute/logging/compute.md index 5111b17e88b47..d11dc6095fa92 100644 --- a/doc/developer/generated/compute/logging/compute.md +++ b/doc/developer/generated/compute/logging/compute.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging/compute.rs -revision: 8864f35b2c +revision: b0fa98e931 --- # mz-compute::logging::compute diff --git a/doc/developer/generated/compute/logging/differential.md b/doc/developer/generated/compute/logging/differential.md index 25f7f8e28e2c2..e96a4de906d74 100644 --- a/doc/developer/generated/compute/logging/differential.md +++ b/doc/developer/generated/compute/logging/differential.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging/differential.rs -revision: 5680493e7d +revision: b0fa98e931 --- # mz-compute::logging::differential diff --git a/doc/developer/generated/compute/logging/initialize.md b/doc/developer/generated/compute/logging/initialize.md index 570ada21e0a45..770ebaab3819a 100644 --- a/doc/developer/generated/compute/logging/initialize.md +++ b/doc/developer/generated/compute/logging/initialize.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging/initialize.rs -revision: 8864f35b2c +revision: b0fa98e931 --- # mz-compute::logging::initialize diff --git a/doc/developer/generated/compute/logging/prometheus.md b/doc/developer/generated/compute/logging/prometheus.md index b3effe6e1ca80..65a9ba08a3df4 100644 --- a/doc/developer/generated/compute/logging/prometheus.md +++ b/doc/developer/generated/compute/logging/prometheus.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging/prometheus.rs -revision: f94584ddef +revision: b0fa98e931 --- # mz-compute::logging::prometheus diff --git a/doc/developer/generated/compute/logging/reachability.md b/doc/developer/generated/compute/logging/reachability.md index 526cfd71d9525..5198b7c8e06a3 100644 --- a/doc/developer/generated/compute/logging/reachability.md +++ b/doc/developer/generated/compute/logging/reachability.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging/reachability.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-compute::logging::reachability diff --git a/doc/developer/generated/compute/logging/timely.md b/doc/developer/generated/compute/logging/timely.md index a396c8b1b8a70..cafa54bd40cfa 100644 --- a/doc/developer/generated/compute/logging/timely.md +++ b/doc/developer/generated/compute/logging/timely.md @@ -1,6 +1,6 @@ --- source: src/compute/src/logging/timely.rs -revision: 834eb88f27 +revision: b0fa98e931 --- # mz-compute::logging::timely diff --git a/doc/developer/generated/compute/render/_module.md b/doc/developer/generated/compute/render/_module.md index 7e4b38de9d475..ede4cba876e9e 100644 --- a/doc/developer/generated/compute/render/_module.md +++ b/doc/developer/generated/compute/render/_module.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render.rs -revision: 84bda5baa0 +revision: b0fa98e931 --- # mz-compute::render diff --git a/doc/developer/generated/compute/render/context.md b/doc/developer/generated/compute/render/context.md index b5bd8117ee229..6471bf31ba5f9 100644 --- a/doc/developer/generated/compute/render/context.md +++ b/doc/developer/generated/compute/render/context.md @@ -1,10 +1,10 @@ --- source: src/compute/src/render/context.rs -revision: 84bda5baa0 +revision: 52f2de096d --- # mz-compute::render::context Defines `Context`, the central data structure that accumulates dataflow-local collections and arrangements while a `RenderPlan` is being translated into Timely operators. -Provides `lookup_id` for resolving `Id` references to either locally rendered collections or imported arrangements, and methods for inserting `AvailableCollections` (both unarranged and arranged) keyed by `GlobalId`. -Handles timestamp refinement: the context is parameterized by a scope `S` whose timestamp implements `RenderTimestamp`, which refines `mz_repr::Timestamp`; the two may differ in the case of regions or iteration, and the context provides methods to enter imported traces into the rendering scope. +Provides `lookup_id` for resolving `Id` references to either locally rendered collections or imported arrangements, and `insert_id`/`update_id`/`remove_id` for managing `CollectionBundle` bindings keyed by `Id`. +Handles timestamp refinement: the context is parameterized by a scope `S` whose timestamp implements `RenderTimestamp`, which refines `mz_repr::Timestamp`; the two may differ in the case of regions or iteration, and the context provides `enter_region` to bring all bindings into a child scope. diff --git a/doc/developer/generated/compute/render/continual_task.md b/doc/developer/generated/compute/render/continual_task.md index 66089d2e87eb9..df2d34812e2c1 100644 --- a/doc/developer/generated/compute/render/continual_task.md +++ b/doc/developer/generated/compute/render/continual_task.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/continual_task.rs -revision: c642b63c77 +revision: b0fa98e931 --- # mz-compute::render::continual_task diff --git a/doc/developer/generated/compute/render/flat_map.md b/doc/developer/generated/compute/render/flat_map.md index 564345830f3c0..0566e2088364c 100644 --- a/doc/developer/generated/compute/render/flat_map.md +++ b/doc/developer/generated/compute/render/flat_map.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/flat_map.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-compute::render::flat_map diff --git a/doc/developer/generated/compute/render/join/delta_join.md b/doc/developer/generated/compute/render/join/delta_join.md index b41a13177c56c..247cfe08b3bda 100644 --- a/doc/developer/generated/compute/render/join/delta_join.md +++ b/doc/developer/generated/compute/render/join/delta_join.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/join/delta_join.rs -revision: ef950e9dbe +revision: b0fa98e931 --- # mz-compute::render::join::delta_join diff --git a/doc/developer/generated/compute/render/join/linear_join.md b/doc/developer/generated/compute/render/join/linear_join.md index 37e01038027ae..cc7c1ca8030c6 100644 --- a/doc/developer/generated/compute/render/join/linear_join.md +++ b/doc/developer/generated/compute/render/join/linear_join.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/join/linear_join.rs -revision: 84bda5baa0 +revision: 52f2de096d --- # mz-compute::render::join::linear_join diff --git a/doc/developer/generated/compute/render/join/mz_join_core.md b/doc/developer/generated/compute/render/join/mz_join_core.md index 28b714abd157c..d11308fa02baf 100644 --- a/doc/developer/generated/compute/render/join/mz_join_core.md +++ b/doc/developer/generated/compute/render/join/mz_join_core.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/join/mz_join_core.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-compute::render::join::mz_join_core diff --git a/doc/developer/generated/compute/render/reduce.md b/doc/developer/generated/compute/render/reduce.md index 4137b467f575c..95827a256604a 100644 --- a/doc/developer/generated/compute/render/reduce.md +++ b/doc/developer/generated/compute/render/reduce.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/reduce.rs -revision: 84bda5baa0 +revision: 52f2de096d --- # mz-compute::render::reduce diff --git a/doc/developer/generated/compute/render/sinks.md b/doc/developer/generated/compute/render/sinks.md index 787f1302e6e2e..9b63f02ae2e93 100644 --- a/doc/developer/generated/compute/render/sinks.md +++ b/doc/developer/generated/compute/render/sinks.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/sinks.rs -revision: e79a6d96d9 +revision: 52f2de096d --- # mz-compute::render::sinks diff --git a/doc/developer/generated/compute/render/threshold.md b/doc/developer/generated/compute/render/threshold.md index ed9c279eeded6..c212973fd45b7 100644 --- a/doc/developer/generated/compute/render/threshold.md +++ b/doc/developer/generated/compute/render/threshold.md @@ -1,9 +1,9 @@ --- source: src/compute/src/render/threshold.rs -revision: 84bda5baa0 +revision: 52f2de096d --- # mz-compute::render::threshold Renders `ThresholdPlan` nodes, which filter out rows with negative or zero multiplicities to implement SQL `EXCEPT ALL` / `DISTINCT`-based deduplication. -`BasicThresholdPlan` arranges the collection and applies a reduce that retains only rows with positive count; the result is passed back as an unarranged collection. +`BasicThresholdPlan` arranges the collection and applies a reduce that retains only rows with positive count; the result is returned as an arranged `CollectionBundle`. diff --git a/doc/developer/generated/compute/render/top_k.md b/doc/developer/generated/compute/render/top_k.md index 5245d9d5acde1..45db206b9a34c 100644 --- a/doc/developer/generated/compute/render/top_k.md +++ b/doc/developer/generated/compute/render/top_k.md @@ -1,6 +1,6 @@ --- source: src/compute/src/render/top_k.rs -revision: 834eb88f27 +revision: b0fa98e931 --- # mz-compute::render::top_k diff --git a/doc/developer/generated/compute/server.md b/doc/developer/generated/compute/server.md index 2ec17b628ab9f..cefe25e0ac0e1 100644 --- a/doc/developer/generated/compute/server.md +++ b/doc/developer/generated/compute/server.md @@ -1,6 +1,6 @@ --- source: src/compute/src/server.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-compute::server diff --git a/doc/developer/generated/compute/sink/copy_to_s3_oneshot.md b/doc/developer/generated/compute/sink/copy_to_s3_oneshot.md index 38da85d2a7b86..5ec0222b8b1e5 100644 --- a/doc/developer/generated/compute/sink/copy_to_s3_oneshot.md +++ b/doc/developer/generated/compute/sink/copy_to_s3_oneshot.md @@ -1,10 +1,10 @@ --- source: src/compute/src/sink/copy_to_s3_oneshot.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-compute::sink::copy_to_s3_oneshot Implements the `CopyToS3Oneshot` sink, which writes a snapshot of a collection to S3 (or compatible object storage) in Parquet format as a one-shot operation initiated by a `COPY TO` statement. -Updates are consolidated per worker and exchanged to a single writer worker that accumulates row groups, respecting configured part size and row-group ratio dyncfgs. +Updates are consolidated per worker and exchanged by batch ID across workers, with each worker responsible for 0 or more batches; each batch is split into one or more files according to the configured `MAX_FILE_SIZE`, respecting part size and row-group ratio dyncfgs. A `CopyToResponse` is returned to the controller with the number of rows written once the upload completes. diff --git a/doc/developer/generated/compute/sink/materialized_view.md b/doc/developer/generated/compute/sink/materialized_view.md index 4f1d9b360d9ee..72b03d0b49a78 100644 --- a/doc/developer/generated/compute/sink/materialized_view.md +++ b/doc/developer/generated/compute/sink/materialized_view.md @@ -1,6 +1,6 @@ --- source: src/compute/src/sink/materialized_view.rs -revision: 7411afe76f +revision: b0fa98e931 --- # mz-compute::sink::materialized_view diff --git a/doc/developer/generated/compute/sink/refresh.md b/doc/developer/generated/compute/sink/refresh.md index 769192cf27876..fa8bc677f2b24 100644 --- a/doc/developer/generated/compute/sink/refresh.md +++ b/doc/developer/generated/compute/sink/refresh.md @@ -1,6 +1,6 @@ --- source: src/compute/src/sink/refresh.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-compute::sink::refresh diff --git a/doc/developer/generated/compute/sink/subscribe.md b/doc/developer/generated/compute/sink/subscribe.md index 5e091cde1019e..854b3d63add3b 100644 --- a/doc/developer/generated/compute/sink/subscribe.md +++ b/doc/developer/generated/compute/sink/subscribe.md @@ -1,6 +1,6 @@ --- source: src/compute/src/sink/subscribe.rs -revision: 2982634c0d +revision: b0fa98e931 --- # mz-compute::sink::subscribe diff --git a/doc/developer/generated/compute/typedefs.md b/doc/developer/generated/compute/typedefs.md index 3830f02c2858e..295c9c43ff8e6 100644 --- a/doc/developer/generated/compute/typedefs.md +++ b/doc/developer/generated/compute/typedefs.md @@ -1,6 +1,6 @@ --- source: src/compute/src/typedefs.rs -revision: f498b6e141 +revision: b0fa98e931 --- # mz-compute::typedefs diff --git a/doc/developer/generated/controller/_crate.md b/doc/developer/generated/controller/_crate.md index 795a12e0a9027..a22349063a2ff 100644 --- a/doc/developer/generated/controller/_crate.md +++ b/doc/developer/generated/controller/_crate.md @@ -1,15 +1,15 @@ --- source: src/controller/src/lib.rs -revision: 681cdf1339 +revision: 00cc513fa5 --- # controller -Provides the top-level `Controller` that unifies the storage controller, storage collections, and compute controller behind a single `ready` / `process` interface consumed by `environmentd`'s coordinator. +Provides the top-level `Controller` that unifies the storage controller, storage collections, and compute controller behind a single `ready` / `process` interface consumed by `environmentd`'s coordinator. `Controller` holds `Box`, `Arc`, and `ComputeController`; it multiplexes their readiness via `tokio::select!` in `ready()` and dispatches to the appropriate subsystem in `process()`. It also implements the _watch set_ mechanism: callers install a set of `GlobalId`s and a timestamp, and the controller fires a `WatchSetFinished` response once all frontier uppers have advanced past that timestamp. -Replica metrics (CPU, memory, disk) are polled from the orchestrator and written into the `ReplicaMetricsHistory` introspection collection. +Replica metrics (CPU, memory, disk, heap) are polled from the orchestrator and written into the `ReplicaMetricsHistory` introspection collection. Key dependencies are `mz-compute-client`, `mz-storage-client`, `mz-storage-controller`, `mz-orchestrator`, and `mz-persist-client`; the crate is consumed exclusively by `environmentd`. ## Modules diff --git a/doc/developer/generated/controller/clusters.md b/doc/developer/generated/controller/clusters.md index ec3870a15785e..dd961b901e969 100644 --- a/doc/developer/generated/controller/clusters.md +++ b/doc/developer/generated/controller/clusters.md @@ -1,6 +1,6 @@ --- source: src/controller/src/clusters.rs -revision: eb105c0ce8 +revision: f4f99cbc37 --- # controller::clusters diff --git a/doc/developer/generated/environmentd/environmentd/main.md b/doc/developer/generated/environmentd/environmentd/main.md index 340d7664e2630..f17e8a6fcf2c4 100644 --- a/doc/developer/generated/environmentd/environmentd/main.md +++ b/doc/developer/generated/environmentd/environmentd/main.md @@ -1,6 +1,6 @@ --- source: src/environmentd/src/environmentd/main.rs -revision: f8348f3bca +revision: 117286ee15 --- # environmentd::environmentd::main diff --git a/doc/developer/generated/environmentd/http/webhook.md b/doc/developer/generated/environmentd/http/webhook.md index 0f0a9b0e6b25e..ea608ad630ad1 100644 --- a/doc/developer/generated/environmentd/http/webhook.md +++ b/doc/developer/generated/environmentd/http/webhook.md @@ -1,6 +1,6 @@ --- source: src/environmentd/src/http/webhook.rs -revision: db271c31b1 +revision: 00cc513fa5 --- # environmentd::http::webhook diff --git a/doc/developer/generated/environmentd/test_util.md b/doc/developer/generated/environmentd/test_util.md index d4049abbb989f..f1a5342a12798 100644 --- a/doc/developer/generated/environmentd/test_util.md +++ b/doc/developer/generated/environmentd/test_util.md @@ -1,6 +1,6 @@ --- source: src/environmentd/src/test_util.rs -revision: b1e123e786 +revision: bce428d203 --- # environmentd::test_util diff --git a/doc/developer/generated/expr/relation/_module.md b/doc/developer/generated/expr/relation/_module.md index c3f6302c78eaa..d047db4a37b36 100644 --- a/doc/developer/generated/expr/relation/_module.md +++ b/doc/developer/generated/expr/relation/_module.md @@ -1,6 +1,6 @@ --- source: src/expr/src/relation.rs -revision: 834eb88f27 +revision: 15f2490e38 --- # mz-expr::relation diff --git a/doc/developer/generated/expr/relation/func.md b/doc/developer/generated/expr/relation/func.md index 3eb6e0b6c3371..aedd561e0308d 100644 --- a/doc/developer/generated/expr/relation/func.md +++ b/doc/developer/generated/expr/relation/func.md @@ -1,6 +1,6 @@ --- source: src/expr/src/relation/func.rs -revision: 5f785f23fd +revision: 15f2490e38 --- # mz-expr::relation::func diff --git a/doc/developer/generated/flows.md b/doc/developer/generated/flows.md index f840f7811c706..9c1a1e00387b8 100644 --- a/doc/developer/generated/flows.md +++ b/doc/developer/generated/flows.md @@ -128,11 +128,11 @@ graph TD SC --> R2[Replica via CTP] ``` -1. **`mz_controller`** — unified `Controller` multiplexing compute + storage +1. **`mz_controller`** — unified `Controller` multiplexing compute + storage 2. **`mz_compute_client::controller`** — `ComputeController` manages compute clusters/replicas 3. **`mz_compute_client::controller::instance`** — per-cluster `Instance` state 4. **`mz_compute_client::controller::replica`** — per-replica connection state -5. **`mz_storage_controller`** — `Controller` for storage; manages ingestion/export lifecycle -6. **`mz_storage_controller::instance`** — per-cluster `Instance` and `Replica` +5. **`mz_storage_controller`** — `Controller` for storage; manages ingestion/export lifecycle +6. **`mz_storage_controller::instance`** — per-cluster `Instance` and `Replica` 7. **`mz_service::transport`** — Cluster Transport Protocol (CTP): length-prefixed bincode over TCP/UDS 8. **`mz_cluster::communication`** — generation-epoch mesh protocol for Timely workers diff --git a/doc/developer/generated/interchange/envelopes.md b/doc/developer/generated/interchange/envelopes.md index f6602130cb967..f2853e72967b1 100644 --- a/doc/developer/generated/interchange/envelopes.md +++ b/doc/developer/generated/interchange/envelopes.md @@ -1,6 +1,6 @@ --- source: src/interchange/src/envelopes.rs -revision: c28d0061a6 +revision: b0fa98e931 --- # interchange::envelopes diff --git a/doc/developer/generated/orchestrator-kubernetes/_crate.md b/doc/developer/generated/orchestrator-kubernetes/_crate.md index f779a625c78dd..e7495e8ba27b6 100644 --- a/doc/developer/generated/orchestrator-kubernetes/_crate.md +++ b/doc/developer/generated/orchestrator-kubernetes/_crate.md @@ -1,6 +1,6 @@ --- source: src/orchestrator-kubernetes/src/lib.rs -revision: b1c657d28e +revision: 117286ee15 --- # mz-orchestrator-kubernetes diff --git a/doc/developer/generated/persist-client/operators/_module.md b/doc/developer/generated/persist-client/operators/_module.md index 9580250612b3d..709868c78cb98 100644 --- a/doc/developer/generated/persist-client/operators/_module.md +++ b/doc/developer/generated/persist-client/operators/_module.md @@ -1,6 +1,6 @@ --- source: src/persist-client/src/operators/shard_source.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # persist-client::operators diff --git a/doc/developer/generated/persist-client/operators/shard_source.md b/doc/developer/generated/persist-client/operators/shard_source.md index 722a063237cb2..52c54d4be4b47 100644 --- a/doc/developer/generated/persist-client/operators/shard_source.md +++ b/doc/developer/generated/persist-client/operators/shard_source.md @@ -1,6 +1,6 @@ --- source: src/persist-client/src/operators/shard_source.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # persist-client::operators::shard_source diff --git a/doc/developer/generated/persist-client/rpc.md b/doc/developer/generated/persist-client/rpc.md index f40a06e8d6f9d..c341e84a44ab9 100644 --- a/doc/developer/generated/persist-client/rpc.md +++ b/doc/developer/generated/persist-client/rpc.md @@ -1,6 +1,6 @@ --- source: src/persist-client/src/rpc.rs -revision: cdc621b496 +revision: 6040296dc3 --- # persist-client::rpc diff --git a/doc/developer/generated/persist/s3.md b/doc/developer/generated/persist/s3.md index 920dfc5a21efb..b238b52576e97 100644 --- a/doc/developer/generated/persist/s3.md +++ b/doc/developer/generated/persist/s3.md @@ -1,6 +1,6 @@ --- source: src/persist/src/s3.rs -revision: 2a6ac3ab4c +revision: bd6360c23a --- # persist::s3 diff --git a/doc/developer/generated/sql-parser/ast/defs/statement.md b/doc/developer/generated/sql-parser/ast/defs/statement.md index 1330ebb5cc5be..0c0269f0e69be 100644 --- a/doc/developer/generated/sql-parser/ast/defs/statement.md +++ b/doc/developer/generated/sql-parser/ast/defs/statement.md @@ -1,6 +1,6 @@ --- source: src/sql-parser/src/ast/defs/statement.rs -revision: 1e050fdd07 +revision: e28be51f38 --- # mz-sql-parser::ast::defs::statement diff --git a/doc/developer/generated/sql-parser/parser.md b/doc/developer/generated/sql-parser/parser.md index 0a3c99832f0e3..a9deb0f6c06ed 100644 --- a/doc/developer/generated/sql-parser/parser.md +++ b/doc/developer/generated/sql-parser/parser.md @@ -1,6 +1,6 @@ --- source: src/sql-parser/src/parser.rs -revision: 0c2379c49c +revision: efdbfc889c --- # mz-sql-parser::parser diff --git a/doc/developer/generated/sql/plan/statement/ddl.md b/doc/developer/generated/sql/plan/statement/ddl.md index da5b0f0ed14f1..60e19fbbd3473 100644 --- a/doc/developer/generated/sql/plan/statement/ddl.md +++ b/doc/developer/generated/sql/plan/statement/ddl.md @@ -1,6 +1,6 @@ --- source: src/sql/src/plan/statement/ddl.rs -revision: 0c2379c49c +revision: 528c8d60fe --- # mz-sql::plan::statement::ddl diff --git a/doc/developer/generated/sql/plan/statement/ddl/_module.md b/doc/developer/generated/sql/plan/statement/ddl/_module.md index 4e0670ed1965e..854f1b3cfecb9 100644 --- a/doc/developer/generated/sql/plan/statement/ddl/_module.md +++ b/doc/developer/generated/sql/plan/statement/ddl/_module.md @@ -1,6 +1,6 @@ --- source: src/sql/src/plan/statement/ddl.rs -revision: 0c2379c49c +revision: 528c8d60fe --- # mz-sql::plan::statement::ddl diff --git a/doc/developer/generated/sql/plan/statement/dml.md b/doc/developer/generated/sql/plan/statement/dml.md index 705536374eff3..fed64253f1e93 100644 --- a/doc/developer/generated/sql/plan/statement/dml.md +++ b/doc/developer/generated/sql/plan/statement/dml.md @@ -1,6 +1,6 @@ --- source: src/sql/src/plan/statement/dml.rs -revision: f7c755e1ed +revision: 7842a7a152 --- # mz-sql::plan::statement::dml diff --git a/doc/developer/generated/sql/plan/typeconv.md b/doc/developer/generated/sql/plan/typeconv.md index e20232729e732..166ea37b0e81c 100644 --- a/doc/developer/generated/sql/plan/typeconv.md +++ b/doc/developer/generated/sql/plan/typeconv.md @@ -1,6 +1,6 @@ --- source: src/sql/src/plan/typeconv.rs -revision: 3af9082af6 +revision: 51c628afa3 --- # mz-sql::plan::typeconv diff --git a/doc/developer/generated/sql/pure/postgres.md b/doc/developer/generated/sql/pure/postgres.md index 86b783a744858..f3b79e7eb5a72 100644 --- a/doc/developer/generated/sql/pure/postgres.md +++ b/doc/developer/generated/sql/pure/postgres.md @@ -1,9 +1,9 @@ --- source: src/sql/src/pure/postgres.rs -revision: 5f785f23fd +revision: f1e12c2e99 --- # mz-sql::pure::postgres -Postgres-specific purification helpers: validates SELECT/RLS/replica-identity privileges on requested tables, maps Postgres column types (including OID-based casts) to Materialize types, and generates `CreateSubsourceStatement` ASTs from `PostgresTableDesc` descriptions. +Postgres-specific purification helpers: validates SELECT/RLS/replica-identity privileges on requested tables, maps Postgres column types to Materialize types by producing `StorageScalarExpr`-based cast expressions (including OID-based casts), and generates `CreateSubsourceStatement` ASTs from `PostgresTableDesc` descriptions. Consumes a live `tokio_postgres::Client` to introspect the upstream publication. diff --git a/doc/developer/generated/sql/session/vars/value.md b/doc/developer/generated/sql/session/vars/value.md index 89b87fa1d1142..3f199a5bcfb23 100644 --- a/doc/developer/generated/sql/session/vars/value.md +++ b/doc/developer/generated/sql/session/vars/value.md @@ -1,6 +1,6 @@ --- source: src/sql/src/session/vars/value.rs -revision: 7ad5109560 +revision: d2a5974dbd --- # mz-sql::session::vars::value diff --git a/doc/developer/generated/storage-client/client.md b/doc/developer/generated/storage-client/client.md index 7ad013e7fd5f6..f4b20192a4c5f 100644 --- a/doc/developer/generated/storage-client/client.md +++ b/doc/developer/generated/storage-client/client.md @@ -1,6 +1,6 @@ --- source: src/storage-client/src/client.rs -revision: c10148c42f +revision: 00cc513fa5 --- # storage-client::client diff --git a/doc/developer/generated/storage-client/controller.md b/doc/developer/generated/storage-client/controller.md index 82e9acb265184..201af6496f9aa 100644 --- a/doc/developer/generated/storage-client/controller.md +++ b/doc/developer/generated/storage-client/controller.md @@ -1,6 +1,6 @@ --- source: src/storage-client/src/controller.rs -revision: 901d0526a1 +revision: 00cc513fa5 --- # storage-client::controller diff --git a/doc/developer/generated/storage-client/metrics.md b/doc/developer/generated/storage-client/metrics.md index 8c2f72010e1cc..8baf9bbf55d51 100644 --- a/doc/developer/generated/storage-client/metrics.md +++ b/doc/developer/generated/storage-client/metrics.md @@ -1,6 +1,6 @@ --- source: src/storage-client/src/metrics.rs -revision: 19608a8469 +revision: 00cc513fa5 --- # storage-client::metrics diff --git a/doc/developer/generated/storage-client/storage_collections/_module.md b/doc/developer/generated/storage-client/storage_collections/_module.md index 957b3e44d98ec..21baad76096f3 100644 --- a/doc/developer/generated/storage-client/storage_collections/_module.md +++ b/doc/developer/generated/storage-client/storage_collections/_module.md @@ -1,6 +1,6 @@ --- source: src/storage-client/src/storage_collections.rs -revision: 901d0526a1 +revision: 00cc513fa5 --- # storage-client::storage_collections diff --git a/doc/developer/generated/storage-controller/_crate.md b/doc/developer/generated/storage-controller/_crate.md index 186878327aa8f..a38d9034e0be7 100644 --- a/doc/developer/generated/storage-controller/_crate.md +++ b/doc/developer/generated/storage-controller/_crate.md @@ -1,20 +1,20 @@ --- source: src/storage-controller/src/lib.rs -revision: 5f785f23fd +revision: 00cc513fa5 --- # storage-controller -Implements `Controller`, the concrete storage controller that satisfies the `StorageController` trait defined in `mz-storage-client`. +Implements `Controller`, the concrete storage controller that satisfies the `StorageController` trait defined in `mz-storage-client`. The controller manages the lifecycle of all sources, sinks, and tables: it maintains per-collection metadata, coordinates writes to table shards via `PersistTableWriteWorker` and the txn-wal system, drives introspection collection updates through `CollectionManager`, and dispatches ingestion and sink commands to storage instances (clusters). ## Module structure -* `lib.rs` — `Controller` struct and the full `StorageController` trait implementation; crate entry point. -* `instance` — `Instance` and `Replica`: manages one storage cluster and its replica connections, including scheduling and command replay. -* `history` — `CommandHistory`: reducible command log used to rehydrate replicas. -* `collection_mgmt` — `CollectionManager`: background tasks that maintain append-only and differential introspection collections. -* `persist_handles` — `PersistTableWriteWorker`: serializes table writes through txn-wal; includes read-only mode fallback. +* `lib.rs` — `Controller` struct and the full `StorageController` trait implementation; crate entry point. +* `instance` — `Instance` and `Replica`: manages one storage cluster and its replica connections, including scheduling and command replay. +* `history` — `CommandHistory`: reducible command log used to rehydrate replicas. +* `collection_mgmt` — `CollectionManager`: background tasks that maintain append-only and differential introspection collections. +* `persist_handles` — `PersistTableWriteWorker`: serializes table writes through txn-wal; includes read-only mode fallback. * `statistics` — Background scrapers that flush source and sink statistics into managed collections. * `rtr` — Real-time recency timestamp resolution for external source connections. diff --git a/doc/developer/generated/storage-controller/collection_mgmt.md b/doc/developer/generated/storage-controller/collection_mgmt.md index eb235c7ca13b4..a2c1da33e2985 100644 --- a/doc/developer/generated/storage-controller/collection_mgmt.md +++ b/doc/developer/generated/storage-controller/collection_mgmt.md @@ -1,11 +1,11 @@ --- source: src/storage-controller/src/collection_mgmt.rs -revision: 5680493e7d +revision: 00cc513fa5 --- # storage-controller::collection_mgmt -Defines `CollectionManager`, the central coordinator for all storage-managed (introspection) collections, which come in two flavors: append-only and differential. +Defines `CollectionManager`, the central coordinator for all storage-managed (introspection) collections, which come in two flavors: append-only and differential. Append-only collections accept blind writes at any timestamp; differential collections maintain an in-memory desired state and continuously reconcile persist to match it, similar to a self-correcting persist sink. In read-only mode the manager maintains the desired state for differential collections so it can immediately begin writing when leaving read-only mode, but does not write to persist. For append-only collections in read-only mode, calls to `blind_write` panic and attempts to append via `MonotonicAppender` return `StorageError::ReadOnly`. diff --git a/doc/developer/generated/storage-controller/history.md b/doc/developer/generated/storage-controller/history.md index d767a4387f3ed..207d4eb258cd8 100644 --- a/doc/developer/generated/storage-controller/history.md +++ b/doc/developer/generated/storage-controller/history.md @@ -1,10 +1,10 @@ --- source: src/storage-controller/src/history.rs -revision: 82d92a7fad +revision: 00cc513fa5 --- # storage-controller::history -Defines `CommandHistory`, a reducible log of `StorageCommand`s used to rehydrate new or reconnecting storage replicas. +Defines `CommandHistory`, a reducible log of `StorageCommand`s used to rehydrate new or reconnecting storage replicas. The history auto-reduces whenever it doubles in size, discarding dropped ingestions and sinks and retaining only the latest definition and compaction frontier for each live object. It tracks per-command-type metrics and exposes an iterator so `Instance` can replay commands to fresh replicas. diff --git a/doc/developer/generated/storage-controller/instance.md b/doc/developer/generated/storage-controller/instance.md index c51aadffa4d05..a026e3623e789 100644 --- a/doc/developer/generated/storage-controller/instance.md +++ b/doc/developer/generated/storage-controller/instance.md @@ -1,11 +1,11 @@ --- source: src/storage-controller/src/instance.rs -revision: 82d92a7fad +revision: 00cc513fa5 --- # storage-controller::instance -Defines `Instance`, which manages communication with all replicas of one storage instance (cluster), and `Replica`, which handles the connection to a single replica over gRPC. +Defines `Instance`, which manages communication with all replicas of one storage instance (cluster), and `Replica`, which handles the connection to a single replica over gRPC. `Instance` maintains per-object scheduling decisions — single-replica objects (most sinks and some sources) run on the lowest-ID replica, while multi-replica objects run on all replicas — and replays the command history to newly added or reconnected replicas via `CommandHistory`. `Replica` runs a background `ReplicaTask` that connects (with retry) and runs a bidirectional message loop forwarding commands from the controller and responses back up. `ReplicaConfig` bundles the build info, network location, and gRPC parameters needed to connect. diff --git a/doc/developer/generated/storage-controller/persist_handles/_module.md b/doc/developer/generated/storage-controller/persist_handles/_module.md index 68a973a91820d..cc7af4d57ab7e 100644 --- a/doc/developer/generated/storage-controller/persist_handles/_module.md +++ b/doc/developer/generated/storage-controller/persist_handles/_module.md @@ -1,10 +1,10 @@ --- source: src/storage-controller/src/persist_handles.rs -revision: 82d92a7fad +revision: 00cc513fa5 --- # storage-controller::persist_handles -Provides `PersistTableWriteWorker`, a cloneable handle to a background Tokio task that serializes all table writes through a txn-wal `TxnsHandle`. +Provides `PersistTableWriteWorker`, a cloneable handle to a background Tokio task that serializes all table writes through a txn-wal `TxnsHandle`. The worker accepts `Register`, `Update`, `DropHandles`, and `Append` commands over an unbounded channel and executes them in order, coordinating timestamp-ordered appends and txns-shard tidying. The submodule `read_only_table_worker` provides an alternative worker implementation for the read-only mode case that writes directly to persist shards, bypassing txn-wal. diff --git a/doc/developer/generated/storage-controller/persist_handles/read_only_table_worker.md b/doc/developer/generated/storage-controller/persist_handles/read_only_table_worker.md index 06b98054b5923..0ecbea8cc015c 100644 --- a/doc/developer/generated/storage-controller/persist_handles/read_only_table_worker.md +++ b/doc/developer/generated/storage-controller/persist_handles/read_only_table_worker.md @@ -1,6 +1,6 @@ --- source: src/storage-controller/src/persist_handles/read_only_table_worker.rs -revision: 82d92a7fad +revision: 00cc513fa5 --- # storage-controller::persist_handles::read_only_table_worker diff --git a/doc/developer/generated/storage-controller/rtr.md b/doc/developer/generated/storage-controller/rtr.md index 37ddd44ef2069..53cf4014f1641 100644 --- a/doc/developer/generated/storage-controller/rtr.md +++ b/doc/developer/generated/storage-controller/rtr.md @@ -1,6 +1,6 @@ --- source: src/storage-controller/src/rtr.rs -revision: 82d92a7fad +revision: 00cc513fa5 --- # storage-controller::rtr diff --git a/doc/developer/generated/storage-controller/statistics.md b/doc/developer/generated/storage-controller/statistics.md index 207ae0187505a..864b81f26c34f 100644 --- a/doc/developer/generated/storage-controller/statistics.md +++ b/doc/developer/generated/storage-controller/statistics.md @@ -1,6 +1,6 @@ --- source: src/storage-controller/src/statistics.rs -revision: 82d92a7fad +revision: 00cc513fa5 --- # storage-controller::statistics diff --git a/doc/developer/generated/storage-operators/oneshot_source/_module.md b/doc/developer/generated/storage-operators/oneshot_source/_module.md index bd35d1a149b50..12deb8c1f9372 100644 --- a/doc/developer/generated/storage-operators/oneshot_source/_module.md +++ b/doc/developer/generated/storage-operators/oneshot_source/_module.md @@ -1,6 +1,6 @@ --- source: src/storage-operators/src/oneshot_source.rs -revision: 8089fa9c25 +revision: b0fa98e931 --- # storage-operators::oneshot_source diff --git a/doc/developer/generated/storage-operators/persist_source.md b/doc/developer/generated/storage-operators/persist_source.md index 73d4d490385b2..51249ffd4e792 100644 --- a/doc/developer/generated/storage-operators/persist_source.md +++ b/doc/developer/generated/storage-operators/persist_source.md @@ -1,6 +1,6 @@ --- source: src/storage-operators/src/persist_source.rs -revision: 55db0f4ab9 +revision: b0fa98e931 --- # storage-operators::persist_source diff --git a/doc/developer/generated/storage-operators/s3_oneshot_sink/_module.md b/doc/developer/generated/storage-operators/s3_oneshot_sink/_module.md index 1d36006420f45..f5a1b50821ea2 100644 --- a/doc/developer/generated/storage-operators/s3_oneshot_sink/_module.md +++ b/doc/developer/generated/storage-operators/s3_oneshot_sink/_module.md @@ -1,6 +1,6 @@ --- source: src/storage-operators/src/s3_oneshot_sink.rs -revision: f498b6e141 +revision: b0fa98e931 --- # storage-operators::s3_oneshot_sink diff --git a/doc/developer/generated/storage-types/controller.md b/doc/developer/generated/storage-types/controller.md index a1a976fa92e0b..9960900322900 100644 --- a/doc/developer/generated/storage-types/controller.md +++ b/doc/developer/generated/storage-types/controller.md @@ -1,6 +1,6 @@ --- source: src/storage-types/src/controller.rs -revision: 7340287c14 +revision: 00cc513fa5 --- # storage-types::controller diff --git a/doc/developer/generated/storage-types/sources/_module.md b/doc/developer/generated/storage-types/sources/_module.md index d239840947218..a508fdfaf3ef8 100644 --- a/doc/developer/generated/storage-types/sources/_module.md +++ b/doc/developer/generated/storage-types/sources/_module.md @@ -1,6 +1,6 @@ --- source: src/storage-types/src/sources.rs -revision: f2656c001e +revision: f1e12c2e99 --- # storage-types::sources @@ -8,4 +8,4 @@ revision: f2656c001e Defines the core types for source ingestion: `IngestionDescription` (the full source plan including export map and metadata), `SourceDesc` (connection + encoding + envelope), `SourceExport`, `SourceData` (the row-or-error type persisted in source shards), and `MzOffset`. The `SourceConnection` and `SourceTimestamp` traits abstract over the five concrete connector types (Kafka, Postgres, MySQL, SQL Server, load generator). `SourceData` implements `mz_persist_types::Codec` using a custom columnar Arrow-based encoding that stores data rows and error rows in separate columns for efficient filter pushdown. -Submodules `encoding`, `envelope`, `kafka`, `load_generator`, `mysql`, `postgres`, and `sql_server` each define the connection-specific structs for their respective source types. +Submodules `casts`, `encoding`, `envelope`, `kafka`, `load_generator`, `mysql`, `postgres`, and `sql_server` each define the connection-specific structs for their respective source types; `casts` defines `StorageScalarExpr` for source cast expressions. diff --git a/doc/developer/generated/storage-types/sources/postgres.md b/doc/developer/generated/storage-types/sources/postgres.md index 2cb2d74c37447..d5b64bc91e866 100644 --- a/doc/developer/generated/storage-types/sources/postgres.md +++ b/doc/developer/generated/storage-types/sources/postgres.md @@ -1,10 +1,10 @@ --- source: src/storage-types/src/sources/postgres.rs -revision: c637d48dd4 +revision: f1e12c2e99 --- # storage-types::sources::postgres -Defines `PostgresSourceConnection` (connection reference, publication name, publication details snapshot) and `PostgresSourceExportDetails` (per-table output column projection and cast expressions). +Defines `PostgresSourceConnection` (connection reference, publication name, publication details snapshot) and `PostgresSourceExportDetails` (per-table output column projection and cast expressions stored as `Vec<(CastType, StorageScalarExpr)>`). `PostgresSourcePublicationDetails` records the list of `PostgresTableDesc`s captured at source creation time. The progress subsource schema (`PG_PROGRESS_DESC`) tracks a single LSN-derived `MzOffset`. diff --git a/doc/developer/generated/storage/decode/_module.md b/doc/developer/generated/storage/decode/_module.md index 91f9b9af3e8a9..985f99ed2f30b 100644 --- a/doc/developer/generated/storage/decode/_module.md +++ b/doc/developer/generated/storage/decode/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/decode.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::decode diff --git a/doc/developer/generated/storage/healthcheck.md b/doc/developer/generated/storage/healthcheck.md index ddc9103daad32..8a9297d838166 100644 --- a/doc/developer/generated/storage/healthcheck.md +++ b/doc/developer/generated/storage/healthcheck.md @@ -1,6 +1,6 @@ --- source: src/storage/src/healthcheck.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::healthcheck diff --git a/doc/developer/generated/storage/internal_control.md b/doc/developer/generated/storage/internal_control.md index 9a6e1318ac430..28561e26cd521 100644 --- a/doc/developer/generated/storage/internal_control.md +++ b/doc/developer/generated/storage/internal_control.md @@ -1,11 +1,11 @@ --- source: src/storage/src/internal_control.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-storage::internal_control Defines the intra-cluster command bus used by storage workers to broadcast commands to each other in a consistent total order. -`InternalStorageCommand` enumerates all commands that can be so broadcast (create/drop ingestion and sink dataflows, update configuration, propagate statistics). +`InternalStorageCommand` enumerates all commands that can be so broadcast (suspend/restart a dataflow, create ingestion and sink dataflows, run oneshot ingestions, drop dataflows, update configuration, propagate statistics). `InternalCommandSender` and `InternalCommandReceiver` wrap an mpsc channel; `setup_command_sequencer` builds a small timely dataflow that sequences commands arriving from any worker through worker 0 before broadcasting them to all workers, ensuring a deterministic global order. `DataflowParameters` holds runtime-tunable dataflow knobs (currently RocksDB configuration). diff --git a/doc/developer/generated/storage/render/_module.md b/doc/developer/generated/storage/render/_module.md index a3921ed3ccea2..449ee4eba7a3a 100644 --- a/doc/developer/generated/storage/render/_module.md +++ b/doc/developer/generated/storage/render/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/render.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-storage::render diff --git a/doc/developer/generated/storage/render/persist_sink.md b/doc/developer/generated/storage/render/persist_sink.md index b26f616fe0d63..40878eb36fab3 100644 --- a/doc/developer/generated/storage/render/persist_sink.md +++ b/doc/developer/generated/storage/render/persist_sink.md @@ -1,6 +1,6 @@ --- source: src/storage/src/render/persist_sink.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::render::persist_sink diff --git a/doc/developer/generated/storage/render/sinks.md b/doc/developer/generated/storage/render/sinks.md index 3f051922c59ee..ce009dc7bbbf4 100644 --- a/doc/developer/generated/storage/render/sinks.md +++ b/doc/developer/generated/storage/render/sinks.md @@ -1,6 +1,6 @@ --- source: src/storage/src/render/sinks.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::render::sinks diff --git a/doc/developer/generated/storage/render/sources.md b/doc/developer/generated/storage/render/sources.md index 369b9f220dea6..4ac6646f04b58 100644 --- a/doc/developer/generated/storage/render/sources.md +++ b/doc/developer/generated/storage/render/sources.md @@ -1,6 +1,6 @@ --- source: src/storage/src/render/sources.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::render::sources diff --git a/doc/developer/generated/storage/server.md b/doc/developer/generated/storage/server.md index a57cad56d049b..ea90027b2a484 100644 --- a/doc/developer/generated/storage/server.md +++ b/doc/developer/generated/storage/server.md @@ -1,6 +1,6 @@ --- source: src/storage/src/server.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-storage::server diff --git a/doc/developer/generated/storage/sink/iceberg.md b/doc/developer/generated/storage/sink/iceberg.md index 163dfee70f717..25b375ca5fcb2 100644 --- a/doc/developer/generated/storage/sink/iceberg.md +++ b/doc/developer/generated/storage/sink/iceberg.md @@ -1,6 +1,6 @@ --- source: src/storage/src/sink/iceberg.rs -revision: 0c2379c49c +revision: b0fa98e931 --- # mz-storage::sink::iceberg diff --git a/doc/developer/generated/storage/sink/kafka.md b/doc/developer/generated/storage/sink/kafka.md index d6c6096d66b5a..9fc923c4cc2b5 100644 --- a/doc/developer/generated/storage/sink/kafka.md +++ b/doc/developer/generated/storage/sink/kafka.md @@ -1,6 +1,6 @@ --- source: src/storage/src/sink/kafka.rs -revision: 0c2379c49c +revision: b0fa98e931 --- # mz-storage::sink::kafka diff --git a/doc/developer/generated/storage/source/generator/_module.md b/doc/developer/generated/storage/source/generator/_module.md index e3db049c77ad2..0aa1799364d5f 100644 --- a/doc/developer/generated/storage/source/generator/_module.md +++ b/doc/developer/generated/storage/source/generator/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/generator.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::generator diff --git a/doc/developer/generated/storage/source/generator/key_value.md b/doc/developer/generated/storage/source/generator/key_value.md index 2027fb2df18e4..22cad26f03d83 100644 --- a/doc/developer/generated/storage/source/generator/key_value.md +++ b/doc/developer/generated/storage/source/generator/key_value.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/generator/key_value.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::generator::key_value diff --git a/doc/developer/generated/storage/source/kafka.md b/doc/developer/generated/storage/source/kafka.md index f42fc7dbbe52f..fa01dba865439 100644 --- a/doc/developer/generated/storage/source/kafka.md +++ b/doc/developer/generated/storage/source/kafka.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/kafka.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::kafka diff --git a/doc/developer/generated/storage/source/mysql/_module.md b/doc/developer/generated/storage/source/mysql/_module.md index d002dbf2a9123..8bb60dcbc4850 100644 --- a/doc/developer/generated/storage/source/mysql/_module.md +++ b/doc/developer/generated/storage/source/mysql/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/mysql.rs -revision: f498b6e141 +revision: b0fa98e931 --- # mz-storage::source::mysql diff --git a/doc/developer/generated/storage/source/mysql/replication/_module.md b/doc/developer/generated/storage/source/mysql/replication/_module.md index c7d66d18b4e74..9d79a8c1a7ec1 100644 --- a/doc/developer/generated/storage/source/mysql/replication/_module.md +++ b/doc/developer/generated/storage/source/mysql/replication/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/mysql/replication.rs -revision: 9e91428d8a +revision: b0fa98e931 --- # mz-storage::source::mysql::replication diff --git a/doc/developer/generated/storage/source/mysql/snapshot.md b/doc/developer/generated/storage/source/mysql/snapshot.md index 826dc9ffd1279..07f132ee3737e 100644 --- a/doc/developer/generated/storage/source/mysql/snapshot.md +++ b/doc/developer/generated/storage/source/mysql/snapshot.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/mysql/snapshot.rs -revision: 9e91428d8a +revision: b0fa98e931 --- # mz-storage::source::mysql::snapshot diff --git a/doc/developer/generated/storage/source/mysql/statistics.md b/doc/developer/generated/storage/source/mysql/statistics.md index 29597f6fbd9e1..1d20a7f49ceaa 100644 --- a/doc/developer/generated/storage/source/mysql/statistics.md +++ b/doc/developer/generated/storage/source/mysql/statistics.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/mysql/statistics.rs -revision: 9e91428d8a +revision: b0fa98e931 --- # mz-storage::source::mysql::statistics diff --git a/doc/developer/generated/storage/source/postgres/_module.md b/doc/developer/generated/storage/source/postgres/_module.md index a902a0efc3b85..0e5a327943d6f 100644 --- a/doc/developer/generated/storage/source/postgres/_module.md +++ b/doc/developer/generated/storage/source/postgres/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/postgres.rs -revision: e79a6d96d9 +revision: f1e12c2e99 --- # mz-storage::source::postgres diff --git a/doc/developer/generated/storage/source/postgres/replication.md b/doc/developer/generated/storage/source/postgres/replication.md index 72a10d00552b0..32f7ef30d7150 100644 --- a/doc/developer/generated/storage/source/postgres/replication.md +++ b/doc/developer/generated/storage/source/postgres/replication.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/postgres/replication.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::postgres::replication diff --git a/doc/developer/generated/storage/source/postgres/snapshot.md b/doc/developer/generated/storage/source/postgres/snapshot.md index 09db11e6423d0..204b5331bf5e2 100644 --- a/doc/developer/generated/storage/source/postgres/snapshot.md +++ b/doc/developer/generated/storage/source/postgres/snapshot.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/postgres/snapshot.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::postgres::snapshot diff --git a/doc/developer/generated/storage/source/source_reader_pipeline.md b/doc/developer/generated/storage/source/source_reader_pipeline.md index 6daadc15da767..e97360f347b14 100644 --- a/doc/developer/generated/storage/source/source_reader_pipeline.md +++ b/doc/developer/generated/storage/source/source_reader_pipeline.md @@ -1,10 +1,10 @@ --- source: src/storage/src/source/source_reader_pipeline.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::source_reader_pipeline Implements `create_raw_source`, the function that turns a `SourceRender` implementation and a `RawSourceCreationConfig` into raw timely streams ready for reclocking and decoding. -It renders the source in a `FromTime` child scope using a `tokio::sync::watch` channel to pass the resumption upper, captures the output streams using `PusherCapture` to cross scope boundaries, creates the remap operator to write timestamp bindings into the remap shard, and feeds bindings into the `reclock` utility to produce an `IntoTime`-timestamped data stream. +It renders the source in a `SourceTimeDomain` root scope using a `tokio::sync::watch` channel to pass the probed upstream frontier, captures the output streams using `PusherCapture` to cross scope boundaries, creates the remap operator to write timestamp bindings into the remap shard, and feeds bindings into the `reclock` utility to produce an `IntoTime`-timestamped data stream. `RawSourceCreationConfig` bundles all per-source creation metadata (id, exports, as-of, resume uppers, metrics, persist clients, etc.). diff --git a/doc/developer/generated/storage/source/sql_server/_module.md b/doc/developer/generated/storage/source/sql_server/_module.md index ec64c88c49bae..fc2f996f02b74 100644 --- a/doc/developer/generated/storage/source/sql_server/_module.md +++ b/doc/developer/generated/storage/source/sql_server/_module.md @@ -1,10 +1,10 @@ --- source: src/storage/src/source/sql_server.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::sql_server Implements `SourceRender` for `SqlServerSourceConnection`, composing replication and progress operators into a SQL Server CDC ingestion dataflow. -Per-capture-instance `SourceOutputInfo` structs carry the decoder, resume LSN, and partition index needed by the replication operator. +Per-capture-instance `SourceOutputInfo` structs carry the decoder, resume LSN, initial LSN, and partition index needed by the replication operator. Submodules `replication` and `progress` implement the two operator families. diff --git a/doc/developer/generated/storage/source/sql_server/progress.md b/doc/developer/generated/storage/source/sql_server/progress.md index 66f3fe573f280..665191ed378bb 100644 --- a/doc/developer/generated/storage/source/sql_server/progress.md +++ b/doc/developer/generated/storage/source/sql_server/progress.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/sql_server/progress.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # mz-storage::source::sql_server::progress diff --git a/doc/developer/generated/storage/source/sql_server/replication.md b/doc/developer/generated/storage/source/sql_server/replication.md index e22bb305a7589..0c57aaf143317 100644 --- a/doc/developer/generated/storage/source/sql_server/replication.md +++ b/doc/developer/generated/storage/source/sql_server/replication.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/sql_server/replication.rs -revision: f498b6e141 +revision: b0fa98e931 --- # mz-storage::source::sql_server::replication diff --git a/doc/developer/generated/storage/source/types.md b/doc/developer/generated/storage/source/types.md index 4fee79abff541..131a9d487214d 100644 --- a/doc/developer/generated/storage/source/types.md +++ b/doc/developer/generated/storage/source/types.md @@ -1,6 +1,6 @@ --- source: src/storage/src/source/types.rs -revision: f498b6e141 +revision: b0fa98e931 --- # mz-storage::source::types diff --git a/doc/developer/generated/storage/storage_state/_module.md b/doc/developer/generated/storage/storage_state/_module.md index 38ac1baa1c08e..1903a0903dcb6 100644 --- a/doc/developer/generated/storage/storage_state/_module.md +++ b/doc/developer/generated/storage/storage_state/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/storage_state.rs -revision: 3ac449b444 +revision: b0fa98e931 --- # mz-storage::storage_state diff --git a/doc/developer/generated/storage/upsert/_module.md b/doc/developer/generated/storage/upsert/_module.md index 633616db33afd..79df12eb92dcd 100644 --- a/doc/developer/generated/storage/upsert/_module.md +++ b/doc/developer/generated/storage/upsert/_module.md @@ -1,6 +1,6 @@ --- source: src/storage/src/upsert.rs -revision: 5680493e7d +revision: b0fa98e931 --- # mz-storage::upsert diff --git a/doc/developer/generated/storage/upsert_continual_feedback.md b/doc/developer/generated/storage/upsert_continual_feedback.md index e8a6108b24370..650c1f830fca7 100644 --- a/doc/developer/generated/storage/upsert_continual_feedback.md +++ b/doc/developer/generated/storage/upsert_continual_feedback.md @@ -1,6 +1,6 @@ --- source: src/storage/src/upsert_continual_feedback.rs -revision: 5680493e7d +revision: b0fa98e931 --- # mz-storage::upsert_continual_feedback diff --git a/doc/developer/generated/timely-util/activator.md b/doc/developer/generated/timely-util/activator.md index da4a2d373977a..00e6653f2f163 100644 --- a/doc/developer/generated/timely-util/activator.md +++ b/doc/developer/generated/timely-util/activator.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/activator.rs -revision: f87a1bceae +revision: b0fa98e931 --- # timely-util::activator diff --git a/doc/developer/generated/timely-util/builder_async.md b/doc/developer/generated/timely-util/builder_async.md index 2f423317deccb..fee44f1338a19 100644 --- a/doc/developer/generated/timely-util/builder_async.md +++ b/doc/developer/generated/timely-util/builder_async.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/builder_async.rs -revision: 834eb88f27 +revision: b0fa98e931 --- # timely-util::builder_async diff --git a/doc/developer/generated/timely-util/operator.md b/doc/developer/generated/timely-util/operator.md index 4185769c24e78..ae34521cf31bd 100644 --- a/doc/developer/generated/timely-util/operator.md +++ b/doc/developer/generated/timely-util/operator.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/operator.rs -revision: 834eb88f27 +revision: b0fa98e931 --- # timely-util::operator diff --git a/doc/developer/generated/timely-util/pact.md b/doc/developer/generated/timely-util/pact.md index dfa92bb227020..771280cf9eb71 100644 --- a/doc/developer/generated/timely-util/pact.md +++ b/doc/developer/generated/timely-util/pact.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/pact.rs -revision: f641a29d4a +revision: b0fa98e931 --- # timely-util::pact diff --git a/doc/developer/generated/timely-util/probe.md b/doc/developer/generated/timely-util/probe.md index 6a8399ed34da1..9ed94acd3bfc2 100644 --- a/doc/developer/generated/timely-util/probe.md +++ b/doc/developer/generated/timely-util/probe.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/probe.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # timely-util::probe diff --git a/doc/developer/generated/timely-util/reclock.md b/doc/developer/generated/timely-util/reclock.md index 7adf7771f2b7a..27b8abb917222 100644 --- a/doc/developer/generated/timely-util/reclock.md +++ b/doc/developer/generated/timely-util/reclock.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/reclock.rs -revision: c642b63c77 +revision: b0fa98e931 --- # timely-util::reclock diff --git a/doc/developer/generated/timely-util/replay.md b/doc/developer/generated/timely-util/replay.md index 3c3646198c443..0f0848cc19752 100644 --- a/doc/developer/generated/timely-util/replay.md +++ b/doc/developer/generated/timely-util/replay.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/replay.rs -revision: e79a6d96d9 +revision: b0fa98e931 --- # timely-util::replay diff --git a/doc/developer/generated/timely-util/scope_label.md b/doc/developer/generated/timely-util/scope_label.md index 0643f802ef887..54622282eeba6 100644 --- a/doc/developer/generated/timely-util/scope_label.md +++ b/doc/developer/generated/timely-util/scope_label.md @@ -1,9 +1,9 @@ --- source: src/timely-util/src/scope_label.rs -revision: c642b63c77 +revision: b0fa98e931 --- # timely-util::scope_label -Provides `LabelledScope` and `LabelledOperator`, which wrap a timely `Scope` and `Operate` implementation to set a profiling label (via `custom_labels::with_label`) on every operator scheduling call. -`ScopeExt::with_label` converts any scope into a `LabelledScope` that uses the scope's name as the label, enabling CPU profilers to attribute work to the enclosing dataflow scope. +Provides the `ScopeExt` extension trait for timely `Scope`, which exposes a `with_label` method. +The method is currently a no-op that returns `self` unchanged; the intended behavior of setting a profiling label before scheduling child operators is not yet implemented. diff --git a/doc/developer/generated/timely-util/temporal.md b/doc/developer/generated/timely-util/temporal.md index 020f94dace973..68f61025f6721 100644 --- a/doc/developer/generated/timely-util/temporal.md +++ b/doc/developer/generated/timely-util/temporal.md @@ -1,6 +1,6 @@ --- source: src/timely-util/src/temporal.rs -revision: a048f88ed6 +revision: d90aafba34 --- # timely-util::temporal diff --git a/doc/developer/generated/txn-wal/operator.md b/doc/developer/generated/txn-wal/operator.md index 930c8fc64f367..a9da6dbe15686 100644 --- a/doc/developer/generated/txn-wal/operator.md +++ b/doc/developer/generated/txn-wal/operator.md @@ -1,6 +1,6 @@ --- source: src/txn-wal/src/operator.rs -revision: 901d0526a1 +revision: b0fa98e931 --- # mz-txn-wal::operator