Skip to content

skills: Redpanda Connect skill updates for v4.115.1 (2026-10-30) - #118

Merged
micheleRP merged 2 commits into
mainfrom
claude/sync-skills-2026-10-30
Oct 31, 2026
Merged

micheleRP merged 2 commits into
mainfrom
claude/sync-skills-2026-10-30

Conversation

@micheleRP

@micheleRP micheleRP commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Scheduled Connect sync against Connect v4.115.1 (current stable tag; changelog section dated 2026-10-23, nothing newer tagged). The benthos dependency is unchanged at v4.84.0 in go.mod at the tag, so no engine, CLI or Bloblang surface moved with the release.

The release itself

Nothing in the v4.115.1 changelog is a structural skill surface:

  • The two kafka entries are fixes inside the franz writer — a data race between the produce-error and record-decoration slices, and the dispatch signal now firing when a record is handed to the producer, restoring the hand-off the ordered per-partition input relies on. No flag, field, component name or documented behaviour changes, and no skill asserted the broken behaviour.
  • The oracledb_cdc entry is a schema-unit change, applied below.

The remaining changes are drift corrections and two durable CDC constraints the skills were missing. Per-field additions elsewhere in the window stay deferred to the generated reference and rpk connect create.

Changes

skills/connect/references/migration.md (Migration row of references/SOURCES.md)

Change Verified in
The migrator input's lag gauge is redpanda_lag, not input_redpanda_migrator_lag (absent from the tree), and it is emitted only when the input has a consumer_group redpanda-data/connect internal/impl/kafka/lag.go (lagMetricNameRedpanda), internal/impl/redpanda/migrator/migrator.go (both specs) and internal/impl/redpanda/migrator/README.md, at v4.115.1
A single migrator pair needs no labels; labels matter only with several pairs in one config, and a mismatched pair fails to connect rather than silently mis-coordinating. Labels dropped from the workflow YAML internal/impl/redpanda/migrator/migrator.go (newMigratorFrom's migratorKey{label, stream} with its "default" fallback, the input spec's pairing paragraph, validateInitialized)
Label syntax replaced with the engine's own rule (lowercase [a-z0-9_-], no leading _/-); the previous "3–128 chars, A-Za-z0-9-_" claim was wrong on both counts redpanda-data/benthos internal/docs/config.go (labelExpression, ValidateLabel) at v4.84.0
Exact consumer-group offset translation for groups in the Empty state via the output's offset header — the skill presented translation as timestamp-based and approximate, full stop internal/impl/redpanda/migrator/migrator_groups.go (tryFindExactOffset, gated on State == "Empty" and a non-empty header) and the offset_header field spec in migrator.go
An existing destination topic with fewer partitions than its source has partitions added to match ("never overwritten" overstated it) the output spec's guarantees in internal/impl/redpanda/migrator/migrator.go

skills/connect-cdc-postgres/ (SKILL.md, references/config-reference.md)

  • postgres_cdc requires the primary instance — a read replica or physical standby is not supported. A product requirement stated in the connector's reference documentation; provenance is recorded in the skill's references/SOURCES.md.
  • The snapshot phase holds the database's xmin horizon for as long as it runs, for every table, so a scan measured in hours defers cleanup of row versions that died after it started; large tables should use an incremental snapshot. Verified in internal/impl/postgresql/pglogicalstream/snapshotter.go (openTxn: read-only REPEATABLE READ on a background context, then SET TRANSACTION SNAPSHOT; rolled back only in releaseSnapshot) and pglogicalstream/logical_stream.go (SnapshotAction: "EXPORT_SNAPSHOT").

No postgres_cdc source changed between v4.115.0 and the tag.

skills/connect-cdc-oracle/references/pipeline-and-output.md

  • TIMESTAMP columns declare microseconds in the schema the schema metadata field carries (DATE keeps milliseconds), so TIMESTAMP(6) is no longer truncated and the snapshot and streaming schemas agree; a consumer holding the pre-4.115.1 schema sees the unit change. Verified in internal/impl/oracledb/schema.go (columnToCommon, oracleTimestampLogical) and internal/impl/oracledb/TYPES.md at the tag. The JSON wire format of a timestamp is unchanged, so the type table still reads ISO-8601.

Checked and deliberately not documented

  • The oracledb_cdc / microsoft_sql_server_cdc batcher refactor onto a shared flush-ticket package, and the second "flush queue is sealed" variant of their rebuild-publisher warning: internal, and no skill quotes those log lines.
  • Durability: no field list, default, Bloblang catalog or component roster was pinned; the migrator's offset_header is named only where its semantics change a documented guarantee.

TODO for a human

Duplicate-label linting versus the documented multi-pair scheme. benthos v4.84.0 registers every component label in one per-run lint context and returns a duplicate-label lint for a label used twice in a config, which appears to collide with the migrator's documented scheme of giving an input and its paired output the same label. migration.md now tells readers to lint a multi-pair config and confirm the behaviour on their version rather than asserting either way. Worth a Connect-team answer on whether a deliberately matched pair lints clean.


🤖 Generated with Claude Code

micheleRP and others added 2 commits October 30, 2026 07:12
Synced the Connect skills against Connect v4.115.1 (benthos v4.84.0,
unchanged). The release itself carries no structural change: its two kafka
entries are franz-writer fixes and its oracledb_cdc entry is a schema-unit
change, handled below. The rest of this change corrects migrator drift and
records two durable CDC constraints the skills were missing.

connect/references/migration.md
- The migrator input's lag gauge is `redpanda_lag`, not
  `input_redpanda_migrator_lag` (which no longer exists in the tree), and it
  is emitted only when the input has a `consumer_group`
  (internal/impl/kafka/lag.go, internal/impl/redpanda/migrator/migrator.go).
- A single migrator pair needs no labels: the shared Migrator is keyed by
  label with a "default" fallback. Labels matter only with several pairs in
  one config, where a mismatch makes the pair fail to connect rather than
  silently mis-coordinate. Dropped the labels from the workflow YAML and
  replaced the incorrect label-syntax claim with the engine's own rule
  (benthos internal/docs/config.go).
- Documented exact consumer-group offset translation for groups in the Empty
  state via the output's offset header, which the "timestamp-based and
  approximate" guarantee had flattened
  (internal/impl/redpanda/migrator/migrator_groups.go).
- Recorded that an existing destination topic with fewer partitions than its
  source has partitions added to match.

connect-cdc-postgres
- postgres_cdc requires the primary instance; a read replica or physical
  standby is not supported. Product requirement stated in the connector's
  reference documentation; provenance in the skill's SOURCES.md.
- The snapshot phase reads through long-lived REPEATABLE READ transactions on
  an exported snapshot, holding the database's xmin horizon for its whole
  duration; large tables should use an incremental snapshot
  (internal/impl/postgresql/pglogicalstream/snapshotter.go, logical_stream.go).

connect-cdc-oracle
- TIMESTAMP columns declare microseconds in the emitted schema (DATE keeps
  milliseconds), so TIMESTAMP(6) is no longer truncated and snapshot and
  streaming now agree (internal/impl/oracledb/schema.go, TYPES.md).

Each skill's SOURCES.md carries the full sync log, including what was checked
and deliberately not documented, and the open question about duplicate-label
linting versus the documented multi-pair scheme.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s best-effort

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@micheleRP
micheleRP merged commit 6ce212a into main Oct 31, 2026
1 check passed
@micheleRP
micheleRP deleted the claude/sync-skills-2026-10-30 branch October 31, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant