hacking.postgres.tv
./menu

postgres=# select * from projects where slug = 'logical-decoding-archives';

Logical decoding from archive-fed standbys

Keep logical slots usable on a standby fed by WAL archives by pausing and safely resuming recovery around conflicts.

active[blueprint #24 ↗][patch PR #40 ↗][test plan #43 ↗]

current state

active

blocker: Crash-safety and promotion behavior need complete end-to-end coverage before upstream submission.

next

  • Complete crash/restart matrix
  • Test promotion and rewind paths
  • Prepare -hackers submission

sessions

5
· 78:11 · youtube ↗

Agentic Hacking Postgres

End-to-end tests for archive-fed logical decoding and pause/resume behavior.

The session refined logical decoding from an archive-fed physical standby as a recoverable, production-independent pipeline: after losing the decoding node, a consumer can restore another standby at the acknowledged LSN and tolerate overlap. They reviewed the three-patch implementation and TAP test, found registration and naming issues, debated lock structure, and judged the code ready for discussion after fixes. They then specified an end-to-end WAL-G test with catalog churn, paused consumption, catch-up, and a final marker.

[timecodes · 8]
  1. Recoverable decoding after a standby is lost
  2. Archive-fed WAL decoding without touching production
  3. Naming the feature: logical decoding from archive
  4. Reviewing the three-patch implementation
  5. TAP test registration and test readability
  6. Reviewing pause-on-conflict code and locking
  7. Patch readiness and concrete submission fixes
  8. Designing an end-to-end WAL-G recovery test
#logical decoding#recovery#WAL#AI-assisted hacking@Nikolay Samokhvalov@Kirk Wolak
test design #43
active

direct mapping
· 53:58 · youtube ↗

Logical replication from physical standbys on WAL shipping

Refine recovery pause behavior, crash safety, promotion, and auto-resume.

The group inspected two AI-produced patch branches for pausing archive recovery when replay would invalidate a logical slot. They reconstructed the patch series, reviewed synchronized-slot and promotion behavior, and clarified that dirty slot state is only persisted at restart points, weakening crash guarantees. The concrete conclusion was to favor correctness, consolidate the work into one reviewable three-patch series, preserve readable code over premature lock optimization, and target the July CommitFest after further testing.

[timecodes · 8]
  1. Returning to logical decoding on WAL-shipping standbys
  2. Recovering useful changes from a broken pull request
  3. Reviewing pause and auto-resume patches
  4. Conflict horizons and synchronized-slot behavior
  5. Dirty slot state and restart-point persistence
  6. Live transcript-assisted code investigation
  7. Restructuring the work into reviewable patches
  8. Crash-safety invariant and correctness-first decision
#logical decoding#recovery#WAL@Nikolay Samokhvalov@Andrey Borodin
PR #40
active

direct mapping
· 93:07 · youtube ↗

Hacking Postgres with Andrey, Kirk, Nik

Pause replay instead of invalidating conflicting logical slots; design auto-resume.

The group connected archive-fed logical decoding to resilient analytical replication: replay should pause before catalog cleanup invalidates a lagging slot, then resume automatically once consumption catches up. They manually audited an AI-generated prototype and its complex TAP test, discovering that the test needed clearer assertions around the actual paused state. They accepted a proof-of-concept pause GUC, rejected manual resume as incomplete, and designed auto-resume behavior for drained, dropped, or otherwise nonblocking slots.

[timecodes · 8]
  1. Archive-fed decoding for resilient analytics
  2. From meeting transcript to AI-built prototype
  3. Logical archive during large initial synchronization
  4. Pause replay instead of blocking primary vacuum
  5. Opening the prototype for manual review
  6. Understanding the pause-on-conflict TAP test
  7. Missing assertions for the paused recovery state
  8. Designing automatic recovery resume
#logical decoding#recovery@Nikolay Samokhvalov@Andrey Borodin@Kirk Wolak
PR #27PR #30
active

direct mapping
· 39:42 · youtube ↗

Logical decoding of archived WALs

Use an archive-fed standby as a logical-decoding engine.

The session began by testing whether Postgres 16's logical decoding on a standby could turn an archive-fed recovery instance into a detached decoding engine. The group concluded a full server is the practical first implementation because it supplies catalog snapshots and output plugins; the missing behavior is to pause replay when it would invalidate a slot and continue when consumption advances. They also identified broader uses: per-record WAL verification, recovery debugging, and filtered, PII-free staging streams.

[timecodes · 8]
  1. Why decode archived WAL away from production
  2. Using an archive-fed standby as the decoder
  3. Standalone WAL tool versus a full Postgres server
  4. Slot conflicts and the need to pause replay
  5. Choosing a recovery mode driven by consumption
  6. Function-based recovery targets
  7. Stepping through WAL for correctness testing
  8. Filtered, PII-free staging from archived WAL
#logical decoding#recovery#WAL@Nikolay Samokhvalov@Andrey Borodin@Kirk Wolak
blueprint PR #24Sprint 0 #25PR #27
active

direct mapping
· 55:36 · youtube ↗

Hacking Postgres with Kirk, Andrey, Nik

Custom command tags and logical decoding from an archive-fed recovery server.

The workshop first revisited customizable command tags, confirming legacy behavior remains the default and its added branch should be negligible, then explored extensible client protocols and better psql completion. The main direction was extracting data from managed Postgres without burdening production: decode archived WAL into a logical stream. They compared a full recovery server with a standalone tool, concluding the latter would need schema state at a starting LSN, checkpoint/full-page-image handling, and careful treatment of file swaps such as pg_repack.

[timecodes · 8]
  1. Command-tag customization and performance feedback
  2. Ideas for an extensible client/server protocol
  3. Language-server completion for psql
  4. Parallelizing logical-replication initialization
  5. Keeping an independent copy outside managed Postgres
  6. Turning archived WAL into a logical stream
  7. Designing a standalone offline decoder
  8. Checkpoints, full-page images, and pg_repack caveats
#logical decoding#recovery#WAL#hacking process@Nikolay Samokhvalov@Andrey Borodin@Kirk Wolak
no public artifact mapped
exploration

direct mapping