hacking.postgres.tv
./menu

postgres=# select * from sessions where topic = 'wait-events';

#wait events

6 related sessions.

· 77:12 · youtube ↗

Agent hacking Postgres

Agent-assisted patch work around the wait-event timing submission.

The group reviewed Kirk Wolak's pg_dump patch for keeping loadable placeholders when table data is excluded, grounding the discussion in his large-table restore workflow. They debated a positive interface, filter-file actions, arbitrary dump queries, PII transformation, and referential-integrity risks, then posted supportive review feedback to pgsql-hackers. The final segment examined Salma El-Sayed's B-tree leaf-page merge design, including merged-page markers, scan correctness, vacuum cleanup, and review plans.

[timecodes · 8]
  1. Postgres 19 development opens and patches await review
  2. Kirk introduces the pg_dump placeholder patch
  3. Large-table dump and one-command restore use case
  4. Reviewing the option name and user interface
  5. Using AI to review design and mailing-list history
  6. Parallel chunks for restoring one large table
  7. Custom dump queries, filtering, and PII transformation
  8. B-tree leaf-page merge design and scan safety
#wait events#B-tree#AI-assisted hacking#hacking process@Nikolay Samokhvalov@Andrey Borodin@Kirk Wolak
-hackersCF #6984
active

correlated mapping
· 93:47 · youtube ↗

Wait-event tracing

Independent benchmarks, observer effect, coverage, and patch restructuring.

Dmitry Fomin presented a seven-patch wait-event tracing series, split for review but intended to provide complete transition timing and query attribution. The group designed an independent benchmark ladder covering vanilla, compiled-disabled, stats, and full trace modes; corrected pgbench workloads; and distinguished static overhead from observer effect. Preliminary runs showed no clear slowdown, but they added pg_stat_statements, wait-event counts, repeated rounds, raw results, and independent review before submission.

[timecodes · 8]
  1. Why sampled wait events miss short transitions
  2. Seven-patch tracing series and review structure
  3. Planning an independent overhead benchmark
  4. Designing high-transition pgbench workloads
  5. Five measurement levels and observer effect
  6. Stats versus full trace capture modes
  7. Correcting workload duration, rate limits, and logging
  8. Preliminary results and adding pg_stat_statements
#wait events#monitoring@Nikolay Samokhvalov@Kirk Wolak@Dmitry Fomin
benchmark #48coverage #49PR #2benchmark briefCF #6984
active

direct mapping
· 60:51 · youtube ↗

Precise wait-event tracing, part 2

Low-overhead per-event timing and trace capture.

The group compared pg_wait_tracer's external eBPF hardware-watchpoint tracing with Dmitri Fomin's new Postgres patch, which records wait transitions into per-backend lock-free ring buffers. They designed a shared-buffer-thrashing benchmark to stress transition overhead, explored transition, concurrency, and query-profile visualizations, and discussed eventual EXPLAIN integration. Preliminary claims of roughly 2% patched overhead were treated as unverified; the agreed next steps were finish benchmarks, publish the tool, review the code, and prepare a pgsql-hackers proposal.

[timecodes · 8]
  1. How precise wait-event tracing works
  2. ASH dashboard, AAS, and wait hierarchy
  3. Why tracing needs a Postgres patch
  4. Designing a transition-heavy overhead benchmark
  5. Per-backend lock-free ring-buffer design
  6. Transition paths and concurrency analysis
  7. Toward wait events in EXPLAIN
  8. Publishing and presenting the patch upstream
#wait events#monitoring@Nikolay Samokhvalov@Kirk Wolak@Dmitry Fomin
PR #2-hackers v1CF #6984
active

direct mapping
· 60:12 · youtube ↗

Wait-event tracing, part 1

pg_wait_tracer, core timing, USDT probes, and observer-effect trade-offs.

Dmitri Fomin demonstrated pg_wait_tracer, an eBPF-based tool that captures every Postgres wait-event transition with microsecond-to-nanosecond timestamps, replayable trace files, latency heatmaps, and query drill-down. The motivating case was intermittent primary-key lookups jumping from milliseconds to a one-second timeout. The group identified inlined pgstat_report_wait_start/end calls as the obstacle to ordinary probes, produced a small USDT-probe prototype, and planned worst-case overhead benchmarks before proposing always-on core instrumentation.

[timecodes · 8]
  1. ASH, wait-event gaps, and pg_wait_tracer
  2. The intermittent one-second query mystery
  3. Sampling versus complete wait tracing
  4. Finding a trace path with hardware watchpoints
  5. Live ASH-style trace visualization
  6. Latency heatmaps and source-code attribution
  7. Transition chains for root-cause analysis
  8. USDT probes, core patch, and benchmark plan
#wait events#monitoring@Nikolay Samokhvalov@Kirk Wolak@Dmitry Fomin
USDT PR #18timing PR #1-hackers
active

direct mapping
· 87:51 · youtube ↗

Hacking Postgres

Discuss wait-event coverage gaps; follow-up work added COPY file and PROGRAM waits upstream.

Starting from an AI-assisted audit of missing wait-event coverage, the group distinguished genuine waits from CPU work and chose COPY as a testable first target. They reproduced an unreported wait in COPY FROM PROGRAM, prototyped events for program input and output, and used pg_stat_activity and pg_wait_sampling to validate them. The investigation also exposed missing coverage for ordinary COPY file reads and writes, expanding the planned patch to all four paths; the file/program work later reached core.

[timecodes · 8]
  1. AI-assisted audit of missing wait events
  2. Review discipline and excluding CPU work
  3. Choosing COPY FROM PROGRAM as the first target
  4. Reproducing the unreported external-program wait
  5. Prototyping COPY PROGRAM wait events
  6. Testing COPY TO PROGRAM and separating code paths
  7. Discovering ordinary COPY file I/O gaps
  8. Validating coverage with pg_wait_sampling
#wait events#MultiXact#monitoring@Nikolay Samokhvalov@Andrey Borodin@Kirk Wolak
gap analysiscoverage PR #2COPY waits threadupstream commit
committed

direct mapping
· 81:56 · youtube ↗

Vibe Hacking with Dmitry Fomin

Wait-event observability and precise tracing ideas.

Dmitry Fomin presented his first core patch after tracing synchronous-replication latency spikes to blocking fdatasync calls in the standby WAL receiver, even under synchronous_commit=remote_write. His prototype batches flushes by byte threshold or timeout. Review uncovered unsafe defaults for stronger synchronous modes and suggested a separate flusher process as a cleaner long-term design. They rebased the old patch onto master, removed redundant unflushed-byte state, compiled it, and prepared it for pgsql-hackers discussion.

[timecodes · 8]
  1. Production SyncRep latency problem
  2. fdatasync stalls in remote_write mode
  3. Batching WAL flushes by bytes or timeout
  4. What synchronous replication actually guarantees
  5. Reviewing the new WAL-receiver settings
  6. Defaults break stronger synchronous-commit modes
  7. Alternative design with a separate WAL flusher
  8. Rebasing, reviewing, and preparing the patch
#wait events#monitoring@Nikolay Samokhvalov@Andrey Borodin@Dmitry Fomin
pg_wait_tracertiming PR #1
active

correlated mapping