hacking.postgres.tv
./menu

postgres@hacking:~$ grep -R '#compression' sessions/

#compression

4 related sessions.

· 72:37 · youtube ↗

AI or not AI; GSoC 2026; improving pglz compression

Evaluate agentic hacking, develop GSoC project ideas, and experiment with pglz compression.

The first half examined AI-assisted Postgres development: community overload, specification-first agent workflows, independent review, and personal responsibility for generated work. The second half audited an agent-produced pglz optimization series. Existing benchmarks mixed runtime and work done, while some optimizations changed compressed output. The group therefore narrowed the project to byte-compatible changes, required old-versus-new output tests, fixed-transaction-count and multi-platform benchmarks, and a small reproducible pgsql-hackers result; a reported 14% workload gain still needed verification.

[timecodes · 8]
  1. AI risks for the Postgres community
  2. Specification-first autonomous agent workflows
  3. Quality, review, and ownership of AI output
  4. AI policy for Google Summer of Code
  5. Reviewing the pglz optimization project
  6. Why the benchmark methodology was misleading
  7. Compression correctness and corruption tests
  8. Byte-compatible scope and revised benchmark plan
#compression#AI-assisted hacking@Nikolay Samokhvalov@Andrey Borodin@Kirk Wolak
pglz experimentGSoC ideaswait coveragepgss scalabilityB-tree merge
exploration

direct mapping
· 55:46 · youtube ↗

Continue with WAL compression

Continue the B-tree/WAL compression prototype.

After noting Kirk Wolak's committed psql ECHO_HIDDEN improvement and comparing Meson with Autotools, the session resumed a WAL full-page-image compression prototype. Earlier work grouped up to 32 images in one WAL record; Andrey began replacing per-page compression with a contiguous buffer, disabling hole stripping and restructuring record-data links. The code became tangled around rdata ownership and was left uncompilable. They agreed to finish offline, then benchmark batch sizes 1, 4, 16, and 32 for compression, CPU, memory, and record-size trade-offs.

[timecodes · 8]
  1. Kirk's first committed psql patch
  2. Meson versus Autotools build benchmarks
  3. Resuming batched WAL image compression
  4. How WAL record assembly handles page images
  5. Removing per-page compression and hole logic
  6. Allocating contiguous multi-page buffers
  7. Record-data links block the implementation
  8. Batch-size rationale and benchmark plan
#B-tree#WAL#compression@Nikolay Samokhvalov
-hackersCF #5621
stalled

correlated mapping
· 64:33 · youtube ↗

B-tree build and batching WAL records

Batch or compress WAL records during B-tree construction.

The group explored reducing WAL volume during B-tree index creation by batching full-page images, which could later let compressors reuse context across similar pages. After tracing the sorted B-tree build and its page-flush routine, they changed it to postpone writes and emit roughly 31 pages in one WAL record. The build and basic tests worked, but compression itself was not implemented, replication coverage was uncertain, and macOS library and overheating problems prevented the intended size and performance comparison.

[timecodes · 8]
  1. Patch status and PostgreSQL CI infrastructure
  2. Why full-page images limit WAL compression
  3. Reusing compression context across B-tree pages
  4. Measuring current multi-page WAL records
  5. Tracing the sorted B-tree build path
  6. Designing deferred page flushing and batching
  7. Implementing the multi-page WAL batch
  8. Successful 31-page records and missing benchmarks
#B-tree#WAL#compression@Nikolay Samokhvalov
-hackersCF #5621
stalled

correlated mapping
· 64:25 · youtube ↗

Compression improvements

Explore compression paths and possible improvements.

The session began by designing an overflow hash table for pg_stat_statements and pg_stat_kcache so new query IDs would not block behind a busy main table; races and merge semantics made it unsuitable for an immediate patch. Discussion then shifted to compression throughout Postgres. Per-column dictionaries were judged invasive, while reusing compression context during B-tree WAL logging looked tractable. They examined pglz and LZ4 stream constraints and concluded that B-tree builds should first batch pages like GiST; no code was written.

[timecodes · 8]
  1. Revisiting pg_stat_kcache lock contention
  2. An overflow hash table for delayed query IDs
  3. Duplicate entries and merge races
  4. From statistics contention to compression
  5. Column dictionaries and system-wide dictionaries
  6. Reusing context for B-tree full-page images
  7. Why appending to compressed streams is difficult
  8. B-tree WAL batching becomes the prerequisite
#compression@Nikolay Samokhvalov@Kirk Wolak
no public artifact mapped
exploration

correlated mapping