postgres=# select * from sessions where video_id = 'wHMNX-fHb2A';

Observer effect in pg_stat_statements and pg_stat_kcache

· 63:53 · Observability

Missing and self-hidden I/O accounting in monitoring extensions.

session summary

The group reproduced an observer effect shared by pg_stat_statements and pg_stat_kcache. Reading their hash tables holds a shared lightweight lock while results are materialized; if the tuplestore spills beyond work_mem, the lock may cover temporary-file I/O, blocking registration of new queries. They prototyped conditional lock acquisition for pg_stat_kcache so a query can execute while its statistics entry is skipped. The workaround succeeded in the demonstration, but needed configuration, benchmarks, and confirmation that it addressed the production contention before submission.

timecodes

  1. Reviewing the committed bounded psql watch
  2. Observer effects in statistics extensions
  3. pg_stat_kcache contention at high TPS
  4. Tracing the shared hash-table lock
  5. Tuplestore spill extends lock duration
  6. Reproducing blocked registration of new queries
  7. Best-effort tracking with conditional locking
  8. Prototype works, but the production cause remains uncertain