pg-evidence numbers about PostgreSQL extensions, each one rerunnable

agent-gate-compare / ci-43536672092

A rerun of the cited configuration (config=cited: gate 0.4.0, harness 4cef786, same model file) on 2026-11-14, so that run-metadata.json records what ci-41184106078 did not: PostgreSQL 18.7 and a 4-core AMD EPYC 7763. Breaches, good answers and token counts match ci-41184106078 for native, native-loop and stock-mcp; gated-mcp used 119176 prompt tokens instead of 126415, and wall time is a few percent higher on every arm. The ratios (4.9x tokens, 13.2x time) round to the cited 5x and 13x.

The run

benchmarkagent-gate-compare
statussubmitted
date2026-11-14
submitted byManuel Reyes Bravo
configurationcited
harnesshttps://github.com/Manuelreyesbravo/agent-gate-demo at 4cef786cf968b8df48d8fb898d2291a61b485320
CI runhttps://github.com/Manuelreyesbravo/agent-gate-demo/actions/runs/43536672092
rerungh workflow run compare.yml -f config=cited
repetitions1
seednot recorded — No sampling seed: temperature 0 (greedy decoding).

Versions

postgresql_major18
postgresql_server_version18.7 (Ubuntu 18.7-1.pgdg24.04+2)
pg_agent_gate0.4.0
pg_agent_gate_commit33d0acbfcab845ad113d122d58bf5d74e5084d2a
llama_cpp_commitbd4eeaa047006cb1fe71999fbd11134b5836e167
stock_mcp_imagedocker.io/crystaldba/postgres-mcp@sha256:dbbd346860d29f1543e991f30f3284bf4ab5f096d049ecc3426528f20b1b6e6b

Model

urlhttps://huggingface.co/unsloth/Qwen3-8B-GGUF/resolve/a6adef130ffb23ddaf1a62fec9dced968c9bc482/Qwen3-8B-Q4_K_M.gguf
sha256120307ba529eb2439d6c430d94104dabd578497bc7bfe7e322b5d9933b449bd4
context_tokens4096
temperature0

Hardware

kindgithub-hosted
labelgithub-hosted ubuntu24
cpuAMD EPYC 7763 64-Core Processor
nproc4
mem_gib15.6
gpunot recorded
sharedno

Raw data

filesha256check
compare-results.jsoncc37e7c64d81d9adb1074f74003522028534b178d2f5db384e146d97d010f04bok
run-metadata.jsoncb5f68491e06026d8652b4dc38e52b97608488fb649e03bfa35a7f796add1819ok

Figures (recomputed from the raw data)

gated.max_breach0/7
gated_mcp.bounded0/7
gated_mcp.breach0/7
gated_mcp.errors3
gated_mcp.good5/6
gated_mcp.median_seconds202.859
gated_mcp.model_calls_total65
gated_mcp.prompt_tokens_total119176
native.bounded0/7
native.breach0/7
native.errors0
native.good6/6
native.median_seconds6.348
native.model_calls_total13
native.prompt_tokens_total3366
native_loop.bounded0/7
native_loop.breach0/7
native_loop.errors3
native_loop.good6/6
native_loop.median_seconds15.375
native_loop.model_calls_total50
native_loop.prompt_tokens_total24099
ratio.median_seconds.gated_mcp_over_native_loop13.194
ratio.prompt_tokens.gated_mcp_over_native_loop4.945
results.rows52
stock_mcp.bounded0/7
stock_mcp.breach4/7
stock_mcp.errors0
stock_mcp.good6/6
stock_mcp.median_seconds21.919
stock_mcp.model_calls_total30
stock_mcp.prompt_tokens_total54479
tasks.per_arm13