This note accompanies the next increment on top of v49 and picks up where the 07-19 posting left off. Tatsuo's 07-21 review moved 2015–2026 up to 1015–1026 and emptied the 2XXX band, so these eighteen are everything that is still unreviewed: bands 3XXX and 4XXX, plus the 8XXX/9XXX pairs. Ten are unchanged from 07-19; seven are new and one consolidates six. Vertical is apply order, horizontal is subject axis; within an axis, later depends on earlier. The band scheme is the one described in the 07-19 mail and is unchanged.
Since the 07-13 posting the pre-commit work has run on two tracks in parallel, and the band numbers follow that division. It is not an administrative split: the matcher's open questions are about how the standard's preferment rules must be read, while the questions around it are about PostgreSQL's own planner and executor invariants. Those are different bodies of knowledge, and separating them let both move at once.
The matcher and the pattern tree. Preferment and cycle handling in the NFA, the compiled representation the matcher walks, the tree rewrites that feed it, and absorption analysis. Cross-checked against Trino 471 and Perl regex semantics where the standard's reading was in question.
Everything the matcher meets at its edges: navigation-offset resolution in the executor, DEFINE volatility across the planner's preprocessing, junk targetlist entries and their resno allocation, subquery pull-up interaction, grammar diagnostics, and deparse. All eight commits in band 4XXX carry Jian He as author.
Coordination between the two tracks is continuous rather than per-posting. Patches, counter-patches and review notes go back and forth offlist, with day-to-day discussion on Discord; several of the commits below began as a patch Jian sent offlist, were revised in that exchange, and are carried here under his authorship. The Patches carried from Jian table traces each attachment to the commit that now holds it, so the provenance is checkable rather than asserted.
One axis carries no band number. Each track reviews the other's patches, and separately Henson runs QA over the range: auditing the whole span before it is posted, producing the defect list, and re-judging that list against the code.
The tables in this note are that output. The 07-11 triage, 53 items, is
the pre-commit defect list, and the evidence column in the burn-down is what was checked
in the current tree rather than what the document claimed. The 07-24
audit swept the preceding eight commits (+1591/−730) and reported eight
items, one of which was a wrong-results regression the suite did not
catch — mergeConsecutiveVars merging a greedy quantifier with
a following reluctant one and changing the preferred match, staying green because the
committed tests only covered reluctant-first (A+? A). Jian's fix arrived the
next day and is folded into 4004.
Verification is the same strand. The row-by-row comparison against Oracle 23ai and Trino 471, plus the Perl regex cross-check, is what backs the standard reading in 3002·3003·3004; and the rebase carried a clean build, the regression suites and pgindent at every patch. Finding the defects and confirming the fixes actually hold runs across both tracks rather than inside either.
jump served as both branch link and group skip → wrong resultsRPRVarMatch · follows Jian's 07-20 reviewEEOP_JUMP_IF_NULL → wrong resultsparsenodes.h field → must follow 4005hasRPR gate — closes the pull-up / EXISTS paths 4003 left open3XXX and 8/9XXX keep the numbers they were posted under. Only 4XXX is renumbered. The new 4001 absorbs all of old 4001–4004, and five previously unposted commits land in the middle of the band; since the rule is that the number follows the apply order, the band has to be renumbered to stay consecutive.
| # | New | Was | Commit | Note |
|---|---|---|---|---|
| 1–5 | 3001–3005 | same | 90a2bd0 … e0c2656 | As posted 07-19 |
| 6 | 3006 | — | 6055522 | New |
| 7 | 4001 | 4001–4004 | 5fa4621 | Absorbs four, plus three unposted |
| 8 | 4002 | 4005 | 93e3337 | Number moved |
| 9 | 4003 | — | 6e67451 | New |
| 10 | 4004 | — | d73897f | New |
| 11 | 4005 | — | 532f237 | New |
| 12 | 4006 | — | 9874c80 | New |
| 13 | 4007 | — | e5f1e08 | New |
| 14 | 4008 | — | 929e5d5 | New |
| 15–18 | 8001 8003 9001 9003 | same | a86915e … a8771d3 | As posted 07-13 (8002/9002 dropped) |
Old 4006 (Introduce execution struct RprNavState) no longer exists as a
separate commit; it is part of the new 4001. In short, old 4001–4006
folded into new 4001–4002 — the cover mail will say so in one line
so the numbering change is traceable from the 07-19 posting.
4005 is the widest patch in the band and its diff does not read in the order its commit message does. This is a map of it.
1 junk TE resno bypasses p_next_resno -> assertion defect 2 dedup compares only varno/varattno -> lateral failure defect 3 define_walker check order -> nav depth misdiagnosed diagnostic 4 gram.y quantifier message omits the token diagnostic 5 four stale comments refreshed comment 6 nav markpos initialized to -1 per the convention no-op 7 rpr_integration tests given numbered sections test parse_rpr.c +29 strands 1 and 2 -- the defects gram.y, ruleutils.c ~40 strands 3 to 5 rpr_integration.sql +168 strand 7 rpr_integration.out +190 strand 7
The line counts are why the map is worth having: strand 7 is 358 of the 467 added
lines, so a reviewer starting at the top of the diff meets a test reorganization
before reaching the 29 lines that fix anything. Strands 1 and 2 first, then 3–5,
then 7, is the order that matches the risk. Strand 6 changes no behaviour —
begin_partition() overwrites the value — and is there so the field
follows the same -1 convention as the other window objects.
Attachments received offlist since 07-11, and where each one lives now. The evidence column is what was checked in the tree rather than what the commit message claims.
| Attachment | Date | Carried by | Evidence |
|---|---|---|---|
| v51-0001-Fix-the-regression-failure.rpr | 07-11 | 4001 | Part of the old 4001–4004 merge |
| v51-0002-Stop-evaluating-navigation-arguments-… | 07-11 | 4002 | Carried as its own commit |
| v51-0003-Introduce-execution-struct-RprNavState.rpr | public | 4001 | RPRNavState 11 sites / old name 0 |
| v1-0001-Remove-validate_rpr_define_volatility…wip | 07-20 | 4003 | validate_rpr_define_volatility 0 sites |
| v1-0001-mergeConsecutiveVars1-is-equivalent-…wip | 07-21 | 4004 | mergeConsecutiveVars1 0 sites |
| v1-0001-further-enhance-RPR-DEFINE-volatility-…patch | 07-21 | 4008 | hasRPR 7 sites |
| v1-0001-code-simplification.20260723.patch | 07-23 | 4006 | initial field gone |
| v51-0001-Fix-wrong-query-results-…mergeConsecutiveVa.patch | 07-24 | 4004 | Guard plus three regression tests |
| remove_ensure_reduced_frame.diff | 07-20 | declined | Declined in the 07-13 reply. The RF_NOT_DETERMINED guard is load-bearing, and where to put it went to the list as two options; Tatsuo picked the helper, which is what the tree has. Raised again 07-20; the answer stands. |
The 07-24 patch is worth singling out. It guards mergeConsecutiveVars
against merging a greedy quantifier with a following reluctant one over the same
variable, which the pair makes observable under leftmost-choice-first. Rather than
applying it at the tip, it was folded into 4004 — the commit that gives
mergeConsecutiveVars its current shape — so the series does not
introduce a wrong-results path and then repair it a few patches later. Two small
departures from the patch as sent: a now-unreachable assignment and its comment were
removed along with it, and the guard carries a comment naming the standard clause.
The burn-down above is what this increment closed. This table is what it
introduced. It draws on two audits: the 07-24 range audit (nine items) and a
07-26 audit of the whole 4XXX band against the last commit of 3XXX
(RPR-henson, seven items). Each is mapped to the patch that introduced it
under the current numbering and re-judged against the tree as it stands.
The evidence column is that check.
Codes follow one form, MM-DD #n. The 07-24 audit's original
A1–B4 were renumbered to 07-24 #1–#8;
the mapping is in that report's header. C10 J2 keeps its own prefix because
it is a worklist item, not an audit finding. The date is there because the triage table
below uses the separate label space of the 07-11 document — its A1 is
the nfa_try_absorb_context absorption problem, a different item from
07-24 #1 here.
| Audit ref | From | Item | Kind | Now | Evidence |
|---|---|---|---|---|---|
| 07-24 #1 | 4004 | mergeConsecutiveVars merges a greedy quantifier with a following reluctant one, changing the preferred match | wrong results | closed | guard + 3 tests in the same patch |
| 07-24 #6 | — | a planner/executor redesign landed under an “Add regression tests” title | commit hygiene | closed | commit gone in the rewrite; content in 4001 |
| 07-24 #3 | 4003 | volatile-DEFINE errors lost their cursor position (the cost of moving the check) | diagnostics | accepted | rpr_define_errposition 0 sitesagreed 07-24 |
| 07-24 #2 | 4004 | tautological INF guard — a finite max-sum equal to INT32_MAX is written into the unbounded sentinel | misclassification | open | rpr.c:312 newmax ≤ INF, INF = PG_INT32_MAXno parser bound |
| 07-24 #4 | 4001 | EXPLAIN of a never-executable navigation shows an underflowed lookahead | display only | open | nodeWindowAgg.c:4270 reach = inner - outer with a “both ≥ 0” comment |
| 07-24 #5 | 4001 | rpr_srf_f(int) defined with the same signature in two concurrently scheduled suites | flaky test | open | rpr_base 4544–4560 / rpr_integration 1067–1122 overlapparallel_schedule:110 same group |
| 07-24 #7 | 4001 | the compound outer-offset tests do not test the fix they document | coverage gap | open | added queries use Const NULL::int8 only — 0 parameterized outer-offset cases |
| 07-24 #8 | 4001 | an EXPLAIN filter bakes plan indentation into the expected output | latent flaky | open | rpr_explain.out:6507 carries 15 leading spaces |
| C10 J2 | 4001 | unused #include "common/int.h" in createplan.c — 4001 removed its last use | hygiene | open | include 1 / pg_*_overflow uses 0 |
| 07-26 #1 | 4001 | BASE's two negative-offset clamps deleted with no replacement → both subtractions in resolve_one_nav overflow int64 (C undefined behaviour) | UB | open | reachable from a plain EXPLAINsibling arms use pg_add_s64_overflow; only the subtractions at :4252 and :4270 are unguardedroot cause of 07-24 #4 |
| 07-26 #2 | 4008 | the hasRPR gate degrades plans for valid non-volatile queries — a hash semi join becomes a correlated SubPlan | performance | open | a Query-wide flag: it looks neither at whether the window is discarded nor at whether the DEFINE is volatile “only plan shape” is true of results alone |
| 07-26 #3 | 4001 | offset validation now aborts at scan start even for an empty partition; BASE checked per row | behaviour | open | BASE (0 rows) → HEAD ERRORnot in the commit message, not pinned by a test the direction matches calculate_frame_offsets |
| 07-26 #4 | 4002 | skipping the argument also suppresses run-time errors it would have raised — whether a query errors becomes data-dependent | behaviour | open | all seven added queries wrap the navigation in IS NULL and cannot observe itthe only message in the band with no design rationale |
| 07-26 #5 | 4008 | “The DEFINE then always survives into a planned subquery” does not hold | diagnostics | open | subqueries the planner discards before set_subquery_pathlist() are never checkedidentical text accepted or rejected on an unrelated qual not a regression against BASE |
| 07-26 #6 | 4005 | only one of the two error sites in the gram.y Op Op rule was brought into line | diagnostics | open | the survivor prints a token with the alternation bar glued on, and an unconditional ?? hint |
| 07-26 #7 | series | CATALOG_VERSION_NO not bumped — 4006 removes a node field and 4008 adds one | — | false positive | an in-development series does not bump it the committer does; not a defect |
No correctness defect remains. The 07-26 audit found no path across the eight commits by which a query result differs from BASE, and the one wrong-results item, 07-24 #1, is closed in the patch that introduced it. What remains splits three ways.
#1 is the one that weighs differently. BASE's two negative-offset
clamps were removed with nothing put in their place, so the subtractions in
resolve_one_nav reach C undefined behaviour, and a plain
EXPLAIN is enough to get there. A build with -ftrapv
or UBSan aborts. 07-24 #4 only flagged the comment at :4270; the invariant
that comment rests on was exactly the deleted clamp, so 07-24 #4 is the symptom and #1
the cause. Restoring the clamp on the non-validating path makes the comment true again.
It belongs in 4001, which alone reaches the overflow, and since the posted branch is
fixed it goes in with the next revision.
#3, #4 and #5 are record-keeping rather than code. All three changes are in the right direction, and none of them appears in a commit message or is pinned by a test. Under claim-based review a change with no claim gets no review, and 4002 is the illustration: the only message in the band with no design rationale, and all seven queries it adds are blind to the change it makes.
07-24 #7 is the different one. The others are known defects still standing; 07-24 #7 says the regression protection is empty. If the runtime outer-check ordering regresses again, every committed test still passes. Since 4001 rewrote that area wholesale, now is the moment to add the differentiating test.
07-24 #5 was set aside as not urgent, but the failure is timing-dependent:
when the two definitions are live at once the second CREATE fails with
function "rpr_srf_f" already exists, and when they are not it passes
quietly. Renaming one of them settles it.
An offlist triage list circulated on 07-11 catalogued candidate defects across the whole patch set, at the level of one entry per distinct failure. This increment closes 30 of 53 — A1 is closed too, but by the patch that is now the review line (1026, was 2026, reviewed 07-21), so it is outside this increment. The 11 that remain open cluster almost entirely in the planner and deparse axes, which this increment does not touch.
Denominator: the list has 60 entries; the 7 that a recheck left refuted are excluded, and the 2 that were promoted to confirmed defects are included. The reference codes below are that document's internal labels, kept so the two can be read side by side.
Every row below was re-judged against the current tree rather than taken from the document's own check marks; the evidence column records the symbol or count that settled it.
| Axis | Ref | Title | Status | Evidence |
|---|---|---|---|---|
| NFA matcher | N1 | visited guard kills legitimate loop-backs — (A+|B){2} yields no match on all-A input | 3004 | nfaVisitedEnds 11 |
| N2 | skip path does not increment the END count, so a group-final optional that is absent loses the whole match | 3004 | nfa_exit_to() | |
| N3 | nfa_advance_alt does not prune branches after FIN → alternation preferment violated, SKIP contaminated | 3004 | matchGen compare | |
| N4 | ((A?){2}?) matches one row — EMPTY_LOOP fast-forward order inverts body preferment | 3004 | RPR_ELEM_EMPTY_PREFERRED 12 | |
| N5 | greedy * empty-iteration preferment — the preferred branch's stop-here completion is dropped by cycle detection | 3004 | nfa_exit_to 10 | |
| N6 | empty-match statistics misclassified — cleanup counts non-target empty matches as failures | 3004 | classification 8 | |
| MR07 | DEFINE predicates evaluated per row even for variables the NFA never tests → match-irrelevant runtime errors | 3006 | RPRVarMatch 15 | |
| Absorption | A1 | nfa_try_absorb_context does not check matchedState — a recorded match is destroyed with the context | 1026 | 760311b = the review lineoutside this increment |
| A2 | SKIP TO NEXT ROW with long matches is O(n2) — completed but unqueried contexts linger (values correct) | not a defect | inherent to per-row match reporting residual comment closed by 4005 | |
| Pattern optimization | P1 | mergeConsecutiveAlts rollup of GROUP{n,n}(ALT) kills all matches on quantified branches | 3003 | Fixed at the matcher root |
| P2 | consecutive GROUP merge violates preferment with a reluctant body or unequal-length alternatives | 3003 | rprBodyRowCount 7 | |
| P3 | SUFFIX merge diverges when the group body has ALT or min<max (PREFIX is safe) | 3003 | rprNodeRowCount 5 | |
| P4 | tryMultiplyQuantifiers changes the preferred match for child.min≥2 with finite max | 3003 | same | |
| Planner / DEFINE | C1 | volatile default argument injected into DEFINE after the check → a different match on every execution, with no error | 4003 | check after preprocessing |
| C2 | flatten_join_alias_vars plants a volatile implicit cast into a JOIN USING merged column | 4003 | closed with C1 | |
| C3 | volatile DEFAULT in a nav offset — init-time frozen trim vs per-row re-evaluation | 4001 4003 | closed twice over | |
| MR09 | post-fold recheck reports a misleading errposition on the SQL-function inlining path | 4003 | rpr_define_errposition 0 | |
| C4 | junk TLE dedup ignores varlevelsup — a LATERAL outer-ref collision makes a valid query fail | 4005 | equal(tle->expr, var) | |
| C5 | function RTE inlining injects PARAM_EXEC into a nav offset; executor init freezes the unset param | 4001 | resolve_nav_offsets | |
| C6 | EXEC_FLAG_EXPLAIN_GENERIC ignored → EXPLAIN (GENERIC_PLAN) fails on a parameterized offset | 4001 | RPR_NAV_OFFSET_NEEDS_EVAL 9 | |
| DEFINE eval / nav | E1 | out-of-range nav evaluates its argument over an all-NULL row instead of returning an operation-level NULL | 4002 | EEOP_JUMP_IF_NULL 5 |
| E2 | compound outer-offset validation skipped when the inner FIRST/LAST position is out of range | 4001 | validation unconditional | |
| E3 | three NEXT_FIRST comments contradict the load-bearing code | 4001 | comments corrected | |
| Parse / targetlist | X1 | multi-window resno collision crash — junk TLE bypasses p_next_resno → assertion | 4005 | p_next_resno++ |
| Optimizer / window | W1 | finalize_plan does not finalize WindowAgg.defineClause → DEFINE-only PARAM_EXEC missing from extParam | 4001 | finalize_primnode |
| W9 | pull_var_clause(defineClause, 0) rests on three undocumented guards | 4005 | documented at the call site | |
| W2 | defineClause sits outside the grouping substitution pipeline → GROUPING SETS/ROLLUP/CUBE with RPR | open | blocked on Z3 | |
| W3 | GROUP BY x+y with DEFINE x+y wrongly rejected | open | — | |
| W4 | unreferenced RPR windows with volatile DEFINE bypass the check via pull-up / EXISTS | partial 2/3 | hasRPR gate in 4008 | |
| W5 | blanket run-condition block forces rn≤N to match the entire input | open | blocked on Z4 | |
| W6 | remove_unused_subquery_outputs WindowFunc retention: comment contradicts code | open | — | |
| W7 | make_windowagg aliases defineClause without copying → shared list cells overwritten | open | make_windowagg 3 | |
| W8 | DEFINE Var retention re-runs pull_var_clause per column — O(cols × DEFINE) | open | — | |
| Rewrite / deparse | D1 | multi-RTE views deparse DEFINE Vars alias-qualified → pg_get_viewdef output is not reparseable | open | inRPRDefine qualifies function names, not Vars |
| D2 | window names pattern/after/initial/seek deparsed unquoted → viewdef reparse failure | open | tied to CF 7056 | |
| D3 | get_rule_windowspec leaves a trailing space after each RPR clause (cosmetic) | open | — | |
| Grammar / parser | G3 | define_walker misreports sibling navs as nested more than two levels deep | 4005 | check order swapped |
| G4 | op-op quantifier error omits the token name and points the cursor at the preceding token | 4005 | gram.y message | |
| G1 | PATTERN (PERMUTE(A)) silently accepted with non-standard semantics | open | PERMUTE 0 in parser | |
| G2 | A*|B*|C builds a nested ALT but its deparse reparses flat — semantics preserved, same-tree contract not | not a defect | deparse text identical 2nd generation is a fixed point dump-safe | |
| Serialization | R3 | _copy/_out/_read equality check under #ifdef NOT_USED | not a defect | NOT_USED 0 |
| R1 | DEFINE variable names live only in a jumble-ignored resname → different DEFINEs share a queryId | open | — | |
| R2 | four contract deviations in the hand-written RPRPattern serialization | partial 3/4 | no_query_jumble added | |
| Performance | PF1 | a single shared read pointer oscillates between matchStart and the frontier → quadratic physical re-reads once spilled | out of scope | correctness unaffected |
| Quality / docs | Q1 | README.rpr dominance argument and the allStatesAbsorbable comments | 4005 | 4 sites corrected |
| Q2 | double-check intent comment in planner.c, placeholder contract in nodeWindowAgg.c | 4005 | both resolved | |
| Q3 | prepare_tuplestore initializes markpos to 0 rather than the -1 convention | 4005 | markpos = -1 | |
| Q4 | executor init writes offsets into the RPRNavExpr of a cached plan tree | 4001 | rprNavOffsets 7 | |
| Decisions | Z1 | dead-branch volatile (CASE WHEN false THEN random()): pre+post double check vs post-fold only | decided | post-fold only |
| Z2 | whether any stock type pair lets select_common_type reach the JOIN USING cast case | not a defect | — | |
| Z3 | phnullingrels (VALUES) variant — design input for W2 | open | precedes W2 | |
| Z4 | safety of lifting the run-condition block — blanket removal is not result-equivalent for count(*) | open | gates W5 | |
| Z5 | WindowAgg.runCondition likewise not finalized by SS_finalize_plan | not a defect | structurally masked |
Six in optimizer / window integration are the larger group. W2 and
W3 both come from defineClause sitting outside the grouping pipeline; W7
is make_windowagg aliasing the clause without copying, so
set_upper_references can overwrite shared list cells. W4 is a partial fix
— two of three paths are closed, and the third is the constant-false pruning path
still under discussion.
Three in deparse, of which D1 matters most: in a multi-RTE view a
DEFINE Var is deparsed alias-qualified, so pg_get_viewdef output does not
reparse and pg_dump / restore breaks. D2 depends on which shape CF 7056
takes upstream.
W2 GROUPING SETS/ROLLUP/CUBE + RPR D1 multi-RTE view deparse unreparseable
W3 GROUP BY x+y + DEFINE x+y D2 window name unquoted (CF 7056)
W5 run-condition block (needs Z4) D3 deparse trailing space (cosmetic)
W6 remove_unused_subquery_outputs G1 PERMUTE(A) accepted
W7 defineClause aliased without copy R1 DEFINE names absent from queryId
W8 pull_var_clause O(cols x DEFINE)
partial W4 volatile DEFINE bypass 2/3 R2 RPRPattern serialization 3/4
PF1 quadratic re-read once spilled (out of scope)
decision Z3 phnullingrels variant (gates W2) Z4 run-condition lift (gates W5)
That FIRST blocks context absorption is the correct rule, not a defect.
Absorption presupposes that future behaviour is independent of where the match started,
and FIRST looks back to match_start by definition, so the premise fails.
A context with navFirstOffset set belongs out of the absorption candidates.
PF1 concerns the layer below that. Whether or not absorption happens, one
tuplestore read pointer is shared by two positions that alternate:
ExecRPRNavGetSlot() winobj = winstate->nav_winobj <- near matchStart rpr_prepare_row() the same winobj <- the frontier (pos) window_gettupleslot repositions by relative tuplestore_skiptuples -> once spilled, two traversals of (pos - matchStart) tuples per row -> Theta(length^2) physical re-reads per match
The measurements separate the two explanations. PREV also blocks
absorption, yet costs 1.7 ms spilled where FIRST costs
5358 ms (n=4000). If blocked absorption were the cause the two would be
comparable. The only difference is whether the read pointer travels, and
EXPLAIN (ANALYZE, BUFFERS) shows temp read=2,154,477 (n=2000)
for those physical re-reads. Results are correct throughout (md5 identical).
A fix would not touch absorption either: splitting the read pointer into a
frontier pointer (advancing one row at a time) and a nav pointer (parked near
matchStart) makes each amortized O(1). The markptr floor
(matchStartRow + navFirstOffset) is unchanged, so neither the retained
window nor the absorption decision moves. It is out of scope here because correctness
is unaffected; it is recorded for after the commit.
CF 7056 — Fix quotation logic for unreserved keywords in
window specifications. When a window name is an unreserved keyword,
get_rule_windowspec() does not quote it, so pg_get_viewdef
output fails to reparse and pg_dump / restore breaks. It reproduces on
stock releases, predates RPR by many years, and is fixed in the same place RPR would
fix D2.
RPR's D2 (pattern / after /
initial / seek deparsed unquoted) is that same defect
extended to the RPR keywords. Depending on the form upstream takes, D2 either
closes automatically or needs a small addition on top:
07-10 Kwangwon Seo posts v1 (quotes the refname)
07-24 public review reply -- prefers the explicit-list variant
-> a view from Tatsuo on the direction would unblock this
-> and Kwangwon's answer on covering the RPR keywords
D1 is adjacent: same function family, but a Var-qualification problem.
D3 is a trailing space in the same function -- can ride along either way.
This is why the three deparse items are hard to start independently: D2 depends on the upstream outcome, and D1 and D3 touch the same function, so the order has to be agreed to avoid conflicts.
A defect list does not show what was not attempted. The items below are intentionally outside this series — unimplemented features, optimizations held for after the commit, and resource questions that do not affect correctness. Stating the boundary avoids their being read as omissions.
| Ref | Item | Track | Status |
|---|---|---|---|
| L01 | SEEK clause support (SQL:2016) | Feature | Semantics analysed · not implemented |
| L02 | PATTERN () empty pattern | Feature | Deferred by agreement · on request |
| L03 | PREFIX pattern absorption | Optimization | Designed · separate follow-up series |
| L04 | Relaxing DEFINE subquery over-rejection | Feature | Low priority — the over-restriction is intentional and noted in code |
| L05 | Lazy / tri-state DEFINE evaluation (short-circuit) | Optimization + correctness | Done in 3006 (with MR07) |
| L06 | Precise WindowAgg removal | Optimization | Conservative guard is in; the precise version follows |
| L07 | RPRContext consolidation | Structural | Deferred by agreement (2026-02-17) — largest, last |
| L08 | CLASSIFIER / row pattern variable storage | Structural | Design note posted 07-04 · open for comment |
| L10 | Unbounded quantifier counters do not saturate at min → frontier growth | NFA resource bounds | Out of scope — answers are correct; this is resource use |
| L11 | Shared read pointer oscillation → quadratic physical re-reads when spilled (PF1) | Performance | Out of scope — correctness unaffected; after the commit |
The suggestion on 07-23 was to lower RPR_DEPTH_MAX from 254 to 10. The
reply argued that depth is the wrong variable: depth enters
nfaStateSize only linearly, whereas the resource that actually grows is
the number of live NFA states, driven by alternation, quantifier ranges and their
combinations. A limit of 10 forbids a harmless depth of 253 while still admitting a
{3,7}? combination at depth 5.
L10 is the concrete case behind that argument. With an unbounded {m,} the
counter never saturates at min, so states that are semantically identical keep
splitting on the counter value alone and the frontier grows without bound.
The answers stay correct — this is resource bounding, not
correctness, which is why it sits outside this series. It is also the first consumer of
the work_mem-style bound on peak NFA states that the reply proposed.
L07 (RPRContext consolidation) is the structural groundwork for
accepting MATCH_RECOGNIZE syntax; it was deferred by agreement with Tatsuo
on 2026-02-17 and is last because it is the largest.
L08 (CLASSIFIER / row pattern variable storage) was posted as a design
note on 07-04, leaving accumulator lifetime and winning-match replay as open engine-side
questions; comment on those two would be welcome. Both are post-commit work, but naming
them here explains some of the present design choices — for instance why the nav
offsets are explicit arguments rather than reached through WindowClause.
RPR-base..RPR.
The cross-engine run is worth a word since it bears on 3002, 3003 and 3004. Every difference found was either a feature gap (11 cases where Oracle implements something PostgreSQL rejects) or a place where PostgreSQL is deliberately stricter (3 cases). No query returned a different answer. If the preferment reading in those three patches is questioned, the Trino and Perl cross-checks are the supporting evidence; the comparison was posted to the list on 07-16.
3002, 3003 and 3004 are one standard clause failing at three layers — the tree rewrite changes the preferred match (3003), the compiled representation conflates a branch link with a group skip (3002), and the matcher breaks the rule directly (3004). Split across postings, the same clause has to be re-read three times. 3001 is defensive code unrelated to preferment, so it needs its own justification for landing now; 3005 is cleanup of the reviewed base and is different in kind.
4001 does not divide. Collecting offset resolution into one place in
the executor, keeping the result in executor state rather than the shared plan tree, and
distinguishing constant / runtime / overflow through RPRNavOffsetKind are
one change. Split apart, the intermediate states either give wrong answers or write into
a shared plan tree.
4004 both introduces and guards. The commit that gives
mergeConsecutiveVars its current shape is the one that would introduce the
wrong-results path, so the guard belongs there rather than in a later fix-up.
Only the unrelated-code axis is fully independent. 8001/9001 and 8003/9003 share neither files nor subject with the rest and can be taken separately or first.
Small items still to fold in, listed so they are not mistaken for oversights:
createplan.c keeps an #include "common/int.h" whose last use
4001 removed, and it belongs in that commit; and the removal of
nav_null_slot is the one item from the v51 review not yet carried.
Separately, Jian's 06-24 message went unanswered. It raised four
cleanups in ParseRPRNavCall — the decoration comment block,
coerce_to_target_type over coerce_to_common_type, dropping the
offtype test, and the extra parentheses around ereport()
— and came in from his other address, which is how it was missed. It went to the
list, not offlist. parse_func.c has not changed since, so all four still
stand as written. They will be reviewed and answered.
One question would help most if answered before the next posting. Tatsuo — a view on the CF 7056 direction would unblock the whole deparse axis, since D1 and D3 touch the same function and cannot be sequenced until D2 is settled.
The verdicts above were checked by building both trees. BASE is
RPR-henson, the last commit of band 3XXX; HEAD is the
branch tip. Each item ran in its own fresh database on both sides, and what
follows is the reproduction query and the before/after output difference
verbatim. Every query creates its own tables, so it runs as pasted.
Each case carries a control whose answer is fixed whether or not the
defect is present — without one, a test proves nothing. X1 is an
assertion failure and needs asserts enabled to fire.
The status on every item here — and in the two tables above — is a working classification, not a settled one. What the measurement fixes is whether BASE and HEAD differ; how that difference is judged can move as each item is worked through in detail, and items may cross between closed, open, partial and not-a-defect.
A plain EXPLAIN reaches it. The two subtractions overflow in opposite directions.
EXPLAIN (COSTS OFF) SELECT count(*) OVER w FROM generate_series(1,10) s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+)
DEFINE A AS v > NEXT(LAST(v, 1), 1)); -- control
EXPLAIN (COSTS OFF) SELECT count(*) OVER w FROM generate_series(1,10) s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+)
DEFINE A AS v > NEXT(LAST(v, (-9223372036854775807-1)::int8), 1));
-- PREV_FIRST arm, same shape
DEFINE A AS v > PREV(FIRST(v, 9223372036854775807), 0); -- control
DEFINE A AS v > PREV(FIRST(v, 9223372036854775807), -1);-- BASE control Nav Mark Lookback: 0 case Nav Mark Lookback: 0 control Nav Mark Lookahead: infinite case Nav Mark Lookahead: infinite -- HEAD control Nav Mark Lookback: 0 case Nav Mark Lookback: 9223372036854775807 control Nav Mark Lookahead: infinite case Nav Mark Lookahead: -9223372036854775808
BASE checks per row, so with no rows there is no error.
CREATE TEMP TABLE emp0(v int); -- empty SELECT count(*) OVER w FROM emp0 WINDOW w AS (ORDER BY v ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS v > PREV(v, 1)); -- control SELECT count(*) OVER w FROM emp0 WINDOW w AS (ORDER BY v ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS v > PREV(v, -1)); SELECT count(*) OVER w FROM emp0 WINDOW w AS (ORDER BY v ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS v > PREV(v, NULL::int8));
-- BASE (0 rows) (0 rows) (0 rows) -- HEAD (0 rows) ERROR: row pattern navigation offset must not be negative ERROR: row pattern navigation offset must not be null
With no target row the argument is never evaluated, so its error disappears. The LAST control has a target row and is identical on both.
CREATE TEMP TABLE r4(id int, val int); INSERT INTO r4 VALUES (1,10); SELECT id, count(*) OVER w FROM r4 WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A) DEFINE A AS PREV(1/coalesce(val*0,0)) IS NULL); SELECT id, count(*) OVER w FROM r4 WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A) DEFINE A AS LAST(1/coalesce(val*0,0)) IS NULL);
-- BASE ERROR: division by zero ERROR: division by zero -- control -- HEAD id | count ----+------- 1 | 1 ERROR: division by zero -- control
Without the WINDOW clause both sides give a hash semi join. The gate is a Query-wide flag, blind to whether the window is discarded or the DEFINE volatile.
CREATE TEMP TABLE ta(k int); CREATE TEMP TABLE tb(x int, k int);
INSERT INTO ta SELECT i FROM generate_series(1,100) i;
INSERT INTO tb SELECT i, i FROM generate_series(1,100) i;
ANALYZE ta; ANALYZE tb;
EXPLAIN (COSTS OFF) SELECT * FROM ta WHERE EXISTS (
SELECT 1 FROM tb WHERE tb.k = ta.k); -- control
EXPLAIN (COSTS OFF) SELECT * FROM ta WHERE EXISTS (
SELECT 1 FROM tb WHERE tb.k = ta.k
WINDOW w AS (ORDER BY k ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS x > 0));-- BASE
control Hash Semi Join
case Hash Semi Join
-- HEAD
control Hash Semi Join
case Seq Scan on ta
Filter: EXISTS(SubPlan exists_1)
SubPlan exists_1
-> Seq Scan on tbBASE does not reject this shape at all. 4003 and 4008 built a check that did not exist; one path, where the planner discards the subquery, still slips through.
CREATE TEMP TABLE nt(id int, val int); INSERT INTO nt SELECT i, i FROM generate_series(1,10) i; SELECT id FROM (SELECT id FROM nt WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS random() > 0.5)) s; SELECT id FROM (SELECT id FROM nt WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS random() > 0.5)) s WHERE false;
-- BASE (10 rows) -- no rejection at all (0 rows) -- HEAD ERROR: volatile functions are not allowed in DEFINE clause (0 rows) -- slips past the check
The cost of moving the check past preprocessing. Accepted as a trade-off on 07-24.
CREATE TEMP TABLE v3(id int, val int); INSERT INTO v3 VALUES (1,1),(2,2); SELECT id, count(*) OVER w FROM v3 WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS random() > 0.5);
-- BASE
ERROR: volatile functions are not allowed in DEFINE clause
LINE 3: PATTERN (A+) DEFINE A AS random() > 0.5);
^
-- HEAD
ERROR: volatile functions are not allowed in DEFINE clauseThe second case is identical on both — the surviving site calls "@*|" a token, alternation bar and all, and suggests "??" for a pattern containing no question mark.
CREATE TEMP TABLE g6(v int); SELECT count(*) OVER w FROM g6 WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A? ??) DEFINE A AS TRUE); SELECT count(*) OVER w FROM g6 WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A@ *|B) DEFINE A AS TRUE, B AS TRUE);
-- BASE ERROR: invalid quantifier combination HINT: Did you mean "??" for reluctant quantifier? ERROR: invalid quantifier combination: "@*|" HINT: Did you mean "??" for reluctant quantifier? -- HEAD ERROR: invalid token "??" after "?" quantifier HINT: Did you mean "??" for reluctant quantifier? ERROR: invalid quantifier combination: "@*|" -- unchanged HINT: Did you mean "??" for reluctant quantifier?
RPR_QUANTITY_INF is not a number but the sentinel meaning unbounded. Writing a finite bound into that slot turns the pattern unbounded, so representing finite as infinite is itself the defect. BASE declines the merge when the sum would equal the sentinel — giving up the optimization is the fallback, and it is the right call. In HEAD the guard is tautological, the merge goes through, the bound lands exactly on the sentinel, and the pattern is recorded as a{2,}. Even where no row count can tell them apart, the optimizer reads the element as unbounded and its absorption analysis follows. The sibling mergeConsecutiveGroups still carries that guard.
CREATE TEMP TABLE q(v int); INSERT INTO q SELECT generate_series(1,5);
EXPLAIN (COSTS OFF) SELECT count(*) OVER w FROM q WINDOW w AS (ORDER BY v
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A{1,1073741823} A{1,1073741823}) DEFINE A AS TRUE); -- control
EXPLAIN (COSTS OFF) SELECT count(*) OVER w FROM q WINDOW w AS (ORDER BY v
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A{1,1073741823} A{1,1073741824}) DEFINE A AS TRUE);-- BASE
control Pattern: a{2,2147483646} -- merged, bound below the sentinel
case Pattern: a{1,1073741823} a{1,1073741824} -- merge declined: the fallback
-- HEAD
control Pattern: a{2,2147483646}
case Pattern: a{2,} -- merged; finite bound became unboundedThis is the wrong-results regression the 07-24 audit found. It reproduced on the branch as it then stood — that build returned 1,1,1,1,1 for A+ A?? — and Jian sent a guard patch the next day. Because the guard was folded into 4004, the patch that introduces the code, no commit in the current series exposes it. That is why BASE and HEAD both match the control.
CREATE TEMP TABLE g(id int); INSERT INTO g SELECT generate_series(1,5); SELECT id, count(*) OVER w AS cnt FROM g WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A+ A??) DEFINE A AS TRUE) ORDER BY id; SELECT id, count(*) OVER w AS cnt FROM g WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A+ B??) DEFINE A AS TRUE, B AS TRUE) ORDER BY id; -- control
-- BASE case 5,0,0,0,0 control 5,0,0,0,0 -- matches -- HEAD case 5,0,0,0,0 control 5,0,0,0,0 -- matches pre-rewrite build: case 1,1,1,1,1 control 5,0,0,0,0 -- the defect as reported
Not a defect. An in-development patch series does not bump this constant: doing so conflicts with every other patch in flight, so the committer handles it once at commit time. The audit called this one wrong.
grep -o '#define CATALOG_VERSION_NO.*' src/include/catalog/catversion.h
-- BASE #define CATALOG_VERSION_NO 202607091 -- HEAD #define CATALOG_VERSION_NO 202607091 -- unchanged
4001 removed the last two uses in createplan.c, leaving the include behind.
grep -c 'common/int\.h' src/backend/optimizer/plan/createplan.c grep -cE 'pg_(add|sub|mul)_[su][0-9]+_overflow' src/backend/optimizer/plan/createplan.c
-- BASE include 1 uses 2 -- HEAD include 1 uses 0 -- the include is now dead
CREATE TEMP TABLE z_n1_t (id int, v text);
INSERT INTO z_n1_t VALUES (1,'A'),(2,'A'),(3,'A');
-- repro 1 (ALT branch head): expect 3,0,0 -- buggy engine returns 0,0,0
SELECT id, count(*) OVER w AS cnt FROM z_n1_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((A+ | B){2}) DEFINE A AS v = 'A', B AS v = 'B');
-- repro 2 (optional VAR skipState, no ALT): expect 2,0 -- buggy: 0,0
SELECT id, count(*) OVER w AS cnt FROM z_n1_t WHERE id <= 2
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((B? A+){2}) DEFINE A AS v = 'A', B AS v = 'B');
-- repro 3 (branch order independent): expect 3,0,0 -- buggy: 0,0,0
SELECT id, count(*) OVER w AS cnt FROM z_n1_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((B | A+){2}) DEFINE A AS v = 'A', B AS v = 'B');
-- repro 4 (hand-unrolled; optimizer re-rolls to {2}): expect 3,0,0 -- buggy: 0,0,0
SELECT id, count(*) OVER w AS cnt FROM z_n1_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((A+ | B) (A+ | B)) DEFINE A AS v = 'A', B AS v = 'B');
DROP TABLE z_n1_t;-- BASE and HEAD identical; tail of the shared output 1 | 3 2 | 0 3 | 0 (3 rows) id | cnt ----+----- 1 | 3 2 | 0 3 | 0 (3 rows)
CREATE TEMP TABLE z_n2_t (id int, flags text[]);
INSERT INTO z_n2_t VALUES (1, ARRAY['B']), (2, ARRAY['B']);
-- Site 1 (nfa_route_to_elem optional VAR skip -> END): expected row1 cnt=2, buggy all 0
SELECT id, count(*) OVER w AS cnt
FROM z_n2_t
WINDOW w AS (
ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN ((B A?){2})
DEFINE A AS 'A' = ANY(flags), B AS 'B' = ANY(flags)
);
-- Site 2 (nfa_advance_begin min=0 group skip -> END): expected row1 cnt=2, buggy all 0
SELECT id, count(*) OVER w AS cnt
FROM z_n2_t
WINDOW w AS (
ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN ((B (A C)?){2})
DEFINE A AS 'A' = ANY(flags), B AS 'B' = ANY(flags), C AS 'C' = ANY(flags)
);
-- Title form: (UP DOWN?)+ with DOWN absent: expected row2 cnt=2, buggy all 0
CREATE TEMP TABLE z_n2_stock (id int, price int);
INSERT INTO z_n2_stock VALUES (1,100),(2,110),(3,120);
SELECT id, price, count(*) OVER w AS cnt
FROM z_n2_stock
WINDOW w AS (
ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
INITIAL
PATTERN ((UP DOWN?)+)
DEFINE UP AS price > PREV(price), DOWN AS price < PREV(price)
);-- BASE and HEAD identical; tail of the shared output ----+----- 1 | 2 2 | 0 (2 rows) id | price | cnt ----+-------+----- 1 | 100 | 0 2 | 110 | 2 3 | 120 | 0 (3 rows)
CREATE TEMP TABLE z_N3_t(id int, val int);
INSERT INTO z_N3_t VALUES (1,-1);
-- Expected cnt=0 (A*'s empty match takes precedence over B's 1-row match), actual 1
SELECT id, count(*) OVER w AS cnt FROM z_N3_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A* | B) DEFINE A AS val > 0, B AS val <= 0) ORDER BY id;
-- Control: expected cnt=1 (B is the first branch), actual 1 (correct) -> proves branch order is ignored
SELECT id, count(*) OVER w AS cnt FROM z_N3_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (B | A*) DEFINE A AS val > 0, B AS val <= 0) ORDER BY id;
-- SKIP contamination: expected 0,0,0 / actual 2,0,0 (the spurious 2-row match contaminates the skip position)
CREATE TEMP TABLE z_N3_k(id int, val int);
INSERT INTO z_N3_k VALUES (1,-1),(2,-1),(3,-1);
SELECT id, count(*) OVER w AS cnt FROM z_N3_k
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN (A* | B B) DEFINE A AS val > 0, B AS val <= 0) ORDER BY id;-- BASE and HEAD identical; tail of the shared output id | cnt ----+----- 1 | 1 (1 row) id | cnt ----+----- 1 | 0 2 | 0 3 | 0 (3 rows)
CREATE TEMP TABLE z_N4_t (id int, v text);
INSERT INTO z_N4_t VALUES (1,'A'),(2,'A');
-- Direction 1: engine 1,1 / standard 2,0 (Perl /\A(?:A?){2}?/ on "AA" -> "AA")
SELECT id, count(*) OVER w AS cnt FROM z_N4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((A?){2}?) DEFINE A AS v = 'A') ORDER BY id;
-- Control (correct): ((A?){2}) -> 2,0
SELECT id, count(*) OVER w AS cnt FROM z_N4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((A?){2}) DEFINE A AS v = 'A') ORDER BY id;
-- Direction 2: engine 2,0 / standard 0,0 (Perl /\A(?:A??){2}/ on "AA" -> "")
SELECT id, count(*) OVER w AS cnt FROM z_N4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN ((A??){2}) DEFINE A AS v = 'A') ORDER BY id;
-- Range extension (T10 family): engine 1,1,1 / standard 2,2,1
CREATE TEMP TABLE z_N4_c (id int, flags text[]);
INSERT INTO z_N4_c VALUES (1,ARRAY['C']),(2,ARRAY['C']),(3,ARRAY['C']);
SELECT id, count(*) OVER w AS cnt FROM z_N4_c
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN ((C | C*){2,3}?) DEFINE C AS 'C' = ANY(flags)) ORDER BY id;-- BASE and HEAD identical; tail of the shared output ----+----- 1 | 0 2 | 0 (2 rows) id | cnt ----+----- 1 | 2 2 | 2 3 | 1 (3 rows)
CREATE TEMP TABLE z_N5_d4(id int, flags text[]);
INSERT INTO z_N5_d4 VALUES (1,ARRAY['A']),(2,ARRAY['B']);
SELECT id, count(*) OVER w AS cnt
FROM z_N5_d4
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN ((A? | B)*)
DEFINE A AS 'A'=ANY(flags), B AS 'B'=ANY(flags))
ORDER BY id;
-- Expected cnt=1,0 / current (buggy) cnt=2,1-- BASE and HEAD identical; tail of the shared output id | cnt ----+----- 1 | 1 2 | 0 (2 rows)
-- N6: empty-match stats misclassified for non-target contexts (instrumentation only)
-- Expected after fix: NFA: 4 matched (len 0/0/0.0), 0 mismatched / 0 pruned
-- Actual today: NFA: 1 matched (len 0/0/0.0), 2 mismatched (len 2/3/2.5) / 1 pruned
EXPLAIN (ANALYZE, BUFFERS OFF, COSTS OFF, TIMING OFF, SUMMARY OFF)
SELECT count(*) OVER w FROM generate_series(1,4) AS s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN (A+ C | B*)
DEFINE A AS v > 0, C AS v < 0, B AS v < 0);
-- Control (baseline, already correct): 4 matched (len 0/0/0.0), 0 mismatched / 0 pruned
EXPLAIN (ANALYZE, BUFFERS OFF, COSTS OFF, TIMING OFF, SUMMARY OFF)
SELECT count(*) OVER w FROM generate_series(1,4) AS s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN (B*)
DEFINE B AS v < 0);
-- Mid-partition variant (proves independence from partition-end finalization)
-- Expected after fix: 4 matched (len 0/0/0.0) / 0 pruned; today: 2 matched / 1 mismatched / 1 pruned
EXPLAIN (ANALYZE, BUFFERS OFF, COSTS OFF, TIMING OFF, SUMMARY OFF)
SELECT count(*) OVER w FROM (VALUES (1),(1),(0),(0)) AS t(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN (A+ B | C*)
DEFINE A AS v > 0, B AS v > 1, C AS v > 9);
-- User-visible sanity: all rows count=0 regardless of fix
SELECT count(*) OVER w FROM generate_series(1,4) AS s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN (A+ C | B*)
DEFINE A AS v > 0, C AS v < 0, B AS v < 0);-- BASE and HEAD identical; tail of the shared output
NFA: 4 matched (len 0/0/0.0), 0 mismatched
-> Values Scan on "*VALUES*" (actual rows=4.00 loops=1)
(9 rows)
count
-------
0
0
0
0
(4 rows)CREATE TEMP TABLE z_MR07_t (id int, v int); INSERT INTO z_MR07_t VALUES (1, 1), (2, 2), (3, 3); -- A is false at every row: per ISO 19075-5 no row is ever tentatively -- mapped to B, so B's condition must never be evaluated. Expected: 3 -- unmatched rows (cnt=0). Actual: ERROR division by zero. SELECT id, v, count(*) OVER w AS cnt FROM z_MR07_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A B) DEFINE A AS v < 0, B AS 1 / (v - v) > 0 ); -- Single-row partition: B needs a second row, can never be tested. CREATE TEMP TABLE z_MR07_t2 (id int, v int); INSERT INTO z_MR07_t2 VALUES (1, 10); SELECT id, v, count(*) OVER w AS cnt FROM z_MR07_t2 WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A B) DEFINE A AS v = 10, B AS 10 / (v - 10) > 0 ); -- Control: CASE guard returns the correct result (cnt=0), proving the -- error comes from eager evaluation of the never-tested variable. SELECT id, v, count(*) OVER w AS cnt FROM z_MR07_t2 WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A B) DEFINE A AS v = 10, B AS CASE WHEN v <> 10 THEN 10 / (v - 10) > 0 ELSE false END );
-- BASE and HEAD identical; tail of the shared output 3 | 3 | 0 (3 rows) id | v | cnt ----+----+----- 1 | 10 | 0 (1 row) id | v | cnt ----+----+----- 1 | 10 | 0 (1 row)
CREATE TEMP TABLE z_A1_t (id int, v text);
INSERT INTO z_A1_t SELECT g, 'a' FROM generate_series(1, 10) g;
-- buggy: rows 2..9 unmatched (NULL); fixed: every row i matches [i,i]
SELECT id,
first_value(id) OVER w AS match_start,
last_value(id) OVER w AS match_end
FROM z_A1_t
WINDOW w AS (
ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN (A+ B | C)
DEFINE A AS v = 'a', B AS v = 'b', C AS v = 'a'
)
ORDER BY id;-- BASE and HEAD identical; tail of the shared output 2 | 2 | 2 3 | 3 | 3 4 | 4 | 4 5 | 5 | 5 6 | 6 | 6 7 | 7 | 7 8 | 8 | 8 9 | 9 | 9 10 | 10 | 10 (10 rows)
-- A2: SKIP TO NEXT ROW + long match => quadratic runtime (values correct; timing-only repro)
CREATE TEMP TABLE z_A2_t AS SELECT g AS id FROM generate_series(1, 5000) g;
SELECT count(*) FROM (
SELECT count(*) OVER w AS c
FROM z_A2_t
WINDOW w AS (
ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
INITIAL
PATTERN (A B*)
DEFINE A AS TRUE, B AS TRUE
)) s;
TRUNCATE z_A2_t;
INSERT INTO z_A2_t SELECT g FROM generate_series(1, 10000) g;
SELECT count(*) FROM (
SELECT count(*) OVER w AS c
FROM z_A2_t
WINDOW w AS (
ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
INITIAL
PATTERN (A B*)
DEFINE A AS TRUE, B AS TRUE
)) s;
-- Expected: second run ~4x the elapsed time of the first (2x rows).
-- Prior measurements: 1.88s @5k -> 7.54s @10k; contrast SKIP PAST LAST ROW, which is near-linear.-- BASE and HEAD identical; tail of the shared output count ------- 5000 (1 row) count ------- 10000 (1 row)
CREATE TEMP TABLE z_p1_t (id int, a bool, b bool);
INSERT INTO z_p1_t VALUES (1,true,false),(2,true,false),(3,true,false),(4,true,false);
-- merged by mergeConsecutiveAlts into (a+ | b){2}: returns 0/0/0/0 (all matches lost)
SELECT id, count(*) OVER w AS len FROM z_p1_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A+ | B) (A+ | B)) DEFINE A AS a, B AS b) ORDER BY id;
-- renamed baseline, cannot merge: returns 4/0/0/0
SELECT id, count(*) OVER w AS len FROM z_p1_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A+ | B) (C+ | D)) DEFINE A AS a, B AS b, C AS a, D AS b) ORDER BY id;
-- executor root proof: user-written rolled form also returns 0/0/0/0
SELECT id, count(*) OVER w AS len FROM z_p1_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A+ | B){2}) DEFINE A AS a, B AS b) ORDER BY id;
-- minimal 2-row preferment case: merged 2/0 vs baseline 1/1
CREATE TEMP TABLE z_p1_u (id int, a bool, b bool);
INSERT INTO z_p1_u VALUES (1,false,true),(2,false,true);
SELECT id, count(*) OVER w AS len FROM z_p1_u
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A? | B) (A? | B)) DEFINE A AS a, B AS b) ORDER BY id;
SELECT id, count(*) OVER w AS len FROM z_p1_u
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A? | B) (C? | D)) DEFINE A AS a, B AS b, C AS a, D AS b) ORDER BY id;-- BASE and HEAD identical; tail of the shared output id | len ----+----- 1 | 0 2 | 0 (2 rows) id | len ----+----- 1 | 0 2 | 0 (2 rows)
-- P2: consecutive GROUP merge preferment violation (rpr.c mergeConsecutiveGroups)
-- Mechanism A: reluctant inside body. Rows: B A B A A B A
CREATE TEMP TABLE z_P2_ba(id int, a bool, b bool);
INSERT INTO z_P2_ba VALUES
(1,false,true),(2,true,false),(3,false,true),(4,true,false),
(5,true,false),(6,false,true),(7,true,false);
-- merged to (b a+?){2,}: returns 4. Standard preferment: 7 = (B A)(B A A)(B A)
SELECT id, count(*) OVER w AS cnt
FROM z_P2_ba
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN ((B A+?)+ (B A+?)+)
DEFINE A AS a, B AS b)
ORDER BY id;
-- control: alias-differentiated twin (merge cannot apply) -> 7
SELECT id, count(*) OVER w AS cnt
FROM z_P2_ba
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN ((B A+?)+ (D C+?)+)
DEFINE A AS a, B AS b, C AS a, D AS b)
ORDER BY id;
-- Mechanism B: unequal-length alternatives in body. Rows: a ab b a
CREATE TEMP TABLE z_P2_ab(id int, val text);
INSERT INTO z_P2_ab VALUES (1,'a'),(2,'ab'),(3,'b'),(4,'a');
-- merged to (a | b{2}){2,}: returns 2. Standard preferment: 4 = A,BB,A
SELECT id, count(*) OVER w AS cnt
FROM z_P2_ab
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN ((A | B B)+ (A | B B)+)
DEFINE A AS strpos(val,'a') > 0, B AS strpos(val,'b') > 0)
ORDER BY id;
-- control: alias-differentiated twin -> 4
SELECT id, count(*) OVER w AS cnt
FROM z_P2_ab
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN ((A | B B)+ (C | D D)+)
DEFINE A AS strpos(val,'a') > 0, B AS strpos(val,'b') > 0,
C AS strpos(val,'a') > 0, D AS strpos(val,'b') > 0)
ORDER BY id;-- BASE and HEAD identical; tail of the shared output 3 | 0 4 | 0 (4 rows) id | cnt ----+----- 1 | 4 2 | 0 3 | 0 4 | 0 (4 rows)
CREATE TEMP TABLE z_P3_t (rn int, kind text);
INSERT INTO z_P3_t VALUES (1,'a'),(2,'ab'),(3,'b'),(4,'a');
-- merged form: returns cnt=2 at rn=1 (WRONG)
SELECT rn, kind, count(*) OVER w AS cnt FROM z_P3_t
WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A | B B){1,2} (A | B B))
DEFINE A AS kind LIKE '%a%', B AS kind LIKE '%b%');
-- control, same predicates renamed so no merge: cnt=4 at rn=1 (correct)
SELECT rn, kind, count(*) OVER w AS cnt FROM z_P3_t
WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A | B B){1,2} (C | D D))
DEFINE A AS kind LIKE '%a%', B AS kind LIKE '%b%',
C AS kind LIKE '%a%', D AS kind LIKE '%b%');
-- evidence of the fold
EXPLAIN (COSTS OFF)
SELECT count(*) OVER w FROM z_P3_t
WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A | B B){1,2} (A | B B))
DEFINE A AS kind LIKE '%a%', B AS kind LIKE '%b%');
-- no-ALT variant: min<max quantifier alone diverges (5 vs 6)
CREATE TEMP TABLE z_P3_t2 (rn int, kind text);
INSERT INTO z_P3_t2 VALUES (1,'a'),(2,'b'),(3,'a'),(4,'b'),(5,'ab'),(6,'b');
SELECT rn, count(*) OVER w AS cnt FROM z_P3_t2
WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN ((A B{1,2}){1,2} (A B{1,2}))
DEFINE A AS kind LIKE '%a%', B AS kind LIKE '%b%');-- BASE and HEAD identical; tail of the shared output (7 rows) rn | cnt ----+----- 1 | 6 2 | 0 3 | 0 4 | 0 5 | 0 6 | 0 (6 rows)
CREATE TEMP TABLE z_P4_t (id int, v int);
INSERT INTO z_P4_t VALUES (1,1),(2,1),(3,1),(4,1),(5,0);
-- folded to a{2,6} (see EXPLAIN Pattern: line); match_end at id=1 is 4,
-- standard/Perl preferred match is rows 1-3 (match_end 3)
EXPLAIN (COSTS OFF)
SELECT count(*) OVER w FROM z_P4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN ((A{2,3}){1,2}) DEFINE A AS v = 1);
SELECT id, v, first_value(id) OVER w AS match_start,
last_value(id) OVER w AS match_end
FROM z_P4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN ((A{2,3}){1,2}) DEFINE A AS v = 1);
-- fold-blocked semantically identical baseline (SEQ child): match_end at id=1 is 3
SELECT id, v, first_value(id) OVER w AS match_start,
last_value(id) OVER w AS match_end
FROM z_P4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW
PATTERN ((A{2} B?){1,2}) DEFINE A AS v = 1, B AS v = 1);-- BASE and HEAD identical; tail of the shared output 5 | 0 | | (5 rows) id | v | match_start | match_end ----+---+-------------+----------- 1 | 1 | 1 | 3 2 | 1 | 2 | 4 3 | 1 | 3 | 4 4 | 1 | | 5 | 0 | | (5 rows)
-- C1 / const-fold V1: volatile DEFAULT arg injected into DEFINE after the volatility check
CREATE TEMP TABLE z_c1_coin(ts int, price int);
INSERT INTO z_c1_coin SELECT g, 50 FROM generate_series(1,10) g;
INSERT INTO z_c1_coin VALUES (11, 10);
CREATE FUNCTION pg_temp.z_c1_rising(cur int, jitter float8 DEFAULT random())
RETURNS bool LANGUAGE sql STABLE AS 'SELECT cur + jitter*100 > 100';
-- baseline: explicit volatile argument is rejected (ERROR: volatile functions are not allowed in DEFINE clause)
SELECT 1 FROM z_c1_coin WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS pg_temp.z_c1_rising(price, random())) LIMIT 1;
-- control: explicit constant argument is deterministic (always 11,0,0,0,0,0,0,0,0,0,0)
SELECT string_agg(c::text,',') FROM (SELECT count(*) OVER w AS c FROM z_c1_coin
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+ B) DEFINE A AS pg_temp.z_c1_rising(price, 0.9), B AS price < 100)) s;
-- V1: same semantics via DEFAULT slips past the check; repeat 3+ times, each run differs, no error
SELECT string_agg(c::text,',') FROM (SELECT count(*) OVER w AS c FROM z_c1_coin
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+ B) DEFINE A AS pg_temp.z_c1_rising(price), B AS price < 100)) s;
-- nextval DEFAULT variant: sequence-op ban bypassed the same way (currval advances to 11)
CREATE TEMP SEQUENCE z_c1_seq;
CREATE FUNCTION pg_temp.z_c1_seqfn(cur int, n bigint DEFAULT nextval('z_c1_seq'))
RETURNS bool LANGUAGE sql STABLE AS 'SELECT cur + n > 0';
SELECT count(*) FROM (SELECT count(*) OVER w AS c FROM z_c1_coin
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS pg_temp.z_c1_seqfn(price))) s;
SELECT currval('z_c1_seq') AS seq_advanced;--- BASE
+++ HEAD
@@ -1,4 +1,2 @@
ERROR: volatile functions are not allowed in DEFINE clause
-LINE 2: ...TTERN (A+) DEFINE A AS pg_temp.z_c1_rising(price, random()))...
- ^
string_agg
@@ -8,15 +6,4 @@
- string_agg
------------------------
- 0,3,0,0,0,4,0,0,0,0,0
-(1 row)
-
- count
--------
- 11
-(1 row)
-
- seq_advanced
---------------
- 11
-(1 row)
+ERROR: volatile functions are not allowed in DEFINE clause
+ERROR: volatile functions are not allowed in DEFINE clause
+ERROR: currval of sequence "z_c1_seq" is not yet defined in this session-- C2 / const-fold V3 repro (self-contained; needs superuser for shell type I/O)
DROP CAST IF EXISTS (z_c2_type AS text);
DROP TYPE IF EXISTS z_c2_type CASCADE;
DROP FUNCTION IF EXISTS z_c2_vol_cast(z_c2_type);
CREATE TYPE z_c2_type;
CREATE FUNCTION z_c2_in(cstring) RETURNS z_c2_type LANGUAGE internal IMMUTABLE STRICT AS 'textin';
CREATE FUNCTION z_c2_out(z_c2_type) RETURNS cstring LANGUAGE internal IMMUTABLE STRICT AS 'textout';
CREATE TYPE z_c2_type (INPUT = z_c2_in, OUTPUT = z_c2_out, INTERNALLENGTH = -1, CATEGORY = 'S');
CREATE FUNCTION z_c2_vol_cast(z_c2_type) RETURNS text LANGUAGE sql VOLATILE
AS $$ SELECT CASE WHEN random() < 0.5 THEN 'x' ELSE 'y' END $$;
CREATE CAST (z_c2_type AS text) WITH FUNCTION z_c2_vol_cast(z_c2_type) AS IMPLICIT;
CREATE TEMP TABLE z_c2_t1(k int, u z_c2_type);
CREATE TEMP TABLE z_c2_t2(k2 int, u text); -- stays empty: join layer deterministic
INSERT INTO z_c2_t1 SELECT g, 'x' FROM generate_series(1, 20) g;
-- Run the following statement several times: matched_rows varies run to run
-- (observed 8, 5, 5), no error, no warning. Expected after fix: ERROR
-- volatile functions are not allowed in DEFINE clause.
SELECT count(*) AS matched_rows FROM (
SELECT first_value(k) OVER w AS fv
FROM z_c2_t1 LEFT JOIN z_c2_t2 USING (u)
WINDOW w AS (ORDER BY k ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
INITIAL PATTERN (A+) DEFINE A AS u = 'x')
) s WHERE fv IS NOT NULL;
-- cleanup
DROP TABLE z_c2_t1, z_c2_t2;
DROP CAST (z_c2_type AS text);
DROP FUNCTION z_c2_vol_cast(z_c2_type);
DROP TYPE z_c2_type CASCADE;--- BASE
+++ HEAD
@@ -7,7 +7,3 @@
^
- matched_rows
---------------
- 7
-(1 row)
-
+ERROR: volatile functions are not allowed in DEFINE clause
NOTICE: drop cascades to 2 other objects-- C3 / const-fold V2 (probabilistic: repeat; fired 29/40 runs live)
CREATE FUNCTION z_C3_off_leak(n bigint DEFAULT (random()*5)::bigint)
RETURNS bigint LANGUAGE sql STABLE AS 'SELECT n';
-- baseline: explicit volatile offset is rejected -> injection path is the only leak
SELECT 1 FROM generate_series(1,5) g(v)
WINDOW w AS (ORDER BY v ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS v > PREV(v, (random()*5)::bigint));
-- repeat this until it fires (\watch c=12 in psql):
SELECT sum(c) FROM (SELECT count(*) OVER w AS c FROM generate_series(1,100) g(v)
WINDOW w AS (ORDER BY v ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS v > PREV(v, z_C3_off_leak()))) s \watch c=12
-- ERROR: cannot fetch row: N before WindowObject's mark position: M
DROP FUNCTION z_C3_off_leak(bigint);--- BASE +++ HEAD @@ -1,4 +1,2 @@ ERROR: volatile functions are not allowed in DEFINE clause -LINE 3: ... PATTERN (A+) DEFINE A AS v > PREV(v, (random()*5... - ^ -ERROR: cannot fetch row: 0 before WindowObject's mark position: 1 +ERROR: volatile functions are not allowed in DEFINE clause
-- MR09: #11 post-hoc recheck (in subquery_planner, right after the defineClause is
-- processed by preprocess_expression, re-invoke validate_rpr_define_volatility)
-- applied; running on such a build shows the mispointed error cursor.
-- On the current branch (no post-hoc check) this is silently accepted, and with debug_print_plan
-- the random() FUNCEXPR :location 7 inside the folded defineClause (an offset relative to
-- the body 'SELECT random()') can be observed.
CREATE FUNCTION z_mr09_vf() RETURNS float8
AS 'SELECT random()' LANGUAGE sql VOLATILE;
CREATE FUNCTION z_mr09_g(x float8 DEFAULT z_mr09_vf()) RETURNS float8
AS 'begin return x; end' LANGUAGE plpgsql STABLE;
CREATE TEMP TABLE z_mr09_t(company text, price float8);
INSERT INTO z_mr09_t VALUES ('a',10),('a',20);
SELECT company, price, first_value(price) OVER w
FROM z_mr09_t
WINDOW w AS (
PARTITION BY company
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
INITIAL
PATTERN (START UP+)
DEFINE START AS TRUE, UP AS price > z_mr09_g()
);
DROP FUNCTION z_mr09_g(float8);
DROP FUNCTION z_mr09_vf();--- BASE +++ HEAD @@ -1,5 +1 @@ - company | price | first_value ----------+-------+------------- - a | 10 | 10 - a | 20 | -(2 rows) +ERROR: volatile functions are not allowed in DEFINE clause
-- C4 / const-fold S2: junk-TLE dedup ignores varlevelsup
CREATE TEMP TABLE z_C4_o(a int, b int); -- lateral o.b => (varno=1, varattno=2, varlevelsup=1)
CREATE TEMP TABLE z_C4_i(x int, y int); -- DEFINE's i.y => (varno=1, varattno=2, varlevelsup=0)
INSERT INTO z_C4_o VALUES (1, 10);
INSERT INTO z_C4_i VALUES (1, 5), (2, 6);
-- collision: ERROR: variable not found in subplan target list (XX000)
SELECT * FROM z_C4_o o, LATERAL (
SELECT o.b AS lat, count(*) OVER w AS c FROM z_C4_i
WINDOW w AS (ORDER BY x ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS y > 0)) s;
-- control: o.a (attno 1) removes the collision => works, returns 2 rows
SELECT * FROM z_C4_o o, LATERAL (
SELECT o.a AS lat, count(*) OVER w AS c FROM z_C4_i
WINDOW w AS (ORDER BY x ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS y > 0)) s;
DROP TABLE z_C4_o, z_C4_i;--- BASE +++ HEAD @@ -1,2 +1,7 @@ -ERROR: variable not found in subplan target list + a | b | lat | c +---+----+-----+--- + 1 | 10 | 10 | 2 + 1 | 10 | 10 | 0 +(2 rows) + a | b | lat | c
CREATE TEMP TABLE z_C5_d(v int);
INSERT INTO z_C5_d SELECT generate_series(1,10);
CREATE TEMP TABLE z_C5_t(x int8);
INSERT INTO z_C5_t VALUES (2);
CREATE FUNCTION pg_temp.z_C5_f(n int8) RETURNS TABLE(cnt bigint)
LANGUAGE sql STABLE AS $$
SELECT count(*) OVER w FROM z_C5_d
WINDOW w AS (ORDER BY v ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS v > PREV(v, n)) $$;
SELECT count(*) FROM pg_temp.z_C5_f(2); -- OK: 10
SELECT * FROM z_C5_t, pg_temp.z_C5_f(z_C5_t.x); -- ERROR: cannot fetch row: 0 before WindowObject's mark position: 1--- BASE +++ HEAD @@ -5,2 +5,14 @@ -ERROR: cannot fetch row: 0 before WindowObject's mark position: 1 + x | cnt +---+----- + 2 | 0 + 2 | 0 + 2 | 8 + 2 | 0 + 2 | 0 + 2 | 0 + 2 | 0 + 2 | 0 + 2 | 0 + 2 | 0 +(10 rows)
-- z_C6_ repro: self-contained, no objects created EXPLAIN (GENERIC_PLAN, COSTS OFF) SELECT count(*) OVER w FROM generate_series(1,10) s(v) WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS v > PREV(v, $1::int)); -- lookahead arm, same root cause: EXPLAIN (GENERIC_PLAN, COSTS OFF) SELECT count(*) OVER w FROM generate_series(1,10) s(v) WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A+) DEFINE A AS NEXT(FIRST(v, $1::int8), $2::int8) IS NOT NULL);
--- BASE +++ HEAD @@ -1,2 +1,18 @@ -ERROR: no value found for parameter 1 -ERROR: no value found for parameter 1 + QUERY PLAN +------------------------------------------------------------------- + WindowAgg + Window: w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a+" + Nav Mark Lookback: runtime + -> Function Scan on generate_series s +(5 rows) + + QUERY PLAN +------------------------------------------------------------------- + WindowAgg + Window: w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) + Pattern: a+ + Nav Mark Lookback: 0 + Nav Mark Lookahead: runtime + -> Function Scan on generate_series s +(6 rows)
CREATE TEMP TABLE z_E1_t (id int, v int); INSERT INTO z_E1_t VALUES (1,10),(2,20),(3,30); -- (a) PREV @ partition start: standard = nav NULL -> unknown -> row 1 must NOT match SELECT id, count(*) OVER w AS cnt FROM z_E1_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A) DEFINE A AS PREV(v IS NULL)); -- (b) COALESCE variant, same expectation SELECT id, count(*) OVER w AS cnt FROM z_E1_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A) DEFINE A AS PREV(COALESCE(v,-1)) = -1); -- (c) NEXT @ partition end SELECT id, count(*) OVER w AS cnt FROM z_E1_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A) DEFINE A AS NEXT(COALESCE(v,-1)) = -1); -- (d) FIRST offset beyond match range SELECT id, count(*) OVER w AS cnt FROM z_E1_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A) DEFINE A AS FIRST(v IS NULL, 5)); -- (e) compound PREV(FIRST(...)) out of range SELECT id, count(*) OVER w AS cnt FROM z_E1_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A) DEFINE A AS PREV(FIRST(v IS NULL, 9), 1)); -- control: IS NULL outside nav -- row 1 match is correct, must stay unchanged SELECT id, count(*) OVER w AS cnt FROM z_E1_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING INITIAL PATTERN (A) DEFINE A AS PREV(v) IS NULL);
--- BASE +++ HEAD @@ -2,3 +2,31 @@ ----+----- - 1 | 1 + 1 | 0 + 2 | 0 + 3 | 0 +(3 rows) + + id | cnt +----+----- + 1 | 0 + 2 | 0 + 3 | 0 +(3 rows) + + id | cnt +----+----- + 1 | 0 + 2 | 0 + 3 | 0 +(3 rows) + + id | cnt +----+----- + 1 | 0 + 2 | 0 + 3 | 0 +(3 rows) + + id | cnt +----+----- + 1 | 0 2 | 0 @@ -13,29 +41 @@ (3 rows) - - id | cnt -----+----- - 1 | 0 - 2 | 0 - 3 | 1 -(3 rows) - - id | cnt -----+----- - 1 | 1 - 2 | 1 - 3 | 1 -(3 rows) - - id | cnt -----+----- - 1 | 1 - 2 | 1 - 3 | 1 -(3 rows) - - id | cnt -----+----- - 1 | 1 - 2 | 0 - 3 | 0 -(3 rows)
CREATE TEMP TABLE z_E2_nav (id int, val int);
INSERT INTO z_E2_nav VALUES (1,10),(2,20),(3,30);
-- Control (inner in-range): errors as expected
SELECT id, val, count(*) OVER w FROM z_E2_nav WINDOW w AS (
ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A B+)
DEFINE A AS TRUE, B AS PREV(FIRST(val, 0), NULL::int8) IS NULL);
SELECT id, val, count(*) OVER w FROM z_E2_nav WINDOW w AS (
ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A B+)
DEFINE A AS TRUE, B AS NEXT(LAST(val, 0), -5) IS NULL);
-- Bug (inner out-of-range, offset 99): same illegal outer offsets silently succeed
SELECT id, val, count(*) OVER w FROM z_E2_nav WINDOW w AS (
ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A B+)
DEFINE A AS TRUE, B AS PREV(FIRST(val, 99), NULL::int8) IS NULL);
SELECT id, val, count(*) OVER w FROM z_E2_nav WINDOW w AS (
ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A B+)
DEFINE A AS TRUE, B AS NEXT(LAST(val, 99), -5) IS NULL);--- BASE +++ HEAD @@ -2,14 +2,3 @@ ERROR: row pattern navigation offset must not be negative - id | val | count -----+-----+------- - 1 | 10 | 3 - 2 | 20 | 0 - 3 | 30 | 0 -(3 rows) - - id | val | count -----+-----+------- - 1 | 10 | 3 - 2 | 20 | 0 - 3 | 30 | 0 -(3 rows) +ERROR: row pattern navigation offset must not be null +ERROR: row pattern navigation offset must not be negative
three NEXT_FIRST comments contradict the code -- not observable by execution
On BASE this takes the whole backend down, so the server spends the next moment in recovery. Wait for it to accept connections again before running the blocks that follow.
CREATE TEMP TABLE z_X1_crash (id int, grp int, val int);
INSERT INTO z_X1_crash VALUES (1,1,10),(2,1,20);
-- assert build: backend SIGABRT (TRAP: dest_tle->resno == src_tle->resno, tlist.c)
SELECT id, count(*) OVER w1 AS c1, count(*) OVER w2 AS c2
FROM z_X1_crash
WINDOW w1 AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (S U+) DEFINE U AS val > PREV(val)),
w2 AS (PARTITION BY grp ORDER BY id);--- BASE +++ HEAD @@ -1,4 +1,5 @@ -server closed the connection unexpectedly - This probably means the server terminated abnormally - before or while processing the request. -error: connection to server was lost + id | c1 | c2 +----+----+---- + 1 | 2 | 1 + 2 | 0 | 2 +(2 rows)
BEGIN;
CREATE TEMP TABLE z_W1_outer (threshold int);
INSERT INTO z_W1_outer VALUES (10), (200);
CREATE TEMP TABLE z_W1_stock (price int);
INSERT INTO z_W1_stock SELECT g FROM generate_series(1, 100) g;
CREATE FUNCTION pg_temp.z_W1_fn(th int) RETURNS SETOF bigint
LANGUAGE SQL STABLE AS $$
SELECT DISTINCT count(*) OVER w FROM z_W1_stock
WINDOW w AS (ORDER BY price ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (a+)
DEFINE a AS price > th) $$;
-- Ground truth: th=200 must yield a single row, 0.
SELECT * FROM pg_temp.z_W1_fn(10) ORDER BY 1;
SELECT * FROM pg_temp.z_W1_fn(200) ORDER BY 1;
SET LOCAL enable_sort = off; -- force HashAgg DISTINCT (chgParam-gated cache)
SELECT o.threshold, f FROM z_W1_outer o, LATERAL pg_temp.z_W1_fn(o.threshold) f
ORDER BY 1, 2;
-- buggy: 10|0 10|90 200|0 200|90 (200|90 is th=10's stale answer)
-- fixed: 10|0 10|90 200|0
ROLLBACK;--- BASE
+++ HEAD
@@ -16,3 +16,2 @@
200 | 0
- 200 | 90
-(4 rows)
+(3 rows)CREATE TEMP TABLE z_W2_t (category text, val int);
INSERT INTO z_W2_t VALUES ('A',10),('A',20),('B',5);
-- control: plain GROUP BY + RPR -> works (A|2, B|0)
SELECT category, count(*) OVER w AS cnt
FROM z_W2_t
GROUP BY category
WINDOW w AS (ORDER BY category ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS category >= 'A');
-- defect variant 1: ROLLUP (also CUBE / 2+ GROUPING SETS)
-- ERROR: wrong varnullingrels (b) (expected (b 2)) for Var 1/1
SELECT category, count(*) OVER w AS cnt
FROM z_W2_t
GROUP BY ROLLUP(category)
WINDOW w AS (ORDER BY category ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS category >= 'A');
-- defect variant 2: VALUES source
-- ERROR: wrong phnullingrels (b) (expected (b 2)) for PlaceHolderVar 1
SELECT category, count(*) OVER w AS cnt
FROM (VALUES ('A',10),('A',20),('B',5)) v(category, val)
GROUP BY ROLLUP(category)
WINDOW w AS (ORDER BY category ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS category >= 'A');
-- boundary: single grouping set collapses to plain GROUP BY -> works
SELECT category, count(*) OVER w AS cnt
FROM z_W2_t
GROUP BY GROUPING SETS ((category))
WINDOW w AS (ORDER BY category ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS category >= 'A');-- BASE and HEAD identical; tail of the shared output A | 2 B | 0 (2 rows) ERROR: wrong varnullingrels (b) (expected (b 2)) for Var 1/1 ERROR: wrong phnullingrels (b) (expected (b 2)) for PlaceHolderVar 1 category | cnt ----------+----- A | 2 B | 0 (2 rows)
CREATE TEMP TABLE z_W3_gb (id int, val int);
INSERT INTO z_W3_gb VALUES (1,10),(2,25);
-- baseline (succeeds): non-RPR window over the same group expression
SELECT val/10 AS bucket, count(*) OVER (ORDER BY val/10) AS cnt
FROM z_W3_gb GROUP BY val/10;
-- false rejection: DEFINE uses exactly the GROUP BY expression
SELECT val/10 AS bucket, count(*) OVER w AS cnt
FROM z_W3_gb
GROUP BY val/10
WINDOW w AS (ORDER BY val/10
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+)
DEFINE A AS val/10 >= 0);-- BASE and HEAD identical; tail of the shared output
bucket | cnt
--------+-----
1 | 1
2 | 2
(2 rows)
ERROR: column "z_w3_gb.val" must appear in the GROUP BY clause or be used in an aggregate function
LINE 7: DEFINE A AS val/10 >= 0);
^CREATE TEMP TABLE z_W4_t(id int);
INSERT INTO z_W4_t VALUES (1),(2);
-- 1. baseline, top level: ERROR (correct)
SELECT id FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5);
-- 2. F5 FROM-subquery pull-up: silently succeeds (BUG)
SELECT id FROM (SELECT id FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5)) s;
-- 3. control, OFFSET 0 fence: ERROR restored
SELECT id FROM (SELECT id FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5) OFFSET 0) s;
-- 4. F9 EXISTS simplification: silently succeeds (BUG)
SELECT 1 WHERE EXISTS (SELECT id FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5));
-- 5. R3 UNION ALL leaf pull-up: silently succeeds (BUG)
SELECT id FROM z_W4_t
UNION ALL
SELECT id FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5);
-- 6. control, window used by OVER: ERROR
SELECT id FROM (SELECT id, count(*) OVER w FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5)) s;
-- 7. unused volatile RPR window in a view: create + read both succeed (BUG)
CREATE TEMP VIEW z_W4_v AS
SELECT id FROM z_W4_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS random() > 0.5);
SELECT id FROM (SELECT id FROM z_W4_v) x;
DROP VIEW z_W4_v;
DROP TABLE z_W4_t;--- BASE +++ HEAD @@ -1,33 +1,7 @@ ERROR: volatile functions are not allowed in DEFINE clause -LINE 3: PATTERN (A+) DEFINE A AS random() > 0.5); - ^ - id ----- - 1 - 2 -(2 rows) - ERROR: volatile functions are not allowed in DEFINE clause -LINE 3: PATTERN (A+) DEFINE A AS random() > 0.5) OFFS... - ^ - ?column? ----------- - 1 -(1 row) - - id ----- - 1 - 2 - 1 - 2 -(4 rows) - ERROR: volatile functions are not allowed in DEFINE clause -LINE 3: PATTERN (A+) DEFINE A AS random() > 0.5)) s; - ^ - id ----- - 1 - 2 -(2 rows) +ERROR: volatile functions are not allowed in DEFINE clause +ERROR: volatile functions are not allowed in DEFINE clause +ERROR: volatile functions are not allowed in DEFINE clause +ERROR: volatile functions are not allowed in DEFINE clause
-- W5 / opt-window F6
CREATE TEMP TABLE z_W5_stock (ts int, price int);
INSERT INTO z_W5_stock SELECT g, 100 + (g % 7) FROM generate_series(1, 1000) g;
-- 1) RPR window: Run Condition absent, filter degraded to Subquery Scan Filter
EXPLAIN (COSTS OFF)
SELECT * FROM (
SELECT ts, row_number() OVER w AS rn FROM z_W5_stock
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW INITIAL
PATTERN (A+) DEFINE A AS price > PREV(price))
) s WHERE rn <= 10;
-- 2) Same shape without PATTERN/DEFINE: Run Condition appears
EXPLAIN (COSTS OFF)
SELECT * FROM (
SELECT ts, row_number() OVER w AS rn FROM z_W5_stock
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)
) s WHERE rn <= 10;
-- 3) Full-unblock is UNSAFE: count(*) over an RPR window is non-monotonic
-- (sequence 2,0,0,2,0), so a DECREASING run condition "cnt > 0" would
-- stop at id=2 and silently drop the id=4 match.
CREATE TEMP TABLE z_W5_mono (id int, v int);
INSERT INTO z_W5_mono VALUES (1,1),(2,2),(3,1),(4,0),(5,5);
SELECT id, v, count(*) OVER w AS cnt
FROM z_W5_mono
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN (A B+) DEFINE B AS v > PREV(v));-- BASE and HEAD identical; tail of the shared output
-> Seq Scan on z_w5_stock
(6 rows)
id | v | cnt
----+---+-----
1 | 1 | 2
2 | 2 | 0
3 | 1 | 0
4 | 0 | 2
5 | 5 | 0
(5 rows)CREATE TEMP TABLE z_W6_t (id int, val int);
INSERT INTO z_W6_t VALUES (1,10),(2,20),(3,15),(4,25),(5,5);
-- Case 1: bare unused RPR window function -> guard fires, WindowAgg retained
EXPLAIN (COSTS OFF)
SELECT count(*) FROM (
SELECT count(*) OVER w AS c FROM z_W6_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS val > PREV(val))) t;
SELECT count(*) FROM (
SELECT count(*) OVER w AS c FROM z_W6_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS val > PREV(val))) t;
-- Case 2: same window function nested in an expression -> guard bypassed,
-- WindowAgg dropped entirely, yet the result is identical and correct
EXPLAIN (COSTS OFF)
SELECT count(*) FROM (
SELECT count(*) OVER w + 0 AS c FROM z_W6_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS val > PREV(val))) t;
SELECT count(*) FROM (
SELECT count(*) OVER w + 0 AS c FROM z_W6_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+) DEFINE A AS val > PREV(val))) t;-- BASE and HEAD identical; tail of the shared output
(1 row)
QUERY PLAN
--------------------------
Aggregate
-> Seq Scan on z_w6_t
(2 rows)
count
-------
5
(1 row)defineClause aliased without copying -- code inspection
CREATE TEMP TABLE z_w8_stock (company text, tdate date, price int);
INSERT INTO z_w8_stock VALUES
('c1','2024-01-01',100),('c1','2024-01-02',200),('c1','2024-01-03',150);
-- w2 is referenced by no window function: select_active_windows drops it and
-- no WindowAgg for w2 is planned, yet remove_unused_subquery_outputs keeps
-- the otherwise-unused column "price" in the subquery output solely because
-- w2's DEFINE references it. (count(*) OVER w1 sets hasWindowFuncs and
-- blocks subquery pullup, so remove_unused_subquery_outputs actually runs.)
EXPLAIN (VERBOSE, COSTS OFF)
SELECT cnt FROM (
SELECT count(*) OVER w1 AS cnt, price
FROM z_w8_stock
WINDOW w1 AS (ORDER BY tdate),
w2 AS (ORDER BY tdate
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (a b+)
DEFINE b AS price > PREV(price))
) t;
-- Defect: the WindowAgg "Output:" line still carries "price" (and Sort /
-- Seq Scan below carry it too) although no plan node ever evaluates w2.
-- Fixed: price is replaced with NULL::integer and drops out of the lower
-- targetlists. (The O(cols x DEFINE) facet is code-shape only; observe via
-- planning profile with a wide DEFINE and many removable output columns.)-- BASE and HEAD identical; tail of the shared output
Output: t.cnt
-> WindowAgg
Output: count(*) OVER w1, z_w8_stock.price, z_w8_stock.tdate
Window: w1 AS (ORDER BY z_w8_stock.tdate)
-> Sort
Output: z_w8_stock.tdate, z_w8_stock.price
Sort Key: z_w8_stock.tdate
-> Seq Scan on pg_temp.z_w8_stock
Output: z_w8_stock.tdate, z_w8_stock.price
(10 rows)undocumented guards around pull_var_clause -- call-site comment
CREATE TEMP TABLE z_D1_a (id int, price int);
CREATE TEMP TABLE z_D1_b (id int);
CREATE TEMP VIEW z_D1_v AS
SELECT a.id, count(*) OVER w AS cnt
FROM z_D1_a a JOIN z_D1_b b ON a.id = b.id
WINDOW w AS (ORDER BY a.id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (UP+) DEFINE UP AS price > PREV(price));
-- bug: DEFINE deparses as "up AS (a.price > PREV(a.price))"
SELECT pg_get_viewdef('z_D1_v');
-- reparsing the deparsed text fails:
CREATE TEMP VIEW z_D1_v_rt AS
SELECT a.id, count(*) OVER w AS cnt
FROM (z_D1_a a JOIN z_D1_b b ON ((a.id = b.id)))
WINDOW w AS (ORDER BY a.id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (up+)
DEFINE up AS (a.price > PREV(a.price)) );
DROP TABLE z_D1_a, z_D1_b CASCADE;-- BASE and HEAD identical; tail of the shared output
AFTER MATCH SKIP PAST LAST ROW +
INITIAL +
PATTERN (up+) +
DEFINE +
up AS (a.price > PREV(a.price)) );
(1 row)
ERROR: range variable qualified expression "a.price" is not allowed in DEFINE clause
LINE 6: DEFINE up AS (a.price > PREV(a.price)) );
^
NOTICE: drop cascades to view z_d1_vCREATE TEMP TABLE z_D2_t (v int);
CREATE TEMP VIEW z_D2_v AS
SELECT count(*) OVER w2 FROM z_D2_t
WINDOW "pattern" AS (PARTITION BY v), w2 AS ("pattern" ORDER BY v);
SELECT pg_get_viewdef('z_D2_v', true);
-- deparsed text as pg_dump would emit it: fails to re-parse
SELECT count(*) OVER w2 AS count
FROM z_D2_t
WINDOW pattern AS (PARTITION BY v), w2 AS (pattern ORDER BY v);-- BASE and HEAD identical; tail of the shared output
pg_get_viewdef
-------------------------------------------------------------------
SELECT count(*) OVER w2 AS count +
FROM z_d2_t +
WINDOW pattern AS (PARTITION BY v), w2 AS (pattern ORDER BY v);
(1 row)
ERROR: syntax error at or near "ORDER"
LINE 3: ...INDOW pattern AS (PARTITION BY v), w2 AS (pattern ORDER BY v...
^CREATE TEMP TABLE z_D3_ws (id int, val int);
CREATE VIEW z_D3_ws_v AS
SELECT id, count(*) OVER w FROM z_D3_ws
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW INITIAL
PATTERN (A+) DEFINE A AS val > 0);
SELECT regexp_replace(pg_get_viewdef('z_D3_ws_v'), ' ', '.', 'g') AS deparse_dots;
SELECT pg_get_viewdef('z_D3_ws_v') ~ ' \n' AS has_trailing_space_before_newline;
SELECT pg_get_viewdef('z_D3_ws_v') ~ ' \)' AS has_space_before_close_paren;
DROP VIEW z_D3_ws_v;
DROP TABLE z_D3_ws;--- BASE +++ HEAD @@ -8,4 +8,4 @@ ..WINDOW.w.AS.(ORDER.BY.id.ROWS.BETWEEN.CURRENT.ROW.AND.UNBOUNDED.FOLLOWING.+ - ..AFTER.MATCH.SKIP.PAST.LAST.ROW.. + - ..INITIAL. + + ..AFTER.MATCH.SKIP.PAST.LAST.ROW + + ..INITIAL + ..PATTERN.(a+). +
CREATE TEMP TABLE z_g1_t (rn int, v int); INSERT INTO z_g1_t VALUES (1, 10), (2, 20), (3, 30); -- silently parsed as VAR(permute) then GROUP(A): shows "Pattern: permute a" EXPLAIN (COSTS OFF, VERBOSE) SELECT rn, v, first_value(v) OVER w FROM z_g1_t WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (PERMUTE(A)) DEFINE A AS v > 0); -- diverges from standard PERMUTE(A) == A: one 2-row match vs three 1-row matches SELECT rn, v, count(*) OVER w FROM z_g1_t WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (PERMUTE(A)) DEFINE A AS v > 0); SELECT rn, v, count(*) OVER w FROM z_g1_t WINDOW w AS (ORDER BY rn ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING PATTERN (A) DEFINE A AS v > 0);
-- BASE and HEAD identical; tail of the shared output 1 | 10 | 2 2 | 20 | 0 3 | 30 | 0 (3 rows) rn | v | count ----+----+------- 1 | 10 | 1 2 | 20 | 1 3 | 30 | 1 (3 rows)
CREATE TEMP TABLE z_g2_t (id int, val int);
CREATE TEMP VIEW z_g2_glued AS
SELECT count(*) OVER w FROM z_g2_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A*|B*|C)
DEFINE A AS val > 20, B AS val <= 20, C AS val = 0);
CREATE TEMP VIEW z_g2_spaced AS
SELECT count(*) OVER w FROM z_g2_t
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A* | B* | C)
DEFINE A AS val > 20, B AS val <= 20, C AS val = 0);
-- deparse text identical (flat canonical form)
SELECT pg_get_viewdef('z_g2_glued'::regclass)
= pg_get_viewdef('z_g2_spaced'::regclass) AS same_deparse_text;
-- BUG: stored parse trees differ (glued built nested ALT(A*, ALT(B*, C)))
SELECT (SELECT ev_action FROM pg_rewrite WHERE ev_class = 'z_g2_glued'::regclass)
= (SELECT ev_action FROM pg_rewrite WHERE ev_class = 'z_g2_spaced'::regclass)
AS same_stored_tree;
-- node count: glued 5 vs spaced 4 (the extra node is the synthesized inner ALT)
SELECT (SELECT (length(ev_action) - length(replace(ev_action,'RPRPATTERNNODE','')))/14
FROM pg_rewrite WHERE ev_class = 'z_g2_glued'::regclass) AS glued_nodes,
(SELECT (length(ev_action) - length(replace(ev_action,'RPRPATTERNNODE','')))/14
FROM pg_rewrite WHERE ev_class = 'z_g2_spaced'::regclass) AS spaced_nodes;-- BASE and HEAD identical; tail of the shared output
t
(1 row)
same_stored_tree
------------------
f
(1 row)
glued_nodes | spaced_nodes
-------------+--------------
5 | 4
(1 row)-- z_G3_v5: sibling inner navs at depth 2 -> wrong "more than two levels deep"
SELECT count(*) OVER w FROM generate_series(1,3) s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+)
DEFINE A AS PREV(FIRST(v) + LAST(v)) > 0);
-- control 1: single inner nav, correct direct-argument message
SELECT count(*) OVER w FROM generate_series(1,3) s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+)
DEFINE A AS PREV(v + FIRST(v)) > 0);
-- control 2: genuine 3-level nesting, deep-nesting message is correct here
SELECT count(*) OVER w FROM generate_series(1,3) s(v)
WINDOW w AS (ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+)
DEFINE A AS PREV(FIRST(LAST(v))) > 0);--- BASE +++ HEAD @@ -1,2 +1,2 @@ -ERROR: cannot nest row pattern navigation more than two levels deep +ERROR: row pattern navigation operation must be a direct argument of the outer navigation LINE 4: DEFINE A AS PREV(FIRST(v) + LAST(v)) > 0);
-- z_G4_: Op Op quantifier second-token diagnostic (raw-parse error; no objects needed)
SELECT count(*) OVER w FROM (VALUES (1)) z_g4_v(id)
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A ? !!) DEFINE A AS TRUE);
-- Contrast: sibling '*' Op branch correctly names the token and carets it
SELECT count(*) OVER w FROM (VALUES (1)) z_g4_v(id)
WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A * !!) DEFINE A AS TRUE);--- BASE +++ HEAD @@ -1,4 +1,4 @@ -ERROR: invalid quantifier combination +ERROR: invalid token "!!" after "?" quantifier LINE 3: PATTERN (A ? !!) DEFINE A AS TRUE); - ^ + ^ HINT: Did you mean "??" for reluctant quantifier?
SET compute_query_id = on; CREATE TEMP TABLE z_R1_t (id int, p int); INSERT INTO z_R1_t VALUES (1, 60), (2, 40), (3, 70), (4, 30); -- Query 1: A means p>50 -> PATTERN (A B) = high then low; returns counts 2,0,2,0 SELECT id, p, count(*) OVER w FROM z_R1_t WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (A B) DEFINE A AS p > 50, B AS p < 50); -- Query 2: names swapped -> PATTERN (A B) = low then high; returns counts 0,2,0,0 SELECT id, p, count(*) OVER w FROM z_R1_t WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (A B) DEFINE B AS p > 50, A AS p < 50); -- Both EXPLAINs print the SAME "Query Identifier:" line despite different semantics EXPLAIN (VERBOSE, COSTS OFF) SELECT id, p, count(*) OVER w FROM z_R1_t WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (A B) DEFINE A AS p > 50, B AS p < 50); EXPLAIN (VERBOSE, COSTS OFF) SELECT id, p, count(*) OVER w FROM z_R1_t WINDOW w AS (ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (A B) DEFINE B AS p > 50, A AS p < 50);
-- BASE and HEAD identical; tail of the shared output
Window: w AS (ORDER BY z_r1_t.id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)
Pattern: a b
Nav Mark Lookback: 0
-> Sort
Output: id, p
Sort Key: z_r1_t.id
-> Seq Scan on pg_temp.z_r1_t
Output: id, p
Query Identifier: N
(11 rows)three of four hand-written serialization contracts -- code inspection
-- z_R3_: RPRPattern serialization roundtrip smoke (assert build only) SET debug_write_read_parse_plan_trees = on; SET debug_copy_parse_plan_trees = on; CREATE TEMP TABLE z_R3_t (id int, v int); INSERT INTO z_R3_t VALUES (1,10),(2,20),(3,15),(4,25),(5,30); SELECT id, v, first_value(id) OVER w AS m FROM z_R3_t WINDOW w AS ( ORDER BY id ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING AFTER MATCH SKIP PAST LAST ROW INITIAL PATTERN (STRT UP+ DOWN+) DEFINE UP AS v > PREV(v), DOWN AS v < PREV(v) ); RESET debug_write_read_parse_plan_trees; RESET debug_copy_parse_plan_trees;
-- BASE and HEAD identical; tail of the shared output id | v | m ----+----+--- 1 | 10 | 1 2 | 20 | 3 | 15 | 4 | 25 | 5 | 30 | (5 rows)
-- PF1: spill + FIRST nav quadratic repro (creates no objects, self-contained)
SET work_mem='64kB';
-- Spill: n=2000 ~0.83s, n=4000 ~5.4s (4-6.5x per doubling)
SELECT count(*) FROM (
SELECT id, count(*) OVER w cnt
FROM (SELECT g AS id, repeat('x', 50) AS pad, (g % 11) AS v
FROM generate_series(0, 3999) g) t
WINDOW w AS (ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN (S A+) DEFINE S AS TRUE, A AS v >= FIRST(v))
) s WHERE cnt > 0;
RESET work_mem;
-- Control 1: same query in-memory ~1.3ms
SELECT count(*) FROM (
SELECT id, count(*) OVER w cnt
FROM (SELECT g AS id, repeat('x', 50) AS pad, (g % 11) AS v
FROM generate_series(0, 3999) g) t
WINDOW w AS (ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN (S A+) DEFINE S AS TRUE, A AS v >= FIRST(v))
) s WHERE cnt > 0;
-- Control 2: even with spill, PREV takes ~1.7ms (FIRST alone is the trigger)
SET work_mem='64kB';
SELECT count(*) FROM (
SELECT id, count(*) OVER w cnt
FROM (SELECT g AS id, repeat('x', 50) AS pad, (g % 11) AS v
FROM generate_series(0, 3999) g) t
WINDOW w AS (ORDER BY id
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP PAST LAST ROW
PATTERN (S A+) DEFINE S AS TRUE, A AS v >= PREV(v))
) s WHERE cnt > 0;
RESET work_mem;-- BASE and HEAD identical; tail of the shared output
1
(1 row)
count
-------
1
(1 row)
count
-------
364
(1 row)four allStatesAbsorbable comments corrected
-- z_Q2_ illustrative (falsifies the "error always precedes" wording; not run here)
-- Completes without error: every match is shorter than 6 rows, so the inner
-- FIRST(v, 5) is always out of range and the negative outer offset -1 is
-- never examined (execExprInterp.c compound short-circuit).
SELECT count(*) OVER w FROM generate_series(1,3) s(v)
WINDOW w AS (
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+)
DEFINE A AS PREV(FIRST(v, 5), -1) IS NULL);
-- Contrast: with 10 rows a match reaches its 6th row, the inner comes in
-- range, and the same query errors: "row pattern navigation offset must not
-- be negative" -- proving the error is data-dependent, not guaranteed.
SELECT count(*) OVER w FROM generate_series(1,10) s(v)
WINDOW w AS (
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (A+)
DEFINE A AS PREV(FIRST(v, 5), -1) IS NULL);--- BASE +++ HEAD @@ -1,8 +1,2 @@ - count -------- - 3 - 0 - 0 -(3 rows) - ERROR: row pattern navigation offset must not be negative +ERROR: row pattern navigation offset must not be negative
markpos initialized to -1 per convention -- no behaviour change
executor init writes into a cached plan tree -- code inspection
CREATE TEMP TABLE z_Z1_stock (company text, tdate date, price int);
INSERT INTO z_Z1_stock VALUES ('c1', '2024-01-01', 100), ('c1', '2024-01-02', 200);
-- Decision probe: dead-branch volatile in DEFINE.
-- Current tree (pre-fold check only, planner.c:1082-1083): rejected with
-- ERROR: volatile functions are not allowed in DEFINE clause
-- Post-fold-only policy would accept it (folding deletes the WHEN false
-- branch), matching FOR PORTION OF (planner.c:1123-1130).
SELECT company, tdate, price, count(*) OVER w
FROM z_Z1_stock
WINDOW w AS (
PARTITION BY company ORDER BY tdate
ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
INITIAL
PATTERN (A+)
DEFINE A AS CASE WHEN false THEN random() < 1
ELSE price >= 0 END
);
DROP TABLE z_Z1_stock;--- BASE +++ HEAD @@ -1 +1,5 @@ -psql: error: connection to server at "localhost" (::1), port 55432 failed: FATAL: the database system is in recovery mode + company | tdate | price | count +---------+------------+-------+------- + c1 | 2024-01-01 | 100 | 2 + c1 | 2024-01-02 | 200 | 0 +(2 rows)
-- z_Z2_ verification query (proof of absence): the stock catalogs contain no volatile cast function at all SELECT c.castsource::regtype, c.casttarget::regtype, c.castcontext, p.provolatile FROM pg_cast c JOIN pg_proc p ON p.oid = c.castfunc WHERE p.provolatile = 'v'; -- Expected: 0 rows (0 not only for implicit 'i' but across all contexts, including assignment/explicit)
-- BASE and HEAD identical; tail of the shared output castsource | casttarget | castcontext | provolatile ------------+------------+-------------+------------- (0 rows)
-- Z3 core: phnullingrels variant (VALUES input), self-contained
SELECT count(*) OVER w
FROM (VALUES ('A',1),('A',2),('B',3)) t(category, id)
GROUP BY GROUPING SETS ((category), ())
WINDOW w AS (ORDER BY category ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (P+) DEFINE P AS category = 'A');
-- expected on current tip: ERROR: wrong phnullingrels (b) (expected (b 2)) for PlaceHolderVar 1
-- companion: varnullingrels variant (real table)
CREATE TEMP TABLE z_Z3_g1 (category text, id int);
INSERT INTO z_Z3_g1 VALUES ('A',1),('A',2),('B',3);
SELECT count(*) OVER w FROM z_Z3_g1
GROUP BY GROUPING SETS ((category), ())
WINDOW w AS (ORDER BY category ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (P+) DEFINE P AS category = 'A');
-- expected: ERROR: wrong varnullingrels (b) (expected (b 2)) for Var 1/1-- BASE and HEAD identical; tail of the shared output ERROR: wrong phnullingrels (b) (expected (b 2)) for PlaceHolderVar 1 ERROR: wrong varnullingrels (b) (expected (b 2)) for Var 1/1
-- Z4 / opt-window F6: run-condition blanket block on RPR windows
CREATE TEMP TABLE z_Z4_t (ts int, price int);
INSERT INTO z_Z4_t SELECT g, (g * 37) % 11 FROM generate_series(1, 30) g;
-- (1) Symptom: RPR window -> no "Run Condition:" line, filter degraded to
-- a Filter above the WindowAgg (full-partition NFA matching forced).
EXPLAIN (COSTS OFF)
SELECT * FROM (
SELECT ts, row_number() OVER w AS rn FROM z_Z4_t
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
AFTER MATCH SKIP TO NEXT ROW INITIAL
PATTERN (a+) DEFINE a AS price > PREV(price))) s
WHERE rn <= 10;
-- (2) Non-RPR baseline: "Run Condition: (row_number() OVER w <= 10)" present.
EXPLAIN (COSTS OFF)
SELECT * FROM (
SELECT ts, row_number() OVER w AS rn FROM z_Z4_t
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)) s
WHERE rn <= 10;
-- (3) Why blanket unblock is unsafe: count(*) claims MONOTONICFUNC_DECREASING
-- from frameOptions alone (int8.c), but over RPR reduced frames cnt is
-- match-dependent and non-monotonic (drops to 0 on unmatched rows, rises
-- again on later matches). Under an unblocked build this query would get
-- a run condition and silently lose all matches after the first unmatched
-- row; on the current tree it returns every cnt>0 row (correct).
SELECT * FROM (
SELECT ts, price, count(*) OVER w AS cnt FROM z_Z4_t
WINDOW w AS (ORDER BY ts ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
PATTERN (b+) DEFINE b AS price > PREV(price))) s
WHERE cnt > 0 ORDER BY ts;-- BASE and HEAD identical; tail of the shared output 7 | 6 | 2 10 | 7 | 1 12 | 4 | 2 15 | 5 | 2 18 | 6 | 2 21 | 7 | 1 23 | 4 | 2 26 | 5 | 2 29 | 6 | 2 (11 rows)
runCondition likewise unfinalized but structurally masked