Skip to content

[IOTDB-17798] Implement table model NEXT fill#17810

Open
DaZuiZui wants to merge 1 commit into
apache:masterfrom
DaZuiZui:fix/next-fill
Open

[IOTDB-17798] Implement table model NEXT fill#17810
DaZuiZui wants to merge 1 commit into
apache:masterfrom
DaZuiZui:fix/next-fill

Conversation

@DaZuiZui
Copy link
Copy Markdown
Contributor

@DaZuiZui DaZuiZui commented Jun 1, 2026

Description

Add table-model NEXT fill syntax and planning

This PR implements FILL METHOD NEXT for the table model.

NEXT fills a null value with the nearest non-null value that appears later in the current operator input order. It directly copies the future value, does not interpolate, and does not search by timestamp order. If no next value exists, the value remains null.

The implementation supports:

  • plain FILL METHOD NEXT
  • TIME_BOUND
  • TIME_COLUMN
  • FILL_GROUP
  • numeric, boolean, binary/text/string/blob/object, date, and timestamp columns

Plain NEXT does not require a helper/time column. A helper/time column is only required or inferred when TIME_BOUND or FILL_GROUP needs it.

FillPolicy.NEXT is appended after existing enum values to keep existing byte/ordinal compatibility, and TABLE_NEXT_FILL_NODE uses a new plan node type id.

Add NEXT fill plan node and optimizer support

This PR adds NextFillNode and wires it through serde, plan visitors, plan graph printing, pruning, symbol rewrite, analyzer, and planner logic.

For TIME_BOUND or FILL_GROUP, the analyzer resolves the required helper/time column similarly to PREVIOUS. For grouped fill, the planner reuses the existing fill-group sorting and grouping-symbol path.

PushLimitOffsetIntoTableScan treats NEXT like LINEAR and avoids pushing down LIMIT/OFFSET, because NEXT fill may need future rows that would otherwise be removed before filling.

Add NEXT fill execution path

This PR adds table NEXT fill operators and the generated fill implementation.

The execution layer reuses AbstractLinearFillOperator for cross-TsBlock look-ahead caching and grouped slicing. The NEXT fill algorithm scans the current value column from right to left, uses the nearest non-null candidate in the current block when available, and otherwise uses the first non-null candidate found in later cached blocks.

For grouped fill, group boundaries reset the next-value state so values are not filled across groups.

Tests

This PR adds and updates tests for:

  • parsing FILL METHOD NEXT
  • NextFillNode serde
  • plain NEXT fill
  • NEXT with TIME_BOUND
  • NEXT with TIME_COLUMN
  • NEXT with FILL_GROUP
  • NEXT with TIME_BOUND + TIME_COLUMN + FILL_GROUP
  • cross-TsBlock fill behavior
  • group boundary isolation
  • trailing null values
  • helper/time column null handling
  • boolean and binary/text-like values
  • current input order semantics, including descending order and sorted subqueries
  • LIMIT behavior without losing future next values

Closes #17798


This PR has:

  • been self-reviewed.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR
  • RelationalSql.g4
  • FillPolicy
  • Fill
  • AstBuilder
  • StatementAnalyzer
  • Analysis.NextFillAnalysis
  • QueryPlanner
  • NextFillNode
  • PlanNodeType
  • CommonPlanNodeDeserializer
  • ICoreQueryPlanVisitor
  • PlanGraphPrinter
  • PruneFillColumns
  • PushLimitOffsetIntoTableScan
  • UnaliasSymbolReferences
  • CommonOperatorUtils
  • TableOperatorGenerator
  • TableNextFillOperator
  • TableNextFillWithGroupOperator
  • fill/next/NextFill
  • nextFill.ftl
  • FillStatementTest
  • NextFillNodeSerdeTest
  • NextFillTest
  • IoTDBFillTableIT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][Table Model] Support NEXT (backward) value fill in the FILL clause

1 participant