-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Open
Description
Expected Behavior or Use Case
Materialized views should support configurable staleness tolerance. Users can specify:
- Staleness tracking mode:
-BY_REFRESH: Staleness is measured by time since last refresh
-BY_BASE_TABLE: Staleness is measured by time since base tables were modified - Staleness window: A duration (e.g., 1h, 30m) that defines acceptable staleness
- Stale read behavior: What happens when staleness exceeds the window:
-FAIL: Query fails with an error
-USE_VIEW_QUERY: Fall back to executing the view query instead of reading the data table
When querying a materialized view:
- If fully materialized (no base table changes), use the data table
- If stale but within the staleness window, use the data table
- If stale beyond the staleness window, apply the configured stale read behavior
- If no staleness config is set, use the session property
materialized_view_stale_read_behavioras the default
Presto Component, Service, or Connector
presto-spi: Core interfaces (MaterializedViewStatus,MaterializedViewDefinition,MaterializedViewStalenessConfig)presto-main-base: Rewrite rule (MaterializedViewRewrite), session propertiespresto-iceberg: Iceberg connector implementationpresto-memory: Memory connector implementation
Possible Implementation
- Add
MaterializedViewStalenessConfigclass with three fields:
-staleReadBehavior(enum:FAIL,USE_VIEW_QUERY)
-stalenessTracking(enum:BY_REFRESH,BY_BASE_TABLE)
-stalenessWindow(Duration) - Add runtime timestamp fields to
MaterializedViewStatus:
-lastRefreshTime: When the MV was last refreshed
-lastBaseTableModificationTime: When base tables were last modified - Update
MaterializedViewRewriterule to:
- Check if view is fully materialized first
- If stale, check staleness config and compare timestamps against the window
- Apply the configured behavior when staleness exceeds tolerance - Add session property
materialized_view_stale_read_behaviorfor default behavior when no staleness config is set on the MV - Update connectors to populate timestamps in
getMaterializedViewStatus()
Example Screenshots (if appropriate):
Context
The desired behavior of how to handle staleness may depend on the view itself and how it is intended to be used. By introducing table properties, we give users flexibility to configure it on a per table basis.
Metadata
Metadata
Assignees
Type
Projects
Status
🆕 Unprioritized