Skip to main content

Materialization

By default, SQL Views are computed on demand at query time. Materialization lets you pre-compute and store the results in Sundial's S3 storage so downstream queries run faster. When you materialize a view, Sundial saves the results so they can be quickly accessed later, rather than having to recalculate them each time.

How it works​

When you materialize a SQL View, Sundial executes the query, resolves all upstream dependencies in the correct order, and writes the results to storage. Subsequent queries against that view read from storage instead of recomputing on the fly.

Materialized upstream views are read directly from storage, while non-materialized ones are recomputed as part of the refresh.

Loading data​

When loading data into a materialized view, you have two options:

Incremental loading​

  • Only loads new data since the last update, based on a timestamp column you specify.
  • Best for large, append-heavy views where reprocessing everything would be too slow.
  • Materialized upstream views are read from storage, while non-materialized ones need a full refresh.

Configuration:

  • Timestamp column — The column used to identify new rows (e.g. created_at, event_timestamp).
  • First timestamp — The earliest date to start processing from.
  • Lookback window — How far back to reprocess on each incremental run, to handle late-arriving data.
  • Partition columns — Optional, for query performance.

Full refresh​

  • Reloads all data from scratch.
  • Best for smaller views or when you need to reprocess everything.
  • Recomputes all upstream views as well.

Configuration:

  • Partition columns — Optional, to improve query performance on frequently filtered fields.

Saving data​

When saving materialized data, Sundial supports two methods:

  1. Full Refresh — Completely replaces the existing table with new data.
  2. Delete + Insert — Removes data for a specific time range and adds new data for that range.

Performance optimization​

You can improve query performance by partitioning tables based on frequently filtered columns. For example, you might partition by date if you often query specific time ranges. These partition settings can be configured in your table settings.

Enabling materialization​

You can enable materialization when creating or editing a SQL View:

  1. Open the view in the Data Catalog.
  2. Navigate to the materialization configuration section.
  3. Toggle materialization on.
  4. Select Full refresh or Timestamp-based refresh.
  5. Fill in the required fields for your chosen mode.
  6. Save.

Materialization config

Manual refresh​

Once materialization is enabled, you can trigger a refresh at any time.

Single view — Click the refresh button on any materialized view in the Data Catalog. Sundial will process that view and all its upstream dependencies.

Manual refresh

Next steps​

Still have questions?
Write to support@sundial.so