Skip to main content

Transforming your data

With Sundial, you transform raw data into clean, structured datasets using modular SQL models. Define each step of your transformation logic in SQL, and Sundial handles dependencies, execution order, and testing automatically. Built for reliability and collaboration, Sundial lets you iterate quickly, document clearly, and scale confidently—turning messy data into trusted insights.

Here’s how you transform your data with Sundial in these steps:

  1. Add a Source - Begin by registering your raw data sources—whether they live in a data warehouse, data lake, or external system. Sundial allows you to define these sources just like any other table, so that its easy to understand how it’s structured, and how it’s used downstream.
  2. Declare Transformations Using SQL-Based Views - Next, write modular SQL models to express your transformation logic. These models are organized as views, layered to incrementally clean, join, filter, and enrich your raw data. Each model is readable, reusable, and version-controlled, making it easy for teams to collaborate and build on each other’s work.
  3. Materialize the transformations - When you're ready, Sundial compiles your models, resolves dependencies, and executes them in the correct order. You can materialize outputs as views or materialized views, depending on your performance and storage needs. Along the way, Sundial enforces data tests and generates documentation automatically—ensuring quality, transparency, and trust in every step of your pipeline.
  4. Set Up Scheduled Triggers - Automate your pipeline by configuring scheduled runs. Whether you need hourly refreshes or daily batch updates, Sundial’s scheduler ensures your data is always up-to-date. Integrate with monitoring tools to get alerts on failures, anomalies, or freshness issues.

In addition to transforming data through models and schedules, Sundial offers powerful capabilities to ensure your pipelines are robust, transparent, and easy to maintain:

  • Data Quality Testing - Add SQL-based or built-in tests (like uniqueness, null checks, row counts) to your models. These tests run automatically on a schedule to catch issues before they impact downstream analytics.
  • Visual Lineage - Trace the lineage of every table and column across your pipeline. Understand how data flows from source to insight, making it easier to debug issues, onboard teammates, and audit changes.
  • Freshness & Staleness Checks - Monitor how up-to-date your data is with built-in freshness tracking. Quickly identify which tables are stale or overdue for a refresh.
  • Schema Browsing - Explore the full schema of any table in your project—columns, data types, and descriptions—directly from the interface.
  • Pending Change Preview - Before running your next materialization, see exactly which changes are queued—new models, edits, renamed fields, so you can review and validate safely.
  • Backfill Configuration - Easily set whether a table should be fully rebuilt or partially refreshed during materialization. Track the backfill status of each model to ensure consistent and performant updates.