Skip to content

SQL Dialects

OrionBelt compiles semantic queries into SQL for eight database dialects. Each dialect has its own identifier quoting, function names, and SQL syntax. The plugin architecture allows adding new dialects without modifying the core compiler.

Supported Dialects

Dialect Identifier Description
BigQuery bigquery Google Cloud analytics warehouse with QUALIFY, STRUCT/ARRAY, semi-structured types
ClickHouse clickhouse Column-oriented OLAP with custom date/aggregation functions
Databricks SQL databricks Spark SQL semantics with backtick identifiers
Dremio dremio Data lakehouse with reduced function surface
DuckDB / MotherDuck duckdb Embedded analytics engine with PostgreSQL-like syntax, QUALIFY, UNION ALL BY NAME
MySQL mysql MySQL 8.0+ with backtick identifiers, DATE_FORMAT time grains, GROUP_CONCAT
PostgreSQL postgres Standard PostgreSQL with strict GROUP BY
Snowflake snowflake Cloud data warehouse with QUALIFY, semi-structured types

Connecting to MotherDuck

MotherDuck is DuckDB served remotely, so it needs no separate dialect — the duckdb codegen is what runs, and every DuckDB capability below applies unchanged. What differs is only the connection:

DUCKDB_DATABASE=md:my_database
MOTHERDUCK_ACCESS_TOKEN=<token>

Create the token at app.motherduck.com under your organization name → Settings → Create token. The default Read/Write token is the right choice for most setups — a read-scaling token cannot create tables, so it cannot load fixtures or seed a database.

A Read Scaling token suits a query-only deployment: it grants SELECT and connects against the read replica pool, which is the better shape for many concurrent readers. It is not required in order to connect read-only — OrionBelt opens read_only=True connections happily with a normal read/write token.

MOTHERDUCK_ACCESS_TOKEN is the canonical spelling, matching the other vendor credentials. MOTHERDUCK_TOKEN and the lowercase motherduck_token that MotherDuck's own tooling uses are both accepted as aliases, so an environment already set up for the MotherDuck CLI needs no changes. A token embedded directly in the database string — either motherduck_token= or read_scaling_token= — is left alone.

A md: database without a token is refused

OrionBelt raises MotherDuckTokenMissingError rather than connecting. This is deliberate. Without an explicit token the DuckDB extension falls back to interactive browser authentication, which on a server does not fail — it hangs, so it surfaces as a stuck worker rather than a configuration error. Refusing up front also prevents an ambient motherduck_token in the environment from silently connecting to a different account than the one you configured.

read_only stays enabled for MotherDuck as it is for a local file: OrionBelt only reads, and a md: connection accepts it.

Capabilities Matrix

Each dialect declares capability flags that the compiler uses to choose SQL generation strategies.

Capability BigQuery ClickHouse Databricks Dremio DuckDB MySQL Postgres Snowflake
supports_cte Yes Yes Yes Yes Yes Yes Yes Yes
supports_qualify Yes No No No Yes No No Yes
supports_arrays Yes Yes Yes No Yes No Yes Yes
supports_window_filters Yes No No No Yes No No Yes
supports_ilike No Yes No No Yes No Yes Yes
supports_time_travel No No No No No No No Yes
supports_semi_structured Yes No No No No No No Yes
supports_union_all_by_name No No No No Yes No No Yes
supports_group_by_all Yes Yes Yes No Yes No No Yes

Identifier Quoting

Dialect Style Example
BigQuery Backticks `column_name`
ClickHouse Double quotes "column_name"
Databricks Backticks `column_name`
Dremio Double quotes "column_name"
DuckDB Double quotes "column_name"
MySQL Backticks `column_name`
Postgres Double quotes "column_name"
Snowflake Double quotes "column_name"

Time Grain Functions

The timeGrain is rendered differently per dialect:

DATE_TRUNC(`order_date`, 'month')
DATE_TRUNC(`order_date`, 'year')
DATE_TRUNC(`order_date`, 'quarter')
DATE_TRUNC(`order_date`, 'ISOWEEK')   -- week
toStartOfMonth("order_date")
toStartOfYear("order_date")
toStartOfQuarter("order_date")
toMonday("order_date")        -- week
toDate("order_date")          -- day
toStartOfHour("order_date")
toStartOfMinute("order_date")
toStartOfSecond("order_date")
date_trunc('month', `order_date`)
date_trunc('year', `order_date`)
DATE_TRUNC('month', "order_date")
DATE_TRUNC('year', "order_date")
date_trunc('month', "order_date")
date_trunc('year', "order_date")
date_trunc('quarter', "order_date")
DATE_FORMAT(`order_date`, '%Y-%m-01')           -- month
DATE_FORMAT(`order_date`, '%Y-01-01')           -- year
DATE_ADD(MAKEDATE(YEAR(`order_date`), 1),
  INTERVAL (QUARTER(`order_date`) - 1) * 3 MONTH)  -- quarter
DATE_FORMAT(`order_date`, '%Y-%u')              -- week (ISO)
DATE_FORMAT(`order_date`, '%Y-%m-%d')           -- day
date_trunc('month', "order_date")
date_trunc('year', "order_date")
date_trunc('quarter', "order_date")
DATE_TRUNC('month', "order_date")
DATE_TRUNC('year', "order_date")
DATE_TRUNC('quarter', "order_date")

String Contains

The contains filter operator is rendered per dialect:

LOWER(`column`) LIKE '%' || LOWER('search') || '%'
"column" ILIKE '%' || 'search' || '%'
lower(`column`) LIKE '%' || lower('search') || '%'
LOWER("column") LIKE '%' || LOWER('search') || '%'
"column" ILIKE '%' || 'search' || '%'
`column` LIKE CONCAT('%', 'search', '%')

MySQL string comparisons are case-insensitive by default with utf8mb4_general_ci collation, so LIKE is sufficient (no ILIKE needed).

"column" ILIKE '%' || 'search' || '%'
CONTAINS("column", 'search')

CAST Handling

OrionBelt automatically wraps aggregate expressions with CAST based on resolved OBML data types. Each dialect maps OBML types to its native SQL types:

OBML Type Postgres Snowflake ClickHouse BigQuery MySQL Databricks DuckDB Dremio
decimal(p, s) NUMERIC(p, s) NUMBER(p, s) Decimal(p, s) NUMERIC(p, s) DECIMAL(p, s) DECIMAL(p, s) DECIMAL(p, s) DECIMAL(p, s)
bigint BIGINT NUMBER(38, 0) Int64 INT64 BIGINT BIGINT BIGINT BIGINT
integer INTEGER NUMBER(38, 0) Int32 INT64 INTEGER INT INTEGER INTEGER
double DOUBLE PRECISION FLOAT Float64 FLOAT64 DOUBLE DOUBLE DOUBLE DOUBLE
string TEXT VARCHAR String STRING VARCHAR(65535) STRING VARCHAR VARCHAR
boolean BOOLEAN BOOLEAN Bool BOOL TINYINT(1) BOOLEAN BOOLEAN BOOLEAN

Maximum decimal precision (values are clamped):

Dialect Max Precision
Postgres 131,072
ClickHouse 76
MySQL 65
Snowflake, DuckDB, Databricks, Dremio 38
BigQuery 38

ClickHouse: decimal division keeps the operand's scale

Dividing two decimals on ClickHouse produces a result at the operands' scale rather than widening it, so a ratio built from decimal(18, 2) columns is truncated to two places:

SELECT toDecimal64(4254, 2) / toDecimal64(10000, 2)   -- 0.42
SELECT toDecimal64(4254, 4) / toDecimal64(10000, 4)   -- 0.4254

Every other supported dialect widens the scale for you. If a metric divides decimal measures and you need the precision, declare the metric's dataType with the scale you want, or give the measures a float resultType so the division happens in floating point. This is a ClickHouse arithmetic rule, not something OrionBelt applies -- it is documented here because a ratio that reads 0.42 on one engine and 0.4254 on another looks like a compiler bug and is not.

CAST(expr AS INTEGER)
CAST(expr AS VARCHAR)
CAST(expr AS DATE)

BigQuery uses its own type names (INT64, FLOAT64, STRING, BOOL) but standard CAST syntax.

ClickHouse uses native conversion functions:

toInt64(expr)      -- int / integer
toFloat64(expr)    -- float / double
toString(expr)     -- string / varchar
toDate(expr)       -- date
-- Other types fall back to CAST
CAST(expr AS DateTime)

Aggregation Functions

Most aggregations (SUM, COUNT, AVG, MIN, MAX) compile identically across dialects. The following aggregations require dialect-specific rendering:

ANY_VALUE

Dialect SQL
BigQuery ANY_VALUE(col)
ClickHouse any(col)
Databricks ANY_VALUE(col)
Dremio ANY_VALUE(col)
DuckDB ANY_VALUE(col)
MySQL ANY_VALUE(col)
Postgres ANY_VALUE(col)
Snowflake ANY_VALUE(col)

MEDIAN

median is the exact, continuous median on every dialect: the middle value of an odd count, and the mean of the two middle values of an even one (6 for 1, 2, 10, 20). NULLs are ignored, and a group of NULLs is NULL. A reaggregate metric with aggregation: median renders its second stage the same way, over the stage-1 values.

Dialect SQL
BigQuery the middle values lower and upper of ARRAY_AGG(col IGNORE NULLS ORDER BY col), at SAFE_OFFSET(DIV(COUNT(col) - 1, 2)) and SAFE_OFFSET(DIV(COUNT(col), 2)): (lower + upper) / NUMERIC '2' when their signs differ, lower + (upper - lower) / NUMERIC '2' when both are positive, upper - (upper - lower) / NUMERIC '2' when both are negative
ClickHouse if(count(col) = 0, NULL, quantileExactInclusive(0.5)(toFloat64(col)))
Databricks MEDIAN(col)
Dremio MEDIAN(CAST(col AS DOUBLE))
DuckDB MEDIAN(CAST(col AS DOUBLE))
MySQL the middle one or two values of GROUP_CONCAT(col ORDER BY col SEPARATOR ','), picked with SUBSTRING_INDEX, read back as DOUBLE, each halved, then added
Postgres PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY col)
Snowflake MEDIAN(col)

Each engine's own shortcut answers something else, measured on all eight: BigQuery's APPROX_QUANTILES is approximate, Postgres' PERCENTILE_DISC takes the lower middle value, ClickHouse's median samples (and has no upper-case MEDIAN), DuckDB's MEDIAN of a DECIMAL rounds to the column's scale, and Dremio's loses small DECIMAL values (0.0 for 0.000000001). The result is a floating-point number on Postgres, DuckDB, ClickHouse, MySQL and Dremio, and computed as one on Snowflake and Databricks (Snowflake's MEDIAN returns a NUMBER, but 9007199254740992 for a single 9007199254740993, as PERCENTILE_CONT does). On BigQuery it is exact NUMERIC for an INT64 or NUMERIC column (rounded once to its scale), BIGNUMERIC for a BIGNUMERIC one and FLOAT64 for a FLOAT64 one, the same for a filtered or multi-fact measure. The midpoint adds two values only of opposite signs and subtracts them only of the same sign, so it never overflows the column's type, and it rounds once, away from zero, like the mean.

Dremio: a median over no rows returns no row

An ungrouped query whose median reads no rows (a filter removed them all) returns no row on Dremio, with every other column of it, where the other engines return one row of NULLs. This is the engine's behavior for MEDIAN and PERCENTILE_CONT alike. Grouped queries are unaffected: a group with no rows has no row anywhere.

Snowflake: a median of a wide NUMBER can be out of range

Snowflake's MEDIAN of a NUMBER(p, s) is a NUMBER(38, s + 3), so a value with more integer digits than 35 - s (26 for a NUMBER(38, 9)) fails with "Number out of representable range" rather than returning a median. PERCENTILE_CONT behaves the same.

PERCENTILE_CONT and PERCENTILE_DISC

percentile_cont interpolates between the two values around position p * (n - 1); percentile_disc is the first value whose cumulative share of the group reaches p. NULLs are ignored, and a group of NULLs is NULL. Measured at 0.00001, 0.1, 0.25, 0.3, 0.5, 0.75 and 0.9 of the values 1 to 10, every dialect gives the same answer. A reaggregate metric with either aggregation renders its second stage the same way.

Dialect percentile_cont percentile_disc
BigQuery lower and upper of ARRAY_AGG(col IGNORE NULLS ORDER BY col) around the position: lower at a whole position or between equal values, else a step up from lower when that is above zero and down from upper otherwise. Each value is split at TRUNC into a whole part, divided by 10^places exactly, and a fraction, whose step is the one rounding, so a tie goes away from zero as a mean's does. Of the same sign the whole parts' difference is taken first; of opposite signs each end's own whole part comes off first, so no sum runs past the type's range at any magnitude, a FLOAT64 near 1.8e308 included the ceil(p * n)-th value of the sorted array
ClickHouse if(count(col) = 0, NULL, quantileExactInclusive(p)(toFloat64(col))) if(count(col) = 0, NULL, arraySort(groupArray(col))[ceil(p * n)])
Databricks PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY col) get(array_sort(collect_list(col)), ceil(p * n) - 1)
Dremio PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY CAST(col AS DOUBLE)) PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col)
DuckDB PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY CAST(col AS DOUBLE)) PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col)
MySQL the values around the position in GROUP_CONCAT(col ORDER BY col SEPARATOR ','), read back as DOUBLE: lower * (1 - w) + upper * w, the weight divided before it multiplies (1e307 and 2e307 would overflow otherwise) the ceil(p * n)-th value, read back as DOUBLE
Postgres PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY col) PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col)
Snowflake PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY col) PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col)

The renderings that find the position themselves (BigQuery, ClickHouse's and Databricks' discrete one, MySQL) compute it in integers from the fraction as a ratio of a power of ten: as a double, 0.3 * 10 is 3.0000000000000004, whose ceiling would be the 4th value. That is why percentile takes at most 9 decimal places. ClickHouse's quantileExact and quantileExactLow are not PERCENTILE_DISC (2, 4 and 10 at 0.1, 0.3 and 0.9 of 1 to 10, where it is 1, 3 and 9), and Databricks' and Dremio's PERCENTILE_DISC return a DOUBLE, which rounds an integer past 2^53. Databricks reads the sorted array instead; Dremio indexes arrays only by a literal, so its discrete percentile stays a DOUBLE, as MySQL's is.

The continuous percentile has the median's types: a floating-point number on Postgres, DuckDB, ClickHouse, MySQL and Dremio, computed as one on Snowflake and Databricks, and exact on BigQuery (BIGNUMERIC over INT64 and NUMERIC values). The discrete one is a value of the column on Postgres, DuckDB, Snowflake, Databricks, ClickHouse and BigQuery. Dremio's empty-group and Snowflake's wide-NUMBER behavior above apply to PERCENTILE_CONT as well.

FIRST and LAST

first / last are the value at the least / greatest withinGroup key, skipping rows without a value or a key, a tie going to the least / greatest value. The vendors' own first / last (DuckDB, Databricks) and ClickHouse's any / anyLast take the first row met rather than an ordered one, and the ordered MAX_BY / arg_max break a tie on the key arbitrarily (Snowflake answered 5 of the tied 5 and 7), so every rendering orders by the (key, value) pair.

Dialect last (first mirrors it)
BigQuery ARRAY_AGG(IF(k IS NULL, NULL, v) IGNORE NULLS ORDER BY k DESC, v DESC LIMIT 1)[SAFE_OFFSET(0)]
ClickHouse if(countIf(...) = 0, NULL, argMaxIf(v, (k, v), isNotNull(k) AND isNotNull(v)))
Databricks max_by(v, CASE WHEN k IS NOT NULL AND v IS NOT NULL THEN struct(k, v) END)
Dremio Not supported: no MAX_BY, no ORDER BY inside ARRAY_AGG, and its ARRAY_AGG ... WITHIN GROUP returns an empty array for the group whose key is NULL, which would read as a NULL last
DuckDB arg_max(v, struct_pack(k := k, v := v)) FILTER (WHERE v IS NOT NULL AND k IS NOT NULL)
MySQL Not supported: GROUP_CONCAT would return a string whatever the column's type
Postgres (ARRAY_AGG(v ORDER BY k DESC, v DESC) FILTER (WHERE v IS NOT NULL AND k IS NOT NULL))[1]
Snowflake MAX_BY(v, IFF(k IS NULL OR v IS NULL, NULL, ARRAY_CONSTRUCT(k, v))) (arrays order as their elements: probed over numbers, dates, timestamps with and without a zone, strings and booleans)

MODE

Dialect SQL
BigQuery APPROX_TOP_COUNT(col, 1)[OFFSET(0)].value
ClickHouse topK(1)(col)[1]
Databricks MODE(col)
Dremio Not supported
DuckDB MODE(col)
MySQL Not supported
Postgres MODE() WITHIN GROUP (ORDER BY col)
Snowflake MODE(col)

LISTAGG

Dialect Base + DISTINCT + ORDER BY
BigQuery STRING_AGG(col, sep) STRING_AGG(DISTINCT col, sep) STRING_AGG(col, sep ORDER BY col)
ClickHouse arrayStringConcat(groupArray(col), sep) arrayStringConcat(groupUniqArray(col), sep) arrayStringConcat(arraySort(groupArray(col)), sep)
Databricks ARRAY_JOIN(COLLECT_LIST(col), sep) ARRAY_JOIN(COLLECT_SET(col), sep) ARRAY_JOIN(SORT_ARRAY(COLLECT_LIST(col)), sep)
Dremio LISTAGG(col, sep) LISTAGG(DISTINCT col, sep) LISTAGG(col, sep) WITHIN GROUP (ORDER BY col)
DuckDB STRING_AGG(col, sep) STRING_AGG(DISTINCT col, sep) STRING_AGG(col, sep ORDER BY col)
MySQL GROUP_CONCAT(col SEPARATOR sep) GROUP_CONCAT(DISTINCT col SEPARATOR sep) GROUP_CONCAT(col ORDER BY col SEPARATOR sep)
Postgres STRING_AGG(col, sep) STRING_AGG(DISTINCT col, sep) STRING_AGG(col, sep ORDER BY col)
Snowflake LISTAGG(col, sep) LISTAGG(DISTINCT col, sep) LISTAGG(col, sep) WITHIN GROUP (ORDER BY col)

LISTAGG ordering limitations

ClickHouse and Databricks only support self-ordering (sorting by the aggregated column). Ordering by a different column raises an error at compile time.

MySQL GROUP_CONCAT limitations

MySQL's GROUP_CONCAT silently cuts its result at group_concat_max_len, 1024 bytes by default. A statement that uses it (a listagg or a median) carries /*+ SET_VAR(group_concat_max_len = 4294967295) */ on its outermost SELECT, which lifts the limit for that statement only; the hint has no effect anywhere else in the statement. Additionally, MySQL silently ignores ORDER BY when DISTINCT is also present in GROUP_CONCAT.

Total not supported

MEDIAN, MODE, LISTAGG, and ANY_VALUE do not support total: true because they cannot be meaningfully re-aggregated via window functions.

Dialect Plugin Architecture

Each dialect implements the abstract Dialect base class:

class Dialect(ABC):
    @property
    @abstractmethod
    def name(self) -> str: ...

    @property
    @abstractmethod
    def capabilities(self) -> DialectCapabilities: ...

    @abstractmethod
    def quote_identifier(self, name: str) -> str: ...

    @abstractmethod
    def render_time_grain(self, column: Expr, grain: TimeGrain) -> Expr: ...

    @abstractmethod
    def render_cast(self, expr: Expr, target_type: str) -> Expr: ...

    def render_string_contains(self, column: Expr, pattern: Expr) -> Expr: ...

    def compile(self, ast: Select) -> str: ...

Dialects register themselves via the @DialectRegistry.register decorator:

@DialectRegistry.register
class PostgresDialect(Dialect):
    @property
    def name(self) -> str:
        return "postgres"
    ...

The registry provides lookup by name:

from orionbelt.dialect.registry import DialectRegistry

dialect = DialectRegistry.get("snowflake")
sql = dialect.compile(ast)

Adding a New Dialect

  1. Create src/orionbelt/dialect/my_dialect.py
  2. Subclass Dialect and implement all abstract methods
  3. Decorate with @DialectRegistry.register
  4. The dialect is automatically available via DialectRegistry.get("my_dialect")

Querying Dialect Info via API

Date Spine Generation

Period-over-period metrics require generating a date series (spine). Each dialect uses a different technique:

Dialect Technique
Postgres generate_series(min, max, INTERVAL)
DuckDB generate_series(min, max, INTERVAL)
Snowflake TABLE(GENERATOR(ROWCOUNT => ...)) + DATEADD
BigQuery UNNEST(GENERATE_DATE_ARRAY(min, max, INTERVAL))
Databricks EXPLODE(SEQUENCE(min, max, INTERVAL))
MySQL Recursive CTE: WITH RECURSIVE dates AS (...)
ClickHouse arrayJoin(range(...)) + date arithmetic
Dremio Recursive CTE: WITH RECURSIVE dates AS (...)

For details, see the Period-over-Period Metrics guide.

Querying Available Dialects

curl http://127.0.0.1:8000/v1/dialects
{
  "dialects": [
    {
      "name": "bigquery",
      "capabilities": {
        "supports_cte": true,
        "supports_qualify": true,
        "supports_arrays": true,
        "supports_window_filters": true,
        "supports_ilike": false,
        "supports_time_travel": false,
        "supports_semi_structured": true,
        "supports_union_all_by_name": false,
        "supports_group_by_all": true
      },
      "supported_aggregations": [
        "any_value", "avg", "corr", "count", "count_distinct", "covar_pop",
        "covar_samp", "listagg", "max", "median", "min", "mode", "stddev",
        "stddev_pop", "sum", "var_pop", "variance"
      ],
      "supported_functions": [
        "abs", "ceil", "coalesce", "concat", "div", "ends_with", "exp", "floor",
        "greatest", "least", "length", "ln", "log", "lower", "lpad", "ltrim",
        "mod", "nullif", "position", "power", "replace", "round", "rpad",
        "rtrim", "sign", "split_part", "sqrt", "starts_with", "substring",
        "trim", "trunc", "upper"
      ]
    },
    ...
  ]
}

capabilities are structural SQL features. The two lists are the vocabulary: every OBML aggregation: value this dialect can compute, and every entry of the portable function catalog it can render. Both are stated positively, so answering "may I use median on this warehouse?" needs no second call. Anything absent is refused at compile time with a 422 rather than emitted and failed at the database — MySQL, for instance, does not list mode.