SQL Dialects¶
OrionBelt compiles semantic queries into SQL for eight database dialects. Each dialect has its own identifier quoting, function names, and SQL syntax. The plugin architecture allows adding new dialects without modifying the core compiler.
Supported Dialects¶
| Dialect | Identifier | Description |
|---|---|---|
| BigQuery | bigquery |
Google Cloud analytics warehouse with QUALIFY, STRUCT/ARRAY, semi-structured types |
| ClickHouse | clickhouse |
Column-oriented OLAP with custom date/aggregation functions |
| Databricks SQL | databricks |
Spark SQL semantics with backtick identifiers |
| Dremio | dremio |
Data lakehouse with reduced function surface |
| DuckDB / MotherDuck | duckdb |
Embedded analytics engine with PostgreSQL-like syntax, QUALIFY, UNION ALL BY NAME |
| MySQL | mysql |
MySQL 8.0+ with backtick identifiers, DATE_FORMAT time grains, GROUP_CONCAT |
| PostgreSQL | postgres |
Standard PostgreSQL with strict GROUP BY |
| Snowflake | snowflake |
Cloud data warehouse with QUALIFY, semi-structured types |
Connecting to MotherDuck¶
MotherDuck is DuckDB served remotely, so it needs no separate dialect — the
duckdb codegen is what runs, and every DuckDB capability below applies
unchanged. What differs is only the connection:
Create the token at app.motherduck.com under your organization name → Settings → Create token. The default Read/Write token is the right choice for most setups — a read-scaling token cannot create tables, so it cannot load fixtures or seed a database.
A Read Scaling token suits a query-only deployment: it grants SELECT
and connects against the read replica pool, which is the better shape for
many concurrent readers. It is not required in order to connect read-only —
OrionBelt opens read_only=True connections happily with a normal
read/write token.
MOTHERDUCK_ACCESS_TOKEN is the canonical spelling, matching the other
vendor credentials. MOTHERDUCK_TOKEN and the lowercase motherduck_token
that MotherDuck's own tooling uses are both accepted as aliases, so an
environment already set up for the MotherDuck CLI needs no changes. A token
embedded directly in the database string — either motherduck_token= or
read_scaling_token= — is left alone.
A md: database without a token is refused
OrionBelt raises MotherDuckTokenMissingError rather than connecting.
This is deliberate. Without an explicit token the DuckDB extension falls
back to interactive browser authentication, which on a server does not
fail — it hangs, so it surfaces as a stuck worker rather than a
configuration error. Refusing up front also prevents an ambient
motherduck_token in the environment from silently connecting to a
different account than the one you configured.
read_only stays enabled for MotherDuck as it is for a local file: OrionBelt
only reads, and a md: connection accepts it.
Capabilities Matrix¶
Each dialect declares capability flags that the compiler uses to choose SQL generation strategies.
| Capability | BigQuery | ClickHouse | Databricks | Dremio | DuckDB | MySQL | Postgres | Snowflake |
|---|---|---|---|---|---|---|---|---|
supports_cte |
Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
supports_qualify |
Yes | No | No | No | Yes | No | No | Yes |
supports_arrays |
Yes | Yes | Yes | No | Yes | No | Yes | Yes |
supports_window_filters |
Yes | No | No | No | Yes | No | No | Yes |
supports_ilike |
No | Yes | No | No | Yes | No | Yes | Yes |
supports_time_travel |
No | No | No | No | No | No | No | Yes |
supports_semi_structured |
Yes | No | No | No | No | No | No | Yes |
supports_union_all_by_name |
No | No | No | No | Yes | No | No | Yes |
supports_group_by_all |
Yes | Yes | Yes | No | Yes | No | No | Yes |
Identifier Quoting¶
| Dialect | Style | Example |
|---|---|---|
| BigQuery | Backticks | `column_name` |
| ClickHouse | Double quotes | "column_name" |
| Databricks | Backticks | `column_name` |
| Dremio | Double quotes | "column_name" |
| DuckDB | Double quotes | "column_name" |
| MySQL | Backticks | `column_name` |
| Postgres | Double quotes | "column_name" |
| Snowflake | Double quotes | "column_name" |
Time Grain Functions¶
The timeGrain is rendered differently per dialect:
String Contains¶
The contains filter operator is rendered per dialect:
MySQL string comparisons are case-insensitive by default with utf8mb4_general_ci collation, so LIKE is sufficient (no ILIKE needed).
CAST Handling¶
OrionBelt automatically wraps aggregate expressions with CAST based on resolved OBML data types. Each dialect maps OBML types to its native SQL types:
| OBML Type | Postgres | Snowflake | ClickHouse | BigQuery | MySQL | Databricks | DuckDB | Dremio |
|---|---|---|---|---|---|---|---|---|
decimal(p, s) |
NUMERIC(p, s) |
NUMBER(p, s) |
Decimal(p, s) |
NUMERIC(p, s) |
DECIMAL(p, s) |
DECIMAL(p, s) |
DECIMAL(p, s) |
DECIMAL(p, s) |
bigint |
BIGINT |
NUMBER(38, 0) |
Int64 |
INT64 |
BIGINT |
BIGINT |
BIGINT |
BIGINT |
integer |
INTEGER |
NUMBER(38, 0) |
Int32 |
INT64 |
INTEGER |
INT |
INTEGER |
INTEGER |
double |
DOUBLE PRECISION |
FLOAT |
Float64 |
FLOAT64 |
DOUBLE |
DOUBLE |
DOUBLE |
DOUBLE |
string |
TEXT |
VARCHAR |
String |
STRING |
VARCHAR(65535) |
STRING |
VARCHAR |
VARCHAR |
boolean |
BOOLEAN |
BOOLEAN |
Bool |
BOOL |
TINYINT(1) |
BOOLEAN |
BOOLEAN |
BOOLEAN |
Maximum decimal precision (values are clamped):
| Dialect | Max Precision |
|---|---|
| Postgres | 131,072 |
| ClickHouse | 76 |
| MySQL | 65 |
| Snowflake, DuckDB, Databricks, Dremio | 38 |
| BigQuery | 38 |
ClickHouse: decimal division keeps the operand's scale
Dividing two decimals on ClickHouse produces a result at the operands'
scale rather than widening it, so a ratio built from decimal(18, 2)
columns is truncated to two places:
SELECT toDecimal64(4254, 2) / toDecimal64(10000, 2) -- 0.42
SELECT toDecimal64(4254, 4) / toDecimal64(10000, 4) -- 0.4254
Every other supported dialect widens the scale for you. If a metric divides
decimal measures and you need the precision, declare the metric's
dataType with the scale you want, or give the measures a float
resultType so the division happens in floating point. This is a
ClickHouse arithmetic rule, not something OrionBelt applies -- it is
documented here because a ratio that reads 0.42 on one engine and
0.4254 on another looks like a compiler bug and is not.
BigQuery uses its own type names (INT64, FLOAT64, STRING, BOOL) but standard CAST syntax.
Aggregation Functions¶
Most aggregations (SUM, COUNT, AVG, MIN, MAX) compile identically across dialects. The following aggregations require dialect-specific rendering:
ANY_VALUE¶
| Dialect | SQL |
|---|---|
| BigQuery | ANY_VALUE(col) |
| ClickHouse | any(col) |
| Databricks | ANY_VALUE(col) |
| Dremio | ANY_VALUE(col) |
| DuckDB | ANY_VALUE(col) |
| MySQL | ANY_VALUE(col) |
| Postgres | ANY_VALUE(col) |
| Snowflake | ANY_VALUE(col) |
MEDIAN¶
median is the exact, continuous median on every dialect: the middle value of an odd count, and the mean of the two middle values of an even one (6 for 1, 2, 10, 20). NULLs are ignored, and a group of NULLs is NULL. A reaggregate metric with aggregation: median renders its second stage the same way, over the stage-1 values.
| Dialect | SQL |
|---|---|
| BigQuery | the middle values lower and upper of ARRAY_AGG(col IGNORE NULLS ORDER BY col), at SAFE_OFFSET(DIV(COUNT(col) - 1, 2)) and SAFE_OFFSET(DIV(COUNT(col), 2)): (lower + upper) / NUMERIC '2' when their signs differ, lower + (upper - lower) / NUMERIC '2' when both are positive, upper - (upper - lower) / NUMERIC '2' when both are negative |
| ClickHouse | if(count(col) = 0, NULL, quantileExactInclusive(0.5)(toFloat64(col))) |
| Databricks | MEDIAN(col) |
| Dremio | MEDIAN(CAST(col AS DOUBLE)) |
| DuckDB | MEDIAN(CAST(col AS DOUBLE)) |
| MySQL | the middle one or two values of GROUP_CONCAT(col ORDER BY col SEPARATOR ','), picked with SUBSTRING_INDEX, read back as DOUBLE, each halved, then added |
| Postgres | PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY col) |
| Snowflake | MEDIAN(col) |
Each engine's own shortcut answers something else, measured on all eight: BigQuery's APPROX_QUANTILES is approximate, Postgres' PERCENTILE_DISC takes the lower middle value, ClickHouse's median samples (and has no upper-case MEDIAN), DuckDB's MEDIAN of a DECIMAL rounds to the column's scale, and Dremio's loses small DECIMAL values (0.0 for 0.000000001). The result is a floating-point number on Postgres, DuckDB, ClickHouse, MySQL and Dremio, and computed as one on Snowflake and Databricks (Snowflake's MEDIAN returns a NUMBER, but 9007199254740992 for a single 9007199254740993, as PERCENTILE_CONT does). On BigQuery it is exact NUMERIC for an INT64 or NUMERIC column (rounded once to its scale), BIGNUMERIC for a BIGNUMERIC one and FLOAT64 for a FLOAT64 one, the same for a filtered or multi-fact measure. The midpoint adds two values only of opposite signs and subtracts them only of the same sign, so it never overflows the column's type, and it rounds once, away from zero, like the mean.
Dremio: a median over no rows returns no row
An ungrouped query whose median reads no rows (a filter removed them all) returns no row on Dremio, with every other column of it, where the other engines return one row of NULLs. This is the engine's behavior for MEDIAN and PERCENTILE_CONT alike. Grouped queries are unaffected: a group with no rows has no row anywhere.
Snowflake: a median of a wide NUMBER can be out of range
Snowflake's MEDIAN of a NUMBER(p, s) is a NUMBER(38, s + 3), so a value with more integer digits than 35 - s (26 for a NUMBER(38, 9)) fails with "Number out of representable range" rather than returning a median. PERCENTILE_CONT behaves the same.
PERCENTILE_CONT and PERCENTILE_DISC¶
percentile_cont interpolates between the two values around position p * (n - 1); percentile_disc is the first value whose cumulative share of the group reaches p. NULLs are ignored, and a group of NULLs is NULL. Measured at 0.00001, 0.1, 0.25, 0.3, 0.5, 0.75 and 0.9 of the values 1 to 10, every dialect gives the same answer. A reaggregate metric with either aggregation renders its second stage the same way.
| Dialect | percentile_cont |
percentile_disc |
|---|---|---|
| BigQuery | lower and upper of ARRAY_AGG(col IGNORE NULLS ORDER BY col) around the position: lower at a whole position or between equal values, else a step up from lower when that is above zero and down from upper otherwise. Each value is split at TRUNC into a whole part, divided by 10^places exactly, and a fraction, whose step is the one rounding, so a tie goes away from zero as a mean's does. Of the same sign the whole parts' difference is taken first; of opposite signs each end's own whole part comes off first, so no sum runs past the type's range at any magnitude, a FLOAT64 near 1.8e308 included |
the ceil(p * n)-th value of the sorted array |
| ClickHouse | if(count(col) = 0, NULL, quantileExactInclusive(p)(toFloat64(col))) |
if(count(col) = 0, NULL, arraySort(groupArray(col))[ceil(p * n)]) |
| Databricks | PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY col) |
get(array_sort(collect_list(col)), ceil(p * n) - 1) |
| Dremio | PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY CAST(col AS DOUBLE)) |
PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col) |
| DuckDB | PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY CAST(col AS DOUBLE)) |
PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col) |
| MySQL | the values around the position in GROUP_CONCAT(col ORDER BY col SEPARATOR ','), read back as DOUBLE: lower * (1 - w) + upper * w, the weight divided before it multiplies (1e307 and 2e307 would overflow otherwise) |
the ceil(p * n)-th value, read back as DOUBLE |
| Postgres | PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY col) |
PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col) |
| Snowflake | PERCENTILE_CONT(p) WITHIN GROUP (ORDER BY col) |
PERCENTILE_DISC(p) WITHIN GROUP (ORDER BY col) |
The renderings that find the position themselves (BigQuery, ClickHouse's and Databricks' discrete one, MySQL) compute it in integers from the fraction as a ratio of a power of ten: as a double, 0.3 * 10 is 3.0000000000000004, whose ceiling would be the 4th value. That is why percentile takes at most 9 decimal places. ClickHouse's quantileExact and quantileExactLow are not PERCENTILE_DISC (2, 4 and 10 at 0.1, 0.3 and 0.9 of 1 to 10, where it is 1, 3 and 9), and Databricks' and Dremio's PERCENTILE_DISC return a DOUBLE, which rounds an integer past 2^53. Databricks reads the sorted array instead; Dremio indexes arrays only by a literal, so its discrete percentile stays a DOUBLE, as MySQL's is.
The continuous percentile has the median's types: a floating-point number on Postgres, DuckDB, ClickHouse, MySQL and Dremio, computed as one on Snowflake and Databricks, and exact on BigQuery (BIGNUMERIC over INT64 and NUMERIC values). The discrete one is a value of the column on Postgres, DuckDB, Snowflake, Databricks, ClickHouse and BigQuery. Dremio's empty-group and Snowflake's wide-NUMBER behavior above apply to PERCENTILE_CONT as well.
FIRST and LAST¶
first / last are the value at the least / greatest withinGroup key, skipping rows without a value or a key, a tie going to the least / greatest value. The vendors' own first / last (DuckDB, Databricks) and ClickHouse's any / anyLast take the first row met rather than an ordered one, and the ordered MAX_BY / arg_max break a tie on the key arbitrarily (Snowflake answered 5 of the tied 5 and 7), so every rendering orders by the (key, value) pair.
| Dialect | last (first mirrors it) |
|---|---|
| BigQuery | ARRAY_AGG(IF(k IS NULL, NULL, v) IGNORE NULLS ORDER BY k DESC, v DESC LIMIT 1)[SAFE_OFFSET(0)] |
| ClickHouse | if(countIf(...) = 0, NULL, argMaxIf(v, (k, v), isNotNull(k) AND isNotNull(v))) |
| Databricks | max_by(v, CASE WHEN k IS NOT NULL AND v IS NOT NULL THEN struct(k, v) END) |
| Dremio | Not supported: no MAX_BY, no ORDER BY inside ARRAY_AGG, and its ARRAY_AGG ... WITHIN GROUP returns an empty array for the group whose key is NULL, which would read as a NULL last |
| DuckDB | arg_max(v, struct_pack(k := k, v := v)) FILTER (WHERE v IS NOT NULL AND k IS NOT NULL) |
| MySQL | Not supported: GROUP_CONCAT would return a string whatever the column's type |
| Postgres | (ARRAY_AGG(v ORDER BY k DESC, v DESC) FILTER (WHERE v IS NOT NULL AND k IS NOT NULL))[1] |
| Snowflake | MAX_BY(v, IFF(k IS NULL OR v IS NULL, NULL, ARRAY_CONSTRUCT(k, v))) (arrays order as their elements: probed over numbers, dates, timestamps with and without a zone, strings and booleans) |
MODE¶
| Dialect | SQL |
|---|---|
| BigQuery | APPROX_TOP_COUNT(col, 1)[OFFSET(0)].value |
| ClickHouse | topK(1)(col)[1] |
| Databricks | MODE(col) |
| Dremio | Not supported |
| DuckDB | MODE(col) |
| MySQL | Not supported |
| Postgres | MODE() WITHIN GROUP (ORDER BY col) |
| Snowflake | MODE(col) |
LISTAGG¶
| Dialect | Base | + DISTINCT | + ORDER BY |
|---|---|---|---|
| BigQuery | STRING_AGG(col, sep) |
STRING_AGG(DISTINCT col, sep) |
STRING_AGG(col, sep ORDER BY col) |
| ClickHouse | arrayStringConcat(groupArray(col), sep) |
arrayStringConcat(groupUniqArray(col), sep) |
arrayStringConcat(arraySort(groupArray(col)), sep) |
| Databricks | ARRAY_JOIN(COLLECT_LIST(col), sep) |
ARRAY_JOIN(COLLECT_SET(col), sep) |
ARRAY_JOIN(SORT_ARRAY(COLLECT_LIST(col)), sep) |
| Dremio | LISTAGG(col, sep) |
LISTAGG(DISTINCT col, sep) |
LISTAGG(col, sep) WITHIN GROUP (ORDER BY col) |
| DuckDB | STRING_AGG(col, sep) |
STRING_AGG(DISTINCT col, sep) |
STRING_AGG(col, sep ORDER BY col) |
| MySQL | GROUP_CONCAT(col SEPARATOR sep) |
GROUP_CONCAT(DISTINCT col SEPARATOR sep) |
GROUP_CONCAT(col ORDER BY col SEPARATOR sep) |
| Postgres | STRING_AGG(col, sep) |
STRING_AGG(DISTINCT col, sep) |
STRING_AGG(col, sep ORDER BY col) |
| Snowflake | LISTAGG(col, sep) |
LISTAGG(DISTINCT col, sep) |
LISTAGG(col, sep) WITHIN GROUP (ORDER BY col) |
LISTAGG ordering limitations
ClickHouse and Databricks only support self-ordering (sorting by the aggregated column). Ordering by a different column raises an error at compile time.
MySQL GROUP_CONCAT limitations
MySQL's GROUP_CONCAT silently cuts its result at group_concat_max_len, 1024 bytes by default. A statement that uses it (a listagg or a median) carries /*+ SET_VAR(group_concat_max_len = 4294967295) */ on its outermost SELECT, which lifts the limit for that statement only; the hint has no effect anywhere else in the statement. Additionally, MySQL silently ignores ORDER BY when DISTINCT is also present in GROUP_CONCAT.
Total not supported
MEDIAN, MODE, LISTAGG, and ANY_VALUE do not support total: true because they cannot be meaningfully re-aggregated via window functions.
Dialect Plugin Architecture¶
Each dialect implements the abstract Dialect base class:
class Dialect(ABC):
@property
@abstractmethod
def name(self) -> str: ...
@property
@abstractmethod
def capabilities(self) -> DialectCapabilities: ...
@abstractmethod
def quote_identifier(self, name: str) -> str: ...
@abstractmethod
def render_time_grain(self, column: Expr, grain: TimeGrain) -> Expr: ...
@abstractmethod
def render_cast(self, expr: Expr, target_type: str) -> Expr: ...
def render_string_contains(self, column: Expr, pattern: Expr) -> Expr: ...
def compile(self, ast: Select) -> str: ...
Dialects register themselves via the @DialectRegistry.register decorator:
@DialectRegistry.register
class PostgresDialect(Dialect):
@property
def name(self) -> str:
return "postgres"
...
The registry provides lookup by name:
from orionbelt.dialect.registry import DialectRegistry
dialect = DialectRegistry.get("snowflake")
sql = dialect.compile(ast)
Adding a New Dialect¶
- Create
src/orionbelt/dialect/my_dialect.py - Subclass
Dialectand implement all abstract methods - Decorate with
@DialectRegistry.register - The dialect is automatically available via
DialectRegistry.get("my_dialect")
Querying Dialect Info via API¶
Date Spine Generation¶
Period-over-period metrics require generating a date series (spine). Each dialect uses a different technique:
| Dialect | Technique |
|---|---|
| Postgres | generate_series(min, max, INTERVAL) |
| DuckDB | generate_series(min, max, INTERVAL) |
| Snowflake | TABLE(GENERATOR(ROWCOUNT => ...)) + DATEADD |
| BigQuery | UNNEST(GENERATE_DATE_ARRAY(min, max, INTERVAL)) |
| Databricks | EXPLODE(SEQUENCE(min, max, INTERVAL)) |
| MySQL | Recursive CTE: WITH RECURSIVE dates AS (...) |
| ClickHouse | arrayJoin(range(...)) + date arithmetic |
| Dremio | Recursive CTE: WITH RECURSIVE dates AS (...) |
For details, see the Period-over-Period Metrics guide.
Querying Available Dialects¶
{
"dialects": [
{
"name": "bigquery",
"capabilities": {
"supports_cte": true,
"supports_qualify": true,
"supports_arrays": true,
"supports_window_filters": true,
"supports_ilike": false,
"supports_time_travel": false,
"supports_semi_structured": true,
"supports_union_all_by_name": false,
"supports_group_by_all": true
},
"supported_aggregations": [
"any_value", "avg", "corr", "count", "count_distinct", "covar_pop",
"covar_samp", "listagg", "max", "median", "min", "mode", "stddev",
"stddev_pop", "sum", "var_pop", "variance"
],
"supported_functions": [
"abs", "ceil", "coalesce", "concat", "div", "ends_with", "exp", "floor",
"greatest", "least", "length", "ln", "log", "lower", "lpad", "ltrim",
"mod", "nullif", "position", "power", "replace", "round", "rpad",
"rtrim", "sign", "split_part", "sqrt", "starts_with", "substring",
"trim", "trunc", "upper"
]
},
...
]
}
capabilities are structural SQL features. The two lists are the vocabulary: every OBML
aggregation: value this dialect can compute, and every entry of the
portable function catalog it can render.
Both are stated positively, so answering "may I use median on this warehouse?" needs no second
call. Anything absent is refused at compile time with a 422 rather than emitted and failed at the
database — MySQL, for instance, does not list mode.