AshClickhouse.DataLayer (AshClickhouse v0.4.0)

Copy Markdown View Source

An Ash data layer for ClickHouse.

This data layer implements the Ash.DataLayer behaviour so that Ash resources can be backed by a ClickHouse columnar OLAP database. It uses the clickhouse client under the hood.

Configuration

defmodule MyApp.MyResource do
  use Ash.Resource,
    data_layer: AshClickhouse.DataLayer

  clickhouse do
    table "my_table"
    repo MyApp.Repo
  end

  attributes do
    uuid_primary_key :id
    attribute :name, :string
  end
end

Features Supported

  • :create / :read / :update / :destroy
  • :filter — full WHERE support
  • :limit / :offset — ClickHouse supports both natively
  • :select — column projection
  • :sortORDER BY
  • :distinctSELECT DISTINCT
  • :multitenancy — database- or attribute-based
  • :bulk_create — batch INSERT
  • :update_query / :destroy_queryALTER TABLE ... UPDATE/DELETE
  • :calculate — in-memory calculations
  • :composite_primary_key
  • :nested_expressions / :boolean_filter
  • :async_engine
  • :expression_calculation
  • {:aggregate, :count | :sum | :avg | :min | :max} — native aggregates

  • {:query_aggregate, :count | :sum | :avg | :min | :max}

  • Combination queries (UNION/INTERSECT) — executed by Ash in memory

Relationship aggregates

attach_aggregates/5 supports {:aggregate, :count | :sum | :avg | :min | :max} over belongs_to, has_many, and has_one relationships. A belongs_to aggregate is a lookup of the related row's scalar field (e.g. customer.tier), not a true aggregation across multiple rows — ClickHouse's lack of JOINs makes this distinction more visible than in a Postgres-backed data layer. has_many/has_one aggregates are real grouped aggregations (e.g. "count of orders per customer"). Multi-hop relationship paths are not supported and fall back to each aggregate's default_value.

Features NOT Supported

  • :transact — ClickHouse has no multi-statement transactions
  • :lock — locking is a no-op
  • :keyset — ClickHouse has no token-based keyset pagination
  • :upsert — ClickHouse has no ON CONFLICT (use :create + :update_query)
  • :expression_calculation_sort — not supported
  • :aggregate_filter / :aggregate_sort — not supported
  • :update_many — use :update_query
  • :composite_type / :through_relationship
  • :join — JOINs are not yet implemented
  • :filter_relationship / {:exists, :unrelated} / {:aggregate_relationship, _}
  • {:query_aggregate, :list | :first | :exists | :custom} — only count/sum/avg/min/max

Summary

Functions

Clears the resource → repo ETS cache.

Returns a stream of Ash records for the given query, consuming ClickHouse's native query stream instead of materializing every row into memory.

Types

t()

@type t() :: AshClickhouse.Query.t()

Functions

clear_repo_cache!()

@spec clear_repo_cache!() :: :ok

Clears the resource → repo ETS cache.

The cache is populated lazily and lives for the life of the VM, which is correct for long-running production apps but painful for test suites that redefine resources or repo configuration between tests. Call this from setup/on_exit in those tests to force re-resolution.

repo(resource)

@spec repo(module()) :: module()

stream(data_layer_query, resource, opts \\ [])

@spec stream(t(), Ash.Resource.t(), keyword()) :: Enumerable.t(Ash.Resource.t())

Returns a stream of Ash records for the given query, consuming ClickHouse's native query stream instead of materializing every row into memory.

This is the natural read path for large OLAP scans/reports. The returned stream yields decoded Ash records one at a time as chunks arrive.

In-memory calculations and aggregates configured on the query are applied to each decoded chunk, so stream/3 returns results identical to run_query/2 (which applies them after fetching all rows). This keeps streaming and non-streaming reads behaviourally consistent.

Options

  • :mutations_sync is ignored for reads.
  • any other options are forwarded to the underlying ClickHouse client.