Using Dagster with Sigma#

This guide provides instructions for using Dagster with Sigma. Your Sigma assets, including datasets and workbooks, can be represented in the Dagster asset graph, allowing you to track lineage and dependencies between Sigma assets and upstream data assets you are already modeling in Dagster.

What you'll learn#

• How to represent Sigma assets in the Dagster asset graph, including lineage to other Dagster assets.
• How to customize asset definition metadata for these Sigma assets.

Represent Sigma assets in the asset graph#

To load Sigma assets into the Dagster asset graph, you must first construct a SigmaOrganization resource, which allows Dagster to communicate with your Sigma organization. You'll need to supply your client ID and client secret alongside the base URL. See Identify your API request URL in the Sigma documentation for more information on how to find your base URL.

Dagster can automatically load all datasets and workbooks ƒrom your Sigma workspace. Call the build_defs() function, which returns a Definitions object containing all the asset definitions for these Sigma assets.

from dagster_sigma import SigmaBaseUrl, SigmaOrganization

from dagster import Definitions, EnvVar
from dagster._core.definitions.decorators.definitions_decorator import definitions
from dagster._core.definitions.definitions_loader import DefinitionsLoadContext

resource = SigmaOrganization(
    base_url=SigmaBaseUrl.AWS_US,
    client_id=EnvVar("SIGMA_CLIENT_ID"),
    client_secret=EnvVar("SIGMA_CLIENT_SECRET"),
)


defs = resource.build_defs()

Customize asset definition metadata for Sigma assets#

By default, Dagster will generate asset keys for each Power BI asset based on its type and name and populate default metadata. You can further customize asset properties by using Definitions.map_asset_specs, which applies a function of your choosing to each asset.

from dagster_sigma import SigmaBaseUrl, SigmaOrganization, SigmaWorkbook

from dagster import AssetSpec, Definitions, EnvVar
from dagster._core.definitions.decorators.definitions_decorator import definitions
from dagster._core.definitions.definitions_loader import DefinitionsLoadContext

resource = SigmaOrganization(
    base_url=SigmaBaseUrl.AWS_US,
    client_id=EnvVar("SIGMA_CLIENT_ID"),
    client_secret=EnvVar("SIGMA_CLIENT_SECRET"),
)


defs = resource.build_defs().map_asset_specs(
    lambda spec: spec._replace(owners=["my_team"])
)

Load Sigma assets from multiple organizations#

Definitions from multiple Sigma organizations can be combined by instantiating multiple SigmaOrganization resources and merging their definitions. This lets you view all your Sigma assets in a single asset graph:

from dagster_sigma import SigmaBaseUrl, SigmaOrganization

from dagster import Definitions, EnvVar
from dagster._core.definitions.decorators.definitions_decorator import definitions
from dagster._core.definitions.definitions_loader import DefinitionsLoadContext

sales_team_organization = SigmaOrganization(
    base_url=SigmaBaseUrl.AWS_US,
    client_id=EnvVar("SALES_SIGMA_CLIENT_ID"),
    client_secret=EnvVar("SALES_SIGMA_CLIENT_SECRET"),
)

marketing_team_organization = SigmaOrganization(
    base_url=SigmaBaseUrl.AWS_US,
    client_id=EnvVar("MARKETING_SIGMA_CLIENT_ID"),
    client_secret=EnvVar("MARKETING_SIGMA_CLIENT_SECRET"),
)
defs = Definitions.merge(
    sales_team_organization.build_defs(),
    marketing_team_organization.build_defs(),
)