Combining Components with Pythonic definitions

As your Dagster project grows, you may want to leverage Components for standardized data pipelines while still maintaining traditional @dg.asset definitions for custom logic. This guide shows how to combine both approaches in a single project using Definitions.merge.

When to use this pattern

This pattern is useful when you:

• Want to use Components for standardized integrations (Sling, dbt, etc.), but need custom Python logic for specific transformations
• Are migrating an existing project to Components incrementally
• Have team members who prefer working with Python code while others benefit from Components' declarative approach

Example project structure

Here's a project that combines Components with traditional Pythonic assets:

tree my_project/src/my_project

my_project/src/my_project
├── __init__.py
├── definitions.py
├── assets
│   ├── __init__.py
│   └── analytics.py
├── resources
│   ├── __init__.py
│   └── warehouse.py
└── defs
    ├── raw_data_sync
    │   └── defs.yaml
    └── analytics_dbt
        └── defs.yaml

5 directories, 8 files

This structure includes:

Directory	Purpose
defs/	Contains Dagster Component definitions
assets/	Traditional @asset definitions written in Python
resources/	Shared resources used by both Component definitions and Pythonic asset definitions

Combining Component and Pythonic asset definitions

The key to combining Component with Pythonic asset definitions is using Definitions.merge. This merges multiple Definitions objects together:

my_project/src/my_project/definitions.py

@dg.definitions
def defs():
    """Combine Components and Pythonic assets."""
    # Load component definitions from the defs/ folder
    component_defs = dg.load_from_defs_folder(path_within_project=Path(__file__).parent)

    # Create definitions for Pythonic assets
    pythonic_defs = dg.Definitions(
        assets=[customer_segmentation, revenue_forecast],
    )

    # Merge component definitions with pythonic definitions
    return dg.Definitions.merge(component_defs, pythonic_defs)

This pattern:

1. Uses load_from_defs_folder to automatically discover and load Component definitions from the defs/ folder
2. Creates a separate Definitions object for your Pythonic assets
3. Merges the Component and Pythonic asset definitions together using Definitions.merge

Sharing resources across both types of definitions

When Component and Pythonic asset definitions need to share resources (like a database connection), you can bind resources when creating your definitions.

There are two recommended patterns for organizing resources:

Pattern 1: resources/ module

Keep resources in a resources/ Python module and bind them in definitions.py. This pattern is best for:

• Complex resource logic that benefits from being in a dedicated module
• Resources that need unit testing
• Sharing resources across multiple code locations

my_project/src/my_project/resources/__init__.py

class MyResource(dg.ConfigurableResource): ...


def get_warehouse_resource():
    return MyResource()

my_project/src/my_project/definitions.py

@dg.definitions
def defs():
    """Combine Components and Pythonic assets with shared resources."""
    # Load component definitions from the defs/ folder
    component_defs = dg.load_from_defs_folder(path_within_project=Path(__file__).parent)

    # Define resources available to ALL assets (components and pythonic)
    resources = {
        "warehouse": get_warehouse_resource(),
    }

    # Create definitions for Pythonic assets
    pythonic_defs = dg.Definitions(
        assets=[customer_segmentation, revenue_forecast],
        resources=resources,
    )

    # Merge component definitions with pythonic definitions
    return dg.Definitions.merge(component_defs, pythonic_defs)

Pattern 2: defs/ folder

Define resources in a defs.py file inside the defs/ folder. This pattern is best for:

• Simple resources with minimal configuration
• Component-centric architectures
• Resources that are primarily used by Components

my_project/src/my_project/
└── defs/
    ├── shared_resources/
    │   └── defs.py      # Returns Definitions with resources
    ├── raw_data_sync/
    │   └── defs.yaml
    └── analytics_dbt/
        └── defs.yaml

my_project/src/my_project/defs/shared_resources/defs.py

@dg.definitions
def defs():
    return dg.Definitions(
        resources={
            "warehouse": MyResource(),
        }
    )

Resources defined this way are automatically discovered by load_from_defs_folder.

Your traditional @asset functions can request resources by parameter name:

my_project/src/my_project/assets/analytics.py

@dg.asset
def customer_segmentation(warehouse: MyResource) -> None: ...


@dg.asset(deps=[customer_segmentation])
def revenue_forecast(warehouse: MyResource) -> None: ...

The resource key (warehouse) in the resources dict matches the parameter name in the asset functions, allowing Dagster to inject the resource automatically.

Where resources should NOT live

Avoid these patterns when organizing resources:

❌ Anti-pattern	Why it's problematic
defs/snowflake/defs.yaml	Resources are not Components. Components produce assets; resources are dependencies. YAML component definitions require a Component class.
resource/snowflake/ (singular)	Inconsistent naming and unnecessary nesting. Use resources/ (plural) as a flat module.

Best practices

When combining Components with Pythonic definitions:

• Use Components for standardized integrations: Sling for data replication, dbt for transformations, etc.
• Use Pythonic assets for custom logic: Complex transformations, ML models, or business logic that doesn't fit a Component
• Share resources through definitions.py: Bind resources at the top level so they're available to all assets
• Keep resource logic in a resources/ module: Makes resources testable and reusable

Next steps

• Learn how to add Components to an existing project
• Explore available Components
• Read about creating custom Components