Multiple code locations in Dagster OSS with shared code (resources, utils) #31890
Unanswered
tspencerdata
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi all,
I’m looking for some guidance on repo structure best practices in Dagster OSS.
Context:
Current repo structure (simplified):
dagster_OSS/
├─ analytics/ # code location 1 (unique env)
│ ├─ defs/
│ ├─ lib/
| └─ pyproject.toml
├─ data_engineering/ # code location 2 (unique env)
│ ├─ defs/
│ ├─ lib/
│ ├─ resources/ # shared resources
│ ├─ utils/ # shared utilities
| └─ pyproject.toml
├─ deploy_files/
├─ workspace.yaml # defines the code locations
└─ pyproject.toml
Each code location may need libraries that would clash with others, so isolation is important. But I also want to avoid duplicating the shared utils/ and resources/ code across locations.
What’s the recommended way to structure repos when you need per-location env isolation but also shared code?
Beta Was this translation helpful? Give feedback.
All reactions