Would it be possible for a DAG to have a static data interval? #59398
Unanswered
plutaniano
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm working with a source system that I can't query using date filters, I have to pull entire dataset every time.
I wanted
data_interval_startanddata_interval_endfor the DAG that fetches this dataset to match this condition, by having static values for the data interval. Is something like this possible? Is it a bad idea?I tried implementing my own Timetable, but it seems that because the
logical_dateis the same everytime, the dag runs only once.StaticIntervalTimetable
Beta Was this translation helpful? Give feedback.
All reactions