log worker identity

[gudata]

Description

we should know where the task is processed. log worker type and identity - ecs.docker and the id cluster/task id

Use case/motivation

when someone have a lot of instance ids or ecs services as workers it would want to know on which of them the task has been executed.

now this is unclear. are the logs coming from the scheler, worker ot from where

Real production could have different kind of workers.

Related issues

example: I am getting timeout on compiling the dags. where should I check for the error? the scheduler or the worker node?

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[jroachgolf84]

Couple of things here:

• When processing DAGs, logs will go to the Scheduler/DAG Process (if you're running a DAG Processor).
• Synchronous Tasks execute on Workers. These logs are output into the Airflow UI (and can also be persisted elsewhere).
• Asynchronous Tasks will typically execute on the Triggerer. This is where you'd find logs for a deferrable Operator/AssetWatcher. I believe these are also passed through to the Airflow UI, but I may be wrong about that.

Do these answers address your requirements/questions?