User metadata #1657

THardy98 · 2025-03-21T19:51:16Z

What was changed

Added functionality for users to provide staticDetails and staticSummary metadata to workflow start commands.
These can be retrieved via describe on the workflow handler and are visibile via the UI/CLI.

Users can set currentDetails - a mutable details string within the workflow - via SetCurrentDetails and retrieve it via GetCurrentDetails. This is visible via the UI/CLI and the __temporal_workflow_metadata internal query.

Users can add a summary (short string description) when setting a timer and when starting an activity.

A notable part of the change is the addition of runWithOptions to activities. Activities can now override their activity options at call time, like so:

* // Setup Activities from module exports
 * const { httpGet, otherActivity } = proxyActivities<typeof activities>({
 *   startToCloseTimeout: '30 minutes',
 * });
 *
 * // Use activities with default options
 * const result1 = await httpGet('http://example.com');
 *
 * // Override options for specific activity calls
 * const result2 = await httpGet.executeWithOptions({
 *   staticSummary: 'Fetches data from external API',
 *   scheduleToCloseTimeout: '5m'
 * }, ['http://api.example.com']);
 *
 * const result3 = await otherActivity.executeWithOptions({
 *   staticSummary: 'Processes the fetched data',
 *   taskQueue: 'special-task-queue'
 * }, [data]);

Closes [Feature Request] Support user metadata #1544
How was this tested:
Couple integration tests
Any docs updates needed?
Maybe

…es field to add a summary to an activity

mjameswh · 2025-05-13T18:03:58Z

packages/common/src/workflow-options.ts

+   * General fixed details for this workflow execution that may appear in UI/CLI.
+   * This can be in Temporal markdown format and can span multiple lines.
+   *
+   * @experimental


Please add a short comment on all @experimental tags (see what we did elsewhere). Benefit isn't huge, but at least, it identifies which "feature" the experimental API is linked with.

I believe i've done so in all the relevant places

packages/client/src/schedule-helpers.ts

packages/client/src/workflow-client.ts

packages/workflow/src/workflow.ts

mjameswh · 2025-05-13T23:51:06Z

packages/workflow/src/interceptors.ts

@@ -80,6 +80,7 @@ export interface ActivityInput {
  readonly options: ActivityOptions;
  readonly headers: Headers;
  readonly seq: number;
+  readonly cmdOpts?: WorkflowCommandOptions;


I don't think we should have this WorkflowCommandOptions type. I even wonder if we should inline summary and details directly in ActivityInput (without the extra UserMetadata nesting). Or add these two to ActivityOptions, though that might pose some minor challenge regarding the type provided to the proxyActivities() function.

What did we do in other SDKs?

Typically they are inline, I've changed it as such (aside from using ActivityOptions for summary/details for an activity, and TimerOptions as you suggested below)

mjameswh · 2025-05-13T23:59:55Z

packages/workflow/src/workflow.ts

 */
-export function sleep(ms: Duration): Promise<void> {
+export function sleep(ms: Duration, summary?: string): Promise<void> {


I'm worried this will evolve badly. What if we then need to add description, and then other properties? Instead, I think I'd define a TimerOptions type, and take this as a second argument here. Then, we can easily add whatever properties we need.

Out of curiosity, what do we do in other SDKs?

typically we inline, but went with TimerOptions here given your reasoning

packages/workflow/src/workflow.ts

mjameswh · 2025-05-14T01:01:28Z

packages/workflow/src/workflow.ts

+   * @param summaries Record mapping activity names to their summary descriptions
+   * @returns A new proxy with the provided summaries
+   */
+  withSummaries(summaries: Record<string, string>): ActivityInterfaceFor<A>;


🤔 That feels a bit awkward, and would prevent having an activity named withSummaries (ok, that's arguably very unlikely, but still means this approach would evolve badly).

I think we could get a better DX by using something similar to this, but at the activity-level rather than at the activities-level (mind the plural). That would also allow any extra option to be set at the call site, which is a recurring request.

For example, we could have:

const { echo } = proxyActivities<MyActivities>({ startToCloseTimeout: '5m' }); // No extra options await echo(myArg1, myArg2); // With extra options, higher order function style await echo.withOptions({ summary: "..." })(myArg1, myArg2); // Some other possible variants to be considered await echo.withOptions({ summary: "..." }, myArg1, myArg2); await echo.withOptions({ summary: "..." }, [myArg1, myArg2]);

I'll let you think you a bit about this, but we should probably include others in this discussion.

opted for:
await echo.runWithOptions(options, args);

to allow users to override their default activity options

renamed to executeWithOptions

THardy98 · 2025-06-03T00:00:27Z

also quickly added lazy decoding summary/details from workflow description

can't do so (or at least annoying to do so) for schedule description because the description type has an invariant where the description must be a superclass of schedule options (which necessitate summary/details being strings)

the alternative being to add new fields that lazily populate the existing fields, but that feels like poor ux

packages/workflow/src/interceptors.ts

mjameswh · 2025-06-05T21:56:12Z

packages/workflow/src/workflow.ts

+        startToFireTimeout: msToTs(durationMs),
+      },
+      userMetadata: options && {
+        summary: options.summary ? activator.payloadConverter.toPayload(options.summary) : undefined,


What if summary is an empty string? As it is now, it will be replaced by undefined. Not sure what would be the desired result.

If we want to keep empty strings, then it might be cleaner to use some toOptionalPayload helper.

And if we don't care about empty strings, then should instead move the terniary out of this object:

userMetadata: options?.summary ? { summary: activator.payloadConverter.toPayload(options.summary), } : undefined,

I think having an empty summary string replaced with undefined is reasonable. I don't see a reason why having an empty summary would be useful (or any more useful than undefined).

mjameswh · 2025-06-05T22:03:11Z

packages/workflow/src/workflow.ts

@@ -130,8 +134,9 @@ function timerNextHandler(input: TimerInput) {
 *
 * @param ms sleep duration - number of milliseconds or {@link https://www.npmjs.com/package/ms | ms-formatted string}.
 * If given a negative number or 0, value will be set to 1.
+ * @param summary a short summary/description of the timer. Can serve as a timer ID.


@param summary is boggus.

yup from old commit
replaced with * @param options optional timer options for additional configuration not sure if that's entirely descriptive..

packages/workflow/src/workflow.ts

mjameswh · 2025-06-06T06:09:53Z

packages/workflow/src/workflow.ts

@@ -301,7 +315,8 @@ export function scheduleActivity<R>(activityType: string, args: any[], options:
 export async function scheduleLocalActivity<R>(
  activityType: string,
  args: any[],
-  options: LocalActivityOptions
+  options: LocalActivityOptions,
+  summary?: string


summary should be part of options, same as for regular activities

also from old commit, removed
uses staticSummary from options now

packages/workflow/src/workflow.ts

mjameswh · 2025-06-06T06:23:48Z

packages/workflow/src/workflow.ts

+          overrideOptions: ActivityOptions,
+          args: any[]
+        ): Promise<unknown> {
+          return scheduleActivity(activityType, args, { ...options, ...overrideOptions });


Not so fast. You will need to merge recursively on retry policy and priority.

added a deepMerge utility in object-helpers.ts for use here (and possibly elsewhere in the future)

packages/workflow/src/workflow.ts

mjameswh · 2025-06-06T06:28:17Z

packages/workflow/src/workflow.ts

+          overrideOptions: ActivityOptions,
+          args: any[]
+        ): Promise<unknown> {
+          return scheduleLocalActivity(activityType, args, { ...options, ...overrideOptions });


Same as before, need to handle recursive options merging on retryOptions and priority.

added a deepMerge utility in object-helpers.ts for use here (and possibly elsewhere in the future)

mjameswh · 2025-06-06T06:30:54Z

packages/common/src/activity-options.ts

+   *
+   * @experimental User metadata is a new API and suspectible to change.
+   */
+  staticSummary?: string;


I think that should be only summary, no static. What's the terminology in other SDKs?

a bit of a mix, but generally static prefix for workflow start methods, otherwise no static, i'ved opted for the same

mjameswh · 2025-06-06T06:31:06Z

packages/common/src/activity-options.ts

+   *
+   * @experimental User metadata is a new API and suspectible to change.
+   */
+  staticSummary?: string;


no static prefix for activity options, changed

mjameswh · 2025-06-06T06:55:23Z

I believe there should also be user metadata on:

child workflows
conditions
signals
update

…ge to merge/override nested fields in objects (used to override activity options with activities' executeWithOptions), add metadata to child workflows and condition calls

THardy98 · 2025-06-16T11:01:18Z

I believe there should also be user metadata on:
* child workflows

* conditions

* signals

* update

Added for conditions and child workflows. Signals/updates/queries have a description parameter which is used for a similar purpose: #1319 (I don't believe we've added user metadata for these in other SDKs as well, it doesn't mention it in the original feature issue).

also going to mention this again, because it's been awhile

THardy98 added 3 commits March 20, 2025 19:33

init impl of user metadata

7b3aa1c

add test, removed proxyActivities overloads instead added withSummari…

2dab40d

…es field to add a summary to an activity

schedule metadata test, fix schedule action decode

6af2e2c

THardy98 requested a review from a team as a code owner March 21, 2025 19:51

Merge branch 'main' into user-metadata

e0489cd

THardy98 force-pushed the user-metadata branch from 9bceb64 to e0489cd Compare March 24, 2025 18:20

test fix

031e295

mjameswh reviewed May 13, 2025

View reviewed changes

packages/client/src/schedule-helpers.ts Outdated Show resolved Hide resolved

mjameswh reviewed May 13, 2025

View reviewed changes

packages/client/src/schedule-helpers.ts Outdated Show resolved Hide resolved

mjameswh reviewed May 13, 2025

View reviewed changes

packages/client/src/workflow-client.ts Outdated Show resolved Hide resolved

mjameswh reviewed May 13, 2025

View reviewed changes

packages/client/src/workflow-client.ts Outdated Show resolved Hide resolved

mjameswh reviewed May 13, 2025

View reviewed changes

packages/workflow/src/workflow.ts Outdated Show resolved Hide resolved

mjameswh reviewed May 13, 2025

View reviewed changes

mjameswh reviewed May 14, 2025

View reviewed changes

packages/workflow/src/workflow.ts Outdated Show resolved Hide resolved

mjameswh reviewed May 14, 2025

View reviewed changes

THardy98 added 2 commits June 2, 2025 21:16

use supplied data converter

d496966

inline options, lazily decode summary/details on workflow description

1efb3f0

THardy98 force-pushed the user-metadata branch from 414967a to 1efb3f0 Compare June 2, 2025 23:42

THardy98 added 2 commits June 3, 2025 01:48

update proxyActivities comment to show runWithOptions usage

ebe745f

Merge branch 'main' into user-metadata

2c8f922

THardy98 added 2 commits June 3, 2025 02:45

add runWithOptions for local activities

67d8e06

allow for undefined userMetadata when scheduling activity

c491683

THardy98 force-pushed the user-metadata branch from d2bb388 to c491683 Compare June 3, 2025 00:48

THardy98 requested a review from mjameswh June 3, 2025 07:08

mjameswh reviewed Jun 5, 2025

View reviewed changes

packages/workflow/src/interceptors.ts Outdated Show resolved Hide resolved

mjameswh reviewed Jun 5, 2025

View reviewed changes

packages/workflow/src/interceptors.ts Outdated Show resolved Hide resolved

mjameswh reviewed Jun 5, 2025

View reviewed changes

packages/workflow/src/workflow.ts Outdated Show resolved Hide resolved

mjameswh reviewed Jun 6, 2025

View reviewed changes

packages/workflow/src/workflow.ts Outdated Show resolved Hide resolved

mjameswh reviewed Jun 6, 2025

View reviewed changes

packages/workflow/src/workflow.ts Show resolved Hide resolved

mjameswh reviewed Jun 6, 2025

View reviewed changes

packages/workflow/src/workflow.ts Outdated Show resolved Hide resolved

mjameswh reviewed Jun 6, 2025

View reviewed changes

packages/workflow/src/workflow.ts Outdated Show resolved Hide resolved

mjameswh reviewed Jun 6, 2025

View reviewed changes

move user metadata encoding/decoding to user-metadata.ts, add deepMer…

60a1e08

…ge to merge/override nested fields in objects (used to override activity options with activities' executeWithOptions), add metadata to child workflows and condition calls

break circular dependency

3cfffa7

THardy98 requested a review from mjameswh June 16, 2025 14:00

add codec testing

e56a117

User metadata #1657

Are you sure you want to change the base?

User metadata #1657

Uh oh!

Conversation

THardy98 commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was changed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

THardy98 Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

THardy98 commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mjameswh Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mjameswh Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

THardy98 commented Mar 21, 2025 •

edited

Loading

THardy98 Jun 2, 2025 •

edited

Loading

THardy98 commented Jun 3, 2025 •

edited

Loading

mjameswh Jun 5, 2025 •

edited

Loading

mjameswh Jun 6, 2025 •

edited

Loading

THardy98 Jun 16, 2025 •

edited

Loading

THardy98 commented Jun 16, 2025 •

edited

Loading