Skip to content

Add proposal for RID-specific .NET Tool packages #333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dsplaisted merged 3 commits into from
Jul 2, 2025
Merged

Add proposal for RID-specific .NET Tool packages #333

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
15301c7
Add proposal for RID-specific .NET Tool packages
dsplaisted Apr 21, 2025
117062f
Updates
dsplaisted Jun 2, 2025
85c81f3
Update production section to match implementation
dsplaisted Jul 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions INDEX.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ Use update-index to regenerate it:
| 2023 | [net8.0-browser TFM for applications running in the browser](accepted/2023/net8.0-browser-tfm.md) | [Javier Calvarro](https://github.com/javiercn) |
| 2024 | [.NET Standard Targeting Recommendations](accepted/2024/net-standard-recommendation.md) | [Immo Landwerth](https://github.com/terrajobst), [Viktor Hofer](https://github.com/ViktorHofer) |
| 2025 | [Provide SDK hint paths in global.json](accepted/2025/local-sdk-global-json.md) | [Jared Parsons](https://github.com/jaredpar) |
| 2025 | [RID-Specific .NET Tool Packages](accepted/2025/rid-specific-tool-packages.md) | [Daniel Plaisted](https://github.com/dsplaisted) |

## Drafts

Expand Down
91 changes: 91 additions & 0 deletions accepted/2025/rid-specific-tool-packages.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# RID-Specific .NET Tool Packages
Copy link
Member

@joelverhagen joelverhagen Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a GitHub issue with more context on the "why" such as customer comments? Just curious!

Copy link
Member Author

@dsplaisted dsplaisted Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if there's an issue for this. This makes tools better overall, and more specifically allows tools to be compiled with NativeAOT, and more specifically could help make it easier to distribute an AI MCP implemented in .NET and compiled as NativeAOT.


**Owner** [Daniel Plaisted](https://github.com/dsplaisted)

## Summary

This proposal adds the ability for a .NET Tool to have separate packages for each supported OS and architecture.

In .NET, the combination of OS and architecture is represented by a Runtime Identifier (RID). Today, .NET Tools support native assets, but the native assets for all of the supported Runtime Identifiers need to be included in the same package. For tools with large native dependencies, or if the entire tool is native (for example via NativeAOT), this multiplies the package download size by the number of supported RIDs.
Copy link
Member

@jkotas jkotas Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exact same problem exists for regular nuget packages too. Any chance we can generalize the solution to solve the problem for all types of nuget packages, and not just tools?

Related question: Is the downloader for the proposed dependent packages going to be implemented in nuget or in the SDK?

Copy link
Member

@akoeplinger akoeplinger Apr 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NuGet/Home#10571 is an issue about the problem for regular nuget packages.

I'd be very happy if this could replace the runtime.json hack.

Copy link
Member Author

@dsplaisted dsplaisted Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related question: Is the downloader for the proposed dependent packages going to be implemented in nuget or in the SDK?

This is implemented in the SDK, but it calls NuGet APIs to download the packages. That is how tools already work, this proposal updates the logic so that once the SDK downloads the primary package it will see that there's a RID-specific package to download and download that one too.

Any chance we can generalize the solution

Lots of people have had this feedback and we also discussed it in a meeting yesterday. I don't think we would want to use the same system for tools as for RID-specific package dependencies. I will add a section to the design explaining this, and I'm sure we'll also discuss it more.

Copy link
Member

@ericstj ericstj Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also NuGet/Home#1660


We would like to add support for RID-specific .NET Tool packages. This would enable a tool to have a separate package for each supported RID, and when the tool is installed only the package for the correct RID would need to be downloaded.

### Goals

- Tools can create RID-specific packages
- Installing a tool with RID-specific packages is transparent but only downloads the package for the current RID

### Non-goals

- Allowing a tool to have assets from a mixture of packages, for example have architecture neutral assets in the primary package and native dependencies in RID-specific packages

## Tool manifests

.NET Tools have a tool manifest file named `DotnetToolSettings.xml`. It is stored alongside the executable tool assets in the `tools\<TargetFramework>\<RuntimeIdentifier>` folder. Currently for most or all tools, the `RuntimeIdentifier` is `any`. Here is an example of a current manifest:

```xml
<DotNetCliTool Version="1">
<Commands>
<Command Name="dotnet-say" EntryPoint="dotnet-say.dll" Runner="dotnet" />
</Commands>
</DotNetCliTool>
```

## Design

A tool with RID-specific packages will consist of a single primary package and RID-specific packages for each supported RID. The primary package will include a tool manifest that lists the RIDs supported by the tool, and the package name and package version of the RID-specific tool package for each one. The primary package will not include any tool implementation assets or shims. The RID-specific packages will have the same layout and format as tool packages currently have. The only difference will be that the NuGet package type will be set to a new `DotnetToolRidPackage` type, in order to prevent tool search results from being cluttered with RID-specific tool packages (the primary package is the one that should show up in the results).
Copy link
Member

@joelverhagen joelverhagen Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Package type has versioning. Currently DotnetTool is the default 0.0 version. If it is bumped to DotnetTool, 2.0.0 then they should still be searchable by DotnetTool but also have a persistent marker that could be later used for filtering, without having to crack open the package again to parse the DotnetToolSettings.xml. Right now we only have package type name searching enabled but we could enable version filtering in the future.

Conceptually, the package type is moving to version 2 as shown in your XML.

Copy link
Member Author

@dsplaisted dsplaisted Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I'm not sure we knew that you could version package types when we originally implemented tools, or maybe at that time it wasn't possible to version them. Since old SDKs don't look at the package type we'll have to update the version in the tool manifest anyway so that old SDKs will error out when trying to install a newer tool. So I'm not sure there's much extra value in also updating the package type version.

Copy link
Member

@joelverhagen joelverhagen Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would allow NuGet.org to filter these by version, and analyze their adoption in NuGet Insights (separate from V1 tools). It could allow customized UI on NuGet.org to show the RID-specific dependencies (after analyzing DotnetToolSettings.xml). In other words, it's the "NuGet" way of versioning package types.

Package type versioning has been there since the beginning. I made it :)

It's there to use if you want it. I think it would be good to have these new packages marked differently, in a nuget-specific way if possible.

Copy link
Member

@joelverhagen joelverhagen Apr 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the RID-specific package be validated to have DotnetToolRidPackage? Or will any existing package be accepted and treated as a tool package?

The relationship is only 2 layers deep, right? 1 DotnetTool to N DotnetToolRidPackage, and no DotnetTool -> DotnetTool -> DotnetToolRidPackage?

Copy link
Member Author

@dsplaisted dsplaisted Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RID-specific package won't have DotnetToolRidPackage in its manifest, that's only for the primary package to redirect to the RID-specific packages. Theoretically a tool package could redirect to any other tool package. There's currently only layers though, it wouldn't keep following redirects.

We might also decide we want to support tool redirects. I think that might be useful as part of the one-shot / dnx / dotnet tool exec work in order to hide the difference between a tool package name and its command name. But that's not part of a proposal yet.

Copy link
Member

@joelverhagen joelverhagen Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think I was confused at first by the use of the term "manifest". In NuGet world we sometimes refer to the .nuspec as the package manifest (this is where package types are expressed).

What I am understanding is that the .NET SDK will only follow one layer of edges in the graph (tree, at this point).

EXISTING BEHAVIOR: install tool package A -> this downloads and operates on A, a single .nupkg, with no package type validation.

NEW BEHAVIOR (in addition to existing behavior): install tool package X -> this downloads and operates on two .nupkgs: X and Y. X has a pointer to Y in the DotnetToolSettings.xml. There will be no package type validation for X or Y.

If Y happens to have its own RID-specific pointer to Z it will not be followed (no recursive graph walk).

Copy link
Member

@joelverhagen joelverhagen Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will package source mapping, vulnerability auditing, signature validation and policies (etc) all apply to the RID-specific package? I think what I'm getting at, is: will security features that are added to restore happen here too automatically?

Copy link
Member Author

@dsplaisted dsplaisted Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The download of the RID-specific package will be handled the same way tool package downloads are already handled today. We call NuGet APIs to download the package. Depending on the feature (security or not), we might not support it by default without a change to the SDK code. When package source mapping was originally introduced we didn't support it for tools (or .NET SDK workload downloads, which also use the same APIs), and we had to update the SDK to support it. Signature validation may already be handled by the APIs we call. I think vulnerability auditing doesn't currently apply to tools, as they aren't referenced as packages in projects and don't go in the assets file.

So in general, whenever NuGet does a security feature we should think about how or if it applies to .NET Tools, which are a different way of consuming packages. However, adding support for RID-specific packages doesn't change anything here, as the RID-specific packages will be handled the same way as other tool packages are. (Note that we also have PackageDownloads in the .NET SDK, which are a bit more similar to normal NuGet package consumption but are still a different case to be considered.)

Copy link
Member

@joelverhagen joelverhagen Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for clarifying. It's unfortunate that some security features don't work for current .NET tools. That seems like a problem but I see how it's orthogonal to this change. I think the reason I asked is because this feature feels like a mini restore operation with a relatively shallow graph. Since there are now transitive nodes it may be harder for users to reason about the package that they use and depend on.

Maybe the only bit of feedback here is to consider in the design how users can discover and audit the RID-specific package with tooling, similar to how we have dependency package tooling.

Copy link
Member Author

@dsplaisted dsplaisted Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought I responded to this but it looks like it got lost or I never submitted the comment.

The RID-specific packages will be handled the same way as tool packages currently are. Whether security (or other) NuGet features apply to tool packages depends on how the feature is implemented. We don't run restore, we call NuGet APIs to download a package. In some cases that may get us new features by default, in other cases it may not. For example, when package source mapping was introduced, we didn't immediately support it for tools (or for workload packages, which are also downloaded without restore). We had to update the downloader code in the .NET SDK to support the new feature.

I think NuGet auditing doesn't apply at all to tools because tools are not expressed as PackageReferences and don't show up in your assets file.

So when new security features are added, we should consider whether they should apply to .NET Tools, and if so whether we need to update the .NET SDK to enable that. But again, RID-specific tools don't really change this, because the RID-specific packages are downloaded the same way as existing tool packages.


The RID-specific tool packages should be named using the convention `<Toolname>.<RID>`. For example, if the `dotnetsay` tool has RID-specific packages, they would be named `dotnetsay.win-x64`, `dotnetsay.linux-x64`, etc.

The tool manifest for the primary package would look like this:

```xml
<DotNetCliTool Version="1">
<Commands>
<Command Name="dotnet-say" />
</Commands>
<RuntimeIdentifierPackages>
<RuntimeIdentifierPackage RuntimeIdentifier="win-x64" Id="dotnet-say.win-x64" Version="1.0.0" />
<RuntimeIdentifierPackage RuntimeIdentifier="linux-x64" Id="dotnet-say.linux-x64" Version="1.0.0" />
</RuntimeIdentifierPackages>
</DotNetCliTool>
```

Note that the Command has no EntryPoint or Runner, as that is defined in the RID-specific package. The command name is still included because that may be useful for tooling.

## Entry points

Recall that the tool manifest file has a section that looks like this:

```xml
<Command Name="dotnet-say" EntryPoint="dotnet-say.dll" Runner="dotnet" />
```

Currently, the only supported value for `Runner` is `dotnet`, which means that the `EntryPoint` is a DLL that can be launched with the `dotnet` executable. When a global tool is installed, the .NET SDK will create a version of the .NET apphost (called a shim) in the global tools folder. This shim has a filename matching the tool command name so that the tool can be launched from the path. The shim also has the relative path to the tool entry point embedded in it so that it can launch the correct tool. Tools can also include their own shims so that they can be properly signed.

To support NativeAOT .NET apps (or non .NET apps), we should support an additional `executable` runner for executables that can be launched natively by the operating system. On Unix-like operating systems, we can create a symlink in the global tools folder to the tool entry point executable. On Windows, we may create a batch file that launches the entry point executable. If this has drawbacks we might create a new type of shim/launcher.

An alternative would be to copy the whole entry point executable to the global tools folder. However, there might be other files in the same folder that the tool depends on, so we don't currently plan to do this.
Copy link
Member

@rolfbjarne rolfbjarne Apr 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, there might be other files in the same folder that the tool depends on, so we don't currently plan to do this.

Here's an example of something that wouldn't work:

  1. A tool using ClangSharp would be an excellent test case for a rid-specific tool package, because the native clang dependency is big.
  2. A tool using ClangSharp would have a dependency on a native dynamic library (dll/dylib), and as such copying just the executable would not work.


## Tool manifest format version

The root element of the tool manifest has a Version attribute. Currently this is set to 1. When creating a tool manifest with a `RuntimeIdentifierPackages` element or a command Runner type of `executable`, we should set the Version to 2, so that prior .NET SDKs that don't support those features will error out gracefully when trying to install the tool. If the tool manifest doesn't use these new features, the version should still be set to 1 so that the tool can be consumed by previous SDKs.

## Production

A tool project will be able to specify the runtime identifiers to create RID-specific packages for using the `ToolPackageRuntimeIdentifier` item. For example:

```xml
<ItemGroup>
<ToolPackageRuntimeIdentifier Include="win-x64"/>
<ToolPackageRuntimeIdentifier Include="linux-x64"/>
</ItemGroup>
```

If `ToolPackageRuntimeIdentifier` is non-empty, then packing the project without a RuntimeIdentifier will create the primary package, and packing the project with a RuntimeIdentifier will create the corresponding RID-specific package. If feasible, we could do an automatic inner pack of the ToolPackageRuntimeIdentifiers before creating the primary tool package.

The package name for the RID-specific packages will be `<ToolPackageName>.<RuntimeIdentifier>`. If not specified, the version number of the RID-specific package will be the same as that of the primary package. `Version` metadata on the `ToolPackageRuntimeIdentifier` item can be used to set the RID-specific package version.
Copy link
Member

@joelverhagen joelverhagen Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have optional Version? Won't it be known at pack time? I am unsure why we should allow a missing version at all.

Copy link
Member Author

@dsplaisted dsplaisted Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are building a NativeAOT app, then you mostly have to build for each Runtime Identifier on a separate machine. So depending on how your build/pipeline system works, you might not have the same version for all of the different packages.

Copy link
Member

@joelverhagen joelverhagen Apr 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the version is not same for all of the different packages, how would the top level tool package come to know about the list of (RID, version) tuples, i.e. to produce the DotnetToolSettings.xml file?

I see the example .csproj XML:

<ItemGroup>
	<ToolPackageRuntimeIdentifier Include="win-x64"/>
	<ToolPackageRuntimeIdentifier Include="linux-x64"/>
</ItemGroup>

This seems to be the non-NativeAOT case, where the whole graph (3 packages in this case?) is produced on one machine.

If the RID-specific packages are being produced on another machine, there seems to be a fan-out then collect type flow.

I'm trying to visualize how the differing version numbers will flow from one build step to the next. It feels like a bit of a mess if the version numbers are allowed to diverge. If we say "all version numbers must match" then there is no need for build steps to happen in a certain order or flow information (RID + selected version number for that RI) from one build step to the next. I see how it can be technically possible but I wonder what the .csproj will look like to handle the split-up, RID-specific build steps.

Copy link
Member Author

@dsplaisted dsplaisted Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primary package would need to be built after all of the rid-specific tool packages, so the version numbers from those builds would be inputs to the primary package build. I don't know how you'd flow the version numbers from one pipeline to another, but at the project level you could pass those versions as properties and it would look like this:

<ItemGroup>
	<ToolPackageRuntimeIdentifier Include="win-x64" Version="$(ToolPackageVersionWin64)"/>
	<ToolPackageRuntimeIdentifier Include="linux-x64" Version="$(ToolPackageVersionLinux64)"/>
</ItemGroup>

I think it's a good idea for us to support this if people need it. If they don't then all the versions can match, and everything is simpler. But if they don't have a good way to have the build numbers on different architectures all match, then it is good to have an escape hatch for them.


Note that NativeAOT is not generally supported for target platforms other than the current platform. So creating a NativeAOT .NET Tool will involve building the RID-specific packages on separate machines and building a primary package that refers to them.

Packing a project with `PackAsTool` set to true will pack the publish output. However, for NativeAOT, the `_IsPublishing` property needs to be set to true. This is normally set to true when invoking `dotnet publish`. We may need to add an `_IsPacking` property that is set to true when running `dotnet pack`, and have logic in the SDK to set `_IsPublishing` to true if `PackAsTool` and `_IsPacking` are both set to true.