Bindless Tracking Issue

[kanerogers]

Update

Lots of spec work has happened in the upstream WebGPU API for bindless. See #8557 for information about the current angle that the spec is looking at.

Overview

This issue tracks enabling various "bindless" functionality for the various native backends.

For a high level guide on what we believe the bindless api should look like, look at https://hackmd.io/@cwfitzgerald/wgpu-bindless

Binding Array Support

• Implement Descriptor Array Extension #711
• Implement SAMPLED_TEXTURE_ARRAY_NON_UNIFORM_INDEXING #715
• Support BindGroups with fewer elements in Binding Array than in BindGroupLayoutEntry #1635
• Use Argument Buffers for binding_array on Metal #3334
• Support binding_array of Storage Buffers on Metal #6741
• Support binding_array of Storage Textures on Metal #6744
• Support binding_array of Storage Buffers on DX12 #6739
• Support binding_array of Storage Textures on DX12 #6743
• Use UPDATE_AFTER_BIND Descriptors for binding_arrays #6737
• No feature combination allows Metal to use STORAGE_RESOURCE_BINDING_ARRAY. #3249

Uniform Buffers

• Allow binding_array of Uniform Buffers Through Validation #7149
• Support binding_array of Uniform Buffers on DX12 #6740
• Support binding_array of Uniform Buffers on Metal #6742
• Binding Array of Uniform Buffer Improperly Annotated Block #6733

Partially bound descriptors

• Support BindGroups with fewer elements in Binding Array than in BindGroupLayoutEntry #1635
• Partially bound descriptors on DX12 #3639
• Partially bound descriptors on Metal #3640

Validation

• BindGroupLayoutEntry::count Validation Does Not Check That Shader Uses Binding Arrays #3648
• Array binding provided zero elements validation error when using PARTIALLY_BOUND_BINDING_ARRAY #3692
• Texture Binding Array not Validated Against Flag #5102
• Disallow Binding Arrays and Dynamic Offsets in Same Bind Group #6736
• Split binding_array Limits into Separate Limit  #6738

GPU Validation

• Validate Binding Array Access is in Bounds on Vulkan
• Validate Binding Array Access is in Bounds on Metal
• Validate Binding Array Access is in Bounds on DX12

Sparse Bind Groups

• Implement Validation Resource Metadata Buffer on Vulkan
• Implement Validation Resource Metadata Buffer on Metal
• Implement Validation Resource Metadata Buffer on DX12

Mutable Bind Groups

• Implement BindGroup::update_bindings Without Holes

Read Only Resources

• Implement Texture::set_usages
• Implement Buffer::set_usages

Temporary Removal

• Implement prototype temporary remove api. Depends on "Sparse Bind Groups"

Non-wgpu Shader Translation

• Naga WGSL->SPIR-V unsized binding arrays create Vulkan validation warnings outside wgpu #7347

Driver Bugs(?)

• MoltenVK Reports binding_array of Storage Buffer is Incorrectly Validated #6745

[John-Nagle]

Bindless mode means you have a big table of all the texture objects in the GPU. Each of those texture objects is a GPU buffer. Buffers either belong to the GPU, and the CPU can't touch them, or they belong to the CPU, and the GPU can't touch them.

So the big table of texture object descriptors and the GPU buffer states have to be kept in sync. At the Vulkan level, this problem belongs to the program calling Vulkan. Vulkano, which is supposed to be a safe Rust interface to Vulkan, has machinery for maintaining that table. But WGPU doesn't have that machinery, so it can't do bindless yet, at least not safely.

GPU/CPU buffer ownership and the bindless descriptor table need to be managed together.

All the targets WGPU currently supports seem to support bindless mode. Even OpenGL, since 2013, has offered it as an extension. You do need OpenGL 4 with extensions, though. Everything in current release seems to have full support now, via Vulkan, Metal, or OpenGL.

The future is bindless. Unreal Engine is now bindless-only, I think.

[SK83RJOSH]

Unreal Engine is now bindless-only, I think.

It is not, in fact. Bindless resources are largely experimental and only enabled by default under Vulkan. They are optionally enabled under DX12 when using SM6 + ray tracing. There's still ongoing work to add support for bindless resources to shader graph as well.

Aside from that, it seems very much like there are targets that they're interested in that do not support bindless. So many materials will likely still need to implement both paths.

[cwfitzgerald]

I have filled out the above issue with the current plan for bindless, and what work has been done previously

[John-Nagle]

Right. I see more of the problems now. Looking at Bindless Investigation and Proposal, it's clear that driver-controlled residency and big arrays of bindless descriptors do not play well together. This is a non-problem for Vulkan, where all assets must be resident in GPU memory. For Metal, there is a residency control API.. Not sure what the plan is for WebGPU, since that's still being defined.

I look at this from the viewpoint of needing game-type performance on large scenes, for target machines comparable to what the average Steam user has. In my own applications, I'm managing residency at the application level, where I switch textures to lower resolutions when memory is tight. Rejection of a buffer allocation request is a normal event which results in LOD reductions. For the Vulkan case, this substitutes for driver initiated eviction.

I've mostly looked at the Vulkan case, where VK_DESCRIPTOR_BINDING_UPDATE_AFTER_BIND_BIT is available and one big array of descriptors is possible. When everything is resident, that leads to simple implementations which don't require barriers during drawing, while allowing concurrent updating. I've written some design materials on that. The basic concept is that the wrapper level (WGPU) takes ownership of a buffer when it is in the descriptor table, which protects it from being deleted while rendering is in progress.

The cram job required to run on smaller targets makes things far more complicated. Driver-initiated eviction really complicates things. Can that be handled without crippling performance on the more powerful targets?

[cwfitzgerald]

To clarify, none of these apis deal with residency at all . The assumption is that all resources that are bound to bind group are resident always. Yes we need to prevent metal from making resources non-resident, but with residency sets this should be easy.

When everything is resident, that leads to simple implementations which don't require barriers during drawing, while allowing concurrent updating

Barrier generation has basically nothing to do with residency. The purpose of the buffer.set_usages API, is to allow wgpu to not need to check if every single resource needs a barrier. Those resources with only read only usages can be ignored and just bound indirectly as part of the binding array

[John-Nagle]

OK, if everything is resident, things are simpler.

So, as you see bindless, what has to be done on a per-draw basis? That's where the overhead comes from. Is there checking that has to be done at each draw, or can the checking be hoisted to once per frame, or once per texture change, or to compile time?

[John-Nagle]

If an allocator is needed for descriptor slots, feel free to use this one I wrote. It's lock-free.