running CUDA on mac

[vinayakarsh]

im new to CUDA.. how to run CUDA programs on mac

[vinayakarsh]

duplicate of #220 (comment)

[karpathy]

You don't, you'd have to reach for Metal. This is not currently "in scope" for this repo, but maybe one of the notable ports is a place to look.

[vinayakarsh]

Thank you

[regrettable-username]

@vinayakarsh
I'm working on a fork that's using Metal over at https://github.com/regrettable-username/llm.metal I've got the forward pass working. I'm in the process of implementing the backward pass. I've been taking a bit of a detour to set up some testing infrastructure similar to what's in dev/cuda in this repo. This will help benchmark and experiment with kernels for each layer as well as implement the layers for the backward pass.

There is also Accelerate.framework which might be used to get things running pretty quick on the CPU side. If you're on Intel based mac, that would definitely be a worthwhile pursuit!