Some Ideas to get GuitarML into 2022

[pgarz]

Firstly, let me introduce myself. I'm Pedro Garzon, professionally an ML engineer at a startup doing AI for scientific simulations. I have a background in AI from Stanford as well. More importantly, I've had the same idea of making a neural guitar pedal before. So glad someone started the groundwork foundations! I've spent most of my free time over quarantine just practicing guitar and exploring tones. I've been catching up on DL for audio and I really think 2022 can be a break-out year to making a supremely affordable neural pedal. I've got some ideas

Here are some technical infrastructure ideas

1. Build PyTorch support

• PyTorch has the capability of re-writing models trained with Python into compilable C++. This can be a huge boost for not having to manually re-write all models in raw C++
• One could also convert PyTorch to ONNX and run the models on the ONNX C++ runtime
• Port existing models into Pytorch and verify they still work
• It appears one could load PyTorch directly on a Rasberry Pi, but the hardware costs go up considerably https://qengineering.eu/install-pytorch-on-raspberry-pi-4.html

2. Build-in support for M1 chip training

• It seems the M1 series of chips have a bright future in audio production: https://www.waves.com/apple-m1-for-music-producers
• It's already possible to run Pytorch directly on M1 chips https://towardsdatascience.com/yes-you-can-run-pytorch-natively-on-m1-macbooks-and-heres-how-35d2eaa07a83
• Leveraging M1 computation could be a computational breakthrough and squeezing real-time performance out of more complicated network architectures

3. Support for on local computer recording

• We're quite limited by the current memory/CPU capabilities of the RasberryPi setup. So it would be ideal to also have Pytorch models be capable of running locally on the host computer running a DAW for recording
• There also might be a way to use the RasberryPi as the main interface for the guitar. So we could have a chain going guitar -> RasberryPi -> host computer for processing -> back to RasberryPi -> real-world amp

3. Logic Pro X support

• I personally use Logic, so would love to use my DAW of preference

And here are some very brief avenues of exploring richer network architectures:
Transformers, VQ-VAE, WaveNet-like architectures, Diffusion models, Fourier Transform tricks

All of which have PyTorch implementations!

Eager to hear feedback! Def the first step is point number 1. for all my ideas