Add GPU-STUMP

[seanlaw]

According to these CuPy examples, it may be pretty straightforward to port the stumpy._stump function over to CuPy code. Specifically, the k-means example shows how to mix a Python for-loop with a CuPy GPU function call.

• Try CuPy
• Google Colab example
• Add to Tutorial 1
• Add unit test

[seanlaw]

From an earlier comment by @mrocklin that I am now re-reading

With the array_function protocol recently landed in NumPy I suspect that many of the operations in core.py would just work without code change. Some operations though, like rolling_window, probably wouldn't (my guess is that cupy doesn't support stride tricks). For this one could probably use Numba to write a CUDA kernel in a small number of lines, and then use something like functools.singledispatch to switch between implementations. I think that this would satisfy the concerns about maintainability above. There wouldn't be much GPU code, and that code would be both readable-ish and well isolated from the mainline CPU code.

Taking a step back though, @seanlaw are users of stumpy currently compute bound by anything to the point of frustration? If so, what operations do they care about?

From my perspective this codebase is nice in that it's fairly small and well formed around a core set of operations, most of which are "just numpy". It would be an interesting case study to see how light a touch one could have in GPU-ifying a codebase, and in the community concerns that come up during that process (@seanlaw 's list above is great).

@mrocklin @datametrician @quasiben It's been 1.5 years since I looked at CuPy but, given its recent developments, I just took another look at the CuPy examples (namely, the k-means example) and I think I see a clearer path to porting over the stumpy._stump function to leverage CuPy.

Over the next few weeks, I will try to find some time to reinstall the OS and drivers on the TD Ameritrade GPU workstations. Thanks for pointing me in the right direction!

[datametrician]

We also have a multinode, multi-gpu (MNMG, read as, just scales with Dask) K-means that will be a bit faster being released in RAPIDS 0.9.
Fully integrates with Dask. Will be a bit faster, and fully follows SKL if you want to replace the CPU and GPU side with SKL API for K-means.

[seanlaw]

@datametrician So, what we'll need to implement isn't K-means but the K-means example code demonstrates how to mix CuPy code along with a Python for-loop (which is what I need). In doing too much work in NumPy, I think I assumed that everything had to be NumPy/CuPy like vectorized code. However, it is clearer to me now that we can use the Python for-loop to dynamically execute different CuPy code that'll run on the GPU. I think I had a mental block that was unblocked by seeing the k-means example and, apparently, I am a "learn-by-example" kinda person.

[geoHeil]

@seanlaw do you have an example at hand how to call out from RAPIDS / cudf to stumpy?

[geoHeil]

Or do I understand correctly that the regular stumpy.stump should work just fine on a cu-df/series instead of a numpy array (i.e. without moving the data over to the CPU and then over to the GPU again).

[seanlaw]

@geoHeil We don't have support for cudf yet (but I can see how this may be useful). Currently, stumpy.gpu_stump takes a NumPy array as input, performs some pre-computations on the CPU side, and then moves it over to the GPU device. Just know that STUMPY does have to perform some initialization work on the CPU side but then the rest of the computation is performed on the GPUs.