Description
What is the URL of the page with the issue?
It's the worker, not frontend
What is your user agent?
Moot for this FR
Screenshot
No response
What did you do?
Ran my own pkgsite worker.
What did you see happen?
It didn't do any work unless I baggered it with /poll and /enqueue.
What did you expect to see?
It should support work queues other than Google Cloud tasks.
At Netflix, I've built a golang-index that indexes our private SCM. I'm running pkgsite on top of this index. It works well, except for the fact that I have to badget the worker to do work with /poll and /enqueue. We use AWS, not GCP, so I can't plug it up to Google Cloud Tasks, which presumedly would allow the worker to work from a work queue.
It'd be nice if we support an in-memory work queue. I realise that if you have multiple workers, there's no coordination between the in-memory queues, but it'd be a good place to start (in our <2000 repos, we are fine with 1 worker).
It'd be even nicer if we supported a simple postgres work queue. Maybe something simple like a work table with CREATE TABLE <work id - maybe scm,org,repo,tag tuple?> STRING, last_worker_started TIMESTAMP, last_worker_finished TIMESTAMP.
FWIW I'm happy to work on this - certainly the in-memory version.