Plans for async vs blocking IO

[timbertson]

This may be a premature discussion, but I'm very interested in what the plan is for blocking IO (or its absence).

Once async IO support is merged in and rock solid on all compile targets, the big question is whether there are two versions of every IO function - one that blocks the thread (as IO does today), and one that's async. Or, whether all the current IO functions (and sleep() etc) are rewritten to be fully async, and we adopt a rule of never adding a blocking IO function to the stdlib.

I think it'd be ideal if we rule out blocking IO entirely, so that there's no choice. The effect system should mean that there is no real usability impact to users, and it's a fantastic proposition to know that it's simply impossible to block the current thread on IO (outside of FFI).

So in my mind it's a no-brainer, I would want everything to be async. I'm coming at it from a usability perspective though, I'm interested to hear the counter-arguments?

Prior art:

One of Go's most powerful aspects (IMO) is that there's no such thing as blocking the current thread. All IO and waiting is async, and that removes one big way to screw things up (in terms of performance and thread starvation).

NodeJS is similar - there exist a few doSomethingSync() versions of various IO operations, but they're frowned upon aside from quick & dirty scripts, essentially everything in apps/libraries will be async. In the early days the main appeal of these sync versions was to avoid writing callbacks, though I imagine async/await has removed most of that friction (I haven't used node for a long time).

Scala on the other hand has many problems with this, because most FP libraries use a (nonblocking) IO monad, whereas all of the Scala stdlib (and almost all Java libraries) happily block the thread and ruin your day. To workaround this requires the user to be careful and constantly aware of the distinction. Libraries like linebacker exist because of this problem (but it's still up to you to be aware of where blocking can happen).

[zephyrtronium]

My intuition is that it should be possible to express the choice between synchronous and asynchronous I/O within the effect system, so that it is a matter of which handler is used. So, perhaps what I would want to see here is an effect like io but with extra ctls to enable scheduling. I think you've already done some work in that direction.

One of Go's most powerful aspects (IMO) is that there's no such thing as blocking the current thread. All IO and waiting is async, and that removes one big way to screw things up (in terms of performance and thread starvation).

Go has only blocking I/O, The approach there is to always let I/O block, but to separate the threads doing I/O from the threads doing work, which is a style enabled by threads themselves being very cheap.

Perhaps there is some confusion of terminology here, though. To me, "async" specifically means concurrency implemented in terms of continuation passing (or effect handling, which generalizes continuation passing, in the case of Koka). It seems like you're using it more to mean what I would call "concurrent."

[timbertson]

express the choice between synchronous and asynchronous I/O within the effect system, so that it is a matter of which handler is used

That's interesting, I hadn't considered that option. IO is currently implemented in the runtime (and that's on the cards for async with libuv), which I think means you can't use a different handler. I'm also not sure why you'd choose sync, if async is just as easy to use?

Perhaps there is some confusion of terminology here, though. To me, "async" specifically means concurrency implemented in terms of continuation passing (or effect handling, which generalizes continuation passing, in the case of Koka). It seems like you're using it more to mean what I would call "concurrent."

Perhaps - I'm thinking of async as "not being synchronous" at the CPU/thread level, regardless of how that's implemented. What I ultimately care about is whether IO operations block the current (OS) thread or not. If IO operations do block the OS thread, then they will interact poorly with async code, e.g:

interleaved { file1.read() } { file2.read() }

Assuming for simplicity that read is terribly inefficient and always takes 1 second, then:

• If read is synchronous, the example will take ~2s to complete
• If read is async (nonblocking), then the example will only take ~1s to complete

The first case is what I would like to systematically avoid, by not having any functions in the stdlib which block the current OS thread.

[TimWhiting]

Terminology is tricky. In a sense, Go doesn't block the OS thread, but it does block the current thread of execution (i.e. the goroutine). But that is the same for any code that depends on something not currently available, in that sense all code is blocking. However, one thing that Goroutines do that this does not necessarily need is schedule onto multiple OS threads.

I think that it is an interesting question whether to separate the scheduler from the async library. After thinking about it a bit, it seems like the only thing that async is doing is being a scheduler for an event loop. Scheduling can happen in other contexts too though. There are lots of ways to schedule dynamically suspended computations (from the perspective of a single OS thread). At the most basic level there is breadth-first and depth-first (which I experimented with for the nondeterminism effect to create a general search library here: https://github.com/koka-community/std/blob/rbtree/std/effects/search.kk). Then on top of that you can add priorities and other things. You can also do like goroutines and also take care of some OS thread pool to actually parallelize computation. This probably would work well in Koka due to purity (I think I've seen some examples with trees), but it'd be interesting to do some actual benchmarks that involve a realistic workload.

As far as your question about sync / async. Like you I don't see any reason for sync since it means blocking the OS thread from doing any useful work. If you have an OS thread pool scheduler, then it might make sense, since you can still proceed with other computation in parallel, but OS threads are more expensive. In simple cases like stat where you are just trying to get some file info (size, permissions), I can see why you would always use the sync version - it should just be a memory lookup for the OS and the overhead of system calls is less when you just use sync, and don't have to register yourself as a listener. Similar things could be said for small files, etc. Personally I don't think it makes sense to have two different file-system handlers one for sync and one for async. I think it would be better to have an api that is flexible to the needs of the user with both async and sync versions. The sync versions wouldn't even need to use the async handler - which removes the overhead of capturing the continuation and handling the event loop. I think async makes the most sense when we talk about network IO, or parsing a file in a streaming fashion. The file system also could be backed by a networked file system. Again, that is use-case specific knowledge that would tell you whether to use async or sync. I think that sleep should always be async for example, but there could be situations where you wish for the sync version because the sleep is short enough, and you don't want the overhead of capturing the continuation.

[TimWhiting]

By the way, I've been thinking of making that PR a community library, which wouldn't be dependent on Koka shipping it as part of the runtime or tying Koka to libuv. I think it would be interesting to also split it into the core async scheduler, and leaving the os-level callback management to backend-specific libraries which you could swap out. I know you did a bit of that work already.

However, I also know Daan would like to really optimize the async support when we merge it in, and make it really fast. I'm not sure how much of that can be done without dipping into the runtime a bit. I personally think that it could be just as fast as an external library due to Koka's great support for external C imports.