lua 5.3 or luajit support needed

[ilovezfs]

I have upgraded lua in Homebrew from 5.2 to 5.3 in Homebrew/homebrew-core#21300. This broke lua support, so mpv now needs

depends_on "lua@5.1"

In Homebrew, @ formulae for older versions are temporary, so depending on lua@5.1 is not a viable long-term solution for mpv.

Some possibilities for mpv in Homebrew:

1. mpv becomes lua 5.3 compatible and switches back to depends_on "lua"
2. mpv switches from depends_on "lua@5.1" to depends_on "luajit" (if that works?)
3. mpv builds without lua support
4. something else that makes people :)

[mia-0]

I think by now it is probably okay to depend on Lua 5.3 while dropping compatibility with older versions of the language.

[ilovezfs]

@lachs0r unfortunately mpv with lua 5.3 doesn't actually work yet otherwise that's exactly what we would do :)

[jeeb]

Luajit is already used by various people and should work (I'm pretty sure I build my Windows builds with it). 5.2 to 5.3 upgrade on the API side is not a simple task, so if you have luajit packaged you might just do that on your side.

If someone gets bored they can of course enable upstream 5.3 to work as well.

edit: I don't have any MacOS build VMs around right now so I can't say much about LuaJIT + mpv on MacOS. Just that in general it seems to work.

[Argon-]

Last time I tried: luajit doesn't work with mpv on OSX. wm4 once told me on IRC and I tried to confirm it, successfully: #1110 (comment) I didn't try to figure out the problem back then as I assumed there was a reason it is (still) broken although known.

A switch to 5.3 will also (most likely?) require modifications to all built-in lua scripts. And then we are still left with dozens of potentially incompatible user scripts: https://github.com/mpv-player/mpv/wiki/User-Scripts
I'm not sure how sever the language differences are to be honest.

[ilovezfs]

By default, upstream lua (and Homebrew) build 5.3 with the 5.2 compat option, so I don't think breaking the user scripts is an issue unless they're doing 5.1-only things.

[ilovezfs]

Last time I tried luajit doesn't work with mpv on OSX.

This appears to still be true. Not sure why that would be macOS specific?

With luajit

==> /usr/local/Cellar/mpv/0.27.0_4/bin/mpv --ao=null /usr/local/Homebrew/Library/Homebrew/test/support/fixtures/test.wav
[osc] Could not initialize Lua.
[osc] Could not load lua script @osc.lua
[ytdl_hook] Could not initialize Lua.
[ytdl_hook] Could not load lua script @ytdl_hook.lua
Playing: /usr/local/Homebrew/Library/Homebrew/test/support/fixtures/test.wav
 (+) Audio --aid=1 (pcm_s16le 1ch 8000Hz)
AO: [null] 8000Hz mono 1ch s16


Exiting... (End of file)
[ao/null] buffer underrun

vs. with lua@5.1

==> /usr/local/Cellar/mpv/0.27.0_3/bin/mpv --ao=null /usr/local/Homebrew/Library/Homebrew/test/support/fixtures/test.wav
Playing: /usr/local/Homebrew/Library/Homebrew/test/support/fixtures/test.wav
 (+) Audio --aid=1 (pcm_s16le 1ch 8000Hz)
AO: [null] 8000Hz mono 1ch s16


Exiting... (End of file)
[ao/null] buffer underrun

Lu 5.1, 5.2, and 5.3 are different languages. You can compare it to Python2 vs. 3, except with worse compatibility. 5.3 in particular adds an integer data type, which requires major changes both to API user and Lua scripts.

The often cited 5.2 compatibility in 5.3 doesn't seem to exist. There are some compatibility flags, which control the visibility of new and/or deprecated symbols, and a flag for something metamethod related. There is nothing that makes 5.3 run unmodified 5.2 scripts, nor anything that removes the need for major changes in API user code.

So this compatibility mode is a red-herring from what I can tell. But even if it weren't, we couldn't rely that everyone who builds mpv builds Lua with this compatibility enabled. It could even be that such a compatibility mode (which would be a hack from the Lua creator's point of view) would not conflict with legitimate Lua 5.3 API users and scripts, which would further decrease the chance of support.

Moreover, LuaJIT does not support 5.3 (apparently because the author dislikes the language changes). Switching to Lua 5.3 would mean dropping LuaJIT support for good.

Even if you ignore the script compatibility issue, supporting both 5.1/5.2/LuaJIT and Lua 5.3 would be a big mess.

There is a large number of user scripts: https://github.com/mpv-player/mpv/wiki/User-Scripts Certainly we don't want to brick them over night, but from what I can tell, the newly introduced integer type could cause random runtime failures because the scripts do not expect it. I don't know how bad it actually would be (there's a small chance all issues can actually be avoided), but at this point it just seems not worth supporting 5.3.

In fact I have yet to hear a good reason to support 5.3 (or even 5.2), other than that it's "newer". But be reminded again, 5.3 is not just a compatible maintenance release over 5.2 (as the version number might suggest), but a new language based on 5.2.

There is a bit of history about this issue. There is a small FAQ entry, at least 2 rejected PRs for adding 5.3 support, and probably a bunch of closed issues. We won't change just because an overzealous maintainer doesn't understand the issue and demands that we do things that were discussed and rejected years ago.

Anyway, as it stands, there are three choices:

• keep the current status (means everything works, but unhappy homebrew maintainers)
• add support for 5.3 without breaking 5.1/5.2 (this would mean user scripts randomly don't work if they were written for a different Lua version)
• drop support for 5.1./5.2/LuaJIT (probably would break most user scripts, but the situation would stabilize, until Lua 5.4 happens)

(LuaJIT does support OSX, but because of some OSX idiocy (typical Apple), a special build flag is needed: http://luajit.org/install.html (search for image_base). I think it's actually in pkg-config, but waf filters it out for god knows what awful reasons.)

[ilovezfs]

We won't change just because a smug maintainer doesn't understand the issue and demands that we do things that were discussed and rejected years ago.

Just so we're all clear: that person is me and I'm not demanding anything.

Well, making an issue that 5.3 support is "needed" sure sounds demanding to me. mpv will probably require 5.1/5.2 for a long time, so dropping support for those in homebrew also implicitly puts pressure on us to move to a different version of Lua (and, if removed knowingly, certainly implies a strong demand).

It seems you're also suggesting LuaJIT. That is merely a third party implementation of Lua 5.1, with improvements. I don't know if it's as portable as official Lua, but there sometimes are problems associated with it doing JIT (such as the OSX runtime problem).

[ilovezfs]

@wm4 what title would you suggest instead? I'm happy to change it.

[ilovezfs]

In fact I have yet to hear a good reason to support 5.3 (or even 5.2), other than that it's "newer". But be reminded again, 5.3 is not just a compatible maintenance release over 5.2 (as the version number might suggest), but a new language based on 5.2.

It seems you're also suggesting LuaJIT.

5.3 has been the current upstream version of Lua for nearly three years now, there hasn't been a new 5.2.x release in as much time, and there hasn't been a 5.1.x release in nearly six years.

By contrast, LuaJIT 2.0.5 and 2.1.0-beta3 were released in May of this year. So, yes, if Lua > 5.2 is going to be in the "never" column, looking toward a language implementation that's acceptable and currently maintained seems reasonable to me.

That said, according to our analytics for the last 30 days

Install events:
Lua: 46,962
LuaJIT: 16,715

Install-on-request events:
Lua: 6,517
LuaJIT: 1,143

So I would recommend trying to stay on the mainline Lua train if possible in terms of what users will likely be wanting to use over the longer term.

As I've been trying to tell you, 5.2 and 5.3 are different, incompatible (even if similar) languages. On some level that's like trying to argue that we should switch to JavaScript because JS is more actively developed.

I expect that users most likely want working user scripts. Also, nobody ever asked us to support Lua 5.3 because they wanted features that are only in 5.3. (Actually, it appears Lua is mostly hated by those who try to write scripts, but for some reason it seems it's still preferred over mpv's JS scripting support.)

Even if 5.3 and 5.2 are very similar, the change will brick user scripts, and that's not really acceptable. Note that I haven't really tried to quantify how much will break, so you're welcome to try to update mpv to use Lua 5.3 and demonstrate that it doesn't break most scripts (Lua 5.3 compiled without compat options, see my previous post why we wouldn't want those).