Forward compat scheme

[hvr]

Generally, a given cabal-install release is only expected to understand (cabal-)spec(ification)-versions that were known at the time of release.

The Cabal-1.12 release, and more recently the Cabal-2.0 release have made it apparent that we don't have a good forward-compatibility story in place, thereby limiting our ability to evolve the .cabal format. Concretely, cabal-install-1.10 is not expected to understand package meta-data declared using the cabal-version:1.12 spec(ification)-version; also, upon encountering such a future .cabal file in the package index, cabal-install-1.10 ought to gracefully recover (instead of failing fatally to parse the package index). A similar situation occurred with the Cabal-2.0 release, which also introduced new syntax which previous parsers would fail to recognise, again resulting in fatal parsing failures.

To prevent this from happening again, the following forward compatibility has been devised, which is intended to make it possible to evolve the lexical and syntactical structure of the .cabal format in future, while not breaking legacy clients.

---

New-style Cabal-Specification Version Declaration

A new-style spec-version declaration at the beginning of a the .cabal file (without any preceding whitespace) and follow the following case-insensitive grammar (expressed in RFC5234 ABNF):

newstyle-spec-version-decl = "cabal-version" *WS ":" *WS newstyle-spec-version *WS

newstyle-spec-version      = NUM "." NUM [ "." NUM ]

NUM    = DIGIT0 / DIGITP 1*DIGIT0
DIGIT0 = %x30-39
DIGITP = %x31-39
WS = %20

The use of a new-style spec-version is

• invalid before spec-version 1.12,
• optional starting with spec-version 1.12,
• recommended starting with spec-version 2.0, and
• mandatory starting with spec-version 2.1.

It's also assumed that the following invariant holds:

• If a new-style spec-version declaration is present, its value must match the spec-version as determined by a full .cabal parser.

Scanning the Cabal-Specification Version

1. If a .cabal file contains a valid new-style spec-version declaration, it is authoritative;
2. otherwise, the spec-version must be below 2.1 (and either a full .cabal parser or a heuristic cabal-version-scanner (tbd) must be used to determine the exact spec-version).

The new-style spec-version declaration is designed to be simple to parse by means of common string operations. A simple implementation is shown below

scanSpecVersion :: ByteString -> Maybe Version
scanSpecVersion bs = do
    fstline':_ <- pure (BS.Char8.lines bs)

    -- parse <newstyle-spec-version-decl>
    -- normalise: remove all whitespace, convert to lower-case
    let fstline = BSW8.map toLowerW8 $ BSW8.filter (/= 0x20) $ BS.toStrict fstline'
    ["cabal-version",vers] <- pure (BSS.split ':' fstline)

    -- parse <spec-version> tolerantly
    ver <- simpleParse (BSS.unpack vers)
    guard $ case versionNumbers ver of
              [_,_]   -> True
              [_,_,_] -> True
              _       -> False

    pure ver
  where
    -- | Translate ['A'..'Z'] to ['a'..'z']
    toLowerW8 :: Word8 -> Word8
    toLowerW8 w | 0x40 < w && w < 0x5b = w+0x20
                | otherwise            = w

Appendix: Compatiblity with clients prior to cabal 2.0

Since this new scheme is only understood properly starting with cabal 2.0, older clients need to avoid being exposed to spec-versions 2.0 and newer.

To this end we exploit that cabal 2.0 started using the incremental secure 01-index.tar package index by default, while cabal versions prior to cabal 2.0 use the (non-incremental/non-secure) 00-index.tar package index by default (NB: cabal 1.24 was the first release that added experimental non-default/opt-in support for the secure 01-index.tar), by establishing the following hackage-side invariant:

• the legacy index 00-index.tar.gz contains only .cabal files with a spec version below 2

This way, the huge install-base of legacy cabal clients prior to cabal 2.0 keep working without requiring modifications, as they won't be exposed to incompatible .cabal files; with the unfortunate exception that the (hopefully uncommon) cabal clients prior to cabal 1.12 may still be exposed to incompatible cabal versions using the >=-less spec-version declarations.

[23Skidoo]

Related: #4448.

[gbaz]

This seems like a good way forward.