Support asconfig.json in addition to CLI arguments

[jtenner]

In an attempt to standardize the way the vscode language server works, we should probably adopt a configuration file definition.

Please file suggestions and discussion on this topic here.

I plan on starting work on this tomorrow in a pull request.

[dcodeIO]

This reminds me that #1134 is somewhat related in that it proposes an asbuild cli utility, with profiles for debug, release or custom. Perhaps there are ways to make these two work well together?

Another nice touch might be to allow the contents of asconfig.json to be, as a fallback if it's not present, part of tsconfig.json as the assembly key. How useful that'll be ultimately depends a bit, but since we have portable AssemblyScript that compiles to both JS and Wasm, which most likely won't go away, it seems good to have.

[jtenner]

EDIT: I will try to keep this updated with the latest iteration as per suggestions.

type EnableFeatures = "sign-extension" | "bulk-memory" | "simd" | "threads" | "reference-types";
type DisableFeatures = "mutable-globals";
type TrapMode = "js" | "clamp" | "allow";
type BinaryenPasses = "AlignmentLowering" |
  "asyncify" |
  "avoid-reinterprets" |
  "coalesce-locals" |
  "coalesce-locals-with-learning" |
  "code-folding" |
  "code-pushing" |
  "const-hoisting" |
  "dae" |
  "dae-optimizing" |
  "data-flow-opts" |
  "dead-code-elimination" |
  "directize" |
  "dward-dump" |
  "duplicate-import-elimination" |
  "duplicate-function-elimination" |
  "emit-target-features" |
  "extract-function" |
  "flatten" |
  "func-cast-emulation" |
  "full-printer" |
  "function-metrics" |
  "generate-stack-ir" |
  "i64-to-i32-Lowering" |
  "inline-main" |
  "inlining" |
  "inlining-optimizing" |
  "legalize-js-interface" |
  "legalize-js-interface-minimally" |
  "limit-segments" |
  "local-cse" |
  "log-execution" |
  "instrument-locals" |
  "instrument-memory" |
  "loop-invariant-code-motion" |
  "memory-packing" |
  "merge-blocks" |
  "merge-locals" |
  "minified-printer" |
  "minify-imports" |
  "minify-imports-and-exports" |
  "minify-imports-and-exports-and-modules" |
  "metrics" |
  "name-list" |
  "no-rxit-runtime" |
  "optimize-added-constants" |
  "optimize-added-constants-propagate" |
  "optimize-instructions" |
  "optimize-stack-ir" |
  "pick-load-signs" |
  "mod-asyncify-always-only-unwind" |
  "mod-asyncify-never-unwind" |
  "post-assemblyscript" |
  "post-assemblyscript-finalize" |
  "post-emscripten" |
  "precompute" |
  "precompute-propagate" |
  "printer" |
  "print-call-graph" |
  "print-features" |
  "print-function-map" |
  "print-stack-ir" |
  "relooper-jump-threading" |
  "remove-non-js-ops" |
  "remove-imports" |
  "remove-memory" |
  "remove-unused-brs" |
  "remove-unused-module-elements" |
  "remove-unused-non-function-module-elements" |
  "remove-unused-names" |
  "reorder-functions" |
  "reorder-locals" |
  "re-reloop" |
  "redundant-set-elimination" |
  "round-trip" |
  "safeHeap" |
  "simplify-locals" |
  "simplify-globals" |
  "simplify-globals-optimizing" |
  "simplify-locals-no-nesting" |
  "simplify-locals-no-tee" |
  "simplify-locals-no-structure" |
  "simplify-locals-no-tee-no-structure" |
  "strip-debug" |
  "strip-dwarf" |
  "strip-producers" |
  "strip-target-features" |
  "souperify" |
  "souperify-single-use" |
  "spill-pointers" |
  "ssaify" |
  "ssaify-no-merge" |
  "trap-mode-c" |
  "trap-mo" |
  "untee" |
  "vacuum";

interface ASConfig {
  options?: {
    optimizeLevel?: 0 | 1 | 2 | 3;
    shrinkLevel?: 0 | 1 | 2 | "s" | "z";
    converge?: boolean;
    noAssert?: boolean;
    runtime?: "full" | "half" | "stub" | "none";
    debug?: boolean;
    importMemory?: boolean;
    sharedMemory?: boolean;
    importTable?: boolean;
    exportTable?: boolean;
    explicitStart?: boolean; 
    enable?: EnableFeatures[];
    disable?: DisableFeatures[];
    use?: {
      [key: string]: string | number;
    };
    memoryBase?: number;
    tableBase?: number;
    validate?: boolean;
    trapMode?: TrapMode;
    runPasses?: BinaryenPasses[];
    baseDir?: string;
    noUnsafe?: boolean;
    noEmit?: boolean;
    measure?: boolean;
    pedantic?: boolean;
    lib?: string[];
    path?: string[];
    traceResolution?: boolean;
    printrtti: boolean;
  };
  output?: {
    outFile?: string;
    binaryFile: string;
    textFile?: string;
    asmjsFile?: string;
    idlFile?: string;
    tsdFile?: string;
    sourceMap?: boolean | string;
  };
  entry?: string[];
}

We can use this as a starting point. I went through the CLI help and generated an asconfig.json shape.

Any suggestions?

[dcodeIO]

Nice! A few comments:

• sourceMap?: boolean is a string. Relative source map if specified without an argument (--sourceMap, string empty), otherwise absolute source map using the specified URL (--sourceMap URL). Perhaps this can be improved to also take a true instead of an empty string.
• Binaryen passes are lowercase and use dashes, or do we postprocess these from uppercase camelcase?
• What do these do? include?: string[]; exclude?: string[]; extends?: string[];

[jtenner]

sourceMap?: boolean is a string. Relative source map if specified without an argument (--sourceMap, string empty), otherwise absolute source map using the specified URL (--sourceMap URL). Perhaps this can be improved to also take a true instead of an empty string.

See edit, switched to boolean | string.

Binaryen passes are lowerdashcase

Got it. Will change it.

What do these do? include?: string[]; exclude?: string[]; extends?: string[];

All of these were copied from the idea of what the tsconfig does, but we have entry points, so instead I replaced them with an entry?: string[] property.

Perhaps there is no need for asconfig extension.

[dcodeIO]

Have been brainstorming with @jtenner on Slack, and this was the latest we came up with:

Let every project have an asconfig.json in its main directory. Like TypeScript, it has a compilerOptions entry specifying the options used for compilation:

{
  "compilerOptions": {
    ...
  }
}

The configuration can extend another configuration using the extends key, using node resolution:

{
  "extends": "./path/to/other/asconfig.json"
}

Now, while one doesn't typically have the need for multiple builds in TypeScript, it is fairly common to build two variants in AssemblyScript. To make this easier and reduce the amount of files one has to deal with, extending this with multiple targets seems convenient:

{
  "compilerOptions": {
    // common options used by all targets
  },
  "targets": {
    "release": {
      // additional release options
    },
    "debug": {
      // additional debug options
    }
  }
}

This feature would be optional, in that if the targets key is not present, just compilerOptions will be used and function just as one would expect from TypeScript (can have multiple files, one per target if one so wishes). Iff the targets key is present, the default target to build would be the first specified (here: release), issuing a warning on the console "No target specified: Defaulting to 'release'".

On the command line, one would either specify just the config file via --config ./path/to/asbuild.json or, if targets is present, --config ./path/to/asbuild.json:release to execute a single one, --config ./path/to/asbuild.json:release,debug to execute multiple or --config ./path/to/asbuild.json:* to execute all. Similarly, if such a config is extended, one would either

{
  "extends": "./path/to/asconfig.json"
}

or, if targets is present,

{
  "extends": "./path/to/asconfig.json:release"
}

If targets is present and the former is used, so a default target is executed, a warning on the console should read "No target specified for './path/to/asconfig.json': Defaulting to 'release'.".

Thoughts? The obvious alternative would be to omit the targets mechanism, but then tooling might have problems determining which asconfig.x.json is the correct one, unless the default one should always be named asconfig.json with additional ones being possible?

[jtenner]

Specifying the first "key" as the target is undefined behavior in js. Object keys are not garunteed to be ordered.

Suggesting to use an Array instead, and always use the first one.

[willemneal]

Typescript currently allows the extend to use node package resolution.

[willemneal]

Also it would be nice to have nested projects so that multiple binaries can be generated. This is particularly of interest to Near with projects with multiple smart contracts.

[jtenner]

@dcodeIO I am suggesting this:

{
  "options": {},
  "targets": [
    {
      "name": "release",
      "options": { ... }
    }
  ]
}

You cannot default to the "first" one, if it's only object keys. Can we resolve this first before I start work?

[dcodeIO]

Hmm, with that, we could even do something fancy like:

{
  "targets": [
    {
      "name": "release",
      "extends": "assemblyscript/presets:release",
      "options": { ... }
    },
    {
      "name": "debug",
      "extends": "assemblyscript/presets:debug",
      "options": { ... }
    }
  ]
}

but raises the question whether we still need top-level options?

[jtenner]

Again. Why is it so inconvenient or wrong to just have single asconfig files per target?

asconfig.release.json #extends asconfig.json
asconfig.debug.json #extends asconfig.json
asconfig.json # default options

[dcodeIO]

My thinking there is that one typically has two builds per project, like one debug and one release. Without targets, it is strictly necessary to have multiple files, like

asconfig.base.json
asconfig.debug.json (extends base)
asconfig.json (extends base, implicit release, default)

where it seems just too easy to overlook something due to having to switch through the files, instead of having a clear view on all the options at once.

Then there is

Also it would be nice to have nested projects so that multiple binaries can be generated

which is impossible to do (in a single command) without targets, since every config makes exactly one binary.

[jtenner]

Right. I'm arguing for simplicity and leaving the complexity for end users.

I am now proposing a recursive configuration algorithm and exactly two cli options with defaults specified below.

--config (path) 
--target (name) 

# or

--config (path)(:name?) 

Steps:

1. Load config
2. If the config extends another config, load it and resolve it first using this algoirthm
3. Override default and parent options at this level, but do not override cli provided options
4. Override options with the first target entry or by the given name (if provided, not overriding the cli passed options)
5. Return the cliOptions object

[jtenner]

I'm sorry this is taking so long. Encapsulating this algorithm feels quite complicated. 😅