diff --git a/RELEASENOTES.md b/RELEASENOTES.md index 76ef9d26..0d8664c5 100644 --- a/RELEASENOTES.md +++ b/RELEASENOTES.md @@ -232,3 +232,7 @@ non-constant expressions (fixes SIMICS-13113). - `release 6 6363` - `release 7 7064` +- `note 6` The `port-dml-module` script has been extended to support converting + multiple modules at once. Doing so also improves how conditional code within + common files is converted. The script also has a new flag `--compat`, which + mirrors `port-dml --compat`, further improving how common code is converted. diff --git a/doc/1.4/port-dml.md b/doc/1.4/port-dml.md index 83c4633f..2d68d9ea 100644 --- a/doc/1.4/port-dml.md +++ b/doc/1.4/port-dml.md @@ -5,25 +5,39 @@ # Porting DML 1.2 to DML 1.4 When porting a DML 1.2 file to DML 1.4, most differences can be taken -care of by the automatic conversion script `port-dml`. The -script relies on the `dmlc` compiler to produce information on -what changes need to be applied. +care of by the automatic conversion script `port-dml`. -The easiest way to invoke `port-dml` is through a wrapper -script `port-dml-module`. The script ports all devices in one -SIMICS module, and all imported files it depends on. The scripts works by -invoking `make` and `port-dml`, and prints how they are invoked, which -is instructive for understanding how to use `port-dml` -standalone. +The porting process consists of two phases: An *analysis phase*, where the +`dmlc` compiler is run to produce a *tag file*, which describes the required +changes; this is followed by a *conversion phase* where the `port-dml` +script reads the tag file and applies changes. -The `port-dml-module` script works well for converting most DML -devices, but has some limitations when it comes to common code that is -shared between many devices. In particular, if some parts of common -code is unused, e.g. if a provided template is never instantiated, -then it may skip some conversions. For this reason, it can be better -to use `port-dml` directly for common code. +The easiest way to perform conversion is through the wrapper script +`port-dml-module`. This script automatizes the porting process by first running an +analysis step using `make` to build one or more modules, and then running a +conversion phase by applying `port-dml` on the DML files that were compiled. +During the analysis phase, if some code is shared between multiple modules, +then it often happens that some unused code is discarded by the compiler before +it has been fully analysed. This happens in particular for unused templates and +top-level `if` statements. Such unused code will only get basic syntactic +conversions, like transforming `parameter` to `param`, and miss conversions +that depend on semantic analysis, like transforming `read_access` +to `read_register`. This +can be solved by including multiple modules in the analysis step; the +conversion phase for common code will then combine the analyses of all modules, +and utilize semantic analysis from any code paths that is active in *any* of the +included modules. +The `port-dml-module` script relies on +rather crude heuristics which often may be +incorrect; for this reason, the script also prints exactly how it invokes the +lower level `make` and `port-dml` commands. This allows each step to be +individually rerun manually with tweaked settings. + +The `port-dml` script can also be used directly without `port-dml-module`; this +mode of operation has a steeper learning curve but provides greater control +which can be advantageous when porting a large code base. ## Using the port-dml script In order to port a DML 1.2 file to DML 1.4, first pass the -P @@ -54,7 +68,7 @@ If you build your device from a Simics project, you can use the variable be set to the absolute path of a file; `make` will pass that in the -P flag to `dmlc`. Note that if you want to re-run an analysis, then you need to first run make clean-module to force -DMLC to re-run on all devices in the SIMICS module. +DMLC to re-run on all devices in the Simics module. If parts of the device source code is unused, e.g. if a template is never instantiated within the device, then DMLC can not perform a full @@ -75,3 +89,99 @@ tags](changes-auto.html) and apply the change manually. +## Porting common code still used from DML 1.2 code + +When porting a large code base to DML 1.4, you likely want to work +incrementally, porting some devices at a time. It can then happen that some of +your newly ported 1.4 files share common code with devices that are still in DML 1.2. + +It is not allowed to import a DML 1.2 file from a DML 1.4 device, but a DML 1.2 device may import a DML 1.4 file with some caveats. Thus, any code common between DML 1.2 and 1.4 must be ported to 1.4 before any device can be converted. +There are two possible strategies for this: Either convert the common file in place, +or duplicate it into separate 1.2 and 1.4 versions. + +### Keep a separate DML 1.4 copy + +After letting conversion tools convert `foo.dml` to DML 1.4, you can rename the +converted file into `foo-dml14.dml`, and restore the original 1.2 version as +`foo-dml12.dml`, and finally create a trampoline file `foo.dml` containing: + +``` +dml 1.4; +#if (dml_1_2) { + import "foo-dml12.dml"; +} #else { + import "foo-dml14.dml"; +} +``` + +This way, existing `import "foo.dml";` statements from both DML 1.2 and 1.4 devices +will continue to work. + +The apparent downside of this approach is that the logic of the common code is +duplicated across two files, which is a problem if the DML 1.2 variant is expected +to be maintained over a longer period of time. However, if all uses from DML 1.2 of +the common code are expected to be ported within a short migration period, +then this is likely the preferred approach. + +After the last DML 1.2 use of the common code has been ported, `foo-dml12.dml` can be removed, and `foo-dml14.dml` can be moved back to `foo.dml`, overwriting the trampoline. + +Note that the `#if` trick in the `foo.dml` trampoline above utilizes an +otherwise undocumented DML feature: DML normally doesn't allow `import` +statements within `#if` blocks, but a special exception was added to permit it +specifically within `#if (dml_1_2)`, in order to support this use case. + +### In-place conversion, preserving DML 1.2 compatibility + +A common file can be ported to DML 1.4 and still be useful from DML 1.2, with a +number of caveats. For instance, devices often implement functionality by +overriding standard methods, and some methods have been renamed between DML 1.2 +and 1.4. For instance, an override of the `read_access` register method in a +DML 1.2 device roughly corresponds to a `read_register` override in a DML 1.4 +device, and an attribute with `parameter allocate_type = "uint64"` in DML 1.2 +corresponds to an event with `is uint64_attr` in DML 1.4. Much of this can be +taken care of by the `dml12-compatibility.dml` layer: A shared DML 1.4 file can +say `import "dml12-compatibility.dml";`. This does nothing when imported from a +DML 1.4, but when imported from DML 1.2, it provides some glue that ties DML +1.4 constructs to the DML 1.2 API. For instance, it defines templates such that +`is uint64_attr` in the DML 1.4 file will expand to define `allocate_type` when +imported from DML 1.2. This file also provides some templates for explicit +instantiation. In particular, the `dml12_compat_read_register` template can be +instantiated on a DML 1.4 register that overrides the `read_register` method; +this has no effect in a DML 1.4 device, but in a DML 1.2 device it overrides +the DML 1.2 method `read_access` to call the provided override. Similarly, +the `dml12_compat_write_register` template can be used on registers that override +`write_register`; `dml12_compat_read_field` and +`dml12_compat_write_field` can be used on field that override the `read_field` or `write_field` method; and `dml12_compat_io_memory_access` can be used on banks that override the `io_memory_access` method. + +Sometimes, the facilities in `dml12-compatibility.dml` are not sufficient for +full DML 1.2 compatibility. For instance, suppose you want to use the `shared` annotation on a `read` method when writing the DML 1.4 version of a template. There are fundamental limitations in DML 1.2 that prevent such overrides. This can be overcome with an `#if (dml_1_2)` block on the top level: +``` +dml 1.4; + +#if (dml_1_2) { + template read_twelve { + method read() -> (uint64) { + log info: "read"; + return 12; + } + } +} #else { + template read_twelve is read { + shared method read() -> (uint64) { + log info: "read"; + return 12; + } + } +} +``` +This is somewhat similar to the `foo-dml14.dml` trampoline approach discussed above, with +the difference that it can be applied selectively only on problematic parts of the +file. + +If the flag `--compat` is passed to the `port-dml` script, then the script will +automatically detect some cases where similar `#if` clauses are needed for +compatibility, and insert them automatically. The script will also add an +`dml12-compatibility.dml` import. The `--compat` flag can also be passed to the +`port-dml-module` script; in this case, the script will pass on `--compat` to +`port-dml` when converting DML files that don't reside in the directory of any +of the ported modules.