Skip to content

Extend port-dml documentation #367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions RELEASENOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,3 +232,7 @@
non-constant expressions (fixes SIMICS-13113).
- `release 6 6363`
- `release 7 7064`
- `note 6` The `port-dml-module` script has been extended to support converting
multiple modules at once. Doing so also improves how conditional code within
common files is converted. The script also has a new flag `--compat`, which
mirrors `port-dml --compat`, further improving how common code is converted.
142 changes: 126 additions & 16 deletions doc/1.4/port-dml.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,39 @@

# Porting DML 1.2 to DML 1.4
When porting a DML 1.2 file to DML 1.4, most differences can be taken
care of by the automatic conversion script `port-dml`. The
script relies on the `dmlc` compiler to produce information on
what changes need to be applied.
care of by the automatic conversion script `port-dml`.

The easiest way to invoke `port-dml` is through a wrapper
script `port-dml-module`. The script ports all devices in one
SIMICS module, and all imported files it depends on. The scripts works by
invoking `make` and `port-dml`, and prints how they are invoked, which
is instructive for understanding how to use `port-dml`
standalone.
The porting process consists of two phases: An *analysis phase*, where the
`dmlc` compiler is run to produce a *tag file*, which describes the required
changes; this is followed by a *conversion phase* where the `port-dml`
script reads the tag file and applies changes.

The `port-dml-module` script works well for converting most DML
devices, but has some limitations when it comes to common code that is
shared between many devices. In particular, if some parts of common
code is unused, e.g. if a provided template is never instantiated,
then it may skip some conversions. For this reason, it can be better
to use `port-dml` directly for common code.
The easiest way to perform conversion is through the wrapper script
`port-dml-module`. This script automatizes the porting process by first running an
analysis step using `make` to build one or more modules, and then running a
conversion phase by applying `port-dml` on the DML files that were compiled.

During the analysis phase, if some code is shared between multiple modules,
then it often happens that some unused code is discarded by the compiler before
it has been fully analysed. This happens in particular for unused templates and
top-level `if` statements. Such unused code will only get basic syntactic
conversions, like transforming `parameter` to `param`, and miss conversions
that depend on semantic analysis, like transforming `read_access`
to `read_register`. This
can be solved by including multiple modules in the analysis step; the
conversion phase for common code will then combine the analyses of all modules,
and utilize semantic analysis from any code paths that is active in *any* of the
included modules.

The `port-dml-module` script relies on
rather crude heuristics which often may be
incorrect; for this reason, the script also prints exactly how it invokes the
lower level `make` and `port-dml` commands. This allows each step to be
individually rerun manually with tweaked settings.

The `port-dml` script can also be used directly without `port-dml-module`; this
mode of operation has a steeper learning curve but provides greater control
which can be advantageous when porting a large code base.

## Using the port-dml script
In order to port a DML 1.2 file to DML 1.4, first pass the <code>-P
Expand Down Expand Up @@ -54,7 +68,7 @@ If you build your device from a Simics project, you can use the variable
be set to the absolute path of a file; `make` will pass that in the
-P flag to `dmlc`. Note that if you want to re-run an analysis,
then you need to first run <code>make clean-<em>module</em></code> to force
DMLC to re-run on all devices in the SIMICS module.
DMLC to re-run on all devices in the Simics module.

If parts of the device source code is unused, e.g. if a template is
never instantiated within the device, then DMLC can not perform a full
Expand All @@ -75,3 +89,99 @@ tags](changes-auto.html) and apply the change manually.

</div>

## Porting common code still used from DML 1.2 code

When porting a large code base to DML 1.4, you likely want to work
incrementally, porting some devices at a time. It can then happen that some of
your newly ported 1.4 files share common code with devices that are still in DML 1.2.

It is not allowed to import a DML 1.2 file from a DML 1.4 device, but a DML 1.2 device may import a DML 1.4 file with some caveats. Thus, any code common between DML 1.2 and 1.4 must be ported to 1.4 before any device can be converted.
There are two possible strategies for this: Either convert the common file in place,
or duplicate it into separate 1.2 and 1.4 versions.

### Keep a separate DML 1.4 copy

After letting conversion tools convert `foo.dml` to DML 1.4, you can rename the
converted file into `foo-dml14.dml`, and restore the original 1.2 version as
`foo-dml12.dml`, and finally create a trampoline file `foo.dml` containing:

```
dml 1.4;
#if (dml_1_2) {
import "foo-dml12.dml";
} #else {
import "foo-dml14.dml";
}
```

This way, existing `import "foo.dml";` statements from both DML 1.2 and 1.4 devices
will continue to work.

The apparent downside of this approach is that the logic of the common code is
duplicated across two files, which is a problem if the DML 1.2 variant is expected
to be maintained over a longer period of time. However, if all uses from DML 1.2 of
the common code are expected to be ported within a short migration period,
then this is likely the preferred approach.

After the last DML 1.2 use of the common code has been ported, `foo-dml12.dml` can be removed, and `foo-dml14.dml` can be moved back to `foo.dml`, overwriting the trampoline.

Note that the `#if` trick in the `foo.dml` trampoline above utilizes an
otherwise undocumented DML feature: DML normally doesn't allow `import`
statements within `#if` blocks, but a special exception was added to permit it
specifically within `#if (dml_1_2)`, in order to support this use case.

### In-place conversion, preserving DML 1.2 compatibility

A common file can be ported to DML 1.4 and still be useful from DML 1.2, with a
number of caveats. For instance, devices often implement functionality by
overriding standard methods, and some methods have been renamed between DML 1.2
and 1.4. For instance, an override of the `read_access` register method in a
DML 1.2 device roughly corresponds to a `read_register` override in a DML 1.4
device, and an attribute with `parameter allocate_type = "uint64"` in DML 1.2
corresponds to an event with `is uint64_attr` in DML 1.4. Much of this can be
taken care of by the `dml12-compatibility.dml` layer: A shared DML 1.4 file can
say `import "dml12-compatibility.dml";`. This does nothing when imported from a
DML 1.4, but when imported from DML 1.2, it provides some glue that ties DML
1.4 constructs to the DML 1.2 API. For instance, it defines templates such that
`is uint64_attr` in the DML 1.4 file will expand to define `allocate_type` when
imported from DML 1.2. This file also provides some templates for explicit
instantiation. In particular, the `dml12_compat_read_register` template can be
instantiated on a DML 1.4 register that overrides the `read_register` method;
this has no effect in a DML 1.4 device, but in a DML 1.2 device it overrides
the DML 1.2 method `read_access` to call the provided override. Similarly,
the `dml12_compat_write_register` template can be used on registers that override
`write_register`; `dml12_compat_read_field` and
`dml12_compat_write_field` can be used on field that override the `read_field` or `write_field` method; and `dml12_compat_io_memory_access` can be used on banks that override the `io_memory_access` method.

Sometimes, the facilities in `dml12-compatibility.dml` are not sufficient for
full DML 1.2 compatibility. For instance, suppose you want to use the `shared` annotation on a `read` method when writing the DML 1.4 version of a template. There are fundamental limitations in DML 1.2 that prevent such overrides. This can be overcome with an `#if (dml_1_2)` block on the top level:
```
dml 1.4;

#if (dml_1_2) {
template read_twelve {
method read() -> (uint64) {
log info: "read";
return 12;
}
}
} #else {
template read_twelve is read {
shared method read() -> (uint64) {
log info: "read";
return 12;
}
}
}
```
This is somewhat similar to the `foo-dml14.dml` trampoline approach discussed above, with
the difference that it can be applied selectively only on problematic parts of the
file.

If the flag `--compat` is passed to the `port-dml` script, then the script will
automatically detect some cases where similar `#if` clauses are needed for
compatibility, and insert them automatically. The script will also add an
`dml12-compatibility.dml` import. The `--compat` flag can also be passed to the
`port-dml-module` script; in this case, the script will pass on `--compat` to
`port-dml` when converting DML files that don't reside in the directory of any
of the ported modules.