registryURL should a valid URL not just hostname

[kadel]

This discussion came up during the demo.

registryURL as name suggest should be URL. As such it should aways have a form as proto://host/path.

There are examples that show registryURL: registry.devfile.io and with the current implementations, this is valid.
Becaue it autmativalyy asumes http as proto. This is implemented in getDevfileFromRegistry function

registry.devfile.io is not a valid URL and it should not be a valid value for registryURL .
The correct and value in this case should be registryUrl: "https://registry.devfile.io"

[veillard]

registryURL should be an URL as defined in RFC 3986

the parent uri property should be an URI Reference with the URL of the current devfile as the base

c.f. section 4.1 of RFC 3986

[elsony]

The existing field supports both hostname and URL, e.g. "https://registry.devfile.io". If supporting hostname is causing more confusion, we can remove hostname support and only support URL. Also, we'll update the existing URI description field to something like "URI Reference of a Devfile yaml file with the current devfile as the base URI" to clarify.

[kadel]

We might not even need two separate fields. just one uri reference will serve the same purpose.

[elsony]

registryURL should be an URL as defined in RFC 3986

the parent uri property should be an URI Reference with the URL of the current devfile as the base

c.f. section 4.1 of RFC 3986

@veillard Note that the parent reference does not be always based on the current devfile context. A devfile should be able to refer to another devfile in another devfile registry. The spec needs to support both scenarios. Also, to make sure we cover all the scenarios that you have in mind, can you provide a concrete example of how it looks like.

[veillard]

registryURL should be an URL as defined in RFC 3986
the parent uri property should be an URI Reference with the URL of the current devfile as the base
c.f. section 4.1 of RFC 3986

@veillard Note that the parent reference does not be always based on the current devfile context. A devfile should be able to
refer to another devfile in another devfile registry. The spec needs to support both scenarios. Also, to make sure we cover all
the scenarios that you have in mind, can you provide a concrete example of how it looks like.

Referencing parent devfile in another registry should be possible by providing the full URL of the parent e.g.:
https://corpregistry.corp.example.com/backend.yaml

Referencing a parent devfile in same registry then becomes a relative reference e.g.:

uri: shared/java.yaml

if the current URL of the devfile is https://ourgroup.corp.example.com/springboot.yaml
then the parent would resolve as

https://ourgroup.corp.example.com/shared/java.yaml

and if another client lacking https support was accessing the springboot devfile over http as

http://ourgroup.corp.example.com/springboot.yaml

then the parent uri value once converted would turn into

http://ourgroup.corp.example.com/shared/java.yaml

and usable by that client too.
Problem is that if you try to support hostnames shortcuts you make it impossible to name the parent with
a path that could be also interpreted as a hostname.

Does that sound right ?

[yangcao77]

@veillard All the cases you mentioned are already supported in devfile library by using parent.uri

The resgiryURL supports both hostname (registry.devfile.io) and URL (https://registry.devfile.io) for user's convenience.
But since the hostname support in the demo brings confusion. I created a PR to restrict the registryURL to use valid URL format, which contains http/https protocol as a prefix: devfile/library#76.

Also created another PR (#387) to update the description of importReference.uri with explanation of usage: can be either a full URL or a relative URI reference.

[yangcao77]

Problem is that if you try to support hostnames shortcuts you make it impossible to name the parent with
a path that could be also interpreted as a hostname.

Does that sound right ?

I agree resgiryURL should be with protocol instead of just the hostname, as it is named as "URL", it should follow the URL format.

However, the use case described should use parent.uri to specify the path, it does not matter if uri is being defined in a hostname-like path. URI reference and Id reference follow different logics.
Using registryURL + Id does not have the flexibility to specify the path. Valid registries are expected to follow the REST API defined: GET http://{registry host}/devfiles/{stack}, where http://{registry host} is the registryURL and {stack} is the Id

[elsony]

The scenarios described in #386 (comment) are all related to the uri field only. As @yangcao77 mentioned, all of those scenarios are already supported by the uri field. Looking at the current spec, I think maybe one source of confusion is where the registryURL is.

The current registryURL field is meant to be used with the id field, however, the current spec have the registryUrl field directly under the parent element. #387 has updated the description of the registryUrl field to clarify it in the spec. If that is meant to be used by the id field only, maybe one of the things that we can consider is to move the registryUrl under the id to make it clear that it is meant to be used by the id reference but that means it is a change on the spec. Or we can deprecate the existing field and add a new one under id to avoid breakage.

[kadel]

Maybe I missed some discussion earlier.

If I understand it correctly there are currently 3 ways to reference parents.

reference to DevWorkspaceTemplate CR

This is done by kubernetes field in parent. This is not relevant to this discussion, so skipping this one.

devfile.yaml file outside of the registry

This is specified by uri field.
It can be a full URL like https://example.com/devfile.yaml or a relative like uri: something.yaml .
If I have a local devfile.yaml that lives in root of my project source code and it specifies parent as uri: something.yaml. Than it will look for someting.yaml in root of my project source code, right?

devfile.yaml in registry

This for some reason requires two fields

• registryUrl
• id
  If I understand it correctly I can set just id: nodejs, and then it should check the current registry for stack called nodejs. But most of the devfile are going to live in project source codes. What registry is going to be used if my local devfile specifies parent just by id?

Would it be safer to require a full definition, including registry?

Another thing is that we have two ways of defining the same thing. If I'm not missing something, because devfile registry has REST API this

registryURL: https://registry.devfile.io
id: nodejs

is equivalent to

uri: https://registry.devfile.io/devfiles/nodejs

Do we even need id and registryURL if I can define the same with just uri field?

[yangcao77]

Maybe I missed some discussion earlier.

It can be a full URL like https://example.com/devfile.yaml or a relative like uri: something.yaml .
If I have a local devfile.yaml that lives in root of my project source code and it specifies parent as uri: something.yaml. Than it will look for someting.yaml in root of my project source code, right?

Notice that the relative uri should be relative to the current devfile. So if parent uri is specified as something.yaml, then it will look for something.yaml in the current devfile's directory.

* `registryUrl`

* `id`
  If I understand it correctly I can set just `id: nodejs`, and then it should check the current registry for stack called `nodejs`. But most of the devfile are going to live in project source codes. **What registry is going to be used if my local devfile specifies parent just by `id`?**

Would it be safer to require a full definition, including registry?

Another thing is that we have two ways of defining the same thing. If I'm not missing something, because devfile registry has REST API this

registryURL: https://registry.devfile.io
id: nodejs

is equivalent to

uri: https://registry.devfile.io/devfiles/nodejs

Do we even need id and registryURL if I can define the same with just uri field?

I think you misunderstood it. Library won't "detect" current registry.
For the case if registryURL is not provided in devfile, as I mentioned in the demo, consumers can pass in valid registries when they call parser, via: https://github.com/devfile/library/blob/3d708859f0b5605f0827d5594b25374d0a0937c8/pkg/devfile/parser/parse.go#L69-L71

In addition, providing the uri as https://registry.devfile.io/devfiles/nodejs is not going to work.
It is because this: #61 (comment), library made this change within devfile/library#73
The uri is expected to be a yaml file path/URL. If the uri not end with .yaml, it will be treated as a directory which contains devfile.yaml/.devfile.yaml. So with uri: https://registry.devfile.io/devfiles/nodejs, library will look for https://registry.devfile.io/devfiles/nodejs/devfile.yaml and https://registry.devfile.io/devfiles/nodejs/.devfile.yaml; which does not even exist

[elsony]

devfile.yaml file outside of the registry

This is specified by uri field.
It can be a full URL like https://example.com/devfile.yaml or a relative like uri: something.yaml .
If I have a local devfile.yaml that lives in root of my project source code and it specifies parent as uri: something.yaml. Than it will look for someting.yaml in root of my project source code, right?

+1 on the described behaviour

devfile.yaml in registry

This for some reason requires two fields

• registryUrl
• id
  If I understand it correctly I can set just id: nodejs, and then it should check the current registry for stack called nodejs. But most of the devfile are going to live in project source codes. What registry is going to be used if my local devfile specifies parent just by id?

Would it be safer to require a full definition, including registry?

Are you suggesting combining them into a single field or making it so that when someone specifies the id, the registryURL will become mandatory as well? If it is the latter, does it mean that we no longer support resolving the id with the current registry? This may be a safer approach to make sure it will get resolved properly as you suggested.

Another thing is that we have two ways of defining the same thing. If I'm not missing something, because devfile registry has REST API this

registryURL: https://registry.devfile.io
id: nodejs

is equivalent to

uri: https://registry.devfile.io/devfiles/nodejs

Do we even need id and registryURL if I can define the same with just uri field?

If I remember correctly when discussing the parent reference spec, the reason that we have id + registryURL is that we want to have an easy way to just refer to a given stack in a given registry (same or a different registry). You have correctly pointed out that there are currently two ways to achieve the same thing but the sample uri https://registry.devfile.io/devfiles/nodejs exposes the implementation details of the registry. That's how a user can access the existing OCI based registry. However, if we decided to implement the registry differently and the access to a stack may be different. The original reason for having id + registryURL is to not exposing the registry implementation details and may not need to know that. Having said that, the discussions on the parent reference were quite a while ago (almost at the beginning of the devfile 2.0 spec discussions where we haven't got any implementation details on the registry design yet). We can always revisit if we don't find it useful.