Please consider the packageName with Android apps

[IzzySoft]

First let me say I really like the idea of your project, so thanks for creating and maintaining it! But currently it's impossible for us to show your "statistics batch" on our projects' pages, as those numbers are misleading:

I've already reported that to your website project for a quick visual work-around, but I guess the underlying issue is rather in the updater:

When it comes to Android apps, results shown are misleading. Example:

Who has the newer version? Don't let the numbers fool you: those are 2 different apps just having the same display name. The 1.4 there was last updated 4 years ago – the 1.2.1 this year. So you compare apples with peaches here: axp.tool.apkextractor v1.4 with io.github.abdurazaaqmohammed.ApkExtractor 1.2.1 – two entirely different apps.

Please eliminate such "false hits" already when comparing. People who are not that tech savvy are misled to think IzzOnDroid does not update properly, thus serving outdated packages. So when comparing for Android apps, also ensure the packageNames match – not only the "display name". It otherwise reflects badly on the repositories you cover. Thanks in advance!

[IzzySoft]

PS: something else is wrong with the updater:

As you can see in our repo browser, there's only one version of flying-carpet available, and that since April already, namely v8.0.2:

LibChecker was just added here recently, after their 2.5.0 release. So why is it "problematic" that outdated versions are missing?

[AMDmi3]

something else is wrong with the updater

It works as intended. libchecker is "ignored" because it uses non-official version format which cannot be compared to other repositories and cannot mark them outdated. flying-carpet is "untrusted" because the version looks valid, but we don't trust newest versions present solely in android repos, as these are likely to use unofficial format like in the former case. It would become green if some trusted repo also provides 8.0.2. This is a precaution mostly, may reconsider it as some point.

[IzzySoft]

It works as intended. libchecker is "ignored" because it uses non-official version format which cannot be compared to other repositories and cannot mark them outdated.

I was rather pointing at the two "outdated" versions in red there, which have never been at IzzyOnDroid.

flying-carpet is "untrusted" because the version looks valid, but we don't trust newest versions present solely in android repos

Ahem… Screenshot from Github:

That was released in March. So it's "solely" at both Android repos AND at the source 🤷‍♂️ So what is a "trusted repo" here that could provide? I guess you just follow the "latest" link there; in fact, 8.0.1 is marked "latest" at Github, but 8.0.2 is definitely present, as the screenshot confirms.

cannot mark them outdated

Speaking of which, a snippet from the "outdated" list:

Should then be ignored as well? It's a different build flavor, yes, and they include a versionNameSuffix for it (a standard feature, though not used that often – but not an "unofficial format", see here). And something I'm afraid you cannot do much about:

That 1.9.2 tag is broken and has no APKs. Guess F-Droid has it established as RB, and thus cannot build it either as there are no APKs. But that's a rather special case I guess we have to live with (unless you want to check for "APK available" as well – but then, that APK could be a debug one we'd not pick up either…

Funny, according to https://github.com/oxen-io/session-android/releases (and tags as well) 1.20.2 is the latest one. Ah, F-Droid uses a fork 🙈 (checking) Now, that was helpful, thanks: Repo moved and we got no notice. Adjusted, will be updated with the next sync at around 7 pm UTC.

These two are quite obvious (and yes, different apps by different authors).

---

That said: thanks for working on it! And knowing the "obstacles", I will now include that "outdated" list with our maintenance links, to check from time to time (e.g. for cases such as Session above, or where to nudge the app authors who forgot to attach an APK etc. So let me once more include a thanks for providing Repology!

[AMDmi3]

Regarding the original issue, it's fundamental and I'm not sure if it is fixable. The first step in Repology processing is grouping packages of a single project together under a single key (dubbed as "project name") derived from package data and refined through a ruleset. Most repositories use common project name as a package name and this works, but android repos don't seem to have that and use either display name (which is not unique) or appid (which won't match with other repositories).

Possible solutions:

1. Switch to appid as a project name for android repos.
   • Unrelated projects no longer alias.
   • F-Droid and Izzy should still be comparable (as long as these use the same appids which I believe is likely true).
   • UI is polluted by long unreadable names.
   • Android packages no longer match with PC ones.
     • To match these, we need manually maintained mapping. In fact, we already have one in a form of whitelist for android apps which are allowed to be matched with other repos, but it's just a list of names which is less complicated than an appid mapping.
2. Hide projects only present in Android repos.
   • Only intersection with PC repos will be visible, aliased projects will be not.
   • Harder to add new projects to the whitelist as these are not visible by default.
   • No way to compare android-only projects between fdroid and izzy.
3. Mark projects only present in Android repos as incomparable.
   • Aliased projects will still be there, but there won't be meaningless version comparison for these.
4. Manually split all aliased projects.
   • This requires a lot of handwork for maintaining a ruleset for android apps.

The choice depends on whether you are interested in Repology support, because IoD currently mostly intersects with F-Droid. IIRC, on mastodon you've mentioned that you already have tool for version comparison with either F-Droid of upstreams so it's okay if you're not. If you are interested, we can do 4 if you can spare some time on maintaining a set of split rules or 1 if that doesn't cause any problems with the site. Otherwise, it would make sense to do 2 which is basically reverting to state before adding IoD (it would have not point to still support it for only 3 projects intersection).

What do you think?

[AMDmi3]

I was rather pointing at the two "outdated" versions in red there, which have never been at IzzyOnDroid.

These are from F-Droid. The table shows all versions in all repos, IoD specific versions are in the "Selected" column (see its tooltip). This suggests UX can be improved here.

So what is a "trusted repo" here that could provide

The others listed (not chocolatey though as it's untrusted as well), or any new repo which packages it.

Speaking of which, a snippet from the "outdated" list:

Should then be ignored as well? It's a different build flavor, yes, and they include a versionNameSuffix for it

It can either be normalized (removing the suffix) or ignored with a rule, or, if versionNameSuffix is something provided with repository metadata, can be accounted for in the repository data parser.

That 1.9.2 tag is broken and has no APKs. Guess F-Droid has it established as RB, and thus cannot build it either as there are no APKs

I believe everything listed in F-Droid is installable as they build APKs themselves. Just confirmed, it's installable.

Now, that was helpful, thanks: Repo moved and we got no notice

It's great it was helpful.

[IzzySoft]

Thanks @AMDmi3! I meanwhile walked the list of 21 entries. 2 of the apps I was able to fix on our end (the one from above, and another one which had changed their tag name formatting), a 3rd one I removed as it was rather "dead" (the latest version at F-Droid (from 2021) had no APK provided so we cannot fetch it, and the one we had was removed from releases). So that list should get a bit shorter after both of us have synced.

From the remaining 18, another few "patterns"/"cases":

There is no 2.1.0 at F-Droid – 2.0.2 is the latest. And there is no official source (anymore), as the repo disappeared in 2021/22 – see here for background.

So 1.7 is outdated, but 1.6 is just "legacy"? And "Americum" an official version format? 😉 (yes, different apps again, and a few more "Metronom"s at F-Droid. Too many apps around with ambiguous names: Metronom, Clock, Compass, BMI Calculator…)

Actually, that 3.1.4 is from 2021, the 2.5.0 at F-Droid from 2017 – while the 2.7.5 at IzzyOnDroid is from 8/2024. No idea where F-Droid got the 3.1.4 from, no such tag listed (IIRC, the 3.x are a different build variant: an experimental one with MQTT support. Latest from that was 3.0.8 created 2/2023). Though not much you can do here to keep the two apart (same repo, same appId) – but how the (non-existing) 3.1.4 got in there I have no clue.

OK, no worries, I stop here: all the different constellations I've encountered should be mentioned now. Wish you all the best to get that tackled; if you need me to test something, or for some "technical details" concerning Android apps / F-Droid repos, let me know and I see how I can do my part here, so we can make this a double-win-win 😃 Thanks again!

[AMDmi3]

There is no 2.1.0 at F-Droid – 2.0.2 is the latest

2.1.0 is shown on that very page. It's just not marked as suggested. Not quite if versions above suggested one should be handled specially, maybe as devel.

So 1.7 is outdated, but 1.6 is just "legacy"?

1.7 outdated because there's 4.12.1, and versions older than the latest in the repository are classified as legacy, assumed to be knowingly and expectedly outdated and preserved for some reason.

And "Americum" an official version format?

Repology won't know it's not offical unless there's a rule. We can have a rule which ignores alphabetic versions, or all versions for this project, but in the global ecosystem it would just be outdated by numeric version and that would be ok.

[AMDmi3]

OK, no worries, I stop here: all the different constellations I've encountered should be mentioned now. Wish you all the best to get that tackled; if you need me to test something, or for some "technical details" concerning Android apps / F-Droid repos, let me know and I see how I can do my part here, so we can make this a double-win-win 😃 Thanks again!

I need your opinion on solutions proposed in #1453 (comment).

[IzzySoft]

Oops, missed a comment while digging through the list, apologies! So here we go:

F-Droid and Izzy should still be comparable (as long as these use the same appids which I believe is likely true).

I guess in 90..95% that is true. A few apps have different "applicationIdSuffixes" (e.g. .libre or .fdroid for the builds at F-Droid, while none or .free at IzzyOnDroid). But those could as well be considered different apps, as they are different builds.

UI is polluted by long unreadable names.

Hm, could maybe mapped to a name (as now), with a numbering appended if there'd be a second match (e.g. "Clock, Clock(2), Clock(3)"). Just a thought, not sure if that fits you.

Android packages no longer match with PC ones.

"Shadow mapping" (via the current mapping way, just as "invisible attribute")? Maybe a few relations missing is better than "false positives" – but if it's too many missing… yeah, I get your point there. And "manual" should be the "last resort" (I love having things automated, too).

Hide projects only present in Android repos.

No idea there, sorry – didn't dig into the Repology concept that deep yet. But it would be good if Android apps were comparable. Helps each of us to detect "missed changes" (devs do not always notify us when they change things, as shown above – and that's not limited to IzzyOnDroid, but happens at F-Droid a lot, too).

Mark projects only present in Android repos as incomparable.

Uncomparable to what? F-Droid vs. IzzyOnDroid should be mostly comparable (and should be definitely when package names match, and even more when they are "reproducible builds" at both ends). Neither allows forks to use the original app's applicationId (unless the project was transferred officially to the new project owner/maintainer by the original one). Of course there could be cases where an app uses the same appId (if the original was not known before and the appId is not too obvious; e.g. an org.fossify.something not coming from Naveen's team would ring a bell, and a church bell at that, if you know what I mean).

Manually split all aliased projects.

Always the last resort, in case all else fails…

The choice depends on whether you are interested in Repology support,

definitely!

because IoD currently mostly intersects with F-Droid. IIRC, on mastodon you've mentioned that you already have tool for version comparison with either F-Droid of upstreams so it's okay if you're not.

I have a tool here showing me which apps are in both repos, yes. But that's a CLI tool which I can use – not something that's an easy option for other people who might wish to check. And I've been pointed to you by several people now, a.o. to integrate your badge. So they obviously like Repology and want to see IoD (IzzyOnDroid) there. So I try my best to support that – and well, I like it myself 😉

If you are interested, we can do 4 if you can spare some time on maintaining a set of split rules or 1 if that doesn't cause any problems with the site.

I have no idea how those "split rules" would look like. In general, I'd prefer automated solutions above manual ones, but I'm open to helping out there. If you could give me an example of how such a rule file could/should look like, I could see if I can automate that here. I have the indices of several F-Droid style repos here and updated automatically.

Otherwise, it would make sense to do 2 which is basically reverting to state before adding IoD (it would have not point to still support it for only 3 projects intersection).

That would remove a good feature (which currently only has the side-effect of "false postives" which could be sorted out when e.g. including the appId when comparing).

only 3 projects intersection

I don't understand what you mean by that, sorry. Maybe an example would help?

[AMDmi3]

Hm, could maybe mapped to a name (as now), with a numbering appended if there'd be a second match (e.g. "Clock, Clock(2), Clock(3)"). Just a thought, not sure if that fits you.

Not really, it won't be stable if applied on the fly (2 gone → 3 shifts to 2 → messup), also doesn't help users to tell which is which (when we split, descriptive suffixes are added to the names, author names as last resort), and if done manually it's solutuion 4.

Uncomparable to what?

To themselves. That is, there's no version comparison for these projects, no newest and outdated versions, just ignored ones.

definitely!

Okay, so we're going to fix it some way.

I have no idea how those "split rules" would look like. In general, I'd prefer automated solutions above manual ones, but I'm open to helping out there. If you could give me an example of how such a rule file could/should look like, I could see if I can automate that here. I have the indices of several F-Droid style repos here and updated automatically.

See any file under https://github.com/repology/repology-rules/tree/master/850.split-ambiguities. In short, withing an ambiguous project name (one which contains multiple unrelated projects) we match by upstream URL and assign more specific names. Packages without no url matches fall into unclassified rule and end up with ${projectname}-unclassified name, so it's easy to spot them to add a missing rule.

Such group of rules would have to be added for each ambiguous project name. Quick lookup shows there are 71 of them (that's both F-Droid and IoD). There's currently no direct way to access appid either, so these would have to use URLs too.

That would be a tedious task, but with rather simple code changes I can add an ability to append appid to a project name, which simplifies the task from writing a rule for each target project, to just listing source (ambiguous) names. That would surely blow up something in the UI, but we can tune CSS wrapping properties which would be useful change anyway.

Does it sound like a plan?

I don't understand what you mean by that, sorry. Maybe an example would help?

There are only 3 projects which interect with any other repositories that f-droid. That means that if we're going to hide android-only projects, there will be only 3 projects left visible in IoD.

[IzzySoft]

split rules

Just taking a look at it, seems doable. I only need to understand what to put there, then it's a one-time setup – and checking every once in a while what needs to be added. Is there something that can be matched against the appId? Display names change from time to time, appIds don't (as that would mean "different app" then – also happens, but different issue). Going by the above examples, let me make up "dummy examples" here to show you what I mean:

- { appId: ca.farrelltonsolar.classic, verpat: "3\\..*", setname: MidniteSolar Classic Monitor (MQTT) }
- { appId: de.moekadu.metronome, setname: Metronome (Moessner) }
- { appId: com.bobek.metronome, setname: Metronom (Bobek) }
- { appId: tech.waelk.radioactive.metronome, setname: Metronome (wael) }
- { appId: james.metronome, setname: Metronome (James) }

I wonder how the Bobek ended up being associated with the other Metronomes, as it does not have the "e" at the end of its display name. And I hope with the MidniteSolar, the other one would show up as before? And could it work that way, matching on the appId (and, where needed, on other parts in addition)? Would wwwpart be applied on the source URL (Github, GitLab, Codeberg etc)? If we cannot match on appId, the match on wwwpart would be the next thing then – though apps moving between repos (e.g. from Github to Codeberg) are no "rare cases" (no frequent ones either, but they happen all the time).

Quick lookup shows there are 71 of them (that's both F-Droid and IoD).

Ah, thanks: that list makes it easier to write up the rules, as I can simply go by the local metadata here instead of clicking through a bunch of URLs. And while I could match on the "old names" there (display name, lower-cased, spaces replaced by - – is that right? And how would I match my examples above then, with the parenthesis?), the apkId would still help making the match "steady".

Does it sound like a plan?

If I got that right, yes. I guess I could setup such rules then. Whether I could set up a script to keep them automatically updated, or at least to spit out what needs attention, remains to be seen. But I'd be willing to give that a try – nothing to do on your end before we see how it goes then (I'd hand in a first sample to I can adjust before running down a wrong track to its end I guess – and when we see the light at the end of the tunnel, I'd "only" need to complete my task, while you could start on your part. Automatically fetching such a list like in your Gist would certainly help in keeping the set up-to-date.

There are only 3 projects which interect with any other repositories that f-droid. That means that if we're going to hide android-only projects, there will be only 3 projects left visible in IoD.

That sounds not as good – but maybe I still don't get the concept of what "project" means in this contest. E.g. currently, there seem to be 1209 project in IoD. We have 1223 apps there as of now, so that's not what "project" means it seems…

[IzzySoft]

OTOH, seeing it's just 11 apps affected by ambiguity, and all of them having the android: prefix: maybe the easiest approach would be to simply mark them "ambiguous"/"ignored". I've managed to bring the all-over number down from 21 to 17, and from the remaining, 6 are correctly pointed out (can't be helped if upstream doesn't provide us the APK; IoD serves the APKs of the developers, we don't sign ourselves though we could build, which we do to confirm reproducible builds). The final 1 left is android:port-authority, which uses a versionNameSuffix for its "free" build flavor, which you maybe could adapt to.