`outdated --accurate*` takes excessively long (5+ minutes) or hangs when lots of MAS apps installed

[HugeIRL]

Configuration

mas ▁▁▁▁ 3.0.1
arch ▁▁▁ arm64
from ▁▁▁ homebrew/core/mas
origin ▁ https://github.com/mas-cli/mas.git
rev ▁▁▁▁ 67e5dfd18882ad294c58441f5d2308d4dfccb862
swift ▁▁ 6.2.1 (swiftlang-6.2.1.4.8 clang-1700.4.4.1)
driver ▁ 1.127.14.1
region ▁ CA
macos ▁▁ 26.1 (25B78)
mac ▁▁▁▁ Mac16,9
cpu ▁▁▁▁ Apple M4 Max

Bug description

I've noticed this on several machines. I wouldn't say I have a ton of MAS apps installed, less than 100. However mas outdated even with a pure 1Gbp/s pipe to my machine that is dedicated to that machine only can take longer then 5 minutes to complete. I have a script locally that exits the command on timeout of 5 minutes, which it always hits.

Updatest users are also reporting excessively long mas outdated runs on 3.0.0, which causes Updatest's internal timeout of 5 minutes to prevent command hanging to fail mas cli.

This did not happen on the old mas outdated command pre 3.0.0. I suspect the addition of a download/cancel to the logic is causing this issue.

Steps to reproduce

• Have an assortment of MAS apps (over 35)
• Run mas outdated
• In most cases, it takes over 5 minutes

[rgoldberg]

@HugeIRL outdated takes around 3 seconds for my 15-20 apps. I can't test this issue because I haven't gotten enough apps from the App Store.

Can you try moving all but 10 of your MAS apps out of /Applications (note that mas will also check any large apps folder you have configured, like /Volumes/<large app volume name>/Applications, so check your app count via mas list).

Then run time mas outdated. Can you continually add 10 apps back to /Applications, run time mas outdated & repeat? Then report back with the times?

If you notice a huge jump in time from N to N+10, could you zero in and find out exactly what count causes the massive jump?

I assume the problem is too many concurrent downloads. I can limit them to a maximum count. I'll make a WIP PR. The testing mentioned above will help me improve the maximum concurrent download count (which will either be hardcoded or processor-dependent for now as mas doesn't yet have a configuration system). If you can run that WIP when it's ready, and report if it helps, that would be very useful.

I was going to release 4.0.0 with the installd/PackageKit fix soon, but, I'd prefer to fix this issue & release 3.0.2 first, if possible, so time is of the essence.

Thanks.

[Accademia]

I’ve run into the same issue. I modified my own update script so that it can launch 16 parallel processes for mas outdated and 8 parallel processes for mas upgrade. After doing this, I’ve never had any timeouts again.

In reality, the App Store’s local background processes will block requests. After a certain number of queries, you must restart them; otherwise, even if you wait the whole night, the commands may never finish. I have nearly 300 MAS apps installed, and with high parallelism I’ve tested launching up to 128 concurrent queries — it can check all MAS apps within about one minute. However, after that, you must restart the App Store background processes, or subsequent MAS operations will stop responding completely (they’re basically stuck).

UpdateFull_For_MacOS

[HugeIRL]

@rgoldberg working atm so I can't really test it yet but I can later this evening.

Can you explain the reasoning behind adding the downloads at all to mas outdated? Just trying to understand the logic here so I can potentially fix this on my end and prevent you from having to do it through mas CLI. I don't believe the old mas outdated command ever tried to initiate a download?

[rgoldberg]

@Accademia Thanks for the limit numbers. That's exactly the type of solution I was planning. Please see my prior comment on this issue.

What CPU & Mac model are you using? The issue might be tied to core count, or it might be independent.

Have you tried larger values for the max concurrent downloads? I'd not, can you try & let me know what seems optimal as I code my WIP?

Thanks.

[Accademia]

I feel that the App Store’s local background processes will block requests. After a certain number of queries, you have to restart them; otherwise, even if you wait the whole night, the operations may never finish. I have nearly 300 MAS apps installed, and with high parallelism I’ve tested running up to 128 concurrent queries — it can check all MAS apps within about one minute. However, after that, you must restart the App Store background processes, or subsequent MAS commands become completely unresponsive (most likely because the processes are stuck).

I also tried increasing concurrency to 200 parallel processes, but macOS started throwing a large number of pop-up warnings saying that the system was not responding. After lowering the parallelism to 128, the problem disappeared entirely. My machine is an M3 Pro with 64GB of RAM. The relevant parallel code is shown below:

⸻

Parrale mas outdated （16 Process = -P 16）

mas list | awk '{print $1}' \
| xargs -n 1 -P 16 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec

⸻

Restart App Store agent process

# End the App Store agent process first (non-root)
killall appstoreagent 2>/dev/null

# Restart the installation daemon process (password required)
sudo killall installd

⸻

Full source code of the script

usercmd_updatefull

⸻

@Accademia Thanks for the limit numbers. That's exactly the type of solution I was planning. Please see my prior comment on this issue.

What CPU & Mac model are you using? The issue might be tied to core count, or it might be independent.

Have you tried larger values for the max concurrent downloads? I'd not, can you try & let me know what seems optimal as I code my WIP?

Thanks.

[rgoldberg]

@HugeIRL mas 3.1.0 will fix it, so no need to do anything except test. Thanks for offering.

There were numerous problems with the old outdated & upgrade that the 3.0.0+ download/cancel logic fixes:

1. The pre-3.0.0 logic compared versions from 2 different sources. The same release could have different version string from each, e.g., 1.8 & 1.0.0.8, or 10.0 & 26. I have no idea why Apple allowed that or developers do that, but it happens.
2. The information the pre-3.0.0 logic from Apple was only for the newest version of each app. That version might not be able to run on your macOS version, but there might be an older pending update that could. mas couldn't know about those.
3. Probably other issues, but the above 2 were bad enough.

The installd/PackageKit issue prompted me to release these changes as 3.0.0 sooner than I had anticipated, because I wanted people to be able to use them without the core changes to mas from the fix for the new issue from the forthcoming 4.0.0. I thus didn't have the chance to get feedback on 3.0.0.

[Accademia]

⸻

Parrale mas outdated （16 Process = -P 16）

mas list | awk '{print $1}' \
| xargs -n 1 -P 16 -I{} sh -lc '
  if gtimeout 120s mas outdated "{}"; then
    exit 0
  fi
  ec=$?
  if [ $ec -eq 60 ]; then
    osascript -e "tell application \"App Store\" to quit" >/dev/null 2>&1 || true
    sleep 1
  fi
  exit $ec

The reason for forcibly terminating a process after 60 seconds on the command line is that some apps will pop up a dialog saying the app is no longer available on the App Store and require the user to click OK; without that click, the script hangs. To allow the update script to run unattended, we need to set a timeout. Under these conditions, each process can only check one app at a time to ensure that an app which hasn’t been checked yet isn’t killed along with the process about to be terminated.

[rgoldberg]

@Accademia Thanks for the info.

It sounds like you tried a max of 16 & a max of 128. Can you tell me how long each took?

If the higher amount took much less time, can you increase from 16 in increments (best to try lower amounts first to not get limited by Apple) to hone in on the optimal max?

Can you reply back with all of your run times for each max value?

Thanks.

[rgoldberg]

@Accademia For the aforementioned testing, can you temporarily move any apps that open dialogs out of your /Applications? Or just have your script skip them by name?

Thanks.

[rgoldberg]

@Accademia Are you saying that, even if you process the apps in 16-app chunks, you still must eventually kill your App Store processes? Or that the 16-app chunks avoid needing to kill the processes?

Thanks again.

[rgoldberg]

@HugeIRL @Accademia if the chunking still requires MAS processes to be restarted, then it might be best to offer both the old logic & the 3.x logic, so people can switch via a flag. I'm debating about which should be default. It really depends on what most users will experience, but, it's probably safest to make the default the old logic.

[HugeIRL]

@HugeIRL @Accademia if the chunking still requires MAS processes to be restarted, then it might be best to offer both the old logic & the 3.x logic, so people can switch via a flag. I'm debating about which should be default. It really depends on what most users will experience, but, it's probably safest to make the default the old logic.

I'd say old logic is safe as a default as it worked relatively fine for most people, and the new logic behind a flag of some sort.