Speed up yadm by caching results of git calls

[rasa]

Is your feature request related to a problem? Please describe.

yadm can be slow on slower systems.

Describe the solution you'd like

Cache expensive git calls.

Additional context

When I run yadm commit on my system, these are the commands that are run, and how often (found using GIT_TRACE=1):

  Count  Command
-------  ----------------------------------------- 
      2  git fetch --update-head-ok
      2  git maintenance run --auto --no-quiet
      2  git merge FETCH_HEAD
      2  git rev-parse --git-dir
      2  git rev-parse --git-path objects
      2  git rev-parse --show-prefix
      2  git rev-parse --show-toplevel
      2  git-submodule summary --cached --for-status --summary-limit -1 HEAD
      2  git-submodule summary --files --for-status --summary-limit -1
      4  git config --file=.config/yadm/config --bool yadm.alt-copy
      4  git config --file=.config/yadm/config --bool yadm.auto-alt
      4  git config --file=.config/yadm/config --bool yadm.auto-perms
      4  git config --file=.config/yadm/config --bool yadm.auto-private-dirs
      4  git config --file=.config/yadm/config --bool yadm.gpg-perms
      4  git config --file=.config/yadm/config --bool yadm.ssh-perms
      4  git config --file=.config/yadm/config yadm.git-program
      4  git config --get-all local.class
      4  git config core.worktree
      4  git config local.arch
      4  git config local.hostname
      4  git config local.os
      4  git config local.user
      4  git ls-files
      4  git-sh-i18n--envsubst 'usage: $dashless $USAGE'
      4  git-sh-i18n--envsubst --variables 'usage: $dashless $USAGE'

The results of at least the git config commands can be cached to speed things up, yes?

If you are open to this enhancement, I will draft a PR for your review.

[erijo]

It would be interesting to see how much difference it makes if you replace the calls with hard coded values (i.e. the optimal case)?

[rasa]

In an iSH emulation environment, running a simple yadm command can take 45+ seconds with auto-alt and auto-perms enabled, and 7.5 seconds when they're disabled.

I will explore how this can be sped up.

#466 should help when auto-alt is enabled.

[erijo]

@rasa: If you have the time, please test with the deep-alt branch and let me know if that improved the performance.

[rasa]

I ran yadm alt several times in an iSH VM, and consistently get:
master:

real    0m9.716s
user    0m4.294s
sys     0m0.000s

deep-alt:

real    0m4.896s
user    0m2.158s
sys     0m0.000s

so it's a 50% speed increase. Worth it!

[rasa]

No speed up on yadm perms, which always shows

real    0m0.693s
user    0m0.446s
sys     0m0.000s

[erijo]

@rasa: Could you please test with the test-workflow branch? It now includes the fixes from the deep-alt branch + a change to how parse_encrypt is implemented which may affect the run time.

[rasa]

@erijo We lost about a second on iSH, using the test-workflow branch:

real    0m5.561s
user    0m2.191s
sys     0m0.000s

[erijo]

Could you please test how the recursive-dir-alt branch performs?

Also of interest is what bash version are you using, and how many alt files do you have in your test?

[erijo]

If you have time @rasa I'm still interesting in getting some more data and info (see previous comment) to better understand the root cause.

[rasa]

@erijo Thanks for the tickle. I will do some testing of recursive-dir-alt soon.