[Core] Optimize LoRA weight loading

[jeejeelee]

Purpose

Gemmi's description is very detailed and can slightly reduce the loading time for LoRA weights.

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request optimizes LoRA weight loading by changing the memory layout of LoRA tensors to avoid costly transpose operations. The changes are consistently applied across all relevant layers and utility functions. The new convention for lora_a is (rank, input_dim) and for lora_b is (output_dim, rank), which matches how they are often stored in checkpoints, thus removing the need for transposition during loading. The slicing and copying logic has been updated accordingly. The changes are correct and contribute to better performance. I have no high or critical severity comments.

[Isotr0py]

LGTM!