Skip to content

Conversation

ShikiSuen
Copy link
Collaborator

@ShikiSuen ShikiSuen commented Jan 25, 2025

Lukhnos 在 Gramambular 2 引入的 DAG-Relax(有向無環圖)組句算法在 Sandy Bridge CPU 電腦上的效能不如這個 PR 引入的經過最佳化處理的 Dijkstra 組句算法。

另外,哪怕是末代的 Intel mac mini(Intel(R) Core(TM) i7-8700B CPU @ 3.20GHz),測試得出的結論也是一樣。

image

平均記憶體佔用雖是 DAG 的兩倍,但峰值記憶體佔用略低、在 i7-8700B 測試下述內容時僅需要 23s(DAG 需要 32s)。

  func test24_asanAgainstDijkstra() throws {
    let useDAG = true
    let ramMeasurement = XCTMemoryMetric()
    measure(metrics: [ramMeasurement]) {
      var compositor = Megrez.Compositor(with: SimpleLM(input: strSampleData))
      compositor.separator = ""
      compositor.clear()
      let rawReadings = "gao1 ke1 ji4 gong1 si1 de5 nian2 zhong1 jiang3 jin1"
      for i in 0 ..< 100 {
        rawReadings.split(separator: " ").forEach { key in
          compositor.insertKey(key.description)
          compositor.walk(useDAG: useDAG)
        }
      }
    }
  }

因為 Dijkstra 算法不涉及到 Retain Cycle,所以 Swift ARC 可以自動完成記憶體清掃工作、不需要手動 ASAN。

@ShikiSuen ShikiSuen changed the title Deprecating Lukhnos Liu's DAG algorithm, using Dijkstra instead. Deprecating Lukhnos Liu's DAG walker, using Dijkstra instead. Jan 25, 2025
@ShikiSuen ShikiSuen force-pushed the Shiki/WalkingAlgorithm branch 8 times, most recently from 7e628d1 to defc371 Compare January 26, 2025 05:06
- DAG is less competitive against optimized Dijkstra while benchmarking on Sandy-Bridge CPU. Note that no manual ASAN required since there's no retain cycle introduced.
@ShikiSuen ShikiSuen force-pushed the Shiki/WalkingAlgorithm branch 3 times, most recently from f29b09b to 749584c Compare January 26, 2025 05:46
@ShikiSuen ShikiSuen force-pushed the Shiki/WalkingAlgorithm branch from 749584c to 5b33cba Compare January 26, 2025 05:50
@ShikiSuen ShikiSuen merged commit 88b10fd into main Jan 26, 2025
1 check passed
@ShikiSuen ShikiSuen deleted the Shiki/WalkingAlgorithm branch January 26, 2025 05:51
ShikiSuen added a commit that referenced this pull request Jan 26, 2025
* Use Dijkstra in lieu of DAG as the default walking algorithm.

* Deprecating Lukhnos Liu's DAG walking algorithm.

- DAG is less competitive against optimized Dijkstra while benchmarking on Sandy-Bridge CPU. Note that no manual ASAN required since there's no retain cycle introduced.

* README // Add notices regarding differences from Gramambular 2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant