Add function to calculate representativeness of statement for groups

[patcon]

• https://github.com/nishio/UTAS-UMAP/blob/main/UTAS-Polis.ipynb#Show-significant-comments
• https://github.com/compdemocracy/analysis-python/blob/main/notebooks/american-assembly-specific/american-assembly-representative-groups-and-comments.ipynb#Comment-Statistics
• https://github.com/compdemocracy/analysis/blob/master/notebooks/jupyter/american-assembly-representative-groups-and-comments.ipynb#Comment-Statistics

[patcon]

First pass from @ThenWho:

#17 (comment)

The main difference I see between yours and the others @ThenWho is that yours count only agree/disagree, whereas the other jupyter notebooks (from compdem and nishio) count any finite values, so agree/disagree/pass.

If the answer isn't immediately evident to you, I will just implement it as adjustable, then re-read the paper and try to figure out how clj code does it :)

[patcon]

k, i think votes is all of agree/disagree/pass.

I think you were misled with this official-looking notebook:

https://github.com/compdemocracy/analysis/blob/master/notebooks/jupyter/american-assembly-representative-groups-and-comments.ipynb

But it seems like it was just added by some random guy who was pretty vocal that he's didn't understand the methods very well, and the bit in red he was confused about:

However, some methodology is not 100% clear to me. I marked them in red in the markdown cells. In case we can resolve my lack of understanding, I am happy to implement the scaling + comment embedding as well.

-- compdemocracy/analysis#4

The paper is a bit ambiguous:

𝑁(𝑔, 𝑐) be the total number of votes on comment c within group g

-- https://hyp.is/U-3w_v1xEe-nROthFyv2ww/gwern.net/doc/sociology/2021-small.pdf

Diving into math clj code, it's pretty apparent that a vote includes all of agree/disagree/pass:

Notes:

• na/nd/ns = n_agreed/n_disagreed/n_seen
• count-votes without an arg counts length of array of any non-missing votes. aka any +1, 0, -1 values.

"seen" is short for any non-null vote value, and is called such in the codebase because "unseen" values are null in vote matrix.

https://github.com/compdemocracy/polis/blob/90bcb43e67dad660629e0888fedc0d32379f375d/math/src/polismath/math/repness.clj#L56-L75

(defn- count-votes
  [votes & [vote]]
  "Utility function for counting the number of votes matching `vote`. Not specifying `vote` returns
  length of vote vector, sans any missed votes."
  (let [filt-fn (if vote #(= vote %) identity)]
    (count (filter filt-fn votes))))


(def comment-stats-graphimpl
  ; The graph implementation is in a closure so that we don't reify classes every time function is called.
  ; This can cause PermGen space crashes. This is used for both consensus and repness comments.
  (gr/eager-compile
    {:na (fnk [votes] (count-votes votes -1))
     :nd (fnk [votes] (count-votes votes  1))
     :ns (fnk [votes] (count-votes votes))
     ; XXX - Change when we flip votes!!!
     :pa (fnk [na ns] (/ (+ 1 na) (+ 2 ns)))
     :pd (fnk [nd ns] (/ (+ 1 nd) (+ 2 ns)))
     :pat (fnk [na ns] (stats/prop-test na ns))
     :pdt (fnk [nd ns] (stats/prop-test nd ns))}))

I wish the notebook wasn't published in such an authoritative place, as it's probably misled/discouraged quite a few ppl.

Anyhow, in conclusion, i feel very confident that vote count should be any votes :)

[patcon]

cc: @nishio since I'm seeing you also may have only counted [-1, 1] as total votes, when it should have been [-1, 0, 1].

https://github.com/nishio/UTAS-UMAP/blob/main/UTAS-Polis.ipynb

        # total votes corresponds to votes with +1 or -1
        # N(g,c)は、グループgでコメントcに+1か-1で投票した人数の合計
        # N(g,c) is the total number of people in group g who voted +1 or -1 on comment c.  
        N_g_c[g, c] = N_v_g_c[0, g, c] + N_v_g_c[2, g, c]

Just in case the docstring was misleading, and i was failing to understand clojure syntax, I confirmed with chatgpt what this clojure code is doing:

[ThenWho]

Thanks for the careful investigation @patcon, you're absolutely right! I was only calculating agree and disagree votes, but the pass ones should definitely be included.

In my defense, Clojure is more clear-cut for this type of thing. Pandas and NumPy arrays treat missing values differently (Pandas uses NaN for missing values while NumPy tends to use masks or sentinel values), which makes it less obvious whether zeros represent missing values or actual "pass" votes.

I saw what you've done with the code and your use of the np.isfinite() function is great - that's exactly the way to go. It properly distinguishes between actual zeros (passes) and missing values.

Overall, thank you very much for the fix. This is exactly right and will make the representativeness calculation much more accurate.