Today we maintain and construct actual state on the world based on the annotations set in the pod at PreScore or PreFilter extension point. This is required because the scheduling cycle for pods in queue can be called and the pods we just processed don't yet have a node assigned. Having this annotations enables us to construct the state of what we know it'll look like after the scheduling is complete.

We need to validate if there are more optimizations around this with maintaining an in memory cache. Ref comment from our discussion in the issue: #23 (comment).