Consider memory deduplication in Object::load method (Cortex 9)

[ldmoser]

Currently we have in our StreamIndexedIO all the information would be required for high level Object deduplication at load time.

The Objects when saved, flush their directories into a sub-index in the StreamIndexedIO, which is deduplicated. So identical Objects are currently saved in identical Sub-index data blocks.

I believe we could add a hash() method to IndexedIO that would uniquely identify the current location and in case of a sub-index, it would use it's offset in the file. Then we could use a ComputationCache in the Object::load method to reuse loaded objects. The main drawback I can see is that for objects that are not repetitive, we would have an additional copy() call once we get them from the cache...

Just an idea to consider. If we have that, then the cache mechanisms in SceneCache reader will be unnecessary and we would save memory when loading models with repetitive structures.

[ldmoser]

An alternative to not affect Object::load method is to, instead, add a method to SampledSceneInterface for returning hashes of the stored IECore::Objects and then the cache mechanism would remain in the SceneCache object. We would still require a hash function in IndexedIO.