Add an generic internal representation for all physical files

[ashvina]

          I am wondering if there is a way to avoid adding these to the `InternalDeletionVector` but rather have this as an implementation detail of how the `ordinalStream` is generated.

Originally posted by @the-other-tim-brown in #634 (comment)