Rotating ImageMobject in 3D is buggy

[sparshg]

Description of bug / unexpected behavior

In the animation below where image 'lies' on NumberPlane, It always remains a rectangle instead of getting distorted into a trapezium.
However, doing same with opengl renderer works as expected.

Bug Animation

test3.mp4

Code

num = NumberPlane()
img = ImageMobject("test.png")
self.add(num, img)
self.play(Rotate(num, -PI / 2, axis=RIGHT), Rotate(img, -PI / 2, axis=RIGHT))

Logs

Terminal output

PASTE HERE OR PROVIDE LINK TO https://pastebin.com/ OR SIMILAR

System specifications

System Details

• OS (with version, e.g., Windows 10 v2004 or macOS 10.15 (Catalina)):
• RAM:
• Python version (python/py/python3 --version):
• Installed modules (provide output from pip list):

PASTE HERE

LaTeX details

• LaTeX distribution (e.g. TeX Live 2020):
• Installed LaTeX packages:

FFMPEG

Output of ffmpeg -version:

PASTE HERE

Additional comments

[kevinlinxc]

This also happens with the camera orientation:

MyScene.mp4

from manim import *

class MyScene(ThreeDScene):
    def construct(self):
        self.video1 = ImageMobject(
                filename_or_array="cat.png",
            ).scale_to_fit_height(3)
        ax = Axes(
            x_range=[0, 10, 1],
            x_length=9,
            y_range=[0, 20, 5],
            y_length=6,
            axis_config={"include_numbers": True, "include_tip": False},

        ).to_edge(DL + RIGHT + UP, buff=1).scale(0.7)
        labels = ax.get_axis_labels()

        self.play(Create(VGroup(ax, labels)))
        self.play(FadeIn(self.video1))
        self.wait(3)
        self.move_camera(phi=0*DEGREES, theta= -90 * DEGREES, zoom= 0.7, run_time=0.4, gamma=0*DEGREES)

        self.begin_ambient_camera_rotation(90 * DEGREES / 3, about='phi')
        self.begin_ambient_camera_rotation(90 * DEGREES / 3, about='theta')
        self.wait(3)

[kleinesfilmroellchen]

I banged my head against this for several hours, but I figured something out that may be worth investigating:

Pillow's Image has a transform method that can do a perspective transform. The parameter appears to just be the homogeneous 3D matrix, so it should be possible to use the inverse of the rotation matrix shifted to the center of the image. (inverse(translation) * rotation * translation). I couldn't get this to work however.

[uwezi]

I don't think you want to mess around with the pixel matrix itself. The Manim renderer should correctly find the pixels in the image matrix and calculate its position on the projected camera's focus plane.