Skip to content

[Bug] Sparksql audit log query is not correct #5709

Open
@Pandas886

Description

@Pandas886

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

paimon 1.0.1

Compute Engine

spark 3.5.0

Minimal reproduce step

  spark.sql(
      """
        |create table my_table (
        |    k int,
        |    v string
        |) tblproperties (
        |    'primary-key' = 'k','changelog-producer' = 'lookup','bucket'='4'
        |);
        |""".stripMargin)
//
//    快照1
    spark.sql(" insert into my_table values (1, 'a'), (2, 'b')")
//    快照2
    spark.sql("delete from my_table where k = 1")
//    快照3
    spark.sql("insert into my_table values (11, 'a'), (2, 'bb')")

What doesn't meet your expectations?

when query like below

    spark.sql(
      """
        |SELECT * FROM paimon_incremental_query('`my_table$audit_log`', '0', '3')
        |""".stripMargin).show()

error result appear:

+-------+---+---+
|rowkind|  k|  v|
+-------+---+---+
|     +I|  1|  a|
|     +I|  2|  b|
+-------+---+---+

In my view, k=1 should -I , k=2 should -U and +U

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions