Is it valid for a remove action to reference a file path that has never been added to the table? #5915
Unanswered
shanisolomon
asked this question in
Ask the Authors
Replies: 2 comments
-
|
@larsk-db perhaps you know the answer? thanks! |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@shanisolomon sorry for the slow response. I had to check with some people. The answer is "yes", readers need to gracefully handle states where there are tombstones for files that never occurred in any AddFile entry before. The expectation is that such tombstones are ignored by readers during state reconstruction, but they are honoured by VACUUM implementations (i.e. such files do not get deleted by VACUUM until the timestamp + file retention on their tombstones has expired). |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Question: Is it valid for a remove action to reference a file path that has never been added to the table (i.e., no corresponding add action exists in any previous version)?
Context: The protocol clearly states that remove actions create tombstones and that invalid actions (like removes that cancel out adds) are handled during action reconciliation. However, it doesn't explicitly state whether a remove for a non-existent path is:
What I've found in the protocol:
- The description implies removes follow adds ("removing a file that has been added")
- Tombstones are described as marking "files that were recently deleted"
- The dataChange=false case mentions removed records must be "contained in one or more add file actions"
But there's no explicit prohibition or guidance for the case where no prior add exists.
Use case: I'm implementing a Delta reader and want to ensure proper handling of edge cases and potentially malformed transaction logs.
Would appreciate any clarification on the expected behavior for both writers and readers in this scenario.
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions