HIVE-29815: Iceberg: [V3] Support ROW LINEAGE in Copy-On-Write DELETE operations

[Aggarwal-Raghav]

What changes were proposed in this pull request?

Check HIVE-29815
This PR updates CopyOnWriteDeleteRewriter.java to explicitly select and preserve ROW__LINEAGE__ID and LAST__UPDATED__SEQUENCE__NUMBER when rewriting files during a Copy-On-Write DELETE operation on Iceberg V3 tables.

Why are the changes needed?

Currently, when a user deletes a row in COW mode, the surviving rows are rewritten into a new file without their original lineage metadata. As a result, they get assigned brand new lineage IDs and sequence numbers

Does this PR introduce any user-facing change?

Yes

How was this patch tested?

With q file and on spark engine as well

[Aggarwal-Raghav]

spark Output:

Hive Before Fix:

[ayushtkn]

thanx @Aggarwal-Raghav for the fix, minor request around the test.

• @kokila-19 can u review as well

[ayushtkn]

can u add a test around a partitioned table as well

[Aggarwal-Raghav]

Done

[ayushtkn]

If the build comes clean, changes LGTM

[kokila-19]

Good Catch @Aggarwal-Raghav
LGTM +1

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Aggarwal-Raghav]

I've found 1 more potential bug/scope of improvement. Please hold on merging it.

[Aggarwal-Raghav]

The new issue i found was for row lineage for MERGE WHEN DELETE query.

MERGE INTO ice_cow_merge_delete_only t
USING merge_source s
ON t.id = s.id
WHEN MATCHED THEN DELETE;

The following code doesn't check for merge when delete. it only checks for hasWhenMatchedUpdateClause

hive/ql/src/java/org/apache/hadoop/hive/ql/metadata/RowLineageUtils.java

Lines 109 to 114 in b88f948

	public static boolean shouldAddRowLineageColumnsForMerge(MergeStatement mergeStatement, Configuration conf) {
	boolean shouldAddRowLineageColumns =
	supportsRowLineage(mergeStatement.getTargetTable()) && mergeStatement.hasWhenMatchedUpdateClause();
	setRowLineage(conf, shouldAddRowLineageColumns);
	return shouldAddRowLineageColumns;
	}

[Aggarwal-Raghav]

@ayushtkn
There is 1 more bug in hive. Merge when delete is adding additonal row instead of deleteing it. I'm filing a new JIRA for it

SPARK:

HIVE:

[Aggarwal-Raghav]

Filed HIVE-29816. You can proceed with this PR merge as per your convenience