feat: Variant Support

[c-thiel]

Which issue does this PR close?

Variant Support.
Arrow value support is currently missing as I am unsure how we want to extend Literal

What changes are included in this PR?

Core: Variant Type

• crates/iceberg/src/spec/datatypes.rs — new Variant type
• crates/iceberg/src/spec/values/literal.rs — Variant literal value
• crates/iceberg/src/spec/schema/ — visitor, index, pruning, mod, id reassigner all handle Variant
• crates/iceberg/src/spec/table_metadata.rs — metadata support

Avro

• crates/iceberg/src/avro/schema.rs — read/write Variant in Avro

Arrow

• crates/iceberg/src/arrow/schema.rs — map Variant to Arrow type
• crates/iceberg/src/arrow/reader.rs — read Variant from Arrow
• crates/iceberg/src/arrow/value.rs — Arrow value conversion
• Minor fixes in caching_delete_file_loader.rs and nan_val_cnt_visitor.rs

Parquet

• crates/iceberg/src/writer/file_writer/parquet_writer.rs — write Variant columns

Tests & Dev

• crates/integration_tests/tests/read_variant.rs — new integration test for reading Variant data
• dev/spark/provision.py — Spark provisioning to generate Variant test data

Are these changes tested?

Sure! Even integration tested :)

[c-thiel]

Before this change existing tests where rightfully failing as we used to create a V2 table with a NS Timestamp column:
https://github.com/apache/iceberg-rust/actions/runs/22522306915/job/65248930667

This new logic determines the min format version required and uses that - but at least V2. Thus we switch now to V3 for ns timestamps.