When prototyping with the new lance_write_fragments api, I noticed the storage_opts doesn't actually support specifying the lance file version but object store params. The WriteParams is using the default file version, which currently is 2.0. We want to test all the file versions.
Suggest create a lance_write_versioned_fragments that allow us to specify a lance file version (at least all versions above 2).
When prototyping with the new lance_write_fragments api, I noticed the storage_opts doesn't actually support specifying the lance file version but object store params. The WriteParams is using the default file version, which currently is 2.0. We want to test all the file versions.
Suggest create a
lance_write_versioned_fragmentsthat allow us to specify a lance file version (at least all versions above 2).