`cargo vendor` should vendor crates in a pristine state

[glandium]

In a large project, where different people might run cargo vendor with different versions of cargo, it would be better to avoid the unwanted noise from each version of cargo introducing changes to the cargo publish output.

Things were mostly fine until cargo 1.80, but since then almost each and every release of cargo has introduced changes that introduces churn on Cargo.toml, Cargo.lock and consequently .cargo-checksum.json.

Things like issue #14965 also wouldn't happen if the vendored crates were unmodified.

[epage]

This is a more general form of several issues we already have. Copying #15080 (comment)

Thanks for the report!

We have a number of cargo vendor issues related to it not replicating the contents of the .crate file because it essentially re-computes the list of files. Because the .crate file is no longer in a git repository, that logic changes (in this case, skipping hidden fiels). #14034 (comment) contains some notes about this (there are several other issues as well, though I don't see any mentioning this particular case).

I don't recall if there are any updates since #12509 (comment) about trying to change that.

[ehuss]

I'm a little unclear from the report if it is related to that issue, or if it is related to Cargo.toml normalization (which changes from version to version). cargo vendor should not be causing any changes to the three files mentioned.

@glandium Can you give a more detailed description of exactly which commands you are running, and which changes you are identifying between cargo versions?

[glandium]

Here are the changes that 1.80 introduced compared to 1.79:
https://hg.mozilla.org/integration/autoland/rev/c49e334c10e111e15d28563459556c06c2ec53dd

Then 1.81:
https://hg.mozilla.org/integration/autoland/rev/c85d718c19422a7b32286ad68e7ddbf2f2d2b018

1.83:
https://hg.mozilla.org/integration/autoland/rev/a50083c5e0877d1dc296d318cf8f697feee35a6f

1.84:
https://hg.mozilla.org/integration/autoland/rev/e4280732cdc2eb713cfdf4087c81869d9b8deb64

The command that is run is simply cargo vendor third_party/rust.

[ehuss]

Ah, I did not realize that was happening with cargo vendor. That's not good, and all the more reason to try again to implement a direct extraction.

[glandium]

(and 1.85 added another round of changes, this time moving where features are declared)

[weihanglo]

Hi. I know in general we want to do direct tarball extraction to fix all #9054, #13191, #14034, #15080, and this one. However, AFAIK except the change in 1.84 which additionally includes Cargo.lock, all the other changed dependencies seems to be git dependencies? Could you help confirm that @glandium? Thank you!

[glandium]

That is a good observation, they are from git dependencies.

[weihanglo]

If it is the case, let's continue the discussion of how git dependencies should be vendored in #7051, and leave this for crates from registries (which will be fixed by #15514).