Expand "CI best practicies" section to the guide

[matklad]

Things to talk about:

• what directories should be cached (probably ~/.cargo, ./target, but keep ~/.cargo/credentials in mind)
• Rust channels (testing on current stable, beta and the oldest supported rustc)
• using --minimal-versions to verify dependencies in Cargo.toml
• setting-up rustfmt/clippy
• examples for popular CI services like GitHub Actions (Suggestion for cargo book Continuous Integration section to include GitHub Actions #7664)
• examples of other actions, like documentation generation, publishing to crates.io, publishing binaries (like GitHub releases), doing more advanced testing, etc. (Suggestion for cargo book Continuous Integration section to include more examples #7665)

Ideally, the section should contain copy-pastable config files for various CI providers.

[matklad]

A blog post about this feature: https://blog.illicitonion.com/2018/06/rust-minimum-versions-semver-is-lie.html

[ehuss]

I assume this means to expand https://github.com/rust-lang/cargo/blob/master/src/doc/src/guide/continuous-integration.md? Should it maybe leverage or at least point to japaric/trust?

[matklad]

Ah, I haven't realized that https://github.com/rust-lang/cargo/blob/master/src/doc/src/guide/continuous-integration.md exists.

Should it maybe leverage or at least point to japaric/trust?

Yep, we should definitely mention it!

[Eh2406]

@ehuss That is a good place for it to go, thanks for the reminder!

The impetus for this is that when we stabilize --minimal-versions we want to craft a message about how to use it. Specifically as a small part of a thorough CI setup. Witch will just lead everyone to ask, "What showed the big parts of the CI setup be?" So we thought we should write up our opinion before we stabilize --minimal-versions.

[withoutboats]

Relevant: rust-lang/rfcs#2483

[aturon]

I've just posted a blog post that talks about version selection in detail, and touches on the questions here as well.

[Eh2406]

That is an excellent read. Thank you for writing this up so articulately. I especially liked the paragraph of:

In the long run, it could even make sense to combine the two approaches, allowing crates to state their toolchain requirements (and have that influence resolution), but encourage core crates to state “LTS” as their requirement.

That sounds like the tradeoff bending I know and love from the rust ethic. I.E. a careful and pedantic system to ensure you are correct but with a large pit of success to make it usable. It also involves the most design work, so it may not be feasible.

In our earlier discussion today you suggested that "equality constraints are rare in the ecosystem" and I resisted being pedantic and pointing out lock files. So I was glad to see them addressed in your blog post. But I do have one nit with:

Similarly, when dependencies are subsequently adjusted, Cargo will “unlock” the affected dependencies and again choose the maximum version.

Currently that adjustment is very conservative, if the version of the sub dependency that appears in the lock file satisfies the requirement of the new dependency then it is not unlocked. Meaning that I am often testing a combination not in the well tested set. So if the new dependency does not have lower-bound precision and my lockfile is old enough then I will brake.

Thanks for the thought provoking guidance of this community.

[Nemo157]

There's also the https://crate-ci.github.io/ project (cc @epage). Maybe the Cargo Guide could have a small CI best practices section and link out to a larger exploration of the area like this?

[withoutboats]

In our earlier discussion today you suggested that "equality constraints are rare in the ecosystem" and I resisted being pedantic and pointing out lock files.

The important difference with lock files is that you once you have one you should have a successful build, and because it was generated using max versions, it will not run into the too low min version problem. Ceiling'd constraints are a problem because they influence version resolution, whereas the lockfile is after version resolution.

You do identify exactly where a problem can arise, though: cargo update -p. It might be worth considering changing the behavior of cargo update -p to update the entire subgraph under the crate you're updating.

[dwijnand]

More blog posts: https://levans.fr/rust_travis_cache.html (https://www.reddit.com/r/rust/comments/9d7sax/beware_the_rust_cache_on_travis_or_why_you_should/)

[Turbo87]

Currently that adjustment is very conservative, if the version of the sub dependency that appears in the lock file satisfies the requirement of the new dependency then it is not unlocked. Meaning that I am often testing a combination not in the well tested set. So if the new dependency does not have lower-bound precision and my lockfile is old enough then I will brake.

I've run into this problem quite a few times lately. I'm using dependabot on some of my projects to automatically update dependencies, and the updates regularly break CI because the lockfile has an older transitive dependency, but some other dependency relies on it being the most recent release.

Examples:

• the trait bound rand_hc::Hc128Rng: rand_core::SeedableRng is not satisfied actix/actix#184
• trust-dns-proto fails to compile actix/actix#177

It’s possible that we will eventually have workflows that depend on the accuracy of lower bounds in Cargo.toml. At the moment, however, this is purely speculative; the Cargo team does not have any ready examples.

Hopefully the examples above are helpful in that regard