Tools to help you write better code - Princeton Wintersession
0 likes204 views
The document discusses various tools to enhance code quality and development efficiency in Python, emphasizing the use of the UV tool for better package management and conflict resolution. It outlines applications like cibuildwheel, jupylite, and ruff, as well as the utility of pre-commit hooks for maintaining code quality. Additionally, it highlights Nox for task automation and mentions the fast adoption of the ruff linter in the Python ecosystem.
![Quick scripts solution!
7
#!/usr/bin/env uv run
# /// script
# dependencies = ["rich"]
# ///
import rich
rich.print("[blue]This worked!")
uv run ./print_blue.py
# OR
./print_blue.py](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-10-320.jpg)
![Writing a nox
fi
le.py
9
import nox
nox.options.default_venv_backend = "uv|virtualenv"
@nox.session(python=["3.9", "3.10", "3.11", "3.12", "3.13"])
def tests(session: nox.Session) -> None:
"""
Run the unit and regular tests.
"""
session.install(".[test]")
session.run("pytest", *session.posargs)](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-12-320.jpg)
![Running nox
10
~/g/s/uproot-browser henryiii/feat/logo *$ nox -s tests-3.9
nox > Running session tests-3.9
nox > Creating virtual environment (virtualenv) using python3.9 in .nox/tests-3-9
nox > python -m pip install '.[test]'
nox > pytest
=========================================== test session starts ===========================================
platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/henryschreiner/git/scikit-hep/uproot-browser, configfile: pyproject.toml, testpaths: tests
collected 3 items
tests/test_dirs.py .. [ 66%]
tests/test_package.py . [100%]
=========================================== 3 passed in 0.01s =============================================
nox > Session tests-3.9 was successful.](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-13-320.jpg)
![Con
fi
guring pre-commit
17
Design
A hook is just a YAML dict
Fields can be overridden
Environments globally cached by git tag
Supports checks and
fi
xers
You can have as many as you want
Must use a static tag
# .pre-commit-config.yaml
hooks:
- repo: https://github.com/abravalheri/validate-pyproject
rev: "0.23"
hooks:
- id: validate-pyproject
# validate-pyproject .pre-commit-hooks.yaml
- id: validate-pyproject
name: Validate pyproject.toml
description: >
Validation library for a simple check
on pyproject.toml, including optional dependencies
language: python
files: ^pyproject.toml$
entry: validate-pyproject
additional_dependencies:
- .[all]
You write this
Formatter author writes this](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-21-320.jpg)
![Ruff con
fi
g example
27
[tool.ruff.lint]
extend-select = [
"B", # flake8-bugbear
"I", # isort
"ARG", # flake8-unused-arguments
"C4", # flake8-comprehensions
"EM", # flake8-errmsg
"ICN", # flake8-import-conventions
"PGH", # pygrep-hooks
"PIE", # flake8-pie
"PL", # pylint
"PT", # flake8-pytest-style
"RET", # flake8-return
"RUF", # Ruff-specific
"SIM", # flake8-simplify
"T20", # flake8-print
"UP", # pyupgrade
"YTT", # flake8-2020
]
typing-modules = ["somepackage._compat.typing"]
[tool.ruff.lint.per-file-ignores]
"tests/**" = ["T20"]
"noxfile.py" = ["T20"]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.9.1"
hooks:
- id: ruff
args: ["--fix", "--show-fixes"]
- id: ruff-format
Flake8 con
fi
g? Try:
pipx run flake8-to-ruff .flake8
https://learn.scienti
fi
c-python.org/development/guides/style/#ru
f](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-37-320.jpg)
![Running Ruff on notebooks
31
pre-commit? This is now the default!:
types_or: [python, pyi, jupyter]
Native support
Linter and formatter support notebooks
md/rst standalone
fi
le support planned](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-42-320.jpg)
![Formatting code snippets
32
Ru
ff
native support planned!
(Already handles docstrings)
- repo: https://github.com/adamchainz/blacken-docs
rev: "1.19.1"
hooks:
- id: blacken-docs
additional_dependencies: [black==24.*]
Blacken docs
Adapts black to md/rst
fi
les](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-43-320.jpg)
![Code I: isort
38
Sort your Python imports
Very con
fi
gurable
Reduces merge con
fl
icts
Grouping imports helps readers
Can inject future imports
args: ["-a", "from __future__ import annotations"]
Default groupings
Future imports
Stdlib imports
Third party packages
Local imports
from __future__ import annotations
import dataclasses
import graphlib
import textwrap
from collections.abc import Mapping, Set
from typing import Any, TypeVar
import markdown_it
from .checks import Check
from .families import Family, collect_families
from .fixtures import pyproject
from .ghpath import EmptyTraversable](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-49-320.jpg)
![Code UP: pyupgrade
39
Update Python syntax
Avoid deprecated or obsolete code
Fairly cautious
Can target a speci
fi
c Python 3 min
(Mostly) not con
fi
gurable
Remove static if sys.version_info blocks
Python 2.7
Set literals
Dictionary comprehensions
Generators in functions
Format speci
fi
er & .format ⚙
Comparison for const literals (3.8 warn)
Invalid escapes
Python 3
Unicode literals
Long literals, octal literals
Modern super()
New style classes
Future import removal
yield from
Remove six compatibility code
io.open -> open
Remove error aliases
Python 3.x
f-strings (partial) (3.6) ⚙
NamedTuple/TypedDict (3.6)
subprocess.run updates (3.7)
lru_cache parens (3.8)
lru_cache(None) -> cache (3.9)
Typing & annotation rewrites (various)
abspath(__file__) removal (3.9)
Before After
for a, b in c:
yield (a, b)
yield from c
"{foo} {bar}".format(foo=foo, bar=bar) f"{foo} {bar}"
dict([(a, b) for a, b in y]) {a: b for a, b in y}](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-50-320.jpg)
![pyupgrade limits
40
PyUpgrade does not over modernize
isinstance(x, (int, str)) -> isinstance(x, int | str) (3.10)
No match statement conversions (3.10)
Nothing converts to using walrus := (3.8) (probably a good thing!)
Except for a bit of typing
Optional[int] -> int | None (I like this one now, though)
❌](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-51-320.jpg)
![hooks:
- repo: https://gitlab.com/pycqa/flake8
rev: "7.1.1"
hooks:
- id: flake8
Flake8
43
Fast simple extendable linter
Very con
fi
gurable: setup.cfg or .
fl
ake8
Doesn’t support pyproject.toml
Many plugins, local plugins easy
No auto-
fi
xers like rubocop (Ruby)
Still great for custom checks
# .flake8
[flake8]
max-complexity = 12
extend-ignore = E203, E501, E722, B950
extend-select = B9](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-54-320.jpg)
![Custom local
fl
ake8 plugin
45
class Visitor(ast.NodeVisitor):
msg = "AK101 exception must be wrapped in ak._v2._util.*error"
def __init__(self) -> None:
self.errors: list[Flake8ASTErrorInfo] = []
def visit_Raise(self, node: ast.Node) -> None:
if isinstance(node.exc, ast.Call):
if isinstance(node.exc.func, ast.Attribute):
if node.exc.func.attr in {"error", "indexerror"}:
return
if node.exc.func.id in {"ImportError"}:
return
self.errors.append(
Flake8ASTErrorInfo(node.lineno, node.col_offset, self.msg, type(self))
)](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-56-320.jpg)
![Custom local
fl
ake8 plugin
46
class AwkwardASTPlugin:
name = "flake8_awkward"
version = "0.0.0"
def __init__(self, tree: ast.AST) -> None:
self._tree = tree
def run(self) -> Iterator[Flake8ASTErrorInfo]:
visitor = Visitor()
visitor.visit(self._tree)
yield from visitor.errors](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-57-320.jpg)
![Custom local
fl
ake8 plugin
47
[flake8:local-plugins]
extension =
AK1 = flake8_awkward:AwkwardASTPlugin
paths =
./dev/
def main(path: str) -> None:
with open(path) as f:
code = f.read()
node = ast.parse(code)
plugin = AwkwardASTPlugin(node)
for err in plugin.run():
print(f"{path}:{err.line_number}:{err.offset} {err.msg}")
if __name__ == "__main__":
for item in sys.argv[1:]:
main(item)](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-58-320.jpg)
![PyLint
48
PyLint recommends having your project installed, so it is not a good pre-commit hook (though you can do it)
It’s also a bit slow, so a good candidate for nox
@nox.session
def pylint(session: nox.Session) -> None:
session.install("-e.")
session.install("pylint")
session.run("pylint", "src", *session.posargs)
# pyproject.toml
[tool.pylint]
master.py-version = "3.9"
master.jobs = "0"
reports.output-format = "colorized"
similarities.ignore-imports = "yes"
messages_control.enable = ["useless-suppression"]
messages_control.disable = [
"design",
"fixme",
"line-too-long",
"wrong-import-position",
]
Code linter
Can be very opinionated
Signal to noise ratio poor
You will need to disable checks - that’s okay!
A bit more advanced / less static than
fl
ake8
But can catch hard to
fi
nd bugs!
For an example of lots of suppressions:
https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/pyproject.toml
Some parts available in Ruff](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-59-320.jpg)
![Con
fi
guring MyPy
52
[tool.mypy]
files = "src"
python_version = "3.9"
warn_unused_configs = true
strict = true
[[tool.mypy.overrides]]
module = [ "numpy.*" ]
ignore_missing_imports = true
Start small
Start without strictness
Add a check at a time
Extra libraries
Try adding them to your environment
You can ignore untyped or slow libraries
You can provide stubs for untyped libraries if you want
Tests?
Adding pytest is rather slow
I prefer to avoid tests, or keep them mostly untyped
(unless the package is very important)](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-64-320.jpg)
![Future annotations
54
Classic code (3.5+)
from typing import Union, List
def f(x: int) -> List[int]:
return list(range(x))
def g(x: Union[str, int]) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
Modern code (3.7+)
from __future__ import annotations
def f(x: int) -> list[int]:
return list(range(x))
def g(x: str | int) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
Ultramodern code (3.10+)
def f(x: int) -> list[int]:
return list(range(x))
def g(x: str | int) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
With the future import, you get all the bene
fi
ts of future code in 3.7+ annotations
Typing is already extra code, simpler is better](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-66-320.jpg)
![Clang-format
56
hooks:
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: "v19.1.6"
hooks:
- id: clang-format
types_or: [c++, c, cuda]
C++ and more code formatter
Very con
fi
gurable: .clang-format
fi
le
Opinion: stay close to llvm style
PyPI clang-format wheels, under 2MB
No more issues with mismatched LLVM!](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-68-320.jpg)
![CMake-format
57
hooks:
- repo: https://github.com/cheshirekow/cmake-format-precommit
rev: "v0.6.13"
hooks:
- id: cmake-format
additional_dependencies: [pyyaml]
CMake code formatter
Very con
fi
gurable: .cmake-format.yaml
fi
le
Anything that helps with CMake!](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-69-320.jpg)
![Markdown & YAML with Prettier
58
hooks:
- repo: https://github.com/rbubley/mirrors-prettier
rev: "v3.4.2"
hooks:
- id: prettier
types_or: [yaml, markdown, html, css, scss, javascript, json]
args: [--prose-wrap=always]
exclude: "^tests"
JavaScript Linter
Lots of formats supported
A few customization points](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-70-320.jpg)
![CodeSpell
60
hooks:
- repo: https://github.com/codespell-project/codespell
rev: "v2.3.0"
hooks:
- id: shellcheck
args: ["-L", "sur,nd"]
Find common misspellings
Inverted spell checker - looks for misspellings
Can con
fi
gure or provide wordlist
Actually can catch bugs!
Pass -w to
fi
x, too](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-72-320.jpg)
![pytest tips
64
Spend time learning pytest
Full of amazing things that really make testing fun!
Tests are code too
Or for C++: Catch2 or doctest, etc.
Also maybe learn Hypothesis for pytest
[tool.pytest.ini_options]
minversion = "6.0"
addopts = [
"-ra",
"--showlocals",
"--strict-markers",
"--strict-config",
]
xfail_strict = true
filterwarnings = [
"error",
]
log_cli_level = "info"
testpaths = [
"tests",
]
Use pytest.approx
Even works on numpy arrays
Remember to test for failures
If you expect a failure, test it!
Test your installed package
That’s how users will get it, not from a directory](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-76-320.jpg)
![pytest tips
64
Spend time learning pytest
Full of amazing things that really make testing fun!
Tests are code too
Or for C++: Catch2 or doctest, etc.
Also maybe learn Hypothesis for pytest
[tool.pytest.ini_options]
minversion = "6.0"
addopts = [
"-ra",
"--showlocals",
"--strict-markers",
"--strict-config",
]
xfail_strict = true
filterwarnings = [
"error",
]
log_cli_level = "info"
testpaths = [
"tests",
]
Don’t let warnings slip by!
Makes logging more useful
Strictness is good
Useful summary
Print out locals on errors
Use pytest.approx
Even works on numpy arrays
Remember to test for failures
If you expect a failure, test it!
Test your installed package
That’s how users will get it, not from a directory](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-77-320.jpg)
![pytest Tricks
65
Mock and Monkeypatch
This is how you make tricky tests “unit” tests
Fixtures
This keeps tests simple and scalable
@pytest.fixture(params=["Linux", "Darwin", "Windows"], autouse=True)
def platform_system(request, monkeypatch):
monkeypatch.setattr(platform, "system", lambda _: request.param)
Parametrize
Directly or in a
fi
xture for reuse
Use conftest.py
Fixtures available in same and nested directories](https://image.slidesharecdn.com/20250122sqat-250122202107-a7cc45ab/85/Tools-to-help-you-write-better-code-Princeton-Wintersession-78-320.jpg)
Tools to help you write better code - Princeton Wintersession
- 1. Tools That Help You Write Better Code Henry Schreiner January 22, 2025 ISciNumPy.dev https://iscinumpy.dev Scienti fi c-Python Development Guide https://learn.scienti fi c-python.org/development https://github.com/henryiii/sqat-example
- 2. Language We’ll focus on Python (especially at fi rst) But the general concepts are around in most other languages You just need to fi nd the matching tool(s)
- 3. Popularity comparison 3 I used to cover single tools like pipx
- 4. Popularity comparison 4 But then this happened:
- 5. One tool to rule them all For packaging related things, we'll use uv wherever we can. You can use other tools, but they are much slower and you need more tools. Install uv before beginning! https://docs.astral.sh/uv/getting-started/installation pip pipx twine venv poetry Partially generated with Copilot
- 6. Packaging aside: uv tool / uvx 6 $ pip install <application> $ <application> I’m sure you’ve seen this: Examples of applications: cibuildwheel: make redistributable wheels nox/tox: Python task runners jupylite: WebAssembly Python site builder ruff: Python code quality tooling pypi-command-line: query PyPI uproot-browser: ROOT file browser (HEP) tiptop: fancy top-style monitor rich-cli: pretty print files cookiecutter: template packages clang-format: format C/C++/CUDA code pre-commit: general CQA tool cmake: build system generator meson: another build system generator ninja: build system (native tool replaced: pipx)
- 7. Packaging aside: uv tool / uvx 6 $ pip install <application> $ <application> I’m sure you’ve seen this: Examples of applications: cibuildwheel: make redistributable wheels nox/tox: Python task runners jupylite: WebAssembly Python site builder ruff: Python code quality tooling pypi-command-line: query PyPI uproot-browser: ROOT file browser (HEP) tiptop: fancy top-style monitor rich-cli: pretty print files cookiecutter: template packages clang-format: format C/C++/CUDA code pre-commit: general CQA tool cmake: build system generator meson: another build system generator ninja: build system Packages can con fl ict Updates get slower over time Lose track of why things are installed Manual updates are painful Hates Python being replaced (native tool replaced: pipx)
- 8. Packaging aside: uv tool / uvx 6 $ pip install <application> $ <application> I’m sure you’ve seen this: Examples of applications: cibuildwheel: make redistributable wheels nox/tox: Python task runners jupylite: WebAssembly Python site builder ruff: Python code quality tooling pypi-command-line: query PyPI uproot-browser: ROOT file browser (HEP) tiptop: fancy top-style monitor rich-cli: pretty print files cookiecutter: template packages clang-format: format C/C++/CUDA code pre-commit: general CQA tool cmake: build system generator meson: another build system generator ninja: build system Packages can con fl ict Updates get slower over time Lose track of why things are installed Manual updates are painful Hates Python being replaced $ uv tool install <application> $ <application> Better! Automatic venv for each package No con fl icts ever Everything updatable / replaceable Doesn’t like Python being replaced (native tool replaced: pipx)
- 9. Packaging aside: uv tool / uvx 6 $ pip install <application> $ <application> I’m sure you’ve seen this: Examples of applications: cibuildwheel: make redistributable wheels nox/tox: Python task runners jupylite: WebAssembly Python site builder ruff: Python code quality tooling pypi-command-line: query PyPI uproot-browser: ROOT file browser (HEP) tiptop: fancy top-style monitor rich-cli: pretty print files cookiecutter: template packages clang-format: format C/C++/CUDA code pre-commit: general CQA tool cmake: build system generator meson: another build system generator ninja: build system Packages can con fl ict Updates get slower over time Lose track of why things are installed Manual updates are painful Hates Python being replaced $ uv tool install <application> $ <application> Better! Automatic venv for each package No con fl icts ever Everything updatable / replaceable Doesn’t like Python being replaced $ uvx <application> Best! Automatic venv caching Never more than a week old No pre-install or setup No maintenance Replace Python at will uvx --from git+https://github.com/henryiii/rich-cli@patch-1 rich pipx has fi rst class support on GHA & Azure, and uv has a native action. (native tool replaced: pipx)
- 10. Quick scripts solution! 7 #!/usr/bin/env uv run # /// script # dependencies = ["rich"] # /// import rich rich.print("[blue]This worked!") uv run ./print_blue.py # OR ./print_blue.py
- 11. Task runner aside: Nox 8 Make fi les Custom language Painful to write Painful to maintain Looks like garbage OS dependent No Python environments Everywhere Tox Custom language Concise to write Tricky to read Ties you to tox OS independent Python environments Python package Nox Python, mimics pytest Simple but verbose Easy to read Teaches commands OS independent Python environments Python package Other task runners available for other purposes, like Rake (Ruby) Hatch TOML con fi g Intermediate Intermediate Integrated with packaging OS independent Python environments Python package Note that with uv, you might not need nox if tasks are all simple!
- 12. Writing a nox fi le.py 9 import nox nox.options.default_venv_backend = "uv|virtualenv" @nox.session(python=["3.9", "3.10", "3.11", "3.12", "3.13"]) def tests(session: nox.Session) -> None: """ Run the unit and regular tests. """ session.install(".[test]") session.run("pytest", *session.posargs)
- 13. Running nox 10 ~/g/s/uproot-browser henryiii/feat/logo *$ nox -s tests-3.9 nox > Running session tests-3.9 nox > Creating virtual environment (virtualenv) using python3.9 in .nox/tests-3-9 nox > python -m pip install '.[test]' nox > pytest =========================================== test session starts =========================================== platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0 rootdir: /Users/henryschreiner/git/scikit-hep/uproot-browser, configfile: pyproject.toml, testpaths: tests collected 3 items tests/test_dirs.py .. [ 66%] tests/test_package.py . [100%] =========================================== 3 passed in 0.01s ============================================= nox > Session tests-3.9 was successful.
- 14. Features of nox 11 Full control over environments Easy fl y-by contributions Transparent, simple .nox directory Conda support Trade speed for reproducibility Some ideas for sessions lint tests docs build bump pylint regenerate update_pins check_manifest make_changelog update_python_dependencies See pypa/cibuildwheel pypa/manylinux scikit-hep/hist scikit-hep/boost-histogram pybind/pybind11 scientific-python/cookie scientific-python/repo-review scikit-hep/scikit-hep.github.io Optional environment reuse Use -R for speed! (Reuse environment and skip installs)
- 15. Python launcher for Unix 12 Rust implementation of “py” for UNIX But also automatically picks up .venv folder! Meant for lazy experts Launcher $ py -m pytest Classic $ . .venv/bin/activate (.venv) $ python -m pytest (.venv) $ deactivate Classic, take 2 $ .venv/bin/python -m pytest
- 16. Cookiecutter 13 Quickly set up a project Takes options scienti fi c-python/cookie is a great cookiecutter for Python! How to run uvx cookiecutter gh:scientific-python/cookie
- 17. Part 0: Intro 14
- 18. Code Quality 15 Why does code quality matter? Improve readability Find errors before they happen Avoid historical baggage Reduce merge con fl icts Warm fuzzy feelings How to run Discussion of checks (Opinionated) Mostly focusing on Python today
- 19. pre-commit 16 Poorly named? Has a pre-commit hook mode You don’t have to use it that way! Generic check runner conda coursier dart docker docker_image dotnet fail golang lua node perl python python_venv r ruby rust swift pygrep script system Written in Python pipx, nox, homebrew, etc. Designed for speed & reproducibility Ultra fast environment caching Locked environments Easy autoupdate command pre-commit.ci Automatic updates Automatic fi xes for PRs Large library of hooks https://pre-commit.com/hooks.html Custom hooks are simple
- 20. Con fi guring pre-commit 17 Design A hook is just a YAML dict Fields can be overridden Environments globally cached by git tag Supports checks and fi xers You can have as many as you want Must use a static tag # .pre-commit-config.yaml hooks: - repo: https://github.com/abravalheri/validate-pyproject rev: "0.23" hooks: - id: validate-pyproject You write this
- 21. Con fi guring pre-commit 17 Design A hook is just a YAML dict Fields can be overridden Environments globally cached by git tag Supports checks and fi xers You can have as many as you want Must use a static tag # .pre-commit-config.yaml hooks: - repo: https://github.com/abravalheri/validate-pyproject rev: "0.23" hooks: - id: validate-pyproject # validate-pyproject .pre-commit-hooks.yaml - id: validate-pyproject name: Validate pyproject.toml description: > Validation library for a simple check on pyproject.toml, including optional dependencies language: python files: ^pyproject.toml$ entry: validate-pyproject additional_dependencies: - .[all] You write this Formatter author writes this
- 22. Options for pre-commit 18 Selected options fi les: explicit include regex exclude: explicit exclude regex types_or/types/exclude_types: fi le types args: control arguments additional_dependencies: extra things to install stages: select the git stage (like manual)
- 23. Running pre-commit 19 Run all checks pre-commit run -a Update all hooks pre-commit autoupdate
- 24. Running pre-commit 19 Run all checks pre-commit run -a Update all hooks pre-commit autoupdate Install as a pre-commit hook pre-commit install (Skip with git commit -n)
- 25. Running pre-commit 19 Run all checks pre-commit run -a Update all hooks pre-commit autoupdate Install as a pre-commit hook pre-commit install (Skip with git commit -n) Skip checks SKIP=… <run> Run one check pre-commit run -a <id> Run manual stage pre-commit run --hook-stage manual
- 26. Examples of pre-commit checks 20 Almost everything following in this talk - repo: local hooks: - id: disallow-caps name: Disallow improper capitalization language: pygrep entry: PyBind|Numpy|Cmake|CCache|Github|PyTest exclude: .pre-commit-config.yaml
- 27. Examples of pre-commit checks 20 Almost everything following in this talk - repo: local hooks: - id: disallow-caps name: Disallow improper capitalization language: pygrep entry: PyBind|Numpy|Cmake|CCache|Github|PyTest exclude: .pre-commit-config.yaml Don’t grep the fi le this is in! “Entry” is the grep, in this case Using pygrep “language” Custom hook
- 28. pre-commit/pre-commit-hooks 21 - repo: https://github.com/pre-commit/pre-commit-hooks rev: "v4.5.0" hooks: - id: check-added-large-files - id: check-case-conflict - id: check-merge-conflict - id: check-symlinks - id: check-yaml - id: debug-statements - id: end-of-file-fixer - id: mixed-line-ending - id: requirements-txt-fixer - id: trailing-whitespace Small common checks Some Python leaning Some pre-commit hook specialization
- 29. pre-commit/pygrep-hooks 22 Small common pygreps - repo: https://github.com/pre-commit/pygrep-hooks rev: "v1.10.0" hooks: - id: rst-backticks - id: rst-directive-colons - id: rst-inline-touching-normal
- 30. CI (GitHub Actions) 23 on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 with: python-version: "3.x" - uses: pre-commit/action@v3.0.1 Great, fast caching, but maintenance only - replaced by pre-commit.ci
- 31. CI (GitHub Actions) 23 on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 with: python-version: "3.x" - uses: pre-commit/action@v3.0.1 Great, fast caching, but maintenance only - replaced by pre-commit.ci on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: pipx run nox -s lint @nox.session def lint(session: nox.Session) -> None: session.install("pre-commit") session.run("pre-commit", "run", "--all-files", "--show-diff-on-failure", *session.posargs)
- 32. Useful GitHub Actions 24 actions/checkout actions/setup-python actions/cache actions/upload-artifact actions/download-artifact ilammy/msvc-dev-cmd jwlawson/actions-setup-cmake excitedleigh/setup-nox pypa/gh-action-pypi-publish pre-commit/action conda-incubator/setup-miniconda peaceiris/actions-gh-pages ruby/setup-miniconda astral-sh/setup-uv Writing your own composite action is really easy!
- 33. Part 1: One tool to rule them all 25 (Again)
- 34. Ruff 26 A new entry in the Python linting/formatting space, amazing adoption in a year(ish) 100x faster than existing Python linters Has support for fi xers! Implements all of (modern) fl ake8’s checks Implements dozens of fl ake8 plugins Fixes many long-standing issues in plugins Over 700 rules (!!!) 0 dependencies Con fi gured with pyproject.toml Has a Black-like formatter too, 30x faster than Black! Only binary platforms (Rust compiled) Doesn’t support user plugins Online version https://play.ru ff .rs 0s 20s 40s 60s Ruff Autoflake Flake8 Pyflakes Pycodestyle Pylint 0.29s 6.18s 12.26s 15.79s 46.92s > 60s
- 35. Ruff 26 A new entry in the Python linting/formatting space, amazing adoption in a year(ish) 100x faster than existing Python linters Has support for fi xers! Implements all of (modern) fl ake8’s checks Implements dozens of fl ake8 plugins Fixes many long-standing issues in plugins Over 700 rules (!!!) 0 dependencies Con fi gured with pyproject.toml Has a Black-like formatter too, 30x faster than Black! Only binary platforms (Rust compiled) Doesn’t support user plugins Online version https://play.ru ff .rs
- 36. Ruff 26 A new entry in the Python linting/formatting space, amazing adoption in a year(ish) 100x faster than existing Python linters Has support for fi xers! Implements all of (modern) fl ake8’s checks Implements dozens of fl ake8 plugins Fixes many long-standing issues in plugins Over 700 rules (!!!) 0 dependencies Con fi gured with pyproject.toml Has a Black-like formatter too, 30x faster than Black! Only binary platforms (Rust compiled) Doesn’t support user plugins Online version https://play.ru ff .rs
- 37. Ruff con fi g example 27 [tool.ruff.lint] extend-select = [ "B", # flake8-bugbear "I", # isort "ARG", # flake8-unused-arguments "C4", # flake8-comprehensions "EM", # flake8-errmsg "ICN", # flake8-import-conventions "PGH", # pygrep-hooks "PIE", # flake8-pie "PL", # pylint "PT", # flake8-pytest-style "RET", # flake8-return "RUF", # Ruff-specific "SIM", # flake8-simplify "T20", # flake8-print "UP", # pyupgrade "YTT", # flake8-2020 ] typing-modules = ["somepackage._compat.typing"] [tool.ruff.lint.per-file-ignores] "tests/**" = ["T20"] "noxfile.py" = ["T20"] - repo: https://github.com/astral-sh/ruff-pre-commit rev: "v0.9.1" hooks: - id: ruff args: ["--fix", "--show-fixes"] - id: ruff-format Flake8 con fi g? Try: pipx run flake8-to-ruff .flake8 https://learn.scienti fi c-python.org/development/guides/style/#ru f
- 38. Ruff-format 28 Black Close to the one true format for Python Almost not con fi gurable (this is a feature) A good standard is better than perfection Designed to reduce merge con fl icts Reading blacked code is fast Write your code to produce nice formatting You can disable line/lines if you have to Workaround for single quotes (use double) Magic trailing comma Ru ff ’s formatter 99.9% compatible with Black A little bit more con fi gurable Fast(er) Already present if using Ru f
- 39. Write for good format 29 raise RuntimeError( "This was not a valid value for some_value: {}".format(repr(some_value)) ) Bad: Ru ff can check for this and rewrite it for you!
- 40. Write for good format 29 raise RuntimeError( "This was not a valid value for some_value: {}".format(repr(some_value)) ) Bad: msg = f"This was not a valid value for some_value: {some_value!r}" raise RuntimeError(msg) Good: Better stacktrace More readable Two lines instead of three Faster (f-string) Ru ff can check for this and rewrite it for you!
- 41. Using code formatters 30 Existing projects Apply all-at-once, not spread out over time Add the format commit to .git-blame-ignore-revs (GitHub now recognizes this fi le, too!)
- 42. Running Ruff on notebooks 31 pre-commit? This is now the default!: types_or: [python, pyi, jupyter] Native support Linter and formatter support notebooks md/rst standalone fi le support planned
- 43. Formatting code snippets 32 Ru ff native support planned! (Already handles docstrings) - repo: https://github.com/adamchainz/blacken-docs rev: "1.19.1" hooks: - id: blacken-docs additional_dependencies: [black==24.*] Blacken docs Adapts black to md/rst fi les
- 44. Ruff linter 33 Groups of rules Most are based on some existing tool / plugin Opt in (recommended) or use ALL --preview enables lots more Fixing code --fix --show-fixes on the command line --unsafe-fixes for even more fi xes Can disable fi xes by code Running Ru ff Doesn’t depend on version of Python! Doesn’t require any environment setup! Easy to run locally as well as in pre-commit Can integrate with VSCode or any LSP editor
- 45. Using code linters 34 Existing projects Feel free to build a long ignore list Work on one or a few at a time You don’t have to have every check
- 46. Default codes 35 F: PyFlakes (default) Unused modules & variables String formatting mistakes No placeholders in f-string Dictionary key repetition Assert a tuple (it’s always true) Various syntax errors Unde fi ned names Rede fi nition of unused var ❤ pytest C90: McCabe Complexity checks E: PyCodeStyle (subset default) Style checks
- 47. Other useful codes 36 B: Bugbear Do not use bare except No mutable argument defaults getattr(x, "const") should be x.const No assert False, use raise AssertionError Pointless comparison ❤ pytest T20: fl ake8-print Avoid leaking debugging print statements D: pydocstyle Documentation requirements PERF: per fl int Detect common expressions with faster idioms SIM: fl ake8-simplify Simpli fi er form for expression C4: fl ake8-comprehensions Comprehension simpli fi cation PTH: fl ake8-use-pathlib Use pathlib instead of os.path And many more!
- 48. Ruff’s own codes 37 NPY: numpy rules Can detect 2.0 upgrade changes RUF codes Unicode checks Unused noqa ( fi xer can remove unused!) Various assorted checks See all the codes at: https://docs.astral.sh/ru ff /rules
- 49. Code I: isort 38 Sort your Python imports Very con fi gurable Reduces merge con fl icts Grouping imports helps readers Can inject future imports args: ["-a", "from __future__ import annotations"] Default groupings Future imports Stdlib imports Third party packages Local imports from __future__ import annotations import dataclasses import graphlib import textwrap from collections.abc import Mapping, Set from typing import Any, TypeVar import markdown_it from .checks import Check from .families import Family, collect_families from .fixtures import pyproject from .ghpath import EmptyTraversable
- 50. Code UP: pyupgrade 39 Update Python syntax Avoid deprecated or obsolete code Fairly cautious Can target a speci fi c Python 3 min (Mostly) not con fi gurable Remove static if sys.version_info blocks Python 2.7 Set literals Dictionary comprehensions Generators in functions Format speci fi er & .format ⚙ Comparison for const literals (3.8 warn) Invalid escapes Python 3 Unicode literals Long literals, octal literals Modern super() New style classes Future import removal yield from Remove six compatibility code io.open -> open Remove error aliases Python 3.x f-strings (partial) (3.6) ⚙ NamedTuple/TypedDict (3.6) subprocess.run updates (3.7) lru_cache parens (3.8) lru_cache(None) -> cache (3.9) Typing & annotation rewrites (various) abspath(__file__) removal (3.9) Before After for a, b in c: yield (a, b) yield from c "{foo} {bar}".format(foo=foo, bar=bar) f"{foo} {bar}" dict([(a, b) for a, b in y]) {a: b for a, b in y}
- 51. pyupgrade limits 40 PyUpgrade does not over modernize isinstance(x, (int, str)) -> isinstance(x, int | str) (3.10) No match statement conversions (3.10) Nothing converts to using walrus := (3.8) (probably a good thing!) Except for a bit of typing Optional[int] -> int | None (I like this one now, though) ❌
- 52. Part 2: Other tools 41
- 53. Notebook cleaner 42 hooks: - repo: https://github.com/kynan/nbstripout rev: "0.8.1" hooks: - id: nbstripout Remove outputs from notebooks Best if not stored in VCS You can render outputs in JupyterBook, etc. Use Binder or JupyterLite
- 54. hooks: - repo: https://gitlab.com/pycqa/flake8 rev: "7.1.1" hooks: - id: flake8 Flake8 43 Fast simple extendable linter Very con fi gurable: setup.cfg or . fl ake8 Doesn’t support pyproject.toml Many plugins, local plugins easy No auto- fi xers like rubocop (Ruby) Still great for custom checks # .flake8 [flake8] max-complexity = 12 extend-ignore = E203, E501, E722, B950 extend-select = B9
- 55. Custom local fl ake8 plugin 44 import ast import sys from typing import NamedTuple, Iterator class Flake8ASTErrorInfo(NamedTuple): line_number: int offset: int msg: str cls: type # unused
- 56. Custom local fl ake8 plugin 45 class Visitor(ast.NodeVisitor): msg = "AK101 exception must be wrapped in ak._v2._util.*error" def __init__(self) -> None: self.errors: list[Flake8ASTErrorInfo] = [] def visit_Raise(self, node: ast.Node) -> None: if isinstance(node.exc, ast.Call): if isinstance(node.exc.func, ast.Attribute): if node.exc.func.attr in {"error", "indexerror"}: return if node.exc.func.id in {"ImportError"}: return self.errors.append( Flake8ASTErrorInfo(node.lineno, node.col_offset, self.msg, type(self)) )
- 57. Custom local fl ake8 plugin 46 class AwkwardASTPlugin: name = "flake8_awkward" version = "0.0.0" def __init__(self, tree: ast.AST) -> None: self._tree = tree def run(self) -> Iterator[Flake8ASTErrorInfo]: visitor = Visitor() visitor.visit(self._tree) yield from visitor.errors
- 58. Custom local fl ake8 plugin 47 [flake8:local-plugins] extension = AK1 = flake8_awkward:AwkwardASTPlugin paths = ./dev/ def main(path: str) -> None: with open(path) as f: code = f.read() node = ast.parse(code) plugin = AwkwardASTPlugin(node) for err in plugin.run(): print(f"{path}:{err.line_number}:{err.offset} {err.msg}") if __name__ == "__main__": for item in sys.argv[1:]: main(item)
- 59. PyLint 48 PyLint recommends having your project installed, so it is not a good pre-commit hook (though you can do it) It’s also a bit slow, so a good candidate for nox @nox.session def pylint(session: nox.Session) -> None: session.install("-e.") session.install("pylint") session.run("pylint", "src", *session.posargs) # pyproject.toml [tool.pylint] master.py-version = "3.9" master.jobs = "0" reports.output-format = "colorized" similarities.ignore-imports = "yes" messages_control.enable = ["useless-suppression"] messages_control.disable = [ "design", "fixme", "line-too-long", "wrong-import-position", ] Code linter Can be very opinionated Signal to noise ratio poor You will need to disable checks - that’s okay! A bit more advanced / less static than fl ake8 But can catch hard to fi nd bugs! For an example of lots of suppressions: https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/pyproject.toml Some parts available in Ruff
- 60. Example PyLint rules 49 Duplicate code Finds large repeated code patterns Attribute de fi ned outside init Only __init__ should de fi ne attributes No self use Can be @classmethod or @staticmethod Unnecessary code Lambdas, comprehensions, etc. Unreachable code Finds things that can’t be reached Consider using in x in {stuff} vs chaining or’s Arguments di ff er Subclass should have matching arguments Consider iterating dictionary Better use of dictionary iteration Consider merging isinstance You can use a tuple in isinstance Useless else on loop They are bad enough when useful :) Consider using enumerate Avoid temp variables, idiomatic Global variable not assigned You should only declare global to assign
- 61. Controversial PyLint rules 50 No else after control- fl ow Guard-style only Can simply complex control fl ow Removes useless indentation if x: return x else: return None # Should be: if x: return x return None # Or: return x if x else None # Or: return x or None Design Too many various things Too few methods Can just silence “design”
- 62. Controversial PyLint rules 50 No else after control- fl ow Guard-style only Can simply complex control fl ow Removes useless indentation if x: return x else: return None # Should be: if x: return x return None # Or: return x if x else None # Or: return x or None Design Too many various things Too few methods Can just silence “design” (I’m on the in-favor side)
- 63. Static type checking: MyPy 51 hooks: - repo: https://gitlab.com/pre-commit/mirrors-mypy rev: "v1.14.1" hooks: - id: mypy files: src Like a linter on steroids Uses Python typing Enforces correct type annotations Designed to be iteratively enabled Should be in a controlled environment (pre-commit or nox) Always specify args (bad hook defaults) Almost always need additional_dependencies Con fi gure in pyproject.toml Pros Can catch many things tests normally catch, without writing tests Therefore it can catch things not covered by tests (yet, hopefully) Code is more readable with types Sort of works without types initially Cons Lots of work to add all types Typing can be tricky in Python Active development area for Python
- 64. Con fi guring MyPy 52 [tool.mypy] files = "src" python_version = "3.9" warn_unused_configs = true strict = true [[tool.mypy.overrides]] module = [ "numpy.*" ] ignore_missing_imports = true Start small Start without strictness Add a check at a time Extra libraries Try adding them to your environment You can ignore untyped or slow libraries You can provide stubs for untyped libraries if you want Tests? Adding pytest is rather slow I prefer to avoid tests, or keep them mostly untyped (unless the package is very important)
- 65. Typing tricks 53 Protocols Better than ABCs, great for duck typing @typing.runtime_checkable class Duck(Protocol): def quack() -> str: ... def f(x: Duck) -> str: return x.quack() class MyDuck: def quack() -> str: return "quack" if typing.TYPE_CHECKING: _: Duck = typing.cast(MyDuck, None) Type Narrowing Integral to how mypy works x: A | B if isinstance(x, A): reveal_type(x) # A else: reveal_type(x) # B Make a typed package Must include py.typed marker fi le Always use sys.version_info Better for readers than try/except, and static Also sys.platform instead of os.name
- 66. Future annotations 54 Classic code (3.5+) from typing import Union, List def f(x: int) -> List[int]: return list(range(x)) def g(x: Union[str, int]) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) Modern code (3.7+) from __future__ import annotations def f(x: int) -> list[int]: return list(range(x)) def g(x: str | int) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) Ultramodern code (3.10+) def f(x: int) -> list[int]: return list(range(x)) def g(x: str | int) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) With the future import, you get all the bene fi ts of future code in 3.7+ annotations Typing is already extra code, simpler is better
- 67. Part 3: Other languages
- 68. Clang-format 56 hooks: - repo: https://github.com/pre-commit/mirrors-clang-format rev: "v19.1.6" hooks: - id: clang-format types_or: [c++, c, cuda] C++ and more code formatter Very con fi gurable: .clang-format fi le Opinion: stay close to llvm style PyPI clang-format wheels, under 2MB No more issues with mismatched LLVM!
- 69. CMake-format 57 hooks: - repo: https://github.com/cheshirekow/cmake-format-precommit rev: "v0.6.13" hooks: - id: cmake-format additional_dependencies: [pyyaml] CMake code formatter Very con fi gurable: .cmake-format.yaml fi le Anything that helps with CMake!
- 70. Markdown & YAML with Prettier 58 hooks: - repo: https://github.com/rbubley/mirrors-prettier rev: "v3.4.2" hooks: - id: prettier types_or: [yaml, markdown, html, css, scss, javascript, json] args: [--prose-wrap=always] exclude: "^tests" JavaScript Linter Lots of formats supported A few customization points
- 71. ShellCheck 59 hooks: - repo: https://github.com/shellcheck-py/shellcheck-py rev: "v0.10.0.1" hooks: - id: shellcheck Linter for bash scripts Can locally disable Prioritizes correctness over terseness
- 72. CodeSpell 60 hooks: - repo: https://github.com/codespell-project/codespell rev: "v2.3.0" hooks: - id: shellcheck args: ["-L", "sur,nd"] Find common misspellings Inverted spell checker - looks for misspellings Can con fi gure or provide wordlist Actually can catch bugs! Pass -w to fi x, too
- 73. Schemas 61 hooks: - repo: https://github.com/python-jsonschema/check-jsonschema rev: "0.31.0" hooks: - id: check-readthedocs - id: check-github-workflows Can validate common fi les Can get more from SchemaStore (Over 900 fi le types supported) Can write custom schemas too hooks: - repo: https://github.com/abravalheri/validate-pyproject rev: "0.23" hooks: - id: validate-pyproject # OR - repo: https://github.com/henryiii/validate-pyproject-schema-store rev: "2025.01.10" hooks: - id: validate-pyproject Specialized for pyproject.toml Supports plugins SchemaStore plugin available Live demo: https://scienti fi c-python.github.io/repo-review/
- 74. See how your repo measures! 62 https://learn.scienti fi c-python.org/development/guides/repo-review/
- 75. Bonus part: pytest tips 63
- 76. pytest tips 64 Spend time learning pytest Full of amazing things that really make testing fun! Tests are code too Or for C++: Catch2 or doctest, etc. Also maybe learn Hypothesis for pytest [tool.pytest.ini_options] minversion = "6.0" addopts = [ "-ra", "--showlocals", "--strict-markers", "--strict-config", ] xfail_strict = true filterwarnings = [ "error", ] log_cli_level = "info" testpaths = [ "tests", ] Use pytest.approx Even works on numpy arrays Remember to test for failures If you expect a failure, test it! Test your installed package That’s how users will get it, not from a directory
- 77. pytest tips 64 Spend time learning pytest Full of amazing things that really make testing fun! Tests are code too Or for C++: Catch2 or doctest, etc. Also maybe learn Hypothesis for pytest [tool.pytest.ini_options] minversion = "6.0" addopts = [ "-ra", "--showlocals", "--strict-markers", "--strict-config", ] xfail_strict = true filterwarnings = [ "error", ] log_cli_level = "info" testpaths = [ "tests", ] Don’t let warnings slip by! Makes logging more useful Strictness is good Useful summary Print out locals on errors Use pytest.approx Even works on numpy arrays Remember to test for failures If you expect a failure, test it! Test your installed package That’s how users will get it, not from a directory
- 78. pytest Tricks 65 Mock and Monkeypatch This is how you make tricky tests “unit” tests Fixtures This keeps tests simple and scalable @pytest.fixture(params=["Linux", "Darwin", "Windows"], autouse=True) def platform_system(request, monkeypatch): monkeypatch.setattr(platform, "system", lambda _: request.param) Parametrize Directly or in a fi xture for reuse Use conftest.py Fixtures available in same and nested directories
- 79. Running pytest 66 Show locals on failure --showlocals/-l Jump into a debugger on failure --pdb Start with last failing test --lf Jump into a debugger immediately --trace or use breakpoint() Run matching tests -k <expression> Run speci fi c test filename.py::testname Run speci fi c marker -m <marker> Control traceback style --tb=<style>
- 80. In conclusion 67 Code quality tools can help a lot with Readability Reducing bugs Boosting developer productivity Consistency Refactoring Teaching others good practice too Hopefully we have had some helpful discussions! It’s okay to disable a check Try to understand why it’s there Remember there are multiple concerns involved in decisions