Developing ado¶
Project Setup¶
To start developing ado, you need to set up a Python environment. We use uv
for project and dependency management.
Installing uv¶
To install uv
, refer to the official installation documentation and choose your preferred method.
Creating a development virtual environment¶
Create a development virtual environment by executing the following commands in the top-level of the ado
repository:
Caution
If you create a development in a different location you must direct uv sync
explicitly to use it with --active
If you do not, it will default to using .venv
in the project top-level directory. See the using a custom location for the venv section for instructions on how to do this.
uv sync
source .venv/bin/activate
Note
This installs ado
in editable mode.
Note
In line with uv's defaults, the uv sync
command creates a .venv
in the top-level of the project's repository. Note that environments created by uv sync
are intended only to be used when developing a specific project and should not be shared across projects.
Caution
uv sync
ensures a reproducible development environment is created by using a lock-file, uv.lock
. Only packages in the lockfile are installed, and other packages found in the virtual environment will be deleted. See Making changes to dependencies for how to add packages to the lockfile.
Using a custom location for the venv¶
If you want to create your development virtual environment at an alternate location, $LOCATION, then run:
uv venv $LOCATION
source $LOCATION/bin/activate
uv sync --active
Code style¶
Note
See the Automating checks with pre-commit section to automate this.
This repository follows the black
style for formatting.
You can format your code by:
- Manually running
black tests/ orchestrator/ plugins/
- Setting up PyCharm to use the
black
integration: https://www.jetbrains.com/help/pycharm/reformat-and-rearrange-code.html#format-python-code-with-black - Using the "Black formatter" extension for VSCode and setting it as the default formatter: https://code.visualstudio.com/docs/python/formatting#_set-a-default-formatter
Linting code with ruff¶
Note
See the Automating checks with pre-commit section to automate this.
This repository uses ruff
to enforce linting rules. Install it using one of the methods described in the official ruff
documentation. To run linting checks, execute:
ruff check --exclude website
Linting markdown with markdownlint-cli2¶
Note
See the Automating checks with pre-commit section to automate this.
This repository uses markdownlint-cli2
to enforce linting rules on markdown files. Install it using one of the methods described in the official documentation. To run linting checks, execute:
markdownlint-cli2 "**/*.md" "#.venv" --fix
Prettier for lines too long¶
Warning
Prettier might undo some changes that markdownlint-cli2
has done. A common error is adding a line after the markdownlint-disable-next-line
comments
Line-too-long errors do not get automatically fixed by markdownlint-cli2
. We recommend using prettier
to autoformat markdown in that case. The official website provides instructions to:
Prettier can be run as a CLI tool with:
prettier -w "**/*.md"
Secret scanning¶
Note
See the Automating checks with pre-commit section to automate this.
This repository uses IBM's detect-secrets to scan for secrets before the code is pushed to GitHub. Follow installation instructions in their repository: https://github.com/ibm/detect-secrets?tab=readme-ov-file#example-usage
To update the secrets database manually, run:
detect-secrets scan --update .secrets.baseline
To audit detected secrets, use:
detect-secrets audit .secrets.baseline
If the pre-commit hook raises an error but the audit command succeeds with just Nothing to audit!
then run detect-secrets scan --update .secrets.baseline
to perform a full scan and then repeat the audit
command.
Commit style¶
We require commit messages to use the conventional commit style.
Conventional Commit messages follow the pattern (NOTE: the scope is optional):
type(scope): subject
extended body
Where type is one of: build, chore, ci, docs, feat, fix, perf, refactor, revert, style, test.
Copyright and license headers¶
We require copyright and SPDX license headers to be added to the source code. This can be automated by using Hashicorp's Copywrite tool: https://github.com/hashicorp/copywrite
Once installed, run
copywrite headers
Website link checking¶
To make it less likely for us to push commits with broken links, we use linkcheck to check if our website contains broken links. You can install it with your preferred method from the ones provided in the official documentation.
On one terminal, navigate to the website directory and start serving the website using mkdocs:
cd website && mkdocs serve --clean
On a different terminal, run the linkchecker:
linkcheck http://127.0.0.1:8000/ado/ \
--skip-file=.linkchecker_skip
Automating checks with pre-commit¶
To automate the checks for code style, linting, and security, you can utilize the provided pre-commit hooks.
Installing the hooks¶
Important
Before installing the hooks, make sure you have the following prerequisites:
- A developer virtual environment created with uv
- A recent version of NodeJS
- On MacOS we suggest installing it via brew for ease of use
pre-commit install
This command will configure pre-commit to run automatically before each commit, highlighting any issues and preventing the commit if problems are found.
Handling pre-commit failures¶
- Black code formatting failures: try committing again,
black
might have reformatted your code in-place.
- If black fails to format your code, your files have syntax errors. Try manually running black.
- Ruff linter failures: run
ruff
as specified in Linting code with ruff and fix the code that is causing the failures.
- In case of false positives, you might need to add
#noqa
annotations. - If your local ruff installation does not detect any failure you may be using an old version that needs updating.
- Detect secrets failures: include
.secrets.baseline
in your commit, it was updated by the pre-commit hook.
- If secrets are detected, audit them as specified in Secret scanning.
- Commit style failures: change your commit message to match conventional commits. See Commit style for more in-depth information.
- Misspellings detected by codespell: fix the misspellings reported or add an inline ignore comment.
- uv export failures: commit the updated
requirements.txt
file. It has been updated following changes to the lock file. - Markdown linter failures:
markdownlint-cli2
usually fixes most issues automatically. If you review its error message and still don’t see a clear explanation or solution, try recommitting your changes and let the tool re-run.
Making changes to dependencies¶
As mentioned in Project Setup, we use uv
to manage dependencies. This means that all changes to dependencies must be done via uv
, and not by manually editing pyproject.toml
.
The relevant documentation on uv
's website is available here , but at a glance:
Adding base dependencies¶
If you are adding (or updating) base dependencies for ado
, you should use the uv add
command:
Note
You can optionally add specific version selectors. By default, uv
will add >=CURRENT_VERSION
.
uv add pydantic
Adding optional dependencies¶
Dependencies may be optional, making them available only when using extras, such as ado-core[my-extra]
. To add these kind of dependencies, use the uv add --optional
command:
uv add --optional validation pydantic
Adding dependency groups¶
Sometimes we might want to include dependencies that have a specific purpose, like testing the code, linting it, or building the documentation. This is a perfect use case for dependency groups, sets of dependencies that do not get published to indices like PyPI and are not installed with ado. A noteworthy dependency group is the dev
group, which uv
installs by default when syncing dependencies.
Users are highly encouraged to read the documentation available both on uv's and Python's website:
- https://docs.astral.sh/uv/concepts/projects/dependencies/#development-dependencies
- https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups
- https://packaging.python.org/en/latest/specifications/dependency-groups
With uv
you can add dependencies to groups using uv add --group NAME
:
Note
For the dev
group there is the shorthand --dev
that replaces --group dev
.
uv add --group dev pytest