Conex, establish trust in community repositories
Published: 2017-02-16 (last updated: 2017-02-20)
Less than two years after the initial proposal, we're happy to present conex 0.9.2. Pleas note that this is still work in progress, to be deployed with opam 2.0 and the opam repository.
Conex is a library to verify and attest release integrity and authenticity of a community repository through the use of cryptographic signatures.
Packages are collected in a community repository to provide an index and allowing cross-references. Authors submit their packages to the repository. which is curated by a team of janitors. Information about a package stored in a repository includes: license, author, releases, their dependencies, build instructions, url, tarball checksum. When someone publishes a new package, the janitors integrate it into the repository, if it compiles and passes some validity checks. For example, its name must not be misleading, nor may it be too general.
Janitors keep an eye on the repository and fix emergent failures. A new compiler release, or a release of a package on which other packages depend, might break the compilation of a package. Janitors usually fix these problems by adding a patch to the build script, or introducing a version constraint in the repository.
Conex ensures that every release of each package has been approved by its author or a quorum of janitors. A conex-aware client initially verifies the repository using janitor key fingerprints as anchor. Afterwards, the on-disk repository is trusted, and every update is verified (as a patch) individually. This incremental verification is accomplished by ensuring all resources that the patch modifies result in a valid repository with sufficient approvals. Additionally, monotonicity is preserved by embedding counters in each resource, and enforcing a counter increment after modification. This mechanism avoids rollback attacks, when an attacker presents you an old version of the repository.
A timestamping service (NYI) will periodically approve a global view of the verified repository, together with a timestamp. This is then used by the client to prevent mix-and-match attacks, where an attacker mixes some old packages and some new ones. Also, the client is able to detect freeze attacks, since at least every day there should be a new signature done by the timestamping service.
The trust is rooted in digital signatures by package authors. The server which hosts the repository does not need to be trusted. Neither does the host serving release tarballs.
If a single janitor would be powerful enough to approve a key for any author, compromising one janitor would be sufficient to enroll any new identities, modify dependencies, build scripts, etc. In conex, a quorum of janitors (let's say 3) have to approve such changes. This is different from current workflows, where a single janitor with access to the repository can merge fixes.
Conex adds metadata, in form of resources, to the repository to ensure integrity and authenticity. There are different kinds of resources:
- Authors, consisting of a unique identifier, public key(s), accounts.
- Teams, sharing the same namespace as authors, containing a set of members.
- Authorisation, one for each package, describing which identities are authorised for the package.
- Package index, for each package, listing all releases.
- Release, for each release, listing checksums of all data files.
Modifications to identities and authorisations need to be approved by a quorum of janitors, package index and release files can be modified either by an authorised id or by a quorum of janitors.
We presented an abstract at OCaml 2016 about an earlier design.
Another article on an earlier design (from 2015) is also available.
The TUF spec has a good overview of attacks and threat model, both of which are shared by conex.
- See issue 7 for a laundry list
- Timestamping service
- Key revocation and rollover
- Tool to approve a PR (for janitors)
- Camelus like opam-repository check bot
- Integration into release management systems
At the moment, our opam repository does not include any metadata needed for signing. We're in a bootstrap phase: we need you to generate a keypair, claim your packages, and approve your releases.
We cannot verify the main opam repository yet, but opam2 has support for a
repository validation command,
builtin, which should then call out to
conex_verify (there is a
flag for the impatient). There is also an example repository which uses the opam validation command.
To reduce the manual work, we analysed 7000 PRs of the opam repository within
the last 4.5 years (more details here.
This resulted in an educated guess who are the people
modifying each package, which we use as a basis whom to authorise for
which packages. Please check with
conex_author status below whether your team
membership and authorised packages were inferred correctly.
Each individual author - you - need to generate their private key, submit their public key and starts approving releases (and old ones after careful checking that the build script, patches, and tarball checksum are valid). Each resource can be approved in multiple versions at the same time.
TODO: remove clone once PR 8494 is merged.
$ git clone -b auth https://github.com/hannesm/opam-repository.git repo $ opam install conex $ cd repo
This will install conex, namely command line utilities,
conex_verify_openssl. All files read and written by conex are in the usual
opam file format. This means can always manually modify them (but be careful,
modifications need to increment counters, add checksums, and be signed). Conex
does not deal with git, you have to manually
git add files and open pull
For the opam repository, we will use GitHub ids as conex ids. Thus, your conex id and your GitHub id should match up.
repo$ conex_author init --repo ~/repo --id hannesm Created keypair hannesm. Join teams, claim your packages, sign your approved resources and open a PR :)
This attempts to parse
~/repo/id/hannesm, errors if it is a team or an author
with a publickey. Otherwise it generates a keypair, writes the private part as
home.hannes.repo.hannesm.private (the absolute path separated by dots,
followed by your id, and
private - if you move your repository, rename your
private key) into
~/.conex/, the checksums of the public part and your
conex_author help init for more
options (esp. additional verbosity
-v can be helpful).
repo$ git status -s M id/hannesm repo$ git diff //abbreviated output - ["counter" 0x0] + ["counter" 0x1] - ["resources" ] + [ + "resources" + [ + [ + ["typ" "key"] + ["name" "hannesm"] + ["index" 0x1] + ["digest" ["SHA256" "ht9ztjjDwWwD/id6LSVi7nKqVyCHQuQu9ORpr8Zo2aY="]] + ] + [ + ["typ" "account"] + ["name" "hannesm"] + ["index" 0x2] + ["digest" ["SHA256" "aCsktJ5M9PI6T+m1NIQtuIFYILFkqoHKwBxwvuzpuzg="]] + ] + +keys: [ + [ + [ + "RSA" + """ +-----BEGIN PUBLIC KEY----- +MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAyUhArwt4XcxLanARyH9S ... +9KQdg6QnLsQh/j74QKLOZacCAwEAAQ== +-----END PUBLIC KEY-----""" + 0x58A3419F + ] + [ + 0x58A79A1D + "RSA-PSS-SHA256" + "HqqicsDx4hG9pFM5E7" + ] + ] +]
If you have a single identity and contribute to a single signed opam repository,
you don't need to specify
--repo from now on.
status subcommand presents an author-specific view on the repository. It
lists the own public keys, team membership, queued resources, and authorised
The opam repository is in a transitionary state, we explicitly pass
0, which means that every checksum is valid (approved by a quorum of 0
repo$ conex_author status --quorum 0 arp author hannesm #1 (created 0) verified 3 resources, 0 queued 4096 bit RSA key created 1487094175 approved, SHA256: ht9ztjjDwWwD/id6LSVi7nKqVyCHQuQu9ORpr8Zo2aY= account GitHub hannesm approved account email email@example.com approved package arp authorisation approved conex_author: [ERROR] package index arp was not found in repository
This shows your key material and accounts, team membership and packages you are authorised to modify (inferred as described here.
--noteam argument limits the package list to only these you are personally
authorised for. The
--id argument presents you with a view of another author,
or from a team perspective. The positional argument is a prefix matching on
package names (leave empty for all).
Each resource needs to be approved individually. Each author has a local queue
for to-be-signed resources, which is extended with
team (all have a
--dry-run flag). The queue can be
conex_author reset. Below shown is
conex_author sign, which
let's you interactively approve queued resources and cryptopgraphically signs
your approved resources afterwards.
The output of
conex_author status listed an authorisation for
which I don't feel responsible for. Let's drop my privileges:
repo$ conex_author authorisation conf-gsl --remove -m hannesm modified authorisation and added resource to your queue.
I checked my arp release careful (checksums of tarballs are correct, opam files do not execute arbitrary shell code, etc.), and approve this package and its single release:
repo$ conex_author release arp conex_author.native: [WARNING] package index arp was not found in repository conex_author.native: [WARNING] release arp.0.1.1 was not found in repository wrote release and added resources to your queue.
Once finished with joining and leaving teams (using the
claiming packages (using the
authorisation subcommand), and approve releases
release subcommand), you have to cryprographically sign your queued
repo$ conex_author sign release arp.0.1.1 #1 (created 1487269425) [descr: SHA256: aCsNvcj3cBKO0GESWG4r3AzoUEnI0pHGSyEDYNPouoE=; opam: SHA256: nqy6lD1UP+kXj3+oPXLt2VMUIENEuHMVlVaG2V4z3p0=; url: SHA256: FaUPievda6cEMjNkWdi0kGVK7t6EpWGfQ4q2NTSTcy0=] approved (yes/No)? package arp #1 (created 1487269425) [arp.0.1.1] approved (yes/No)?y authorisation conf-gsl #1 (created 0) empty approved (yes/No)?y wrote hannesm to disk repo$ conex_author status --quorum 0 arp author hannesm #1 (created 0) verified 7 resources, 0 queued 4096 bit RSA key created 1487094175 approved, SHA256: ht9ztjjDwWwD/id6LSVi7nKqVyCHQuQu9ORpr8Zo2aY= account GitHub hannesm approved account email firstname.lastname@example.org approved package arp authorisation approved package index approved release arp.0.1.1: approved
If you now modify anything in
packages/arp (add subdirectories, modify opam,
etc.), this will not be automatically approved (see below for how to do this).
You manually need to
git add some created files.
repo$ git status -s M id/hannesm M packages/conf-gsl/authorisation ?? packages/arp/arp.0.1.1/release ?? packages/arp/package repo$ git add packages/arp/arp.0.1.1/release packages/arp/package repo$ git commit -m "hannesm key enrollment and some fixes" id packages
Now push this to your fork, and open a PR on opam-repository!
Editing a package
If you need to modify a released package, you modify the opam file (as before,
e.g. introducing a conflict with a dependency), and then approve the
modifications. After your local modifications,
conex_author status will
repo$ conex_author status arp --quorum 0 package arp authorisation approved package index approved release arp.0.1.1: checksums for arp.0.1.1 differ, missing on disk: empty, missing in checksums file: empty, checksums differ: [have opam: SHA256: QSGUU9HdPOrwoRs6XJka4cZpd8h+8NN1Auu5IMN8ew4= want opam: SHA256: nqy6lD1UP+kXj3+oPXLt2VMUIENEuHMVlVaG2V4z3p0=] repo$ conex_author release arp.0.1.1 released and added resources to your resource list. repo$ conex_author sign release arp.0.1.1 #1 (created 1487269943) [descr: SHA256: aCsNvcj3cBKO0GESWG4r3AzoUEnI0pHGSyEDYNPouoE=; opam: SHA256: QSGUU9HdPOrwoRs6XJka4cZpd8h+8NN1Auu5IMN8ew4=; url: SHA256: FaUPievda6cEMjNkWdi0kGVK7t6EpWGfQ4q2NTSTcy0=] approved (yes/No)? y wrote hannesm to disk
release subcommand recomputed the checksums, incremented the counter, and
added it to your queue. The
sign command signed the approved resource.
repo$ git status -s M id/hannesm M packages/arp/arp.0.1.1/opam M packages/arp/arp.0.1.1/package repo$ git commit -m "fixed broken arp package" id packages
Janitors need to approve teams, keys, accounts, and authorisations.
To approve resources which are already in the repository on disk,
key subcommand queues approval of keys and accounts of the provided author:
repo$ conex_author key avsm added keys and accounts to your resource list.
authorisation subcommand, and
team subcommand behave similarly for
authorisations and teams.
Bulk operations are supported as well:
conex_author authorisation all
This will approve all authorisations of the repository which are not yet
approved by you. Similar for the
team subcommands, which also
Don't forget to
conex_author sign afterwards (or
yes | conex_author sign).
The two command line utlities,
conex_verify_nocrypto contain the same logic and same command line arguments.
For bootstrapping purposes (
nocrypto is an opam package with dependencies),
conex_verify_openssl relies on the openssl command line tool (version 1.0.0
and above) for digest computation and verification of the RSA-PSS signature.
The goal is to use the opam2 provided hooks, but before we have signatures we cannot enable them.
See the example repository for initial verification experiments, and opam2 integration.