-
Notifications
You must be signed in to change notification settings - Fork 191
feat: tool/copyright #4983
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
feat: tool/copyright #4983
Changes from all commits
9e49976
7a8af2b
47aa0b9
b3dec32
10d0078
f076aa1
5f96a8b
c2b3059
3839800
5326173
7768c8f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| load("@rules_go//go:def.bzl", "go_binary", "go_library") | ||
| load("//tools:go.bzl", "go_test") | ||
|
|
||
| go_library( | ||
| name = "go_default_library", | ||
| srcs = [ | ||
| "affiliation_history.go", | ||
| "config.go", | ||
| "doc.go", | ||
| "header.go", | ||
| "history.go", | ||
| "main.go", | ||
| ], | ||
| embedsrcs = ["affiliations.json"], | ||
| importpath = "github.com/scionproto/scion/tools/copyright", | ||
| visibility = ["//visibility:private"], | ||
| ) | ||
|
|
||
| go_binary( | ||
| name = "copyright", | ||
| embed = [":go_default_library"], | ||
| visibility = ["//visibility:public"], | ||
| ) | ||
|
|
||
| go_test( | ||
| name = "go_default_test", | ||
| srcs = [ | ||
| "affiliation_history_test.go", | ||
| "config_test.go", | ||
| "header_test.go", | ||
| "history_test.go", | ||
| ], | ||
| embed = [":go_default_library"], | ||
| deps = [ | ||
| "@com_github_stretchr_testify//assert:go_default_library", | ||
| "@com_github_stretchr_testify//require:go_default_library", | ||
| ], | ||
| ) |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,167 @@ | ||||||
| # copyright | ||||||
|
|
||||||
| See [doc.go](doc.go), or `go doc ./tools/copyright`. | ||||||
|
|
||||||
| By default it reports how many files are out of date, and which identities it could not | ||||||
| attribute. Under `-v`, every line it adds or moves cites the contribution behind it: | ||||||
|
|
||||||
| ```txt | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This example looks very much like it is adding lines with attribution date and commit ID. Is it not? Maybe explain what this example shows: to me this looks like the output of |
||||||
| private/storage/beacon/sqlite/db.go | ||||||
| - // Copyright 2019 Anapaya Systems | ||||||
| + // Copyright 2025 Anapaya Systems (roos@anapaya.net, 2025-09-30 38e79c43f) | ||||||
| + // Copyright 2025 SCION Association (roman.scharkov@gmail.com, 2025-02-12 33de3e514) | ||||||
| ``` | ||||||
|
|
||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe explain what happens now. Is the output above good or bad? |
||||||
| ## What it will not do | ||||||
|
|
||||||
| It only adds claims and moves years forward, never drops one or moves a year back: | ||||||
| git history is not the only evidence of authorship. | ||||||
| `-verify` is the only way to ask it to remove claims. | ||||||
|
|
||||||
| Files it declines to touch, all listed under `-v`: | ||||||
|
|
||||||
| - Generated files. | ||||||
| - Third-party notices: a header claiming copyright in any form other than | ||||||
| `// Copyright <year> <configured organization>`. `// MIT License`, | ||||||
| `// Copyright (c) 2016 Max Mustermann` and `// Copyright 2020 Some Other Labs, Inc.` | ||||||
| (two spaces) all occur here. | ||||||
| - A well-formed line held by an undeclared organization, | ||||||
| such as `// Copyright 2013 The Prometheus Authors`. | ||||||
| - Files with no header. Claims go above an Apache license block, | ||||||
| and a missing license is not invented; `goheader` flags those. | ||||||
| - Files whose header does not open with the copyright line, an SPDX tag above it included. | ||||||
| The parser reads claims from the first line of the leading comment block, | ||||||
| so a tag belongs below the license block, as in `private/underlay/ebpf`. | ||||||
|
|
||||||
| An identity with no known affiliation claims nothing, and is reported. | ||||||
| Leaving an identity out is safe. Guessing its affiliation is not. | ||||||
|
|
||||||
| Uncommitted work is attributed to `git config user.email` and the current year; | ||||||
| having no date of its own, the affiliation is read at the end of that year. | ||||||
| Without a `user.email` the tool stops. `-committed-only` needs no identity. | ||||||
|
|
||||||
| ## affiliations.json | ||||||
|
|
||||||
| Resolution order for an (email, date) pair: | ||||||
|
|
||||||
| 1. an entry in `contributors` whose `emails` contains the address | ||||||
| 2. the domain of the address in `domains` | ||||||
| 3. otherwise nothing is claimed | ||||||
|
|
||||||
| ```json | ||||||
| { | ||||||
| "name": "Max Mustermann", | ||||||
| "emails": ["max.m@example.com", "max@another.org"], | ||||||
| "affiliation": "Muster Org" | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| One organization per person, undated, so it answers for every day that address | ||||||
| contributed. It lists the people contributing now: an entry whose addresses have | ||||||
| not appeared for a year is dropped. Earlier spans live in the file below, which | ||||||
| is neither embedded nor read by default — the repository does not publish when | ||||||
| someone worked where. | ||||||
|
|
||||||
| The entries have no dates, so a plain run must skip the old commits. `-base` | ||||||
| draws the line between old and new: a commit that `-base` already contains is old, | ||||||
| and the tool reads only the commits that it does not. | ||||||
|
|
||||||
| ```sh | ||||||
| go run ./tools/copyright # default -base origin/master: this branch only | ||||||
| go run ./tools/copyright -base upstream/master # on a fork, name the real upstream | ||||||
| go run ./tools/copyright -base '' # read every commit, with today's entries | ||||||
| ``` | ||||||
|
|
||||||
| The commits `-base` reaches are done. Their claims are already in the headers. | ||||||
| Reading them again would use today's entries, and give that old work to the | ||||||
| organization the author works for now. | ||||||
|
|
||||||
| If `-base` doesn't exist, the tool stops with an error. | ||||||
| It doesn't quietly read the whole history instead. | ||||||
|
|
||||||
| `ignoreEmails` drops bots. There is deliberately no list of excluded paths: | ||||||
| which files are ours to edit is read from each file's header. | ||||||
|
|
||||||
| Adding an entry can only add claims, so re-running after an edit is safe. | ||||||
| `go test ./tools/copyright` validates the embedded configuration. | ||||||
|
|
||||||
| ## affiliation-history.json | ||||||
|
|
||||||
| People change employer while keeping the same address, and the snapshot has | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don´t quite understand this. What is the snapshot and how can it drop things? It is also not clear to me why we need this history? |
||||||
| dropped others outright. `-history` points at a file of dated spans that | ||||||
| replaces the contributors of affiliations.json, so it carries its own addresses: | ||||||
|
|
||||||
| ```json | ||||||
| [ | ||||||
| { | ||||||
| "name": "Some Contributor", | ||||||
| "emails": ["some@example.com", "contributor@scion.org"], | ||||||
| "affiliations": [ | ||||||
| {"org": "ETH Zurich", "from": "2018-09-01", "until": "2022-11-30"}, | ||||||
| {"org": "SCION Association", "from": "2022-12-01"} | ||||||
| ] | ||||||
| } | ||||||
| ] | ||||||
| ``` | ||||||
|
|
||||||
| ```sh | ||||||
| go run ./tools/copyright -w -history tools/copyright/affiliation-history.json | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is the hostory private? I think, git history should contain all that information publicly?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I initially raised a concern to Roman about this. Then he mentioned that this is anyway public in Git history. So if that is the case, we can keep this file public too
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So if it is in the git history, why store it in a file? |
||||||
| ``` | ||||||
|
|
||||||
| `.gitignore` reserves that path, but the file may live anywhere. Through `make`, | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, maybe the path shouldn't be hardcoded in .gitignore if it can live anywhere? Usually, I'd say, ignored file should live in the project root or a dedicated subfolder, not somewhere in the code? |
||||||
| give an absolute path: `bazel run` does not start in the repository root. | ||||||
|
|
||||||
| `from` and `until` are inclusive `YYYY-MM-DD` days. `from` is required, | ||||||
| so no span reaches further back than meant; `until` may be omitted while an | ||||||
| affiliation holds. Where the day is unknown, `-01-01` and `-12-31` bound the year. | ||||||
| A day no span covers claims nothing. | ||||||
|
|
||||||
| `name` only labels the entry; contributions are matched by address. | ||||||
| The file answers alone, so everyone whose work is to be claimed belongs in it, | ||||||
| the people in affiliations.json included; an address it omits is reported as a gap. | ||||||
| Its organizations must be declared in affiliations.json, but need not be current ones. | ||||||
| Only `organizations`, `domains` and `ignoreEmails` are still read from there: | ||||||
| those hold whenever an address was used. | ||||||
|
|
||||||
| Dating every revision is what puts the whole history in scope, so `-history` | ||||||
| ignores `-base`. Within a branch's own commits the two agree; over older history | ||||||
| the snapshot claims more, since someone who left an organization keeps claiming | ||||||
| for it, and the people it dropped claim nothing. | ||||||
|
|
||||||
| ## Claims nothing accounts for | ||||||
|
|
||||||
| By default an existing claim is taken as given. `-verify` checks them too, | ||||||
| and reports every organization no contribution to that file accounts for: | ||||||
|
|
||||||
| ```txt | ||||||
| pkg/private/util/fs.go | ||||||
| - // Copyright 2019 ETH Zurich (no contribution from ETH Zurich) | ||||||
| + // Copyright 2022 Anapaya Systems (roos@anapaya.net, 2022-03-12 15d455f11) | ||||||
| ``` | ||||||
|
|
||||||
| Read-only without `-w`: | ||||||
|
|
||||||
| ```sh | ||||||
| make copyright-check COPYRIGHT_FLAGS="-v -verify -history $PWD/tools/copyright/affiliation-history.json" > report.txt 2>/dev/null | ||||||
| ``` | ||||||
|
|
||||||
| A shared line keeps its other holders, and no file loses all its claims: a file | ||||||
| is only processed when it has contributions, and those get claims of their own. | ||||||
|
|
||||||
| Two limits: | ||||||
|
|
||||||
| - It needs `-history`: affiliations.json says where people work today, | ||||||
| so a claim for an organization its contributor has left would look unaccounted for. | ||||||
| - It leaves a file alone when someone who touched it has no known affiliation: | ||||||
| that person may be who a claim rests on. `-v` lists those files. | ||||||
|
|
||||||
| **A claim with nothing behind it is a question, not a verdict.** Code gets copied | ||||||
| between files by hand, work can predate this repository, and a commit can carry | ||||||
| someone else's patch. Read the report and decide; don't pipe it into `-w`. | ||||||
|
|
||||||
| ## Not part of make lint | ||||||
|
|
||||||
| `goheader` in `.golangci.yml` enforces the shape of the header and the presence | ||||||
| of the license text. It does not know who worked on the file. | ||||||
|
|
||||||
| `make copyright-check` reports what is out of date; `make copyright-update` rewrites it. | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find it a bit confusing that part of the documentation lives in README and the other part in doc.go.
Maybe merge them?
Also, it seems the reading order is README.md, the doc.go.