pack has unresolved deltas with git-aggregator 3.0.0 and 3.0.1 with git 2.20.1 (debian buster, 10)

[ap-wtioit]

not yet reproduced with public commits (will update tomorrow)

when using git-aggregator 3.0.0 or 3.0.1 with git 2.20.1 and performing gitaggregate --expand-env --config .../repo.yaml aggregate:

./odoo:
  defaults:
    depth: $DEPTH_MERGE
  merges:
  - example $ODOO_VERSION
  - example 13.0-patch_branch
  remotes:
    odoo: https://gitlab-ci-token:...@gitlab.example.com/group_a/odoo.git
    example: https://gitlab-ci-token:...@gitlab.example.com/group_b/odoo.git
    example-odoo: https://gitlab-ci-token:...@gitlab.example.com/group_a/odoo.git
  target: example $ODOO_VERSION

with

• ODOO_VERSION 13.0
• DEPTH_MERGE 100
  we get the following error:

(INFO) [14:45:42] git_aggregator.repo  odoo   Start aggregation of /tmp/tmppzdk14gj/odoo
(INFO) [14:45:42] git_aggregator.repo  odoo   Cloning git repository https://gitlab-ci-token:...@gitlab.example.com/group_b/odoo.git in /tmp/tmppzdk14gj/odoo
Cloning into '/tmp/tmppzdk14gj/odoo'...
Checking out files: 100% (25549/25549), done.
(INFO) [14:46:17] git_aggregator.repo  odoo   Switch to branch 13.0
Reset branch '13.0'
(INFO) [14:46:18] git_aggregator.repo  odoo   Adding remote odoo <https://gitlab-ci-token:...@gitlab.example.com/group_a/odoo.git>
(INFO) [14:46:18] git_aggregator.repo  odoo   Adding remote example <https://gitlab-ci-token:...@gitlab.example.com/group_b/odoo.git>
Your branch is up to date with 'origin/13.0'.
(INFO) [14:46:18] git_aggregator.repo  odoo   Adding remote example-odoo <https://gitlab-ci-token:...@gitlab.example.com/group_b/odoo.git>
(INFO) [14:46:18] git_aggregator.repo  odoo   Fetching required remotes
From https://gitlab.example.com/group_b/odoo
 * branch              13.0       -> FETCH_HEAD
 * [new branch]        13.0       -> example/13.0
fatal: pack has 42 unresolved deltas
fatal: index-pack failed
(ERROR) [14:46:26] git_aggregator.repo  odoo   /tmp/tmppzdk14gj/odoo> error calling ('git', 'fetch', '--depth', '100', 'example', '13.0-patch_branch')
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/main.py", line 230, in aggregate_repo
    repo.aggregate()
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/repo.py", line 183, in aggregate
    self.fetch()
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/repo.py", line 248, in fetch
    self.log_call(cmd, cwd=self.cwd)
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/repo.py", line 160, in log_call
    ret = callwith(cmd, **kw)
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '('git', 'fetch', '--depth', '100', 'example', '13.0-patch_branch')' returned non-zero exit status 128.

When i increase DEPTH_MERGE to 1000 i get fatal: pack has 228 unresolved deltas and the rest stays the same.

Those errors do not happen if i use git-aggregator 2.1.0 (or my local dev environment with git 2.34.1 + python 3.10).

It does look like it's related to the --filter stuff but i think the root cause is somewhere in git.

[pedrobaeza]

I'm having the same problem when merging several sources.

[LoisRForgeFlow]

We are also facing the same issue.

[sbidoul]

I don't have a reproducer. Can someone try to dig into this?
In the meantime, I'm going to yank the 3.0.x releases.

[sbidoul]

Can you try on a fresh clone without the depth option?

[ap-wtioit]

@sbidoul i reproduced it in https://github.com/ap-wtioit/git-aggregator_test_68 with public commits

running ./test_issue_68.sh gives you the following output (shortened to the relevant part):

Successfully tagged test_issue_68-99292-3.0.1:latest
(INFO) [06:54:52] git_aggregator.repo  odoo   Start aggregation of /root/odoo
(INFO) [06:54:52] git_aggregator.repo  odoo   Cloning git repository https://github.com/odoo/odoo.git in /root/odoo
Cloning into '/root/odoo'...
Checking out files: 100% (25549/25549), done.
(INFO) [06:56:01] git_aggregator.repo  odoo   Switch to branch 13.0
Reset branch '13.0'
Your branch is up to date with 'origin/13.0'.
(INFO) [06:56:02] git_aggregator.repo  odoo   Adding remote odoo <https://github.com/odoo/odoo.git>
(INFO) [06:56:02] git_aggregator.repo  odoo   Fetching required remotes
From https://github.com/odoo/odoo
 * branch              13.0       -> FETCH_HEAD
 * [new branch]        13.0       -> odoo/13.0
fatal: pack has 34 unresolved deltas
fatal: index-pack failed
(ERROR) [06:56:11] git_aggregator.repo  odoo   /root/odoo> error calling ('git', 'fetch', '--depth', '100', 'odoo', 'refs/pull/99292/head')
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/main.py", line 230, in aggregate_repo
    repo.aggregate()
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/repo.py", line 183, in aggregate
    self.fetch()
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/repo.py", line 248, in fetch
    self.log_call(cmd, cwd=self.cwd)
  File "/usr/local/lib/python3.7/dist-packages/git_aggregator/repo.py", line 160, in log_call
    ret = callwith(cmd, **kw)
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '('git', 'fetch', '--depth', '100', 'odoo', 'refs/pull/99292/head')' returned non-zero exit status 128.

When cloning current odoo 13.0 (odoo/odoo@cd1a85a) with the new --filter option and merging pull request 99292 (odoo/odoo@ddb768a) into it, git-aggregate fails in version 3.0.1 where it worked before

Update 09:26:
could reproduce the issue in plain git (2.20.1) commands (./test_issue_68_git.sh):

git clone --filter=blob:none -b 13.0 --depth 100 https://github.com/odoo/odoo.git /root/odoo
cd /root/odoo
git checkout -B 13.0
git remote add odoo https://github.com/odoo/odoo.git
git fetch --depth 100 odoo 13.0
git fetch --depth 100 odoo refs/pull/99292/head

reproduces the error while the following (using remote origin) does work:

git clone --filter=blob:none -b 13.0 --depth 100 https://github.com/odoo/odoo.git /root/odoo
cd /root/odoo
git checkout -B 13.0
git fetch --depth 100 origin 13.0
git fetch --depth 100 origin refs/pull/99292/head

Update 10:06:

• https://stackoverflow.com/a/52916879/925125 mentions bugfixes for git clone --filter up to git 2.33
• --filter only appeared in the man page in 2.27 (https://git-scm.com/docs/git-clone/2.27.0 vs https://git-scm.com/docs/git-clone/2.25.0)
• using the newest ./test_issue_68_git.sh that drops to an interactive bash upon the error happening
  • GIT_TRACE=1 git fetch --verbose --depth 20 odoo refs/pull/99292/head shows the error
  • GIT_TRACE=1 git fetch --verbose --depth 19 odoo refs/pull/99292/head does work

Update 10:30:
i did not find any options that would make the git clone --filter work with the following git fetches with git 2.20.1, my recommendation would be to only use git clone --filter starting with git 2.33 (or at least starting with 2.27 when it's in the man page).
As our pip installs still pick up the 3.0.1 version from pypi https://pypi.org/project/git-aggregator/#history (not yet yanked) i will now focus on fixing our builds for the weekend.

[sbidoul]

Thanks for the investigation. I ultimately did not yank because it seems to work fine in many cases.

I tried your reproducer on Ubuntu 20.04 (git 2.25.1) and it works fine:

git clone --filter=blob:none -b 13.0 --depth 100 https://github.com/odoo/odoo.git /root/odoo
cd /root/odoo
git checkout -B 13.0
git remote add odoo https://github.com/odoo/odoo.git
git fetch --depth 100 odoo 13.0
git fetch --depth 100 odoo refs/pull/99292/head

So my proposal would be to enable --filter blob:none with git >= 2.25.1.

I would also add a note in the docs that says that the depth option may not help that much with performance anymore.

[nicomacr]

We also face this, any news about it?

[wtaferner]

bump

[sbidoul]

Until a fix is proposed, I'd recommend upgrading git to 2.25.1+ and maybe not use the depth option.