Repository navigation
linkcheck inconsistent behavior from client #12030
Description
Describe the bug
Running on this project: https://github.com/crim-ca/weaver/
Local Ubuntu 20.04.6 LTS instance, Python 3.10.13 produces :
( changes: line 1633) broken https://github.com/geopython/pywps/commit/343d82539576b1e73eee3102654749c3d3137cff - 406 Client Error: Not Acceptable for url: https://github.com/geopython/pywps/commit/343d82539576b1e73eee3102654749c3d3137cff
(autoapi/weaver/base/index: line 72) redirect https://stackoverflow.com/a/5191224 - with Found to https://stackoverflow.com/questions/5189699/how-to-make-a-class-property/5191224#5191224
On GitHub CI, Python 3.10.13 produces (https://github.com/crim-ca/weaver/actions/runs/7962895777/job/22037962351?pr=604) :
( changes: line 1633) ok https://github.com/geopython/pywps/commit/343d82539576b1e73eee3102654749c3d3137cff
(autoapi/weaver/base/index: line 72) broken https://stackoverflow.com/a/5191224 - 403 Client Error: Forbidden for url: https://stackoverflow.com/a/5191224
Both locations run Sphinx 7.2.6
The errors for the same URLs are replicated over multiple runs, but somehow do not align on different machines.
How to Reproduce
Use the listed URLs in some RST document, and see if they get flagged by linkcheck.
The command make check-links can be used from https://github.com/crim-ca/weaver/ to test exactly the repo where the problem occurs.
Environment Information
Platform: linux; (Linux-5.15.0-97-generic-x86_64-with-glibc2.31)
Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0])
Python implementation: CPython
Sphinx version: 7.2.6
Docutils version: 0.18.1
Jinja2 version: 3.1.2
Pygments version: 2.17.1
> pip freeze
alabaster==0.7.13
alembic==1.12.1
amqp==5.2.0
anyascii==0.3.2
argcomplete==3.1.6
asciitree==0.3.3
astor==0.8.1
astroid==2.15.8
attrs==23.1.0
Babel==2.13.1
bandit==1.7.5
Beaker==1.12.1
beautifulsoup4==4.12.2
billiard==4.2.0
boto3==1.29.4
boto3-stubs==1.29.4
botocore==1.32.4
botocore-stubs==1.32.4
bump2version==1.0.1
CacheControl==0.13.1
celery==5.3.5
certifi==2023.11.17
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.3.0
cligj==0.7.2
codacy-coverage==1.3.11
colander==1.8.3
coloredlogs==15.0.1
contextlib2==21.6.0
cornice==6.0.1
cornice-swagger @ git+https://github.com/fmigneault/cornice.ext.swagger.git@b66bd4d85f5f5550f39d046742ae9799aee1f153
coverage==7.3.2
cryptography==41.0.5
cwl-upgrader==1.2.10
cwl-utils==0.31
cwltool==3.1.20230906142556
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.7
dnspython==2.4.2
doc8==0.11.2
docformatter==1.5.0
docker==6.1.3
docutils==0.18.1
dparse==0.6.3
duration==1.1.1
esgf-compute-api @ git+https://github.com/ESGF/esgf-compute-api.git@8faf829abb3ba0434715c4db44440cfb9c71b55f
execnet==2.0.2
fasteners==0.19
filelock==3.13.1
fiona==1.9.5
flake8==5.0.4
flynt==1.0.1
future==0.18.3
geotiff==0.2.10
gitdb==4.0.11
GitPython==3.1.40
greenlet==3.0.1
gunicorn==21.2.0
humanfriendly==10.0
humanize==4.8.0
hupper==1.12
idna==3.4
imagesize==1.4.1
importlib-resources==6.1.1
iniconfig==2.0.0
iso8601==2.1.0
isodate==0.6.1
isort==5.12.0
Jinja2==3.1.2
jmespath==1.0.1
json2xml==4.1.0
jsonschema==4.20.0
jsonschema-specifications==2023.11.1
kombu==5.3.4
lazy-object-proxy==1.9.0
lxml==4.9.3
Mako==1.3.0
markdown-it-py==3.0.0
MarkupSafe==2.1.3
mccabe==0.7.0
mdurl==0.1.2
mistune==2.0.5
mock==3.0.5
moto==4.2.14
msgpack==1.0.7
mypy==1.7.0
mypy-boto3-s3==1.29.3
mypy-extensions==1.0.0
networkx==3.2.1
numcodecs==0.12.1
numpy==1.26.2
oauthlib==3.2.2
OWSLib==0.29.3
packaging==21.3
parameterized==0.9.0
Paste==3.7.1
PasteDeploy==3.1.0
path==16.7.1
path.py==12.5.0
pbr==6.0.0
Pint==0.22
plaster==1.1.2
plaster-pastedeploy==1.0.1
platformdirs==4.0.0
pluggy==1.3.0
prompt-toolkit==3.0.41
prov==1.5.1
psutil==5.9.6
py==1.11.0
pycodestyle==2.9.1
pycparser==2.21
pydocstyle==6.3.0
pydot==1.4.2
pyflakes==2.5.0
Pygments==2.17.1
pylint==2.17.7
pylint-per-file-ignores==1.3.2
pylint-quotes==0.2.3
pymongo==4.6.0
pyparsing==3.1.1
pyproj==3.6.1
pyramid==2.0.2
pyramid-beaker==0.8
pyramid-celery @ git+https://github.com/crim-ca/pyramid_celery.git@3d4c5096661176b176cab4b7d31b282317c70301
pyramid-mako==1.1.0
pyramid-rewrite==0.2
pyramid-storage==1.3.0
pytest==6.2.5
pytest-fixture-config==1.7.0
pytest-httpserver==1.0.8
pytest-rerunfailures==12.0
pytest-server-fixtures==1.7.0
pytest-shutil==1.7.0
python-dateutil==2.8.2
pytz==2023.3.post1
pywps==4.6.0
PyYAML==6.0.1
rdflib==7.0.0
referencing==0.31.0
requests==2.31.0
requests-file==1.5.1
requests-magpie @ git+https://github.com/Ouranosinc/requests-magpie@b26ed3c2cee4395c1bd745c7e102a442ba2199ae
responses==0.24.1
restructuredtext-lint==1.4.0
retry==0.9.2
rich==13.7.0
rpds-py==0.13.1
ruamel.yaml==0.17.40
ruamel.yaml.clib==0.2.8
s3transfer==0.7.0
safety==2.3.5
schema-salad==8.4.20231117150958
shapely==2.0.2
shellescape==3.8.1
simplejson==3.19.2
six==1.16.0
smmap==5.0.1
snowballstemmer==2.2.0
soupsieve==2.5
Sphinx==7.2.6
sphinx-argparse==0.4.0
sphinx-autoapi==3.0.0
sphinx-autodoc-typehints==1.25.2
sphinx-paramlinks==0.6.0
sphinx-rtd-theme==1.3.0
sphinxcontrib-applehelp==1.0.7
sphinxcontrib-devhelp==1.0.5
sphinxcontrib-htmlhelp==2.0.4
sphinxcontrib-jquery==4.1
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.6
sphinxcontrib-redoc==1.6.0
sphinxcontrib-serializinghtml==1.1.9
SQLAlchemy==2.0.23
stevedore==5.1.0
stopit==1.1.2
termcolor==2.3.0
tifffile==2023.9.26
toml==0.10.2
tomli==2.0.1
tomlkit==0.12.3
translationstring==1.4
types-awscrt==0.19.13
types-s3transfer==0.7.0
typing_extensions==4.8.0
tzdata==2023.3
untokenize==0.1.1
urllib3==2.0.7
urlmatch==1.0.1
venusian==3.1.0
vine==5.1.0
waitress==2.1.2
wcwidth==0.2.11
-e git+ssh://git@github-crim/crim-ca/weaver.git@781dac1bb5429beba99580a4f1118224ef46b6a1#egg=weaver
WebOb==1.8.7
websocket-client==1.6.4
WebTest==3.0.0
Werkzeug==3.0.1
wrapt==1.16.0
WSGIProxy==0.2.2
WSGIProxy2==0.5.1
xmltodict==0.13.0
zarr==2.12.0
zope.deprecation==5.0
zope.interface==6.1Sphinx extensions
https://github.com/crim-ca/weaver/blob/d911c161721c2e25ee4663d34a49d2b218261710/docs/source/conf.py#L54-L68Additional context
Error seen on crim-ca/weaver#604 (but unrelated changes)
Last run that did not raise any issue by linkcheck as on crim-ca/weaver@d234e11 (ie: https://github.com/crim-ca/weaver/actions/runs/7890099969/job/21545963203)
No apparent change to documentation links between them (crim-ca/weaver@d234e11...cmd-err-handling).
Activity
jayaddison commentedon Mar 1, 2024
Hi @fmigneault - unfortunately I think that this is most likely to be due to stackoverflow choosing to respond differently to HTTP requests depending on the origin. If that's so, then in the short-term may have to adapt by using
linkcheckconfiguration options.I'm going to convert this issue into a discussion topic since it isn't immediately clear that there is a bug or enhancement request for the Sphinx codebase here - but perhaps we'll discover one as a result.
Merge pull request #604 from crim-ca/cmd-err-handling
ignore linkcheck HTTP 405 (relates to sphinx-doc/sphinx#12030)