Unknown node: `pending_xref_condition`

[ruro]

To Reproduce

python -m venv .venv
. .venv/bin/activate
pip install sphinx==4.0.1 sphinx-qt-documentation==0.3 PyQt5==5.15.4
mkdir docs

foo.py:

from PyQt5 import QtGui

def bar() -> QtGui.QIcon:
    pass

docs/conf.py:

import os
import sys

sys.path.insert(0, os.path.abspath("./"))

extensions = [
    "sphinx.ext.autodoc",
    "sphinx.ext.intersphinx",
    "sphinx_qt_documentation",
]
intersphinx_mapping = {
    "PyQt5": ("https://riverbankcomputing.com/static/Docs/PyQt5/", None),
}
python_use_unqualified_type_names = True

docs/index.rst:

test
====

.. automodule:: foo
   :members:
   :undoc-members:

Run:

python -m sphinx -b dirhtml docs .out

Result

# Sphinx version: 4.0.1
# Python version: 3.9.4 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 2.11.3
# Last messages:
#   
#   looking for now-outdated files...
#   none found
#   pickling environment...
#   done
#   checking consistency...
#   done
#   preparing documents...
#   done
#   writing output... [100%] index
# Loaded extensions:
#   sphinx.ext.mathjax (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (1.0.3) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.4) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/alabaster/__init__.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.intersphinx (4.0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/ext/intersphinx.py
#   sphinx_qt_documentation (0.1) from /tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx_qt_documentation/__init__.py
Traceback (most recent call last):
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/application.py", line 350, in build
    self.builder.build_update()
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 292, in build_update
    self.build(to_build,
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 356, in build
    self.write(docnames, list(updated_docnames), method)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 530, in write
    self._write_serial(sorted(docnames))
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 540, in _write_serial
    self.write_doc(docname, doctree)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/builders/html/__init__.py", line 615, in write_doc
    self.docwriter.write(doctree, destination)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/writers/html.py", line 70, in translate
    self.document.walkabout(visitor)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  [Previous line repeated 3 more times]
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 219, in walkabout
    visitor.dispatch_visit(self)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/util/docutils.py", line 472, in dispatch_visit
    super().dispatch_visit(node)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/docutils/nodes.py", line 2021, in dispatch_visit
    return method(node)
  File "/tmp/tmp.oLe3FEHJrF/.venv/lib/python3.9/site-packages/sphinx/writers/html5.py", line 799, in unknown_visit
    raise NotImplementedError('Unknown node: ' + node.__class__.__name__)
NotImplementedError: Unknown node: pending_xref_condition

Environment info

• OS: Linux
• Python version: 3.9.4
• Sphinx version: 4.0.1
• Sphinx extensions: sphinx.ext.autodoc, sphinx.ext.intersphinx, sphinx_qt_documentation

Additional context

This is similar to the issue that was fixed in #8996. I think, that I've tracked the issue to the sphinx-qt-documentation plugin. This plugin connects itself to missing-reference events like this

def setup(app: Sphinx) -> Dict[str, Any]:
    ... # snip
    app.connect("missing-reference", missing_reference)
    ... # snip

however, their missing_reference implementation doesn't have the

    content = find_pending_xref_condition(node, 'resolved')
    if content:
        contnode = content.children[0]  # type: ignore

snippet, which eventually leads to the above Unknown node: pending_xref_condition error.

I could submit this as a bug report to the author of sphinx-qt-documentation, but I wanted to make sure, that this behaviour is intended. It seems to me like a potential antipattern. For example, the above snippet (with minor variations) is already duplicated internally in

• sphinx/domains/python.py:PythonDomain.resolve_xref
• sphinx/domains/python.py:PythonDomain.resolve_any_xref
• sphinx/domains/python.py:builtin_resolver
• sphinx/ext/intersphinx.py:missing_reference

And any plugin that connects to the missing-reference event must also now add this snippet (which is complicated by the fact that find_pending_xref_condition doesn't seem to be available in sphinx<4). Do you think that maybe the contnode value should be resolved when the missing-reference event is created in sphinx/transforms/post_transforms/__init__.py?

[tk0miya]

Thank you for reporting. I guess Sphinx's post processing has a bug when 3rd party extension resolves a missing-reference.

Could you check this patch works fine?

diff --git a/sphinx/transforms/post_transforms/__init__.py b/sphinx/transforms/post_transforms/__init__.py
index e2899d994..54cab4ed6 100644
--- a/sphinx/transforms/post_transforms/__init__.py
+++ b/sphinx/transforms/post_transforms/__init__.py
@@ -96,10 +96,18 @@ class ReferencesResolver(SphinxPostTransform):
                     newnode = self.app.emit_firstresult('missing-reference', self.env,
                                                         node, contnode,
                                                         allowed_exceptions=(NoUri,))
-                    # still not found? warn if node wishes to be warned about or
-                    # we are in nit-picky mode
                     if newnode is None:
+                        # still not found? warn if node wishes to be warned about or
+                        # we are in nit-picky mode
                         self.warn_missing_reference(refdoc, typ, target, node, domain)
+                    elif isinstance(newnode[0], addnodes.pending_xref_condition):
+                        matched = find_pending_xref_condition(node, "*")
+                        if matched:
+                            newnode = matched[0]
+                        else:
+                            logger.warning(__('Could not determine the fallback text for the '
+                                              'cross-reference. Might be a bug.'),
+                                           location=node)
             except NoUri:
                 newnode = None

(I'll try this tomorrow. But I need to build the environment to build the example project. I don't know PyQt at all...)

[ruro]

Thank you for reporting. I guess Sphinx's post processing has a bug when 3rd party extension resolves a missing-reference.

Could you check this patch works fine?

Not quite. It removes the exception, but the PyQt5.QtGui.QIcon return type is still rendered including with the full module name, and it doesn't link to the Qt docs.

I was thinking of something more along the lines of

--- a/sphinx/transforms/post_transforms/__init__.py
+++ b/sphinx/transforms/post_transforms/__init__.py
@@ -69,11 +69,15 @@
 
     default_priority = 10
 
     def run(self, **kwargs: Any) -> None:
         for node in self.document.traverse(addnodes.pending_xref):
-            contnode = cast(nodes.TextElement, node[0].deepcopy())
+            content = find_pending_xref_condition(node, 'resolved')
+            if content:
+                contnode = content.children[0]  # type: ignore
+            else:
+                contnode = cast(nodes.TextElement, node[0].deepcopy())
             newnode = None
 
             typ = node['reftype']
             target = node['reftarget']
             refdoc = node.get('refdoc', self.env.docname)

(although you might need to somehow handle the case, where content has multiple children, I am not 100% sure)

The above patch fixes the error and produces the desired results for the minimal test case, that I have provided, but I am not sure, if it plays well with other components of the reference resolution process.

---

With a little refactoring you should also be able to remove the duplicated code fragments in the following functions:

• sphinx/domains/python.py:PythonDomain.resolve_xref
• sphinx/domains/python.py:PythonDomain.resolve_any_xref
• sphinx/domains/python.py:builtin_resolver
• sphinx/ext/intersphinx.py:missing_reference

All these functions are called from ReferenceResolver.run either directly:

• PythonDomain.resolve_xref by domain.resolve_xref
• PythonDomain.resolve_any_xref from self.resolve_anyref by domain.resolve_any_xref

or via connecting to the missing-reference event:

• domains/python.py:builtin_resolver by app.connect('missing-reference', builtin_resolver, priority=900)
• ext/intersphinx.py:missing_reference by app.connect('missing-reference', missing_reference))

---

(I'll try this tomorrow. But I need to build the environment to build the example project. I don't know PyQt at all...)

You shouldn't need any PyQt knowledge for this. The reason why Qt is involved here is that Qt doesn't have a full Python documentation, because it's a C++ wrapper. So the sphinx-qt-documentation plugin just makes it possible to automatically link from a python type to documentation of the equivalent C++ type. For example, the PyQt5.QtGui.QIcon return type in foo.py should link to the C++ docs for the QIcon class.

[tk0miya]

Thank you for confirmation. And it seems your proposal is better than mine. I posted #9246 to resolve this issue. Could you confirm this, please?

You shouldn't need any PyQt knowledge for this.

On my local, I can't install PyQt5. So I'd like to know the way to build the environment.

root@5de8942ddfb0:/# python
Python 3.8.5 (default, Aug  4 2020, 16:24:08)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5 import QtGui
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libGL.so.1: cannot open shared object file: No such file or directory

[ruro]

Could you confirm this, please?

I've tried the 9246 fix, and it seems to work fine for me.

ImportError: libGL.so.1: cannot open shared object file: No such file or directory

Ah, do you have a headless machine or something? Unfortunately, Qt has some binary requirements, that you can't install into the virtual environment (at least not easily). So you will have to install some packages to your actual system. Try installing python3-pyqt5 directly from apt-get or your package manager. This should also install all the binary dependencies, so the PyQt inside the virtual environment should now work.

On my local, I can't install PyQt5.

Why not? Do you mean that you don't have the admin privileges? Unfortunately, I don't think there's anything you can do in that case other than contacting your admin.