Linksmith RFC#

This document originates from an excellent conversation at pueblo’s sketch about adding an inventory decoder for Sphinx.

It is an informal living document, and may be enhanced through gists from other and subsequent conversations as we go. Contributions of any kind are very much appreciated and welcome.

Objectives#

Exploring a few ideas around the topics shared below, in order to create an optimal linksmith program, mostly just a gobble of glue code, calling out to existing tools and libraries, and poly-filling the gaps.

  • sphobjinv doesn’t know how to introspect conf.py:intersphinx_mapping, to go from project name to docset URL.

  • sphobjinv is meant to be able to run whether or not the user is in a Sphinx documentation directory tree.

  • People are asking for Markdown Output, but sphobjinv will unlikely implement it.

  • People are asking for Multi-Project Support, but sphobjinv doesn’t have it.

  • Colleagues are running Community Operations like Sviatoslav’s intersphinx-untangled, or Brian’s intersphinx-gist, with code that is not packaged for reusability.

  • MyST support, specifically MEP 0002 - Cross Reference Simplifications using Markdown Links, should not be left behind, but instead should be considered from the very beginning. Primarily using Markdown will certainly be the future of technical writing with Sphinx.

  • Focus on technically relevant details around Sphinx references, cross- references, and inventories, but also don’t forget about style, using the new Hyper role, and friends.

Markdown Output#

Maybe Markdown output has not been considered yet?

Markdown output? Why?

We usually reach out to Markdown generation these days, because the output can be used both as a text-only variant, and for upgrading to HTML by sending it through a corresponding Markdown renderer.

In this way, text output from utility programs becomes super versatile, because it can be easily shared on all systems understanding Markdown, like Sphinx/MyST, GitHub, Discourse, or even Grafana, all tools we use most of the time for information sharing and conveyance.

By using Markdown, it was easy to convey different sections of output into a single output file. In this case, the inventory decoder needed to take into account that we wanted to include intersphinx references for multiple projects, effectively all inventory files listed in sphinx-inventories.txt.

This has now converged into https://github.com/crate/crate-docs/issues/105, not quite reaching the goal yet, because the generated Markdown became too large to be shared on behalf of a GitHub comment 🙈.

Multi-Project Support#

Provide different means of seeding / wrapping around sphobjinv using multiple projects.

It would be straightforward to write a wrapper [to handle multi-project] processing. […] [However,] I don’t consider it within sphobjinv’s purview to actually be that consolidating wrapper, though.

We will be happy to explore this on behalf of linksmith, by pulling in sphobjinv. Excellent! Because we have the need for standalone usage (i.e. without access to conf.py), we consider this wrapper to receive the following features aka. variants, to seed a list of projects:

  • Parse intersphinx_mapping out of conf.py, like suggested by @chrisjsewell.

  • Use a list of paths or URLs to multiple objects.inv files, like sphinx-inventories.txt, either by providing a file, or on stdin/cmdline. This reflects the use case we have, based on the code in tasks.py.

  • Autodiscovery-like features on top of both variants: For example, a conf.py can be local or on a remote URL, possibly within a doc or docs folder within a Git project pointed at. Similarly, an objects.inv file can be located within a {doc,docs}/_build/ folder within a typical Git project, or within a /en/latest URL path when hosted on RTD, for example. That would be just a bunch of convenience heuristics on top for people who want them.

  • Consider how the new references builder in Sphinx by @chrisjsewell could be used in this context.

Community Operations#

  • Consider the use case of @webknjaz, to run a multi-tenant static intersphinx generator operation, on behalf of a list of projects like sphinx-websites.yml, based on relevant code to generate a static website, if he doesn’t have any objections about it?

  • In order to further increase community support and leverage, fragments of @bskinn’s community operations around intersphinx-gist and intersphinx_mappings.txt could also be absorbed into code instead of hosting it on behalf of a text file in a gist.

My thinking here is that if you already have a tool in hand that takes care about those topics in a generalized way, it would feel like an artificial boundary why it would only work on local projects.

In the same spirit like @webknjaz curated a list of popular Python projects on behalf of intersphinx-untangled in YAML format, and @bskinn did on behalf of the intersphinx_mappings.txt Gist, those lists could also be written down or wrapped through Python, in order to provide easy command-line access, right?