Translate README files in reStructuredText with Sphinx

reStructuredText (reST) is a markup language that is popular in the Python developers community. reST is the standard markup language for docutils, Sphinx documentation generator, and the Python Package Index (PyPI). However, reST by now is still not popular enough. Most translation platforms, including Crowdin which I’m using now for EH Forwarder Bot, have no support to reST documents.

Sphinx has provided a plug-in sphinx-intl, which extract strings from reST documents and compile into GNU gettext message catalog template (.pot) files, and build new documents with translated strings in other languages. GNU gettext formats are widely accepted by translation platforms, making our life much easier. This would work out-of-box if you are generating HTML or PDF documentations, but not so simple if you want a reST output.

To utilize what we have in sphinx-intl, what we need is instead a document writer for Sphinx that outputs reST. There is a plug-in for called restbuilder that serves this very purpose, but it has not been updated for over a year. restbuilder is currently looking for maintainers. Unfortunately I don’t really have much time to maintain such a complex project. What I did is just forked the project, included some other fixes from existing PRs, and fixed some more stuff myself.

Extract strings

Since sphinx-build in general works with directories, we need to create a new temporary directory to isolate README.rst. We don’t want to include write a conf.py file just for the translation either as everything can be indicated in command line.

<span>mkdir</span> .build_readme
<span>cp </span>README.rst .build_readme/README.rst
sphinx-build <span>-b</span> gettext <span>-C</span> <span>-D</span> <span>master_doc</span><span>=</span>README <span>-D</span> <span>gettext_additional_targets</span><span>=</span>literal-block,image .build_readme ./readme_translations/locale/ .build_readme/README.rst
<span>rm</span> <span>-rf</span> .build_readme
<span>mkdir</span> .build_readme
<span>cp </span>README.rst .build_readme/README.rst
sphinx-build <span>-b</span> gettext <span>-C</span> <span>-D</span> <span>master_doc</span><span>=</span>README <span>-D</span> <span>gettext_additional_targets</span><span>=</span>literal-block,image .build_readme ./readme_translations/locale/ .build_readme/README.rst
<span>rm</span> <span>-rf</span> .build_readme
mkdir .build_readme cp README.rst .build_readme/README.rst sphinx-build -b gettext -C -D master_doc=README -D gettext_additional_targets=literal-block,image .build_readme ./readme_translations/locale/ .build_readme/README.rst rm -rf .build_readme

Enter fullscreen mode Exit fullscreen mode

Flags used on line 3:

  • -b gettext: Use gettext string extractor as builder
  • -C: Use no config file at all, only -D options
  • -D master_doc=README: Set the front page name as README (that’s the only file we have here)
  • -D gettext_additional_targets=literal-block,image: Include code blocks and images (and captions) into the message catalog.
  • .build_readme: Source directory
  • ./readme_translations/locale/: Destination directory
  • .build_readme/README.rst: File to build

This will extract strings into ./readme_translations/locale/README.pot. This file can then be uploaded onto any translation platform or directly passed over to translators.

Build translated README files

When you have got translated strings from your translators, you need to arrange it into a folder structure that sphinx-intl can recognize, namely {language_code}/LC_MESSAGES/README.(po|mo). I have arranged them in readme_translations/locale/{language_code}/LC_MESSAGES/ in my case.

To build translated README files, we need to install the restbuilder plugin for Sphinx. Here I’ll use my fork as example.

git clone https://github.com/blueset/restbuilder.git
<span>cd </span>restbuilder.git
python3 setup.py
git clone https://github.com/blueset/restbuilder.git
<span>cd </span>restbuilder.git
python3 setup.py
git clone https://github.com/blueset/restbuilder.git cd restbuilder.git python3 setup.py

Enter fullscreen mode Exit fullscreen mode

Then, write a script to iterate through the locale folder and get a list of languages available, and use that to build a list of commands to run.

<span>import</span> <span>glob</span>
<span>from</span> <span>pathlib</span> <span>import</span> <span>Path</span>
<span># My language code is defined in a POSIX-like style. E.g. en_US </span><span>languages</span> <span>=</span> <span>[</span><span>i</span><span>[</span><span>i</span><span>.</span><span>rfind</span><span>(</span><span>'</span><span>/</span><span>'</span><span>)</span><span>+</span><span>1</span><span>:]</span> <span>for</span> <span>i</span> <span>in</span> <span>glob</span><span>.</span><span>glob</span><span>(</span><span>"</span><span>./readme_translations/locale/*_*</span><span>"</span><span>)]</span>
<span># Compile .po files to .mo </span><span>sources</span> <span>=</span> <span>glob</span><span>.</span><span>glob</span><span>(</span><span>"</span><span>./**/*.po</span><span>"</span><span>,</span> <span>recursive</span><span>=</span><span>True</span><span>)</span>
<span>dests</span> <span>=</span> <span>[</span><span>i</span><span>[:</span><span>-</span><span>3</span><span>]</span> <span>+</span> <span>"</span><span>.mo</span><span>"</span> <span>for</span> <span>i</span> <span>in</span> <span>sources</span><span>]</span>
<span>actions</span> <span>=</span> <span>[[</span><span>"</span><span>msgfmt</span><span>"</span><span>,</span> <span>sources</span><span>[</span><span>i</span><span>],</span> <span>"</span><span>-o</span><span>"</span><span>,</span> <span>dests</span><span>[</span><span>i</span><span>]]</span> <span>for</span> <span>i</span> <span>in</span> <span>range</span><span>(</span><span>len</span><span>(</span><span>sources</span><span>))]</span>
<span># Build translated README files </span><span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>mkdir</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source</span><span>"</span><span>])</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>cp</span><span>"</span><span>,</span> <span>"</span><span>README.rst</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source/README.rst</span><span>"</span><span>])</span>
<span>locale_dirs</span> <span>=</span> <span>(</span><span>Path</span><span>(</span><span>'</span><span>.</span><span>'</span><span>)</span> <span>/</span> <span>"</span><span>readme_translations</span><span>"</span> <span>/</span> <span>"</span><span>locale</span><span>"</span><span>).</span><span>absolute</span><span>()</span>
<span>for</span> <span>i</span> <span>in</span> <span>languages</span><span>:</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>sphinx-build</span><span>"</span><span>,</span> <span>"</span><span>-b</span><span>"</span><span>,</span> <span>"</span><span>rst</span><span>"</span><span>,</span> <span>"</span><span>-C</span><span>"</span><span>,</span>
<span>"</span><span>-D</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>language=</span><span>{</span><span>i</span><span>}</span><span>"</span><span>,</span> <span>"</span><span>-D</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>locale_dirs=</span><span>{</span><span>locale_dirs</span><span>}</span><span>"</span><span>,</span>
<span>"</span><span>-D</span><span>"</span><span>,</span> <span>"</span><span>extensions=sphinxcontrib.restbuilder</span><span>"</span><span>,</span>
<span>"</span><span>-D</span><span>"</span><span>,</span> <span>"</span><span>master_doc=README</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./.build_readme/</span><span>{</span><span>i</span><span>}</span><span>"</span><span>])</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>mv</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./.build_readme/</span><span>{</span><span>i</span><span>}</span><span>/README.rst</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./readme_translations/</span><span>{</span><span>i</span><span>}</span><span>.rst</span><span>"</span><span>])</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>rm</span><span>"</span><span>,</span> <span>"</span><span>-rf</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./.build_readme/</span><span>{</span><span>i</span><span>}</span><span>"</span><span>])</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>rm</span><span>"</span><span>,</span> <span>"</span><span>-rf</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source</span><span>"</span><span>])</span>
<span>import</span> <span>glob</span>
<span>from</span> <span>pathlib</span> <span>import</span> <span>Path</span>

<span># My language code is defined in a POSIX-like style. E.g. en_US </span><span>languages</span> <span>=</span> <span>[</span><span>i</span><span>[</span><span>i</span><span>.</span><span>rfind</span><span>(</span><span>'</span><span>/</span><span>'</span><span>)</span><span>+</span><span>1</span><span>:]</span> <span>for</span> <span>i</span> <span>in</span> <span>glob</span><span>.</span><span>glob</span><span>(</span><span>"</span><span>./readme_translations/locale/*_*</span><span>"</span><span>)]</span>

<span># Compile .po files to .mo </span><span>sources</span> <span>=</span> <span>glob</span><span>.</span><span>glob</span><span>(</span><span>"</span><span>./**/*.po</span><span>"</span><span>,</span> <span>recursive</span><span>=</span><span>True</span><span>)</span>
<span>dests</span> <span>=</span> <span>[</span><span>i</span><span>[:</span><span>-</span><span>3</span><span>]</span> <span>+</span> <span>"</span><span>.mo</span><span>"</span> <span>for</span> <span>i</span> <span>in</span> <span>sources</span><span>]</span>
<span>actions</span> <span>=</span> <span>[[</span><span>"</span><span>msgfmt</span><span>"</span><span>,</span> <span>sources</span><span>[</span><span>i</span><span>],</span> <span>"</span><span>-o</span><span>"</span><span>,</span> <span>dests</span><span>[</span><span>i</span><span>]]</span> <span>for</span> <span>i</span> <span>in</span> <span>range</span><span>(</span><span>len</span><span>(</span><span>sources</span><span>))]</span>

<span># Build translated README files </span><span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>mkdir</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source</span><span>"</span><span>])</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>cp</span><span>"</span><span>,</span> <span>"</span><span>README.rst</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source/README.rst</span><span>"</span><span>])</span>

<span>locale_dirs</span> <span>=</span> <span>(</span><span>Path</span><span>(</span><span>'</span><span>.</span><span>'</span><span>)</span> <span>/</span> <span>"</span><span>readme_translations</span><span>"</span> <span>/</span> <span>"</span><span>locale</span><span>"</span><span>).</span><span>absolute</span><span>()</span>
<span>for</span> <span>i</span> <span>in</span> <span>languages</span><span>:</span>
    <span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>sphinx-build</span><span>"</span><span>,</span> <span>"</span><span>-b</span><span>"</span><span>,</span> <span>"</span><span>rst</span><span>"</span><span>,</span> <span>"</span><span>-C</span><span>"</span><span>,</span>
                    <span>"</span><span>-D</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>language=</span><span>{</span><span>i</span><span>}</span><span>"</span><span>,</span> <span>"</span><span>-D</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>locale_dirs=</span><span>{</span><span>locale_dirs</span><span>}</span><span>"</span><span>,</span>
                    <span>"</span><span>-D</span><span>"</span><span>,</span> <span>"</span><span>extensions=sphinxcontrib.restbuilder</span><span>"</span><span>,</span>
                    <span>"</span><span>-D</span><span>"</span><span>,</span> <span>"</span><span>master_doc=README</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./.build_readme/</span><span>{</span><span>i</span><span>}</span><span>"</span><span>])</span>
    <span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>mv</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./.build_readme/</span><span>{</span><span>i</span><span>}</span><span>/README.rst</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./readme_translations/</span><span>{</span><span>i</span><span>}</span><span>.rst</span><span>"</span><span>])</span>
    <span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>rm</span><span>"</span><span>,</span> <span>"</span><span>-rf</span><span>"</span><span>,</span> <span>f</span><span>"</span><span>./.build_readme/</span><span>{</span><span>i</span><span>}</span><span>"</span><span>])</span>
<span>actions</span><span>.</span><span>append</span><span>([</span><span>"</span><span>rm</span><span>"</span><span>,</span> <span>"</span><span>-rf</span><span>"</span><span>,</span> <span>"</span><span>./.build_readme/source</span><span>"</span><span>])</span>
import glob from pathlib import Path # My language code is defined in a POSIX-like style. E.g. en_US languages = [i[i.rfind('/')+1:] for i in glob.glob("./readme_translations/locale/*_*")] # Compile .po files to .mo sources = glob.glob("./**/*.po", recursive=True) dests = [i[:-3] + ".mo" for i in sources] actions = [["msgfmt", sources[i], "-o", dests[i]] for i in range(len(sources))] # Build translated README files actions.append(["mkdir", "./.build_readme/source"]) actions.append(["cp", "README.rst", "./.build_readme/source/README.rst"]) locale_dirs = (Path('.') / "readme_translations" / "locale").absolute() for i in languages: actions.append(["sphinx-build", "-b", "rst", "-C", "-D", f"language={i}", "-D", f"locale_dirs={locale_dirs}", "-D", "extensions=sphinxcontrib.restbuilder", "-D", "master_doc=README", "./.build_readme/source", f"./.build_readme/{i}"]) actions.append(["mv", f"./.build_readme/{i}/README.rst", f"./readme_translations/{i}.rst"]) actions.append(["rm", "-rf", f"./.build_readme/{i}"]) actions.append(["rm", "-rf", "./.build_readme/source"])

Enter fullscreen mode Exit fullscreen mode

More flags used:

  • -b rst: Use reST as output format
  • -D language={language}: Indicate language code to use
  • -D extensions=sphinx.contrib.restbuilder: Load the restbuilder extension installed

With the commands above, you can build translated README files automatically with Sphinx and GNU gettext message catalogs. For full sample code with doit automation, visit the script in EFB Telegram Master Channel repository, and look for task_gettext and task_msgfmt methods.

The post Translate README files in reStructuredText with Sphinx appeared first on 1A23 Blog.

原文链接:Translate README files in reStructuredText with Sphinx

© 版权声明
THE END
喜欢就支持一下吧
点赞5 分享
Do not find excuses for failure, to chase success reasons.
不要找失败的借口,去追成功的理由
评论 抢沙发

请登录后发表评论

    暂无评论内容