This is just a set of scripts to generate documentation from any programming language. The idea is to divide the process into two steps:
-
Extract the documentation from the source code and save it to a JSON file. The scripts implementing this part should be named
x2doc, wherexis the programming language. At the moment, onlypy2docis implemented. -
Format the documentation in Markdown. The script implementing this part is
doc2md. It just copies verbatim the documentation from the JSON file to the Markdown file. The only exceptions to this rule are the following ones:- Every element in the JSON file is a section of the Markdown file. The script formats the heading.
- For each generated Markdown file, the script injects all known link destinations at the end of the file.
Any additional formatting should be done in the documentation itself.
usage: py2doc.py [-h] [-o OUTPUT] [-e ENCODING]
INPUT[:NAME] [INPUT[:NAME] ...]
Extract docstrings from Python
positional arguments:
INPUT[:NAME] Input file. If the input is followed by a colon, the
text before the colon is used as the module name
options:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file. If not specified, the output is written
to stdout
-e ENCODING, --encoding ENCODING
Encoding of the input files
usage: doc2md.py [-h] [-i INPUT] [-o OUTPUT] [-l LEVEL] [-u URL]
Generate markdown documentation from JSON
options:
-h, --help show this help message and exit
-i INPUT, --input INPUT
Input file. If not specified, the input is read from
stdin
-o OUTPUT, --output OUTPUT
Output file. If not specified, the output is written
to stdout
-l LEVEL, --level LEVEL
Start level for the headers
-u URL, --url URL URL template for the links
usage: md2html.py [-h] INPUT_FILE.md OUTPUT_FILE.html
Render markdown to html using the GitHub API
positional arguments:
INPUT_FILE.md Input Markdown file
OUTPUT_FILE.html Output HTML file
options:
-h, --help show this help message and exit
At this moment, doctest.testmod does not allow to specify a custom doctesr.DocTestParser. This is a problem if you want to wrap a doctest example with a fenced code block.
import doctest
def sum(a, b):
""" This function sums two numbers.
```python
>>> sum(1, 2)
3
```
"""
return a + b
result = doctest.testmod()
exit(int(bool(result.failed)))**********************************************************************
File "__main__", line 8, in __main__.sum
Failed example:
sum(1, 2)
Expected:
3
```
Got:
3
**********************************************************************
1 items had failures:
1 of 1 in __main__.sum
***Test Failed*** 1 failures.
To solve this problem, you can use doctest_utils.MarkdownDocTestParser through doctest_utils.testmod.
import doctest
import doctest_utils
def sum(a, b):
""" This function sums two numbers.
```python
>>> sum(1, 2)
3
```
"""
return a + b
parser = doctest_utils.MarkdownDocTestParser()
result = doctest_utils.testmod(parser=parser)
exit(int(bool(result.failed)))Module doc2md
Format the documentation from JSON to markdown.
Function doc2md.format
def format(doc, level, url=None): ...Format the documentation.
Function doc2md.format.format_node
def format_node(node, level, prefix): ...Format a node.
Function doc2md.format.format_refs
def format_refs(refs): ...Format the references.
It generates two destination link for every element in the JSON file. One with the text as the destination, and another with the text surrounded by backticks.
Function doc2md.escape_markdown
def escape_markdown(text): ...Escape the markdown characters.
Function doc2md.collect_refs
def collect_refs(node): ...Collect the references from the JSON file.
To compose the link, it uses the standard GitHub approach:
- Start with the header text.
- Convert all letters to lowercase.
- Replace all spaces and non-alphanumeric characters with hyphens.
Function doc2md.reader
@contextmanager
def reader(input): ...Open the input file, or stdin if not specified.
Function doc2md.writer
@contextmanager
def writer(output): ...Open the output file, or stdout if not specified.
Module doctest_utils
Extension of doctest to allow testing Markdown texts.
A doctest.DocTestParser that allows use Markdown fences as doctest examples.
This class just patches the original DocTestParser._EXAMPLE_RE to exclude
Markdown fences (``` or ~~~) from the WANT group.
NOTE: This is much better that just removing the fences from the source. Removing lines from the source will make useless the line numbers in the traceback.
Function doctest_utils.testmod
def testmod(m=None, name=None, globs=None, verbose=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, exclude_empty=False, parser=DocTestParser()): ...Same as doctest.testmod but allows to specify a custom DocTestParser.
All master related code is removed.
Module md2html
Convert a markdown file to HTML using GitHub API.
Module py2doc
Extract the docstrings from a Python file and store them in a JSON file. This script doesn't care about the format of the docstrings, it just extracts them. The only exception to this rule is when it finds a doctest, in which case it labels this code snippet as Python code.
Function py2doc.process_node
def process_node(node, path): ...Extract the information from an AST node.
If the docstring contains doctests, the docstring is split into chunks. The doctests are labeled as Python code, and the rest of the docstring is labeled as text.
Function py2doc._compose_definition
def _compose_definition(code): ...Compose the definition of a function or method from its code.
>>> _compose_definition("def foo(a, b): return a + b")
'def foo(a, b): ...'
>>> _compose_definition("async def foo(a, b): return a + b")
'async def foo(a, b): ...'
>>> _compose_definition("def foo(a : int, b : int) -> int: return a + b")
'def foo(a : int, b : int) -> int: ...'Function py2doc.writer
@contextmanager
def writer(output): ...Open a file for writing or use stdout if the output is None.