Automatically generate contracts documentation [WIP]

[michalinacienciala]

We are creating a GH Actions workflow which automatically generates the
contracts documentation based on the functions and the NatSpec-format comments
in the Solidity files stored in the contracts folder in
keep-network/tbtc-v2/solidity. For certain workflow triggers, the generated
documentation gets published to the api-docs.threshold.network preview or main
GCP bucket.

Workflow triggers:

• workflow_dispatch - If the worklow gets triggered manually, it will just
  build the docs, but will not publish them to the GCP bucket.
• pull_request - If the workflow gets triggered by a PR, it will check if the
  changes in the PR modify the './.github/workflows/docs.yml' file or files in
  the './solidity/contracts' folder. If yes, the workflow will build the HTML
  documentation and publish it to the preview GCP bucket
  (api-docs.threshold.network/solidity/<branch_name>) and will publish a link
  to the generated file in the PR's comment.
• push - If the workflow gets triggered by the push to the Solidity release
  branch (branch, which name starts with releases/mainnet/solidity/), the
  workflow will build the HTML documentation and publish it to the preview GCP
  bucket (api-docs.threshold.network/solidity/<branch_name>).
• release - If the workflow gets triggered by the Solidity release (release,
  which tag starts with refs/tags/solidity/), the workflow will build the HTML
  documentation and publish it to the main GCP bucket
  (api-docs.threshold.network/solidity/).

How HTML documentation gets created:

The documentation gets created based on the content of the Solidity files in
keep-network/tbtc-v2/solidity. We first transform it to the Markdown format,
then translate it to Asciidoc and finally, based on that, create an HTML output
file with contracts documentation, which we publish to the GCP bucket. Here is
the description of that process:

1. Before we run the Docgen tool generating Markdown files, we need to perform
   some slight changes to the input files, as some of the formatting we use in
   our .sol files is interpreted by Docgen not the way we would like or is not
   completely in line with the NatSpec format:
   • In many @dev clauses in the Solidity files we have the lists of
     requirements or other items which are constructed like this:

      /// @dev Requirements:
      /// - Item one. Lorem ipsum dolor sit amet, consectetur adipiscing elit.
      ///   Nulla sed porttitor enim, sit amet venenatis enim. Donec tincidunt
      ///   auctor velit non eleifend. Nunc sit amet est non ligula condimentum
      ///   mattis.
      /// - Item two. Quisque purus massa, pellentesque in viverra tempus,
      ///   aliquet nec urna.
     

     This doesn't get recognized by Docgen as a list and is translated to
     regular text, which is displayed as one continuous line. But when the space
     characters between /// and the text get removed from he .sol files, the
     lists are recognized correctly (without breaking interpretation of other
     features). That's why we decided to run sed -i 's_///[[:blank:]]*_///_'
     command on all Solidity files.
   • In one line of the BitcoinTx.sol file there's an incorrectly used //
     comment inside the @dev clause, which makes the Docgen think think that
     the clause is ending in the line with // (which results in wierd and
     incomplete display of the description of the BitcoinTx function). Due to
     that we are removing the problematic line from the file before running
     Docgen by running sed -i ':a;N;$!ba;s_///\n//\n_///\n_g' on the file.
2. Once the files are ready, we use the Docgen tool
   (https://github.com/OpenZeppelin/solidity-docgen) to generate Markdown
   index.md file with the contracts documentation. The tool is
   configured to
   • export the documentation of all contracts into one common file (default
     behavior),
   • place the generated file into generated-docs folder,
   • don't generate documentation for contracts in the contracts/test folder
     (as those are test/stub contracts which are not used on Mainnet),
   • use custom template for Markdown generation (based on the default
     https://github.com/OpenZeppelin/solidity-docgen/blob/master/src/themes/markdown/common.hbs
     template, but with removed italicisation of the {{{natspec.dev}}} element
     • because it wasn't working well with the lists in the @dev clauses).
3. Then we transform the index.md Markdown file into tbtc-v2-contracts.adoc
   Asciidoc file using tool called Kramdoc-AsciiDoc
   (https://github.com/asciidoctor/kramdown-asciidoc).
4. Next we inject two lines at the beginning of that file, to add the table of
   contens:

   :toc: left
   :toclevels: 1
   

5. Once that is done, we build the tbtc-v2-contracts.html' HTML file using custom thesis/asciidoctor-actionGH Acion. The action generates that file into./asciidoc-out/solidity/generated-docs` folder.
6. We then use thesis/gcp-storage-bucket-action to push the content of that
   folder to the api-docs.threshold.network/solidity main GCP bucket (or
   api-docs.threshold.network/solidity/<branch_name> preview GCP bucket)

Alternative tools for Solidity documentation generation that were considered:

• Doxity (https://github.com/DigixGlobal/doxity) - I tried to use it, but hit a
  bunch of issues during installation - some of which I resolved, but got stuck
  on can't build project DigixGlobal/doxity#25. The project is not
  maintained anymore, latest commits are from year 2017. I figured that
  exploring the tool more and trying to find a workaround for the issue I
  observed may be a waste of time.
• Dodoc (https://github.com/primitivefinance/primitive-dodoc) - yields similar
  results as Docgen (although slightly worse - some @dev/@notice comments were
  not displayed at all), but does not have an option to output the documentation
  to one combo file - each .sol file generates a single .md file.
• Solidoc (https://github.com/binodnp/solidoc) - operates on .NET, I haven't
  explored that tool
• Remix IDE plugin (https://remix-ethdoc-plugin.readthedocs.io/en/latest/) - I
  don't use Remix IDE, also I don't think we could use this plugin in automation
• Hardhat Docgen (https://www.npmjs.com/package/hardhat-docgen) - I already had
  most of the work with my workflow done when I learned about this tool. I tried
  it on, but hit errors when running hardhat-compile. Couldn't find a quick
  solution, decided not to investigate further.

[github-actions]

Documentation preview uploaded to https://api-docs.threshold.network/solidity/document-contracts/tbtc-v2-contracts.html.

[github-actions]

Documentation preview uploaded to https://docs.keep.network/document-contracts/tbtc-v2-contracts.html.

[github-actions]

Documentation preview uploaded to api-docs.threshold.network/solidity/557/merge/tbtc-v2-contracts.html.

[github-actions]

Documentation preview uploaded to https://api-docs.threshold.network/solidity/557/merge/tbtc-v2-contracts.html.

[github-actions]

Documentation preview uploaded to https://api-docs.threshold.network/solidity/557/merge/tbtc-v2-contracts.html.

[michalinacienciala]

Closed in favor of #584.