When I decided to release Sawppy to the world, I thought briefly about how to best organize all the information I want to convey for rover assembly. I quickly fell into a state of Analysis Paralysis and, as a path out of that state, decided that it was better to have something written down whatever the format. No matter how unorganized, is still better than keeping it all in my head.

I first tried putting it in the “Build Instructions” section, but that feature has some strange and unpredictable limitations that became annoying as the length of instructions grew. The final straw was when I noticed that images and instructions for earlier steps were disappearing as I added later steps. That made me… unhappy, so I went to something else.

The second attempt is what I have as of today: a loose collection of Markdown files on a Github repository. Edited in a code editor rather than a word processor, I struggled with typos and grammatical errors as I lacked the usual automated proofreading tools present in a word processor. Still, with a large helping of assembly pictures, it was just barely enough for people to build their own rovers.

I was painfully aware of the fact there is a ton of obvious room for improvement. This was just the “get it written down” first stage and at some point I need to revisit the various problems still open. The most significant of which is lack of structure beyond an index page with links to all the other pages. The index suggested a relative ordering that matched my personal assembly order, but that doesn’t necessarily work for anyone else. And worse, they would be stuck if they wanted to ask some specific questions my layout is unable to answer.

As an example of problems with Sawppy’s documentation, look at a single component: a Sawppy wheel axle. There are at least three entirely separate pages relating to the wheel axle:

1. The page of parts list, telling a builder to buy 8mm metal shaft.
2. The page for 8mm shaft modification, where I describe how to cut the long shaft into shorter segments. Followed by steps to turn these segments into wheel axles. Other segments were turned into steering shafts, plus those turned into rocker-bogie suspension pivots.
3. The page for wheel hub assembly, which incorporates a single segment of the 8mm wheel axle shaft.

While these three files were all linked from the index page, there’s no obvious way to retrieve the relationship between them in the context of wheel axles. I can manually add links between them, but this is time consuming and perpetually incomplete. Even worse, as the number of relationships grew, it will quickly become a maintenance nightmare.

Thus I started my quest to find a system to document and track these types of relationships without losing too much of the advantages of my current system. This means every candidate system must meet the following challenges:

Challenge A: Easy to Contribute

When a Markdown file is hosted on GitHub, a potential contributor can click the “Edit” button to create a fork of the repository, make a quick fix, and create a pull request all via GitHub website. This presents an extremely low barrier to entry for contributors which is a feature I want to preserve. If contributors were required to install and learn some piece of documentation software, that would discourage many people from participating before we even talk about the learning curve for that software.

Challenge B: Easy to Manage

When using GitHub’s web interface to edit a Markdown file, visualizing the change is as simple as clicking over to the “Preview” tab of the editor. Sadly such ease can’t be matched by any system external to GitHub, but it would be nice to have some way to let a contributor see what the end results look like. Failing that, I must have a way to visualize the final impact of changes before I merge a pull request. It is unacceptable to not see changes until after merging into the main branch.

Challenge C: Easy to Query

The desired documentation system will take metadata written by document author and build an interactive presentation. This way rover builders can find information without being constrained by the current linear narrative. Here are some examples of questions I’ve received for Sawppy, rephrased in terms of wheel axle.

• (Easy: positive query) I’m on the wheel hub assembly page, and it needs the wheel axle which I guess I forgot to build. Where do I need to go?
• (Hard: negative query) My order of 8mm shafts got delayed. What can I work on now that’s not dependent on having these shafts?
• (Both of above) How can a teacher most effectively divide up rover tasks so multiple students teams can build the rover in parallel?

Challenge D: Free

It would be nice for the system to be free as in freedom, but at the very least it must be free as in beer. Design for Sawppy is given away at no cost to rover fans everywhere, there is no profit to cover monetary expense of a documentation system.

---

This was the problem statement I presented to the combined Write the Docs LA/SGVLUG meetup on the evening of October 8th. Sawppy’s current system of a loose set of Markdown files score highly on challenges A and B, but fall flat on C. What can I look into that helps with C without giving up the rest?

---

DITA

Fundamentally speaking, challenge C is not new, as it would be desirable in any large scale engineering project. The novel twist here is the desire to do it all with a system that is inviting for public contribution. As a general rule, documentarians for large engineering projects are professionals who have undergone training and have licenses for proprietary software tools.

Most such tools are excluded from consideration due to cost, but many of them deal with DITA, an open XML-based data model for authoring and publishing under the custody of the OASIS consortium. It is the standard answer to reassure customers wary of being locked in to proprietary file formats. And since it is an open format, there exists a DITA Open Toolkit to transform DITA data to desired output formats… HTML, PDF, even Markdown! There are learning resources at https://learningdita.com/

As a XML (and thus text) based format, DITA would be GitHub friendly for branching and merging. It is very flexible for creating any organization we want (creating a “Subject Schema” for DITA) but taking advantage of that power will take work. DITA Open Toolkit functions at a lower level relative to the other tools discussed here. A quick web search found many commercial offerings built on DITA (example: https://easydita.com/) but failed to find free open source counterpart.

Bill of Materials

Similar to documentation, the challenge of tracking parts and components is well-tread ground for engineering projects. People at the meeting with industry experience suggested a few terms. MRP or Materials Resource & Planning was one, plus several variants on BOM or Bill of Materials. This area is big business! Sadly free open source tools are scarce. Someone gave a pointer to OpenBOM.com but upon further research I was disappointed to find that despite the name, this tool is not actually open.

That leaves us with few choices between “use a spreadsheet” and big ticket enterprise software. Even if numerous choices were available, such tools are focused on a very small subset of the overall problem. I do want to set up some kind of parts management, but bringing in a full fledged BOM tool adds a lot of complexity I’m not sure is justified for a small scale project like Sawppy.

Model All The Things

I have the same concern about another series of suggestions. The idea is to fully model everything about Sawppy using a modeling language like SysML or PlantUML. I agree doing so will result in a complete picture, breaking down every part of the rover project. Such data could then feed into software packages that generate visualizations plotting dependencies between components. That sounds good, but the amount of work also feels disproportionate to the benefit it would bring to a project of this scale.

Static Site Generators

I’m glad I had the chance to learn about the tools listed above. They are great tools for their respective niches, but adopting any of them would require major changes and I prefer to take a smaller incremental step. Which brings us to static site generators. (SSG) This category of software tools see a lot of open source development, giving us many options to choose from.

Background: As input, SSGs take content in various formats. A set of rules and templates are applied to that content, generating as output a set of HTML and CSS (and maybe JavaScript) files that can be served by any web server. “Static” in this context means the web server does not have to run any code to modify the files, they are transmitted directly to users’ web browsers as-is. (As opposed to a server-side systems like PHP.)

A large number of SSGs accept Markdown as a text content input format, meaning Sawppy’s existing Markdown documentation can be used with very few changes. This also preserves the advantages of using Markdown, meeting most of challenges A and B.

Every SSG offers customization of the rules and templates that it applies to content. At the minimum there are themes for cosmetic appearance, but plugins and extensions allow more extensive functionality. This is where I hope to create something that meets my challenge C including a lightweight BOM tool. Around this point Eric spoke up that the JPL Open Source Rover documentation system has a script that generates parts list from document content, but the generated dependency tree is not exposed to the viewer. I want to build upon that precedent and also make this kind of information available to rover builders.

To be pedantic, Sawppy documentation is already using a SSG because GitHub does not display raw Markdown files. A simple transformation to HTML has been applied for display. However, the reason I started this investigation is because the simple GitHub transformation is very limited. GitHub is aware of this, and as an upgrade, has built-in support for a SSG called Jekyll for generating GitHub Pages.

As another example, most of us had experience reading technical software documentation on some subdomain of ReadTheDocs. All of these content were generated by a SSG. MkDocsand Sphinx are two popular SSGs for ReadTheDocs, and a lot of the default functionality (automatic indexing, references, etc.) useful for technical software documentation would be useful for Sawppy as well.

But features like a lightweight BOM tool would be outside the scope of software documentation, so there were several recommendations for investigating Hugo. It is apparently the current darling for flexibly transforming content (including Markdown text) into varying different presentations. Associated with Hugo is Docsy, a Hugo theme focused on technical documentation.

I plan to start with Hugo with the Docsy theme and expand my experimentation from there.

Preview Changes

But using a SSG also introduces the problem of previewing pages, because we’re no longer directly using the simple Markdown transformation. Contributors should not be expected to set up a SSG environment on their computer in order to see the results of their work, and asking maintainers to review every change on their own SSG environment would not scale.

The answer is to bring in tools from the continuous integration world, where tools exist to preview changes to a website before deploying live. Some use services like Netlify, which is not itself open source but there is a free tier available.

One example: look at the repository for Write the Docs website. Open one of the pending pull requests and click on “Show all checks”. One of them is “Read the Docs build succeeded!” and clicking “Details” will bring up a version of the site built with changes in the pull request.

---

At this point I ran out of steam, and the meeting ran out of time, but I have a big treasure trove of pointers to investigate and keep me busy for a while.

Other notes:

• Project Tetra is a hardware project that has a documentation publishing system that uses Docsy and Hugo. It also uses PlantUML and Graphvis to render diagrams. https://tetrabiodistributed.github.io/project-tetra-docs/explanations/documentation-overview/
• Markdown is not the only lightweight markup language, here’s a list on Wikipedia: https://en.wikipedia.org/wiki/Lightweight_markup_language
• It is already quite clear Hugo is not the only documentation system out there, but Wikipedia has many more in a list: https://en.wikipedia.org/wiki/Comparison_of_documentation_generators