Mmark: a powerful markdown processor in Go geared towards the IETF

title date aliases
About
2018-07-22 14:05:51 +0100
/about/

Build Status

Mmark is a powerful markdown processor written in Go, geared towards writing IETF documents. It is, however, also suited for writing complete books and other technical documentation, like the Learning Go book (mmark source, and I-D text output).

Also see this repository on how to write RFC using Markdown.

It provides an advanced markdown dialect that processes file(s) to produce internet-drafts in XML RFC 7991 format. Mmark can produce xml2rfc (aforementioned RFC 7991), RFC 7749 (xml2rfc version 2 - now deprecated), HTML5 output, markdown and manual pages.

Example RFCs in Mmark format can be found in the Github repository.

Mmark uses gomarkdown which is a fork of blackfriday. See its README.md for more documentation.

If you like Go and parsing text, drop me (mailto:[email protected]) a line if you want to be part of the Mmarkdown Github org, and help develop Mmark!

Syntax

Mmark's syntax and the extra features compared to plain Markdown are detailed in syntax.md.

Mmark adds the following syntax elements to gomarkdown/markdown:

Usage

You can download a binary or optionally build mmark your self. You'll need a working Go environment, then check out the code and:

% go get && go build
% ./mmark -version
2.0.0

To output XML2RFC v3 xml just give it a markdown file and:

% ./mmark rfc/3514.md

Making a draft in text form (v3 output)

% ./mmark rfc/3514.md > x.xml
% xml2rfc --v3 --text x.xml

Outputting HTML5 is done with the -html switch. Outputting markdown is done with the -markdown switch (optionally you can use -width to set the text width).

Example RFC

The rfc/ directory contains a couple of example RFCs that can be build via the v2 or v3 tool chain. The build the text files, just run:

cd rfc
make txt

Official RFCs are in rfc/orig (so you can compare the text output from mmark).

Using Mmark as a library

By default Mmark gives you a binary you can run, if you want to include the parser and renderers in your own code you'll have to lift some of it out of mmark.go.

Create a parser with the correct options and flags. The that init is used to track file includes. In this snippet we set if to fileName which is the file we're currently reading. If reading from standard input, this can be set to "".

p := parser.NewWithExtensions(mparser.Extensions)
init := mparser.NewInitial(fileName)
documentTitle := "" // hack to get document title from TOML title block and then set it here.
p.Opts = parser.Options{
    ParserHook: func(data []byte) (ast.Node, []byte, int) {
        node, data, consumed := mparser.Hook(data)
        if t, ok := node.(*mast.Title); ok {
            if !t.IsTriggerDash() {
                documentTitle = t.TitleData.Title
            }
        }
        return node, data, consumed
    },
    ReadIncludeFn: init.ReadInclude,
    Flags:         parserFlags,
}

Then parser the document (d is a []byte containing the document text):

doc := markdown.Parse(d, p)
mparser.AddBibliography(doc)
mparser.AddIndex(doc)

After this doc is ready to be rendered. Create a renderer, with a bunch of options.

opts := html.RendererOptions{
    Comments:       [][]byte{[]byte("//"), []byte("#")}, // used for callouts.
	RenderNodeHook: mhtml.RenderHook,
	Flags:          html.CommonFlags | html.FootnoteNoHRTag | html.FootnoteReturnLinks| html.CompletePage,
	Generator:      `  <meta name="GENERATOR" content="github.com/mmarkdown/mmark Mmark Markdown Processor - mmark.miek.nl`,
}
opts.Title = documentTitle // hack to add-in discovered title

renderer := html.NewRenderer(opts)

Next we we only need to generate the HTML: x := markdown.Render(doc, renderer). Now x contains a []byte with the HTML.

Also See

Kramdown-rfc2629 is another tool to process markdown and output XML2RFC XML.

Owner
Comments
  • xml: images as artwork doesn't work.

    xml: images as artwork doesn't work.

    for rfc7991 output we can't output images as artwork, as a image element is not an (block)level artwork, this means we must inhibit outputting <t>s, but that is hard. Also using the title for the name attribute looks weird.

    A figureblock works, but must be present in the markdown code.

  • Is it possible to reference table numbers

    Is it possible to reference table numbers

    Is there syntax to include a table number in the document? For example, is something like this available?

    Please refer to table :syntax-inserting-number-of-useful-info-table: for more information
    
    [table]
    Table: useful info {useful-info-table}
    
  • Support other `sectionFormat` options for xrefs

    Support other `sectionFormat` options for xrefs

    What went wrong?

    I attempted to reference another RFC, with the desired output being text of the form "See [RFC1234], Section 1.2."

    To achieve this goal, I wrote See [@RFC1234, section 1.2]., as suggested by the docs. However, this resulted in text of the form "See 1.2".

    In order to get the result I wanted, I had to type the very verbose/redundant See [@RFC1234], Section [@RFC1234, section 1.2].

    This is because mmark produces xrefs with sectionFormat=bare. There are many other sectionFormats that have useful behavior. It would be good to be able to specify that we want the "of", "comma", or "parens" behavior.

    What version of mmark are you using?

    2.2.16

    What is the most minimal markdown snippets that shows the problem?

    See [@RFC1234, section 1.2]

  • Missing docName

    Missing docName

    What went wrong?

    xml2rfc --v3 complained with Warning: Expected a 'docName' attribute in the <rfc/> element, but found none., and https://datatracker.ietf.org/submit/ rejected the XML.

    Manually adding the docName attribute fixed the issue.

    What version of mmark are you using?

    2.0.46

    What command line did you use?

    mmark

    What platform (Linux/Apple/Windows)?

    • [ ] Linux
    • [x] Apple
    • [ ] Windows

    What is the most minimal markdown snippets that shows the problem?

    %%%
    
        Title = "The ristretto255 Group"
        abbrev = "ristretto255"
        ipr = "trust200902"
        category = "info"
        area = "Internet"
        workgroup = "Crypto Forum Research Group"
    
        [seriesInfo]
        status = "informational"
        name = "Internet-Draft"
        value = "draft-hdevalence-cfrg-ristretto-01"
        stream = "IETF"
    
    [...]
    
  • Extra content at the end of the document

    Extra content at the end of the document

    Execute the command: ./mark.exe mqtt.md mqtt.xml on rfc/7511.md Then xml2rfc.exe --v3 mqtt.xml I got this error: "Extra content at the end of the document"

    The errors persist with all the examples

  • xml:refcontent is stripped from the resulting xml file

    xml:refcontent is stripped from the resulting xml file

    When using XML in the markdown, when <reference> attribute contains <refcontent> it is stripped in the resulting XML file.

    For example, in my markdown file I added the following:

    <reference anchor="ECMA-262" target="https://www.ecma-international.org/ecma-262/10.0/index.html">
      <front>
        <title>ECMAScript 2019 Language Specification</title>
        <author>
          <organization>ECMA International</organization>
        </author>
        <date year="2019" month="June"/>
      </front>
      <refcontent>Standard ECMA-262 10th Edition</refcontent>
    </reference>
    

    but in the generated XML, I get this content

    <reference anchor="ECMA-262" target="https://www.ecma-international.org/ecma-262/10.0/index.html">
      <front>
        <title>ECMAScript 2019 Language Specification</title>
        <author>
          <organization>ECMA International</organization>
        </author>
        <date year="2019" month="June"></date>
      </front>
    </reference>
    

    Any reason why this would be the case?

  • The order of <reference> element is random on each call

    The order of element is random on each call

    Everytime you touch a source file and run mmark on it, the list of references at the end has a different order.

    That makes it difficult to track changes in the code or compare with a previous version.

  • submissionType is hardcoded to

    submissionType is hardcoded to "IETF"

    What went wrong?

    The submissionType in the XML output is hardcoded to IETF (https://github.com/mmarkdown/mmark/blob/master/render/xml/title.go#L42) and setting seriesInfo.stream to "independent" as per https://mmark.miek.nl/post/syntax/#title-block causes xml2rfc to barf:

    Error: The stream setting of <seriesInfo> is inconsistent with the submissionType of <rfc>.  Found independent, IETF
    

    What version of mmark are you using?

    2.0.48

  • Minimum version of xml2rfc has recently changed

    Minimum version of xml2rfc has recently changed

    To help us debug your issue please add these details.

    What went wrong?

    The minimum required version for xml2rfc seems to have changed. Until a few weeks ago it was around 2.47.

    What is the new minimum version?

    I notice that v3 output is specified, including by default, so I suspect xml2rfc might need to be v3 as well. That isn't a problem, but it would be helpful to know what is intended.

    mmark/rfc/1035.xml(2373): Error: Did not expect an <iref> here, skipping it 
      :
    1035.xml(0): Error: Type IDREF doesn't allow value 'rfc.index.*', at /rfc/back/section[1]/t/xref[1]
      :
    1035.xml(0): Error: Element li has extra content: t, at /rfc/back/section[1]/ul/li[1]/t   
    

    What version of mmark are you using?

    Git master as of today, 2022-01-29

    mmark -version reports 2.2.25

    What is the most minimal markdown snippets that shows the problem?

    cd rfc
    make txt  
    
  • "Tighten lists" patch leads to illegal XML

    What went wrong?

    minimal.xml(25): Error: Element li has extra content: ol, at /rfc/middle/section/ol/li[2]/ol
    minimal.xml(3): Error: Invalid document before running preptool.
    Unable to complete processing minimal.xml
    

    The problem is that elements are not generated where they should.

    What version of mmark are you using?

    2.2.4; the bug does not appear in 2.2.3

    What command line did you use?

    mmark minimal.md > minimal.xml; xml2rfc --html minimal.xml

    What platform (Linux/Apple/Windows)?

    • [X] Linux
    • [ ] Apple
    • [ ] Windows

    What is the most minimal markdown snippets that shows the problem?

    Define "most minimal" ;-) This is pretty close:

    # x
    
     1. a
     1. b
         1. c
         2. d
     1. e
    
  • WIP: GopherJS integration

    WIP: GopherJS integration

    My earlier work on draftr and draftr-js was pretty half-hearted, but the "no install" experience has been useful to a few people. It would be nice, though, if we could have a web-based Internet-draft authoring tool that kept pace with the more real tooling -- mmark.

    This PR is a proof of concept for how this could be done fairly simply using GopherJS. To see how this works:

    > go get -u github.com/gopherjs/gopherjs
    > cd ${GOPATH}/src/github.com/mmarkdown/mmark/js
    > gopherjs serve
    
    # In browser opened to http://localhost:8080/github.com/mmarkdown/mmark/js
    # Open browser console, and:
    > doc = mmark.NewDocument(mmark.SampleData)
    > html = doc.HTMLFragment()
    
  • mmark output not accepted by xml2rfc

    mmark output not accepted by xml2rfc

    The following input renders fine

    %%%
    title = "Test"
    
    [seriesInfo]
    name = "Internet-Draft"
    stream = "IETF"
    status = "standard"
    value = "0"
    
    [[author]]
    fullname="Martijn van Beurden"
    
    %%%
    
    .# Abstract
    
    Test
    
    {mainmatter}
    
    # bit Rice parameter
    
    More test
    

    But if I change the section title to # 5 bit Rice parameter I get

    test.xml(20): Error: Invalid attribute anchor for element section, at /rfc/middle/section
    /home/pi/bin/flac-specification/test.xml(3): Error: Invalid document before running preptool.
    Unable to complete processing test.xml
    

    Don't know whether this is deliberate or not, it seems xml2rfc doesn't like anchors starting with a digit?

Blackfriday: a markdown processor for Go

Blackfriday Blackfriday is a Markdown processor implemented in Go. It is paranoid about its input (so you can safely feed it user-supplied data), it i

Dec 24, 2021
Enhanced Markdown template processor for golang

emd Enhanced Markdown template processor. See emd README file TOC Install glide

Jan 2, 2022
🚩 TOC, zero configuration table of content generator for Markdown files, create table of contents from any Markdown file with ease.
🚩 TOC, zero configuration table of content generator for Markdown files, create table of contents from any Markdown file with ease.

toc toc TOC, table of content generator for Markdown files Table of Contents Table of Contents Usage Installation Packages Arch Linux Homebrew Docker

Dec 29, 2022
Markdown - Markdown converter for golang

markdown ?? Talks ?? Join ?? Youtube ❤️ Sponsor Install via nami nami install ma

Jun 2, 2022
Mdfmt - A Markdown formatter that follow the CommonMark. Like gofmt, but for Markdown

Introduction A Markdown formatter that follow the CommonMark. Like gofmt, but fo

Dec 18, 2022
pdfcpu is a PDF processor written in Go.
pdfcpu is a PDF processor written in Go.

pdfcpu is a PDF processing library written in Go supporting encryption. It provides both an API and a CLI. Supported are all versions up to PDF 1.7 (ISO-32000).

Jan 4, 2023
⚙️ Convert HTML to Markdown. Even works with entire websites and can be extended through rules.
⚙️ Convert HTML to Markdown. Even works with entire websites and can be extended through rules.

html-to-markdown Convert HTML into Markdown with Go. It is using an HTML Parser to avoid the use of regexp as much as possible. That should prevent so

Jan 6, 2023
Produces a set of tags from given source. Source can be either an HTML page, Markdown document or a plain text. Supports English, Russian, Chinese, Hindi, Spanish, Arabic, Japanese, German, Hebrew, French and Korean languages.
Produces a set of tags from given source. Source can be either an HTML page, Markdown document or a plain text. Supports English, Russian, Chinese, Hindi, Spanish, Arabic, Japanese, German, Hebrew, French and Korean languages.

Tagify Gets STDIN, file or HTTP address as an input and returns a list of most popular words ordered by popularity as an output. More info about what

Dec 19, 2022
Upskirt markdown library bindings for Go

Goskirt Package goskirt provides Go-bindings for the excellent Sundown Markdown parser. (F/K/A Upskirt). To use goskirt, create a new Goskirt-value wi

Oct 23, 2022
A CLI markdown converter written in Go.

MDConv is a markdown converter written in Go. It is able to create PDF and HTML files from Markdown without using LaTeX. Instead MDConv u

Dec 20, 2022
A markdown renderer package for the terminal
A markdown renderer package for the terminal

go-term-markdown go-term-markdown is a go package implementing a Markdown renderer for the terminal. Note: Markdown being originally designed to rende

Nov 25, 2022
A markdown parser written in Go. Easy to extend, standard(CommonMark) compliant, well structured.

goldmark A Markdown parser written in Go. Easy to extend, standards-compliant, well-structured. goldmark is compliant with CommonMark 0.29. Motivation

Dec 29, 2022
:triangular_ruler:gofmtmd formats go source code block in Markdown. detects fenced code & formats code using gofmt.
:triangular_ruler:gofmtmd formats go source code block in Markdown. detects fenced code & formats code using gofmt.

gofmtmd gofmtmd formats go source code block in Markdown. detects fenced code & formats code using gofmt. Installation $ go get github.com/po3rin/gofm

Oct 31, 2022
Convert Microsoft Word Document to Markdown
Convert Microsoft Word Document to Markdown

docx2md Convert Microsoft Word Document to Markdown Usage $ docx2md NewDocument.docx Installation $ go get github.com/mattn/docx2md Supported Styles

Jan 4, 2023
Stylesheet-based markdown rendering for your CLI apps 💇🏻‍♀️
Stylesheet-based markdown rendering for your CLI apps 💇🏻‍♀️

Glamour Write handsome command-line tools with Glamour. glamour lets you render markdown documents & templates on ANSI compatible terminals. You can c

Jan 1, 2023
go-md2man - 转换 Markdown 为 man 手册内容

go-md2man Converts markdown into roff (man pages). Uses blackfriday to process markdown into man pages. Usage ./md2man -in /path/to/markdownfile.md -o

Dec 22, 2022
A PDF renderer for the goldmark markdown parser.
A PDF renderer for the goldmark markdown parser.

goldmark-pdf goldmark-pdf is a renderer for goldmark that allows rendering to PDF. Reference See https://pkg.go.dev/github.com/stephenafamo/goldmark-p

Jan 7, 2023
Markdown to Webpage app

mark2web Markdown to webpage link Usage $ mark2web test.md https://mark2web.test/aa32d8f230ef9d44c3a7acb55b572c8599502701 $ mark2web /tmp/session/test

Apr 18, 2021
Markdown Powered Graph API

What is Arachne? Arachne, (Greek: “Spider”) in [[greek/mythology]], the [[Arachne:daughter of:Idmon of Colophon]] in Lydia, a dyer in purple. Arachne

Dec 19, 2021