simple GitHub action to parse Markdown Links into a .yaml file for Hugo

Obsidian Link Scrapper

Used by Quartz

This repository comes to you in two parts.

  1. GitHub Action (scrapes links into a .yml file)
  2. Hugo Partial (turns .yml file into graphs and tables)

GitHub Action

GitHub action and binary to scrape Obsidian vault for links and exposes them as a .yml file for easy consumption by Hugo.

Example Usage (Binary)

Read Markdown from the /content folder and place the resulting linkIndex.yaml into /data

# Installation
go install github.com/jackyzha0/hugo-obsidian

# Run
hugo-obsidian -input=content -output=data

Example Usage (GitHub Action)

Add 'Build Link Index' as a build step in your workflow file (e.g. .github/workflows/deploy.yaml)

...

jobs:
  deploy:
    runs-on: ubuntu-18.04
    steps:
      - uses: actions/checkout@v2
      - name: Build Link Index
        uses: jackyzha0/[email protected]
        with:
          input: content # input folder
          output: data   # output folder
      ...

Hugo Partial

To then embed this information in your Hugo site, you can copy and use the provided partials in /partials. Graph provides a graph view of all nodes and links and Backlinks provides a list of pages that link to this page.

To start, create a graphConfig.yaml file in /data in your Hugo folder. This will be our main point of configuration for the graph partial.

Then, in one of your Hugo templates, do something like the following to render the graph.

{{partial "graph_partial.html" .}}
">
<div id="graph-container">
    {{partial "graph_partial.html" .}}
div>

Configuration

Example:

enableLegend: false
enableDrag: true
enableZoom: false
base:
  node: "#284b63"
  activeNode: "#f28482"
  inactiveNode: "#a8b3bd"
  link: "#babdbf"
  activeLink: "#5a7282"
paths:
  - /toc: "#4388cc"
  - /newsletters: "#e0b152"
  - /posts: "#42c988"
Owner
Jacky Zhao
writes code, words, and sometimes climbs rocks
Jacky Zhao
Comments
  • Use the filename when no title attribute is present

    Use the filename when no title attribute is present

    Hey @jackyzha0, I found that for my digital garden use case many of my files did not have the necessary title frontmatter and this seemed like a sensible default (at least for me). If it's useful to you, feel free to use it, if not my feelings won't be hurt either.

  • failed to install Hugo-obsidian

    failed to install Hugo-obsidian

    I tried to install Hugo-obsidan, but it seemed that I didn't install it

    $ go install github.com/jackyzha0/hugo-obsidian
    go install: version is required when current directory is not in a module
    	Try 'go install github.com/jackyzha0/hugo-obsidian@latest' to install the latest version
    MacBook-Pro:quartz leo$ go install github.com/jackyzha0/hugo-obsidian@latest
    MacBook-Pro:quartz leo$ hugo-obsidian -input=content -output=data
    -bash: hugo-obsidian: command not found
    

    Would you please help out? Thanks

    Best regards

  • [BUG]Multiple nodes for files with spaces in their name

    [BUG]Multiple nodes for files with spaces in their name

    Title is self explanatory. Problem doesn't occur without spaces.

    Here's my attempt at understanding the problem, I will try tomorrow or in the next days to see if I can confirm and maybe fix it.

    For the linkIndex.yaml generation, if I understand correctly, the program will use the file name+path for the source , and the link inside the page for target . With markdown styled links obsidian uses encoded urls (' ' -> '%20'). This creates a name difference between the target("a%20b") and the source("a b"), resulting in 2 different nodes pointing at the same file.

  • Links to domain .md could not be recognized correctly

    Links to domain .md could not be recognized correctly

    • I tried to add a markdown link [Obsidian](https://obsidian.md) on the website using quartz. However, when click the link on the website, it forward the address tohttps://obsidian/, an address without .md.
    • I think maybe quartz recognize the domian .md as a markdown file, thus forward it to a wrong address.
    • I also changed the https://obsidian.md to https://www.obsidian.md, but it's the same result without .md
    • I tried this on chrome browser both on android and macOS.
    • Here is what the image of what has been rendered: image

    Please check it out, Regards.

    This issue is recommended to be filed here from https://github.com/jackyzha0/quartz/issues/11#issue-975445797

  • [Feature Request] Grey out broken links

    [Feature Request] Grey out broken links

    Obsidian Publish, when serving a note that contains links to non-public notes, shows the link doesn't exist by greying it out and making it not redirect when clicked. Quartz makes the link look like an existent link, but when clicked the link 404's, which is less clean.

    In the case in which a vault has a large proportion of links to non-public notes (either because of only wanting to serve a portion of the true vault or by using links as note ToDos) this makes the vault navigable only if you can put up with a significant proportion of your clicks being 404'd.

  • [Bug] Fail to render Hugo sites on Windows 11

    [Bug] Fail to render Hugo sites on Windows 11

    Cannot render local website by running hugo-obsidian -input=content -output=data -index -root=. then hugo server. Returns

    ERROR 2022/02/12 10:31:26 render of "page" failed: execute of template failed: template: _default/single.html:34:6: executing "_default/single.html" at <partial "footer.html" .>: error calling partial: execute of template failed: template: partials/footer.html:4:10: executing "partials/footer.html" at <partial "backlinks.html" .>: error calling partial: "..\layouts\partials\backlinks.html:12:30": execute of template failed: template: partials/backlinks.html:12:30: executing "partials/backlinks.html" at <index (index $contentTable .) "title">: error calling index: index of untyped nil
    

    The problem is caused by Line 12 in layouts\partials\backlinks.html:

    <a href="{{.}}">{{index (index $contentTable .) "title"}}</a>
    

    This error can be reproduced on Windows 11 and Golang version 1.17.6 Windows/AMD 64.

  • Fix wikilink processing bug

    Fix wikilink processing bug

    • With WikiLink header syntax such as [[file.md#header-name|link name]], the target becomes file.md.html#header-name, so we must split the target by # before stripping .md.html.
    • Previously we were attempting to strip .md.html and then split by #, so all wikilinks to specific tags were not processed properly (and thus being treated as external links)
  • v2.6

    v2.6

    Changelog

    • split code
    • actually parse ignored (fixes https://github.com/jackyzha0/quartz/issues/24)
    • add last modified time metadata for content indexing
  • Images' links can't parse correctly

    Images' links can't parse correctly

    Such as assets/xxx.png won't parse to relative path. If add / at beginning, it's ok. But the markdown files like xxx/xxx.md can be parsed to ../../xxx/xxx.md

  • Footnotes are treated as links

    Footnotes are treated as links

    Hey, I want to thank you first for your great work! I just noticed that footnotes are treated as links between file, and this leads to broken nodes in graph-view in quartz.

    Example: 图片 In quartz, this footnote is represented as follows in linkIndex.json: 图片

    Because linkIndex.json is generated by hugo-obsidian, so I think the problem can be fixed by excluding all the footnotes when generating linkIndex.json.

  • fix: unescape target URL

    fix: unescape target URL

    For URLs in non-ascii, the target name should un-escaped to be shown properly.

    For example, before this commit, target written in Arabic "/البرمجة" is shown as /%D8%A7%D9%84%D8%A8%D8%B1%D9%85%D8%AC%D8%A9.

    After this fix (last link at bottom right): image

  • Orphaned Links are added to graph indices

    Orphaned Links are added to graph indices

    Describe the bug Orphaned links are added in the indices. If the notes contain many orphan links, this would results in increased node size and unnecessary clutter in graph.

    Expected Behavior Non existent nodes either should not be added to indices or represented separately in graph. Further A config can also be provided to enable or disable orphan nodes.

    Screenshot I added these two links markdownlink,[[wikilink]] orphan_link orphan_link2

  • URL escape is different from hugo file name escape

    URL escape is different from hugo file name escape

    Such as, YUNG's Better Caves will be YUNGs-Better-Caves but YUNG's-Better Caves in plugin

    Maybe need something like https://github.com/flytam/filenamify/blob/master/filenamify.go

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Jan 3, 2023
yaml-patch is a version of Evan Phoenix's json-patch, which is an implementation of JSON Patch, directly transposed to YAML

yaml-patch yaml-patch is a version of Evan Phoenix's json-patch, which is an implementation of JavaScript Object Notation (JSON) Patch, directly trans

Jan 15, 2022
A Github action to codon optimize sequences.

codon-optimize A Github action to codon optimize sequences. codon-optimize is a Github Action that receives a path for an amino acid fasta file (faa),

Jul 28, 2022
tail a text file and trigger an action if a new line occurs [wip]

Tailpipe Synopsis Config Help Synopsis Tail a file and trigger an action if a new line occurs. Currently only mailing the new line somewhere is suppor

Dec 23, 2021
Q2entities - Parse the entities string from a Quake 2 .bsp map file. Written in Go

Q2Entities A simple command-line utility to extract the entities string from a Quake 2 map file. Entities? Binary Space Partitioning maps (.bsp) conta

Apr 9, 2022
CUE utilities and helpers for working with tree based objects in any combination of CUE, Yaml, and JSON.

Cuetils CUE utilities and helpers for working with tree based objects in any combination of CUE, Yaml, and JSON. Using As a command line binary The cu

Dec 24, 2022
libraries for various programming languages that make it easy to generate per-process trace files that can be loaded into chrome://tracing
libraries for various programming languages that make it easy to generate per-process trace files that can be loaded into chrome://tracing

chrometracing: chrome://tracing trace_event files The chrometracing directory contains libraries for various programming languages that make it easy t

Oct 6, 2022
A port of the parser from graphql-js into golang

gqlparser This is a parser for graphql, written to mirror the graphql-js reference implementation as closely while remaining idiomatic and easy to use

Dec 27, 2022
Go library for decoding generic map values into native Go structures and vice versa.

mapstructure mapstructure is a Go library for decoding generic map values to structures and vice versa, while providing helpful error handling. This l

Dec 28, 2022
go-i18n is a Go package and a command that helps you translate Go programs into multiple languages.

go-i18n is a Go package and a command that helps you translate Go programs into multiple languages.

Jan 2, 2023
Chaos Engineering tool for introducing failure into syscalls

Syscall monkey Chaos Engineering tool for tampering with syscalls.

Jun 11, 2022
A Go utility to convert Go example tests into jupyter notebooks.

go2colab Scientists (my main project's users) love jupyter notebook tutorials pkg.dev.go's runnable playground doesn't support file IO but I love exam

Jul 10, 2022
Tiny Go tool for running multiple functions concurrently and collecting their results into an error slice.

Overview Short for "ConCurrent". Tiny Go tool for running multiple functions concurrently and collecting their results into an error slice. Dependency

Nov 22, 2021
Small tool for splitting files found in a path into multiple groups

Small tool for splitting files found in a path into multiple groups. Usefull for parallelisation of whatever can be paralleled with multiple files.

Jan 30, 2022
Getting into Golang 1.18

Go 1.18 New features of Golang 1.18 with tutorial and examples. In this repository I introduce new features of Golang version 1.18 with their examples

Aug 28, 2022
sigurls is a reconnaissance tool, it fetches URLs from AlienVault's OTX, Common Crawl, URLScan, Github and the Wayback Machine.

sigurls is a reconnaissance tool, it fetches URLs from AlienVault's OTX, Common Crawl, URLScan, Github and the Wayback Machine. DiSCLAIMER: fe

May 22, 2021
📚 Clone all your repositories from GitHub with one command!

An utility to clone easily all your repositories from GitHub. The process is done concurrently so it is quite fast.

Apr 6, 2022
Utility to add network config file in apk

Utility to add network config file in apk. Which bypass the proxy intercept restriction for user installed burpsuit CA certificate.

Aug 19, 2022
Package tail implements file tailing with fsnotify.

Tail Package tail implements file tailing with fsnotify. Fork of nxadm/tail, simplified, reworked and optimized. Fully supports only posix-compatible

Nov 30, 2022