hdq - HTML DOM Query Language for Go+

hdq - HTML DOM Query Language for Go+

Build Status Go Report Card GitHub release Coverage Status Language GoDoc

Summary about hdq

hdq is a Go+ package for processing HTML documents.

Tutorials

Collect links of a html page

How to collect all links of a html page? If you use hdq, it is very easy.

import "github.com/qiniu/hdq"

func links(url interface{}) []string {
	doc := hdq.Source(url)
	return [link for a <- doc.any.a, link := a.href?:""; link != ""]
}

At first, we call hdq.Source(url) to create a node set named doc. doc is a node set which only contains one node, the root node.

Then, select all a elements by doc.any.a. Here doc.any means all nodes in the html document.

Then, we visit all these a elements, get href attribute value and assign it to the variable link. If link is not empty, collect it.

At last, we return all collected links. Goto tutorial/01-Links to get the full source code.

Owner
Qiniu Cloud
Connect Data, Redefine Value.
Qiniu Cloud
Comments
  • Bump github.com/goplus/gop from 1.1.0-rc2 to 1.1.0

    Bump github.com/goplus/gop from 1.1.0-rc2 to 1.1.0

    Bumps github.com/goplus/gop from 1.1.0-rc2 to 1.1.0.

    Release notes

    Sourced from github.com/goplus/gop's releases.

    Release v1.1.0

    Go+ v1.1.0 vs. Go+ v1.0.39

    First, it introduces the github.com/goplus/mod module. This module provides support for Go/Go+ module related capabilities. It can be thought of as enhancements to golang.org/x/mod which include:

    Second, it has improved gop mod (module management) related commands, especially the gop mod tidy command needs to be considered in many cases.

    Third, it enhances the completeness of gop command functions such as gop build/install/run/test, which are mainly reflected in:

    • Support for remote packages. For example, in Go+ v1.0 you could only gop run local directories or files, now gop run is supported to run a remote package.
    • Consistency of gop build/install/run/test. Previously, the codes of gop run and gop build/install were written separately, lacking a unified support framework, and their behaviors would be different. Now they are unified into the same implementation framework.
    • The gop cache bug has been eliminated. In Go+ v1.0, due to the poor implementation performance of the import package, the gop cache module cache was introduced, which led to some bugs in module update detection, and an infinite loop of importing some complex packages.

    Fourth, it adds some syntax features that v1.0 does not have. This mainly includes:

    • Overloading of ++, -- operators is supported.
    • Expression type() is supported to represent the zero value of type type. For example, string() is equivalent to "", int32() is equivalent to int32(0), and so on.
    • Supports uint128, int128 types.
    • Supports the any type, which is equivalent to the empty interface.

    Fifth, it adds support for comment instructions in structure and function headers. E.g:

    //go:notinheap
    type S struct{ x int }
    

    //go:noinline //go:uintptrescapes func test(s string, p, q uintptr, rest ...uintptr) int { ... }

    Of course, the comment instruction still needs to be further improved. For example, conditional compilation instructions, etc., are currently not implemented.

    Sixth, fix many bugs. It includes Go+ bugs (issue #888, #1056, #1228, #1243, etc.) and new test cases have been added in go1.8 version, which may not all pass in Go+ v1.0 version (issues #1195, #1196, #1197, #1198, etc.).

    Seventh, it supports Go/Go+ hybrid projects. For details, please refer to the gop/testdata/mixgo example. The following is an introduction to this in the project's README file:

    Eighth, support for calling C from Go+. This includes:

    • The gop c command (equivalent to the stand-alone c2go command) can be used to convert a C project to a Go project.
    • import "C" and import "C/xxx" are used to import a C project converted by c2go. where import "C" is short for import "C/github.com/goplus/libc".
    • The C"xxx" syntax represents C-style string constants.

    Here is an example to show how Go+ interacts with C.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • Bump github.com/goplus/gop from 1.1.0 to 1.1.2

    Bump github.com/goplus/gop from 1.1.0 to 1.1.2

    Bumps github.com/goplus/gop from 1.1.0 to 1.1.2.

    Release notes

    Sourced from github.com/goplus/gop's releases.

    Release v1.1.2

    documentations:

    • Go+ doesn't support DSL (Domain Specific Language), but it's Specific Domain Friendly (see DSL vs. SDF)

    changes:

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • Bump github.com/goplus/gop from 1.1.0-rc1 to 1.1.0-rc2

    Bump github.com/goplus/gop from 1.1.0-rc1 to 1.1.0-rc2

    Bumps github.com/goplus/gop from 1.1.0-rc1 to 1.1.0-rc2.

    Release notes

    Sourced from github.com/goplus/gop's releases.

    Release v1.1.0-rc2

    new features:

    changes:

    incompatible changes:

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • Bump github.com/goplus/gop from 1.0.0-beta5 to 1.1.0-rc1

    Bump github.com/goplus/gop from 1.0.0-beta5 to 1.1.0-rc1

    Bumps github.com/goplus/gop from 1.0.0-beta5 to 1.1.0-rc1.

    Release notes

    Sourced from github.com/goplus/gop's releases.

    Release v1.1.0-rc1

    new features:

    incompatible changes:

    changes:

    Release v1.1.0-beta5

    new features:

    incompatible changes:

    changes:

    Release v1.1.0-beta3

    new features:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • Bump github.com/goplus/gop from 1.1.2 to 1.1.3

    Bump github.com/goplus/gop from 1.1.2 to 1.1.3

    Bumps github.com/goplus/gop from 1.1.2 to 1.1.3.

    Release notes

    Sourced from github.com/goplus/gop's releases.

    Release v1.1.3

    new features:

    changes:

    Commits
    • 8455cf6 Merge pull request #1343 from goplus/main
    • 0c434fe Merge pull request #1342 from goplus/dependabot/go_modules/github.com/goplus/...
    • c8a5885 build(deps): bump github.com/goplus/libc from 0.3.11 to 0.3.12
    • d71b06f Merge pull request #1341 from goplus/dependabot/go_modules/github.com/goplus/...
    • bef3300 build(deps): bump github.com/goplus/c2go from 0.7.9 to 0.7.10
    • b4d8f64 Update go.yml
    • d365b8a Merge pull request #1340 from goplus/dependabot/go_modules/github.com/goplus/...
    • c08a796 build(deps): bump github.com/goplus/gox from 1.11.19 to 1.11.21
    • 5897c98 Merge pull request #1337 from goplus/dependabot/go_modules/github.com/goplus/...
    • 32e9150 build(deps): bump github.com/goplus/libc from 0.3.10 to 0.3.11
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Simple HTML Modification in Go

Simple HTML Modification in Go Do you grin at the sight of html.Node ? Me too. Modifying HTML in Go should be simple. ???? Human friendly: query langu

Sep 29, 2021
⚙️ Concept of Golang HTML render engine with frontend components and dynamic behavior

SSC Engine An HTML render engine concept that brings frontend-like components experience to the server side with native html/template on steroids. Sup

Nov 25, 2022
An idiomatic Go query builder for ElasticSearch

esquery A non-obtrusive, idiomatic and easy-to-use query and aggregation builder for the official Go client for ElasticSearch. Table of Contents Descr

Jan 5, 2023
dqlx is a fully featured DGraph Schema and Query Builder for Go.

dqlx is a fully featured DGraph Schema and Query Builder for Go. It aims to simplify the interaction with the awesome Dgraph database allowing you to fluently compose any queries and mutations of any complexity. It also comes with a rich Schema builder to easily develop and maintain your Dgraph schema.

Dec 17, 2022
Steampipe plugin to query your Scalingo apps, addons and more

Scalingo plugin for Steampipe Use SQL to query infrastructure including applications and addons from Scalingo. Get started → Documentation: Table defi

Nov 4, 2022
Quickly query a Terraform provider's data type.

Terraform Query Quickly query a Terraform provider's data type. Such as a GitHub repository: ➜ ~ tfq github_repository full_name hashicorp/terraform |

Oct 12, 2021
a decision & trigger framework backed by Google's Common Expression Language used in graphikDB

a decision & trigger framework backed by Google's Common Expression Language used in graphikDB

Nov 15, 2022
Clojure-esque extension language for Go.
Clojure-esque extension language for Go.

let-go Greetings loafers! (λ-gophers haha, get it?) This is supposed to be a compiler and bytecode VM for a language resembling Clojure as close as po

Jan 1, 2023
This is an open source project for commonly used functions for the Go programming language.

Common Functions This is an open source project for commonly used functions for the Go programming language. This package need >= go 1.3 Code Conventi

Jan 8, 2023
Rest Api Generator for Golang Programming Language

Rest Api Generator for Golang Programming Language

Nov 29, 2021
Go Lang Web Assembly bindings for DOM, HTML etc

WebAPI Go Language Web Assembly bindings for DOM, HTML etc WARNING: The current API is in very early state and should be consider to be expremental. T

Dec 28, 2022
DOM library for Go and WASM

Go DOM binding (and more) for WebAssembly This library provides a Go API for different Web APIs for WebAssembly target. It's in an active development,

Dec 23, 2022
domui: DOM UI framework for Go

domui: DOM UI framework for Go

Jul 23, 2022
Gom: DOM building using Go

gom DOM building using Go Usage package main import "github.com/hadihammurabi/gom" func main() { dom := gom.H("html").Children( gom.H("head").Chi

Dec 16, 2021
Dom - A Go API for different Web APIs for WebAssembly target

Go DOM binding (and more) for WebAssembly This library provides a Go API for dif

Jan 7, 2023
bluemonday: a fast golang HTML sanitizer (inspired by the OWASP Java HTML Sanitizer) to scrub user generated content of XSS

bluemonday bluemonday is a HTML sanitizer implemented in Go. It is fast and highly configurable. bluemonday takes untrusted user generated content as

Jan 4, 2023
Simple system for writing HTML/XML as Go code. Better-performing replacement for html/template and text/template

Simple system for writing HTML as Go code. Use normal Go conditionals, loops and functions. Benefit from typing and code analysis. Better performance than templating. Tiny and dependency-free.

Dec 5, 2022
VMail - check the markup (HTML, CSS) of HTML email template compatibility with email clients
VMail - check the markup (HTML, CSS) of HTML email template compatibility with email clients

VMail - check the markup (HTML, CSS) of HTML email template compatibility with email clients Email clients use different rendering standards. This is

Dec 17, 2022
This command line converts .html file into .html with images embed.

embed-html This command line converts .html file into .html with images embed. Install > go get github.com/gonejack/embed-html Usage > embed-html *.ht

Oct 6, 2022
htmlquery is golang XPath package for HTML query.

htmlquery Overview htmlquery is an XPath query package for HTML, lets you extract data or evaluate from HTML documents by an XPath expression. htmlque

Jan 4, 2023