🌭 The hotdog web browser and browser engine 🌭

thdwb

This is the hotdog web browser project. It's a web browser with its own layout and rendering engine, parsers, and UI toolkit!

It's made from scratch entirely in golang. External dependencies are only OpenGL and GLFW, even go dependencies are kept to a minimum.

The main goal of this project is to learn how web browsers work under the hood by implementing one. The browser is far from stable, spec-compliant, or even really useful, but, I'm slowly working on bringing more features and supporting more sites.

🌭 🌭 🌭

Components

  • ketchup (html parser and DOM Tree builder)
  • mayo (css parser and Render Tree builder)
  • mustard (UI Toolkit, events and OpenGL)
  • sauce (requests, cache and filesystem)
  • bun (css layout calculator)
  • gg (drawing routines and text rendering)

Getting started

  • Running

    make

    This command will start the browser and load an example page

  • Testing

    make test

    This command will run all the configurated unit tests

  • Building

    make build

    This command will build the binary version

Screenshots

Comments
  • List Dependencies

    List Dependencies

    Could you update this project to include its dependencies? Every time I try to build it, it tells me I'm missing something, I have to look up what the name of the package that includes that header is, which often times isn't even helpful, install something, and then get a totally new dependency error. I don't know what cannot find -lXxf86vm is even supposed to mean, and nowhere in the repo does it say that this depends on a tonne of packages including xorg-dev, libx11-dev, libgl1-mesa-dev, and libxcursor-dev. What do I actually need to install and build this project?

  • Can't Download/Install

    Can't Download/Install

    Tried downloading as a ZIP file and it refuses to build, saying that it's not a valid repo and cannot find build:

    root@MACHINE:/mnt/c/Users/USERNAME/Downloads/thdwb-master/zip/thdwb-master# make
    fatal: not a git repository (or any parent up to mount point /mnt)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
    fatal: not a git repository (or any parent up to mount point /mnt)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
    # pkg-config --cflags  -- gl gl
    pkg-config: exec: "pkg-config": executable file not found in $PATH
    # github.com/go-gl/glfw/v3.3/glfw
    cgo: exec gcc: exec: "gcc": executable file not found in $PATH
    make: *** [Makefile:11: run] Error 2
    root@MACHINE:/mnt/c/Users/USERNAME/Downloads/thdwb-master/zip/thdwb-master#
    

    Made sure to grab a fresh install of make, pkg-config, and gcc, and it still refuses to build:

    root@MACHINE:/mnt/c/Users/USERNAME/Downloads/thdwb-master/zip/thdwb-master# make
    fatal: not a git repository (or any parent up to mount point /mnt)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
    fatal: not a git repository (or any parent up to mount point /mnt)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
    # github.com/go-gl/glfw/v3.3/glfw
    In file included from ./glfw/src/internal.h:188,
                     from ./glfw/src/context.c:30,
                     from /root/go/pkg/mod/github.com/go-gl/glfw/v3.3/[email protected]/c_glfw.go:4:
    ./glfw/src/x11_platform.h:33:10: fatal error: X11/Xlib.h: No such file or directory
       33 | #include <X11/Xlib.h>
          |          ^~~~~~~~~~~~
    compilation terminated.
    # pkg-config --cflags  -- gl gl
    Package gl was not found in the pkg-config search path.
    Perhaps you should add the directory containing `gl.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'gl' found
    Package gl was not found in the pkg-config search path.
    Perhaps you should add the directory containing `gl.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'gl' found
    pkg-config: exit status 1
    make: *** [Makefile:11: run] Error 2
    

    Tried downloading using Git and GH.exe CLI and it fails to even download every single time:

    PS C:\Users\USERNAME\Downloads\thdwb-master> gh.exe repo clone danfragoso/thdwb
    Cloning into 'thdwb'...
    remote: Enumerating objects: 2143, done.
    remote: Counting objects: 100% (44/44), done.
    remote: Compressing objects: 100% (20/20), done.
    error: RPC failed; curl 56 OpenSSL SSL_read: Connection was reset, errno 10054
    error: 3527 bytes of body are still expected
    fetch-pack: unexpected disconnect while reading sideband packet
    fatal: early EOF
    fatal: fetch-pack: invalid index-pack output
    exit status 128
    
  • Error creating shader on Windows11

    Error creating shader on Windows11

    Exception 0xc0000005 0x8 0x0 0x0 PC=0x0 signal arrived during external code execution

    runtime.cgocall(0xab1c30, 0xc000109c28) D:/go1.19.1/go1.19.1/src/runtime/cgocall.go:158 +0x4a fp=0xc000109c00 sp=0xc000109bc8 pc=0x704e8a github.com/go-gl/gl/v3.2-core/gl._Cfunc_glowCreateShader(0x0, 0x8b31) _cgo_gotypes.go:5527 +0x5a fp=0xc000109c28 sp=0xc000109c00 pc=0x88779a github.com/go-gl/gl/v3.2-core/gl.CreateShader(...) C:/Users/15306/go/pkg/mod/github.com/go-gl/[email protected]/v3.2-core/gl/package.go:9310 github.com/danfragoso/thdwb/mustard.compileShader({0xbe664c, 0x9b}, 0x761e07?) C:/Users/15306/Desktop/thdwb/mustard/shaders.go:39 +0x4a fp=0xc000109cd0 sp=0xc000109c28 pc=0x8a522a github.com/danfragoso/thdwb/mustard.createNewGLProgram({0xbe664c?, 0xacef30?}, {0xbe65bb, 0x91}) C:/Users/15306/Desktop/thdwb/mustard/gl.go:21 +0x3a fp=0xc000109d58 sp=0xc000109cd0 pc=0x8a30fa github.com/danfragoso/thdwb/mustard.createGLBackend() C:/Users/15306/Desktop/thdwb/mustard/gl.go:56 +0x45 fp=0xc000109de8 sp=0xc000109d58 pc=0x8a34e5 github.com/danfragoso/thdwb/mustard.CreateNewWindow({0xbc1036, 0x5}, 0x384, 0x258, 0x1) C:/Users/15306/Desktop/thdwb/mustard/window.go:52 +0x29d fp=0xc000109e78 sp=0xc000109de8 pc=0x8a825d main.main() C:/Users/15306/Desktop/thdwb/browser/main.go:46 +0x2c7 fp=0xc000109f80 sp=0xc000109e78 pc=0xaabc27 runtime.main() D:/go1.19.1/go1.19.1/src/runtime/proc.go:250 +0x1fe fp=0xc000109fe0 sp=0xc000109f80 pc=0x73c01e runtime.goexit() D:/go1.19.1/go1.19.1/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000109fe8 sp=0xc000109fe0 pc=0x766f61

    goroutine 2 [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) D:/go1.19.1/go1.19.1/src/runtime/proc.go:363 +0xd6 fp=0xc000047fb0 sp=0xc000047f90 pc=0x73c3b6 runtime.goparkunlock(...) D:/go1.19.1/go1.19.1/src/runtime/proc.go:369 runtime.forcegchelper() D:/go1.19.1/go1.19.1/src/runtime/proc.go:302 +0xb1 fp=0xc000047fe0 sp=0xc000047fb0 pc=0x73c251 runtime.goexit() D:/go1.19.1/go1.19.1/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000047fe8 sp=0xc000047fe0 pc=0x766f61 created by runtime.init.6 D:/go1.19.1/go1.19.1/src/runtime/proc.go:290 +0x25

    goroutine 3 [GC sweep wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) D:/go1.19.1/go1.19.1/src/runtime/proc.go:363 +0xd6 fp=0xc000049f90 sp=0xc000049f70 pc=0x73c3b6 runtime.goparkunlock(...) D:/go1.19.1/go1.19.1/src/runtime/proc.go:369 runtime.bgsweep(0x0?) D:/go1.19.1/go1.19.1/src/runtime/mgcsweep.go:278 +0x8e fp=0xc000049fc8 sp=0xc000049f90 pc=0x7267ee runtime.gcenable.func1() D:/go1.19.1/go1.19.1/src/runtime/mgc.go:178 +0x26 fp=0xc000049fe0 sp=0xc000049fc8 pc=0x71b386 runtime.goexit() D:/go1.19.1/go1.19.1/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000049fe8 sp=0xc000049fe0 pc=0x766f61 created by runtime.gcenable D:/go1.19.1/go1.19.1/src/runtime/mgc.go:178 +0x6b

    goroutine 4 [GC scavenge wait]: runtime.gopark(0xc000012620?, 0xc7ab18?, 0x1?, 0x0?, 0x0?) D:/go1.19.1/go1.19.1/src/runtime/proc.go:363 +0xd6 fp=0xc000059f70 sp=0xc000059f50 pc=0x73c3b6 runtime.goparkunlock(...) D:/go1.19.1/go1.19.1/src/runtime/proc.go:369 runtime.(*scavengerState).park(0xf4e3a0) D:/go1.19.1/go1.19.1/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc000059fa0 sp=0xc000059f70 pc=0x724873 runtime.bgscavenge(0x0?) D:/go1.19.1/go1.19.1/src/runtime/mgcscavenge.go:617 +0x45 fp=0xc000059fc8 sp=0xc000059fa0 pc=0x724e65 runtime.gcenable.func2() D:/go1.19.1/go1.19.1/src/runtime/mgc.go:179 +0x26 fp=0xc000059fe0 sp=0xc000059fc8 pc=0x71b326 runtime.goexit() D:/go1.19.1/go1.19.1/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000059fe8 sp=0xc000059fe0 pc=0x766f61 created by runtime.gcenable D:/go1.19.1/go1.19.1/src/runtime/mgc.go:179 +0xaa

    goroutine 5 [finalizer wait]: runtime.gopark(0xf4ea00?, 0xc000044ea0?, 0x0?, 0x0?, 0xc00004bf70?) D:/go1.19.1/go1.19.1/src/runtime/proc.go:363 +0xd6 fp=0xc00004be28 sp=0xc00004be08 pc=0x73c3b6 runtime.goparkunlock(...) D:/go1.19.1/go1.19.1/src/runtime/proc.go:369 runtime.runfinq() D:/go1.19.1/go1.19.1/src/runtime/mfinal.go:180 +0x10f fp=0xc00004bfe0 sp=0xc00004be28 pc=0x71a48f runtime.goexit() D:/go1.19.1/go1.19.1/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004bfe8 sp=0xc00004bfe0 pc=0x766f61 created by runtime.createfing D:/go1.19.1/go1.19.1/src/runtime/mfinal.go:157 +0x45 rax 0xc00010a000 rbx 0xc000109c28 rcx 0x8b31 rdi 0xc00010a000 rsi 0xf4e6a0 rbp 0xc000109bf0 rsp 0x8ee89ff6a8 r8 0xf4ea00 r9 0x0 r10 0x146f r11 0x7ffa5f620710 r12 0xc000109ca8 r13 0x140 r14 0xc000044000 r15 0x2030000 rip 0x0 rflags 0x10202 cs 0x33 fs 0x53 gs 0x2b

  • can't build

    can't build

    make : The term 'make' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:1

    • make build
    •   + CategoryInfo          : ObjectNotFound: (make:String) [], CommandNotFoundException
        + FullyQualifiedErrorId : CommandNotFoundException
  • Cannot build

    Cannot build

    thdwb $  make go: downloading github.com/go-gl/glfw/v3.3/glfw v0.0.0-20210410170116-ea3d685f79fb go: downloading github.com/goki/freetype v0.0.0-20181231101311-fa8a33aabaff go: downloading github.com/go-gl/gl v0.0.0-20210501111010-69f74958bac0 go: downloading golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1 go: downloading golang.org/x/image v0.0.0-20210220032944-ac19c3e999fb

    command-line-arguments

    /usr/bin/ld: $WORK/b001/pkg2.a(_x004.o): undefined reference to symbol 'pthread_key_delete@@GLIBC_2.2.5' /usr/bin/ld: /usr/lib/libpthread.so.0: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make: *** [Makefile:11: run] Error 2

Extract structured data from web sites. Web sites scraping.
Extract structured data from web sites. Web sites scraping.

Dataflow kit Dataflow kit ("DFK") is a Web Scraping framework for Gophers. It extracts data from web pages, following the specified CSS Selectors. You

Jan 7, 2023
A complete Liquid template engine in Go
A complete Liquid template engine in Go

Liquid Template Parser liquid is a pure Go implementation of Shopify Liquid templates. It was developed for use in the Gojekyll port of the Jekyll sta

Dec 15, 2022
In-memory, full-text search engine built in Go. For no particular reason.
In-memory, full-text search engine built in Go. For no particular reason.

Motivation I just wanted to learn how to write a search engine from scratch without any prior experience. Features Index content Search content Index

Sep 1, 2022
In-memory, full-text search engine built in Go. For no particular reason.
In-memory, full-text search engine built in Go. For no particular reason.

Motivation I just wanted to learn how to write a search engine from scratch without any prior experience. Features Index content Search content Index

Sep 1, 2022
yview is a lightweight, minimalist and idiomatic template library based on golang html/template for building Go web application.

wview wview is a lightweight, minimalist and idiomatic template library based on golang html/template for building Go web application. Contents Instal

Dec 5, 2021
Build and deploy resilient web applications.

Archived Due to the security concerns surrounding XML, this package is now archived. Go server overview : Template engine. Built in request tracer. we

Dec 15, 2020
Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Geziyor Geziyor is a blazing fast web crawling and web scraping framework. It can be used to crawl websites and extract structured data from them. Gez

Dec 29, 2022
Go minifiers for web formats

Minify Online demo if you need to minify files now. Command line tool that minifies concurrently and watches file changes. Releases of CLI for various

Jan 6, 2023
Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

mxj - to/from maps, XML and JSON Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-

Dec 29, 2022
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler
Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

Pagser Pagser inspired by page parser。 Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and str

Dec 13, 2022
[Go] Package of validators and sanitizers for strings, numerics, slices and structs

govalidator A package of validators and sanitizers for strings, structs and collections. Based on validator.js. Installation Make sure that Go is inst

Dec 28, 2022
Take screenshots of websites and create PDF from HTML pages using chromium and docker

gochro is a small docker image with chromium installed and a golang based webserver to interact wit it. It can be used to take screenshots of w

Nov 23, 2022
Parse data and test fixtures from markdown files, and patch them programmatically, too.

go-testmark Do you need test fixtures and example data for your project, in a language agnostic way? Do you want it to be easy to combine with documen

Oct 31, 2022
Watches container registries for new and changed tags and creates an RSS feed for detected changes.

Tagwatch Watches container registries for new and changed tags and creates an RSS feed for detected changes. Configuration Tagwatch is configured thro

Jan 7, 2022
A general purpose application and library for aligning text.

align A general purpose application that aligns text The focus of this application is to provide a fast, efficient, and useful tool for aligning text.

Sep 27, 2022
Parse placeholder and wildcard text commands

allot allot is a small Golang library to match and parse commands with pre-defined strings. For example use allot to define a list of commands your CL

Nov 24, 2022
:evergreen_tree: Parses indented code and returns a tree structure.

codetree Parses indented code (Python, Pug, Stylus, Pixy, codetree, etc.) and returns a tree structure. Installation go get github.com/aerogo/codetree

Sep 27, 2022
Elegant Scraper and Crawler Framework for Golang

Colly Lightning Fast and Elegant Scraping Framework for Gophers Colly provides a clean interface to write any kind of crawler/scraper/spider. With Col

Dec 30, 2022
Encoding and decoding for fixed-width formatted data

fixedwidth Package fixedwidth provides encoding and decoding for fixed-width formatted Data. go get github.com/ianlopshire/go-fixedwidth Usage Struct

Dec 16, 2022