41 Resources
Golang scraper Libraries
🔎 scan the internet to find "private" proxies.
🔎 scan the internet to find "private" proxies. 🧠 HTTP/SOCKS4/SOCKS5 Proxies. 📌 Installation: - sudo apt-get install git zmap golang
Bulk image downloader for reddit.
rrip Program to bulk-download image from reddit subreddits. Features Set max size of file, max total size, minimum score etc.. Filter by post title or
GoLang ads.txt scraper
Collects and parses ads.txt GoLang program scrapes sites for ads.txt and stores its significant details to PostgreSQL database. Give it a file with CS
Warhammer40K faction scraper written in Golang, powered by colly.
Wascra Description Wascra is a tool written in Golang, which lets you extract all relevant Datasheet info from a Warhammer40K (9th edition) faction fr
A simple scraper to export data from buildkite to honeycomb using opentelemetry SDK
A quick scraper program that let you export builds on BuildKite as OpenTelemetry data and then send them to honeycomb.io for slice-n-dice high cardinality analysis.
Ratemyprof scraper - Ratemyprof scraper with golang
ratemyprof scraper visit https://ratemyprof-api.vercel.app/api/getProf to try ou
Depth controllable Web scraper and Sitemap Generator in Go
Scrape Scrape is minimalistic depth controlled web scraping project. It can be used as command-line tool or integrate it in your project. Scrape also
A scraper to generate ASL Anki Cards
ASL Pro Video Downloader Directions In the same folder as asl-pro.exe: Create fi
👾 Fast and simple video download library and CLI tool written in Go
👾 Annie is a fast, simple and clean video downloader built with Go. Installation Prerequisites Install via go install Homebrew (macOS only) Arch Linu
Best Room Price Scraper from Booking.com
Best Room Price Scraper from Booking.com This repo is a tutorial of Large Scale
A cli scraper of gocomics.com made in go
goComic goComic is a cli tool written in go that scrapes your favorite childhood favorite comic from gocomics.com. It will give you a single days comi
Financial markets data library implemented in go.
finance-go Summary This go package aims to provide a go application with access to current and historical financial markets data in streamlined, well-
Scraper to download school attendance data from the DfE's statistics website
💪 Simple to use. Scrape attendance data with a single command! 🏇 Super fast. A
Golang scraper to get data from Google Play Store
google-play-scraper Golang scraper to get data from Google Play Store This project is inspired by the google-play-scraper node.js project Installation
Terminal interface to search the llvm C wrapper (llvm-c) API
Introduction This tool scrapes the LLVM-C API and (currently) prints all available API functions to stdout. It came to be from the lack of search opti
Webpage summary extractor using Facebook Open Graph and arc90's readability
goreadability goreadability is a tool for extracting the primary readable content of a webpage. It is a Go port of arc90's readability project, based
Port information web scraper written in Go.
Whatport is an open source tool that scrapes port information from SpeedGuide's Port Database Usage whatport [port(s)] (Seperate ports with a space)
Simple price scraper with HTTP server/exporter for use with Prometheus
priceserver v0.3 Simple price scraper with HTTP server/exporter for use with Prometheus Currently working with Bitrue.com exchange but easily adaptabl
Scrape the web in the eink era. Convert websites into books.
Scrape the web in the eink era. Convert websites into books.
An auto-scraper for EmulationStation written in Go using hashes.
An auto-scraper for EmulationStation written in Go using hashes. This currently works with NES, SNES, N64, GB, GBC, GBA, MD, SMS, 32X, GG, PCE, A2600, LNX, MAME/FBA(see below), Dreamcast(bin/gdi), PSX(bin/cue), ScummVM, SegaCD, WonderSwan, WonderSwan Color ROMs.
Golang based web site opengraph data scraper with caching
Snapper A Web microservice for capturing a website's OpenGraph data built in Golang Building Snapper building the binary git clone https://github.com/
scrapes codewars solution
Codewars Solution Scraper Currently the codewars api provides the completed katas, but not the actual solution code you wrote to complete it. This go
A crawler/scraper based on golang + colly, configurable via JSON
Super-Simple Scraper This a very thin layer on top of Colly which allows configuration from a JSON file. The output is JSONL which is ready to be impo
A crawler/scraper based on golang + colly, configurable via JSON
A crawler/scraper based on golang + colly, configurable via JSON
DorkScout - Golang tool to automate google dork scan against the entiere internet or specific targets
dorkscout dokrscout is a tool to automate the finding of vulnerable applications or secret files around the internet throught google searches, dorksco
DataHen Till is a standalone tool that instantly makes your existing web scraper scalable, maintainable, and more unblockable, with minimal code changes on your scraper.
DataHen Till is a standalone tool that instantly makes your existing web scraper scalable, maintainable, and more unblockable, with minimal code changes on your scraper.
Getting new films from baskino.me
Filmparser Description Every time I want to watch a film, I have to go to baskino.me to check if it is available on torrent trackers to download it th
Web Scraper in Go, similar to BeautifulSoup
soup Web Scraper in Go, similar to BeautifulSoup soup is a small web scraper package for Go, with its interface highly similar to that of BeautifulSou
Take a list of domains and scan for endpoints, secrets, api keys, file extensions, tokens and more...
Take a list of domains and scan for endpoints, secrets, api keys, file extensions, tokens and more... Coded with 💙 by edoardottt. Share on Twitter! P
Declarative web scraping
Ferret Try it! Docs CLI Test runner Web worker What is it? ferret is a web scraping system. It aims to simplify data extraction from the web for UI te
ant (alpha) is a web crawler for Go.
The package includes functions that can scan data from the page into your structs or slice of structs, this allows you to reduce the noise and complexity in your source-code.
Elegant Scraper and Crawler Framework for Golang
Colly Lightning Fast and Elegant Scraping Framework for Gophers Colly provides a clean interface to write any kind of crawler/scraper/spider. With Col
📱🥴 TikTok terminal client for browsing & downloading videos
tiktik 📱 🥴 - TikTok Terminal Client Name inspiration - this yt vid Huge Thanks to Egor who wrote the parsing part Installation 🔨 go get -u github.c
Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Geziyor Geziyor is a blazing fast web crawling and web scraping framework. It can be used to crawl websites and extract structured data from them. Gez
弹弹play 资源搜索节点 API 实现,基于 Cloudflare Workers 或 Golang 的两种实现。 || API implementations for "dandanplay" resource search service.
dandanplay-resource-service API implementations for "dandanplay" resource search service. 弹弹play 资源搜索节点的 API 实现。 提供基于 Cloudflare Workers 和 Golang 的两种实
Extract structured data from web sites. Web sites scraping.
Dataflow kit Dataflow kit ("DFK") is a Web Scraping framework for Gophers. It extracts data from web pages, following the specified CSS Selectors. You
Scrape the Twitter Frontend API without authentication with Golang.
Twitter Scraper Twitter's API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has it's own API, which I re
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Goribot 一个分布式友好的轻量的 Golang 爬虫框架。 完整文档 | Document !! Warning !! Goribot 已经被迁移到 Gospider|github.com/zhshch2002/gospider。修复了一些调度问题并分离了网络请求部分到另一个仓库。此仓库会继续
Extract structured data from web sites. Web sites scraping.
Dataflow kit Dataflow kit ("DFK") is a Web Scraping framework for Gophers. It extracts data from web pages, following the specified CSS Selectors. You
Elegant Scraper and Crawler Framework for Golang
Colly Lightning Fast and Elegant Scraping Framework for Gophers Colly provides a clean interface to write any kind of crawler/scraper/spider. With Col
A Devtools driver for web automation and scraping
Overview Documentation | API reference Rod is a high-level driver directly based on DevTools Protocol. It's designed for web automation and scraping.