ssdt - Survey security.txt files

ssdt - Survey security.txt files

A program to quickly survey security.txt files found on the Alexa Top 1 Million websites. The program takes about 15 hours to run over a 1.5Mbit residential DSL connection. It could go much faster over high-speed links.

$ ps -p 165199 -o etime
  ELAPSED
  15:06:42

Sample output

{"website" ["contacts"] "expires"}
{"github.com" ["https://hackerone.com/github"] ""}
{"google.com" ["https://g.co/vulnz" "mailto:[email protected]"] ""}
{"facebook.com" ["https://www.facebook.com/whitehat/report/"] ""}
{"linkedin.com" ["mailto:[email protected]" "https://www.linkedin.com/help/linkedin/answer/62924"] ""}
{"cloudflare.com" ["https://hackerone.com/cloudflare" "mailto:[email protected]" "https://www.cloudflare.com/abuse/"] "sat, 20 mar 2021 13:24:05 -0700"}

Build the program

$ make

Run the program

$ ./ssdt -hosts top-1m-alexa.csv 2> err.txt > out.txt

Remove invalid security.txt entries

$ grep -v "\[\]" out.txt

Count results

$ grep -v "\[\]" out.txt | wc -l

Notes

  • You may need to adjust the nofile limit in /etc/security/limits.conf before running ssdt. Otherwise, you may exceed the open file limit.
  • Read my blog post about why I wrote this program.
Similar Resources

Hex dump and read values of files quickly and swiftly with Go-Hex a program designed to dump any file in a hexadecimal format

Go-Hex Hex dump and read values of files quickly and swiftly with Go-Hex a program designed to dump any file in a hexadecimal format Dump Hashes ----

Oct 10, 2021

🔍 A jq-like tool that queries files via glob.

🔍 fq A jq-like tool that queries files via glob. ✅ Prerequisites Go 1.17+ jq (installed and on PATH) 📦 Installation $ go get github.com/siketyan/fq

Dec 22, 2021

Package macho implements access to and creation of Mach-O object files.

go-macho [WIP] 🚧 Package macho implements access to and creation of Mach-O object files. Why 🤔 This package goes beyond the Go's debug/macho to: Cov

Sep 2, 2022

A utility library to do files/io/bytes processing/parsing in file-system or network.

goreader A utility library to do files/io/bytes processing/parsing in file-system or network. These features are really common to be implemented for a

Nov 1, 2021

Go-library that facilitates the usage of .env files

Goenv Golang-library that facilitates the use of .env files. Installation go get github.com/fabioelizandro/goenv Usage Place a .env file in the root

Nov 7, 2021

Backme - A backup files organizer

backme --- A backup files organizer Quite often big files (like database dumps)

Jan 26, 2022

Small tool for splitting files found in a path into multiple groups

Small tool for splitting files found in a path into multiple groups. Usefull for parallelisation of whatever can be paralleled with multiple files.

Jan 30, 2022

mesh-kridik is an open-source security scanner that performs various security checks on a Kubernetes cluster with istio service mesh and is leveraged by OPA (Open Policy Agent) to enforce security rules.

mesh-kridik is an open-source security scanner that performs various security checks on a Kubernetes cluster with istio service mesh and is leveraged by OPA (Open Policy Agent) to enforce security rules.

mesh-kridik Enhance your Kubernetes service mesh security !! mesh-kridik is an open-source security scanner that performs various security checks on a

Dec 14, 2022

This small Docker project is the easiest way to send notifications directly via .txt files to services like: Gotify, Telegram, SMTP (Email) or Webhook.

This small Docker project is the easiest way to send notifications directly via .txt files to services like: Gotify, Telegram, SMTP (Email) or Webhook.

This small Docker project is the easiest way to send notifications directly via .txt files to services like: Gotify, Telegram, SMTP (Email) or Webhook.

Oct 5, 2022

Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert

Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert

DeepAlert DeepAlert is a serverless framework for automatic response of security alert. Overview DeepAlert receives a security alert that is event of

Jan 3, 2023

Web-Security-Academy - Web Security Academy, developed in GO

Web-Security-Academy - Web Security Academy, developed in GO

Feb 23, 2022

grobotstxt is a native Go port of Google's robots.txt parser and matcher library.

grobotstxt grobotstxt is a native Go port of Google's robots.txt parser and matcher C++ library. Direct function-for-function conversion/port Preserve

Dec 27, 2022

A CoreDNS plugin to serve temporary TXT records for validation purposes (eg. Let's Encrypt DNS-01)

temptxt Name temptxt - serves TXT records for validation purposes (eg. ACME DNS-01 challenge) updated through a HTTP api. Description The temptxt plug

Aug 23, 2022

DNStxt-exp - 一个提供查询 TXT 记录的 DNS 服务利用工具。例如:可配合 Windows 下的 certutil 工具传输小文件(64KB)

DNStxt-exp 一个提供查询 TXT 记录的 DNS 服务利用工具。例如:可配合 Windows 下的 certutil 工具传输小文件(64KB) HE

Nov 9, 2022

Txt-lsp - A toy project with Language Server Protocol (LSP)

txt-lsp txt-lsp is a toy project where I play around with Language Server Protoc

Jan 22, 2022

Service that calls uzma24/project1 service, takes input from .txt file and prints JSON output returned from the service.

Service that calls uzma24/project1 service, takes input from .txt file and prints JSON output returned from the service. Program can take large input files.

Feb 6, 2022

GoLang ads.txt scraper

Collects and parses ads.txt GoLang program scrapes sites for ads.txt and stores its significant details to PostgreSQL database. Give it a file with CS

Feb 11, 2022

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Dec 29, 2022

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Jan 3, 2023
Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Jan 3, 2023
A golang library to validate and format swiss social security numbers

s3n is a golang library to validate and format swiss social security numbers (aka. AVS in french and AHV in german).

Nov 15, 2021
a tool for creating exploited media files for discord

Discord-Exploits A program for creating exploited media files for discord written in Go. Usage discord-exploits is a command line utility, meaning you

Dec 29, 2021
A tool to check problems about meta files of Unity
A tool to check problems about meta files of Unity

A tool to check problems about meta files of Unity on Git repositories, and also the tool can do limited autofix for meta files of auto-generated files.

Dec 22, 2022
libraries for various programming languages that make it easy to generate per-process trace files that can be loaded into chrome://tracing
libraries for various programming languages that make it easy to generate per-process trace files that can be loaded into chrome://tracing

chrometracing: chrome://tracing trace_event files The chrometracing directory contains libraries for various programming languages that make it easy t

Oct 6, 2022
A program for extracting information from chrome session files.

What A tool for dumping the contents of chrome session files. Why? Most tools for reading SNSS files (chrome session files) are either broken or outda

Nov 23, 2022
Small utility to allow simpler, quicker testing of parsing files in crowdsec

cs_parser_test Small utility to allow simpler, quicker testing of parsing files in crowdsec Usage $ sudo cs_parser_test -t syslog /var/log/mail.log N

Jul 13, 2021
a sharded store to hold large IPLD graphs efficiently, packaged as location-transparent attachable CAR files, with mechanical sympathy

DAG store This README will be populated soon. In the meantime, please refer to the design document.

Oct 31, 2022
LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.
LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.

LogAnalyzer Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory

May 31, 2022
Helm plugin to reference value files packaged in dependency charts

Helm Octopus Plugin This Helm plugin allows to reference packaged value files (other than the default values.yaml). Install helm plugin install https:

Sep 23, 2021