Split text files into gzip files with x lines

hakgzsplit

split lines of text into multiple gzip files

Install/update

go get -u github.com/hakluke/hakgzsplit

Usage

The command below will take 10mhostnames.txt, which contains 10 million hostnames separated by lines, and split it into 10 gzip files called hosts1.gz, hosts2.gz, etc.

cat 10mhostnames.txt | hakgzsplit -b 1000000 -f hosts
Owner
Luke Stephens (hakluke)
Internet Ninja.
Luke Stephens (hakluke)
Similar Resources

Easily create Go files from stub files

go-stubs Easily create .go files from stub files in your projects. Usage go get github.com/nwby/go-stubs Create a stub file: package stubs type {{.Mo

Jan 27, 2022

app-services-go-linter plugin analyze source tree of Go files and validates the availability of i18n strings in *.toml files

app-services-go-linter app-services-go-linter plugin analyze source tree of Go files and validates the availability of i18n strings in *.toml files. A

Nov 29, 2021

Compute message digest for large files in Go

checksum test coverage Compute message digest, like MD5 and SHA256, in golang for potentially large files. Usage package main import ( "fmt" "githu

Dec 28, 2022

copy files for humans

Go-Decent-Copy go-decent-copy provides a copy file for humans Usage package main import "github.com/hugocarreira/go-decent-copy" func main() { e

Sep 26, 2022

Golang wrapper for Exiftool : extract as much metadata as possible (EXIF, ...) from files (pictures, pdf, office documents, ...)

go-exiftool go-exiftool is a golang library that wraps ExifTool. ExifTool's purpose is to extract as much metadata as possible (EXIF, IPTC, XMP, GPS,

Dec 28, 2022

Load GTFS files in golang

go-gtfs Load GTFS files in Go. The project is in maintenance mode. It is kept compatible with changes in the Go ecosystem but no new features will be

Dec 5, 2022

An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Sep 26, 2022

QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enable you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to CSV file

Dec 22, 2021

A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Oct 14, 2022
Embed files into a Go executable

statik statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem. Is this a crazy idea? No

Jan 6, 2023
A tool for moving files into directories by file extensions
A tool for moving files into directories by file extensions

The tool for moving files into directories by file extensions Example before moving structure: moving into same extension dir result: moving into diff

Dec 6, 2021
Allows parsing CSV files into custom structs and implements required fields that can't be empty

Welcome to Go Custom CSV Parser ?? Allows parsing CSV files into custom structs and implements required fields that can't be empty ?? Homepage Install

Nov 9, 2021
A PDF document generator with high level support for text, drawing and images

GoFPDF document generator Package go-pdf/fpdf implements a PDF document generator with high level support for text, drawing and images. Features UTF-8

Jan 4, 2023
A solution to convert PDFs into audiobooks (offline).

A solution to convert PDFs into audiobooks (offline). This solution kit consists of an Opentts engine hosted on docker and a CLI client that parses the given PDF file for text content & connects with Opentts to generate audio files.

Dec 28, 2022
Embed arbitrary resources into a go executable at runtime, after the executable has been built.

ember Ember is a lightweight library and tool for embedding arbitrary resources into a go executable at runtime. The resources don't need to exist at

Nov 9, 2022
Simple go script that converts csv file into a json document

csv-go-parser Simple go script that converts csv file into a json document. CSV Input: id,first_name,last_name,email,avatar,ip_address 1,Pauline,Hirth

Jun 4, 2022
Ghostinthepdf - This is a small tool that helps to embed a PostScript file into a PDF

This is a small tool that helps to embed a PostScript file into a PDF in a way that GhostScript will run the PostScript code during the

Dec 20, 2022
RtxTest - Extract this zip file into your golang development environment

Documentation 1. Clone or extract file extract this zip file into your golang de

May 12, 2022