An optimal, byte-aligned, LZ+RLE hybrid encoder, designed to maximize decoding speed on NMOS 6502 and derived CPUs

TSCrunch

TSCrunch is an optimal, byte-aligned, LZ+RLE hybrid encoder, designed to maximize decoding speed on NMOS 6502 and derived CPUs, while keeping decent compression performance (for a bytecruncher, that is). TSCrunch was designed as the default asset cruncher for the upcoming game A Pig Quest, and, as such, it's optimized for in-memory level compression, but at as of version 1.0 it can also create SFX executables for off-line prg crunching.

Requirements

TSCrunch requires python 3.x with scipy library installed, or a windows x64 machine(A pre-compiled windows x64 command line executable is also provided). A GO version is also supplied to facilitate building on your OS of choice. The memory decrunchers requires Kick Assembler, but it should be quite easy to port it to your assembler of choice.

Usage

tscrunch [option] infile outfile

Crunching examples:

tscrunch -x $0820 game.prg crunched.prg

Crunches the file game.prg and generates a self executable crunched.prg, using $0820 as post-decrunch jmp address

tscrunch -p game.prg crunched.bin

Mem-crunches the file game.prg, stripping the 2-byte header and generates a binary file crunched.bin

tscrunch data.bin crunched.bin

Mem-crunches the file data.bin and generates a binary file crunched.bin

tscrunch -i data.prg crunched.prg

Mem-crunches the file data.prg for in-place use, and generates a prg file crunched.prg with the appropriate load address

Please refer to the inline help (tscrunch -h) for a detailed description of the different crunching options. Note that with the exception of self executables and inplace, all the files generated by TSCrunch are headless binaries, that is they don't come with a 2 byte loader offset.

Decrunching files from code

For memory decrunching, please #include decrunch.asm and include the crunched binaries in your code, then use the macro TS_DECRUNCH, as explained by the following code fragment

	.pc = $1000 "test"
	//decrunches data to $4000
	:TS_DECRUNCH(compressed_data,$4000) 
	jmp *

	.align $100
	#include "decrunch.asm"
	
	compressed_data:
	.import binary "data.bin"

For inplace decrunching, please #define INPLACE before including the decruncher code, as explained by the following code fragment

	#define INPLACE

	.pc = $1000 "test"
	//decrunches data inplace
	:TS_DECRUNCH(compressed_data) 
	jmp *

	.align $100
	#include "decrunch.asm"
	
	.pc = LoadAddress //as provided by the cruncher
	compressed_data:
	.import c64 "data.bin"

decruncher.asm is the recommended decruncher for the general case, but other than it two alternative decrunchers are supplied: a small version, which saves some bytes at the cost of speed, and an extreme version which is generally marginally faster, but comes with a increased footprint.

Performance

TSCrunch is designed for ultra-fast decrunching while keeping a decent compression ratio. Being a byte-cruncher, it falls short of popular bit-crunchers, such as exomizer or B2, when comparing compression efficiency, but it is usually much faster at decoding. Furthermore, you can expect a 20% to 40% speed bump compared to popular byte-crunchers with similar compression efficiency. The following benchmark compares TSCrunch performance with those of a fast byte-cruncher, TinyCrunch, and a two fast bit-crunchers, B2 and dali, on a real-case compression scenario: Chopper Command, from the same author.

benchmark

Similar Resources

Easily and dynamically generate maps from Go static structures

structomap This package helps you to transform your struct into map easily. It provides a structomap.Serializer interface implemented by the structoma

Dec 9, 2022

Go package for dealing with maps, slices, JSON and other data.

Objx Objx - Go package for dealing with maps, slices, JSON and other data. Get started: Install Objx with one line of code, or update it with another

Dec 27, 2022

Encode and decode Go (golang) struct types via protocol buffers.

protostructure protostructure is a Go library for encoding and decoding a struct type over the wire. This library is useful when you want to send arbi

Nov 15, 2022

Simple, specialised, and efficient binary marshaling

🔌 surge Documentation A library for fast binary (un)marshaling. Designed to be used in Byzantine networks, 🔌 surge never explicitly panics, protects

Oct 4, 2022

Kitex byte-dance internal Golang microservice RPC framework with high performance and strong scalability, customized extensions for byte internal.

Kitex byte-dance internal Golang microservice RPC framework with high performance and strong scalability, customized extensions for byte internal.

Kitex 字节跳动内部的 Golang 微服务 RPC 框架,具有高性能、强可扩展的特点,针对字节内部做了定制扩展。

Jan 9, 2023

Wrap byte read options with uniform interface for io.Reader and byte slice

nibbler Nibble chunks from Reader streams and slice in a common way Overview This is a golang module that provides an interface for treating a Reader

Dec 23, 2021

Beautifully simple single author blog in Go - derived from Bear Blog.

Beautifully simple single author blog in Go - derived from Bear Blog.

Polar Bear Blog 🐻‍❄️ Lightweight blogging system for a single author. Written in Go and deploys to your own GCP project with a few commands. It's a d

Jan 6, 2023

Run your MapReduce workloads as a single binary on a single machine with multiple CPUs and high memory. Pricing of a lot of small machines vs heavy machines is the same on most cloud providers.

gomap Run your MapReduce workloads as a single binary on a single machine with multiple CPUs and high memory. Pricing of a lot of small machines vs he

Sep 16, 2022

Go library providing algorithms optimized to leverage the characteristics of modern CPUs

asm Go library providing algorithms optimized to leverage the characteristics of modern CPUs. Motivation With the development of Cloud technologies, a

Dec 29, 2022

Optimal implementation of ordered maps for Golang - ie maps that remember the order in which keys were inserted.

Goland Ordered Maps Same as regular maps, but also remembers the order in which keys were inserted, akin to Python's collections.OrderedDicts. It offe

Jan 3, 2023

Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm

Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm

go-orca Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm Disclaimer This project is under active development and i

Nov 22, 2022

Tidb - An open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads

Tidb - An open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads

What is TiDB? TiDB ("Ti" stands for Titanium) is an open-source NewSQL database

Jan 5, 2022

Hexa is the open-source, standards-based policy orchestration software for multi-cloud and hybrid businesses.

Hexa Policy Orchestrator Hexa is the open-source, standards-based policy orchestration software for multi-cloud and hybrid businesses. The Hexa projec

Dec 22, 2022

Arche - Smart Hybrid Workforce Manager: A system that aims to provide companies an easy to use platform for managing company resources by allowing employees to book company spaces and resources.

Arche - Smart Hybrid Workforce Manager: A system that aims to provide companies an easy to use platform for managing company resources by allowing employees to book company spaces and resources.

Description Smart Hybrid Workforce Manager is a system that aims to provide companies an easy to use system for managing company resources by allowing

Dec 8, 2022

A terminal designed for anyone to use and designed for any platform

A terminal designed for anyone to use and designed for any platform. Which includes the basic features of any terminal and includes friendly commands to perform tools such as ping, traceroute, generate key pairs, encrypt/decrypt, router security actions, etc. All of the source code is done in Go.

Jan 25, 2022

A component on PolarStack, a hybrid cloud Alibaba Cloud DBaaS product

A component on PolarStack, a hybrid cloud Alibaba Cloud DBaaS product

What is PolarDB Stack Daemon? PolarDB Stack Daemon is a component on PolarStack, a hybrid cloud Alibaba Cloud DBaaS product, and is designed mainly fo

Nov 21, 2021

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang, i.e. Cloudpods is a cloud on clouds. Cloudpods is able to manage not only on-premise KVM/baremetals, but also resources from many cloud accounts across many cloud providers. It hides the differences of underlying cloud providers and exposes one set of APIs that allow programatically interacting with these many clouds.

Jan 11, 2022

NYAGOS - The hybrid Commandline Shell between UNIX & DOS

NYAGOS - The hybrid Commandline Shell between UNIX & DOS

The Nihongo Yet Another GOing Shell English / Japanese NYAGOS is the commandline-shell written with the Programming Language GO and Lua. There are som

Dec 30, 2022
Comments
  • Separate TSCrunch into its own library and cli tools

    Separate TSCrunch into its own library and cli tools

    Hi Tony,

    Wanted to send you this PR with some improvements for your TSCrunch, without changing any of the logic or characteristics:

    • Separate TSCrunch into its own library and cli tools.
    • Refactor so that TSCrunch implements the io.WriterTo interface and accepts an io.Reader as input.
    • Added multitscrunch cli tool, for crunching many files in parallel.
    • Separated the boot binary into it's own boot.prg file for easy future updates.
    • Add go.mod and go.sum to support Go modules.

    To build, just run: go build ./cmd/tscrunch and you'll get an executable that works exactly the same as your current version. If people want to install from source, they can just do go install github.com/tonysavon/TSCrunch/cmd/tscrunch. I've tried to keep the changes as clean as possible. Hope you can merge it in :)

    regards, Enno

  • Bad error message

    Bad error message

    If you leave out the destination name, the parser picks the wrong "filename" part to complain about:

    python3 ~/c64/src/TSCrunch/tscrunch.py -x '$080d' hires.prg
    Traceback (most recent call last):
      File "/Users/jj/c64/src/TSCrunch/tscrunch.py", line 483, in <module>
        fr = open(sys.argv[-2],"rb")
    FileNotFoundError: [Errno 2] No such file or directory: '$080d'
    

    if I (correctly) set the output name:

    python3 ~/c64/src/TSCrunch/tscrunch.py -x '$080d' hires.prg hires.out
    populating LZ layer
    closing gaps
    

    Nice program though!

Some Golang types based on builtin. Implements interfaces Value / Scan and MarshalJSON / UnmarshalJSON for simple working with database NULL-values and Base64 encoding / decoding.

gotypes Some simple types based on builtin Golang types that implement interfaces for working with DB (Scan / Value) and JSON (Marshal / Unmarshal). N

Feb 12, 2022
Go library for decoding generic map values into native Go structures and vice versa.

mapstructure mapstructure is a Go library for decoding generic map values to structures and vice versa, while providing helpful error handling. This l

Jan 1, 2023
GED - Global-purpose Encoding / Decoding library

GED - Global-purpose Encoding / Decoding library This library lets you use common encoding/decoding schemes and allows you to define custom ones. Use

Nov 28, 2021
Fixed width file parser (encoder/decoder) in GO (golang)

Fixed width file parser (encoder/decoder) for GO (golang) This library is using to parse fixed-width table data like: Name Address

Sep 27, 2022
PHP session encoder/decoder written in Go

php_session_decoder PHP session encoder/decoder written in Go Installation Install: The recommended way to install is using gonuts.io: nut get yvasiya

Sep 27, 2022
Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

Version 1.0 vs 2.0 Update 2015 Sept 20: Big news! Version 2.0 of the go-bindings, authored by Ross Light, is now released and newly available! It feat

Nov 29, 2022
csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.
csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.

csvutil Package csvutil provides fast and idiomatic mapping between CSV and Go (golang) values. This package does not provide a CSV parser itself, it

Jan 6, 2023
Encode and decode binary message and file formats in Go

Encode and Decode Binary Formats in Go This module wraps the package encoding/binary of the Go standard library and provides the missing Marshal() and

Dec 22, 2022
Asn.1 BER and DER encoding library for golang.

WARNING This repo has been archived! NO further developement will be made in the foreseen future. asn1 -- import "github.com/PromonLogicalis/asn1" Pac

Nov 14, 2022
idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go]

go-codec This repository contains the go-codec library, the codecgen tool and benchmarks for comparing against other libraries. This is a High Perform

Dec 19, 2022