Read the text of memes, then inject that text into the image as searchable metadata.

Make Meme Text Searchable

I have an extensive set of memes I've been collecting since the early days of Flickr. #icanhascheeseburger

It's a pain in the ass to not be able to search my memes (now stored in Apple’s Photos.app) to find what I'm looking for when I need it.

I had something for this.

This project uses Amazon Rekognition to read the text from the images, then go-exif to write the text into the image metadata as a caption/description. Photo apps and services should be able to parse and index this data, making your images (memes, really) searchable by the text that's in the image.

General Program Flow

  1. Read the binary image data into memory.

  2. Rekognition only supports PNG and JPEG formats, so…

    1. If the image is already a PNG or JPEG, skip to the next step.

    2. If the image is a GIF, WEBP, or HEIF format, convert the in-memory representation of the image to JPEG format.

  3. Submit the (PNG or JPEG) bytes of the image to Rekognition.

  4. Get back the results. Merge, deduplicate, and munge the resulting text matches, whatever they are. Words don't always come back in the right order, so think of them less as a sentence and more of a collection of keywords.

    Wait a minute, I had something for this.

    …might become…

    a for had i minute something this wait

  5. Using the EXIF library, write these words into the file (the one we read into memory), into the ImageDescription EXIF field of the metadata.

  6. Optionally, we can:

    1. Overwrite the original file with an updated description (destructive).

    2. Read the file from one location, and write an updated copy to a new location (non-destructive).

      1. This new location can even be a different format, such as JPEG or PNG. Whatever Go's standard library supports.

Why Go?

Go (aka, "Golang") compiles down to a static binary, and is stupidly fast. It can also compile to WebAssembly, which means it can run in Node.js or web browsers. It also has the fastest boot time on AWS Lambda (more-or-less tied for first place with Rust), so creating a POST endpoint should be easy as well.

Someday, I want to learn how to develop mobile apps so that I can solve this user problem. Compiled Go code can be called from Android and iOS.

Progress

Library

  • Importable with go get.
  • Handles image bytestreams and decoding.
  • Handles converting the image to JPEG before passing to Rekognition.
  • Sends data to Rekognition.
  • Parses the results from Rekognition into words.
  • Converts the image bytestream to JPEG before sending to Rekognition.
  • Preserves any existing EXIF data.
  • Writes the words into the EXIF data.

CLI Tool

  • Supports reading a file.
  • Supports reading a directory.
  • Supports reading a glob.
  • Supports verbose logging.
  • Supports AWS credentials as environment variables.
  • Supports AWS credentials as a profile reference.
  • Supports -v.
  • Supports -vv and -vvv.
  • Supports -q.
  • Supports outputting a copy to a new directory.
  • Supports outputting a copy in a new format.
  • Supports writing the Rekognition results into EXIF data at all.
  • Supports status updates for jobs.
  • Supports an index of already-processed images to facilitate restarting a failed queue.

Usage

Library

Incomplete example. Error handling removed for brevity.

import "github.com/skyzyx/make-meme-text-searchable/meme"

func main() {
    // Open the file as am io.Reader.
    r, _ := os.Open("./images/paris-airport.heic")

    // Read the io.Reader, decode the image, then re-encode the image data as JPEG format.
    buf, _, _, _ := meme.ReadImage(r, meme.DefaultJPEGQuality)

    // Pass the image data to AWS Rekognition.
    results, _ := meme.DetectText(ctx, &awsConfig, buf)

    // Sanitize, de-dupe, remove punctuation, and sort the resulting words.
    words := meme.GetSanitizedText(results)

    // Write the string of words back to the image into the EXIF ImageDescription field.
    _ := meme.WriteImageDescription(r, words)
}

CLI

(Brainstorming) Something like…

meme-text [--report=TEXT|JSON] [--out=FILE] [--outdir=DIR] [--outformat=GIF|HEIC|JPG|PNG|WEBP] [--quiet] [--verbose] [--force] INPUT...
  • INPUT is one or more files, directories of files, or globs of files. Supports: GIF, HEIC, JPEG, PNG, WEBP. Also works with STDIN.
  • --report will write data to STDOUT in the specified format.
  • --quiet will silence all output.
  • --verbose maybe be specified up to 3 times, with increasing levels of verbosity. The default value is equivalent to WARNING. -v, -vv, and -vvv are equivalent to INFO, DEBUG, and TRACE (respectively).
  • --force disables any interactive prompts.

Web UI

This should be relatively simple to write as long as people can drag-and-drop/upload their images into the webpage, then provide an email address to send the results to (asychronously). A simple desktop app is also possible — maybe with Electron, Wails, or Tauri?

Things to read

Things I need to read and understand. Apparently writing EXIF data can be non-trivial.

Owner
Ryan Parman
Builder of digital things used by millions. Proud dad. Xennial. Gamer. Ambivert. Curious. Perpetual learner. Not a coffee drinker. Get vaccinated.
Ryan Parman
Similar Resources

Image - This repository holds supplementary Go image librariesThis repository holds supplementary Go image libraries

Go Images This repository holds supplementary Go image libraries. Download/Insta

Jan 5, 2022

k8s-image-swapper Mirror images into your own registry and swap image references automatically.

k8s-image-swapper Mirror images into your own registry and swap image references automatically.

k8s-image-swapper Mirror images into your own registry and swap image references automatically. k8s-image-swapper is a mutating webhook for Kubernetes

Dec 27, 2022

Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Dec 19, 2022

K8s_dns_chaos: enables inject DNS chaos in a Kubernetes cluster for Chaos Engineering

k8s_dns_chaos Name k8s_dns_chaos - enables inject DNS chaos in a Kubernetes cluster for Chaos Engineering. Description This plugin implements the Kube

Dec 12, 2021

Simple-read-file - Example of how to read file in Go

simple-read-file This repository contains a simple example of how to read file i

Jan 11, 2022

Litestream-read-replica-demo - A demo application for running live read replication on fly.io with Litestream

Litestream Read Replica Demo A demo application for running live read replicatio

Oct 18, 2022

You had one job, or more then one, which can be done in steps

Leprechaun Leprechaun is tool where you can schedule your recurring tasks to be performed over and over. In Leprechaun tasks are recipes, lets observe

Nov 23, 2022

QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enable you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to CSV file

Dec 22, 2021

Get any cryptocurrencies ticker and trade data in real time from multiple exchanges and then save it in multiple storage systems.

Get any cryptocurrencies ticker and trade data in real time from multiple exchanges and then save it in multiple storage systems.

Cryptogalaxy is an app which will get any cryptocurrencies ticker and trade data in real time from multiple exchanges and then saves it in multiple storage systems.

Jan 4, 2023

A web application attack surface mapping tool. It takes in a list of urls then performs numerous probes

sigurlscann3r A web application attack surface mapping tool. It takes in a list of urls then performs numerous probes Resources Features Installation

Sep 24, 2022

Simple application that waits for a given time and attempts and then exits

Wait Simple application that waits for a given time and attempts and then exits. WAIT_HOSTS is a list of hosts to wait for. e.g. WAIT_HOSTS=tcp://app:

Nov 24, 2021

A pay later service to allow our users to buy goods from a merchant now, and then allow them to pay for those goods at a later date.

simple-pay-later A pay later service to allow our users to buy goods from a merchant now, and then allow them to pay for those goods at a later date.

Dec 11, 2021

Peerconnection_explainer - PeerConnection-Explainer parses WebRTC Offers/Answers then provides summaries and suggestions

PeerConnection Explainer PeerConnection Explainer decodes WebRTC... so you don't have too! PeerConnection Explainer parses WebRTC Offers/Answers then

Oct 31, 2022

A simple javascript website that takes user input, queries a Go based backend which then creates ascii art and sends it back to the frontend

A simple javascript website that takes user input, queries a Go based backend which then creates ascii art and sends it back to the frontend. Finally the site displays the ascii art and offers the option to download as multiple file types.

Jan 7, 2022

Simple tool that updates Visual Studio Code workspace(s) to include Go modules in gopath/src, then launches VSCode if only one modified.

Simple tool that updates Visual Studio Code workspace(s) to include Go modules in gopath/src, then launches VSCode if only one modified.

Jan 27, 2022

Upgit - Upgit helps you simply upload any file to your Github repository and then get a raw URL for it

Upgit - Upgit helps you simply upload any file to your Github repository and then get a raw URL for it

Upgit - Upgit helps you simply upload any file to your Github repository and then get a raw URL for it

Dec 27, 2022

Muxer - This repo is designed to trancode RTMP streams from the ingester and then push them to be muxed

Muxer - This repo is designed to trancode RTMP streams from the ingester and then push them to be muxed

VidersMuxer This repo is designed to trancode RTMP streams from the ingester and

Feb 3, 2022

Moviefetch: a simple program to search and download for movies from websites like 1337x and then stream them

MovieFetch Disclaimer I am NOT responisble for any legal issues or other you enc

Dec 2, 2022
Hntoebook - Converts the best HN stories to .mobi format to be read using an e-reader
Hntoebook - Converts the best HN stories to .mobi format to be read using an e-reader

HN to E-Book What? This program converts the best HN stories to .mobi format to

Jan 2, 2023
💾 Wolke API is the API behind Wolke image storage and processing aswell as user management

?? Wolke API Wolke API is the API behind Wolke image storage and processing aswell as user management Deploying To deploy Wolke Bot you'll need podman

Dec 21, 2021
Dedugo - Simple duplicate image finder for golang

Dedugo Image Duplicate Finder De-duplicate in Go (get it?) Summary This simple p

Dec 27, 2022
Convert JPEG images from S3 bucket to BMP, GIF, PNG into another bucket
Convert JPEG images from S3 bucket to BMP, GIF, PNG into another bucket

aws-lambda Convert JPEG images from S3 bucket to BMP, GIF, PNG into another bucket Setup two buckets jpeg-images for source jpeg images converted-jpeg

Feb 13, 2022
Ydb-go-yc-metadata - Helpers to connect to YDB inside yandex-cloud using metadata service

ydb-go-yc-metadata helpers to connect to YDB inside yandex-cloud using metadata

Nov 28, 2022
View reddit memes and posts from ur terminal with golang and webscraping

goddit View reddit memes and posts from your terminal with golang and webscraping Installation run the following commands on your terminal to install

Feb 22, 2021
Lightweight go web server that provides a searchable directory index.

autoindex Lightweight go web server that provides a searchable directory index. Optimized for handling large numbers of files (100k+) and remote file

Jan 5, 2023
A k8s vault webhook is a Kubernetes webhook that can inject secrets into Kubernetes resources by connecting to multiple secret managers
A k8s vault webhook is a Kubernetes webhook that can inject secrets into Kubernetes resources by connecting to multiple secret managers

k8s-vault-webhook is a Kubernetes admission webhook which listen for the events related to Kubernetes resources for injecting secret directly from sec

Oct 15, 2022
Hotdog is a set of OCI hooks used to inject the Log4j Hot Patch into containers.

Hotdog Hotdog is a set of OCI hooks used to inject the Log4j Hot Patch into containers. How it works When runc sets up the container, it invokes hotdo

Nov 12, 2022
Script to inject Aliucord into a Discord ipa.

Aliucord-Patcher Script to patch Discord's ipa with a custom build of hermes, apply a custom icon and apply changes to Info.plist. Usage: go run cmds/

Jan 15, 2022