Go bindings for libmagic to detect MIME types

magicmime Build Status GoDoc

magicmime is a Go package which allows you to discover a file's mimetype by looking for magic numbers in its content. It could be used as a supplementary for Go's mime package which only interprets the file extension to detect mimetypes. Internally, it implements libmagic(3) bindings.

Tested on Linux and Mac OS X, should be working on BSD. You could be able to build and make it working with Cygwin on Windows.

Prerequisites

This uses the libmagic library from the file tool, so you might need to install the development package for libmagic. On Debian or Ubuntu: apt-get install libmagic-dev. On RHEL, CentOS or Fedora: yum install file-devel. On Mac OS X: brew install libmagic. If you don't have the required package, compilation will be terminated by an error saying magic.h cannot be found.

Usage

In order to start, go get this repository:

go get github.com/rakyll/magicmime

Examples

See godoc examples.

Docs

https://godoc.org/github.com/rakyll/magicmime

License

Copyright 2013 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
     http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Comments
  • Refactor to use Decoder struct

    Refactor to use Decoder struct

    All the main logic has been moved to functions on the Decoder struct and the package level functions merely construct and use a global detector that is protected by a mutex so that is is safe for concurrent use. This will solve #20

    NOTE: previously I had added TypeByFileHandle so that detection can be done on a *os.File, but this had troubles on linux, so I removed it.

  • Support synchronized access by multiple goroutines

    Support synchronized access by multiple goroutines

    The magicmime library uses a global variable to hold the file descriptor associated with the libmagic database. This is not safe for use across mutliple goroutines. This pull-request makes it possible to use a libmagic file-description in unsynchronized and synchronized modes.

    This pull-request changes the Open function API signature to add a parameter indicating the synchronization mode (SYNCHRONIZED or UNSYNCHRONIZED), and returns a pointer to a struct (type MimeDB) that keeps a reference to the libmagic file descriptor and mutex (if synchronized).

    The TypeByBuffer, TypeByFile, and Close functions are now receiver functions that operate on the new MimeDB struct type. If the Open function was called with SYNCHRONIZED SyncMode then these functions will use a mutex to coordinate calls to the libmagic library.

  • Fix test and Travis build

    Fix test and Travis build

    I solved the multiple definitions issue by changing the package of example_test.go to magicmime.

    After that I noticed TestMissingFile started panicking on Travis and saw that the mimetype DB was not set up. Interestingly, on OS X and the Linux vm I have tested it on, it doesn't panic.

    Maybe it should never panic and the version of Travis is outdated / panics, but I haven't had the time to verify it.

    The last commit adds Coveralls to your project, if you want it. If you decide to do it, first you have to add your project on Coveralls, then go to rakyll/magicmime Travis' settings and add environment variable COVERALLS_REPO_TOKEN with the token value you got on Coveralls. If you don't, just don't merge it :)

    This closes #17.

  • fatal error: 'magic.h' file not found

    fatal error: 'magic.h' file not found

    When I try to download this package using go get github.com/rakyll/magicmime I get an error saying

    # github.com/rakyll/magicmime
    ../../rakyll/magicmime/magicmime.go:28:11: fatal error: 'magic.h' file not found
     #include <magic.h>
              ^
    1 error generated.
    

    What could be the possible reason for this error?

  • CI: Travis build is failing, but why?

    CI: Travis build is failing, but why?

    Hey @rakyll,

    just wondering...

    have you figured out a way to make it work with Travis?

    I have tried with https://github.com/henvic/magicmime/commit/d917cf88fca4ef77448f3196c87db5039d27ad44

    but it doesn't work out https://travis-ci.org/henvic/magicmime/builds/83209266

    due to multiple definitions, but I don't get how it is happening. On my machine it works just fine.

  • Version tags

    Version tags

    Hello @rakyll !

    Is it possible to cut (semantic) version tags for magicmime? it would be easier to track it with dep.

    Thank you for you work on the project.

  • README.md: fix/improve prereq section

    README.md: fix/improve prereq section

    The changes are, in the order of importance:

    1. On CentOS, the package name is file-devel not libmagic-dev.
    2. As it was done for Mac OS X, add the actual commands to install needed package.
    3. Add Fedora and RHEL next to CentOS.
    4. Mention this comes from 'file' tool.
    5. It's 'package' not 'packages'.
  • cannot open; magic mime db is already open

    cannot open; magic mime db is already open

    Hi

    I get cannot open; magic mime db is already open after my vscode crashed during tests and it obviously didn't close the db. However I cannot get it working again. If I ignore the error that it's already open, when I run magicmime.Close() I get invalid memory address or nil pointer dereference.

    EDIT: this is still broken after rebooting

  • Open must Close if magic_setflags or magic_load fails

    Open must Close if magic_setflags or magic_load fails

    Currently, Open sequence doesn't handle the case where magic_setflags and magic_load fails. We need to close magic and free the underlying resources.

    Also, we must document that Close must be called once user is done with magic.

  • The example in example_test.go has memeroy leak

    The example in example_test.go has memeroy leak

    The example in example_test.go has memeroy leak.

    package main
    
    import (
        "fmt"
    
        "github.com/rakyll/magicmime"
    )
    
    func main() {
        mm, err := magicmime.New(magicmime.MAGIC_MIME_TYPE | magicmime.MAGIC_SYMLINK | magicmime.MAGIC_ERROR)
        if err != nil {
            panic(err)
        }
       defer mm.Close()
        filepath := "/bin/ls"
    
        mimetype, err := mm.TypeByFile(filepath)
        if err != nil {
            fmt.Printf("Something went wrong: %s", err)
            return
        }
    
        fmt.Printf("%s -> %s\n", filepath, mimetype)
    }
    

    Take special attention for defer mm.Close()

  • Error: build constraints exclude all Go files in /home/user/go/bin/pkg/mod/github.com/rakyll/magicmime@v0.1.0

    Error: build constraints exclude all Go files in /home/user/go/bin/pkg/mod/github.com/rakyll/[email protected]

    I am attempting to install Pistol, and I am running into an error involving magic mime.

    https://github.com/doronbehar/pistol/issues/76

    The error states:

    imports github.com/rakyll/magicmime: build constraints exclude all Go files in /home/user/go/bin/pkg/mod/github.com/rakyll/[email protected]
    

    This is a new one for both me and the developer of pistol, and after doing some digging, an issue was discovered in the golang repository that mentions a potential solution to the issue by ensuring package main is used to denote the main script of the package.

    https://github.com/golang/go/issues/24068#issuecomment-497928701

    I attempted the suggested resolution, which created another error, because there is no main function in the source code.

    So I am inquiring about a workaround that would allow me to use magicmime to install pistol.

  • Handle empty byte slices in TypeByBuffer

    Handle empty byte slices in TypeByBuffer

    Previously, passing an empty []byte paniced when trying to get a pointer to the first element. Since the slice is empty, that resulted in a index out of range runtime error.

  • Error in `./app': free(): invalid next size

    Error in `./app': free(): invalid next size

    *** Error in `./app': free(): invalid next size (fast): 0x00007f79400b5140 ***

    ======= Backtrace: =========
    /lib64/libc.so.6(+0x7c503)[0x7f7971af5503]
    /usr/lib64/libmagic.so.1(file_vprintf+0x6c)[0x7f797247330c]
    /usr/lib64/libmagic.so.1(file_printf+0x87)[0x7f79724733e7]
    /usr/lib64/libmagic.so.1(file_trycdf+0x4c7)[0x7f7972477157]
    /usr/lib64/libmagic.so.1(file_buffer+0x1c6)[0x7f7972473726]
    /usr/lib64/libmagic.so.1(magic_buffer+0x31)[0x7f7972467b61]
    ./app(_cgo_f7c060554b06_Cfunc_magic_buffer+0x1f)[0x8093df]
    ./app[0x459c60]
    

    I'm using magicmime with multiple goroutines but with a mutex around magicmime.TypeByBuffer(blob)

Powerful and versatile MIME sniffing package using pre-compiled glob patterns, magic number signatures, XML document namespaces, and tree magic for mounted volumes, generated from the XDG shared-mime-info database.

mimemagic Powerful and versatile MIME sniffing package using pre-compiled glob patterns, magic number signatures, xml document namespaces, and tree ma

Nov 3, 2022
Plugs module to see different types of plug types needed in different countries, and a comparison tool between two countries plug socket types.

plugs Importing the module: go get github.com/matthewboyd/plugs "github.com/matthewboyd/plugs" How to use the module: There are two functions wi

Dec 28, 2021
Library to work with MimeHeaders and another mime types. Library support wildcards and parameters.

Mime header Motivation This library created to help people to parse media type data, like headers, and store and match it. The main features of the li

Nov 9, 2022
Mimetype - This package provides mime-types as constants.

mimetype This package provides mimetypes as constants. The constants are generated for the IONA website: https://www.iana.org/assignments/media-types/

Dec 27, 2022
Simple interface to libmagic for Go Programming Language

File Magic in Go Introduction Provides simple interface to libmagic for Go Programming Language. Table of Contents Contributing Versioning Author Copy

Dec 22, 2021
Library providing opanapi3 and Go types for store/validation and transfer of ISO-4217, ISO-3166, and other types.

go-types This library has been created with the purpose to facilitate the store, validation, and transfer of Go ISO-3166/ISO-4217/timezones/emails/URL

Nov 9, 2022
Go linter that checks types that are json encoded - reports unsupported types and unnecessary error checks

Checks types passed to the json encoding functions. Reports unsupported types and reports occations, where the check for the returned error can be omited.

Oct 7, 2022
Optional is a library that provides option types for Go types.

Option Optional is a library that provides option types for Go types. Installation Golang version 1.18 + required go get -u github.com/eatmoreapple/op

Nov 9, 2022
An email MIME artist for golang

Marcel is a tool to generate IETF compliant emails in raw MIME format. I mainly use this for generating emails with attachments and sending them via amazon SES. If that's what you're doing too, you may want notifications

Nov 7, 2022
MIME mail encoding and decoding package for Go

enmime enmime is a MIME encoding and decoding library for Go, focused on generating and parsing MIME encoded emails. It is being developed in tandem w

Nov 30, 2022
Send markdown files as MIME-encoded electronic mail.

Send markdown files as MIME-encoded electronic mail.

Aug 9, 2022
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster

Kube-Knark Project Trace your kubernetes runtime !! Kube-Knark is an open source tracer uses pcap & ebpf technology to perform runtime tracing on a de

Sep 19, 2022
A Go program that takes an image, uses pigo to detect a face, and creates a gif that zooms in on the face
A Go program that takes an image, uses pigo to detect a face, and creates a gif that zooms in on the face

ok-zoomer Every GIF is a gift. How it works face detection with esimov/pigo color quantization / dithering with esimov/colorquant image resizing with

Nov 27, 2022
Aegis is a library that allows you detect if your software is being debugged or not on Linux, FreeBSD, NetBSD, OpenBSD and Windows
Aegis is a library that allows you detect if your software is being debugged or not on Linux, FreeBSD, NetBSD, OpenBSD and Windows

Aegis is a library that allows you detect if your software is being debugged or not on Linux, FreeBSD, NetBSD, OpenBSD and Windows. You can use it natively from C or use the Go bind.

Aug 29, 2022
A simple business indicator tool that uses a sliding window to detect whether the indicator exceeds the threshold

melon A simple business indicator tool that uses a sliding window to detect whether the indicator exceeds the threshold Usage //create the metric //th

Jul 11, 2021
a Go code to detect leaks in JS files via regex patterns

a Go code to detect leaks in JS files via regex patterns

Nov 13, 2022
A simple tool to detect WAN IP changes and updates your cloudflare DNS entries.

Cloudflare IP Updater A simple tool to detect WAN IP changes and updates your cloudflare DNS entries. Usage Create a new Cloudflare API token Copy the

Dec 1, 2022
Retnilnil is a static analysis tool to detect `return nil, nil`

retnilnil retnilnil is a static analysis tool for Golang that detects return nil, nil in functions with (*T, error) as the return type. func f() (*T,

Jun 9, 2022
Generate mega-workflows using Wappalyzer outputs and existing tech-detect

Usage Usage of ./build/generate-nuclei-templates: -clone-path string Path to clone Wappalyzer repository (default "./wappalyzer") -debug

Nov 9, 2022
go-ip-fraud-check has a feature to detect fraud from ip addresss.

go-ip-fraud-check go-ip-fraud-check has a feature to detect fraud from ip addresss. go-ip-fraud-check provides both of cli binary and golang API. Supp

Dec 16, 2022