docconv
A Go wrapper library to convert PDF, DOC, DOCX, XML, HTML, RTF, ODT, Pages documents and images (see optional dependencies below) to plain text.
Note for returning users: the Go import path for this package changed to
code.sajari.com/docconv.
Installation
If you haven't setup Go before, you first need to install Go.
To fetch and build the code:
$ go get code.sajari.com/docconv/...
This will also build the command line tool docd into $GOPATH/bin. Make sure that $GOPATH/bin is in your PATH environment variable.
Dependencies
tidy, wv, popplerutils, unrtf, https://github.com/JalfResi/justext
Example install of dependencies (not all systems):
$ sudo apt-get install poppler-utils wv unrtf tidy
$ go get github.com/JalfResi/justext
Optional dependencies
To add image support to the docconv library you first need to install and build gosseract.
Now you can add -tags ocr to any go command when building/fetching/testing docconv to include support for processing images:
$ go get -tags ocr code.sajari.com/docconv/...
This may complain on macOS, which you can fix by installing tesseract via brew:
$ brew install tesseract
docd tool
The docd tool runs as either:
-
a service on port 8888 (by default)
Documents can be sent as a multipart POST request and the plain text (body) and meta information are then returned as a JSON object.
-
a service exposed from within a Docker container
This also runs as a service, but from within a Docker container. Official images are published at https://hub.docker.com/r/sajari/docd.
Optionally you can build it yourself:
cd docd docker build -t docd . -
via the command line.
Documents can be sent as an argument, e.g.
$ docd -input document.pdf
Optional flags
addr- the bind address for the HTTP server, default is ":8888"log-level- 0: errors & critical info
- 1: inclues 0 and logs each request as well
- 2: include 1 and logs the response payloads
readability-length-low- sets the readability length low if the ?readability=1 parameter is setreadability-length-high- sets the readability length high if the ?readability=1 parameter is setreadability-stopwords-low- sets the readability stopwords low if the ?readability=1 parameter is setreadability-stopwords-high- sets the readability stopwords high if the ?readability=1 parameter is setreadability-max-link-density- sets the readability max link density if the ?readability=1 parameter is setreadability-max-heading-distance- sets the readability max heading distance if the ?readability=1 parameter is setreadability-use-classes- comma separated list of readability classes to use if the ?readability=1 parameter is set
How to start the service
$ # This will only log errors and critical info
$ docd -log-level 0
$ # This will run on port 8000 and log each request
$ docd -addr :8000 -log-level 1
Example usage (code)
Some basic code is shown below, but normally you would accept the file by HTTP or open it from the file system.
This should be enough to get you started though.
Use case 1: run locally
Note: this assumes you have the dependencies installed.
package main
import (
"fmt"
"log"
"code.sajari.com/docconv"
)
func main() {
res, err := docconv.ConvertPath("your-file.pdf")
if err != nil {
log.Fatal(err)
}
fmt.Println(res)
}
Use case 2: request over the network
package main
import (
"fmt"
"log"
"code.sajari.com/docconv/client"
)
func main() {
// Create a new client, using the default endpoint (localhost:8888)
c := client.New()
res, err := client.ConvertPath(c, "your-file.pdf")
if err != nil {
log.Fatal(err)
}
fmt.Println(res)
}
Alternatively, via a curl:
curl -s -F input=your-file.pdf http://localhost:8888/convert
