Go (Golang) GNU gettext utilities package

GitHub release MIT license Gotext build Go Report Card PkgGoDev

Gotext

GNU gettext utilities for Go.

Features

  • Implements GNU gettext support in native Go.
  • Complete support for PO files including:
  • Support for MO files.
  • Thread-safe: This package is safe for concurrent use across multiple goroutines.
  • It works with UTF-8 encoding as it's the default for Go language.
  • Unit tests available.
  • Language codes are automatically simplified from the form en_UK to en if the first isn't available.
  • Ready to use inside Go templates.
  • Objects are serializable to []byte to store them in cache.
  • Support for Go Modules.

License

MIT license

Documentation

Refer to the Godoc package documentation at (https://godoc.org/github.com/leonelquinteros/gotext)

Installation

go get github.com/leonelquinteros/gotext
  • There are no requirements or dependencies to use this package.
  • No need to install GNU gettext utilities (unless specific needs of CLI tools).
  • No need for environment variables. Some naming conventions are applied but not needed.

Version vendoring

Stable releases use semantic versioning tagging on this repository.

You can rely on this to use your preferred vendoring tool or to manually retrieve the corresponding release tag from the GitHub repository.

Vendoring with Go Modules (Recommended)

Add github.com/leonelquinteros/gotext inside the require section in your go.mod file.

i.e.

require (
    github.com/leonelquinteros/gotext v1.4.0
)

Vendoring with dep

To use last stable version (v1.4.0 at the moment of writing)

dep ensure -add github.com/leonelquinteros/[email protected]

Import as

import "github.com/leonelquinteros/gotext"

Vendoring with gopkg.in

http://gopkg.in/leonelquinteros/gotext.v1

To get the latest v1 package stable release, execute:

go get gopkg.in/leonelquinteros/gotext.v1

Import as

import "gopkg.in/leonelquinteros/gotext.v1"

Refer to it as gotext.

Locales directories structure

The package will assume a directories structure starting with a base path that will be provided to the package configuration or to object constructors depending on the use, but either will use the same convention to lookup inside the base path.

Inside the base directory where will be the language directories named using the language and country 2-letter codes (en_US, es_AR, ...). All package functions can lookup after the simplified version for each language in case the full code isn't present but the more general language code exists. So if the language set is en_UK, but there is no directory named after that code and there is a directory named en, all package functions will be able to resolve this generalization and provide translations for the more general library.

The language codes are assumed to be ISO 639-1 codes (2-letter codes). That said, most functions will work with any coding standard as long the directory name matches the language code set on the configuration.

Then, there can be a LC_MESSAGES containing all PO files or the PO files themselves. A library directory structure can look like:

/path/to/locales
/path/to/locales/en_US
/path/to/locales/en_US/LC_MESSAGES
/path/to/locales/en_US/LC_MESSAGES/default.po
/path/to/locales/en_US/LC_MESSAGES/extras.po
/path/to/locales/en_UK
/path/to/locales/en_UK/LC_MESSAGES
/path/to/locales/en_UK/LC_MESSAGES/default.po
/path/to/locales/en_UK/LC_MESSAGES/extras.po
/path/to/locales/en_AU
/path/to/locales/en_AU/LC_MESSAGES
/path/to/locales/en_AU/LC_MESSAGES/default.po
/path/to/locales/en_AU/LC_MESSAGES/extras.po
/path/to/locales/es
/path/to/locales/es/default.po
/path/to/locales/es/extras.po
/path/to/locales/es_ES
/path/to/locales/es_ES/default.po
/path/to/locales/es_ES/extras.po
/path/to/locales/fr
/path/to/locales/fr/default.po
/path/to/locales/fr/extras.po

And so on...

Usage examples

Using package for single language/domain settings

For quick/simple translations you can use the package level functions directly.

import (
    "fmt"
    "github.com/leonelquinteros/gotext"
)

func main() {
    // Configure package
    gotext.Configure("/path/to/locales/root/dir", "en_UK", "domain-name")

    // Translate text from default domain
    fmt.Println(gotext.Get("My text on 'domain-name' domain"))

    // Translate text from a different domain without reconfigure
    fmt.Println(gotext.GetD("domain2", "Another text on a different domain"))
}

Using dynamic variables on translations

All translation strings support dynamic variables to be inserted without translate. Use the fmt.Printf syntax (from Go's "fmt" package) to specify how to print the non-translated variable inside the translation string.

import (
    "fmt"
    "github.com/leonelquinteros/gotext"
)

func main() {
    // Configure package
    gotext.Configure("/path/to/locales/root/dir", "en_UK", "domain-name")

    // Set variables
    name := "John"

    // Translate text with variables
    fmt.Println(gotext.Get("Hi, my name is %s", name))
}

Using Locale object

When having multiple languages/domains/libraries at the same time, you can create Locale objects for each variation so you can handle each settings on their own.

import (
    "fmt"
    "github.com/leonelquinteros/gotext"
)

func main() {
    // Create Locale with library path and language code
    l := gotext.NewLocale("/path/to/locales/root/dir", "es_UY")

    // Load domain '/path/to/locales/root/dir/es_UY/default.po'
    l.AddDomain("default")

    // Translate text from default domain
    fmt.Println(l.Get("Translate this"))

    // Load different domain
    l.AddDomain("translations")

    // Translate text from domain
    fmt.Println(l.GetD("translations", "Translate this"))
}

This is also helpful for using inside templates (from the "text/template" package), where you can pass the Locale object to the template. If you set the Locale object as "Loc" in the template, then the template code would look like:

{{ .Loc.Get "Translate this" }}

Using the Po object to handle .po files and PO-formatted strings

For when you need to work with PO files and strings, you can directly use the Po object to parse it and access the translations in there in the same way.

import (
    "fmt"
    "github.com/leonelquinteros/gotext"
)

func main() {
    // Set PO content
    str := `
msgid "Translate this"
msgstr "Translated text"

msgid "Another string"
msgstr ""

msgid "One with var: %s"
msgstr "This one sets the var: %s"
`

    // Create Po object
    po := new(gotext.Po)
    po.Parse(str)

    fmt.Println(po.Get("Translate this"))
}

Use plural forms of translations

PO format supports defining one or more plural forms for the same translation. Relying on the PO file headers, a Plural-Forms formula can be set on the translation file as defined in (https://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/Plural-forms.html)

import (
    "fmt"
    "github.com/leonelquinteros/gotext"
)

func main() {
    // Set PO content
    str := `
msgid ""
msgstr ""

# Header below
"Plural-Forms: nplurals=2; plural=(n != 1);\n"

msgid "Translate this"
msgstr "Translated text"

msgid "Another string"
msgstr ""

msgid "One with var: %s"
msgid_plural "Several with vars: %s"
msgstr[0] "This one is the singular: %s"
msgstr[1] "This one is the plural: %s"
`

    // Create Po object
    po := new(gotext.Po)
    po.Parse(str)

    fmt.Println(po.GetN("One with var: %s", "Several with vars: %s", 54, v))
    // "This one is the plural: Variable"
}

Contribute

  • Please, contribute.
  • Use the package on your projects.
  • Report issues on Github.
  • Send pull requests for bugfixes and improvements.
  • Send proposals on Github issues.
Owner
Leonel Quinteros
This is some of my code. I have more, but isn't here.
Leonel Quinteros
Comments
  • Extract strings into po files

    Extract strings into po files

    Hey!

    I haven't used gettext in a while, and I was looking into a golang variant. Kind of too bad you can't do: _("the message") in golang, like you can everywhere else :/

    In any case, IIRC (it's been a while) gettext tooling gives you a way to extract all the TO-BE-TRANSLATED strings into the .po files. That way you know what's left to do, and you can update these as you add new strings...

    Do you have a script or command that does this? Maybe someone has done some quick golang lexer/parser/token hacking? Not sure how to proceed without it. Thanks!

  • Refactoring of gettext/locale constructs

    Refactoring of gettext/locale constructs

    Please describe your issue

    Is this a bug, an improvement, a proposal or something else?

    • [x] Improvement
    • [x] Proposal

    Unfortunately it is not so ideal for a multilingual web application if the files are reloaded for each request, but it should be possible to use a cache for each locale, and with an option (devel) and function to grant the possibility of rebuild.

    Therefore I consider it necessary to revise the modules.

    Since I notice that it is delivered with 50k simultaneous requests with some the wrong language. Rather, it should be possible to determine the locale for each request and access it in this request, instead of a global "memory".

    Regards, Josef

  • Create MO parser

    Create MO parser

    Refactored a bit too, so we can use interfaces to take Mo and Po files

    added fixtures

    found that the parser for Po files have a bug... but it works... so not touched See: https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html or maybe all "text Po Strings" are "defect"...

    What does this change implement?

    The MO file is now supported. For this, something had to be refactored so that everything could be implemented cleanly. I think that also makes sense as a basis for future work.

    Is this a fix or an improvement?

    • [X] Improvement

    Have discussed this change in an issue?

    • [X] Yes, fixes #2 and fixes #15

    Was some test failing because of this issue or change needed?

    • [X] No

    Are you including tests to cover this change?

    • [X] Yes
  • Do not use Sprintf if no vars are given

    Do not use Sprintf if no vars are given

    I use source sentences that contain named placeholders. For example: Hello %username% When using fmt.Sprintf this wil result in Hello %!u(MISSING)sername%!(NOVERB)

    My changes make sure that if no translation is found and no vars are given fmt.Sprintf is not used.

  • call Translation.Get & GetC directly

    call Translation.Get & GetC directly

    instead of calling Translation.GetN with a default of plural=1

    • add correct Arabic plural rules to test

    Before creating your Pull Request...

    • New Pull Requests should include a good description of what's being merged.
    • Ideally, all Pull Requests are preceded by a discussion initiated in an Issue on this repository.
    • For bug fixes is mandatory to have tests that cover and fail when the bug is present and will pass after this Pull Request.
    • For changes and improvements, new tests have to be provided to cover the new features.

    Is this a fix, improvement or something else?

    Fix (https://github.com/leonelquinteros/gotext/issues/39) ...

    What does this change implement/fix?

    Adding the correct Pluralform rules in the existing Arabic test case caused it to fail in a way that's consistent with https://github.com/leonelquinteros/gotext/issues/39 The code changes provided allowed the test to pass ...

    I have ...

    • [x] answered the 2 questions above,
    • [x] discussed this change in an issue,
    • [x] included tests to cover this changes.
  • LocaleGet use globalConfig instead config from receiver

    LocaleGet use globalConfig instead config from receiver

    Is this a bug, an improvement, a proposal or something else?

    • [x] Bug
    • [ ] Improvement
    • [ ] Proposal
    • [ ] Something else

    Briefly explain your issue

    The variable created by NewLocale(...) when querying a string through Get(str, vars...) for some reason refers to the DOM from the globalConfig, not from the configuration of the variable. As a consequence, all logic breaks.

    What's the expected behaviour?

    I add some Domain via myLocale.AddDomain("hello") and myLocale.Get(str, vars...) must use content from "hello" domain in my local variable.

    What's the actual behaviour?

    myLocale.Get(str, vars...) call globalConfig for content from my "hello" domain, but found nothing and use "default".

    Comments

    See next parts: https://github.com/leonelquinteros/gotext/blob/92b69ffa4cc0895b83b42f50065e9de5a3c32960/locale.go#L90-L104

    (See other Get... functions which use l.domains) https://github.com/leonelquinteros/gotext/blob/92b69ffa4cc0895b83b42f50065e9de5a3c32960/locale.go#L108-L110

    https://github.com/leonelquinteros/gotext/blob/92b69ffa4cc0895b83b42f50065e9de5a3c32960/gotext.go#L76-L82

  • Parse header in PO file is no longer compliant with GNU PO format

    Parse header in PO file is no longer compliant with GNU PO format

    It seems after this commit, the PO header requires a different format than https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html.

    An empty untranslated-string is reserved to contain the header entry with the meta information (see Header Entry). This header entry should be the first entry of the file. The empty untranslated-string is reserved for this purpose and must not be used anywhere else.

    A compliant header looks like:

    msgid   ""
    msgstr  "Project-Id-Version: %s\n"
            "Report-Msgid-Bugs-To: %s\n"
    
  • Format quotation marks and newlines for PO export

    Format quotation marks and newlines for PO export

    Fixes https://github.com/leonelquinteros/gotext/issues/60.

    As described in the issue, this:

    	po := gotext.NewPo()
    	strings := map[string]string{"myid": "test string\nwith \"newline\""}
    	for k, v := range strings {
    		po.Set(k, v)
    	}
    	po_bytes, _ := po.MarshalText()
    	fmt.Println(string(po_bytes))
    

    currently outputs this:

    msgid ""
    msgstr ""
    
    msgid "myid"
    msgstr "test string
    with "newline""
    

    This breaks format and causes xgettext to throw errors if the file is used. After this PR, the output is:

    [...]
    msgid "myid"
    msgstr "test string"
    "with \"newline\""
    

    My apologies for not adding tests. Please feel free to edit/push to this branch.

  • PO(T) Export, for message extraction

    PO(T) Export, for message extraction

    Permits the writing of message extractors, using gotext. With round-tripping support to update existing translations (while preserving contextual details like copyright notices & source references).

  • Issue with Arabic localization not returning any strings

    Issue with Arabic localization not returning any strings

    Please describe your issue

    First off, thanks for all the work on this project. It's exactly what I need for my use case.

    I get no string returned when localizing Arabic. I've tracked it down to the complex Pluralization rules of Arabic. In this case, I'm not trying to use plurals.

    Is this a bug, an improvement, a proposal or something else? Describe it.

    Bug ..

    What's the expected behaviour, the current behaviour and the steps to reproduce it?

    I've created a sample project that shows the behavior: https://github.com/chrisvaughn/ar_gotext The key code being

    enLocale := gotext.NewLocale("locale", "en_US")
    enLocale.AddDomain("messages")
    
    arLocale := gotext.NewLocale("locale", "ar_SA")
    arLocale.AddDomain("messages")
    
    po := new(gotext.Po)
    po.ParseFile("locale/ar_SA/LC_MESSAGES/messages.po")
    
    fmt.Printf("English With Just a Get: %s\n", enLocale.Get("This is a sample string"))
    fmt.Printf("Arabic With Just a Get: %s\n", arLocale.Get("This is a sample string"))
    fmt.Printf("Arabic Plural To 0: %s\n", arLocale.GetN("This is a sample string", "This is a sample string", 0))
    fmt.Printf("Arabic Using po.Get: %s\n", po.Get("This is a sample string"))
    

    the output of running this code is

    English With Just a Get: This is a sample string
    Arabic With Just a Get: 
    Arabic Plural To 0: هذه سلسلة عينة
    Arabic Using po.Get: هذه سلسلة عينة
    

    I would expect arLocale.Get("This is a sample string") and po.Get("This is a sample string") to return the same thing in this case. ...

    Comments

    I don't think it's safe to assume you can always call GetND under the hood with a default plural of 1. I'm happy to submit an MR to attempt to address this.

    I see that there is a test for Arabic from a previous issue but in that test the plural rules used are "Plural-Forms: nplurals=2; plural=(n != 1);\n" the actual plural rules for Arabic are "Plural-Forms: nplurals=6; plural=(n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5);\n"

  • Doesn't translate to Arabic

    Doesn't translate to Arabic

    The package can not work with Arabic translations:

    My ar/categories.po translation:

    ...
    msgid "Alcohol & Tobacco"
    msgstr "الكحول والتبغ"
    ...
    

    Go example:

    locale := gotext.NewLocale("i18n/", "ar")
    locale.AddDomain("categories")
    log.Println(locale.GetD("categories", "Alcohol & Tobacco", nil))
    

    Output:

    2018/08/14 17:33:18 %!(EXTRA <nil>)
    
  • Keep order in po files

    Keep order in po files

    I'm using the library to apply some changes to some existing PO files (from some open source projects), but due to how the library keeps the PO file internally, it's pretty hard to be able to use it:

    When "writing back" the po file to text, it reorders all translations, which makes it impractical to later submit an MR to the upstream project.

    This simple code (with input and output files attached) demonstrates the problem.

    func main() {
    	inPo := filepath.Join(projectpath.Root, "./in.po")
    	outPo := filepath.Join(projectpath.Root, "./out.po")
    
    	po := gotext.NewPo()
    	po.ParseFile(inPo)
    
    	data, _ := po.MarshalText()
    
    	os.WriteFile(outPo, data, 0644)
    }
    

    in.po.txt out.po.txt

    Is there any way to "reconstruct" the pofile to how it was before parsing it?

  • Get and similar functions don't always process formatting sequences

    Get and similar functions don't always process formatting sequences

    Please describe your issue

    Is this a bug, an improvement, a proposal or something else? Describe it.

    This is a bug.

    The Printf function used to format strings doesn't always invoke fmt.Sprintf. As a result, the expansion of %% sequences differs if there are arguments or not.

    What's the expected behaviour, the current behaviour and the steps to reproduce it?

    I expect that I can create a locale object, tr, and then do this:

    a := tr.Get("My string with an escaped value: %%s")
    fmt.Printf(a, "text")
    

    And that it prints My string with an escaped value: text. I happen to need this because I have a map of integers to format strings, where the format strings contain %% sequences.

    Unfortunately, the behavior is inconsistent, so things are only passed through fmt.Sprintf if there are arguments. This makes the behavior hard to reason about.

    Comments

    If there were separate functions to read strings that were formatted and ones that were not, this wouldn't be a problem.

  • Project status and feature request to use embed po files

    Project status and feature request to use embed po files

    Hi !

    I find your lib very attractive since it implements gettext in native Go and supports .po files, that are not the case for the other libs.

    The wip extractor tool in CLI seems to be very useful although it's not include in v1.4.0.

    I wonder if the project is still maintain. Indeed the last stable version was released on 2018 whereas some cool features (such as the cli) are in the pipes.

    Furthermore, it would be great to accept a File instead of a raw path to point a .po file in Configure. It will offer the possibility to embed a po file with the new embed api (since Go 1.16).

    I'm open to contribute and send PR if necessary.

  • Feature request - ways to detect missing translations at runtime

    Feature request - ways to detect missing translations at runtime

    Currently, a translation that doesn't exist just defaults to the passed message ID.

    It can be helpful to be able to catch these missing cases e.g. to save to a log, so it might be helpful to have variants of the functions that return a boolean "ok" or an error or something

    Here's a PR #41 that tries to implement this.

    What do you think of this approach?

  • gotext cli tool should mimic xgettext cli

    gotext cli tool should mimic xgettext cli

    I know there is a lot to implement if one wants to copy the complete xgettext cli but I would at least implement the behaviour with reading in files. I.e. that the last argument is a filelist to be read in. I already made a fork and would implement that there. This would allow a tool like poedit to automatically extract strings from code.

  • Write formal documentation

    Write formal documentation

    Please describe your issue

    The only documentation for this package is Godoc and the README.md file from this repo. We need to write better docs, some tutorial and use case examples to make package adoption easier.

    Is this a bug, an improvement, a proposal or something else? Describe it.

    Improvement

    What's the expected behaviour, the current behaviour and the steps to reproduce it?

    We don't have documentation. We want to have documentation

    Comments

    Decide between a Github page or the Wiki for this.

:ab: GNU gettext for Go (Imported By Kubernetes)

Go语言QQ群: 102319854, 1055927514 凹语言(凹读音“Wa”)(The Wa Programming Language): https://github.com/wa-lang/wa gettext-go: GNU gettext for Go (Imported By Ku

Nov 12, 2022
String i18n utilities for the Go Programming Language

About polyglot polyglot is a String translation package and tool for Go. Setup Make sure you have a working Go installation. See Getting Started Now r

Dec 22, 2022
A golang package to work with Decentralized Identifiers (DIDs)

did did is a Go package that provides tools to work with Decentralized Identifiers (DIDs). Install go get github.com/ockam-network/did Example packag

Nov 25, 2022
htmlquery is golang XPath package for HTML query.

htmlquery Overview htmlquery is an XPath query package for HTML, lets you extract data or evaluate from HTML documents by an XPath expression. htmlque

Jan 4, 2023
Package sanitize provides functions for sanitizing text in golang strings.

sanitize Package sanitize provides functions to sanitize html and paths with go (golang). FUNCTIONS sanitize.Accents(s string) string Accents replaces

Dec 5, 2022
A minimalistic emoji package for Go (golang)

emoji ?? ?? ?? emoji is a minimalistic emoji library for Go. It lets you use emoji characters in strings. Inspired by spatie/emoji Install ?? go get g

Dec 14, 2022
xmlquery is Golang XPath package for XML query.

xmlquery Overview xmlquery is an XPath query package for XML documents, allowing you to extract data or evaluate from XML documents with an XPath expr

Jan 1, 2023
Frongo is a Golang package to create HTML/CSS components using only the Go language.

Frongo Frongo is a Go tool to make HTML/CSS document out of Golang code. It was designed with readability and usability in mind, so HTML objects are c

Jul 29, 2021
This package provides Go (golang) types and helper functions to do some basic but useful things with mxGraph diagrams in XML, which is most famously used by app.diagrams.net, the new name of draw.io.

Go Draw - Golang MX This package provides types and helper functions to do some basic but useful things with mxGraph diagrams in XML, which is most fa

Aug 30, 2022
Genex package for Go

genex Genex package for Go Easy and efficient package to expand any given regex into all the possible strings that it can match. This is the code that

Nov 2, 2022
A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library

goq Example import ( "log" "net/http" "astuart.co/goq" ) // Structured representation for github file name table type example struct { Title str

Dec 12, 2022
csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

csvplus Package csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream processing operations, indices and joins. The

Apr 9, 2022
[Go] Package of validators and sanitizers for strings, numerics, slices and structs

govalidator A package of validators and sanitizers for strings, structs and collections. Based on validator.js. Installation Make sure that Go is inst

Dec 28, 2022
Package strit introduces a new type of string iterator, along with a number of iterator constructors, wrappers and combinators.

strit Package strit (STRing ITerator) assists in development of string processing pipelines by providing a simple iteration model that allows for easy

Jun 21, 2022
A markdown renderer package for the terminal
A markdown renderer package for the terminal

go-term-markdown go-term-markdown is a go package implementing a Markdown renderer for the terminal. Note: Markdown being originally designed to rende

Nov 25, 2022
Go package for syntax highlighting of code

syntaxhighlight Package syntaxhighlight provides syntax highlighting for code. It currently uses a language-independent lexer and performs decently on

Nov 18, 2022
ByNom is a Go package for parsing byte sequences, suitable for parsing text and binary data

ByNom is a Go package for parsing byte sequences. Its goal is to provide tools to build safe byte parsers without compromising the speed or memo

May 5, 2021
Package i18n is a middleware that provides internationalization and localization for Flamego
Package i18n is a middleware that provides internationalization and localization for Flamego

i18n Package i18n is a middleware that provides internationalization and localization for Flamego. Installation The minimum requirement of Go is 1.16.

Dec 14, 2022
A dead simple parser package for Go
A dead simple parser package for Go

A dead simple parser package for Go V2 Introduction Tutorial Tag syntax Overview Grammar syntax Capturing Capturing boolean value Streaming Lexing Sta

Dec 30, 2022