Natural language detection package in pure Go

getlang

GoDoc Go Report Card Build Status cover.run go

getlang provides fast natural language detection in Go.

Features

  • Offline -- no internet connection required
  • Supports 29 languages
  • Provides ISO 639 language codes
  • Fast

Getting started

Installation:

    go get -u github.com/rylans/getlang

example:

package main

import (
	"fmt"
	"github.com/rylans/getlang"
)

func main(){
  info := getlang.FromString("Wszyscy ludzie rodzą się wolni i równi w swojej godności i prawach")
  fmt.Println(info.LanguageCode(), info.Confidence())
}

Documentation

getlang on godoc

License

MIT

Acknowledgements and Citations

  • Thanks to abadojack for the trigram generation logic in whatlanggo
  • Cavnar, William B., and John M. Trenkle. "N-gram-based text categorization." Ann arbor mi 48113.2 (1994): 161-175.
Comments
  • Looking for the text case explaination or documentation.

    Looking for the text case explaination or documentation.

    I added Punjabi language support to this package and run on my local machine. When I submitted the pull request there was a build failed indicated by Tarvis. I want to contribute to this Repo and want to fix the error of my code. If I could get any documentation or guidance it will help me a lot. Thank you Bharat Sewani

  • Adding Persian language detection

    Adding Persian language detection

    Also Arabic detection mechanism has been changed, because Arabic and Persian script is the same, so I had added Arabic profile as well in the profiles.go file

  • Add support for CJK languages

    Add support for CJK languages

    Support language detection for Chinese, Japanese and Korean.

    The 3gram character approach will not work for these languages. Consider detecting these languages based on which part of unicode the CJK runes map to.

  • czech language support, introduced dynamic weight for unicode blocks

    czech language support, introduced dynamic weight for unicode blocks

    Hello, I've added support for a czech language as well as support for dynamic weight for scripts (scriptCountFactor).

    The reason for that is that in a czech alphabet, there are few characters like ř, š, ů which you won't find in any other language.

    I hope you'll like my take on that.

  • Add language detection for Kannada

    Add language detection for Kannada

    Can add support for this language by its script. Add this to the scripts variable (defined on line 42 of getlang.go) "kn": []*unicode.RangeTable{unicode.Kannada},

Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

Dec 25, 2022
Self-contained Machine Learning and Natural Language Processing library in Go

If you like the project, please ★ star this repository to show your support! ?? A Machine Learning library written in pure Go designed to support rele

Dec 30, 2022
A natural language date/time parser with pluggable rules

when when is a natural language date/time parser with pluggable rules and merge strategies Examples tonight at 11:10 pm at Friday afternoon the deadli

Dec 26, 2022
Cross platform locale detection for Golang

go-locale go-locale is a Golang lib for cross platform locale detection. OS Support Support all OS that Golang supported, except android: aix: IBM AIX

Aug 20, 2022
Self-contained Japanese Morphological Analyzer written in pure Go
Self-contained Japanese Morphological Analyzer written in pure Go

Kagome v2 Kagome is an open source Japanese morphological analyzer written in pure golang. The dictionary/statistical models such as MeCab-IPADIC, Uni

Dec 24, 2022
Stemmer packages for Go programming language. Includes English, German and Dutch stemmers.

Stemmer package for Go Stemmer package provides an interface for stemmers and includes English, German and Dutch stemmers as sub-packages: porter2 sub

Dec 14, 2022
Gopher-translator - A HTTP API that accepts english word or sentences and translates them to Gopher language

Gopher Translator Service An interview assignment project. To see the full assig

Jan 25, 2022
Complete Translation - translate a document to another language
 Complete Translation - translate a document to another language

Complete Translation This project is to translate a document to another language. The initial target is English to Korean. Consider this project is no

Feb 25, 2022
The shamoji (杓文字) is a word filtering package

shamoji About The shamoji (杓文字) is word filtering package. Install $ go get -u github.com/osamingo/shamoji Usage package main import ( "fmt" "sync

Sep 27, 2022
A Go package for n-gram based text categorization, with support for utf-8 and raw text

A Go package for n-gram based text categorization, with support for utf-8 and raw text. To do: write documentation make it faster Keywords: text categ

Nov 28, 2022
Package i18n provides internationalization and localization for your Go applications.

i18n Package i18n provides internationalization and localization for your Go applications. Installation The minimum requirement of Go is 1.16. go get

Nov 9, 2022
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.

aScan A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection. Fu

Aug 15, 2022
Fast face detection, pupil/eyes localization and facial landmark points detection library in pure Go.
Fast face detection, pupil/eyes localization and facial landmark points detection library in pure Go.

Pigo is a pure Go face detection, pupil/eyes localization and facial landmark points detection library based on Pixel Intensity Comparison-based Objec

Dec 24, 2022
Natural language detection library for Go
Natural language detection library for Go

Whatlanggo Natural language detection for Go. Features Supports 84 languages 100% written in Go No external dependencies Fast Recognizes not only a la

Dec 28, 2022
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

?? The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

Dec 25, 2022
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

Its task is simple: It tells you which language some provided textual data is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages.

Dec 29, 2022
Natural-deploy - A natural and simple way to deploy workloads or anything on other machines.

Natural Deploy Its Go way of doing Ansibles: Motivation: Have you ever felt when using ansible or any declarative type of program that is used for dep

Jan 3, 2022
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Dec 30, 2022
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

Dec 25, 2022
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

Dec 25, 2022