Go-linq - A powerful language integrated query (LINQ) library for Golang

go-linq GoDoc Build Status Coverage Status Go Report Card

A powerful language integrated query (LINQ) library for Go.

  • Written in vanilla Go, no dependencies!
  • Complete lazy evaluation with iterator pattern
  • Safe for concurrent use
  • Supports generic functions to make your code cleaner and free of type assertions
  • Supports arrays, slices, maps, strings, channels and custom collections

Installation

When used with Go modules, use the following import path:

go get github.com/ahmetb/go-linq/v3

Older versions of Go using different dependency management tools can use the following import path to prevent breaking API changes:

go get gopkg.in/ahmetb/go-linq.v3

Quickstart

Usage is as easy as chaining methods like:

From(slice) .Where(predicate) .Select(selector) .Union(data)

Example 1: Find all owners of cars manufactured after 2015

import . "github.com/ahmetb/go-linq/v3"

type Car struct {
    year int
    owner, model string
}

...


var owners []string

From(cars).Where(func(c interface{}) bool {
	return c.(Car).year >= 2015
}).Select(func(c interface{}) interface{} {
	return c.(Car).owner
}).ToSlice(&owners)

Or, you can use generic functions, like WhereT and SelectT to simplify your code (at a performance penalty):

var owners []string

From(cars).WhereT(func(c Car) bool {
	return c.year >= 2015
}).SelectT(func(c Car) string {
	return c.owner
}).ToSlice(&owners)

Example 2: Find the author who has written the most books

import . "github.com/ahmetb/go-linq/v3"

type Book struct {
	id      int
	title   string
	authors []string
}

author := From(books).SelectMany( // make a flat array of authors
	func(book interface{}) Query {
		return From(book.(Book).authors)
	}).GroupBy( // group by author
	func(author interface{}) interface{} {
		return author // author as key
	}, func(author interface{}) interface{} {
		return author // author as value
	}).OrderByDescending( // sort groups by its length
	func(group interface{}) interface{} {
		return len(group.(Group).Group)
	}).Select( // get authors out of groups
	func(group interface{}) interface{} {
		return group.(Group).Key
	}).First() // take the first author

Example 3: Implement a custom method that leaves only values greater than the specified threshold

type MyQuery Query

func (q MyQuery) GreaterThan(threshold int) Query {
	return Query{
		Iterate: func() Iterator {
			next := q.Iterate()

			return func() (item interface{}, ok bool) {
				for item, ok = next(); ok; item, ok = next() {
					if item.(int) > threshold {
						return
					}
				}

				return
			}
		},
	}
}

result := MyQuery(Range(1,10)).GreaterThan(5).Results()

Generic Functions

Although Go doesn't implement generics, with some reflection tricks, you can use go-linq without typing interface{}s and type assertions. This will introduce a performance penalty (5x-10x slower) but will yield in a cleaner and more readable code.

Methods with T suffix (such as WhereT) accept functions with generic types. So instead of

.Select(func(v interface{}) interface{} {...})

you can type:

.SelectT(func(v YourType) YourOtherType {...})

This will make your code free of interface{} and type assertions.

Example 4: "MapReduce" in a slice of string sentences to list the top 5 most used words using generic functions

var results []string

From(sentences).
	// split sentences to words
	SelectManyT(func(sentence string) Query {
		return From(strings.Split(sentence, " "))
	}).
	// group the words
	GroupByT(
		func(word string) string { return word },
		func(word string) string { return word },
	).
	// order by count
	OrderByDescendingT(func(wordGroup Group) int {
		return len(wordGroup.Group)
	}).
	// order by the word
	ThenByT(func(wordGroup Group) string {
		return wordGroup.Key.(string)
	}).
	Take(5).  // take the top 5
	// project the words using the index as rank
	SelectIndexedT(func(index int, wordGroup Group) string {
		return fmt.Sprintf("Rank: #%d, Word: %s, Counts: %d", index+1, wordGroup.Key, len(wordGroup.Group))
	}).
	ToSlice(&results)

More examples can be found in the documentation.

Release Notes

v3.2.0 (2020-12-29)
* Added FromChannelT().
* Added DefaultIfEmpty().

v3.1.0 (2019-07-09)
* Support for Go modules
* Added IndexOf()/IndexOfT().

v3.0.0 (2017-01-10)
* Breaking change: ToSlice() now overwrites existing slice starting
  from index 0 and grows/reslices it as needed.
* Generic methods support (thanks @cleitonmarx!)
  - Accepting parametrized functions was originally proposed in #26
  - You can now avoid type assertions and interface{}s
  - Functions with generic methods are named as "MethodNameT" and
    signature for the existing LINQ methods are unchanged.
* Added ForEach(), ForEachIndexed() and AggregateWithSeedBy().

v2.0.0 (2016-09-02)
* IMPORTANT: This release is a BREAKING CHANGE. The old version
  is archived at the 'archive/0.9' branch or the 0.9 tags.
* A COMPLETE REWRITE of go-linq with better performance and memory
  efficiency. (thanks @kalaninja!)
* API has significantly changed. Most notably:
  - linq.T removed in favor of interface{}
  - library methods no longer return errors
  - PLINQ removed for now (see channels support)
  - support for channels, custom collections and comparables

v0.9-rc4
* GroupBy()

v0.9-rc3.2
* bugfix: All() iterating over values instead of indices

v0.9-rc3.1
* bugfix: modifying result slice affects subsequent query methods

v0.9-rc3
* removed FirstOrNil, LastOrNil, ElementAtOrNil methods

v0.9-rc2.5
* slice-accepting methods accept slices of any type with reflections

v0.9-rc2
* parallel linq (plinq) implemented
* Queryable separated into Query & ParallelQuery
* fixed early termination for All

v0.9-rc1
* many linq methods are implemented
* methods have error handling support
* type assertion limitations are unresolved
* travis-ci.org build integrated
* open sourced on github, master & dev branches
Owner
Ahmet Alp Balkan
Software Engineer at Google Cloud on cloud-native technologies and serverless platforms like Kubernetes/GKE and Cloud Run.
Ahmet Alp Balkan
Comments
  • [WIP] Generic functions proposal

    [WIP] Generic functions proposal

    Hey guys,

    This is just a proposal, I'd love to hear some feedback.

    Cheers,

    Cleiton

    var r []person
    From(listPerson).WhereT(func(i person) bool {
          return i.Age > 18
    }).ToSlice(&r)
    
  • Production use?

    Production use?

    Really a huge fan of LINQ. I use it all the time in the front-end (map, reduce are basically the same ideas). Would love to use this package but the one question I have is---"Is this used in production by anyone?"

  • ToSlice dose not re-empty the dist slice

    ToSlice dose not re-empty the dist slice

    Codes:

    func main() {
    	l := []string{"foo", "bar", "baz"}
    	linq.From(l).ToSlice(&l)
    	fmt.Println(l)
    }
    

    Result:

    [foo bar baz foo bar baz]
    

    go-linq version:

    - name: github.com/ahmetalpbalkan/go-linq
      version: 8985ec99e11a8bff7eb19dd0a0b2187770dab23a
    

    I don't know whether this is a bug or a feature, but if this is a feature, I think it's better to have this behavior documented on ToSlice's API doc.

  • Updated travis and added go mod.

    Updated travis and added go mod.

    This is an attempt at adding go mod to this library.

    @ahmetb I'm not sure if you had any plans on adding this, but I've started the process to migrate some of my projects from dep to go mod and thought I'd add this too.

    Should resolve #77

  • runtime error: hash of unhashable type map[string]interface {}

    runtime error: hash of unhashable type map[string]interface {}

    Unable to perform group by query on the following dataset

    rows := []map[string]interface{}{
    	{"name": "Mostain", "salary": 30000, "ShipmentTag": "b001"},
    	{"name": "Sanzida", "salary": 20000, "ShipmentTag": "b001"},
    	{"name": "Riaz", "salary": 21000, "ShipmentTag": "b002"},
    	{"name": "Pallobi", "salary": 30000, "ShipmentTag": "b001"},
    	{"name": "Moaz", "salary": 20000, "ShipmentTag": "b001"},
    	{"name": "Tareq", "salary": 21000, "ShipmentTag": "b002"},
    }
    
    	data := From(rows).Where(func(c interface{}) bool {
    		cmap := c.(map[string]interface{})
    		return len(cmap["name"].(string)) > 0
    
    	}).GroupBy(func(ShipmentTag interface{}) interface{} {
    		return ShipmentTag
    
    	}, func(ShipmentTag interface{}) interface{} {
    		return ShipmentTag
    
    	}).First()
    
    	fmt.Println(data)
    

    Error details:

    goroutine 1 [running]: github.com/ahmetb/go-linq/v3.Query.GroupBy.func1() E:/GOLANG/pkg/mod/github.com/ahmetb/go-linq/[email protected]/groupby.go:21 +0x15f github.com/ahmetb/go-linq/v3.Query.First({0x5f0aa0?}) E:/GOLANG/pkg/mod/github.com/ahmetb/go-linq/[email protected]/result.go:188 +0x1a main.main()

  • v2.0

    v2.0

    Hi there, I've managed to deal with all my "busy man" problems. So, back online and ready for some opensource.

    Here is the PR for v2.0. It doesn't have a readme file yet, because I am unaware about the links to godoc, travis, etc. I assume that if we tag the new commit with as "v2.0", the link to godoc would be smth like https://godoc.org/gopkg.in/ahmetalpbalkan/go-linq.v2.0 and it seems that tha branch for travis and coveralls can be specified at their urls. Is it correct? Should I tag it myself?

  • question: usage with KV store.

    question: usage with KV store.

    this sounds like a good match with boltdb, in terms of doing queries in memory.

    i currently use boltdb and also riak. i kind of hate riak but am stuck with it still. i was wondering if you think that linq would help me for writing many queries i need to do on top of boltdb ?

  • bug in WhereIndexed

    bug in WhereIndexed

    I believe WhereIndexed doesn't work as design:

    https://play.golang.org/p/HD5-rxIkpx0

    package main
    
    import (
    	"fmt"
    
    	"github.com/ahmetb/go-linq/v3"
    )
    
    func main() {
    
    	r := linq.Range(1, 10).
    		WhereIndexed(func(i int, _ interface{}) bool {
    			return i%3 == 0
    		}).
    		Results()
    	fmt.Println(r)
    }
    

    output of the code above is [1 2 3 4 5 6 7 8 9 10] instead of [3 6 9]

    maybe the bug is here

    https://github.com/ahmetb/go-linq/blob/master/where.go#L61

    thanks!

  • How to implement complex SQL functions

    How to implement complex SQL functions

    How to implement complex SQL functions similar to the following

    type TransactionRecord struct {
    	ID        uint64
    	AccountID string
    	TCode     string
    	TDate     string
    	TType     uint8
    	Amount    float64
    }
    
    var tds []TransactionRecord
    t1 := TransactionRecord{ID: 1, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 23.0}
    t2 := TransactionRecord{ID: 2, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 99.0}
    t3 := TransactionRecord{ID: 3, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 25.0}
    
    t4 := TransactionRecord{ID: 1, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 13.0}
    t5 := TransactionRecord{ID: 2, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 69.0}
    t6 := TransactionRecord{ID: 3, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 85.0}
    		
    t7 := TransactionRecord{ID: 4, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 11.0}
    t8 := TransactionRecord{ID: 5, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 33.0}
    t9 := TransactionRecord{ID: 6, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 57.0}
    		
    tds = append(tds, t1)
    tds = append(tds, t2)
    tds = append(tds, t3)
    tds = append(tds, t4)
    tds = append(tds, t5)
    tds = append(tds, t6)
    tds = append(tds, t7)
    tds = append(tds, t8)
    tds = append(tds, t9)
    

    I want to implement the following SQL functions, how to write code with go-linq package

    select AccountID,TCode, TDate,TType, sum(Amount)  as am from TransactionDetails where TType = 17 or TType = 24 GROUP BY AccountID, TCode,TDate, TType ORDER BY AccountID, TCode,TDate, TType;
    
  • gopkg.in/ahmetb/go-linq.v3: gopkg.in/ahmetb/go-linq.v3@v3.1.0: parsing go.mod:         module declares its path as: github.com/ahmetb/go-linq/v3 but was required as: gopkg.in/ahmetb/go-linq.v3

    gopkg.in/ahmetb/go-linq.v3: gopkg.in/ahmetb/[email protected]: parsing go.mod: module declares its path as: github.com/ahmetb/go-linq/v3 but was required as: gopkg.in/ahmetb/go-linq.v3

    I am trying to do create go.mod in my project. And this error arrives what should I do. I have imported gopkg.in/ahmetb/go-linq.v3 this package in my project files

  • Typed channels support

    Typed channels support

    • introduced a new method FromTypedChannel()
    • From() calls FromChannel() if channel is of type chan interface{} and FromTypedChannel for all other channels
    • benchmark for FromTypedChannel() and FromChannel() performance comparison
  • Unit test ExampleOrderedQuery_ThenByDescending fails on golang-1.19.2

    Unit test ExampleOrderedQuery_ThenByDescending fails on golang-1.19.2

          testing: github.com/ahmetb/go-linq
    github.com/ahmetb/go-linq
    --- FAIL: ExampleOrderedQuery_ThenByDescending (0.00s)
    got:
    apPLe
    apple
    apPLE
    APple
    orange
    baNanA
    ORANGE
    BAnana
    want:
    apPLe
    apPLE
    apple
    APple
    orange
    baNanA
    ORANGE
    BAnana
    FAIL
    exit status 1
    FAIL	github.com/ahmetb/go-linq	0.021s
    
  • Add Generics to go-linq

    Add Generics to go-linq

    Hey guys, I like go-linq very much and I'd like to add generics support for this lib long ago, this weekend I've tried.

    The trickiest part is how to create parameterized method since golang cannot do that, for example we cannot declare the following method:

    func (QueryG[TOut]) GroupJoin[TOut, TInner, TKey, TResult any](inner QueryG[TInner],
    	outerKeySelector func(TOut) TKey,
    	innerKeySelector func(TInner) TKey,
    	resultSelector func(outer TOut, inners []TInner) TResult) QueryG[TResult]
    

    My way is declare some Expander type to do that. For this GroupJoin example:

    func TestGroupJoinG(t *testing.T) {
    	outer := []int{0, 1, 2}
    	inner := []uint{1, 2, 3, 4, 5, 6, 7, 8, 9}
    	want := []KeyValueG[int, []uint]{
    		{0, []uint{2, 4, 6, 8}},
    		{1, []uint{1, 3, 5, 7, 9}},
    		{2, []uint{}},
    	}
    
    	actual := FromSliceG(outer).Expend4(To4[int, uint, int, KeyValueG[int, []uint]]()).(*Expended4[int, uint, int, KeyValueG[int, []uint]]).GroupJoin(
    		FromSliceG(inner),
    		func(i int) int { return i },
    		func(ui uint) int { return int(ui) % 2 },
    		func(outer int, inners []uint) KeyValueG[int, []uint] {
    			return KeyValueG[int, []uint]{outer, inners}
    		},
    	).ToSlice()
    	assert.Equal(t, want, actual)
    }
    

    We can declare an Expander object to capture the extended type parameters we need, then cast the type immediately, then we decalre these type extended method on these Expander type. The only potential risk point that have no protection from the compiler is this type casting, but they're so close so I think the risk is limited.

    This pr is composed to learn go's generics and thanks to @ahmetb I had a lot of fun.

  • Try out the new Go generics proposal with go-linq

    Try out the new Go generics proposal with go-linq

    Go now has a prototype of generics implementation. Here are some resources:

    • https://blog.golang.org/generics-next-step
    • https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md
    • https://rakyll.org/generics-proposal/

    This potentially could give a severe performance boost to go-linq, as well as actually make this library useful.

    I haven't taken a closer look at how we would do this yet. For example, we might still end up having to do some type assertions. However, it might help us redesign the package and release a v4 if the generics proposal is adopted.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Antch Antch, inspired by Scrapy. If you're familiar with scrapy, you can quickly get started. Antch is a fast, powerful and extensible web crawling &

Jan 6, 2023
DSV Parallel Processor takes input files and query specification via a spec file

DSV Parallel Processor Spec file DSV Parallel Processor takes input files and query specification via a spec file (conventionally named "spec.toml").

Oct 9, 2021
FreeSWITCH Event Socket library for the Go programming language.

eventsocket FreeSWITCH Event Socket library for the Go programming language. It supports both inbound and outbound event socket connections, acting ei

Dec 11, 2022
Some utilities for Persian language in Go (Golang)

persian Some utilities for Persian language in Go (Golang). Installation go get github.com/mavihq/persian API .ToPersianDigits Converts all English d

Oct 22, 2022
A modern programming language written in Golang.

MangoScript A modern programming language written in Golang. Here is what I want MangoScript to look like: struct Rectangle { width: number he

Nov 12, 2021
A toy language parser, lexer and interpreter written in Golang

Monkey - A toy programming language Monkey is a toy programming language used to learn how to write a lexer, parser and interpreter. The language is i

Nov 16, 2021
Unit tests generator for Go programming language
Unit tests generator for Go programming language

GoUnit GoUnit is a commandline tool that generates tests stubs based on source function or method signature. There are plugins for Vim Emacs Atom Subl

Jan 1, 2023
Simple interface to libmagic for Go Programming Language

File Magic in Go Introduction Provides simple interface to libmagic for Go Programming Language. Table of Contents Contributing Versioning Author Copy

Dec 22, 2021
Go language interface to the PAPI performance API

go-papi Description go-papi provides a Go interface to PAPI, the Performance Application Programming Interface. PAPI provides convenient access to har

Dec 22, 2022
go language generics system

Gotgo This document describes the third iteration of my attempt at a reasonable implementation of generics for go based on the idea of template packag

Dec 31, 2022
The Gorilla Programming Language
The Gorilla Programming Language

Gorilla Programming Language Gorilla is a tiny, dynamically typed, multi-engine programming language It has flexible syntax, a compiler, as well as an

Apr 16, 2022
Elastic is an Elasticsearch client for the Go programming language.

Elastic is an Elasticsearch client for the Go programming language.

Jan 9, 2023
CodePlayground is a playground tool for go and rust language.

CodePlayground CodePlayground is a playground tool for go and rust language. Installation Use homebrews to install code-playground. brew tap trendyol/

Mar 5, 2022
Assembly syntax that makes you feel like you're writing code in a high-level language.

shasm Assembly syntax that makes you feel like you're writing code in a high-level language. Shasm is not an Assembler. Shasm simply compiles Shasm sy

Jun 5, 2021
The new home of the CUE language! Validate and define text-based and dynamic configuration

The CUE Data Constraint Language Configure, Unify, Execute CUE is an open source data constraint language which aims to simplify tasks involving defin

Dec 31, 2022
👩🏼‍💻A simple compiled programming language
👩🏼‍💻A simple compiled programming language

The language is written in Go and the target language is C. The built-in library is written in C too

Nov 29, 2022
feedme project porting in Go language

newsapi-go This project has the intent to provide a valid interface for newsapi (https://newsapi.org/). usage The project is still under mantainance a

Oct 28, 2021
Go specs implemented as a scripting language in Rust.

Goscript A script language like Python or Lua written in Rust, with exactly the same syntax as Go's. The Goal Runs most pure Go code, probably add som

Jan 8, 2023
Lithia is an experimental functional programming language with an implicit but strong and dynamic type system.

Lithia is an experimental functional programming language with an implicit but strong and dynamic type system. Lithia is designed around a few core concepts in mind all language features contribute to.

Dec 24, 2022