.NET LINQ capabilities in Go

go-linq GoDoc Build Status Coverage Status Go Report Card

A powerful language integrated query (LINQ) library for Go.

  • Written in vanilla Go, no dependencies!
  • Complete lazy evaluation with iterator pattern
  • Safe for concurrent use
  • Supports generic functions to make your code cleaner and free of type assertions
  • Supports arrays, slices, maps, strings, channels and custom collections

Installation

When used with Go modules, use the following import path:

go get github.com/ahmetb/go-linq/v3

Older versions of Go using different dependency management tools can use the following import path to prevent breaking API changes:

go get gopkg.in/ahmetb/go-linq.v3

Quickstart

Usage is as easy as chaining methods like:

From(slice) .Where(predicate) .Select(selector) .Union(data)

Example 1: Find all owners of cars manufactured after 2015

import . "github.com/ahmetb/go-linq/v3"

type Car struct {
    year int
    owner, model string
}

...


var owners []string

From(cars).Where(func(c interface{}) bool {
	return c.(Car).year >= 2015
}).Select(func(c interface{}) interface{} {
	return c.(Car).owner
}).ToSlice(&owners)

Or, you can use generic functions, like WhereT and SelectT to simplify your code (at a performance penalty):

var owners []string

From(cars).WhereT(func(c Car) bool {
	return c.year >= 2015
}).SelectT(func(c Car) string {
	return c.owner
}).ToSlice(&owners)

Example 2: Find the author who has written the most books

import . "github.com/ahmetb/go-linq/v3"

type Book struct {
	id      int
	title   string
	authors []string
}

author := From(books).SelectMany( // make a flat array of authors
	func(book interface{}) Query {
		return From(book.(Book).authors)
	}).GroupBy( // group by author
	func(author interface{}) interface{} {
		return author // author as key
	}, func(author interface{}) interface{} {
		return author // author as value
	}).OrderByDescending( // sort groups by its length
	func(group interface{}) interface{} {
		return len(group.(Group).Group)
	}).Select( // get authors out of groups
	func(group interface{}) interface{} {
		return group.(Group).Key
	}).First() // take the first author

Example 3: Implement a custom method that leaves only values greater than the specified threshold

type MyQuery Query

func (q MyQuery) GreaterThan(threshold int) Query {
	return Query{
		Iterate: func() Iterator {
			next := q.Iterate()

			return func() (item interface{}, ok bool) {
				for item, ok = next(); ok; item, ok = next() {
					if item.(int) > threshold {
						return
					}
				}

				return
			}
		},
	}
}

result := MyQuery(Range(1,10)).GreaterThan(5).Results()

Generic Functions

Although Go doesn't implement generics, with some reflection tricks, you can use go-linq without typing interface{}s and type assertions. This will introduce a performance penalty (5x-10x slower) but will yield in a cleaner and more readable code.

Methods with T suffix (such as WhereT) accept functions with generic types. So instead of

.Select(func(v interface{}) interface{} {...})

you can type:

.SelectT(func(v YourType) YourOtherType {...})

This will make your code free of interface{} and type assertions.

Example 4: "MapReduce" in a slice of string sentences to list the top 5 most used words using generic functions

var results []string

From(sentences).
	// split sentences to words
	SelectManyT(func(sentence string) Query {
		return From(strings.Split(sentence, " "))
	}).
	// group the words
	GroupByT(
		func(word string) string { return word },
		func(word string) string { return word },
	).
	// order by count
	OrderByDescendingT(func(wordGroup Group) int {
		return len(wordGroup.Group)
	}).
	// order by the word
	ThenByT(func(wordGroup Group) string {
		return wordGroup.Key.(string)
	}).
	Take(5).  // take the top 5
	// project the words using the index as rank
	SelectIndexedT(func(index int, wordGroup Group) string {
		return fmt.Sprintf("Rank: #%d, Word: %s, Counts: %d", index+1, wordGroup.Key, len(wordGroup.Group))
	}).
	ToSlice(&results)

More examples can be found in the documentation.

Release Notes

v3.2.0 (2020-12-29)
* Added FromChannelT().
* Added DefaultIfEmpty().

v3.1.0 (2019-07-09)
* Support for Go modules
* Added IndexOf()/IndexOfT().

v3.0.0 (2017-01-10)
* Breaking change: ToSlice() now overwrites existing slice starting
  from index 0 and grows/reslices it as needed.
* Generic methods support (thanks @cleitonmarx!)
  - Accepting parametrized functions was originally proposed in #26
  - You can now avoid type assertions and interface{}s
  - Functions with generic methods are named as "MethodNameT" and
    signature for the existing LINQ methods are unchanged.
* Added ForEach(), ForEachIndexed() and AggregateWithSeedBy().

v2.0.0 (2016-09-02)
* IMPORTANT: This release is a BREAKING CHANGE. The old version
  is archived at the 'archive/0.9' branch or the 0.9 tags.
* A COMPLETE REWRITE of go-linq with better performance and memory
  efficiency. (thanks @kalaninja!)
* API has significantly changed. Most notably:
  - linq.T removed in favor of interface{}
  - library methods no longer return errors
  - PLINQ removed for now (see channels support)
  - support for channels, custom collections and comparables

v0.9-rc4
* GroupBy()

v0.9-rc3.2
* bugfix: All() iterating over values instead of indices

v0.9-rc3.1
* bugfix: modifying result slice affects subsequent query methods

v0.9-rc3
* removed FirstOrNil, LastOrNil, ElementAtOrNil methods

v0.9-rc2.5
* slice-accepting methods accept slices of any type with reflections

v0.9-rc2
* parallel linq (plinq) implemented
* Queryable separated into Query & ParallelQuery
* fixed early termination for All

v0.9-rc1
* many linq methods are implemented
* methods have error handling support
* type assertion limitations are unresolved
* travis-ci.org build integrated
* open sourced on github, master & dev branches
Owner
Ahmet Alp Balkan
Senior software engineer at Google Cloud, on cloud-native technologies: Kubernetes/GKE and Cloud Run.
Ahmet Alp Balkan
Comments
  • [WIP] Generic functions proposal

    [WIP] Generic functions proposal

    Hey guys,

    This is just a proposal, I'd love to hear some feedback.

    Cheers,

    Cleiton

    var r []person
    From(listPerson).WhereT(func(i person) bool {
          return i.Age > 18
    }).ToSlice(&r)
    
  • Production use?

    Production use?

    Really a huge fan of LINQ. I use it all the time in the front-end (map, reduce are basically the same ideas). Would love to use this package but the one question I have is---"Is this used in production by anyone?"

  • ToSlice dose not re-empty the dist slice

    ToSlice dose not re-empty the dist slice

    Codes:

    func main() {
    	l := []string{"foo", "bar", "baz"}
    	linq.From(l).ToSlice(&l)
    	fmt.Println(l)
    }
    

    Result:

    [foo bar baz foo bar baz]
    

    go-linq version:

    - name: github.com/ahmetalpbalkan/go-linq
      version: 8985ec99e11a8bff7eb19dd0a0b2187770dab23a
    

    I don't know whether this is a bug or a feature, but if this is a feature, I think it's better to have this behavior documented on ToSlice's API doc.

  • Updated travis and added go mod.

    Updated travis and added go mod.

    This is an attempt at adding go mod to this library.

    @ahmetb I'm not sure if you had any plans on adding this, but I've started the process to migrate some of my projects from dep to go mod and thought I'd add this too.

    Should resolve #77

  • runtime error: hash of unhashable type map[string]interface {}

    runtime error: hash of unhashable type map[string]interface {}

    Unable to perform group by query on the following dataset

    rows := []map[string]interface{}{
    	{"name": "Mostain", "salary": 30000, "ShipmentTag": "b001"},
    	{"name": "Sanzida", "salary": 20000, "ShipmentTag": "b001"},
    	{"name": "Riaz", "salary": 21000, "ShipmentTag": "b002"},
    	{"name": "Pallobi", "salary": 30000, "ShipmentTag": "b001"},
    	{"name": "Moaz", "salary": 20000, "ShipmentTag": "b001"},
    	{"name": "Tareq", "salary": 21000, "ShipmentTag": "b002"},
    }
    
    	data := From(rows).Where(func(c interface{}) bool {
    		cmap := c.(map[string]interface{})
    		return len(cmap["name"].(string)) > 0
    
    	}).GroupBy(func(ShipmentTag interface{}) interface{} {
    		return ShipmentTag
    
    	}, func(ShipmentTag interface{}) interface{} {
    		return ShipmentTag
    
    	}).First()
    
    	fmt.Println(data)
    

    Error details:

    goroutine 1 [running]: github.com/ahmetb/go-linq/v3.Query.GroupBy.func1() E:/GOLANG/pkg/mod/github.com/ahmetb/go-linq/[email protected]/groupby.go:21 +0x15f github.com/ahmetb/go-linq/v3.Query.First({0x5f0aa0?}) E:/GOLANG/pkg/mod/github.com/ahmetb/go-linq/[email protected]/result.go:188 +0x1a main.main()

  • v2.0

    v2.0

    Hi there, I've managed to deal with all my "busy man" problems. So, back online and ready for some opensource.

    Here is the PR for v2.0. It doesn't have a readme file yet, because I am unaware about the links to godoc, travis, etc. I assume that if we tag the new commit with as "v2.0", the link to godoc would be smth like https://godoc.org/gopkg.in/ahmetalpbalkan/go-linq.v2.0 and it seems that tha branch for travis and coveralls can be specified at their urls. Is it correct? Should I tag it myself?

  • question: usage with KV store.

    question: usage with KV store.

    this sounds like a good match with boltdb, in terms of doing queries in memory.

    i currently use boltdb and also riak. i kind of hate riak but am stuck with it still. i was wondering if you think that linq would help me for writing many queries i need to do on top of boltdb ?

  • bug in WhereIndexed

    bug in WhereIndexed

    I believe WhereIndexed doesn't work as design:

    https://play.golang.org/p/HD5-rxIkpx0

    package main
    
    import (
    	"fmt"
    
    	"github.com/ahmetb/go-linq/v3"
    )
    
    func main() {
    
    	r := linq.Range(1, 10).
    		WhereIndexed(func(i int, _ interface{}) bool {
    			return i%3 == 0
    		}).
    		Results()
    	fmt.Println(r)
    }
    

    output of the code above is [1 2 3 4 5 6 7 8 9 10] instead of [3 6 9]

    maybe the bug is here

    https://github.com/ahmetb/go-linq/blob/master/where.go#L61

    thanks!

  • How to implement complex SQL functions

    How to implement complex SQL functions

    How to implement complex SQL functions similar to the following

    type TransactionRecord struct {
    	ID        uint64
    	AccountID string
    	TCode     string
    	TDate     string
    	TType     uint8
    	Amount    float64
    }
    
    var tds []TransactionRecord
    t1 := TransactionRecord{ID: 1, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 23.0}
    t2 := TransactionRecord{ID: 2, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 99.0}
    t3 := TransactionRecord{ID: 3, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 25.0}
    
    t4 := TransactionRecord{ID: 1, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 13.0}
    t5 := TransactionRecord{ID: 2, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 69.0}
    t6 := TransactionRecord{ID: 3, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 85.0}
    		
    t7 := TransactionRecord{ID: 4, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 11.0}
    t8 := TransactionRecord{ID: 5, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 33.0}
    t9 := TransactionRecord{ID: 6, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 57.0}
    		
    tds = append(tds, t1)
    tds = append(tds, t2)
    tds = append(tds, t3)
    tds = append(tds, t4)
    tds = append(tds, t5)
    tds = append(tds, t6)
    tds = append(tds, t7)
    tds = append(tds, t8)
    tds = append(tds, t9)
    

    I want to implement the following SQL functions, how to write code with go-linq package

    select AccountID,TCode, TDate,TType, sum(Amount)  as am from TransactionDetails where TType = 17 or TType = 24 GROUP BY AccountID, TCode,TDate, TType ORDER BY AccountID, TCode,TDate, TType;
    
  • gopkg.in/ahmetb/go-linq.v3: gopkg.in/ahmetb/go-linq.v3@v3.1.0: parsing go.mod:         module declares its path as: github.com/ahmetb/go-linq/v3 but was required as: gopkg.in/ahmetb/go-linq.v3

    gopkg.in/ahmetb/go-linq.v3: gopkg.in/ahmetb/[email protected]: parsing go.mod: module declares its path as: github.com/ahmetb/go-linq/v3 but was required as: gopkg.in/ahmetb/go-linq.v3

    I am trying to do create go.mod in my project. And this error arrives what should I do. I have imported gopkg.in/ahmetb/go-linq.v3 this package in my project files

  • Typed channels support

    Typed channels support

    • introduced a new method FromTypedChannel()
    • From() calls FromChannel() if channel is of type chan interface{} and FromTypedChannel for all other channels
    • benchmark for FromTypedChannel() and FromChannel() performance comparison
  • Unit test ExampleOrderedQuery_ThenByDescending fails on golang-1.19.2

    Unit test ExampleOrderedQuery_ThenByDescending fails on golang-1.19.2

          testing: github.com/ahmetb/go-linq
    github.com/ahmetb/go-linq
    --- FAIL: ExampleOrderedQuery_ThenByDescending (0.00s)
    got:
    apPLe
    apple
    apPLE
    APple
    orange
    baNanA
    ORANGE
    BAnana
    want:
    apPLe
    apPLE
    apple
    APple
    orange
    baNanA
    ORANGE
    BAnana
    FAIL
    exit status 1
    FAIL	github.com/ahmetb/go-linq	0.021s
    
  • Add Generics to go-linq

    Add Generics to go-linq

    Hey guys, I like go-linq very much and I'd like to add generics support for this lib long ago, this weekend I've tried.

    The trickiest part is how to create parameterized method since golang cannot do that, for example we cannot declare the following method:

    func (QueryG[TOut]) GroupJoin[TOut, TInner, TKey, TResult any](inner QueryG[TInner],
    	outerKeySelector func(TOut) TKey,
    	innerKeySelector func(TInner) TKey,
    	resultSelector func(outer TOut, inners []TInner) TResult) QueryG[TResult]
    

    My way is declare some Expander type to do that. For this GroupJoin example:

    func TestGroupJoinG(t *testing.T) {
    	outer := []int{0, 1, 2}
    	inner := []uint{1, 2, 3, 4, 5, 6, 7, 8, 9}
    	want := []KeyValueG[int, []uint]{
    		{0, []uint{2, 4, 6, 8}},
    		{1, []uint{1, 3, 5, 7, 9}},
    		{2, []uint{}},
    	}
    
    	actual := FromSliceG(outer).Expend4(To4[int, uint, int, KeyValueG[int, []uint]]()).(*Expended4[int, uint, int, KeyValueG[int, []uint]]).GroupJoin(
    		FromSliceG(inner),
    		func(i int) int { return i },
    		func(ui uint) int { return int(ui) % 2 },
    		func(outer int, inners []uint) KeyValueG[int, []uint] {
    			return KeyValueG[int, []uint]{outer, inners}
    		},
    	).ToSlice()
    	assert.Equal(t, want, actual)
    }
    

    We can declare an Expander object to capture the extended type parameters we need, then cast the type immediately, then we decalre these type extended method on these Expander type. The only potential risk point that have no protection from the compiler is this type casting, but they're so close so I think the risk is limited.

    This pr is composed to learn go's generics and thanks to @ahmetb I had a lot of fun.

  • Try out the new Go generics proposal with go-linq

    Try out the new Go generics proposal with go-linq

    Go now has a prototype of generics implementation. Here are some resources:

    • https://blog.golang.org/generics-next-step
    • https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md
    • https://rakyll.org/generics-proposal/

    This potentially could give a severe performance boost to go-linq, as well as actually make this library useful.

    I haven't taken a closer look at how we would do this yet. For example, we might still end up having to do some type assertions. However, it might help us redesign the package and release a v4 if the generics proposal is adopted.

This library provides an ASTERIX Frame(binary data) decoding/parsing(json,xml) capabilities for Go.

GoAsterix This library provides an ASTERIX Frame(binary data) decoding/parsing(json,xml) capabilities for Go. ASTERIX ASTERIX (All Purpose Structured

Dec 13, 2022
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Dec 28, 2022
.NET LINQ capabilities in Go

go-linq A powerful language integrated query (LINQ) library for Go. Written in vanilla Go, no dependencies! Complete lazy evaluation with iterator pat

Jan 7, 2023
.NET LINQ capabilities in Go

go-linq A powerful language integrated query (LINQ) library for Go. Written in vanilla Go, no dependencies! Complete lazy evaluation with iterator pat

Jan 7, 2023
List your dependencies capabilities and monitor if updates require more capabilities.

A take on supply chain security in Go List your dependencies capabilities and monitor if dependency updates require more capabilities. The Problem Rec

Nov 16, 2022
Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces

lossy Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces. Its main usage is to test robustness of ap

Oct 14, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Jan 1, 2023
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Dec 31, 2022
A Go (golang) package that enhances the standard database/sql package by providing powerful data retrieval methods as well as DB-agnostic query building capabilities.

ozzo-dbx Summary Description Requirements Installation Supported Databases Getting Started Connecting to Database Executing Queries Binding Parameters

Dec 31, 2022
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.
Project Flogo is an open source ecosystem of opinionated  event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.

Project Flogo is an Open Source ecosystem for event-driven apps Ecosystem | Core | Flows | Streams | Flogo Rules | Go Developers | When to use Flogo |

Dec 31, 2022
Provides packet processing capabilities for Go

GoPacket This library provides packet decoding capabilities for Go. See godoc for more details. Minimum Go version required is 1.5 except for pcapgo/E

Dec 29, 2022
GoLang Library for Browser Capabilities Project

Browser Capabilities GoLang Project PHP has get_browser() function which tells what the user's browser is capable of. You can check original documenta

Sep 27, 2022
Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

Version 1.0 vs 2.0 Update 2015 Sept 20: Big news! Version 2.0 of the go-bindings, authored by Ross Light, is now released and newly available! It feat

Nov 29, 2022
SSRFuzz is a tool to find Server Side Request Forgery vulnerabilities, with CRLF chaining capabilities

SSRFuzz is a tool to find Server Side Request Forgery vulnerabilities, with CRLF chaining capabilities Why?

Dec 8, 2022
Git-like capabilities for your object storage
Git-like capabilities for your object storage

What is lakeFS lakeFS is an open source layer that delivers resilience and manageability to object-storage based data lakes. With lakeFS you can build

Dec 30, 2022
Extend KIND networking capabilities with plugins using the KIND API

kind-networking-plugins Plugins to extend KIND networking capabilities with plugins using the KIND API These plugins were used for the Kubecon EU 2021

Nov 10, 2022
Layotto is an application runtime developed using Golang, which provides various distributed capabilities for applications
Layotto is an application runtime developed using Golang, which provides various distributed capabilities for applications

Layotto is an application runtime developed using Golang, which provides various distributed capabilities for applications, such as state management, configuration management, and event pub/sub capabilities to simplify application development.

Jan 8, 2023
PolarDB Stack is a DBaaS implementation for PolarDB-for-Postgres, as an operator creates and manages PolarDB/PostgreSQL clusters running in Kubernetes. It provides re-construct, failover swtich-over, scale up/out, high-available capabilities for each clusters.
PolarDB Stack is a DBaaS implementation for PolarDB-for-Postgres, as an operator creates and manages PolarDB/PostgreSQL clusters running in Kubernetes. It provides re-construct, failover swtich-over, scale up/out, high-available capabilities for each clusters.

PolarDB Stack开源版生命周期 1 系统概述 PolarDB是阿里云自研的云原生关系型数据库,采用了基于Shared-Storage的存储计算分离架构。数据库由传统的Share-Nothing,转变成了Shared-Storage架构。由原来的N份计算+N份存储,转变成了N份计算+1份存储

Nov 8, 2022
This library provides an ASTERIX Frame(binary data) decoding/parsing(json,xml) capabilities for Go.

GoAsterix This library provides an ASTERIX Frame(binary data) decoding/parsing(json,xml) capabilities for Go. ASTERIX ASTERIX (All Purpose Structured

Dec 13, 2022
GoLang Library for Browser Capabilities Project

Browser Capabilities GoLang Project PHP has get_browser() function which tells what the user's browser is capable of. You can check original documenta

Nov 23, 2021