A simple Go package to Query over JSON/YAML/XML/CSV Data

Change to enable queries on standalone objects. This makes queries like this possible:

j := gojsonq.New().JSONString(`{"name":{"first":"Tom","last":"Hanks"},"age":61}`)
ct := j.Where("age", ">=", 60).Where("name.last", "eq", "Hanks").Count() // 1

I am trying to build a server that can fetch certain json entries based on a parameter. So for example if a user enters localhost:8080/store/1 It returns the json entry with the ID 1.

The code i currently have works if i hardcode the id field into the query, but trying to put in parameters i have so far had no luck. Trying this:

func getGame(w http.ResponseWriter, r *http.Request) {
	params := mux.Vars(r)
	jq := gojsonq.New().File("./data/games.json")
	res := jq.From("games").Where("id", "=", params["id"]).Get()
	fmt.Println(res)
}

Suggestions please, before i throw something out the window.

Is it possible to traverse multiple nested arrays in this library without knowing the array indexes? I have a scenario that works well with where clauses on the first array, but I haven't had any success with a second array below that.

This is a successful query using Find (but assumes I know the index to each array):

Find("data.items.[1].nestedA.[3].nestedB.value")

I've tried many variations of: From("data.items").Where("itemValue", "=", "asdf").From("nestedA").Where("nestedBOtherValue", "eq", "fdsa").Find("nestedB.value") without any luck.

issue

When using :

tempTags = []Tag{} // Structure
jsonResult.As(&tempTags)

This error is displayed an error from reflect library:

panic: reflect: call of reflect.Value.Call on zero Value [recovered]
        panic: reflect: call of reflect.Value.Call on zero Value

Expected

Should display a more specific error that give us more clues in order to understand what is going on

Panic: the method '*[]repository.Tag' must be part of 'map[*[]bool:BoolSlice *[]float32:Float32Slice *[]float64:Float64Slice *[]int:IntSlice *[]int16:Int16Slice *[]int32:Int32Slice *[]int8:Int8Slice *[]string:StringSlice *[]time.Duration:DurationSlice *[]uint:UintSlice *[]uint16:Uint16Slice *[]uint32:Uint32Slice *[]uint8:Uint8Slice *bool:Bool *float32:Float32 *float64:Float64 *int:Int *int16:Int16 *int32:Int32 *int8:Int8 *string:String *time.Duration:Duration *uint:Uint *uint16:Uint16 *uint32:Uint32 *uint8:Uint8]' [recovered]
        panic: the method '*[]repository.Tag' must be part of 'map[*[]bool:BoolSlice *[]float32:Float32Slice *[]float64:Float64Slice *[]int:IntSlice *[]int16:Int16Slice *[]int32:Int32Slice *[]int8:Int8Slice *[]string:StringSlice *[]time.Duration:DurationSlice *[]uint:UintSlice *[]uint16:Uint16Slice *[]uint32:Uint32Slice *[]uint8:Uint8Slice *bool:Bool *float32:Float32 *float64:Float64 *int:Int *int16:Int16 *int32:Int32 *int8:Int8 *string:String *time.Duration:Duration *uint:Uint *uint16:Uint16 *uint32:Uint32 *uint8:Uint8]'

Hello,

This is an "info request".

I have the following json and I would like to check if the input contains a father (fid) with two children.

{
  "data": [
    {
      "fid": 10,
      "children": [
        {
          "cid": 123
        },
        {
          "cid": 124
        }
       
      ]
    }
  ]
}

The code so far is

package main

import (
	"fmt"

	"github.com/thedevsaddam/gojsonq"
)

const json = `{
	"data": [
	  {
		"fid": 10,
		"children": [
		  {
			"cid": 123
		  },
		  {
			"cid": 124
		  }
		 
		]
	  }
	]
  }`

func main() {

	q := gojsonq.New().JSONString(json).From("data").
		Where("fid", "=", 10)
	fmt.Println(q.Get())
}

How can I modify the source code to decide if fid=10 AND children.size() = 2?

In principle how can I use the "array size" to query the json file/input?

Thanks

Doesn't seem to work when you don't have nicely sliceable structures.

IE: https://www.iinet.net.au/status/event/events.json

I tried Where("DRW", "=", "Darwin") but kept getting nothing back

Hi there, Is there any way to load JSON document from map[string]interface{}? Simplest use case is querying MongoDB, which returns result documents in that format(without using schema struct). I am willing to make a PR if little help is supplied. Thanks

go-critic linter found some issues , here corresponding log:

check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 0 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 1 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 2 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 3 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 4 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 5 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 6 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:271:3: range ValCopy: each iteration copies 48 bytes (consider pointers or indexing) check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:45:18: typeU nparen: could simplify to [][]query

In #18 I propose fixing for them

Why replacerange with for loop? You must create an issue regarding this refactor before sending PR.

Thanks, good point. The way that I made in my PR not very accurate and should be replaced into this one as written in description to rangeValCopy.

I saw a post on r/golang probably yours. And I wondered why you did not add the examples in the linked article to your project, so I did add some and of your README-file. These will be shown on https://godoc.org/github.com/thedevsaddam/gojsonq the godoc page.

You have to edit the import-path of the examples_test.go file to your repo as I plan to remove my fork as soon as the changes are merged.

Thanks for sharing this project. One question. Why do you do this:

// strStrictContains checks if x contains y
// This is case sensitive search
func strStrictContains(x, y interface{}) (bool, error) {
	xv, okX := x.(string)
	if !okX {
		return false, fmt.Errorf("%v must be string", x)
	}
	yv, okY := y.(string)
	if !okY {
		return false, fmt.Errorf("%v must be string", y)
	}
	return strings.Contains(xv, yv), nil
}

...and not this:

// strStrictContains checks if x contains y
// This is case sensitive search
func strStrictContains(x, y string) (bool) {
	return strings.Contains(xv, yv)
}

...or in this case, simply strings.Contains(xv, yv) without a wrapper.

In this example, you are only allowing string inputs anyway, so why not use strongly typed strings as input params instead of interface{}?

I have an array of JSON objects which all contain a few common keys and then a whole bunch of keys that are different. I want to select a subset of this array that drops a single common key.

The only way to do that right now is to use a select or only with all the possible keys the object can contain minus the one I want to drop. It would be nice to support something like "unselect" or "drop" where I can drop a specific key from the result while keeping the rest.

data:

{
  "appStatus": {
    "displayName": "JobServer",
    "instanceStatus": {
      "autoRestartSuspended": false,
      "autoRestartSuspendedForApp": false,
      "autoRestartableApp": false,
      "instance": {
        "active": true,
        "applicationDisplayName": "JobServer",
        "applicationId": 42,
        "applicationName": "JobServer",
        "custom": false,
        "id": 10,
        "name": 1,
        "serverName": "xxx"
      },
      "lastStartTime": "2022-11-06 11:53:52",
      "lastStatusChangeTime": "2022-12-14 16:37:31",
      "lastStatusEvent": "Stop",
      "lastStopTime": "2022-12-14 16:37:31",
      "status": "STOPPED"
    },
    "name": "JobServer",
    "numberOfInstancesDown": 1,
    "numberOfInstancesRunning": 0,
    "outageCountInLast24Hrs": 1,
    "pecentageUpInstance": 0,
    "status": "Red",
    "totalNumberOfInstances": 1
  }
}

code:

notStopped := jq.New().File("out.json").
	From("appStatus.instanceStatus").
	Where("status", "neq", "STOPPED")

log.Println(notStoppped)
log.Println(notStopped.Get())

output:

Content: {
  "appStatus": {
    "displayName": "JobServer",
    "instanceStatus": {
      "autoRestartSuspended": false,
      "autoRestartSuspendedForApp": false,
      "autoRestartableApp": false,
      "instance": {
        "active": true,
        "applicationDisplayName": "JobServer",
        "applicationId": 42,
        "applicationName": "JobServer",
        "custom": false,
        "id": 10,
        "name": 1,
        "serverName": "xxx"
      },
      "lastStartTime": "2022-11-06 11:53:52",
      "lastStatusChangeTime": "2022-12-14 16:37:31",
      "lastStatusEvent": "Stop",
      "lastStopTime": "2022-12-14 16:37:31",
      "status": "STOPPED"
    },
    "name": "JobServer",
    "numberOfInstancesDown": 1,
    "numberOfInstancesRunning": 0,
    "outageCountInLast24Hrs": 1,
    "pecentageUpInstance": 0,
    "status": "Red",
    "totalNumberOfInstances": 1
  }
}

Queries:[[{status eq STOPPED}]]

2022/12/14 20:48:31 map[autoRestartSuspended:false autoRestartSuspendedForApp:false autoRestartableApp:false instance:map[active:true applicationDisplayName:JobServer applicationId:42 applicationName:JobServer custom:false id:10 name:1 serverName:xxx] lastStartTime:2022-11-06 11:53:52 lastStatusChangeTime:2022-12-14 16:37:31 lastStatusEvent:Stop lastStopTime:2022-12-14 16:37:31 status:STOPPED]

Changing where to eq produces exactly same output. I would think that neq should return an empty map instead?

Another test with neq:

notStopped := jq.New().File("out.json").
	From("appStatus").
	From("instanceStatus.instance").
	Where("name", "!=", 1)

Returns:

map[active:true applicationDisplayName:JobServer applicationId:42 applicationName:JobServer custom:false id:10 name:1 serverName:xxx]

So it would appear that neq and != do not work as expected?

Any help in getting this working? Am I doing smth wrong?

PS: is this lib maintained? Seems like not much activity going on lately.

Added support for:

1. Reading JSON byteArray - Decode byte array in to root content
2. Default values - Support to provide default value for any given non-existing property
3. Object Processing along with JSON Array - Support to have filter & projection on a single JSON object along with the JSON Array

https://github.com/thedevsaddam/gojsonq/issues/99

Can we add support for the

1. Reading JSON byteArray - Decode byte array in to root content
2. Default values - Support to provide default value for any given non-existing property
3. Object Processing along with JSON Array - Support to have filter & projection on a single JSON object along with the JSON Array

a unit test is failing when compiling with golang1.19

 % go version
go version go1.19 darwin/amd64

% make test
go test -timeout 60s  ./...
--- FAIL: TestJSONQ_SortBy_float_ascending_order (0.00s)
    jsonq_testdata_test.go:134: Tag: sorting array of object by its key (price-float64) in ascending desc
        Expected: [{"id":null,"name":"HP core i3 SSD","price":850},{"id":4,"name":"Fujitsu","price":850},{"id":5,"key":2300,"name":"HP core i5","price":850},{"id":6,"name":"HP core i7","price":950},{"id":3,"name":"Sony VAIO","price":1200},{"id":1,"name":"MacBook Pro 13 inch retina","price":1350},{"id":2,"name":"MacBook Pro 15 inch retina","price":1700}]
        Got: [{"id":4,"name":"Fujitsu","price":850},{"id":5,"key":2300,"name":"HP core i5","price":850},{"id":null,"name":"HP core i3 SSD","price":850},{"id":6,"name":"HP core i7","price":950},{"id":3,"name":"Sony VAIO","price":1200},{"id":1,"name":"MacBook Pro 13 inch retina","price":1350},{"id":2,"name":"MacBook Pro 15 inch retina","price":1700}]
FAIL
FAIL	github.com/thedevsaddam/gojsonq/v2	0.353s
FAIL

Supposed that we have list of objects which have internalScore and externalScore. OrderBy can only sort one field only, either those two. If there are 2 records having the same internalScore, when it's already sorted by internalScore, then externalScore cannot be sorted. So, at the moment OrderBy cannot do this.

I wish it allowed us to call OrderBy multiple times which adds up instead of overriding it.

It's similar to SQL functionality:

SELECT * FROM tbl_Employee ORDER BY lastName ASC, Salary DESC