Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel

Go Report Card PkgGoDev Test Build codecov Mentioned in Awesome Go GitHub All Releases Downloads GitHub License GitHub tag (latest by date)

Dasel (short for data-selector) allows you to query and modify data structures using selector strings.

Comparable to jq / yq, but supports JSON, YAML, TOML, XML and CSV with zero runtime dependencies.

One tool to rule them all

Say good bye to learning new tools just to work with a different data format.

Dasel uses a standard selector syntax no matter the data format. This means that once you learn how to use dasel you immediately have the ability to query/modify any of the supported data types without any additional tools or effort.

Update Kubernetes Manifest

Issue vs Discussion

I have enabled discussions on this repository.

I am aware there may be some confusion when deciding where you should communicate when reporting issues, asking questions or raising feature requests so this section aims to help us align on that.

Please raise an issue if:

  • You find a bug.
  • You have a feature request and can clearly describe your request.

Please open a discussion if:

  • You have a question.
  • You're not sure how to achieve something with dasel.
  • You have an idea but don't quite know how you would like it to work.
  • You have achieved something cool with dasel and want to show it off.
  • Anything else!

Features

Table of contents

Playground

You can test out dasel commands using the playground.

Source code for the playground can be found at github.com/TomWright/daselplayground.

Installation

You can import dasel as a package and use it in your applications, or you can use a pre-built binary to modify files from the command line.

Docker

Run dasel in docker using the image ghcr.io/tomwright/dasel.

echo '{"name": "Tom"}' | docker run -i --rm ghcr.io/tomwright/dasel:latest -p json '.name'
"Tom"

Docker images are pushed to the github container repository: tomwright/dasel.

If you want to use a specific version of dasel simply change latest to the desired version.

  • latest - The latest released version.
  • dev - The latest build from master branch.
  • v*.*.* - The build from the given release.

ASDF (Cross Platform)

Using asdf-vm and the asdf-dasel plugin

asdf plugin add dasel https://github.com/asdf-community/asdf-dasel.git
asdf list all dasel
asdf install dasel <version>
asdf global dasel <version>

Manual

You can download a compiled executable from the latest release.

Note: Don't forget to put the binary somewhere in your PATH.

Linux

This one liner should work for you - be sure to change the targeted release executable if needed. It currently targets dasel_linux_amd64.

curl -s https://api.github.com/repos/tomwright/dasel/releases/latest | grep browser_download_url | grep linux_amd64 | cut -d '"' -f 4 | wget -qi - && mv dasel_linux_amd64 dasel && chmod +x dasel
mv ./dasel /usr/local/bin/dasel

MacOS

You may have to brew install wget in order for this to work.

curl -s https://api.github.com/repos/tomwright/dasel/releases/latest | grep browser_download_url | grep darwin_amd64 | cut -d '"' -f 4 | wget -qi - && mv dasel_darwin_amd64 dasel && chmod +x dasel
mv ./dasel /usr/local/bin/dasel

Windows 10

The most convenient installation method is via scoop command-line installer. Issue the following commands in your terminal and dasel will be available:

scoop bucket add extras
scoop install dasel

You can then use

scoop update dasel

to update later on.

Self Update

If you have dasel installed you can easily upgrade to the latest release using:

dasel update

If you have a development version of dasel you should run the following:

dasel update --dev

Note: This is a dasel feature and will cause the current executable to be replaced. This may cause side effects when used in conjuction with a package manager.

Development Version

You can go get the main package to build and install dasel for you.

go get github.com/tomwright/dasel/cmd/[email protected]

You may need to prefix the command with GO111MODULE=on in order for this to work.

Note that doing this will set the version to development

Using dasel as a go package

As with any other go package, just use go get.

go get github.com/tomwright/dasel

Once imported you do something like the following:

package main
import (
    "encoding/json"
    "fmt"
    "github.com/tomwright/dasel"
)

func main() {
    var data interface{}
    _ = json.Unmarshal([]byte(`[{"name": "Tom"}, {"name": "Jim"}]`), &data)

    rootNode := dasel.New(data)

    result, _ := rootNode.Query(".[0].name")
    printNodeValue(result) // Tom
    
    results, _ := rootNode.QueryMultiple(".[*].name")
    printNodeValue(results...) // Tom\nJim

    _ = rootNode.Put(".[0].name", "Frank")
    printNodeValue(rootNode) // [map[name:Frank] map[name:Jim]]

    _ = rootNode.PutMultiple(".[*].name", "Joe")
    printNodeValue(rootNode) // [map[name:Joe] map[name:Joe]]
    
    outputBytes, _ := json.Marshal(rootNode.InterfaceValue())
    fmt.Println(string(outputBytes)) // [{"name":"Joe"},{"name":"Joe"}]
}

func printNodeValue(nodes ...*dasel.Node) {
    for _, n := range nodes {
        fmt.Println(n.InterfaceValue())
    }
}

From then on the rest of the docs should be enough.

Just know that when using the command-line tool the -m,--multiple flag tells dasel to use QueryMultiple/PutMultiple instead of Query/Put.

If the information provided here isn't good enough please open a discussion.

Notes

Preserved formatting and ordering

The formatting of files can be changed while being processed. Dasel itself doesn't make these changes, rather the act of marshaling the results.

In short, the output files may have properties in a different order but the actual contents will be as expected.

Memory usage

Dasel's method of querying data requires that the entire input document is stored in memory.

You should keep this in mind as the maximum filesize it can process will be limited by your system's available resources (specifically RAM).

Converting between formats

Dasel allows you to specify different input/output formats using the -r,--read and -w,--write flags.

E.g.

echo '{"name": "Tom"}{"name": "Jim"}' | dasel -r json -w yaml .
name: Tom
---
name: Jim

This works well in general but you may run into issues when converting between data formats that don't typically play well together.

If you have any questions or concerns around this please raise a discussion.

Usage

dasel -h

An important note is that if no sub-command is given, dasel will default to select.

Select

dasel select -f <file> -p <parser> -r <read_parser> -w <write_parser> -m <selector>

Arguments

-f, --file

Specify the file to query. This is required unless you are piping in data.

If piping in data you can optionally pass -f stdin/-f -.

-r, --read

Specify the parser to use when reading the input data.

This is required if you are piping in data, otherwise dasel will use the given file extension to guess which parser to use.

See supported parsers.

-w, --write

Specify the parser to use when writing the output data.

If not provided dasel will attempt to use the --out and --read flags to determine which parser to use.

See supported parsers.

-p, --parser

Shorthand for -r <value> -w <value>

-m, --multiple

Tells dasel to select multiple items.

This causes the dynamic selector to return all matching results rather than the first, and enables the any index selector.

All matches will be output on a new line.

E.g.

echo '[{"name": "Tom"}, {"name": "Jim"}]' | dasel -p json -m '.[*].name'
"Tom"
"Jim"
-s, --selector, <selector>

Specify the selector to use. See Selectors for more information.

If no selector flag is given, dasel assumes the first argument given is the selector.

This is required.

--plain

By default, dasel formats the output using the specified parser.

If this flag is used no formatting occurs and the results output as a string.

-n, --null

This flag tells dasel to output null instead of ValueNotFound errors.

With the flag:

$ echo '[1]' | dasel -p json -n '.[1]'
null

Without the flag:

$ echo '[1]' | dasel -p json '.[1]'   
Error: could not query node: could not find value: no value found for selector: .[1]: [1]
Usage:
  dasel select -f <file> -p <json,yaml> -s <selector> [flags]

Flags:
  -f, --file string       The file to query.
  -h, --help              help for select
  -m, --multiple          Select multiple results.
  -n, --null              Output null instead of value not found errors.
  -p, --parser string     Shorthand for -r FORMAT -w FORMAT.
      --plain             Alias of -w plain
  -r, --read string       The parser to use when reading.
  -s, --selector string   The selector to use when querying the data structure.
  -w, --write string      The parser to use when writing.

Error: could not write custom output: could not query node: could not find value: no value found for selector: .[1]: [1]
exit status 1
-c, --compact

This tells dasel to output compact data where possible. E.g. not pretty printing JSON.

Example

Select the image within a kubernetes deployment manifest file:
dasel select -f deployment.yaml "spec.template.spec.containers.(name=auth).image"
"tomwright/auth:v1.0.0"
Piping data into the select:
cat deployment.yaml | dasel select -p yaml "spec.template.spec.containers.(name=auth).image"
"tomwright/auth:v1.0.0"

Put

dasel put <type> -f <file> -o <out> -p <parser> -m <selector> <value>
echo "name: Tom" | ./dasel put string -p yaml "name" Jim
name: Jim

Arguments

type

The type of value you want to put.

Available arguments:

-f, --file

Specify the file to query. This is required unless you are piping in data.

If piping in data you can optionally pass -f stdin/-f -.

-o, --out

Specify the output file. If present, results will be written to the given file. If not present, results will be written to the input file (or stdout if none given).

To force output to be written to stdout, pass -o stdout/-o -.

-r, --read

Specify the parser to use when reading the input data.

This is required if you are piping in data, otherwise dasel will use the given file extension to guess which parser to use.

See supported parsers.

-w, --write

Specify the parser to use when writing the output data.

If not provided dasel will attempt to use the --out and --read flags to determine which parser to use.

See supported parsers.

-p, --parser

Shorthand for -r <value> -w <value>

-m, --multiple

Tells dasel to put multiple items.

This causes the dynamic selector to return all matching results rather than the first, and enables the any index selector.

E.g.

echo '[{"name": "Tom"}, {"name": "Jim"}]' | dasel put string -p json -m '.[*].name' Frank
[
  {
    "name": "Frank"
  },
  {
    "name": "Frank"
  }
]
-s, --selector, <selector>

Specify the selector to use. See Selectors for more information.

If no selector flag is given, dasel assumes the first argument given is the selector.

This is required.

-c, --compact

This tells dasel to output compact data where possible. E.g. not pretty printing JSON.

value

The value to write.

Dasel will parse this value as a string, int, or bool from this value depending on the given type.

This is required.

Creating properties

When putting data dasel will create items if they don't exist.

You can create an entire record from scratch by piping in an empty record, and then piping dasel commands together.

echo '' | dasel put string -p yaml -s '.propa' A | dasel put string -p yaml -s '.propb' B
propa: A
propb: B

This can be used to change multiple values or to create an entire document.

Put Object

Putting objects works slightly differently to a standard put, but the same principles apply.

dasel put object -f <file> -o <out> -p <parser> -m -t <type> <selector> <values>

If you want to create an empty object just omit the type flag and the values.

Arguments

-t, --type

The type of value you want to put.

You must repeat this argument for each value provided.

Available arguments:

  • string
  • int
  • bool
-f, --file

Specify the file to query. This is required unless you are piping in data.

If piping in data you can optionally pass -f stdin/-f -.

-o, --out

Specify the output file. If present, results will be written to the given file. If not present, results will be written to the input file (or stdout if none given).

To force output to be written to stdout, pass -o stdout/-o -.

-r, --read

Specify the parser to use when reading the input data.

This is required if you are piping in data, otherwise dasel will use the given file extension to guess which parser to use.

See supported parsers.

-w, --write

Specify the parser to use when writing the output data.

If not provided dasel will attempt to use the --out and --read flags to determine which parser to use.

See supported parsers.

-p, --parser

Shorthand for -r <value> -w <value>

-m, --multiple

Tells dasel to put multiple items.

This causes the dynamic selector to return all matching results rather than the first, and enables the any index selector.

E.g.

echo '[{"name": "Tom"}, {"name": "Jim"}]' | dasel put object -p json -m -t string '.[*]' 'name=Frank'
[
  {
    "name": "Frank"
  },
  {
    "name": "Frank"
  }
]
-s, --selector, <selector>

Specify the selector to use. See Selectors for more information.

If no selector flag is given, dasel assumes the first argument given is the selector.

This is required.

-c, --compact

This tells dasel to output compact data where possible. E.g. not pretty printing JSON.

values

A space-separated list of key=value pairs.

Dasel will parse each value as a string, int, or bool depending on the related type.

Example

echo "" | dasel put object -p yaml -t string -t int "my.favourites" colour=red number=3

Results in the following:

my:
  favourites:
    colour: red
    number: 3

Put Document

You are able to put entire documents using put document.

dasel put document -f <file> -o <out> -p <parser> -m <selector> -d <document-parser> <document>

Arguments

-f, --file

Specify the file to query. This is required unless you are piping in data.

If piping in data you can optionally pass -f stdin/-f -.

-o, --out

Specify the output file. If present, results will be written to the given file. If not present, results will be written to the input file (or stdout if none given).

To force output to be written to stdout, pass -o stdout/-o -.

-d, --document-parser, <document-parser>

Specify the parser to use when reading the document value.

If no value is provided, the read parser is used.

See supported parsers.

-r, --read

Specify the parser to use when reading the input data.

This is required if you are piping in data, otherwise dasel will use the given file extension to guess which parser to use.

See supported parsers.

-w, --write

Specify the parser to use when writing the output data.

If not provided dasel will attempt to use the --out and --read flags to determine which parser to use.

See supported parsers.

-p, --parser

Shorthand for -r <value> -w <value>

-m, --multiple

Tells dasel to put multiple items.

This causes the dynamic selector to return all matching results rather than the first, and enables the any index selector.

-s, --selector, <selector>

Specify the selector to use. See Selectors for more information.

If no selector flag is given, dasel assumes the first argument given is the selector.

This is required.

-c, --compact

This tells dasel to output compact data where possible. E.g. not pretty printing JSON.

document, <document>

The document you want to put, as a string.

This is required.

Example

echo '{"people":[]}' | dasel put document -p json -d yaml '.people.[]' 'name: Tom
colours:
- red
- green
- blue'

Results in the following:

{
  "people": [
    {
      "colours": [
        "red",
        "green",
        "blue"
      ],
      "name": "Tom"
    }
  ]
}

Supported file types

Dasel attempts to find the correct parser for the given file type, but if that fails you can choose which parser to use with the -p or --parser flag.

JSON

-p json

Using golang.org/pkg/encoding/json.

Multi-document files

Multi-document files are decoded into an array, with [0] being the first document, [1] being the second and so on.

Once decoded, you can access them using any of the standard selectors provided by Dasel.

TOML

-p toml

Using github.com/pelletier/go-toml.

YAML

-p yaml

Using gopkg.in/yaml.v2.

Multi-document files

Multi-document files are decoded into an array, with [0] being the first document, [1] being the second and so on.

Once decoded, you can access them using any of the standard selectors provided by Dasel.

XML

-p xml

Using github.com/clbanning/mxj.

XML Documents

XML documents within dasel are stored as a map of values.

This is just how dasel stores data and is required for the general functionality to work. An example of a simple documents representation is as follows:

<Person active="true">
  <Name main="yes">Tom</Name>
  <Age>27</Age>
</Person>
map[
  Person:map[
    -active:true
    Age:27
    Name:map[
      #text:Tom
      -main:true
    ]
  ]
]

In general this won't affect you, but on the odd occasion in specific instances it could lead to unexpected output.

If you are struggling with this please open a discussion for support. This will also help me know when the docs aren't sufficient.

Debugging

You can run select commands with the --plain flag to see the raw data that is stored within dasel. This can help you figure out the exact properties you may need to target when it isn't immediately obvious.

Arrays/Lists

Due to the way that XML is decoded, dasel can only detect something as a list if there are at least 2 items.

If you try to use list selectors (dynamic, index, append) when there are less than 2 items in the list you will get an error.

There are no plans to introduce a workaround for this but if there is enough demand it may be worked on in the future.

CSV

-p csv

Using golang.org/pkg/encoding/csv.

Plain

-p plain

This outputs the data using fmt.Sprint(x), displaying whatever underlying value is present as a string.

Adding data

New columns will be detected and added to the end of the CSV output.

Column deletion is not supported.

Selectors

Selectors define a path through a set of data.

Selectors are made up of different parts separated by a dot ., each part being used to identify the next node in the chain.

The following YAML data structure will be used as a reference in the following examples.

name: Tom
preferences:
  favouriteColour: red
colours:
- red
- green
- blue
colourCodes:
- name: red
  rgb: ff0000
- name: green
  rgb: 00ff00
- name: blue
  rgb: 0000ff

You can escape values in selectors using a backslash \. The main use for this is to allow you to target fields that contain a dot or space in their name.

Property

Property selectors are used to reference a single property of an object.

Just use the property name as a string.

dasel select -f ./tests/assets/example.yaml -s "name"
Tom
  • name == Tom

Keys and Indexes

You can use the property selector with a value of - to return a list of all the keys/indexes in the current node.

echo '{"a":{"c": [1, 2, 3]},"b":{}}' | dasel -p json -m '.a.c.-'
"0"
"1"
"2"

This must be used in conjunction with -m,--multiple and cannot be used in put commands.

Child Elements

Just separate the child element from the parent element using a .:

dasel select -f ./tests/assets/example.yaml -s "preferences.favouriteColour"
red
  • preferences.favouriteColour == red

Index

When you have a list, you can use square brackets to access a specific item in the list by its index.

dasel select -f ./tests/assets/example.yaml -s "colours.[1]"
green
  • colours.[0] == red
  • colours.[1] == green
  • colours.[2] == blue

Next Available Index

The next available index selector is used when adding to a list of items. It allows you to append to a list.

  • colours.[]

Any Index

The any index selector is used to select all items of a list or map.

  • colours.[*]

This must be used in conjunction with -m,--multiple.

Dynamic

Dynamic selectors are used with lists/maps when you don't know the index/property of the item, but instead want to find the index based on some other criteria.

Dasel currently supports key/query=value checks but I aim to support more check types in the future.

Look ups are defined in brackets. You can use multiple dynamic selectors within the same part to perform multiple checks.

dasel select -f ./tests/assets/example.yaml -s "colourCodes.(name=red).rgb"
ff0000

dasel select -f ./tests/assets/example.yaml -s "colourCodes.(name=blue)(rgb=0000ff)"
map[name:blue rgb:0000ff]
  • colourCodes.(name=red).rgb == ff0000
  • colourCodes.(name=green).rgb == 00ff00
  • colourCodes.(name=blue).rgb == 0000ff
  • colourCodes.(name=blue)(rgb=0000ff).rgb == 0000ff

If you want to refer to the value of a non-object value in a list, just define the key as value or ., meaning the current value. This may look something like (value=2).

Using queries in dynamic selectors

When performing a check dasel creates a new root node at the current position and then selects data using the given key as the query.

This allows you to perform complex queries such as...

echo `{
  "users": [
    {
      "name": {
        "first": "Tom",
        "last": "Wright"
      },
      "addresses": [
        {
          "primary": true,
          "number": 123
        },
        {
          "primary": false,
          "number": 456
        }
      ]
   }
  ]
}` | dasel -p json '.users.(.addresses.(.primary=true).number=123).name.first'
"Tom"

The above query in plain English may read as...

Give me the first name of the user who's primary address is at number 123

The resolution of that query looks something like this:

.users.(.addresses.(.primary=true).number=123).name.first
.users.(.addresses.[0].number=123).name.first
.users.[0].name.first

Search

Search selectors recursively search all the data below the current node and returns all the results - this means they can only be used in multi select/put commands.

The syntax is as follows:

.(?:key=value)

If key is:

  • . or value - dasel checks if the current nodes value is value.
  • - or keyValue - dasel checks if the current nodes key/name/index value is value.
  • Else dasel uses the key as a selector itself and compares the result against value.

Search Example

{
  "users": [
    {
      "primary": true,
      "name": {
        "first": "Tom",
        "last": "Wright"
      }
    },
    {
      "primary": false,
      "extra": {
        "name": {
          "first": "Joe",
          "last": "Blogs"
        }
      },
      "name": {
        "first": "Jim",
        "last": "Wright"
      }
    }
  ]
}

Search for all objects with a key of name and output the first name of each:

dasel -p json -m '.(?:-=name).first'
"Tom"
"Joe"
"Jim"

Search for all objects with a last name of Wright and output the first name of each:

dasel -p json -m '.(?:name.last=Wright).name.first'
"Tom"
"Jim"

Examples

General

Filter JSON API results

The following line will return the download URL for the latest macOS dasel release:

curl https://api.github.com/repos/tomwright/dasel/releases/latest | dasel -p json --plain '.assets.(name=dasel_darwin_amd64).browser_download_url'

jq to dasel

The follow examples show a set of jq commands and the equivalent in dasel.

Select a single value

echo '{"name": "Tom"}' | jq '.name'
"Tom"

echo '{"name": "Tom"}' | dasel -p json '.name'
"Tom"

Select a nested value

echo '{"user": {"name": "Tom", "age": 27}}' | jq '.user.age'
27

echo '{"user": {"name": "Tom", "age": 27}}' | dasel -p json '.user.age'
27

Select an array index

echo '[1, 2, 3]' | jq '.[1]'
2

echo '[1, 2, 3]' | dasel -p json '.[1]'
2

Append to an array of strings

echo '["a", "b", "c"]' | jq '. += ["d"]'
[
  "a",
  "b",
  "c",
  "d"
]

echo '["a", "b", "c"]' | dasel put string -p json -s '.[]' d
[
  "a",
  "b",
  "c",
  "d"
]

Update a string value

echo '["a", "b", "c"]' | jq '.[1] = "d"'
[
  "a",
  "d",
  "c"
]

echo '["a", "b", "c"]' | dasel put string -p json -s '.[1]' d
[
  "a",
  "d",
  "c"
]

Update an int value

echo '[1, 2, 3]' | jq '.[1] = 5'
[
  1,
  5,
  3
]

echo '[1, 2, 3]' | dasel put int -p json -s '.[1]' 5
[
  1,
  5,
  3
]

Overwrite an object

echo '{"user": {"name": "Tom", "age": 27}}' | jq '.user = {"name": "Frank", "age": 25}'
{
  "user": {
    "name": "Frank",
    "age": 25
  }
}

echo '{"user": {"name": "Tom", "age": 27}}' | dasel put object -p json -s '.user' -t string -t int name=Frank age=25
{
  "user": {
    "age": 25,
    "name": "Frank"
  }
}

Append to an array of objects

echo '{"users": [{"name": "Tom"}]}' | jq '.users += [{"name": "Frank"}]'
{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "Frank"
    }
  ]
}

echo '{"users": [{"name": "Tom"}]}' | dasel put object -p json -s '.users.[]' -t string name=Frank
{
  "users": [
    {
      "name": "Tom"
    },
    {
      "name": "Frank"
    }
  ]
}

yq to dasel

The follow examples show a set of yq commands and the equivalent in dasel.

Select a single value

echo 'name: Tom' | yq '.name'
"Tom"

echo 'name: Tom' | dasel -p yaml '.name'
Tom

Select a nested value

echo 'user:
  name: Tom
  age: 27' | yq '.user.age'
27

echo 'user:
       name: Tom
       age: 27' | dasel -p yaml '.user.age'
27

Select an array index

echo '- 1
- 2
- 3' | yq '.[1]'
2

echo '- 1
- 2
- 3' | dasel -p yaml '.[1]'
2

Append to an array of strings

echo '- a
- b
- c' | yq --yaml-output '. += ["d"]'
- a
- b
- c
- d

echo '- a
- b
- c' | dasel put string -p yaml -s '.[]' d
- a
- b
- c
- d

Update a string value

echo '- a
- b
- c' | yq --yaml-output '.[1] = "d"'
- a
- d
- c

echo '- a
- b
- c' | dasel put string -p yaml -s '.[1]' d
- a
- d
- c

Update an int value

echo '- 1
- 2
- 3' | yq --yaml-output '.[1] = 5'
- 1
- 5
- 3

echo '- 1
- 2
- 3' | dasel put int -p yaml -s '.[1]' 5
- 1
- 5
- 3

Overwrite an object

echo 'user:
  name: Tom
  age: 27' | yq --yaml-output '.user = {"name": "Frank", "age": 25}'
user:
  name: Frank
  age: 25


echo 'user:
  name: Tom
  age: 27' | dasel put object -p yaml -s '.user' -t string -t int name=Frank age=25
user:
  age: 25
  name: Frank

Append to an array of objects

echo 'users:
- name: Tom' | yq --yaml-output '.users += [{"name": "Frank"}]'
users:
  - name: Tom
  - name: Frank


echo 'users:
- name: Tom' | dasel put object -p yaml -s '.users.[]' -t string name=Frank
users:
- name: Tom
- name: Frank

Kubernetes

The following should work on a kubernetes deployment manifest. While kubernetes isn't for everyone, it does give some good example use-cases.

Select the image for a container named auth

dasel select -f deployment.yaml -s "spec.template.spec.containers.(name=auth).image"
tomwright/x:v2.0.0

Change the image for a container named auth

dasel put string -f deployment.yaml -s "spec.template.spec.containers.(name=auth).image" "tomwright/x:v2.0.0"

Update replicas to 3

dasel put int -f deployment.yaml -s "spec.replicas" 3

Add a new env var

dasel put object -f deployment.yaml -s "spec.template.spec.containers.(name=auth).env.[]" -t string -t string name=MY_NEW_ENV_VAR value=MY_NEW_VALUE

Update an existing env var

dasel put string -f deployment.yaml -s "spec.template.spec.containers.(name=auth).env.(name=MY_NEW_ENV_VAR).value" NEW_VALUE

XML Examples

XML has some slight differences (such as attributes) that should be documented.

Query attributes

Decoded attributes are set as properties on the related object with a prefix of -.

echo '<data>
    <users primary="true">
        <name>Tom</name>
    </users>
    <users primary="false">
        <name>Frank</name>
    </users>
</data>' | dasel -p xml '.data.users[0].-primary'
true

Filtering on attributes

We can also filter on attributes since they are defined against the related object.

echo '<data>
    <users primary="true">
        <name>Tom</name>
    </users>
    <users primary="false">
        <name>Frank</name>
    </users>
</data>' | dasel -p xml '.data.users.(-primary=true).name'
Tom

Benchmarks

In my tests dasel has been up to 3x faster than jq and 15x faster than yq.

See the benchmark directory.

Comments
  • How do I create a new yaml document

    How do I create a new yaml document

    Hello @TomWright ,

    Small question, but I am unable to figure it out, how do I create this yaml output?

    metadata:
      annotations:
         node.longhorn.io/default-disks-config: '[ { "name":"fast",  "path":"/mnt/data-fast1", "allowScheduling":true, "tags":["fast"]}, { "name":"slow",  "path":"/mnt/data-slow1", "allowScheduling":true, "tags":["slow"]} ]'
      labels:
        node.longhorn.io/create-default-disk: config
    

    Thanks!

  • Add file read/write support to Go package

    Add file read/write support to Go package

    Add helper functions/receivers to make it easer to read/write files.

    Functions

    • NewFromFile function which accepts filename and parser strings.
    • NewFromReader function which accepts Reader and parser strings.

    Receivers

    • WriteFile receiver which accepts filename, parser string and compact / escapeHTML options.
    • Write receiver which accepts Writer, parser string and compact / escapeHTML options.

    Misc

    • Fix parser descriptions

    Closes #187

  • support for a validation command

    support for a validation command

    Is your feature request related to a problem? Please describe. dasel is awesome! since i love it too much and although dasel is not a validator, i do use it as such.

    Describe the solution you'd like i would love to see an option to provide dasel with a list of files (of any supported types), and have dasel report whether any error is found. e.g.

    $ dasel validate dir/foo.json dir/foo.yaml
    pass dir/foo.json
    fail dir/foo.yaml
    $ echo $?
    1
    
    $ dasel validate 'dir/**/*.{json,yaml}'
    pass dir/foo.json
    fail dir/foo.yaml
    $ echo $?
    1
    

    Describe alternatives you've considered wrapping dasel with a helper

    find . -type f -name '*.yaml' -o -name '*.json' | while read f; do dasel -f $f; done
    
  • support for a validation pre-commit hooks

    support for a validation pre-commit hooks

    Pre-commit hooks for dasel validate.

    Notes

    1. The version specified in the documentation (v1.25.1) refers to a new release that includes .pre-commit-hooks.yaml
    2. ~The version specified in .pre-commit-config.yaml (master) will work only after merging this change. For testing (see below) the commit hash for this pull-request should be used (e.g. https://github.com/TomWright/dasel/pull/232/commits/1217e03c614c14d0c98b0a7cdc5c65e93132b378). This file is optional, and needed only if pre-commit is not in used for maintaining dasel project~
    3. ~I couldn't find the source code for the documentation files to update them accordingly, and that is the reason that the link in the Features section is left empty. If you will point me to them, I would love to make a contribution~

    Tests

    Execute pre-commit within the repository (added for sanity tests)

    $ pre-commit run -a dasel-validate --verbose
    Validate JSON, YAML, XML, TOML...........................................Failed
    - hook id: dasel-validate
    - duration: 0.04s
    - exit code: 1
    
    fail tests/assets/broken.json could not load input: could not unmarshal data: invalid character '}' after array element
    pass benchmark/data.json
    pass benchmark/data.yaml
    pass .github/workflows/codeql-analysis.yml
    Error: 1 files failed validation
    pass tests/assets/example.yaml
    pass .github/FUNDING.yml
    pass .github/workflows/build-test.yaml
    pass benchmark/data/nested_property.json
    pass benchmark/data/root_object.json
    pass benchmark/data/delete_property.json
    pass .github/workflows/build.yaml
    pass tests/assets/deployment.yaml
    pass tests/assets/example.json
    pass .github/workflows/build-dev.yaml
    pass benchmark/data/array_index.json
    pass benchmark/data/list_array_keys.json
    pass .github/workflows/bump-homebrew.yaml
    pass benchmark/data/top_level_property.json
    pass .pre-commit-hooks.yaml
    pass tests/assets/example.xml
    fail tests/assets/broken.xml could not load input: could not unmarshal data: xml.Decoder.Token() - XML syntax error on line 1: element <a> closed by </b>
    pass benchmark/data/append_array_of_strings.json
    pass .github/workflows/test.yaml
    pass codecov.yaml
    Error: 1 files failed validation
    pass benchmark/data/overwrite_object.json
    pass benchmark/data/update_string.json
    

    cc @TomWright

  • Can not set negative integer value in command line

    Can not set negative integer value in command line

    Describe the bug Can not set negative value in command line , it'll treated as a command line flag rather than a value, and the escape character does not work.

    To Reproduce Steps to reproduce the behavior:

    #  dasel put int -p toml -f /etc/containerd/config.toml '.oom_score' '-900'
    Error: unknown shorthand flag: '9' in -900
    
    #  dasel put int -p toml -f /etc/containerd/config.toml '.oom_score' "\-900"
    Error: could not parse int [\-900]: strconv.ParseInt: parsing "\\-900": invalid syntax
    Usage:
      dasel put int -f <file> -s <selector> <value> [flags]
    
    Flags:
      -h, --help   help for int
    
    Global Flags:
      -f, --file string       The file to query.
      -m, --multiple          Select multiple results.
      -o, --out string        Output destination.
      -p, --parser string     Shorthand for -r FORMAT -w FORMAT.
      -r, --read string       The parser to use when reading.
      -s, --selector string   The selector to use when querying the data structure.
      -w, --write string      The parser to use when writing.
    
    Error: could not parse int [\-900]: strconv.ParseInt: parsing "\\-900": invalid syntax
    

    Desktop (please complete the following information):

    • OS: linux/ubuntu 18.06
    • Version: dasel version v1.11.0-2-gbe51ab2
  • PUT a multi-line file inside a YAML key

    PUT a multi-line file inside a YAML key

    Is your feature request related to a problem? Please describe.

    I could not find a way to edit a YAML file and insert a multi-line plain text document (e.g. a certificate in PEM file format) as a yaml key value (e.g. `-w yaml -s 'server.ssl.certificates.[0].certificate')

    Describe the solution you'd like

    dasel put string -f conf/config.yaml -w yaml -s 'server.ssl.certificates.[0].certificate' < ./certificates/certificate-chain.pem
    

    Had to find an alternative, used yq like this:

    cat ./config-template.yaml \
      | yq --yaml-roundtrip ".server.ssl.certificates[0].certificate = \"$(cat ./certificates/certificate-chain.pem)\"" \
      | yq --yaml-roundtrip ".server.ssl.certificates[0].privateKey = \"$(cat ./certificates/private.key)\"" \
      > ./conf/config.yaml
    
  • How do I do with this kind of error

    How do I do with this kind of error

    Describe the desired outcome Using dasel to select json file, but this error occur

    Error: could not load input: could not unmarshal config data: invalid character '{' after top-level value

    Resources My json file content is like this {"topic": "test", "device_info": {"device_name": "apple"}, "message": "failed"}

  • Question on replacing value(s) single and multiple occurrences

    Question on replacing value(s) single and multiple occurrences

    Hello,

    I was pointed out by someone that this tool was better to replace (multiple) values in a yaml file than using sed. I have tried some examples, but it is not clear to me how to replace certain values in https://raw.githubusercontent.com/c4po/harbor_exporter/master/kubernetes/harbor-exporter.yaml

    I need to replace multiple occurrences in this file, however, I am only able to select 1. How do I add a line with the namespace value?

    metadata:
      name: harbor-exporter
    
    

    to

    metadata:
      name: harbor-exporter
      namespace: test
    

    Next to this, I also need to replace the value of "- harbor #......" to test and keep the remaining lines and this is failing with dasel put string -f he.yaml -s "spec.namespaceSelector.matchNames" "test"

      namespaceSelector:
        matchNames:
        - harbor # change to the namespace where you deployed Harbor
    

    Thanks!

  • Support handling of Struct types

    Support handling of Struct types

    First of all, thank you for this awesome OSS @TomWright


    Is your feature request related to a problem? Please describe.

    I have the use case of using a package like Dasel programmatically to manipulate runtime objects.

    Without going into specific examples, runtime objects does not only have Slices and Maps, but also Structs.


    Describe the solution you'd like

    I'd like Dasel to also support Structs (that are very similar functionally to Maps) when doing operations.

    I've already made some local changes to dasel and it seems to work just fine. An example:

    If we add the following else if to findValueProperty (https://github.com/TomWright/dasel/blob/master/node_query.go#L66):

    if value.Kind() == reflect.Map {
    	...
    } else if value.Kind() == reflect.Struct {
    	fieldV := value.FieldByName(n.Selector.Property)
    	  
    	if fieldV.IsValid() {
    		return fieldV, nil
    	}
    	return nilValue(), &ValueNotFound{Selector: n.Selector.Current, PreviousValue: n.Previous.Value}
    }
    

    Then query is able to operate on interface{} that contain Structs.


    Describe alternatives you've considered

    I have an alternative private package that does similar functions to Dasel, and I'm in the ideation phase of open-sourcing it, and I would have liked to base it on Dasel, the only thing that stops me doing so ATM is the support for Structs.


    As a last note, I'll be happy to help in integrating such a change into Dasel and would like your opinion.

    Thanks!

  • Characters in strings getting replaced with unicode escape

    Characters in strings getting replaced with unicode escape

    Describe the bug < char in json strings is getting replaced with \u003c. Similar thing happens to >.

    To Reproduce Steps to reproduce the behavior:

    1. Create file with this content:
    {
      "name": "John Doe <[email protected]>"
    }
    
    1. dasel -f filename.json
    {
      "name": "John Doe \[email protected]\u003e"
    }
    

    Expected behavior No characters changed.

    Desktop (please complete the following information):

    • OS: macOS
    • Version 1.20.0
  • More examples

    More examples

    When trying to select from more complex files, I'm at a loss on what's possible, and keep failing at creating a proper query.

    Take the following yaml:

    ---
    plugins:
      - "@semantic-release/commit-analyzer"
      - "@semantic-release/release-notes-generator"
      - "@semantic-release/gitlab"
      - - "@semantic-release/git"
        - assets:
            - "tbump.toml"
            - "**/pyproject.toml"
            - "**/setup.py"
            - README.md
          message: |-
            chore(release): ${nextRelease.version}
    
            ${nextRelease.notes}
    

    I would like to retrieve the list under assets. How would I write a query to achieve this?

    Some more examples of all the selector features in the documenation would probably help achieve this.

    When converted to json, I can get it using dasel -f .releaserc -r json '.plugins.[3].[1].assets', but that would require a consistent ordering, which isn't guaranteed.

  • csv output converts to scientific notation

    csv output converts to scientific notation

    $ echo '{"name": 10000000000.1}' | dasel -r json -w csv name 1.00000000001e+10

    Converting the output numbers to scientific notation makes it difficult to use. Can't it just print the values like they come in.

  • Unexpected float, decimal output format

    Unexpected float, decimal output format

    Discussion? While using Dasel to convert a json format to csv containing float, decimal values, I see that the output is wrongly formated.

    Describe the bug Unexpected float, decimal output format

    To Reproduce dasel playground - latest version Arguments : -r json -w csv

    input json : { "x": 123456.1, "y": 1234567.1 } output csv : x,y 123456.1,1.2345671e+06

    Expected behavior x,y 123456.1,1234567.1 > Preserve original number format and do not convert into scientific format (exponential notation)

    Additional context I need to convert json containing WGS84 coordinates and scientific notation is not usable. I didn't found any option to force the output format.

    Thanks for help.

  • Support a terminal view and traversal path discovery

    Support a terminal view and traversal path discovery

    Is your feature request related to a problem? Please describe. When working with a big and non flat data stored in JSON/YAML/XML, it is not always comfortable to examine the data when it is expended automatically and/or find the JSON/YAML/XML traversal path (select query) to select the data

    Describe the solution you'd like It would be nice if dasel would support a terminal view (folding/expanding of nested object, coloring, etc) while assisting in displaying\discovering the traversal path which is examined, in the same spirit as fx does (see path discovery at the bottom)

    fx

    Describe alternatives you've considered There are other tools which has a support for these features. E.g. fx, though it is inconvenient to use other tools since:

    1. Each tool needs to be installed separately
    2. Each tool has its own traversal path syntax per a data structure

    Yet, there is no single tool like dasel, which supports the same traversal path (select query) for all supported data structures (file types)

    Additional context I assume that dasel uses the same data structure when parsing each type (JSON/YAML/XML/etc.), thus displaying this data structure on the terminal and be able to traverse it while discovering the traversal path might be a killer feature

  • Create Backup Before File Modification

    Create Backup Before File Modification

    Hi @TomWright,

    when I use dasel I like to create a backup if it modifies a file.

    Currently I use cp before running dasel to create the backup. But I would prefer that dasel takes care of it. What about implementing the following (global) commandline flags ?

     -b, --backup                  create a backup before writing the file
     --backup-suffix string        suffix to use for backup file (default .~)
    
  • Compact xml output

    Compact xml output

    Is your feature request related to a problem? Please describe

    I would love to see compact xml output. I see that compact -c works with json and others

    Describe the solution you'd like

    Removed line breaks and whitespace.

    Describe alternatives you've considered

    gsed -E 's/^\s+//' test.xml | tr -d '\n'

A simple command line for convert CSV in JSON
A simple command line for convert CSV in JSON

C2J A simple command line for convert CSV in JSON list of objects based on header. Install With Go 1.17 or higher: go install github.com/edermanoel94/

Jul 18, 2022
word2text - a tool is to convert word documents (DocX) to text on the CLI with zero dependencies for free
word2text - a tool is to convert word documents (DocX) to text on the CLI with zero dependencies for free

This tool is to convert word documents (DocX) to text on the CLI with zero dependencies for free. This tool has been tested on: - Linux 32bit and 64 bit - Windows 32 bit and 64 bit - OpenBSD 64 bit

Apr 19, 2021
A simple command line functionality to convert your Kaspersky Password Manager exported file to CSV format

A simple command line functionality to convert your Kaspersky Password Manager exported file to CSV format

Apr 20, 2022
Go package to make lightweight ASCII line graph ╭┈╯ in command line apps with no other dependencies.
Go package to make lightweight ASCII line graph ╭┈╯ in command line apps with no other dependencies.

asciigraph Go package to make lightweight ASCII line graphs ╭┈╯. Installation go get github.com/guptarohit/asciigraph Usage Basic graph package main

Sep 22, 2022
🔄 A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON

proto2yaml ?? A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON. Currently supported exports are for: Packages Service

Jul 25, 2022
Command line tool for processing client transaction data in CSV format

command line tool for processing client transaction data in CSV format. thank you for looking! build instructions: $ git clone https://github.com/lnit

Oct 29, 2021
From the command line, quickly explore data from a CSV file.
From the command line, quickly explore data from a CSV file.

shallow-explore From the command line, quickly explore data from a CSV file. shallow-explore is a Golang backed command-line tool for iterating over c

Aug 16, 2022
CLI for SendGrid, which helps in managing SSO users, can install and update users from yaml config

Sendgrid API This script is needed to add new users to SendGrid as SSO teammates. Previously, all users were manually added and manually migrating the

Jul 20, 2022
Command line XML beautifier and content extractor. Similar to jq.
Command line XML beautifier and content extractor. Similar to jq.

Command line XML beautifier and content extractor. Similar to jq.

Sep 24, 2022
Fonts is a package that provides helpers to access font details and easily retrive font bytes with ZERO dependencies

Fonts Fonts is a package that provides helpers to access font details and easily retrieve font bytes. This package has ZERO 3rd-party dependencies. Fo

Mar 3, 2022
Simple tool to convert a Parquet file to a CSV written in Go/ Golang

Parquet2CSV Parquet2CSV is a simple tool to convert a Parquet file to a CSV written in Go/ Golang Usage: parquet2csv <PATH_TO_PARQUET_FILE> This will

Apr 18, 2022
Program to convert plain text to CSV file which can imported into Anki.

Program to convert plain text to CSV file which can imported into Anki. The motivation of this program is to save time by automatically coverting Question and Answer into CSV file which can be imported directly into Anki.

May 22, 2022
Command-line tool to load csv and excel (xlsx) files and run sql commands
Command-line tool to load csv and excel (xlsx) files and run sql commands

csv-sql supports loading and saving results as CSV and XLSX files with data processing with SQLite compatible sql commands including joins.

Jul 7, 2022
Smart and simple CSV processing on the command line

csvquote smart and simple CSV processing on the command line Are you looking for a way to process CSV data with standard UNIX shell commands? Are you

Aug 29, 2022
Utility CLI to convert Spring Boot Yaml configuration into external configuration

boot-config-export Utility CLI to convert Spring Boot Yaml configuration into external configuration (as environment variables). The variables are tra

Nov 17, 2021
Json-match - Command line util for matching values in a JSON input

json-match Match JSON input by specifying key and value > json-match -json '{\"p

Jan 12, 2022
yq is a portable command-line YAML processor

yq a lightweight and portable command-line YAML processor. yq uses jq like syntax but works with yaml files as well as json. It doesn't yet support ev

Sep 27, 2022
Command-line tool to customize the official Spotify client. Supports Windows, MacOS and Linux.
Command-line tool to customize the official Spotify client. Supports Windows, MacOS and Linux.

Command-line tool to customize the official Spotify client. Supports Windows, MacOS and Linux. Features Change colors whole UI Inject CSS for advanced

Sep 20, 2022
PretGO - asic cli for format json,html and xml!
 PretGO - asic cli for format json,html and xml!

PretGO So basic cli for format json,html and xml! Table of contents Screenshots Setup Status Contact Screenshots Setup First clone project git clone h

Sep 2, 2022