gsheet is a CLI tool (and Golang package) for piping csv data to and from Google Sheets

gsheet

Introduction

gsheet is a cli tool for sending and receiving csv data to and from Google Sheets.

With gsheet you can:

  • Pipe csv data from stdin to a Google Sheet range
    cat data.csv | gsheet csv --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --range 'Sheet1'

  • Pipe csv data from a Google Sheet range to stdout
    gsheet csv --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --range "'Sheet!'A1:D20" > data.csv

  • Clear a Google Sheet range
    gsheet clear --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --range Sheet2

  • Create and delete sheets of a Spreadsheet document
    gsheet newSheet --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --name NewSheet

  • Sort a sheet by a column
    gsheet sort --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --name Sheet1 -c 2

Google Drive operations (with special handling for .csv):

  • Upload .csv files to Google Drive as Sheet documents
    gsheet upload --parent PARENT_ID ./path/to/data.csv

  • Download Google Sheet documents from Google Drive as .csv files
    gsheet download 2o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ > data.csv

  • Create a new empty folder
    gsheet createFolder --parent PARENT_ID 'New Folder Name'

You can also upload/delete and get info about arbitrary files on Google Drive, but gsheet isn’t trying to be a general purpose gdrive interface. If you need a full Google Drive command-line client, check out odeke-em/drive, or something that provides a filesystem interface like rclone.

For more on how to use gsheet see CLI Usage below.

gsheet supports authenticating with Google using service accounts, which is a simple way to allow scripts to access Google Sheets (see Authentication and Authorization below).

Why?

I often write scripts and cron jobs for clients that create tabular reports, and I’ve found that Google Sheets is a convenient way to expose them on the web (easy to use, built-in access control). In addition, I’ve found that setting up a Sheet with a few fields that my script can read is a convenient way to allow clients to configure apps without needing to build a web interface just to get a few runtime config values.

My typical workflow is to create a Sheet using Google’s web interface, set up the formatting and any formulas, and then use gsheet to update the data from scripts.

Installation

gsheet is not yet packaged for any package manager yet (let me know if you can help with that!), but you can download binaries for major platforms from the Releases page.

Otherwise, if you have Go installed you can:

# Install with go get
go get github.com/cristoper/gsheet/cmd/gsheet

# or build/install from git repo
git clone https://github.com/cristoper/gsheet.git
cd gsheet
go install ./cmd/gsheet

However, building from source will pull in the build dependencies (Google’s API SDKs) which are big (200MB+) so can be slow to download.

Authentication and Authorization

The hardest part about getting started with gsheet is creating the API credentials so that the program can access Google Drive, but it’s not so bad and you only have to do it once. In order for gsheet to read and update Sheets documents, it must 1) be provided credentials to authenticate with Google and 2) be granted access to whichever Google Drive folders/documents it should be able to read/write:

  1. Create a service account and download the credentials .json file to the computer you will use gsheet on. Set the GOOGLE_APPLICATION_CREDENTIALS envvar to the absolute path to that file before running gsheet.

    • From the Google Cloud API Dashboard create a new project and enable both the Google Drive API and the Google Sheets API on it. See Google’s Getting Started Guide for creating a project and enabling APIs.

    • From your project’s dashboard Click on "IAM & Admin" > "Service Accounts". Then click "Create Service Account". For the service account’s role I recommend "Basic>Editor".

    • Once the service account is created, click on it to manage its details. From the "Keys" tab click "Add Key" to create credentials in a .json file for the service account that gsheet will use to authenticate as the user.

    • Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to the .json file. This is how gsheet finds the credentials when it runs. NOTE: GOOGLE_APPLICATION_CREDENTIALS must containt the absolute path to the .json file. (See Google’s Authentication Overview.)

  2. In Google Drive grant share files and folders with the service account (the email address generated when you created the service account above) as if it were any other user. Note that if you only want to store files in the service account’s own drive (not viewable from the Google Drive web interface), then you don’t actually have to grant it access to any of your folders or documents.

What about OAuth authentication?

Currently gsheet only supports service accounts which, despite the convoluted instructions above, are simple to generate and use once you figure it out (you just need a single .json file with credentials to grant a script access to Google services).

However, if an OAuth workflow (where you can authenticate gsheet using your own Google account) would be useful to someone, I’m open to implementing it. Feel free to create an issue. In the mean time check out simon3z/gsheetcsv which is another simple cli tool in Go which allows interacting with Google Sheets and uses OAuth for authentication.

CLI Usage

To get an overview of all the commands provided by gsheet run:

gsheet help

You can also run gsheet help CMD to get help for each command. Below are some further usage hints.

Remember that for any of the commands to work you must have the GOOGLE_APPLICATION_CREDENTIALS environment variable set to a path containing the service account credentials in a .json file.

Sheet commands

csv and clear

The csv command is the heart of gsheet. If you pipe csv data to it on std input, it sends the data to the specified range of the Sheets document identified by the --id flag. If you don’t connect stdin to a pipe, then it will read the specified range and output it to stdout in csv format.

Note
csv does not clear the range before updating data in a Sheets document. If the piped data is smaller (fewer rows or columns) than the specified range, then any pre-existing data in the spreadsheet will remain after the update. Use gsheet clear to clear a range.
# Replace an entire sheet of a Spreadsheet doc with the contents of data.csv
gsheet --id SHEETS_DOC_ID clear --range Sheet1
cat data.csv | gsheet --id SHEETS_DOC_ID --range Sheet1

# Read a specific range of a sheet to output.csv
# (You can always single quote sheet names and include the exclamation point in
# the single quotes so that the shell doesn't try to interpret it.)
gsheet --id SHEETS_DOC_ID --range 'Sheet1!'A2:C5 > output.csv

sort

An existing sheet can be sorted by any (single) column in either descending (default) or ascending order:

# Sort sheet by B coloumn in ascending order
sort --id SHEET_NAME -name Sheet1 --column=1 --asc

newSheet and deleteSheet

These commands simply create and delete sheets from a spreadsheet document. The new sheets appear after all other visible sheets.

Note
sheets are deleted by name (the title of the sheet) and not by id; this is a bit fragile because if a user changes the title of a sheet in Google Docs then a script depending on gsheet deleteSheet may break.
# After running this you should see a new sheet called "SHEET_NAME" in the
# spreadsheet with id "SHEETS_DOC_ID"
gsheet newSheet --id SHEETS_DOC_ID --name SHEET_NAME

# After running this it should be gone again
gsheet deleteSheet --id SHEETS_DOC_ID --name SHEET_NAME

Drive commands

upload and download

The upload and download commands can be used to upload and download arbitrary files to Google Drive. They provide special handling for .csv files: uploading a .csv file will import it to Google Drive as a Sheets document, and downloading a Sheets document will export the first visible sheet as a .csv file.

Downloading any other Google Workspace document types will attempt to export them as plain text files.

# Upload data.csv as a Sheets document in the service account's root directory
gsheet upload --parent root data.csv

# Download an image from drive
# Note that download takes a single positional argument: the id of the google
# drive file to download, and it sends its output to stdout.
gsheet download DRIVE_DOC_ID > image.png

delete

The delete command can be used to delete one or more files by id (list each id as a positional argument). Outputs a confirmation as each file is deleted.

Note
delete immediately deletes a file and does not move it to the trash.

list

# List all files and their ids that are in the service account's root folder
gsheet list --parent root

createFolder

Sometimes it is nice if a script can create a new folder to keep all of its own files in. The output of the createFolder command includes the id of the created folder.

# Create a foler in service account's root (specify --parent to use a different
# folder)
$ gsheet createFolder FOLDER_NAME
Created directory named FOLDER_NAME with id 1ApMOHtZtTVM_UU7HyUCvMIIa3R5fDf6N

Ranges

The csv and other commands make use of ranges in A1 notation. Examples of A1 notation can be found in the Google documentation here:

Finding document and parent IDs

Many of the commands operate on the Google Drive ID of a document or a "parent" folder. A convenient way to get these IDs is to just use a web browser and open a file or folder on https://drive.google.com/ to see the ID in the URL. But you can also use gsheet list to list all of the files and folders the service account knows about along with their IDs.

Environment Variables

GOOGLE_APPLICATION_CREDENTIALS

Must be set to the absolute path of a .json file containing credentials for a service account

GSHEET_ID

Can be set instead of setting the --id flag on any command that accepts that flag to identify a spreadsheet document to operate on

GSHEET_PARENT

Can be set instead of setting the --parent flag on any command that accepts that flag to identify a drive folder to operate on

Use as Golang Package

In addition to the cli tool, gsheet can be used as a Golang package to simplify access to Google Sheets and Google Drive from Go.

All of the Sheets related functions are in the gsheets pacakge (gsheets/sheets.go), and all of the Drive related functions are in the gdrive pacakge (gdrive/files.go).

Online godoc documentation for the packages can be found here:

For a quick-and-dirty example of how to use the packages look at the integration_test.go file included in each package.

Hack

To run tests:

make test

To build:

make build

To build binaries for various platforms in build/:

make xbuild

To release:

tbd

Similar Resources

A command line tool for mainly exporting logbook records from Google Spreadsheet to PDF file in EASA format

A command line tool for mainly exporting logbook records from Google Spreadsheet to PDF file in EASA format

Logbook CLI This is a command line tool for mainly exporting logbook records from Google Spreadsheet to PDF file in EASA format. It also supports rend

Feb 6, 2022

Utility for working with files and folders stored on Google Drive

skicka Utility for working with files and folders stored on Google Drive. Note: skicka is not an official Google product! Intro skicka makes it easy t

Nov 15, 2021

Download an upload large files to Google Drive (API v3)

gdriver gdriver is a command-line tool, written in Go, used for uploading and downloading large personal files from Google Drive (API v3). The tool pr

Nov 30, 2022

Go program for uploading files to Google Drive

gdrive-upload Go program for uploading files to Google Drive. Installation Install Go Homebrew: sudo brew install go apt-get: sudo apt-get install gol

Nov 13, 2021

Read data from rss, convert in pdf and send to kindle. Amazon automatically convert them in azw3.

Kindle-RSS-PDF-AZW3 The Kindle RSS PDF AZW3 is a personal project. The Kindle RSS PDF AZW3 is a personal project. I received a Kindle for Christmas, a

Jan 10, 2022

A virtual file system for small to medium sized datasets (MB or GB, not TB or PB). Like Docker, but for data.

AetherFS assists in the production, distribution, and replication of embedded databases and in-memory datasets. You can think of it like Docker, but f

Feb 9, 2022

An API for handling file meta data

dp-files-api An API for handling file meta data Getting started Run make debug Dependencies No further dependencies other than those defined in go.mod

Jan 4, 2022

gh cli extension to display a dashboard of PRs - configurable with a beautiful UI.

gh cli extension to display a dashboard of PRs - configurable with a beautiful UI.

gh-prs A gh cli extension to display a dashboard with pull requests by filters you care about. Installation Installation requires a minimum version (2

Jan 2, 2023
Comments
  • Do not check that CSV files are rectangles

    Do not check that CSV files are rectangles

    Hey :)

    Sometimes we want to upload CSV files that are not perfect rectangles. Like, some lines have 10 fields, some have 7.

    By default Go checks against that. This small commits changes this. I think it's not the role of your tool to check whether our CSV makes sense.

    WDYT?

    Have fun :) happy hacking, super useful tool.

    Matcha

csv reader/writer and csv generator.

IO csv reader sample version 0.0.1-SNAPSHOT Goals: main: read huge file, hex substring, write to new file. repo has 2 version. v1 can read a file and

Nov 4, 2021
GoCsv is a library written in pure Go to use csv data more comfortable

GoCsv GoCsv is a library written in pure Go to use csv data more comfortable Supported Go version golang >= 1.13 Installation go get github.com/shr004

Nov 1, 2022
CSV asynchronous I/O package for go.

golib/csvio selectでの非同期処理に対応した、csvベースでの読み込み/書き込みを行うライブラリです。 csvio.Config CSVのコンマの値や、クォートの有無など、扱うCSVに対するフォーマットを指定できます。 csvio.Reader selectでの非同期処理に対応した、

Jan 21, 2022
FujiSimuRecipesGen - A Golang program to generate Fujifilm simulations from recipes of such simulations in CSV

FujiSimuRecipesGen - A Golang program to generate Fujifilm simulations from recipes of such simulations in CSV

Aug 12, 2022
Allows parsing CSV files into custom structs and implements required fields that can't be empty

Welcome to Go Custom CSV Parser ?? Allows parsing CSV files into custom structs and implements required fields that can't be empty ?? Homepage Install

Nov 9, 2021
Extract profiles and tasks information from CSV file

Footsite-Bot ideas from jw6602 Extract profiles and tasks information from CSV f

Nov 25, 2022
Read csv file from go using tags

go-csv-tag Read csv file from Go using tags The project is in maintenance mode. It is kept compatible with changes in the Go ecosystem but no new feat

Nov 16, 2022
A simple utility for validating CSV columns

gompare A simple utility for validating CSV columns Building In project directly, run go build Usage ./gompare --template-file=template.csv --input-fi

Feb 3, 2022
Simple go script that converts csv file into a json document

csv-go-parser Simple go script that converts csv file into a json document. CSV Input: id,first_name,last_name,email,avatar,ip_address 1,Pauline,Hirth

Jun 4, 2022
This program let you create a DataSet (.CSV) with all TedTalks

TedTalks-Scraper This program let you create a file .CSV with all information from TedTalks, including: Title Description Views (Number of Views) Auth

Dec 26, 2021