Independent set of GDScript tools - parser, linter and formatter

GDScript Toolkit

License: MIT Code style: black

This project provides a set of tools for daily work with GDScript. At the moment it provides:

  • A parser that produces a parse tree for debugging and educational purposes.
  • A linter that performs a static analysis according to some predefined configuration.
  • A formatter that formats the code according to some predefined rules.

Installation

To install this project you need python3 and pip.

Use this command in your terminal to install from this git repository:

pip3 install gdtoolkit

Linting with gdlint

To run a linter you need to perform installation first. Once done, you can run e.g.:

$ gdlint misc/MarkovianPCG.gd

Which outputs messages like:

misc/MarkovianPCG.gd:96: Error: Function argument name "aOrigin" is not valid (function-argument-name)
misc/MarkovianPCG.gd:96: Error: Function argument name "aPos" is not valid (function-argument-name)

To tweak the default check settings, you can dump the default config to a file:

$ gdlint -d
$ cat gdlintrc
class-load-variable-name: (([A-Z][a-z0-9]*)+|_?[a-z][a-z0-9]*(_[a-z0-9]+)*)
class-name: ([A-Z][a-z0-9]*)+
class-variable-name: _?[a-z][a-z0-9]*(_[a-z0-9]+)*
constant-name: '[A-Z][A-Z0-9]*(_[A-Z0-9]+)*'
disable: []
enum-element-name: '[A-Z][A-Z0-9]*(_[A-Z0-9]+)*'
enum-name: ([A-Z][a-z0-9]*)+
function-argument-name: _?[a-z][a-z0-9]*(_[a-z0-9]+)*
function-arguments-number: 10
function-preload-variable-name: ([A-Z][a-z0-9]*)+
function-name: (_on_([A-Z][a-z0-9]*)+(_[a-z0-9]+)*|_?[a-z][a-z0-9]*(_[a-z0-9]+)*)
function-variable-name: '[a-z][a-z0-9]*(_[a-z0-9]+)*'
load-constant-name: (([A-Z][a-z0-9]*)+|[A-Z][A-Z0-9]*(_[A-Z0-9]+)*)
loop-variable-name: _?[a-z][a-z0-9]*(_[a-z0-9]+)*
signal-name: '[a-z][a-z0-9]*(_[a-z0-9]+)*'
sub-class-name: _?([A-Z][a-z0-9]*)+

Once the dump is performed, you can modify the gdlintrc file and optionally rename it to .gdlintrc. From now on, linter will use this config file to override the default config.

Formatting with gdformat

Formatting may lead to data loss, so it's highly recommended to use it along with Version Control System (VCS) e.g. git

gdformat is the uncompromising GDScript code formatter. The only configurable thing is max line length allowed (--line-length). The rest will be taken care of by gdformat in a one, consistent way.

To run a formatter you need to perform installation first. Once done, given a test.gd file:

class X:
	var x=[1,2,{'a':1}]
	var y=[1,2,3,]     # trailing comma
	func foo(a:int,b,c=[1,2,3]):
		if a in c and \
		   b > 100:
			print('foo')
func bar():
	print('bar')

when you run gdformat test.gd, the test.gd file will be reformatted as follows:

class X:
	var x = [1, 2, {'a': 1}]
	var y = [
		1,
		2,
		3,
	]  # trailing comma

	func foo(a: int, b, c = [1, 2, 3]):
		if a in c and b > 100:
			print('foo')


func bar():
	print('bar')

If the program formats your files successfully, it will return the exit code 0. Non-zero code will be returned otherwise.

Parsing with gdparse

To parse a file, use the gdparse program:

gdparse tests/valid-gd-scripts/recursive_tool.gd -p

The parser outputs a tree that represents your code's structure:

start
  class_def
    X
    class_body
      tool_stmt
      signal_stmt	sss
  class_def
    Y
    class_body
      tool_stmt
      signal_stmt	sss
  tool_stmt

If the program parses your file sucessfully, it will return the exit code 0.

Running tests

To run tests you need tox, a tool to automate testing in Python.

Install it with:

pip3 install tox

To invoke entire test suite, in the godot-gdscript-toolkit project directory, run:

tox

You can run only selected test cases:

tox -e py3 -- -k lint

For manual testing, you can consider installing package in editable mode:

pip3 install -e .
Owner
Pawel Lampe
Researcher, Developer and GameDev Hobbyist
Pawel Lampe
Comments
  • Need help is creating cross platform executable versions

    Need help is creating cross platform executable versions

    Hi @Scony ,

    I loved your plugin. But I wanted to create a inEditor version plugin of it.

    tool
    extends EditorPlugin
    
    var files = []
    
    
    func _ready():
        var script_editor := get_editor_interface().get_script_editor()
        var filesystem_interface := get_editor_interface().get_resource_filesystem()
        var filesystem := filesystem_interface.get_filesystem()
        walk_files(filesystem)
        print('Scripts: ' + str(files))
        for script in files:
            var array = [script]
            var args = PoolStringArray(array)
            OS.execute("/Library/Frameworks/Python.framework/Versions/3.8/bin/gdformat", args, true)
            filesystem_interface.update_file(script)
    
    
    func walk_files(dir: EditorFileSystemDirectory):
        for i in range(dir.get_file_count()):
            if dir.get_file_type(i) == "GDScript":
                files.append(ProjectSettings.globalize_path(dir.get_file_path(i)))
        for j in range(dir.get_subdir_count()):
            walk_files(dir.get_subdir(j))
    

    The above script works perfectly and it looks through all the GDscripts in the current project directory and then runs the gdformatter against them and then updates the file in Godot.

    But as you can see the path to gdformat script is hard coded. "/Library/Frameworks/Python.framework/Versions/3.8/bin/gdformat"

    To resolve this problem I need to create an compiled executable version gdformatter and I can add the compiled version in the plugin directory itself and depending upon OS I can switch the executable and run everything locally so end user does not have to install python/this pacakge.

    I tried to create an executable with PyInstaller but it threw a bunch of errors. I am not expert in python so maybe you can help in this regard :)

  • GDScript Formatter vscode extension errors

    GDScript Formatter vscode extension errors

    GDScript Formatter vscode extension VSCode Marketplace Shows this as the source for the extension. The latest version published was 1.2.2 on June 2021. The newest code in this repository is from May 2020 for version 1.1.2.

    #179 aims to targets the extension. It is unclear where the extension is being published from. I was unable to get the extension to work properly and was looking for additional information.

  • Code ordering tool

    Code ordering tool

    @Razoric480 contributed a code ordering tool to the VSCode addon, that relies on the formatter. How about having a new formatter option to order the code, so we can have this from the cli and in every code editor? I'd gladly contribute that.

    It would also help with performances, as currently in his tool, François has to call gdformat twice.

    Suggestions on how to implement it are welcome. I was thinking of formatting first, then reordering the code if the option is in:

    1. having an enum that defines the blocks' order
    2. Parsing the blocks in the current script, storing them as a list of objects with a value from the enum
    3. calling sorted(block_objects, key=operator.attrgetter("value"))
    4. Writing them back to the file
  • Tabs shouldn't be removed on empty lines that are still within function scope

    Tabs shouldn't be removed on empty lines that are still within function scope

    Wrong: image

    Correct: image

    Tabs are rendered in Godot, which helps visualise scope. If an empty line is both preceded and followed by lines within the same scope, it should keep the same indentation level.

  • The vs code gdscript format extension doesnt work

    The vs code gdscript format extension doesnt work

    I installed the vs code gdscript formatter extension, python, pip and the GDToolkit module. Then i reloaded the engine and when i clicked save it didnt format(i have format on save enabled and ut works with other formatters). It also didnt execute the commands organize code and multiline into singleline comments. Btw i saw in the bottom panek that vs code is recognizing the files as gdscript. Thanks in advance for any help,

  • Save document after applyEdit

    Save document after applyEdit

    After formatting, the document is left in an unsaved situation.

    I checked the promise returned from vscode.workspace.applyEdit() and save the document when edit was success

  • gdtoolkit doesn't exist

    gdtoolkit doesn't exist

    $ pip3 install gdtoolkit
    Defaulting to user installation because normal site-packages is not writeable
    ERROR: Could not find a version that satisfies the requirement gdtoolkit
    ERROR: No matching distribution found for gdtoolkit
    
    
  • Converting Python AST to GDScript

    Converting Python AST to GDScript

    I understand that it is not possible to port all Python features to GDScript, but for simple functions implementing algorithms without external dependencies, is it possible to use https://github.com/Scony/godot-gdscript-toolkit/wiki/2.-Parser to reconstruct GDScript AST from Python AST?

  • Error while formating document in VS Code

    Error while formating document in VS Code

    I installed extension in VS Code and GDTollkit with pip3. Formating with GDToolkit works in terminal, but when trying to run in VS Code, it throws an error: ImportError: No module named gdtoolkit.formatter.

    I work on macOS 10.15.5

  • ImportError: No module named gdtoolkit.formatter

    ImportError: No module named gdtoolkit.formatter

    Hi,

    have installed the VS Code extension and installed gdtoolkit via pip3 install gdtoolkit. But I'm getting an error ImportError: No module named gdtoolkit.formatter in VS Code.

    macOS Catalina VS Code: 1.47.3 GDScript Formatter: 1.2.1 gdtoolkit: 3.2.7

    I can run gdformat ... just fine from my terminal.

    Am I missing something?

    Thank you :)

  • gdformat: find GDScript files recursively in a directory

    gdformat: find GDScript files recursively in a directory

    Closes #75

    Before testing the code, I'd like to know if the approach matches your expectations. If only passing a directory as an argument, the program will recursively find .gd files in it and pass them to gdformat.

    Other options:

    • Do that for every directory passed as an argument. For example, gdformat src/Autoload src/Main test.gd would recursively find .gd files in two directories and format test.gd.
    • Same as above, but have an explicit --recursive option. Passing a directory would find files in that directory non-recursively unless the option is provided.
  • GDScript 2.0 - multiline lambdas not supported

    GDScript 2.0 - multiline lambdas not supported

    This is flagged as error on gd-format but completely legit syntax in GDScript 2.0:

    func f():
      return func():
        pass
    

    Error:

    return func():
                          ^
    
    Unexpected token Token('_NL', '\n\t\t') at line 11, column 16.
    Expected one of:
            * NOT
            * HEX
            * BANG
            * BIN
            * VAR
            * AWAIT
            * PERCENT
            * NUMBER
            * LPAR
            * LBRACE
            * REGULAR_STRING
            * CIRCUMFLEX
            * TILDE
            * LSQB
            * RETURN
            * NAME
            * LONG_STRING
            * DOLLAR
            * MINUS
            * FUNC
            * PLUS
            * AMPERSAND
            * PASS
    
  • class-definition-order issues with enums/setters/getters

    class-definition-order issues with enums/setters/getters

    I am experiencing 2 issues with enums and setters/getters in Godot 4 not respecting https://github.com/Scony/godot-gdscript-toolkit/wiki/3.-Linter#class-checks

    image image

    (Here, commenting out the enum fixes the issue)

    image image

    (Here, commenting out the setter fixes the issue)

  • VSCode extension should use gdformat executable directly

    VSCode extension should use gdformat executable directly

    I use pipx to install system utilities in isolated environments so the vscode plugin doesn't work since it cannot source the import from its vscode python environment.

    It would be better to use the executable directly.

    As of now I solved using another vscode extension (Run on Save) with the following setting:

        "emeraldwalk.runonsave": {
            "commands": [
                {
                    "match": "\\.gd$",
                    "isAsync": true,
                    "cmd": "gdformat ${file}"
                }
            ]
        }
    
  • spawn ENAMETOOLONG

    spawn ENAMETOOLONG

    Hi! I'm getting the error spawn ENAMETOOLONG when using gdformat.

    Perhaps it's because there's too many characters in my script?

    Lines: 559 Characters: 32,296

  • gdformat creates a line longer than 100 symbols and gdlint complains about it

    gdformat creates a line longer than 100 symbols and gdlint complains about it

    In my project I run both gdlint and gdformat and I found something what seems like a bug.

    I have a file with the following content:

    static func on_quest_finished(_state: GameState, _quest_type: String, _iker_id: int, _etadata: Dictionary):
    	pass
    

    The line with function declaration is 107 characters long, so gdlint complains about it, but gdformat is perfectly fine with this code:

    $ gdlint tmp/test.gd
    tmp/test.gd:1: Error: Max allowed line length (100) exceeded (max-line-length)
    Failure: 1 problem found
    $ gdformat tmp/test.gd  --diff
    1 file would be left unchanged
    

    If I manually change the code so it fits in 100 symbols:

    static func on_quest_finished(
    	_state: GameState, _quest_type: String, _iker_id: int, _etadata: Dictionary
    ):
    	pass
    

    gdlint is happy, but gdformat wants to revert it to oneline (and violate the 100 length rule):

    $ gdlint tmp/test.gd
    Success: no problems found
    $ gdformat tmp/test.gd  --diff
    would reformat tmp/test.gd
    --- tmp/test.gd
    +++ tmp/test.gd
    @@ -1,4 +1,2 @@
    -static func on_quest_finished(
    -	_state: GameState, _quest_type: String, _iker_id: int, _etadata: Dictionary
    -):
    +static func on_quest_finished(_state: GameState, _quest_type: String, _iker_id: int, _etadata: Dictionary):
     	pass
    1 file would be reformatted, 0 files would be left unchanged.
    

    If I make the line in the original file 1 symbol longer (note _metadata vs _etadata):

    static func on_quest_finished(_state: GameState, _quest_type: String, _iker_id: int, _metadata: Dictionary):
    	pass
    

    both gdlint and gdformat start to complain about it (as expeected):

    $ gdlint tmp/test.gd
    tmp/test.gd:1: Error: Max allowed line length (100) exceeded (max-line-length)
    Failure: 1 problem found
    $ gdformat tmp/test.gd  --diff
    would reformat tmp/test.gd
    --- tmp/test.gd
    +++ tmp/test.gd
    @@ -1,2 +1,4 @@
    -static func on_quest_finished(_state: GameState, _quest_type: String, _iker_id: int, _metadata: Dictionary):
    +static func on_quest_finished(
    +	_state: GameState, _quest_type: String, _iker_id: int, _metadata: Dictionary
    +):
     	pass
    1 file would be reformatted, 0 files would be left unchanged.
    

    The workaround is to silence gdlint with # gdlint:ignore = max-line-length, so it's not super-critical, but still it's a bug.

Mdfmt - A Markdown formatter that follow the CommonMark. Like gofmt, but for Markdown

Introduction A Markdown formatter that follow the CommonMark. Like gofmt, but fo

Dec 18, 2022
Produces a set of tags from given source. Source can be either an HTML page, Markdown document or a plain text. Supports English, Russian, Chinese, Hindi, Spanish, Arabic, Japanese, German, Hebrew, French and Korean languages.
Produces a set of tags from given source. Source can be either an HTML page, Markdown document or a plain text. Supports English, Russian, Chinese, Hindi, Spanish, Arabic, Japanese, German, Hebrew, French and Korean languages.

Tagify Gets STDIN, file or HTTP address as an input and returns a list of most popular words ordered by popularity as an output. More info about what

Dec 19, 2022
character-set conversion library implemented in Go

mahonia character-set conversion library implemented in Go. Mahonia is a character-set conversion library implemented in Go. All data is compiled into

Dec 22, 2022
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

omniparser Omniparser is a native Golang ETL parser that ingests input data of various formats (CSV, txt, fixed length/width, XML, EDI/X12/EDIFACT, JS

Jan 4, 2023
A simple CSS parser and inliner in Go

douceur A simple CSS parser and inliner in Golang. Parser is vaguely inspired by CSS Syntax Module Level 3 and corresponding JS parser. Inliner only p

Dec 12, 2022
Unified diff parser and printer for Go

go-diff Diff parser and printer for Go. Installing go get -u github.com/sourcegraph/go-diff/diff Usage It doesn't actually compute a diff. It only rea

Dec 14, 2022
Quick and simple parser for PFSense XML configuration files, good for auditing firewall rules

pfcfg-parser version 0.0.1 : 13 January 2022 A quick and simple parser for PFSense XML configuration files to generate a plain text file of the main c

Jan 13, 2022
A NMEA parser library in pure Go

go-nmea This is a NMEA library for the Go programming language (Golang). Features Parse individual NMEA 0183 sentences Support for sentences with NMEA

Dec 20, 2022
TOML parser for Golang with reflection.

THIS PROJECT IS UNMAINTAINED The last commit to this repo before writing this message occurred over two years ago. While it was never my intention to

Dec 30, 2022
User agent string parser in golang

User agent parsing useragent is a library written in golang to parse user agent strings. Usage First install the library with: go get xojoc.pw/userage

Aug 2, 2021
Simple HCL (HashiCorp Configuration Language) parser for your vars.

HCL to Markdown About To write a good documentation for terraform module, quite often we just need to print all our input variables as a fancy table.

Dec 14, 2021
A markdown parser written in Go. Easy to extend, standard(CommonMark) compliant, well structured.

goldmark A Markdown parser written in Go. Easy to extend, standards-compliant, well-structured. goldmark is compliant with CommonMark 0.29. Motivation

Dec 29, 2022
A PDF renderer for the goldmark markdown parser.
A PDF renderer for the goldmark markdown parser.

goldmark-pdf goldmark-pdf is a renderer for goldmark that allows rendering to PDF. Reference See https://pkg.go.dev/github.com/stephenafamo/goldmark-p

Jan 7, 2023
Experimental parser Angular template

Experimental parser Angular template This repository only shows what a parser on the Go might look like Benchmark 100k line of template Parser ms @ang

Dec 15, 2021
A dead simple parser package for Go
A dead simple parser package for Go

A dead simple parser package for Go V2 Introduction Tutorial Tag syntax Overview Grammar syntax Capturing Capturing boolean value Streaming Lexing Sta

Dec 30, 2022
Freestyle xml parser with golang

fxml - FreeStyle XML Parser This package provides a simple parser which reads a XML document and output a tree structure, which does not need a pre-de

Jul 1, 2022
An extension to the Goldmark Markdown Parser

Goldmark-Highlight An extension to the Goldmark Markdown Parser which adds parsing / rendering capabilities for rendering highlighted text. Highlighte

May 25, 2022
A parser combinator library for Go.

Takenoco A parser combinator library for Go. Examples CSV parser Dust - toy scripting language Usage Define the parser: package csv import ( "err

Oct 30, 2022
A simple json parser built using golang

jsonparser A simple json parser built using golang Installation: go get -u githu

Dec 29, 2021