Hi everybody,

There were some discussions at #694 with regard to ideas of a possible future major release. GitHub discussions are not too good for these discussions so I decided to open this issue to be more centralized and visible (issues can be pinned).

Also, that probably needs some direction and organization, as having more people involved means we now have different opinions we need to reconcile.

@ghostsquad started working on some ideas on his fork, but I consider that to be a Proof of Concept ™️, and things will likely be different in its final version in the repository.

This issue is a WIP and will be iterated as time passes.

Principles

• Minimal breaking changes
  • Most people should be able to change version: '3' to version: '4' and their Taskfile should keep working mostly the same, with minimal changes
  • This means no unnecessary changes to the schema, like renaming cmds to something else, for example. Most changes should be for new features
  • The same applies to avoiding changes to CLI flags
  • There may be exceptions, of course, but they needs a good justification. Variables will probably be the biggest one, as @ghostsquad has a proposal to make they lazy.
  • Also, for Taskfiles with version: '3' everything should keep working as before, with rare exceptions
  • I'm undecided about remove code related to keep version: '2' working
• Development process
  • Pull requests should be atomic, one for each feature, so they can be reviewed in isolation and discussion can happen with more efficiency about each feature (with regard to UX and implementation)
  • Tests should be written covering the relevant code

Steps

Preparation

• We should try to do this in the master branch IMO, to avoid future conflicts between master and v3
• Code re-organization? We have some old directory structure and may want to improve that
• Tests refactor? Would help as we will write new tests for v3 stuff.
  • Some ideas here: #699

Implementation

• At this point we may decide to have a v3 branch
• Variables refactor
  • Probably the first step, as it's so core an will affects discussions about other changes
  • Make then lazily evaluated (@ghostsquad's proposal, #694)
  • Apply that to ENVs in the process? #693
  • Non-string variables? #140
• Loop / files / matrix
  • Supports looping files? #82
  • Some kind of matrix like GitHub Actions? #675
  • We probably need a proposal to try to solve both use cases
  • #82 is very old but still valid. It's something Make can do that Task still can't
• We need to figure it out if/how to proceed about setup: and hooks proposals: #204, #667, #668
• ...

This is a ongoing document and will be improved...

Hi,

Request

Add the possibility to use import or includes to imports some others tasks. With the default strategy as namespace.

Example

File: docker/v1.0/Taskfile.yaml

version: "2"

task:
  build:
    cmds:
      - docker build...

File: go/v1.0/Taskfile.yaml

version: "2"

task:
  build:
    cmds:
      - go build ...

File (in my project): Taskfile.yaml

imports:
  docker: https://.../docker/v1.0/Taskfile.yaml
  go: file://.../go/v1.0/Taskfile.yaml

— or —

includes:
  docker: https://.../docker/v1.0/Taskfile.yaml
  go: file://.../go/v1.0/Taskfile.yaml

task:
  build:
    - task: "docker:build"
    - task: "go:build"

How to use?

task go:build go:docker

— or —

task build

Finally

What do you think?

UPDATE: This is a general feature request to not run equal dependencies more than once. E.g. if task A and task B both depend on taskC, running both task A and task B in parallel (witin one instance of task), should be able to resolve the dependencies so that taskC is only run once, and this is the interesting port, if it's equal, which we will get to next.

what makes a task equal?

Since you can pass variables to tasks, it's not enough to just look at the name. The best option is probably to rely on a hash of the name + sorted and resolved input variables. A close second could be to rely on unresolved but still sorted input variables, which is not perfect, but could be an acceptable trade-off if it's somehow easier in code.

Original description

I have multiple tasks in my Taskfile that depend on a single task called go-dep that runs the command glide install. When I run task to build multiple targets, I would expect go-dep to be called at most 1 time (zero if it's found to be up-to-date). Instead, I see it's called once for each task that depend on it:

$ task
task: No argument given, trying default task
glide install
glide install
glide install
glide install

PS! go-deps in this case is called with no vars, but the rule should really apply to run a dep at most 1 time when the vars are exactly the same.

When integrating with other tools (like vscode extension, or when trying to script exporting tasks to github actions) we really want to avoid having to parse the free-form output, especially as this will likely change and break integration, even between minor versions.

If we would output as JSON, we can make the parsing far more reliable and less likely to break between versions, especially as if we add extra data, it would not render the original format obsolete.

Things to be included in JSON output:

• all available tasks as keys inside a mapping
• each task dependencies
• task description and summary
• task status (built or not)

I often want to use the directory of where the Taskfile is located as an input to other commands (not the current working directory), but it seems like sprig really locks down access to the filesystem.

Is this possible to do with Task as a first-class idea?

For example, something like:

tasks:
  show-current-dir:
    desc: "show where Task is executing from"
    cmds:
      - echo {{ .TASKFILE_LOCATION }}

Note that this isn't the same as just using $PWD or -d because the working directory might be different; Task should probably provide this as a first-class feature if it doesn't already.

This is a general issue to track proposals with small or big breaking changes, that could be released in the next major version.

This list is a work in progress and can change at any time.

Development is happening on the v3 branch.

Cleanup

• [x] Change module name from github.com/go-task/task/v2 to v3
• [x] Remove all code related support of version 1 (#237)
• [x] Remove automatic inclusion of Taskvars.yml and Taskfile_{{OS}}.yml. This was done before we had a generic inclusion of Taskfiles implemented. We should add options to
  • [ ] Don't include the file if it doesn't exist
  • [x] Allow interpolation on the file name to make it possible to include a OS specific file
• [x] Finally delete the vendor directory? (#214)

Breaking changes

• [x] Added global method: option. Default changed from timestamp to checksum (#246)
• [x] Refactor vars and templates to reduce the number of bugs and undesired behavior (#218)
• [ ] Make CLI arguments override everything (#181) (EDIT: What about #336?)
• [ ] Make up-to-date resolution work when only "generates" is given (#187)
• [ ] Don't run the same dependency (with the same args) more than once by default (#53, #193)
• [ ] Allow forcing specific tasks (#189)

New stuff

• [x] Colored output messages (#207)
• [x] Expose .TIMESTAMP and .CHECKSUM to status: (#216)
• [x] Allow shorter syntax for tasks with default configuration (#194)
• [ ] Allow use as a library (#121)

Documentation

• [x] Mention changes related to method: (shouldn't also check the main guide)
• [x] Update "Taskfile versions" article

There are more issues and feature request that could be added to this release.

For some startup tasks or cleanup paths commands could run in parallel while others should be run in order. Add a Parallel property to task to allow switching between modes

Hi everybody 👋

Once in a while, people ask me why the development is slow, questions got unanswered, bug reports are not responded, features not added, PRs not reviewed, etc.

The answer is simple: the project has grown to become bigger than I ever expected. With a full time job and a life outside work, I have limited time to dedicate to this project. More demand is generated from users than I could keep up even if I doubled the time I dedicate to this project.

So first of all I'd like to ask for your patience and understanding.

Secondly, I'd like to hear opinions and ideas on how to keep the project moving.

And lastly, I'ld like to ask for some help. I had two other people with write access to the repo in the past, but they quickly lost interest in the project. Other than that short period, I've being alone. You could help:

1. Answering questions, validating bug reports, giving opinions on feature requests, closing issues/PRs when they do not make sense, etc.
2. Writing code and reviewing pull requests, specially bug fixes and small features. (I'd still like to have the final word on bigger features as sometimes people suggest changes that breaks backward compatibility or add more complexity than desired)
3. Some people have helped the project by contributing to mvdan/sh as well, and I'm thankful for that. Help on mvdan/sh#93 in particular would hugely benefit Windows users if you're interested to help.

Adding a --summary flag for task to get more a summary of the task

See #107

New field in task called summary

version: '2'
tasks:
  release:
    deps: [build]
    summary: |
      Release your project to github

      It will build your project before starting the release it.
      Please make sure that you have set GITHUB_TOKEN before starting.
    cmds:
      - your-release-tool
  build:
    cmds:
      - your-build-tool

Output for task --summary release:

task: release

Release your project to github

It will build your project before starting the release it.
Please make sure that you have set GITHUB_TOKEN before starting.

dependencies:
 - build

commands:
 - your-release-tool

Behaviour:

• --summary will not execute the task
• --summary will display the summary of a task (see above)
• --summary will use description as fallback, if summary does not exist
• --summary will output a warning (or information), if no description/summary is present
• --summary will output the dependencies if there are any
• --summary will output the commands if there are any

Tests:

• Because there are a lot of edge-cases I decided to extract out a package (summary), which is completly unit tested with above named edge cases.
• There is an integration like test in task_test.go

Documentation is present for that feature

Hi there,

I would like to install go-task on my Linux distro with homebrew but unfortunately, there is already a built-in Formula named task in homebrew-core for taskwarrior that I happen to use.

Since the latter is quite more ancient than go-task, would you consider renaming the binary in your homebrew tap ?

Thanks in advance !

Does go-task currently offer the option of "passthrough" arguments? For example, if I have currently have a task to run the AWS CLI out of a Docker container:

tasks:
  aws:
    cmds:
      - docker-compose run --rm aws		

Is it possible to have the ability for a command like task aws -- ls s3://my-bucket to be translated to docker-compose run --rm aws ls s3://my-bucket, meaning task will pass through anything after -- to the underlying shell command?

This is a more convenient way to pass extra variables compared to the current approach: task write-file FILE=file.txt "CONTENT=Hello, World!". Currently, using this approach allows me to write task aws CMD="ls s3://my-bucket" but the extra variable definition and quotes creates a lot of friction that I think can be alleviated with my proposal.

If this is not possible, it would be a really nice capability to have in the future.

Fixes #976

I noticed that the timestamp method doesn't seem to work. It reruns the tasks all the time even though nothing has changed.

This happens when the task generates some artifacts in the first run and doesn't re-generate them in the subsequent runs. This causes the following run to return false. https://github.com/go-task/task/blob/63c50d13eec142694ff5a074c3f9cda6c53d8be3/internal/status/timestamp.go#L41

To fix this issue, task should include the last run time in the generates. This way, the time of the generates will be newer than the sources, and so there is no need for rerunning.

This solution

• would also fix the issue with an empty generates array.
• is consistent with the checksum method, and these two should work the same in most cases.
• is way faster than the checksum method because the source times are checked lazily and getting the metadata of a file is faster.

I noticed that the timestamp method doesn't seem to work. It reruns the tasks all the time even though nothing has changed.

This happens when the task generates some artifacts in the first run and doesn't re-generate them in the subsequent runs. This causes the following run to return false. https://github.com/go-task/task/blob/63c50d13eec142694ff5a074c3f9cda6c53d8be3/internal/status/timestamp.go#L41

To fix this issue, task should include the last run time in the generates. This way, the time of the generates will be newer than the sources, and so there is no need for rerunning.

This solution

• would also fix the issue with an empty generates array.
• is consistent with the checksum method, and these two should work the same in most cases.
• is way faster than the checksum method because the source times are checked lazily and getting the metadata of a file is faster.

I'd like to change the method of checking for the state of a task to timestamp for all the tasks because the checksum method is slow. Is there a way to do this?

NOTE: This is incomplete work. It's a proof-of-concept that is going well, but the actual behavior and schema will change until it reaches master, so avoid using this unless for testing purposes.

Feedback is welcome on how you expect this to work 🙂.

Implementation

Credits given to @no0dles for the first PoC on #285 and its our tool hammerkit.

On his implementation, @no0dles used the official Docker client library for Go. Sounds reasonable, but I decided to try a different approach: I'm calling docker run ... directly. There's a couple of reasons this may be interesting:

1. It requires no additional dependencies to the tool.
2. It allow us to add a flags prop on the YAML to allow the user to set specific Docker flags (flags: [--cpus=1, --pull=always]). This would be harder to implement if using the Go client, would potentially require a prop to each setting. List of flags available here.
3. The idea of avoiding using libraries make the implementation a bit more generic, opening room to potentially adding support to other container systems in the future, like Podman, without too much additional work.

TODO

• [ ] Add props to highly common settings.
  • Mounting directories?
  • Exposing ports?
  • We don't need to do all because with the generic flags it's also possible to do this.
• [ ] Think about how to handle the dir prop and its relation with the mounts prop.
• [ ] What do you about method/sources/generates? Try to run them inside the container? Run outside? Forbid usage on when task is a container?
  • We could maybe forbid usage on the first release and figure it out the best behavior later.
• [ ] Dynamic variables are still being computed outside the container. Is that OK?
• [ ] Make implementation more generic (change to a interface) to allow Podman/others in the future?
• [ ] Add global container prop, so it's possible to set all tasks on run in a given container?
  • Not sure how useful that would be. It's probably better to wait for feedback before doing this.
• [ ] Write tests
  • These tests should be behind a Go flag so they won't run by default (but should ideally run on CI).
• [ ] Write documentation
  • Usage
  • API
  • JSON Schema

This is meant to be a transitioning implementation for #681 before it becomes the default behavior on a later version. When reviewing this PR, please take into consideration that I don't have much of an experience writing in Go.

The optional command line flag for this feature "--propagate-status," without a shortcut.

I've noticed that Task kills the process when reloading it in Watch mode instead of gracefully terminating it with SIGTERM:

As far as I can tell, this happens because the task context gets cancelled here, which seems to just kill the process instead of sending an interrupt signal.

It would be nice if there was an option to gracefully stop it instead (and wait for it to stop, in case the cleanup takes a while).

I think it should also behave like InterceptInterruptSignals and forcefully exit if 3 consecutive interrupts are detected (in case shutdown hangs).

My machine is running Windows 10 by the way.