Initially suggested by @D630 in #77.

Open questions needed to be answered before we decide whether or not to support it:

• Where is the formal specification of its syntax? The man page (https://www.mirbsd.org/htman/i386/man1/false.htm) seems to be the only reference.
• What language features does it have that we don't yet support via either POSIX Shell or Bash?
• Is it compatible with the POSIX Shell syntax? i.e., do they conflict?
• Is it compatible with the Bash syntax? i.e., do they conflict?

And of course, whether or not this is worth the effort. I myself haven't seen any mksh scripts in the wild, whereas sh and bash are both very common.

I'd be great if you could provide a docker image on docker hub. My use case is to use it with drone.io, there are other CIs that can use docker images natively. Something like:

FROM alpine:latest
ENTRYPOINT ["/usr/bin/shfmt"]
ADD https://github.com/mvdan/sh/releases/download/v1.2.0/shfmt_v1.2.0_linux_amd64 /usr/bin/shfmt
RUN chmod +x /usr/bin/shfmt

would work for me.

And maybe the following for automated builds:

FROM golang:alpine
ENTRYPOINT ["shfmt"]
COPY . /go/src/app
WORKDIR /go/src/app
RUN set -ex \
       && apk add --no-cache --virtual .build-deps git \
       && go-wrapper download ./... \
       && go-wrapper install ./... \
       && apk del .build-deps 

Right now the code:

[ -z "$HOME" ] && foo || bar
[ -z "$HOME" ] &&
    foo ||
    bar

will be formatted:

[ -z "$HOME" ] && foo || bar
[ -z "$HOME" ] \
    && foo \
    || bar

Can you add an option or something to control the formatting of the line breaks ?

Thanks for the great tool. I have started using this tool with the VS Code. I would like to provide one feedback w.r.t formatting functions (which would at least help couple of us).

The opening brace of the function definition should be in next line.

Currently it's as below...

function fun() {
}

Suggested way...

function fun()
{
}

Refer examples here: https://www.shellscript.sh/functions.html

One advantage of following the suggested format is, in the terminal editors like (vim), you can jump between functions by pressing [ and ] character twice. This is very important feature for specially scripts - as once you are done developing the script on your desktop, you would copy/run it on a system where there may not be graphical interfaces or IDEs.

Thanks for this tool. In our shell script, we have lines like

 	local cmd="docker build -t $DOCKER_REGISTRY/$IMG:$TAG ."
	echo $cmd; $cmd

We would like to not break lines like echo $cmd; $cmd into 2 separate lines. I did not see any flag to support this. Can this be supported?

When recursing it would be nice to be able to ignore certain directories or files (for example, node_modules).

Either a --exclude=node_modules, or --ignore=node_modules, or a .shfmtignore-type functionality would be nice.

I'm starting to use shfmt (via Visual Studio Code editor), but one behaviour that prevents me from using it wholeheartedly is the way shfmt changes the whitespace inside source lines. I wonder if there can be an option to disable this behaviour, but it would continue to change the white spaces at the beginning or end of the line.

The motivation is to not mess up custom alignment of code, creating layout that might be used to help the code read better.

im sure that this is definitely intentional, but this is actually undesirable in my use case im developing hilbish and use this module to run sh of course, but used to only use this as a last option fallback instead of running my custom builtins or lua code. but now im considering actually using an exec handler for the runner but realized cd wasnt behaving how i expected it to be since it's this module's cd and not mine

so is there a way for me to change how the builtin works from outside the package? i havent found anything in docs like that

--- FAIL: TestRunnerTerminalExec (0.00s)
    --- FAIL: TestRunnerTerminalExec/Pseudo (0.14s)
##[error]        terminal_test.go:129: signal: hangup
FAIL
FAIL	mvdan.cc/sh/v3/interp	4.548s

@theclapp perhaps you could help me reproduce this again? Do you have any ideas?

Worst case scenario, we can temporarily skip the pty tests on Mac.

For simple echo it's working, but:

$ ls >> out
ls: write error: Bad file descriptor

ls is from Windows Git (C:\Program Files\Git\usr\bin\ls.exe).

Replace in interp\module.go, fix it, but it's bad implementation:

var DefaultOpen = ModuleOpen(func(ctx context.Context, path string, flag int, perm os.FileMode) (io.ReadWriteCloser, error) {
	if flag&os.O_APPEND != 0 {
		flag = flag & (^os.O_APPEND)
		f, err := os.OpenFile(path, flag, perm)
		f.Seek(0, 2)
		return f, err
	}
	return os.OpenFile(path, flag, perm)
})

Use the following actions instead of manual Docker buildx setup:

• https://github.com/docker/login-action
• https://github.com/docker/setup-qemu-action
• https://github.com/docker/build-push-action

Notes:

• It would be good to merge "Build and push" and "Build and push (alpine)" steps but I have no solution to propose for now;
• there are some linting changes (using prettier).

WIP because CI needs to be tested against upstream repo:

• create a DOCKER_USER secret so that forks can test the CI (push);
• I am using token and 2FA on DockerHub so the update description step could not be tested.

Regarding the last step (Update DockerHub description), we may consider using this https://github.com/marketplace/actions/docker-hub-description.

Take this piece of shell:

$ cat f.sh
if
	true \
	; then
	echo yes
fi
$ bash f.sh
yes
$ shfmt f.sh
if
	true \
		;
then
	echo yes
fi

Note how it runs, but shfmt didn't improve it that much. It could do better, instead producing:

$ cat f2.sh
if
	true
then
	echo yes
fi
$ bash f2.sh 
yes
$ shfmt f2.sh
if
	true
then
	echo yes
fi

In my scripts, I have a few commands that follow this general pattern:

env\
	VAR1='1'\
	VAR2='2'\
	somecommand\
	--big-flag-1='big_flag_value_1'\
	--big-flag-2='big_flag_value_2'\
	--big-flag-3='big_flag_value_3'\
	-x\
	-y\
	-z\
	file_with_a_very_long_name_1\
	file_with_a_very_long_name_2\
	file_with_a_very_long_name_3\
	file_with_a_very_long_name_4\
	;

This is line-oriented to simplify diffing, sorting, etc. I would expect shfmt -p to either leave the lines as-is (maybe adding or removing a space before the final \), but the result is rather unexpected and weird:

env VAR1='1' \
       VAR2='2' \
       somecommand --big-flag-1='big_flag_value_1' \
       --big-flag-2='big_flag_value_2' \
       --big-flag-3='big_flag_value_3' \
       -x -y -z file_with_a_very_long_name_1 file_with_a_very_long_name_2 file_with_a_very_long_name_3 file_with_a_very_long_name_4

I get the addition of extra spaces, but then it also combines all of the single-letter flags and all non-option arguments into one unreadable line. But not the envs or the long-form flags.

I feel like shfmt shouldn't interfere with the placement of arguments on separate lines at all, but if it should, it should also at the very least not produce these overlong lines of mixed short options and positional arguments.

Hi, I'm using shfmt -f . to list files I want to format. However, many files aren't found as they don't have a .sh/.bash extension, but are named something like mkosi.finalize and I can't change the name as it is expected by some consumer. The files have, however, a valid shebang.

The files are excluded because of the following line:

https://github.com/mvdan/sh/blob/279b806d8f2417af933a40a726ad8299395d9183/fileutil/file.go#L80

Would be great to just omit it, and do the shebang check in any case.

Sometimes, to improve readability, I put logfile redirections at the beginning of the line:

>>$LOGFILE echo "[label]" 1 2 3
>>$LOGFILE some_command 4 5 6

This is currently, as of shfmt v3.5.1, incorrectly reformatted as follows:

echo >> $LOGFILE "[label]" 1 2 3
some_command >> $LOGFILE 4 5 6

Research

Putting redirects at various places in-between arguments and at the end seems to have been supported from the very beginning of sh in 1971. Putting redirects at the beginning was not supported in UNIX V1, but emerged sometime later. I haven't found out when.

UNIX V1 manual (1971-03-11) (and code):

Each command is a sequence of non-blank command arguments separated by blanks. [...] Two characters [i.e., "<" or ">"] cause the immediately following string to be interpreted as a special argument to the shell itself, not passed to the command. An argument of the form <arg causes the file arg to be used as the standard input file of the given command; an argument of the form “>arg” causes file “arg” to be used as the standard output file for the given command.

Bash documentation, section 3.6 (2022-11-13):

The following redirection operators may precede or appear anywhere within a simple command or may follow a command. Redirections are processed in the order they appear, from left to right. [...]

Bash documentation, section 3.7.1 (2022-11-13):

When a simple command is executed, the shell performs the following expansions, assignments, and redirections, from left to right, in the following order: 1. The words that the parser has marked as variable assignments (those preceding the command name) and redirections are saved for later processing. [...]

shfmt is a great tool, and it would be fantastic if it was more widely adopted. Unfortunately many potential users don't have an easy way to run it: the only *nix machines they have access to are managed by others, and they don't have the expertise to set up something like a VM. To help those users out, I'm wondering if someone would be willing to host a playground like ShellCheck where users can simply paste their code and get a formatted version back in another part of the page.

IsIncomplete reports whether a Parser error could have been avoided with extra input bytes. For example, if an io.EOF was encountered while there was an unclosed quote or parenthesis.

Given this godoc, it seems like it should work with all parse APIs, like Parse or Words, but it does not. For example, see https://go.dev/play/p/DxfQ8ZlBFuF.

We should either make it work everyhwere, or document its limitation. Ideally the former.