Bumps moment from 2.29.1 to 2.29.2.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

This PR changes the type of payload header from map[string]string to map[string]interface{} to allow passing non-string content (ex. Content-Length) to request's headers.

1. Add http timeout prompt
2. Change the way to get host name, by using the standard pkg: net/url (I think this new way is more reasonable, but it requires that the scheme must be specified in the input URL)
3. If DNS resolution fails, give a prompt and return
4. DNS resolution failure caused by "the test endpoint is the machine that made the request" is not fixed. However, this situation triggers 3., so the user is prompted at this point

• Add progress bar UI by using github.com/gizak/termui
• When the test finished, you can press the "Up" and "Down" keys to scroll through the results list and then press the "Q" key to exit

This PR contains

• remove unused code in the codebase
• fix rule 920270: fix validateByteRange
• fix rule 941310: fix t:jsDecode
• fix rule 944200: add flag to notify RegexGeneration that do not use UTF-8 encoding

This PR adds

• automatically fetch the coreruleset and WAFBench repo from Github to repo directory
• add warning for user to install Docker when failed to init Docker daemon

This PR

• added a post-processing module within AddVariable to check if it add any illegal character into the request struct
• added a ErrNotSupport and ErrReject Error struct for better error handling at upstream

As title suggested, the gunicorn worker boot check brought more bugs than the problem (receiving request while gunicorn is not fully prepared) it prevents. Therefore, we decide to remove this check from the docker restart workflow.

This add following functionalities

• allow the backend to run without Docker Engine
• add /api/get-docker-health to check Docker's health at backend
• call /api/get-docker-health everytime before calling /api/get-result at frontend

Bumps moment from 2.29.1 to 2.29.4.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

root@docker-desktop:/go/src/app/repos/coreruleset# git checkout v3.4/dev
Switched to branch 'v3.4/dev'
Your branch is up to date with 'origin/v3.4/dev'.
root@docker-desktop:/go/src/app/repos/coreruleset# cd ../..
root@docker-desktop:/go/src/app# waflab generate
repos/coreruleset/rules
2021/12/16 13:41:41 Err while parsing rule string SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:932011,phase:1,pass,nolog,skipAfter:END-REQUEST-932-APPLICATION-ATTACK-RCE"
SecRule TX:EXECUTING_PARANOIA_LEVEL "@lt 1" "id:932012,phase:2,pass,nolog,skipAfter:END-REQUEST-932-APPLICATION-ATTACK-RCE"

Hi! I'm wondering how are you reading logs for testing that the waf is matching.

• from reading the logContains method in https://github.com/microsoft/waflab/blob/5a38a39175b04b2b9bf78816bc98ec60dd8f9718/cmd/command/test.go#L304, the match looks only a regexp match in the file
• this will not work unless each test fires up one and just one rule per test. That is not the common case.

👉 what ftw and go-ftw have been doing is reading the log backwards until the time of the request. I don't see that here.

Can you please explain what is your use case, and how does it work for you?

Hi Team!

Cannot get waflab generate to work. After building the container, generate panics:

root@docker-desktop:/go/src/app# waflab generate repos/coreruleset/rules/ repos/coreruleset/tests/regression/ -c 1
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xbbc209]

goroutine 1 [running]:
github.com/waflab/waflab/cmd/command.generateTestfile.func1(0x0, 0x0, 0x0, 0x0, 0xe66c20, 0xc00036d380, 0x2, 0x2)
	/go/src/app/cmd/command/utils.go:21 +0x49
path/filepath.Walk(0x0, 0x0, 0xc00035fc70, 0x32, 0x135e840)
	/usr/local/go/src/path/filepath/path.go:499 +0x7e
github.com/waflab/waflab/cmd/command.generateTestfile(0x0, 0x0, 0x1, 0x3, 0x3, 0x2, 0xc00000f980, 0xc00035fd58)
	/go/src/app/cmd/command/utils.go:20 +0x85
github.com/waflab/waflab/cmd/command.generate(0x1320c60, 0xc0003a0e80, 0x2, 0x4)
	/go/src/app/cmd/command/generate.go:37 +0x9a
github.com/spf13/cobra.(*Command).execute(0x1320c60, 0xc0003a0e40, 0x4, 0x4, 0x1320c60, 0xc0003a0e40)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x1320ee0, 0xc00035ff58, 0x2, 0x2)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:897
github.com/waflab/waflab/cmd/command.Execute()
	/go/src/app/cmd/command/root.go:35 +0x7d
main.main()
	/go/src/app/cmd/main.go:6 +0x25

Trying to get this working, but fails on docker compose up:

❯ docker compose up
[+] Running 2/0
 ⠿ Container waflab_frontend_1  Created                                                                           0.0s
 ⠿ Container waflab_backend_1   Created                                                                           0.0s
Attaching to backend_1, frontend_1
backend_1   | {"status":"Pulling from waflab/wafbench","id":"latest"}
backend_1   | {"status":"Digest: sha256:1e94302e53cee0e390b2b7a13fb28f971fa39ac99f9a9b0a0d7a9fef7e0520c0"}
backend_1   | {"status":"Status: Image is up to date for waflab/wafbench:latest"}
backend_1   | panic: Error response from daemon: container cannot be disconnected from host network or connected to host network
backend_1   |
backend_1   | goroutine 1 [running]:
backend_1   | github.com/waflab/waflab/docker.MakeMaster(0x5, 0x200000003)
backend_1   | 	/go/src/app/docker/master.go:134 +0x410
backend_1   | github.com/waflab/waflab/object.InitMaster(...)
backend_1   | 	/go/src/app/object/run.go:18
backend_1   | main.main()
backend_1   | 	/go/src/app/main.go:15 +0x34
backend_1 exited with code 2

Is something additionally required?