A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.

About chromedp Go Reference

Package chromedp is a faster, simpler way to drive browsers supporting the Chrome DevTools Protocol in Go without external dependencies.

Installing

Install in the usual Go way:

$ go get -u github.com/chromedp/chromedp

Examples

Refer to the Go reference for the documentation and examples. Additionally, the examples repository contains more examples on complex actions, and other common high-level tasks such as taking full page screenshots.

Frequently Asked Questions

I can't see any Chrome browser window

By default, Chrome is run in headless mode. See DefaultExecAllocatorOptions, and an example to override the default options.

I'm seeing "context canceled" errors

When the connection to the browser is lost, chromedp cancels the context, and it may result in this error. This occurs, for example, if the browser is closed manually, or if the browser process has been killed or otherwise terminated.

Chrome exits as soon as my Go program finishes

On Linux, chromedp is configured to avoid leaking resources by force-killing any started Chrome child processes. If you need to launch a long-running Chrome instance, manually start Chrome and connect using RemoteAllocator.

Executing an action without Run results in "invalid context"

By default, a chromedp context does not have an executor, however one can be specified manually if necessary; see issue #326 for an example.

I can't use an Action with Run because it returns many values

Wrap it with an ActionFunc:

ctx, cancel := chromedp.NewContext()
defer cancel()
chromedp.Run(ctx, chromedp.ActionFunc(func(ctx context.Context) error {
	_, err := domain.SomeAction().Do(ctx)
	return err
}))

I want to use chromedp on a headless environment

The simplest way is to run the Go program that uses chromedp inside the chromedp/headless-shell image. That image contains headless-shell, a smaller headless build of Chrome, which chromedp is able to find out of the box.

Resources

Owner
Chrome DevTools Protocol clients and tools for Go
Chrome DevTools Protocol clients and tools for Go
Chrome DevTools Protocol clients and tools for Go
https://github.com/chromedp/chromedp
Comments
  • domEvent: timeout waiting for node

    domEvent: timeout waiting for node

    This is more a question than a bug report.

    When a dom event listener is about to wait for a node and the node gets concurrently removed (or its parent), WaitNode fails with timeout. The problem here is when that happens, the whole action handlers stale for 10 seconds. At least this is what I think happens. The behaviour is random, if WaitNode finishes before it's removed, then everything works as expected (I think also Go scheduler contributes to the racy behaviour).

    Debug log for timeout scenario:

    scrap.go:91: -> {"method":"DOM.childNodeInserted","params":{"parentNodeId":7,"previousNodeId":8,"node":{"nodeId":9,"backendNodeId":9,"nodeType":1,"nodeName":"DIV","localName":"div","nodeValue":"","childNodeCount":0,"attributes":[]}}}
scrap.go:91: -> {"method":"DOM.attributeModified","params":{"nodeId":9,"name":"id","value":"someotherid"}}
scrap.go:91: -> {"method":"DOM.childNodeInserted","params":{"parentNodeId":7,"previousNodeId":0,"node":{"nodeId":11,"backendNodeId":11,"nodeType":1,"nodeName":"BODY","localName":"body","nodeValue":"","childNodeCount":1,"attributes":["style","visibility: hidden; width: 0px; height: 0px; border: 0px; margin: 0px; background: none;"]}}}
scrap.go:91: -> {"method":"DOM.childNodeCountUpdated","params":{"nodeId":11,"childNodeCount":0}}
scrap.go:91: -> {"method":"DOM.childNodeRemoved","params":{"parentNodeId":7,"nodeId":9}}
scrap.go:91: -> {"method":"DOM.childNodeRemoved","params":{"parentNodeId":7,"nodeId":11}}
scrap.go:91: could not perform (attributeModified) operation on node 9 (wait node): timeout waiting for node `9`
scrap.go:91: could not perform (childNodeCountUpdated) operation on node 11 (wait node): timeout waiting for node `11`

    I was thinking about fixing this (the 10s stales are really bugging me) by introducing some kind of context cancellation / listening channel, so we break out of waiting loop in WaitNode when the node we're waiting for gets removed.

    Wdyt?

  • won't work out of the box with a headless-shell container running as root

    won't work out of the box with a headless-shell container running as root

    What versions are you running?

    $ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.1.4-0.20190409142133-92a77355f6df
$ chromium --version
Chromium 73.0.3683.75 built on Debian 9.8, running on Debian 9.8
$ go version
go version go1.12.3 linux/amd64

    What did you do?

    I wrote the following test

    package main

import (
        "context"
        "testing"
        "time"

        "github.com/chromedp/chromedp"
)

func Test(t *testing.T) {
        ctx, cancel := chromedp.NewContext(
                context.Background(),
                chromedp.WithLogf(t.Logf),
        )
        defer cancel()

        ctx, cancel = context.WithTimeout(ctx, 10*time.Second)
        defer cancel()

        err := chromedp.Run(ctx,
                chromedp.Navigate("http://localhost:3000/"),
        )
        if err != nil {
                t.Fatal(err)
        }
}

    and tried to go test.

    What did you expect to see?

    go test succeeds.

    What did you see instead?

    main_test.go:25: malformed ws or wss URL

    and, sometimes, also a panic:

    panic: sync: negative WaitGroup counter

goroutine 13 [running]:
sync.(*WaitGroup).Add(0xc000332044, 0xffffffffffffffff)
        /usr/local/go/src/sync/waitgroup.go:74 +0x135
sync.(*WaitGroup).Done(...)
        /usr/local/go/src/sync/waitgroup.go:99
github.com/chromedp/chromedp.(*ExecAllocator).Allocate.func1(0xcdac40, 0xc000332060, 0xc000010888, 0xc000128e01, 0xc0000377c0, 0x1d, 0xc000332000, 0xc0002e4ea0)
        /go/pkg/mod/github.com/chromedp/[email protected]/allocate.go:126 +0xa2
created by github.com/chromedp/chromedp.(*ExecAllocator).Allocate
        /go/pkg/mod/github.com/chromedp/[email protected]/allocate.go:109 +0x740
  • How to set http proxy user and password in chromedp?

    How to set http proxy user and password in chromedp?

    Hi, I know the chromedp could set http proxy on option "runner.Proxy(http://localhost:8000/)", but how to set the proxy's user and password? Thanks

  • An example to capture a screenshot of the full page

    An example to capture a screenshot of the full page

    Is it possible to get a screenshot of the full page, even when setting a specific width and height to emulate a mobile device?

    For example, here is a screenshot using dev tools and an iphone 6.

    I'm looking to achieve the same result.

    http://i.imgur.com/0Je8sgT.png

    So far, I'm getting an image which is cut off at the specified height.

  • bypass headless chrome detection

    bypass headless chrome detection

    What versions are you running?

    $ go list -m github.com/chromedp/chromedp
go list -m: not using modules

$ google-chrome --version
/Applications/Chromium.app/Contents/MacOS/Chromium --version
Chromium 77.0.3830.0

$ go version
go version go1.12.6 darwin/amd64

    What did you do? Include clear steps.

    package main

import (
	"context"
	"fmt"
	"time"

	"github.com/pkg/errors"

	"github.com/chromedp/cdproto/runtime"
	"github.com/chromedp/chromedp"
)

func main() {
	opts := []chromedp.ExecAllocatorOption{
		chromedp.ExecPath(`/Applications/Chromium.app/Contents/MacOS/Chromium`),
		chromedp.UserAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3830.0 Safari/537.36"),
		chromedp.WindowSize(1920, 1080),
		chromedp.NoFirstRun,
		chromedp.NoDefaultBrowserCheck,
		chromedp.Headless,
		chromedp.DisableGPU,
	}

	ctx, cancel := chromedp.NewExecAllocator(context.Background(), opts...)
	defer cancel()

	ctx, cancel = chromedp.NewContext(ctx)
	defer cancel()

	err := chromedp.Run(ctx,
		evalJS(`Object.defineProperty(navigator, 'plugins', {get: () => [1, 2, 3, 4, 5, 7, 8, 9]});`),
		evalJS(`navigator.plugins.length`),
		chromedp.Navigate("https://intoli.com/blog/not-possible-to-block-chrome-headless/chrome-headless-test.html"),
		chromedp.ActionFunc(func(ctx context.Context) error {
			time.Sleep(3 * time.Second)
			return nil
		}),
		evalJS(`navigator.plugins.length`),
	)
	if err != nil {
		panic(err)
	}
}

func evalJS(js string) chromedp.Tasks {
	var res *runtime.RemoteObject
	return chromedp.Tasks{
		chromedp.EvaluateAsDevTools(js, &res),
		chromedp.ActionFunc(func(ctx context.Context) error {
			b, err := res.MarshalJSON()
			if err != nil {
				return errors.Wrap(err, "marshal")
			}
			fmt.Println("result: ", string(b))
			return nil
		}),
	}
}

    What did you expect to see?

    result:  {"type":"object","className":"Navigator","description":"Navigator","objectId":"{\"injectedScriptId\":1,\"id\":1}"}
result:  {"type":"number","value":8,"description":"8"}
result:  {"type":"number","value":8,"description":"8"}

    I am trying to bypass headless browser detection as written in this article https://intoli.com/blog/not-possible-to-block-chrome-headless/. This works fine for puppeteer, but doesn't work for chromedp. I need to understand how to execute js code in chromedp, as in puppeteer. Because every time after run chromedp.Navigate() values are reset to default.

    It is possible to do with chromedp?

    What did you see instead?

    result:  {"type":"object","className":"Navigator","description":"Navigator","objectId":"{\"injectedScriptId\":1,\"id\":1}"}
result:  {"type":"number","value":8,"description":"8"}
result:  {"type":"number","value":0,"description":"0"} // after chromedp.Navigate() navigator.plugins.length has default value equal 0

    Works fine with puppeteer.

    const puppeteer = require('puppeteer');

const userAgent = 'Mozilla/5.0 (X11; Linux x86_64)' +
  'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.39 Safari/537.36';

(async () => {
  const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();
  
  await page.setUserAgent(userAgent);

  await page.evaluateOnNewDocument(() => {
    // Overwrite the `plugins` property to use a custom getter.
    Object.defineProperty(navigator, 'plugins', {
      // This just needs to have `length > 0` for the current test,
      // but we could mock the plugins too if necessary.
      get: () => [1, 2, 3, 4, 5, 6, 7, 8],
    });
  });

  await page.goto('https://intoli.com/blog/not-possible-to-block-chrome-headless/chrome-headless-test.html');
  await page.screenshot({path: 'result.png'});

  await browser.close();
})();
  • x509: certificate signed by unknown authority

    x509: certificate signed by unknown authority

    go install in docker image report x509: certificate signed by unknown

    What versions are you running?

    uname -r

    5.4.72-microsoft-standard-WSL2

    cat /etc/os-release

    PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

    $ go list -m github.com/chromedp/chromedp go: github.com/chromedp/[email protected]: missing go.sum entry; to add it: go mod download github.com/chromedp/chromedp

    $ google-chrome --version sh: 137: google-chrome: not found

    $ go version go version go1.16.5 linux/amd64

    What did you do? Include clear steps.

    docker pull chromedp/headless-shell:74.0.3717.1 //download go1.16.5.linux-amd64.tar.gz and put into local /goinstall directoy docker run -v ${PWD}/goinstall/:/tmp/ -d --name chromedpcontainer chromedp/headless-shell

    docker exec -it sh cd /tmp tar -C /usr/local -xzf go1.16.5.linux-amd64.tar.gz export PATH=$PATH:/usr/local/go/bin cd /tmp go mod init helloworld go get -x github.com/chromedp/chromedp

    What did you expect to see?

    github.com/chromedp/chromedp installed. below added into go.mod

    require ( github.com/chromedp/chromedp v0.6.10 )

    What did you see instead?

    go: github.com/chromedp/[email protected]: Get "https://proxy.golang.org/github.com/chromedp/chromedp/@v/v0.6.10.mod": x509: certificate signed by unknown authority

  • Problem with aws lambda

    Problem with aws lambda

    What versions are you running?

    $ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.8.2

$ docker pull chromedp/headless-shell:latest
latest: Pulling from chromedp/headless-shell
Digest: sha256:8fe1cbf5aba60474079fedf56867a3a3576340ab21b6e222674edcb6610a584d
Status: Image is up to date for chromedp/headless-shell:latest
docker.io/chromedp/headless-shell:latest

$ docker run --rm golang:alpine go version
go version go1.18.2 linux/amd64

    What did you do? Include clear steps.

    Here is an example repo. I want to deploy it to aws lambda.

    Code snippet

    func Handler(_ context.Context, _ json.RawMessage) error {
	opts := []chromedp.ExecAllocatorOption{
		chromedp.NoFirstRun,
		chromedp.NoDefaultBrowserCheck,
		chromedp.NoSandbox,
		chromedp.DisableGPU,
		chromedp.Headless,
		chromedp.Flag("no-zygote", true),
		chromedp.Flag("user-data-dir", "/tmp/chrome-user-data-dir"),
		chromedp.Flag("homedir", "/tmp/chrome-home"),
		chromedp.Flag("data-path", "/tmp/chrome-data-path"),
		chromedp.Flag("disk-cache-dir", "/tmp/chrome-disk-cache-dir"),
		chromedp.Flag("remote-debugging-port", "9222"),
		chromedp.Flag("remote-debugging-address", "0.0.0.0"),
		chromedp.Flag("disable-dev-shm-usage", true),
		chromedp.Flag("enable-features", "NetworkService,NetworkServiceInProcess"),
	}
	ctx, cancel := chromedp.NewExecAllocator(context.Background(), opts...)
	defer cancel()

	ctx, cancel = chromedp.NewContext(ctx, chromedp.WithDebugf(log.Printf))
	defer cancel()

	var content string
	if err := chromedp.Run(ctx, chromedp.Tasks{
		chromedp.Navigate("https://example.com/"),
		chromedp.Text("body > div > p:nth-child(2)", &content),
	}); err != nil {
		log.Fatal(err)
	}
	fmt.Println(content)
	return nil
}

    FROM golang:alpine AS builder

RUN apk add git

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY . .

RUN go build -o main

FROM chromedp/headless-shell

RUN apt-get update && \
    apt-get install -y dumb-init musl-dev && \
    ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1 &&\
    rm -rf /var/lib/apt/lists/*

COPY --from=builder /app/main .

ENTRYPOINT [ "dumb-init", "--" ]
CMD [ "./main" ]

    This code (without docker) run perfectly on my M1 macbook, and built docker container run perfectly on AWS EC2.

    But when I deploy it to aws lambda, it just stuck after last log 2022/06/01 04:13:38 -> {"id":3,"sessionId":"26082765C593E944A783B371227B397A","method":"Runtime.enable"}.

    What did you expect to see?

    I can run it on aws lambda.

    Log on ec2 running perfectly

    2022/06/01 05:13:03 -> {"id":1,"method":"Target.setDiscoverTargets","params":{"discover":true}}
2022/06/01 05:13:03 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"11278bc8-3ff8-436d-901f-a6404ecb61d1","type":"browser","title":"","url":"","attached":false,"canAccessOpener":false}}}
2022/06/01 05:13:03 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"","url":"about:blank","attached":false,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"}}}
2022/06/01 05:13:03 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"e42b9ee5-9df7-49b6-b6f2-0314165479f6","type":"browser","title":"","url":"","attached":true,"canAccessOpener":false}}}
2022/06/01 05:13:03 <- {"id":1,"result":{}}
2022/06/01 05:13:03 -> {"id":2,"method":"Target.attachToTarget","params":{"targetId":"412DC8003B59C78196B600AFBB7C0231","flatten":true}}
2022/06/01 05:13:03 <- {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"}}}
2022/06/01 05:13:03 <- {"method":"Target.attachedToTarget","params":{"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"},"waitingForDebugger":false}}
2022/06/01 05:13:03 <- {"id":2,"result":{"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}}
2022/06/01 05:13:04 -> {"id":3,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Runtime.enable"}
2022/06/01 05:13:04 <- {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"}}}
2022/06/01 05:13:04 <- {"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","uniqueId":"5877140344713506672.6727992831272504814","auxData":{"isDefault":true,"type":"default","frameId":"412DC8003B59C78196B600AFBB7C0231"}}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"id":3,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":4,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Runtime.evaluate","params":{"expression":"self"}}
2022/06/01 05:13:04 <- {"id":4,"result":{"result":{"type":"object","className":"Window","description":"Window","objectId":"5836470857787727263.1.1"}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":5,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Log.enable"}
2022/06/01 05:13:04 <- {"id":5,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":6,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Network.enable","params":{}}
2022/06/01 05:13:04 <- {"id":6,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":7,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Inspector.enable"}
2022/06/01 05:13:04 <- {"id":7,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":8,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Page.enable"}
2022/06/01 05:13:04 <- {"id":8,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":9,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.enable","params":{}}
2022/06/01 05:13:04 <- {"id":9,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":10,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"CSS.enable"}
2022/06/01 05:13:04 <- {"id":10,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":11,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Target.setDiscoverTargets","params":{"discover":true}}
2022/06/01 05:13:04 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"11278bc8-3ff8-436d-901f-a6404ecb61d1","type":"browser","title":"","url":"","attached":false,"canAccessOpener":false}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"about:blank","url":"about:blank","attached":true,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"e42b9ee5-9df7-49b6-b6f2-0314165479f6","type":"browser","title":"","url":"","attached":true,"canAccessOpener":false}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"id":11,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":12,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Target.setAutoAttach","params":{"autoAttach":true,"waitForDebuggerOnStart":false,"flatten":true}}
2022/06/01 05:13:04 <- {"id":12,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":13,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Page.setLifecycleEventsEnabled","params":{"enabled":true}}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"commit","timestamp":55583.941934},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"DOMContentLoaded","timestamp":55583.942021},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"load","timestamp":55583.942425},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"networkAlmostIdle","timestamp":55583.942883},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"networkIdle","timestamp":55583.942883},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"id":13,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":14,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Page.navigate","params":{"url":"https://example.com/"}}
2022/06/01 05:13:04 <- {"method":"Network.requestWillBeSent","params":{"requestId":"C9980CD6274A43FBE9998E7B5E95629B","loaderId":"C9980CD6274A43FBE9998E7B5E95629B","documentURL":"https://example.com/","request":{"url":"https://example.com/","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36"},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"strict-origin-when-cross-origin","isSameSite":true},"timestamp":55583.963522,"wallTime":1654060384.050588,"initiator":{"type":"other"},"redirectHasExtraInfo":false,"type":"Document","frameId":"412DC8003B59C78196B600AFBB7C0231","hasUserGesture":false},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Network.requestWillBeSentExtraInfo","params":{"requestId":"C9980CD6274A43FBE9998E7B5E95629B","associatedCookies":[],"headers":{":authority":"example.com",":method":"GET",":path":"/",":scheme":"https","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","accept-encoding":"gzip, deflate, br","accept-language":"en-US@posix","sec-fetch-dest":"document","sec-fetch-mode":"navigate","sec-fetch-site":"none","sec-fetch-user":"?1","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36"},"connectTiming":{"requestTime":55583.965293}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Network.responseReceivedExtraInfo","params":{"requestId":"C9980CD6274A43FBE9998E7B5E95629B","blockedCookies":[],"headers":{"accept-ranges":"bytes","age":"267163","cache-control":"max-age=604800","content-encoding":"gzip","content-length":"648","content-type":"text/html; charset=UTF-8","date":"Wed, 01 Jun 2022 05:13:04 GMT","etag":"\"3147526947\"","expires":"Wed, 08 Jun 2022 05:13:04 GMT","last-modified":"Thu, 17 Oct 2019 07:18:26 GMT","server":"ECS (oxr/8370)","vary":"Accept-Encoding","x-cache":"HIT"},"resourceIPAddressSpace":"Public","statusCode":200},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Network.responseReceived","params":{"requestId":"C9980CD6274A43FBE9998E7B5E95629B","loaderId":"C9980CD6274A43FBE9998E7B5E95629B","timestamp":55584.700562,"type":"Document","response":{"url":"https://example.com/","status":200,"statusText":"","headers":{"accept-ranges":"bytes","age":"267163","cache-control":"max-age=604800","content-encoding":"gzip","content-length":"648","content-type":"text/html; charset=UTF-8","date":"Wed, 01 Jun 2022 05:13:04 GMT","etag":"\"3147526947\"","expires":"Wed, 08 Jun 2022 05:13:04 GMT","last-modified":"Thu, 17 Oct 2019 07:18:26 GMT","server":"ECS (oxr/8370)","vary":"Accept-Encoding","x-cache":"HIT"},"mimeType":"text/html","connectionReused":false,"connectionId":13,"remoteIPAddress":"93.184.216.34","remotePort":443,"fromDiskCache":false,"fromServiceWorker":false,"fromPrefetchCache":false,"encodedDataLength":193,"timing":{"requestTime":55583.965293,"proxyStart":-1,"proxyEnd":-1,"dnsStart":23.354,"dnsEnd":24.929,"connectStart":24.929,"connectEnd":562.419,"sslStart":194.584,"sslEnd":562.39,"workerStart":-1,"workerReady":-1,"workerFetchStart":-1,"workerRespondWithSettled":-1,"sendStart":562.684,"sendEnd":562.847,"pushStart":0,"pushEnd":0,"receiveHeadersEnd":733.232},"responseTime":1.654060384785279e+12,"protocol":"h2","securityState":"secure","securityDetails":{"protocol":"TLS 1.3","keyExchange":"","keyExchangeGroup":"P-256","cipher":"AES_256_GCM","certificateId":0,"subjectName":"www.example.org","sanList":["www.example.org","example.net","example.edu","example.com","example.org","www.example.com","www.example.edu","www.example.net"],"issuer":"DigiCert TLS RSA SHA256 2020 CA1","validFrom":1647216000,"validTo":1678838399,"signedCertificateTimestampList":[],"certificateTransparencyCompliance":"unknown"}},"hasExtraInfo":true,"frameId":"412DC8003B59C78196B600AFBB7C0231"},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"id":14,"result":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"C9980CD6274A43FBE9998E7B5E95629B"},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"networkAlmostIdle","timestamp":55583.942883},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"8E275FC88E861B68C971049B227D2C13","name":"networkIdle","timestamp":55583.942883},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.frameStartedLoading","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231"},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"C9980CD6274A43FBE9998E7B5E95629B","name":"init","timestamp":55584.704893},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"https://example.com","url":"https://example.com/","attached":true,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"}}}
2022/06/01 05:13:04 <- {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"412DC8003B59C78196B600AFBB7C0231","type":"page","title":"https://example.com","url":"https://example.com/","attached":true,"canAccessOpener":false,"browserContextId":"1488447107EADF5939A51B3C61EA1F3B"}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Runtime.executionContextsCleared","params":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.frameNavigated","params":{"frame":{"id":"412DC8003B59C78196B600AFBB7C0231","loaderId":"C9980CD6274A43FBE9998E7B5E95629B","url":"https://example.com/","domainAndRegistry":"example.com","securityOrigin":"https://example.com","mimeType":"text/html","adFrameStatus":{"adFrameType":"none"},"secureContextType":"Secure","crossOriginIsolatedContextType":"NotIsolated","gatedAPIFeatures":[]},"type":"Navigation"},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"DOM.documentUpdated","params":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Network.dataReceived","params":{"requestId":"C9980CD6274A43FBE9998E7B5E95629B","timestamp":55584.707376,"dataLength":1256,"encodedDataLength":0},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Runtime.executionContextCreated","params":{"context":{"id":2,"origin":"https://example.com","name":"","uniqueId":"964267974370550981.7417314755448916594","auxData":{"isDefault":true,"type":"default","frameId":"412DC8003B59C78196B600AFBB7C0231"}}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":15,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.getDocument","params":{}}
2022/06/01 05:13:04 <- {"method":"Network.loadingFinished","params":{"requestId":"C9980CD6274A43FBE9998E7B5E95629B","timestamp":55584.699583,"encodedDataLength":850,"shouldReportCorbBlocking":false},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"CSS.styleSheetAdded","params":{"header":{"styleSheetId":"38.0","frameId":"412DC8003B59C78196B600AFBB7C0231","sourceURL":"https://example.com/","origin":"regular","title":"","ownerNode":9,"disabled":false,"isInline":true,"isMutable":false,"isConstructed":false,"startLine":8,"startColumn":27,"length":650,"endLine":34,"endColumn":4}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"id":15,"result":{"root":{"nodeId":1,"backendNodeId":3,"nodeType":9,"nodeName":"#document","localName":"","nodeValue":"","childNodeCount":2,"children":[{"nodeId":2,"parentId":1,"backendNodeId":10,"nodeType":10,"nodeName":"html","localName":"","nodeValue":"","publicId":"","systemId":""},{"nodeId":3,"parentId":1,"backendNodeId":4,"nodeType":1,"nodeName":"HTML","localName":"html","nodeValue":"","childNodeCount":2,"children":[{"nodeId":4,"parentId":3,"backendNodeId":11,"nodeType":1,"nodeName":"HEAD","localName":"head","nodeValue":"","childNodeCount":5,"attributes":[]},{"nodeId":5,"parentId":3,"backendNodeId":12,"nodeType":1,"nodeName":"BODY","localName":"body","nodeValue":"","childNodeCount":1,"attributes":[]}],"attributes":[],"frameId":"412DC8003B59C78196B600AFBB7C0231"}],"documentURL":"https://example.com/","baseURL":"https://example.com/","xmlVersion":"","compatibilityMode":"NoQuirksMode"}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.loadEventFired","params":{"timestamp":55584.726037},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"C9980CD6274A43FBE9998E7B5E95629B","name":"load","timestamp":55584.726037},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.frameStoppedLoading","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231"},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"DOM.documentUpdated","params":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.domContentEventFired","params":{"timestamp":55584.726372},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"Page.lifecycleEvent","params":{"frameId":"412DC8003B59C78196B600AFBB7C0231","loaderId":"C9980CD6274A43FBE9998E7B5E95629B","name":"DOMContentLoaded","timestamp":55584.726372},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":16,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.getDocument","params":{}}
2022/06/01 05:13:04 <- {"id":16,"result":{"root":{"nodeId":6,"backendNodeId":3,"nodeType":9,"nodeName":"#document","localName":"","nodeValue":"","childNodeCount":2,"children":[{"nodeId":7,"parentId":6,"backendNodeId":10,"nodeType":10,"nodeName":"html","localName":"","nodeValue":"","publicId":"","systemId":""},{"nodeId":8,"parentId":6,"backendNodeId":4,"nodeType":1,"nodeName":"HTML","localName":"html","nodeValue":"","childNodeCount":2,"children":[{"nodeId":9,"parentId":8,"backendNodeId":11,"nodeType":1,"nodeName":"HEAD","localName":"head","nodeValue":"","childNodeCount":5,"attributes":[]},{"nodeId":10,"parentId":8,"backendNodeId":12,"nodeType":1,"nodeName":"BODY","localName":"body","nodeValue":"","childNodeCount":1,"attributes":[]}],"attributes":[],"frameId":"412DC8003B59C78196B600AFBB7C0231"}],"documentURL":"https://example.com/","baseURL":"https://example.com/","xmlVersion":"","compatibilityMode":"NoQuirksMode"}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":17,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.performSearch","params":{"query":"body \u003e div \u003e p:nth-child(2)"}}
2022/06/01 05:13:04 <- {"id":17,"result":{"searchId":"38.1","resultCount":1},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":18,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.getSearchResults","params":{"searchId":"38.1","fromIndex":0,"toIndex":1}}
2022/06/01 05:13:04 <- {"method":"DOM.setChildNodes","params":{"parentId":10,"nodes":[{"nodeId":11,"parentId":10,"backendNodeId":5,"nodeType":1,"nodeName":"DIV","localName":"div","nodeValue":"","childNodeCount":3,"attributes":[]}]},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"method":"DOM.setChildNodes","params":{"parentId":11,"nodes":[{"nodeId":12,"parentId":11,"backendNodeId":13,"nodeType":1,"nodeName":"H1","localName":"h1","nodeValue":"","childNodeCount":1,"children":[{"nodeId":13,"parentId":12,"backendNodeId":6,"nodeType":3,"nodeName":"#text","localName":"","nodeValue":"Example Domain"}],"attributes":[]},{"nodeId":14,"parentId":11,"backendNodeId":14,"nodeType":1,"nodeName":"P","localName":"p","nodeValue":"","childNodeCount":1,"children":[{"nodeId":15,"parentId":14,"backendNodeId":7,"nodeType":3,"nodeName":"#text","localName":"","nodeValue":"This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission."}],"attributes":[]},{"nodeId":16,"parentId":11,"backendNodeId":15,"nodeType":1,"nodeName":"P","localName":"p","nodeValue":"","childNodeCount":1,"attributes":[]}]},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 <- {"id":18,"result":{"nodeIds":[14]},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":19,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.performSearch","params":{"query":"body \u003e div \u003e p:nth-child(2)"}}
2022/06/01 05:13:04 <- {"id":19,"result":{"searchId":"38.2","resultCount":1},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":20,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.getSearchResults","params":{"searchId":"38.2","fromIndex":0,"toIndex":1}}
2022/06/01 05:13:04 <- {"id":20,"result":{"nodeIds":[14]},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":21,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"DOM.resolveNode","params":{"nodeId":14}}
2022/06/01 05:13:04 <- {"id":21,"result":{"object":{"type":"object","subtype":"node","className":"HTMLParagraphElement","description":"p","objectId":"5836470857787727263.2.1"}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":22,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Runtime.callFunctionOn","params":{"functionDeclaration":"function text() {\n    if (this.offsetWidth || this.offsetHeight || this.getClientRects().length) {\n        return this.innerText;\n    }\n    return '';\n}\n","objectId":"5836470857787727263.2.1","silent":true,"returnByValue":true}}
2022/06/01 05:13:04 <- {"id":22,"result":{"result":{"type":"string","value":"This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission."}},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
2022/06/01 05:13:04 -> {"id":23,"sessionId":"0F3F699532F154F976242DBF4AC0DFD2","method":"Runtime.releaseObject","params":{"objectId":"5836470857787727263.2.1"}}
2022/06/01 05:13:04 <- {"id":23,"result":{},"sessionId":"0F3F699532F154F976242DBF4AC0DFD2"}
This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.

    What did you see instead?

    It stuck and timeout (aws lambda timeout).

    Log on aws lambda.

    START RequestId: f829c5ca-e956-4bfe-bef0-dae4f36d50d1 Version: $LATEST
2022/06/01 04:13:38 -> {"id":1,"method":"Target.setDiscoverTargets","params":{"discover":true}}
2022/06/01 04:13:38 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"D86C302EE0B44DAED223378F7C17BB78","type":"page","title":"","url":"about:blank","attached":false,"canAccessOpener":false,"browserContextId":"0C22F0D3B02A5878B60E8F3500944994"}}}
2022/06/01 04:13:38 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"b08ebc85-843e-44d0-bcb3-42ea7bd5a146","type":"browser","title":"","url":"","attached":true,"canAccessOpener":false}}}
2022/06/01 04:13:38 <- {"method":"Target.targetCreated","params":{"targetInfo":{"targetId":"edd4fc71-a027-494d-8a43-eba9239cd0bb","type":"browser","title":"","url":"","attached":false,"canAccessOpener":false}}}
2022/06/01 04:13:38 <- {"id":1,"result":{}}
2022/06/01 04:13:38 -> {"id":2,"method":"Target.attachToTarget","params":{"targetId":"D86C302EE0B44DAED223378F7C17BB78","flatten":true}}
2022/06/01 04:13:38 <- {"method":"Target.targetInfoChanged","params":{"targetInfo":{"targetId":"D86C302EE0B44DAED223378F7C17BB78","type":"page","title":"","url":"","attached":true,"canAccessOpener":false,"browserContextId":"0C22F0D3B02A5878B60E8F3500944994"}}}
2022/06/01 04:13:38 <- {"method":"Target.attachedToTarget","params":{"sessionId":"26082765C593E944A783B371227B397A","targetInfo":{"targetId":"D86C302EE0B44DAED223378F7C17BB78","type":"page","title":"","url":"","attached":true,"canAccessOpener":false,"browserContextId":"0C22F0D3B02A5878B60E8F3500944994"},"waitingForDebugger":false}}
2022/06/01 04:13:38 <- {"id":2,"result":{"sessionId":"26082765C593E944A783B371227B397A"}}
2022/06/01 04:13:38 -> {"id":3,"sessionId":"26082765C593E944A783B371227B397A","method":"Runtime.enable"}
END RequestId: f829c5ca-e956-4bfe-bef0-dae4f36d50d1
REPORT RequestId: f829c5ca-e956-4bfe-bef0-dae4f36d50d1	Duration: 20019.61 ms	Billed Duration: 20552 ms	Memory Size: 10240 MB	Max Memory Used: 125 MB	Init Duration: 532.33 ms	
2022-06-01T04:13:57.470Z f829c5ca-e956-4bfe-bef0-dae4f36d50d1 Task timed out after 20.02 seconds
  • add a ModifyCmdFunc ExecAllocatorOption

    add a ModifyCmdFunc ExecAllocatorOption

    ModifyCmdFunc allows a function to be passed which will run on the exec.Cmd used to start the browser. This can be used to prevent chromedp from automatically passing os signals through to managed browsers, and to prevent browsers from being automatically killed on program exit.

    Addresses #670

  • Has anyone successfully run chromedp in Lambda?

    Has anyone successfully run chromedp in Lambda?

    What versions are you running?

    github.com/chromedp/chromedp v0.1.3
go version go1.12.1

    What did you do?

    I've written a small headless chromedp utility which runs fine on my laptop (Ubuntu) but it fails in Lambda with this error "fork/exec /usr/bin/google-chrome: operation not permitted".
    This appears to be an issue with the restricted Lambda environment

    What did you expect to see?

    I expected my CLI tool to run successfully.

    What did you see instead?

    The above error.

  • PDF & emulation.SetDefaultBackgroundColorOverride: does it work?

    PDF & emulation.SetDefaultBackgroundColorOverride: does it work?

    Hello 👋 ,

    I'm currently implementing a feature in Gotenberg that mimic the omitBackground feature from Puppeteer.

    I did exactly the same thing as they do (i.e., using emulation to override the default white background), but without success so far.

    I was wondering if you may have some insights to share, because I'm kinda lost here 😄

    What versions are you running?

    $ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.8.6
$ google-chrome --version
Google Chrome 106.0.5249.119
$ go version
go version go1.19 darwin/amd64

    What did you do? Include clear steps.

    I added the following ActionFunc in my list of chromedp.Tasks:

    chromedp.ActionFunc(func(ctx context.Context) error {
  // See https://github.com/gotenberg/gotenberg/issues/226.
  if !options.OmitBackground {
	  logger.Debug("default white background not hidden")
  
	  return nil
  }
  
  logger.Debug("hide default white background")
  
  err := emulation.SetDefaultBackgroundColorOverride().WithColor(
	  &cdp.RGBA{
		  R: 0,
		  G: 0,
		  B: 0,
		  A: 0,
	  }).Do(ctx)
  
  if err == nil {
	  return nil
  }
  
  return fmt.Errorf("hide default white background: %w", err)
}),

    You may find the related PR here: https://github.com/gotenberg/gotenberg/pull/528

    What did you expect to see?

    I did expect the PDFs to have a transparent background.

    What did you see instead?

    The PDFs still have a white background, only the size differ (a few bytes).

  • Have Cancel gracefully close the Browser

    Have Cancel gracefully close the Browser

    I am doing some work collecting code coverage data on Chromium (not JavaScript coverage, but coverage of the browser source/C++). In order for Chromium to correctly generate the output files containing this coverage data, I need it to close gracefully. I believe the DevTools Protocol Browser.close() would work fine.

    1. My understanding is that when you cancel a context, the associated browser is closed with SIGKILL. Is that accurate?

    2. If so, is there an easy way I can circumvent that to close the browser gracefully? I think that right now, this if statement is preventing it. Will there be unintended effects if I just remove that check?

  • Click or WaitNotVisible/Present Not Working for Page

    Click or WaitNotVisible/Present Not Working for Page

    On this Javascript app driven page, if I run this in the Javascript console,

    document.querySelector('#gallery_next')

    it will change all IDs matching .gallery_item, thus signifying that navigation to the next page has occurred.

    However, when I try the same Click command using chromedp, it times out waiting for an element matching what should be changing ID to be no longer visible or present (I have tried both).

    package main

import (
	"context"
	"fmt"
	"time"

	"github.com/chromedp/chromedp"
)

func handleErr(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	ctx, cancel := chromedp.NewContext(context.Background())
	defer cancel()
	time.AfterFunc(10*time.Second, func() {
		fmt.Println("check timed out")
		cancel()
	})

	var id string
	err := chromedp.Run(ctx,
		chromedp.Navigate("https://orgasmsoundlibrary.com/#gallery"),
		chromedp.WaitVisible(`.gallery_item`),
		chromedp.AttributeValue(`.gallery_item`, "id", &id, nil),
	)
	handleErr(err)

	err = chromedp.Run(ctx,
		chromedp.Click("#gallery_next"),
		// These both always time out
		// chromedp.WaitNotVisible("#"+id),
		chromedp.WaitNotPresent("#"+id),
	)
	handleErr(err)
}
  • add exposefunc feature

    add exposefunc feature

    In some scenarios, need to bind go func to the browser environment. when js called, the function executes in go process and returns a Promise refer https://pptr.dev/api/puppeteer.page.exposefunction/

  • Cepture screenshot returns full height screenshot instead of viewport height

    Cepture screenshot returns full height screenshot instead of viewport height

    Hey! I'm using chromedp to capture the website's screenshot for the given viewport. But when I call chromedp.CaptureScreenshot(&buf), full height screenshot is taken, instead of just the viewport height.

    What versions are you running?

    $ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.8.6
$ google-chrome --version
Chromium 108.0.5359.94 built on Debian 11.5, running on Debian 11.5
$ go version
go version go1.18.9 linux/arm64

    What did you do? Include clear steps.

    package crawler

import (
    "context"
    "github.com/chromedp/chromedp"
    "log"
    "time"
)

type Crawler struct {
    URL            string
    ViewportWidth  int64
    ViewportHeight int64
}

func (c *Crawler) GetScreenshot() (error, []byte) {
    ctx, cancel := chromedp.NewContext(
        context.Background(),
    )
    defer cancel()

    ctx, cancel = context.WithTimeout(ctx, 10*time.Second)
    defer cancel()

    var buf []byte

    println(c.ViewportWidth, c.ViewportHeight)

    if err := chromedp.Run(ctx,
        chromedp.EmulateViewport(c.ViewportWidth, c.ViewportHeight),
        chromedp.Navigate(c.URL),
        chromedp.CaptureScreenshot(&buf),
    ); err != nil {
        log.Println(err)
        return err, nil
    }

    return nil, buf
}

    You can also check the source code here: https://github.com/marketron-app/image-engine/blob/main/crawler/crawler.go

    What did you expect to see?

    Get a screenshot with height of a viewport.

    What did you see instead?

    Got a screenshot with full height of the website.

  • How to reuse tab

    How to reuse tab

    What versions are you running?

    $ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.8.6hromedp
$ google-chrome --version
Google Chrome 108.0.5359.94
$ go version
19

    What did you do? Include clear steps.

    chromedp.Run(ctx,

		chromedp.ActionFunc(func(c context.Context) error {
			for u := range urls {
				p := PlayStoreData{}
				p.URL = u
				chromedp.Navigate(u).Do(c)
							
					chromedp.Text(`div[class="TT9eCd"]`, &p.Rating, chromedp.BySearch, chromedp.NodeVisible).Do(c)
					chromedp.Text(`div[class=xg1aie]`, &p.UpdatedDate, chromedp.BySearch, chromedp.NodeVisible).Do(c)
					chromedp.Click(`button[class="VfPpkd-Bz112c-LgbsSe yHy1rc eT1oJ QDwDD mN1ivc VxpoF"]`, chromedp.ByQuery).Do(c)
					chromedp.Text(`(//div[.="Version"])[1]//following-sibling::div`, &p.Version, chromedp.BySearch, chromedp.NodeVisible).Do(c)

				}
				data <- p
			}

			return nil
		}),

    What did you expect to see?

    tchromedp reusing the same tab

    What did you see instead?

    after a few successful navigation on samepage the tab crashes/closes.

Related tags
Testing Frameworks testing golang unit-testing chrome chrome-devtools chrome-debugging-protocol headless
Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language.

cdp Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language. The bindings are generated

Jan 7, 2023
A Devtools driver for web automation and scraping

Overview Documentation | API reference Rod is a high-level driver directly based on DevTools Protocol. It's designed for web automation and scraping.

Dec 30, 2022
Selenium Hub successor running browsers within containers. Scalable, immutable, self hosted Selenium-Grid on any platform with single binary.
 Selenium Hub successor running browsers within containers. Scalable, immutable, self hosted Selenium-Grid on any platform with single binary.

Selenoid Selenoid is a powerful implementation of Selenium hub using Docker containers to launch browsers. Features One-command Installation Start bro

Jan 5, 2023
Prototype pollution scanner using headless chrome
 Prototype pollution scanner using headless chrome

plution Prototype pollution scanner using headless chrome What this is Plution is a convenient way to scan at scale for pages that are vulnerable to c

Jan 1, 2023
Generate PlantUML diagrams from Chrome or Firefox network inspections

hoofli Generate PlantUML diagrams from Chrome or Firefox network inspections This tool reads browser HAR files stored on your local disk and transform

Nov 15, 2022
Drive higher confidence in making changes by detecting large blocks of untested functionality

go-coverage Increase code coverage of Go projects Table of Contents About The Project Getting Started Prerequisites Installation Usage Roadmap About T

Nov 16, 2022
Easier way to unit test terraform

Unit testing terraform (WIP) Disclaimer Currently, the only way to compare values is using JSON query path and all types are strings. want := terraf

Aug 16, 2022
A clipboard-based mocking framework for Go that gets out of your way.
 A clipboard-based mocking framework for Go that gets out of your way.

A clipboard-based mocking framework for Go that gets out of your way. This tool has been built with inspiration lovingly taken from Moq, and fuelled b

Nov 18, 2022
Toy-redis-mock - Experimentation with parsing the redis wire protocol from scratch

Overview Simple app for practicing implementing server-side Redis wire protocol

Jan 9, 2022
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.

About chromedp Package chromedp is a faster, simpler way to drive browsers supporting the Chrome DevTools Protocol in Go without external dependencies

Dec 28, 2022
Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language.

cdp Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language. The bindings are generated

Jan 7, 2023
Chrome-Password-Dumper - Chrome password dumper written in Go for Linux and Windows

Chrome-Password-Dumper Chrome password dumper written in Go for Linux and Window

Dec 19, 2022
htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster.
 htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster.

htf htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster. All you need to do is populate the htf configura

Nov 28, 2021
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files

Website | Documentation | Download | Contributing | Changelog | Installation | Forum Rclone Rclone ("rsync for cloud storage") is a command-line progr

Jan 9, 2023
azqlite is a lightweight wrapper around Azure's SDK to interact with the Azure Storage Queue service in a simpler and more idiomatic way.

azqlite azqlite is a lightweight wrapper around github.com/Azure/azure-storage-queue-go to interact with the Azure Storage Queue service in a simpler

Mar 12, 2022
A Devtools driver for web automation and scraping

Overview Documentation | API reference Rod is a high-level driver directly based on DevTools Protocol. It's designed for web automation and scraping.

Dec 30, 2022
The web framework for writing faster sites, faster

Gondola The web framework for writing faster sites, faster. Written in Go. View documentation at http://gondolaweb.com. Unless indicated otherwise at

Nov 20, 2022
A faster RWLock primitive in Go, 2-3 times faster than RWMutex. A Go implementation of concurrency control algorithm in paper

Go Left Right Concurrency A Go implementation of the left-right concurrency control algorithm in paper <Left-Right - A Concurrency Control Technique w

Jan 6, 2023
Selenium Hub successor running browsers within containers. Scalable, immutable, self hosted Selenium-Grid on any platform with single binary.
 Selenium Hub successor running browsers within containers. Scalable, immutable, self hosted Selenium-Grid on any platform with single binary.

Selenoid Selenoid is a powerful implementation of Selenium hub using Docker containers to launch browsers. Features One-command Installation Start bro

Jan 5, 2023
Interact with Chromium-based browsers' debug port to view open tabs, installed extensions, and cookies
 Interact with Chromium-based browsers' debug port to view open tabs, installed extensions, and cookies

WhiteChocolateMacademiaNut Description Interacts with Chromium-based browsers' debug port to view open tabs, installed extensions, and cookies. Tested

Nov 2, 2022