A Devtools driver for web automation and scraping

Overview

Go Reference Discord Chat

Documentation | API reference

Rod is a high-level driver directly based on DevTools Protocol. It's designed for web automation and scraping. Rod is designed for both high-level and low-level use, senior programmers can use the low-level packages and functions to easily customize or build up their own version of Rod, the high-level functions are just examples to build a default version of Rod.

Features

  • Chained context design, intuitive to timeout or cancel the long-running task
  • Debugging friendly, auto input tracing, remote monitoring headless browser
  • Thread-safe for all operations
  • Automatically find or download browser
  • Lightweight, no third-party dependencies, CI tested on Linux, Mac, and Windows
  • High-level helpers like WaitStable, WaitRequestIdle, HijackRequests, WaitDownload, etc
  • Two-step WaitEvent design, never miss an event (how it works)
  • Correctly handles nested iframes or shadow DOMs
  • No zombie browser process after the crash (how it works)

Examples

Please check the examples_test.go file first, then check the examples folder.

For more detailed examples, please search the unit tests. Such as the usage of method HandleAuth, you can search all the *_test.go files that contain HandleAuth, for example, use Github online search in repository. You can also search the GitHub issues, they contain a lot of usage examples too.

Here is a comparison of the examples between rod and Chromedp.

If you have questions, please raise an issue or join the chat room.

How it works

Here's the common start process of rod:

  1. Try to connect to a Devtools endpoint (WebSocket), if not found try to launch a local browser, if still not found try to download one, then connect again. The lib to handle it is launcher.

  2. Use the JSON-RPC to talk to the Devtools endpoint to control the browser. The lib handles it is cdp.

  3. Use the type definitions of the JSON-RPC to perform high-level actions. The lib handles it is proto.

Object model:

object model

Become a maintainer

Please check this doc.

Owner
Rod
A Devtools driver to make web automation and scraping easy
Rod
Comments
  • Why enable the browser to have two tabs and not open the web properly?

    Why enable the browser to have two tabs and not open the web properly?

    Rod Version: v0.84.3 chrome Version: 版本 87.0.4280.88(正式版本) go Version : go1.14.4 windows/amd64 Windows7

    sample code:

    package main
    
    import (
    	"github.com/go-rod/rod"
    	"time"
    )
    
    func main() {
    	page := rod.New().MustConnect().MustPage("https://www.baidu.com/")
    	page.MustWaitLoad().MustScreenshot("a.png")
    	time.Sleep(time.Hour)
    }
    

    When running the code above, chrome will open two browser window directly, one of the Windows web site: file:///C:/Users/ADMINI~1/AppData/Local/Temp/rod/self-close.html, the other is a blank page, baidu.com is not normal to open

    Help me, thank you 1 2

  • Create docker image builds for linux/arm and linux/arm64

    Create docker image builds for linux/arm and linux/arm64

    Is your feature request related to a problem

    The go-rod/rod image provided in the docker image repository is built for the x86 architecuture. With the rise of arm architecture on smaller portable devices (i.e. raspberry pi) it makes sense to also create a arm based image.

    Describe the solution you'd like

    Add images for arm.

  • 获取frame内的元素直接panic

    获取frame内的元素直接panic

    Rod Version: v0.102.1

    // Please make sure your code is minimal and standalone
    func main() {
    defer func() {
    		if i := recover(); i != nil {
    			fmt.Println("panic:", i)
    		}
    		fmt.Println("app will quit in 10s...")
    		time.Sleep(time.Second * 10)
    	}()
    	browser := rod.New().ControlURL("debugURL").MustConnect().NoDefaultDevice()
    	pages := browser.MustPages()
    	page := pages[0]
    	fmt.Println("got page ", page.TargetID)
    	page.MustNavigate("https://coinlist.co/login")
    	page.MustWaitLoad()
    
    	e := page.MustElement("div:not([style*='display:']) > iframe[data-hcaptcha-widget-id]")
    	fmt.Println("element got")
    	frame := e.MustFrame()
    	fmt.Println("frame got ", frame.IsIframe())
    	c, err := frame.Element("#checkbox")
    	if err != nil {
    		fmt.Println("query checkbox:", err)
    		return
    	}
    	fmt.Println("element got")
    	c.MustClick()
    	select {}
    }
    

    运行结果

    c, err := frame.Element("#checkbox")语句会直接空指针panic

  • Fix hijacking when fulfilling requests with empty body.

    Fix hijacking when fulfilling requests with empty body.

    The issue occurred when I was writing some tests for a backend that uses Protobuf messages (with Twirp as the transport layer). An empty body is the optimized version of "all fields" in the response message are empty.

    If I hijack such a request and just call 'ctx.MustLoadResponse()` without doing anything else, this request suddenly failed. I also could not create this kind of response when I needed it for testing.

    After removing the omitempty, it works fine in all my test cases. I actually don't see any reason why you would want to omit when empty. However, maybe I am forgetting some use cases.

  • 导航时地址输入到了地址栏但是并没有导航过去

    导航时地址输入到了地址栏但是并没有导航过去

    Rod Version: v0.101.8

    示例代码释义

    代码环境:Windows系统 根据实际代码改编如下demo,每次运行程序时: 如果存在历史chrome时杀死并开启新的debug的chrome, 开启协程(blockingMonitorPage)监听当前page对象被关闭事件, 开启携程(blockingMonitorNewDoc)监听当前页面的DOM文档切换。

    示例代码

    import (
    	"context"
    	"fmt"
           "encoding/json"
    	"github.com/go-rod/rod"
    	"github.com/go-rod/rod/lib/proto"
            wapi "github.com/iamacarpet/go-win64api"
    	"github.com/pkg/errors"
    	"log"
    	"os"
    	"os/exec"
    )
    
    func TestNavigate(t *testing.T) {
    	ctx, cancel := context.WithCancel(context.TODO())
    	defer cancel()
    
    	// 杀死历史chrome
    	if err := killChrome(); err != nil {
    		t.Fatal(err)
    	}
    	// 开启新的debug chrome
    	err := exec.Command("cmd", "/c", "start chrome --remote-debugging-port=9222").Run()
    	if err != nil {
    		t.Fatal(err)
    	}
    	
    	time.Sleep(time.Second * 5)
    	debugURL, err := GetBrowserDebugURL()
    	if err != nil {
    		t.Fatal(err)
    	}
    	browser := rod.New().ControlURL(debugURL).MustConnect().NoDefaultDevice()
    	b := &Browser{
    		browser:  browser,
    		currPage: (browser.MustPages())[0],
    	}
    	go b.blockingMonitorNewDoc(ctx)
    	go b.blockingMonitorPage(ctx)
    	
    	// 开始导航
    	err = b.navigate(ctx, "https://www.baidu.com/?1=1")
    	if err != nil {
    		t.Fatal(err)
    	}
    	select {}
    }
    
    // 相关实现
    type Browser struct {
    	browser  *rod.Browser
    	currPage *rod.Page
    }
    
    func (c *Browser) navigate(ctx context.Context, url string) error {
    	if c.currPage == nil {
    		return errors.New("want navigate but nil page")
    	}
    	err := c.currPage.Context(ctx).Navigate(url)
    	if err != nil {
    		return errors.Wrap(err, "navigate page")
    	}
    	// 等待页面加载完成
    	if err := c.currPage.Context(ctx).Timeout(time.Second * 30).WaitLoad(); err != nil {
    		return errors.Wrap(err, "wait page navigated")
    	}
    	return nil
    }
    
    func (c *Browser) blockingMonitorPage(ctx context.Context) {
    	defer func() {
    		if i := recover(); i != nil {
    			log.Printf("[tab monitor] - panic:%v\r\n", i)
    		}
    		log.Printf("blocking monitor browser tabs stopped...")
    	}()
    	events := c.browser.Event()
    	for {
    		select {
    		case <-ctx.Done():
    		case ev := <-events:
    			// 等待目标页面被关闭的事件
    			if ev.Method != "Target.targetDestroyed" {
    				continue
    			}
    			var entity proto.TargetTargetDestroyed
    			if !ev.Load(&entity) {
    				fmt.Println("got Target.targetDestroyed but not loaded")
    				continue
    			}
    			if entity.TargetID != c.currPage.TargetID {
    				continue
    			}
    			// 当前页面被关闭了
    			log.Printf("current page closed...\n")
                             // 自定义更多逻辑
    		}
    	}
    }
    
    func (c *Browser) blockingMonitorNewDoc(ctx context.Context) {
    	defer func() {
    		if i := recover(); i != nil {
    			log.Printf("[page monitor] - panic:%v", i)
    		}
    		log.Println("[page monitor] - browser or current page closed manually!!!")
    	}()
    	log.Println("[page monitor] - blocking monitor new page...")
    	events := c.currPage.Context(ctx).Event()
    	for {
    		select {
    		case <-ctx.Done():
    			log.Println("[page monitor] - canceling...")
    			return
    		case ev, ok := <-events:
    			if !ok {
    				return
    			}
    			if ev.Method != "Page.domContentEventFired" {
    				continue
    			}
    			info, err := c.currPage.Context(ctx).Info()
    			if err != nil {
    				log.Println(err)
    				continue
    			}
    			log.Printf("[page monitor] - new page:%v %v\n", info.URL, info.Title)
    		}
    	}
    }
    
    func GetBrowserDebugURL() (string, error) {
    	resp, err := http.Get("http://localhost:9222/json/version")
    	if err != nil {
    		return "", err
    	}
    	var result map[string]interface{}
    	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
    		return "", err
    	}
    	return result["webSocketDebuggerUrl"].(string), nil
    }
    
    func killChrome() error {
    	pr, err := wapi.ProcessList()
    	if err != nil {
    		return err
    	}
    	for _, p := range pr {
    		if p.Executable == "chrome.exe" {
    			log.Println("killing history chrome instance...")
    			_, err := wapi.ProcessKill(uint32(p.Pid))
    			return err
    		}
    	}
    	return nil
    }
    

    问题描述

    生产环境中,发现某些服务器执行导航www.baidu.com时,www.baidu.com出现在了浏览器导航栏但未导航成功的现象(并未加载对应页面),同时软件收到了当前页Target.targetDestroyed的事件,此blockingMonitorNewDoc携程也会结束(这个是正常行为吧?)。 实际情况是,浏览器外部看起来正常,标签页也未关闭,去浏览器的导航栏手动enter,也可以正常导航至www.baidu.com。

  • Translate documentation site into Chinese

    Translate documentation site into Chinese

    If you don't know Chinese you can ignore this issue.

    这个 issue 将用于讨论中文文档翻译的工作。

    任务描述

    英文项目地址为 https://github.com/go-rod/go-rod.github.io

    翻译的任务可以直接在 crowdin 网站完成 https://crowdin.com/project/go-rod 点击想要翻译的文件就可以开始翻译了。

    项目已经在这里翻译了很多,你可以直接拿来用 ,但是很多都是机翻,可以直接拿来用,只要保证最终整体都是中文且语义通顺就可以了。

    关于领赏

    因为都是公开的,应该不存在你干了我赖账的情况,所以直接微信汇款啥的应该没问题,实在不放心的可以上猪八戒开账户接单,地址是 https://task.zbj.com/327973710956883968?pst=i-mytask-list-zbj-23844463-1-1

    500 块嫌少的话可以再讨论,我也是第一次尝试这种想法。如果这个模式可行,以后英文文档更新就可以反复使用。

    常见问题

    你可能会问我为啥不自己翻译呢?主要是文档后期更新需要有更自动化的方式同步,目前没有足够强大的 AI 能替换人这个部件,只能花钱来转换问题?钱可以驱动人来自动化这个步骤,只不过运行的不是程序而是人。

  • Reconnect to the same browser and use WaitRequestIdle will hang forever

    Reconnect to the same browser and use WaitRequestIdle will hang forever

    Rod Version: v0.52.0

    To Reproduce

    func main() {
    	u := launcher.New().Headless(false).RemoteDebuggingPort(8765).MustLaunch()
    	b := rod.New().Timeout(30 * time.Second).ControlURL(u).MustConnect()
    
    	page := b.MustPages()[0]
    	nav(page)
    	fmt.Println("1")
    	nav(page)
    	fmt.Println("2")
    }
    
    func nav(page *rod.Page) {
    	wait := page.MustWaitRequestIdle()
    	page.MustNavigate("https://www.taobao.com")
    	wait()
    }
    
    go run ./main.go
    go run ./main.go
    

    Expected behavior

    1
    2
    1
    2
    

    What you expected to get.

    Actual result

    1
    2
    1
    panic: context deadline exceeded
    
  • memory leak by use HijackRequests()

    memory leak by use HijackRequests()

    Rod Version: v0.111.0

    The code to demonstrate your question

    func PageNavigateWithProxy(page *rod.Page, proxyUrl string, desURL string, timeOut time.Duration) (*rod.Page, *proto.NetworkResponseReceived, error) {
    
    	router := page.HijackRequests()
    	defer router.Stop()
    
    	router.MustAdd("*", func(ctx *rod.Hijack) {
    		px, _ := url.Parse(proxyUrl)
    		err := ctx.LoadResponse(&http.Client{
    			Transport: &http.Transport{
    				Proxy:           http.ProxyURL(px),
    				TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
    			},
    		}, true)
    		if err != nil {
    			return
    		}
    	})
    	go router.Run()
    
    	err := page.SetUserAgent(&proto.NetworkSetUserAgentOverride{
    		UserAgent: RandomUserAgent(true),
    	})
    	if err != nil {
    		if page != nil {
    			page.Close()
    		}
    		return nil, nil, err
    	}
    	var e proto.NetworkResponseReceived
    	wait := page.WaitEvent(&e)
    	err = rod.Try(func() {
    		page.Timeout(timeOut).MustNavigate(desURL).MustWaitLoad()
    		wait()
    	})
    	if err != nil {
    		return page, &e, err
    	}
    	if page == nil {
    		return nil, nil, errors.New("page is nil")
    	}
    
    	return page, &e, nil
    }
    

    What you got

    对于本项目的一些代码进行了封装,下面这个函数的目标是在每一个 page 独立使用一个代理去访问页面。 As a wrapper around some of the code for this project, the goal of the following function is to use a proxy on each page to access the page independently

    my code

    这里的 page 都会在使用完毕后进行 page.Close() 操作。 The page here is page.close () when used.

    1. 我不确定是 rod 的 HijackRequests 的问题,还是我使用的问题导致的 I'm not sure this problem if it was rod's HijackRequests or my use
    2. 下面会给出我的代码,以及 pprof 的两个文件 My code is given below, along with the two files for pprof
    3. 目前反复使用这个函数一段时间后,内存会缓慢的增长 Currently, after repeated use of this function over a period of time, memory will slowly grow pprof Files:

    查看这两个文件的差异 Look at the differences between the two files

    go tool pprof -http=':8081'  -diff_base .\pprof.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz .\pprof.alloc_objects.alloc_space.inuse_objects.inuse_space.002.pb.gz
    

    这个是截图 This is a screenshot image

    内存缓慢的增长。 Memory growth is slow.

    What you expected to see

    希望能够给出一个方向如何定位这个看起来想泄露的问题,是我使用不挡,还是确实是 bug。 I hope to give some direction on how to locate this seemingly leaky issue, whether I'm using it or whether it's actually a bug.

    What have you tried to solve the question

    我查看了 rod 项目中相关的两个 test 文件的示例: I looked at two examples of test files associated with Project rod

    • https://github.com/go-rod/rod/blob/62a3ffffcb6ea591cc592d3e98b4e319b0ca35ec/hijack_test.go
    • https://github.com/go-rod/rod/blob/e82236fa80dc054e4d55a217da1ecc31909ebe9a/examples_test.go
  • How do you determine if the load is complete for requests like XHR and FETCH?

    How do you determine if the load is complete for requests like XHR and FETCH?

    Some websites are partially refreshed by XHR. The wait function does not seem to work. You can only use time.sleep, but time.sleep does not know how long to sleep, if not sleep, Mustclick will produce panic. Do you have any good solutions? thank you

  • Send cdp request to non-exist sessions will cause hang

    Send cdp request to non-exist sessions will cause hang

    Rod Version: v0.80.0

    The code to demonstrate your question

    // in the top level rod directory
    go test -run 'Test/Hijack'
    

    Also two other unit tests reliably hang for me. 'Test/PageWait' and 'Test/Mouse'. The other 160 odd pass.

    What have you tried to solve the question

    I've tried up to yesterday's commit, 'make test more stable', and the results are the same.

    Performing a git bisect, on my darwin system, with the latest Chrome (version 87), the last commit where they pass is a32cd66. The next one, 39dcf1c, causes each of the three tests listed above to hang.

  • 可以增加一个真实登录成功保存cookie,并后续读取导入到浏览器里的例子吗

    可以增加一个真实登录成功保存cookie,并后续读取导入到浏览器里的例子吗

    请教大家有用json格式cookie进行登录的嘛 我试了可能是构造&proto.NetworkCookieParam的参数或者其他问题,导致FB的账号10个能有1个能登录成功,而用selenium都可以登录成功,这种是怎么回事呢,大家谁能提供下这方面的例子吗,万分感谢。 cookie格式是

    {"domain":"facebook.com","expirationDate":1648615149,"hostOnly":false,"httpOnly":false,"name":"datr", "path":"/","sameSite":"no_restriction","secure":false,"session":false,"storeId":"0","value":"tr5y48r94r5410WNz","id":0}
    

    相关代码:

    cookies := &proto.NetworkCookieParam{
    			Name:  name,
    			Value: value,
    			URL:   “https://www.facebook.com”, 
    			Domain:   domain,
    			Expires:  &proto.TimeSinceEpoch{Time: gconv.Time(expires)}, 
    			HTTPOnly: httpOnly,
    			Path:     path,
    			SameSite: proto.NetworkCookieSameSite(sameSite),
    			Secure:   secure,
    		}
    page.SetCookies(cookies)
    

    我尝试了这种,设置所有NetworkCookieParam参数的方式,也试了只有Name,Value,URL的方式,也试了Name,Value,Domain的方式,效果是一样的,都是极少数能登录成功,我觉得可能是我参数构造的方式有问题,希望能添加这方面的例子,谢谢!

  • Proper Incognito initialization and user data/cookie separation between different instances

    Proper Incognito initialization and user data/cookie separation between different instances

    Rod Version: v0.112.2

    The code to demonstrate your question

    // This is the template to demonstrate how to test Rod.
    func TestRod(t *testing.T) {
    	userDataDir := filepath.Join(os.TempDir(), fmt.Sprintf("bot-%s", "some-name"))
    
    	url := launcher.New().
    		UserDataDir(userDataDir).
    		Proxy("127.0.0.1:8080").
    		WorkingDir("").
    		Devtools(false).
    		Headless(false)
    
    	_ = rod.New().
    		ControlURL(url.MustLaunch()).
    		MustIncognito().
    		NoDefaultDevice()
    }
    

    What you got

    Such as what error you see.

    parallel test 16
    --- FAIL: TestRod (0.21s)
    panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    	panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x76c24e]
    
    goroutine 72 [running]:
    testing.tRunner.func1.2({0x863800, 0xfe3100})
    	/usr/lib/go/src/testing/testing.go:1396 +0x24e
    testing.tRunner.func1()
    	/usr/lib/go/src/testing/testing.go:1399 +0x39f
    panic({0x863800, 0xfe3100})
    	/usr/lib/go/src/runtime/panic.go:884 +0x212
    github.com/go-rod/rod.(*Browser).Call(0xc00012c000, {0xd5e650?, 0xc000030120?}, {0x0, 0x0}, {0x9257db, 0x1b}, {0x8dff00, 0xc00013a080})
    	/home/mamadou/dev/rod/browser.go:235 +0x6e
    github.com/go-rod/rod/lib/proto.call({0x9257db, 0x1b}, {0x8dff00, 0xc00013a080}, {0x8322e0?, 0xc0005320f0}, {0xd5a760, 0xc00012c000})
    	/home/mamadou/dev/rod/lib/proto/a_interface.go:63 +0x111
    github.com/go-rod/rod/lib/proto.TargetCreateBrowserContext.Call(...)
    	/home/mamadou/dev/rod/lib/proto/target.go:240
    github.com/go-rod/rod.(*Browser).Incognito(0xc00012c000?)
    	/home/mamadou/dev/rod/browser.go:82 +0xe5
    github.com/go-rod/rod.(*Browser).MustIncognito(0xc00012c000)
    	/home/mamadou/dev/rod/must.go:62 +0x25
    github.com/go-rod/rod_test.TestRod(0x0?)
    	/home/mamadou/dev/rod/rod_test.go:26 +0x1b2
    testing.tRunner(0xc000103a00, 0xbb1340)
    	/usr/lib/go/src/testing/testing.go:1446 +0x10b
    created by testing.(*T).Run
    	/usr/lib/go/src/testing/testing.go:1493 +0x35f
    exit status 2
    FAIL	github.com/go-rod/rod	6.570s
    

    What you expected to see

    The browser window is in incognito mode.

    What have you tried to solve the question

    Nothing.

  • page request error [net::ERR_ABORTED 521]

    page request error [net::ERR_ABORTED 521]

    Rod Version: v0.112.2

    The code to demonstrate question

    func main() {
    	u := launcher.New().
    		Headless(false).
    		Set("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36").
    		MustLaunch()
    
    	browser := rod.New().
    		ControlURL(u).
    		MustConnect().
    		MustIncognito().
    		MustIgnoreCertErrors(true)
    
    	page := stealth.MustPage(browser)
    	err := page.Navigate("https://ggzyjyzx.tl.gov.cn/tlsggzy/ShowInfo/Jysearch.aspx?infotype&fbdate=all&jyly=&ywtype=006&zbfs=")
    	if err != nil {
    		fmt.Print(err)
    		return
    	}
    	selector := "#result > div.ewb-list_bd > ul > li:nth-child(1) > div > a"
    	err = page.Timeout(10*time.Second).WaitElementsMoreThan(selector, 0)
    	if err != nil {
    		fmt.Print(err)
    		return
    	}
    
    	page.MustElement(selector).MustClick()
    }
    

    What you got

    click page request error [net::ERR_ABORTED 521]

    What you expected to see

    request success

    What have you tried to solve the question

    solve code

    Set("user-agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36")
    stealth.MustPage(browser); //navicate is ok
    
    // new page clear cookie,inject stealth.JS
    go browser.EachEvent(func(e *proto.TargetTargetCreated) {
    		if e.TargetInfo.Type != proto.TargetTargetInfoTypePage {
    			return
    		}
    		page := browser.MustSetCookies().MustPageFromTargetID(e.TargetInfo.TargetID)
    		_, err := page.EvalOnNewDocument(stealth.JS)
    		if err != nil {
    			return
    		}
    	})()
    
    
  • Better way to Get LocalStroage and Set LocalStroage

    Better way to Get LocalStroage and Set LocalStroage

    Rod Version: v0.112.2

    //GET LOCALSTROAGE
    localStorage := page.MustEval(`k => Object.assign({}, window.localStorage)`).Map()
    for key, value := range localStorage {
    fmt.Println("Key:", key, "=>", "Value:", value)
    }
    
    //SET LOCALSTROAGE
    page.MustNavigate("target url")
    
       for key, value := range localStorage {
          page.MustEval(`(key,value) => {
          window.localStorage.setItem(key,value)
       }`, key, value)
    }
    
    page.MustNavigate("target url")
    

    What you expected to see

    Set LocalStorage of target website before navigate

  • DefaultNavigationTimeout and catch requestfailed in go-rod

    DefaultNavigationTimeout and catch requestfailed in go-rod

    Rod Version: v0.112.2

    We can do it with the following code with Node.js Puppeteer

    page.setDefaultNavigationTimeout(120 * 1000)
    page.on('requestfailed', request => {
       if(request.frame() == page.mainFrame() && request.isNavigationRequest()){
           page.reload();
       }
    });
    

    What you expected to see

    When clicked a button , it will or will not navigate to other unknown page. I expected to see that the page will reload when navigation fail or timeout

    What have you tried to solve the question

    As we know that , we can use the following code to catch the navigation timeout or error of "known url"

    err := page.Timeout(10*time.Second).Navigate(url)
    if err != nil {
     fmt.Println(err.Error())
    }
    

    But this situation is nevigating to an "unknown url"

  • Cannot launch 2+ profiles from the one userDataDir

    Cannot launch 2+ profiles from the one userDataDir

    Rod Version: v0.112.2

    Hi. Trying to launch two profiles from one user data dir then close it and just reopen after 10 seconds and repeat it...

    package main
    
    import (
    	"strconv"
    	"time"
    
    	"github.com/go-rod/rod"
    	"github.com/go-rod/rod/lib/launcher"
    	"github.com/go-rod/rod/lib/utils"
    )
    
    func main() {
    
    	for i := 0; i < 2; i++ {
    		go func(index int) {
    			profileIndex := strconv.Itoa(index)
    
    			ln := launcher.NewUserMode()
    
    			ctl := ln.
    				Bin("C:/Program Files/Google/Chrome/Application/chrome.exe").
    				Leakless(false).
    				Headless(false).
    				Set("user-data-dir", "C:/ChromeProfiles").
    				Set("disable-web-security").
    				Set("allow-running-insecure-content").
    				Set("profile-directory", "Profile "+profileIndex).
    				MustLaunch()
    
    			for {
    				browser := rod.New().
    					ControlURL(ctl).
    					SlowMotion(5 * time.Millisecond).
    					NoDefaultDevice().
    					MustConnect()
    
    				browser.MustPage("https://example.com").MustSetWindow(50, 50, 800, 600)
    
    				utils.Sleep(30)
    
    				browser.Close()
    
    				utils.Sleep(5)
    			}
    		}(i)
    
    		utils.Sleep(1)
    	}
    
    	utils.Pause()
    
    }
    
    

    Trying to launch two profiles from the some user data dir and control it... but cant, why? maybe im do something wrong or how can i launch two profiles as different instances from the some user data dir?

  • Can't access downloaded files when using remote browser

    Can't access downloaded files when using remote browser

    Rod Version: v0.112.0

    Hi! I'm facing an issue when trying to access downloaded files of a remotely managed browser.

    The code to demonstrate your question

    Before running the code you need to run the rod image docker run -p 7317:7317 ghcr.io/go-rod/rod

    see: https://go-rod.github.io/#/custom-launch?id=remotely-manage-the-launcher

    func TestRod(t *testing.T) {
    	l := launcher.MustNewManaged("")
    
    	browser := rod.New().
    		Client(l.MustClient()).
    		MustConnect()
    	defer browser.MustClose()
    
    	page := browser.MustPage("https://file-examples.com/index.php/sample-documents-download/sample-pdf-download/")
    
    	wait := browser.MustWaitDownload()
    
    	page.MustElementR("a", "DOWNLOAD SAMPLE PDF FILE").MustClick()
    
    	_ = utils.OutputFile("t.pdf", wait()) //test fail here trying to access downloaded file
    }
    
    

    What you got

    ➜ go test -run TestRod parallel test 8 --- FAIL: TestRod (6.23s) panic: open /tmp/rod/downloads/980336a8-57d8-4bb2-be56-50d0cdca9670: no such file or directory [recovered]

    What you expected to see

    ➜ go test -run TestRod parallel test 8 PASS ok github.com/go-rod/rod 6.615s

    Test passed and file t.pdf created

    What have you tried to solve the question

    Using a local browser works fine and the file is created.

    I think (maybe) that I can make this work by hijacking the requests and getting the response from the hijack function, but I'm not quite sure because inspecting the browser network resources doesn't show any http request with the downloaded file in the response.

    Is there any way to make the MustWaitDownload() function work with a remote browser?

A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.

About chromedp Package chromedp is a faster, simpler way to drive browsers supporting the Chrome DevTools Protocol in Go without external dependencies

Jan 4, 2023
A faster, simpler way to drive browsers supporting the Chrome DevTools Protocol.

About chromedp Package chromedp is a faster, simpler way to drive browsers supporting the Chrome DevTools Protocol in Go without external dependencies

Dec 28, 2022
End to end functional test and automation framework
End to end functional test and automation framework

Declarative end to end functional testing (endly) This library is compatible with Go 1.12+ Please refer to CHANGELOG.md if you encounter breaking chan

Jan 6, 2023
Playwright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.
Playwright for Go a browser automation library to control Chromium, Firefox and WebKit with a single API.

?? Playwright for API reference | Example recipes Playwright is a Go library to automate Chromium, Firefox and WebKit with a single API. Playwright is

Jan 1, 2023
Ritchie CLI is an open-source tool that allows to create, store and share any kind of automation, executing them through command lines, to run operations or start workflows ⚙️ 🖥 💡
Ritchie CLI is an open-source tool that allows to create, store and share any kind of automation, executing them through command lines, to run operations or start workflows ⚙️ 🖥 💡

Table of contents 1. About 2. Getting Started i. Installation ii. Initialize rit locally iii. Add your first formulas repository iv. Run the Hello Wor

Dec 29, 2022
Sql mock driver for golang to test database interactions

Sql driver mock for Golang sqlmock is a mock library implementing sql/driver. Which has one and only purpose - to simulate any sql driver behavior in

Dec 31, 2022
Immutable transaction isolated sql driver for golang

Single transaction based sql.Driver for GO Package txdb is a single transaction based database sql driver. When the connection is opened, it starts a

Jan 6, 2023
This benchmark provides a overview of the different SQLite driver performances available in Go.

SQLite/HTTP Server Performance Benchmark This benchmark provides a overview of the different SQLite driver performances available in Go. For benchmark

Aug 8, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Jan 2, 2023
Plow is a high-performance HTTP benchmarking tool with real-time web UI and terminal displaying
Plow is a high-performance HTTP benchmarking tool with real-time web UI and terminal displaying

Plow is a HTTP(S) benchmarking tool, written in Golang. It uses excellent fasthttp instead of Go's default net/http due to its lightning fast performance.

Jan 9, 2023
Test your command line interfaces on windows, linux and osx and nodes viá ssh and docker

Commander Define language independent tests for your command line scripts and programs in simple yaml files. It runs on windows, osx and linux It can

Dec 17, 2022
Record and replay your HTTP interactions for fast, deterministic and accurate tests

go-vcr go-vcr simplifies testing by recording your HTTP interactions and replaying them in future runs in order to provide fast, deterministic and acc

Dec 25, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Nov 28, 2022
Quick and easy expression matching for JSON schemas used in requests and responses

schema schema makes it easier to check if map/array structures match a certain schema. Great for testing JSON API's or validating the format of incomi

Dec 24, 2022
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Dec 12, 2022
bencode is a golang package for bencoding and bdecoding data from and from to equivalents.

Bencode bencode is a golang package for bencoding and bdecoding data from and from to equivalents. Bencode (pronounced like Ben-code) is the encoding

Jan 8, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Dec 22, 2022
Minimal and Beautiful Go testing framework
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

Dec 25, 2022
A collection of packages to augment the go testing package and support common patterns.

gotest.tools A collection of packages to augment testing and support common patterns. Usage With Go modules enabled (go1.11+) $ go get gotest.tools/v3

Jan 4, 2023