URL-Collector url采集器
Usage
NAME:
URL-Collector - Collect URLs based on dork
USAGE:
url-collector
VERSION:
v0.2
AUTHOR:
无在无不在 <[email protected]>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--input value, -i value input from a file
--output value, -o value specify the output file
--engine value, -e value specify the search engine(google,bing,baidu,google-image) (default: "google-image")
--routine-count value, -r value specify the count of goroutine (default: 5)
--keyword value, -k value specify the keyword
--config-file value, -c value specify the config file
--format value, -f value specify output format(url、domain、protocol_domain) (default: "url")
--help, -h show help (default: false)
--version, -v print the version (default: false)
ERRO[0000] specify -f or -k please
Screenshot
Demo
#根据指定关键字采集
url-collector -k ".php?id="
#批量采集文件中所有的关键字，并将结果保存到result.txt
url-collector -f google-dork.txt -o result.txt
#和sqlmap联动
url-collector -f google-dork.txt -o result.txt && python3 sqlmap.py -m result.txt --batch --random-agents
#默认采用google镜像站点，如果你可以访问外网，可以手动指定搜索引擎为google
url-collector -e google -k ".php?id="
#百度url采集
url-collector -e baidu -k ".php?id=1"
#将常用配置写到配置文件中
url-collector -c config.json
#输出域名
url-collector -e baidu -k ".php?id=1" -f domain
#输出协议加域名
url-collector -e baidu -k ".php?id=1" -f protocol_domain
config.json 格式
{
"output_file_path":"",
"input_file_path":"",
"keyword":"inurl:.php?id=",
"search_engine":"google-image",
"format":"url",
"base_url":{
"google": "https://www.google.com/search?q=$keyword",
"google-image": "https://g.luciaz.me/search?q=$keyword",
"bing": "https://cn.bing.com/search?q=$keyword",
"baidu": "https://www.baidu.com/s?wd=$keyword"
},
"routine_count":5,
"black_list":[
"gov",
"baidu.com",
"cache.baiducontent.com",
"g3.luciaz.me",
"www.youtube.com",
"gitee.com",
"github.com",
"stackoverflow.com",
"developer.aliyun.com",
"cloud.tencent.com",
"www.zhihu.com/question",
"blog.51cto.com",
"zhidao.baidu.com",
"www.cnblogs.com",
"coding.m.imooc.com",
"weibo.cn",
"www.taobao.com",
"www.google.com",
"go.microsoft.com",
"facebook.com",
"blog.csdn.net",
"books.google.com",
"policies.google.com",
"webcache.googleusercontent.com",
"translate.google.com"
]
}
google-dork demo
<<<<<<< HEAD
=======
18300e2f68cfa2602849ad50a26f14965b7858d2
