Home etc Blogs FOSS Security Tools: Subfinder

FOSS Security Tools: Subfinder

0
43
Subfinder

The fourth article in the ongoing series on FOSS security tools introduces the subdomain discovery tool called subfinder.

Subfinder is optimised for speed and is easy to use on websites. Designed to use passive online sources to maximise the results, it includes wildcard elimination modules. You can produce the results to standard output, file or JSON formats. It is written using the Go programming language and released under the MIT licence.

We will use Go version 1.22.4 installed on Parabola GNU/Linux-libre (x86-64) for the subfinder examples.

Installation

The Parabola GNU/Linux-libre (x86-64) is a free, lightweight, Libre rolling-release distribution which is based on Arch Linux.

$ cat /etc/os-release
NAME=Parabola
PRETTY_NAME=”Parabola GNU/Linux-libre”
ID=parabola
ID_LIKE=arch
BUILD_ID=rolling
VARIANT=”x86_64 SystemD Edition”
VARIANT_ID=”x86_64-systemd”
ANSI_COLOR=”1;35”
HOME_URL=”https://www.parabola.nu/”
DOCUMENTATION_URL=”https://wiki.parabola.nu/”
SUPPORT_URL=”ircs://irc.libera.chat/#parabola”
BUG_REPORT_URL=”https://labs.parabola.nu/”
LOGO=”parabola-logo”

You need to first install Go on the system using the ‘pacman’ package management tool as follows:

$ sudo pacman -S go

The GOPATH and PATH environment variables need to be set in ~/.bashrc.

GOPATH=$HOME/go
PATH=$PATH:$GOROOT/bin:$GOPATH/bin

You will then need to source the ~/.bashrc file for the settings to take effect:

$ source ~/.bashrc

You can verify the version of Go installed using the following command:

$ go version
go version go1.22.4 linux/amd64

You can now install subfinder using the “go” executable as shown below:

$ go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
go: downloading github.com/projectdiscovery/subfinder/v2 v2.6.6
go: downloading github.com/projectdiscovery/subfinder v2.6.6+incompatible
go: downloading github.com/projectdiscovery/fdmax v0.0.4
...
github.com/projectdiscovery/utils/update
github.com/projectdiscovery/subfinder/v2/pkg/runner
github.com/projectdiscovery/subfinder/v2/cmd/subfinder

Docker

You can also use the subfinder Docker image instead of doing a local installation in order to use the tool. For example:

$ sudo docker pull projectdiscovery/subfinder:latest

If you want to pass specific flags to the invocation with Docker, you can pass the options to the Docker command as follows:

$ sudo docker run projectdiscovery/subfinder:latest -d google.com

Version

The ‘-version’ option of subfinder prints the installed version of subfinder. As of writing this article, the latest version is v2.6.6.

$ subfinder -version 
[INF] Current Version: v2.6.6 
[INF] Subfinder Config Directory: /home/guest/.config/subfinder

Usage

The ‘-h’ option of subfinder provides a quick summary and description of the various option flags supported by the tool.

$ subfinder -h
Subfinder is a subdomain discovery tool that discovers subdomains for websites by using passive online sources.
Usage:
subfinder [flags]
Flags:
INPUT:
-d, -domain string[] domains to find subdomains for
-dL, -list string file containing list of domains for subdomain discovery
SOURCE:
-s, -sources string[] specific sources to use for discovery (-s crtsh,github). Use -ls to display all available sources.
-recursive use only sources that can handle subdomains recursively (e.g. subdomain.domain.tld vs domain.tld)
-all use all sources for enumeration (slow)
-es, -exclude-sources string[] sources to exclude from enumeration (-es alienvault,zoomeyeapi)
FILTER:
-m, -match string[] subdomain or list of subdomain to match (file or comma separated)
-f, -filter string[] subdomain or list of subdomain to filter (file or comma separated)
...

Sources

The ‘-ls’ or ‘-list-sources’ option provides the list of all available sources as demonstrated below:

$ subfinder -ls
__ _____ __
_______ __/ /_ / __(_)___ ____/ /__ _____
/ ___/ / / / __ \/ /_/ / __ \/ __ / _ \/ ___/
(__ ) /_/ / /_/ / __/ / / / / /_/ / __/ /
/____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/

projectdiscovery.io

[INF] Current subfinder version v2.6.6 (latest)
[INF] Current list of available sources. [39]
[INF] Sources marked with an * need key(s) or token(s) to work.
[INF] You can modify /home/guest/.config/subfinder/provider-config.yaml to configure your keys/tokens.

alienvault
anubis
bevigil *
binaryedge *
bufferover *
c99 *
censys *
certspotter *
chaos *
...
waybackarchive
whoisxmlapi *
zoomeyeapi *
facebook *
builtwith *

Domain

You need to use the ‘-d’ or ‘-domain string[]’ option to specify the domain for which you want to find all the sub-domains. In the following example, we are searching for all the sub-domains for google.com.

$ subfinder -d google.com
...
googleproxy-66-249-80-193.google.com
googleproxy-64-233-172-24.google.com
ww.google.com
firebaserules.clients6.google.com
googleproxy-66-102-7-180.google.com
googleproxy-64-233-173-21.google.com
...
[INF] Enumerating subdomains for google.com
[INF] Found 5875 subdomains for google.com in 2 seconds 434 milliseconds

Verbose

You can request for a more verbose output using the ‘-v’ option of subfinder as demonstrated below:

$ subfinder -d google.com -v
...
[alienvault] tuf-autopush.corp.google.com
[alienvault] staging-git.corp.google.com
[alienvault] docs-qa.corp.google.com
...
...
[crtsh] docs.google.com
[crtsh] mail.google.com
[crtsh] plus.google.com
...
[dnsdumpster] google-proxy-74-125-212-200.google.com
[dnsdumpster] google-proxy-64-233-172-200.google.com
[dnsdumpster] google-proxy-66-249-82-200.google.com
[INF] Found 5875 subdomains for google.com in 2 seconds 459 milliseconds

Match

The ‘-m’ or ‘-match’ option allows you to match only a specific string in the subdomain or list of subdomains. In the following example, we are searching for the subdomains that have ‘googleproxy’ in the name.

$ subfinder -m “googleproxy-*” -d google.com
...
googleproxy-66-249-82-201.google.com
googleproxy-64-233-172-33.google.com
googleproxy-66-102-7-221.google.com
googleproxy-66-249-84-51.google.com
...
googleproxy-66-249-84-91.google.com
googleproxy-66-249-88-34.google.com
googleproxy-64-233-172-180.google.com
[INF] Found 1761 subdomains for google.com in 4 seconds 692 milliseconds

Filter

You can omit certain patterns in the output using the ‘-f’ or ‘-filter string[]’ option to exclude a subdomain or list of subdomains. For example:

$ subfinder -f “googleproxy-*” -d google.com
...
mailiu0-f50.google.com
ratelimited-proxy-209-85-238-74.google.com
mailua0-f193.google.com
mailua0-f254.google.com
lens.google.com
ratelimited-proxy-203-208-38-180.google.com
mail-vs1-f10.google.com
[INF] Found 4114 subdomains for google.com in 4 seconds 700 milliseconds

Rate limit

The ‘-rl’ or ‘-rate-limit int’ option can restrict the number of HTTP requests sent per second to the website. It accepts an integer as an argument. For example:

$ subfinder -rl 2 -d google.com
...
rate-limited-proxy-74-125-150-100.google.com
rate-limited-proxy-108-177-69-120.google.com
r2.sn-npoeene6.c.docs.google.com
mailio0-f171.google.com
...
[INF] Enumerating subdomains for google.com
[INF] Found 5875 subdomains for google.com in 10 seconds 343 millisecond

Concurrency

You can also specify the number of concurrent goroutines to be executed using the ‘-t int’ option. It accepts an integer argument, and the default value is 10.

$ subfinder -t 20 -d google.com
...
mailqt0-f203.google.com
mail-vs1-f110.google.com
cag.ext.google.com
ratelimited-proxy-66-249-91-10.google.com
...
[INF] Enumerating subdomains for google.com
[INF] Found 5875 subdomains for google.com in 11 seconds 616 milliseconds

Output

subfinder supports multiple output formats to show the results. The ‘-o’ or ‘-output string’ option writes the results to a file. For example:

$ subfinder -m “googleproxy-*” -d google.com -o /tmp/google.txt

The ‘-oJ’ or ‘-json’ flag converts each entry into a JSON structure that can be consumed by downstream applications.

$ subfinder -m “googleproxy-*” -d google.com -oJ
...
{“host”:”googleproxy-66-249-83-11.google.com”,”input”:”google.com”,”source”:”anubis”}
{“host”:”googleproxy-66-249-93-163.google.com”,”input”:”google.com”,”source”:”anubis”}
{“host”:”googleproxy-66-249-88-170.google.com”,”input”:”google.com”,”source”:”anubis”}
{“host”:”googleproxy-66-249-80-64.google.com”,”input”:”google.com”,”source”:”anubis”}
[INF] Found 1761 subdomains for google.com in 3 seconds 562 milliseconds

You can also use the ‘-oD’ or ‘-output-dir string’ option to write the output to a directory. The ‘-cs’ or ‘-collect-sources’ option includes all the sources in the results when used only with the ‘-json’ flag. The ‘-oI’ or ‘-ip’ option includes the host IP address in the output.

Statistics

The information on source statistics can be obtained using the ‘-stats’ flag. A sample execution is given below:

$ subfinder -d google.com -stats
...
[INF] Found 5875 subdomains for google.com in 7 seconds 292 milliseconds
[INF] Printing source statistics for google.com
Source Duration Results Errors
────────────────────────────────────────────────────────
alienvault 7.291s 144 0
anubis 51ms 5197 0
builtwith 0s 0 0
crtsh 2.025s 105 0
digitorus 863ms 21 0
dnsdumpster 2.438s 1 0
hackertarget 1.31s 405 0
leakix 1.022s 2 0
virustotal 0s 0 0 
...

Configuration

The subfinder configuration is present in ~/.config/subfinder/config.yaml. The first few lines of the file are given below:

# subfinder config file
# generated by https://github.com/projectdiscovery/goflags
# domains to find subdomains for
#domain: []
# file containing list of domains for subdomain discovery
#list:
# specific sources to use for discovery (-s crtsh,github). use -ls to display all available sources.
#sources: []
# use only sources that can handle subdomains recursively (e.g. subdomain.domain.tld vs domain.tld)
#recursive: false
# use all sources for enumeration (slow)
#all: false
# sources to exclude from enumeration (-es alienvault,zoomeyeapi)
#exclude-sources: []
...

You can explicitly specify a configuration file using the ‘-config string’ option of subfinder. The provider configuration file is present in ~/.config/subfinder/provider-config.yaml. For example:

bevigil: []
binaryedge: []
bufferover: []
builtwith: []
c99: []
censys: []
certspotter: []
chaos: []
chinaz: []
dnsdb: []
...

The ‘-pc’ or ‘-provider-config string’ option can be used to specify a different configuration file. The list of resolvers can be specified using the ‘-r string[]’ option. You can also include them in a file and pass it to the ‘-rL’ or ‘-rlist string’ option of subfinder.

Optimisation

The ‘-timeout int’ option makes the subfinder wait for the given number of seconds before timing out. The default value is 30 seconds. The ‘-max-time int’ option specifies the time in minutes to wait for the results. The default value is 10 minutes. If you do not want colour in the output, you can use the ‘-nc’ or ‘-no-color’ option. You can also use the tool as a library in your application code.

You are encouraged to read the subfinder documentation to learn more on its usage and flag options.

NO COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here