OSINT: Finding Subdomains

Many of us are in the security consulting business, or bug bounties, or even network intelligence and have now and then come across a need to find subdomains. The requirement can be from either side of the table - a consultant assessing a client’s internet presence, or a company validating its own digital footprint. In more than a decade, it has happened so many times that people are not aware of what old assets are they running, and hence can be exploited to either damage the brand image, or actual networks. These assets can also be used as the proxy or hops to gain access to thought-so-well guarded data.

Most common way to search for subdomains (that I have used) so far is old school Google search with dorks: site:example.com. To dig deeper, iterate it with all visible subdomains from results, i.e. site:example.com -www or site:example.com -www -test. It will exclude example.com, www.example.com and test.example.com and so on. Later I found some more tools like, [Pentest Tool Subdomain]¹, [DNS Dumster]², [Cloudpiercer]³, [Netcraft]⁴ etc. All these tools are either expensive or don’t do the job well. Meh!

Finally, while having a [conversation with the SPYSE team]⁵ (the astounding squad behind [CertDB project]⁶) and I got to know about their new project - [FindSubDomains]⁷, a free and fantastic tool/ project to find subdomains for a given domain. Last time I covered their [CertDB project in detail]⁸, and now after being impressed by FindSubDomains, it was time to review and share with you this excellent tool! It not only lists subdomains but a whole lot of intelligence behind it like,

1. IP Addresses
2. DNS records
3. Countries
4. Subnets
5. AS Blocks
6. Organization names etc.

Any of these parameters can be used to filter the list of subdomains, or search - I mean, it’s terrific!

But how does this stack against the common known tools? Let’s find out. For the sake of testing, let’s take the domain apple.com and try finding the subdomains with different tools/ mediums. Let’s start with old school google search,

Only after 4-5 searches/iterations, it became a tedious process. And, when you try to automate it; Google merely pops up re-captcha challenge. In general, it’s worth to search few targeted domains, but worthless to query wild subdomains. Not recommended for such tasks!

How about using the pentest-tools tool? First thing first, it is not a free service and would require you to buy credits. I just performed a free search, and the results were not convincing with pentest-tools,

After the search, it could only find 87 subdomains of apple.com, and the details included subdomain and respective IP addresses. Netcraft and DNSDumster also had the same disappointing results - the first found 180 records with no scope to download or filter them, and the later was capped at 150 results with lousy UI/UX. To summarise none of the tools could deliver a straightforward and intelligent list of subdomains.

Is it any different; any better?

To give you a straight answer - Hell yes! Kudos to the [SPYSE team]⁹, it is way better than the ones I were using before.

The same apple.com subdomain search performed via [FindSubDomains]¹⁰resulted in 1900+ results. It is remarkable!

I mean when others failed miserably to provide even 200 results, [FindSubDomains]¹¹ just nailed it with 1900+ results. Bravo!

All of these 1900+ results are at your disposal without paying a single cent, pop-up advertisement, credits or cap etc. You not only can list these results on the UI but also download them as TXT file. Also, you can view the IP address, geographical region, IP segment and respective AS block details for each subdomain. That is some remarkable open source intelligence in a second without scripts or endless iterations!

To me, the [SPYSE team]¹² 100% justify their project objective,

FindSubDomains is designed to automate subdomains discovering. This task often comes before system security specialists who study the company in the form of a black box in order to search for vulnerabilities, as well as for marketers and entrepreneurs whose goal is to competitively analyze other players on the market in the hope of quickly learning about the emergence of a new vector in the development of a competitor, as well as obtaining information about the internal infrastructure.

Search and Filter

On top of the search, their filters are amazing if you have to search specific information on a domain, subdomain or it’s respective fields as discussed. They also have some pre-filtered results or trivia points,

1. Top 100 sites and their subdomains: [https://findsubdomains.com/world-top]
2. Sites with the highest number of subdomains: [https://findsubdomains.com/top-subdomain-count]
3. Top countries with the highest number of subdomains: [https://findsubdomains.com/countries]
   • UNITED STATES: [https://findsubdomains.com/world-top?country=US]
   • INDIA: [https://findsubdomains.com/world-top?country=IN]
   • CHINA: [https://findsubdomains.com/world-top?country=CN]
4. Top names for subdomains (my favourite) or most common subdomains: [https://findsubdomains.com/top-subdomain-name-count]

The last one is convenient when network surveying a client, or shocking client with their digital footprint.

Dashboard and Custom Tasks

Now, when I signed-in (sign-up is easy) I was welcomed by a Dashboard which shows Total, Ongoing and Remaining tasks. I can start a new task by either using the domain or a word to search. The word search is great if I don’t know the complete domain name. This task executing capability is to supplement anything you don’t find on their main page, or existing database (which believe me is huge). For every task, it can list up to 50,000 subdomains for and takes around 6 minutes (you can setup alert, and the platform will notify you via email on its completion).

To execute the task of finding subdomains, it uses various techniques,

1. Many subdomains can be defined using the site’s crawling and analyzing its pages, as well as resource files;
2. AXFR (DNS Zone Transfer) requests for some domains often reveal a lot of valuable information about them;
3. Search and analysis of historical data often matches with the search term.

While the tool is impressive, and I can’t repeat enough; I would have appreciated the capability to execute the tasks via an API, and having some programmable way to automate via command-line/ terminal. I know, I may find ways to do with curl, but API key would have made things more comfortable, and convenient.

Usage Scenarios

Here are some scenarios I can use this tool,

1. During pentest reconnaissance phase, collecting information on the target network.
2. As a supporting tool to gather network intelligence on firms and their respective domains.
3. Assessing your company’s network, and digital footprint. Many a times you will be surprised to find the wide unaccounted exposure.
4. Keeping a track of external facing subdomains - UAT, SIT, STAGING etc. which ideally should either be locked down or white-listed. Imagine how insecure are these platforms which often even contain production data.

To summarize, this is yet another amazing tool after [CertDB]¹³ which shows the potential of SPYSE team. The [FindSubDomains]¹⁴ has made my search so easier and efficient. I would highly recommend the readers to use this tool in finding subdomains.

Cover Image: Photo by [Himesh Kumar Behera]¹⁵