Monday, January 23, 2006

Web Site Discovery with SensePost

Today I needed to discover Web sites for a client. I'll demonstrate part of my methodology here, using as a sample domain. I relied on a technique outlined in Johnny Long's Google Hacking for Penetration Testers. He mentions a SensePost tool called The script uses Google to extract sub domains and DNS names for a given domain. You have to register with SensePost to retrieve; they email a username and password once you register.

The first requirement is having a license key for the Google API. You put your key into, thus:

#$key = "----YOUR GOOGLE API KEY HERE----";

Since I am running the script on FreeBSD, I realized I needed the net/p5-SOAP-Lite package. I added the latest version from the STABLE package collection.

Finally I needed the file

orr:/home/richard$ fetch
fetch: size of remote file is not known
GoogleSearch.wsdl 7496 B 145 kBps

Now I'm ready to find Web sites.

orr:/home/richard$ perl ./

Adding word [site]
0 1 0 1
Adding word [web]
0 1 0 1
Adding word [document]
0 1 0 1
Adding word []
0 1 0 1
DNS names:

Sub domains:

That's it. You'll notice I found domains that end in but are not part of, like Still, this is a powerful way to use Google to identify Web servers.


Anonymous said...

You can use Netcraft too.*

Richard Bejtlich said...

Thank you -- that's a great tip!

Anonymous said...

where does one get a Google API key from ?

Richard Bejtlich said...

You have to apply for a key at

I've noticed the link to no longer works, but I found the file at

cmlh said...


GoogleSearch.wsdl can still be downloaded from

If you have installed AURA (also from SensePost), then the issue may be you have modified the host file from to localhost.