Saturday, May 22, 2010

Watch Your WHOIS Entries

Thanks to sites like the Sucuri Security blog, domain name administrators should be learning that it is important to watch for updates to WHOIS records. Companies like Sucuri offer such a service free for one domain but charge for additional domains while providing extended services. If you'd just like to monitor your own WHOIS records using a simple script, you can be inspired by last year's article Network-based integrity monitoring keeps website hacks in check by David Davidson.

I decided to create the following simple script to watch two of my domains.

richard@macmini:~/check$ cat check.sh
#/bin/sh
/usr/bin/whois bejtlich.net > /home/richard/check/bejtlich.net.whois.new.txt
/usr/bin/whois taosecurity.com > /home/richard/check/taosecurity.com.whois.new.txt

/usr/bin/diff -u /home/richard/check/bejtlich.net.whois.old.txt \
/home/richard/check/bejtlich.net.whois.new.txt | mail -s "bejtlich.net whois check" taosecurity@gmail.com
/usr/bin/diff -u /home/richard/check/taosecurity.com.whois.old.txt \
/home/richard/check/taosecurity.com.whois.new.txt | mail -s "taosecurity.com whois check" taosecurity@gmail.com

mv /home/richard/check/bejtlich.net.whois.new.txt /home/richard/check \
/bejtlich.net.whois.old.txt
mv /home/richard/check/taosecurity.com.whois.new.txt /home/richard/check \
/taosecurity.com.whois.old.txt

Is this the world's greatest shell script? No, I wrote it in 60 seconds to make my point. Feel free to create something uber-cool and post it here. :)

Next I created empty files:

$ echo "" > bejtlich.net.whois.old.txt
$ echo "" > taosecurity.com.whois.old.txt

Finally I ran the check:

$ ./check.sh

Checking my email, I got two. Here's the one for bejtlich.net:

--- /home/richard/check/bejtlich.net.whois.old.txt 2010-05-22 20:52:58.000000000 -0400
+++ /home/richard/check/bejtlich.net.whois.new.txt 2010-05-22 20:53:05.000000000 -0400
@@ -1 +1,106 @@

+Whois Server Version 2.0
+
+Domain names in the .com and .net domains can now be registered
+with many different competing registrars. Go to http://www.internic.net
+for detailed information.
+
+ Domain Name: BEJTLICH.NET
+ Registrar: GODADDY.COM, INC.
+ Whois Server: whois.godaddy.com
+ Referral URL: http://registrar.godaddy.com
+ Name Server: NS18.ZONEEDIT.COM
+ Name Server: NS8.ZONEEDIT.COM
+ Status: clientDeleteProhibited
+ Status: clientRenewProhibited
+ Status: clientTransferProhibited
+ Status: clientUpdateProhibited
+ Updated Date: 22-may-2010
+ Creation Date: 01-jul-2000
+ Expiration Date: 01-jul-2011
...truncated...

As you can see it's "all new" because the old file was empty.

When I run the check again, I should get no significant changes via email.

--- /home/richard/check/bejtlich.net.whois.old.txt 2010-05-22 20:53:05.000000000 -0400
+++ /home/richard/check/bejtlich.net.whois.new.txt 2010-05-22 20:55:28.000000000 -0400
@@ -19,7 +19,7 @@
Creation Date: 01-jul-2000
Expiration Date: 01-jul-2011

->>> Last update of whois database: Sun, 23 May 2010 00:52:33 UTC <<<
+>>> Last update of whois database: Sun, 23 May 2010 00:54:20 UTC <<<
- Hide quoted text -

NOTICE: The expiration date displayed in this record is the date the
registrar's sponsorship of the domain name registration in the registry is

You could argue not to use diff -u to simplify the output. Sure, you could. I just prefer seeing some context when changes do occur.

Now I'm going to add another DNS server to my WHOIS record and see if my script catches the change.

Reading email...

--- /home/richard/check/bejtlich.net.whois.old.txt 2010-05-22 20:55:28.000000000 -0400
+++ /home/richard/check/bejtlich.net.whois.new.txt 2010-05-22 20:58:09.000000000 -0400
@@ -10,6 +10,7 @@
Whois Server: whois.godaddy.com
Referral URL: http://registrar.godaddy.com
Name Server: NS18.ZONEEDIT.COM
+ Name Server: NS5.ZONEEDIT.COM
Name Server: NS8.ZONEEDIT.COM
Status: clientDeleteProhibited
Status: clientRenewProhibited
@@ -19,7 +20,7 @@
Creation Date: 01-jul-2000
Expiration Date: 01-jul-2011

->>> Last update of whois database: Sun, 23 May 2010 00:54:20 UTC <<<
+>>> Last update of whois database: Sun, 23 May 2010 00:57:09 UTC <<<

NOTICE: The expiration date displayed in this record is the date the
registrar's sponsorship of the domain name registration in the registry is
@@ -103,4 +104,5 @@
Domain servers in listed order:
NS18.ZONEEDIT.COM
NS8.ZONEEDIT.COM
+ NS5.ZONEEDIT.COM

There it is -- ns5.zoneedit.com. If I hadn't made that change, then I would know someone has compromised my account.

The next evolution of this script is to run it from cron, and better yet modify it so I only get an email if there is a change. For now, I have a simple way to watch for changes. Again, Sucuri should take credit for bringing this to people's attention during the last 2 years or so.

8 comments:

Anonymous said...

How often would you recommend running the script looking for differences? Daily, weekly, monthly?

Richard Bejtlich said...

I'd say start daily. If you have an incident, increase the frequency.

. said...

all your post was true.
but i can't do that.
now i'm use free hosting, because im worrying if in the next day the bad guys change this domain's dns record.
oh sure, it's wasn't an important site.but i'm afraid if someday the domain used to spreading malware or some unaware activity.
because you haven't control in real. and how if the interface was fake.
finally, choose the right registrar if you want having full control on your domain.

Royce said...

I like this idea, Richard. Thanks for the nudge. Here's a version that's abstracted a bit for more general use.

It looks like PRE tags aren't accepted, so please excuse the un-indented formatting.

#!/bin/sh

# Description: check whois for domains of interest.

MYDOMAINS="example.com"
MYEMAIL=royce@example.net
MYWORKDIR=/home/royce/check
DIFF=/usr/bin/diff
WHOIS=/usr/bin/whois

for domain in ${MYDOMAINS}; do

# If the working directory does not exist, create it.
[ ! -d ${MYWORKDIR} ] && mkdir ${MYWORKDIR}

# If baseline files don't exist, create them.
if [ ! -f ${MYWORKDIR}/${domain}.whois.old.txt ]; then
touch ${WORKDIR}/${domain}.whois.old.txt
exit 0
fi

# Fetch WHOIS record for the domain.
${WHOIS} ${domain} > ${MYWORKDIR}/${domain}.whois.new.txt

# Compare old and new WHOIS records.
${DIFF} -u ${MYWORKDIR}/${domain}.whois.old.txt \
${MYWORKDIR}/${domain}.whois.new.txt >/dev/null

# If they differ, send an email.
if [ $? != 0 ]; then
${DIFF} -u ${MYWORKDIR}/${domain}.whois.old.txt \
${MYWORKDIR}/${domain}.whois.new.txt \
| mail -s "whois check: ${domain}" ${MYEMAIL}
fi

# Save state from this run.
mv ${MYWORKDIR}/${domain}.whois.new.txt \
${MYWORKDIR}/${domain}.whois.old.txt

done

fermetabouche said...

Hi,

This is really interesting, efficient and ...cheap.

- When vendors sell Data leak prevention, you can implement a single ngrep with a predefined pattern identifying documents classification (internal, secret...etc)

- When there's a need to implement file monitoring: tripwire e.g with fine selection of WHAT files to monitor is possible, without paying that much.

- I particularily loved this article:
"http://www.ghacks.net/2009/10/22/asset-scanning-with-nmap-and-ndiff/"

I plan to implement monitoring with nagios and scripts this way.
I personnaly plan to identify first in my perimeter which files to monitor and why.. It's like waf or web app security, you can't connect a waf and leave, which is most of the time deficient ... You have to identify first with apps developpers which parameter HAS to be checked with what value.. and why :)
Like in OSSEC, i first have to identify assets, apps, files to monitor..

I maybe be able to check all the things i want ..one day...

Anyway, Richard (and Royce !), Thanks for the whois one !

Regards

Grymoire said...

Here's mine. Run this in cron, and it will send you mail when anything changes


#!/bin/sh
for i in domain1.com domain2.com
do
whois $i | grep -v -i database >/tmp/whois.$i
diff /var/log/whois.$i /tmp/whois.$i >/tmp/diff.$i
if [ -s /tmp/diff.$i ]
then
echo DIFF in $i:
cat /tmp/diff.$i
fi
cat /tmp/whois.$i >/var/log/whois.$i
done

Royce said...

I'd forgotten that some registrars include the global WHOIS database's last modification time in their output, which can cause frequent false positives:

>>> Last update of whois database: Sun, 23 May 2010 12:29:08 UTC <<<

In order to ignore this date change without ignoring other date changes, we have to make the script slightly more fragile (by depending on the registrar's output format). I'm not sure of a better way to handle this case; comments welcome.

Also, my previously posted script has a logic bug. It exits the script entirely after only creating one missing baseline file.

My version is quite a bit more verbose than Grymoire's, but there are a couple of deliberate reasons:

* I'm manually sending the email (rather than letting cron's output do so) because some OSes (Solaris) put no useful information in the subject line generated by cron.

* I'm not using /tmp in order to avoid race conditions.

* I'm automatically setting up the workspace and initial baseline files, making it work "out of the box" without manual setup steps.

* cron paths can vary by OS and local preference; also, multiple versions of a binary can be present on a given system. This makes hard-coding the paths to specific binaries attractive (though it does harm portability).

* I'm abstracting some customizable parameters (work directory, domain list) in a separate area to adhere to DRY (Don't Repeat Yourself) and to improve legibility.

Here's a new version, taking all of the above into account, with some other legibility improvements.

#!/bin/sh

# Description: check whois for domains of interest.

MYDOMAINS="example.com example.net"
MYEMAIL=royce@example.org
MYWORKDIR=/home/royce/check

DIFF=/usr/bin/diff
EGREP=/usr/bin/egrep
WHOIS=/usr/bin/whois

for domain in ${MYDOMAINS}; do

OLDFILE=${MYWORKDIR}/${domain}.whois.old.txt
NEWFILE=${MYWORKDIR}/${domain}.whois.new.txt

# If the working directory does not exist, create it or exit.
if [ ! -d ${MYWORKDIR} ]; then
mkdir ${MYWORKDIR} || (echo "Could not create ${MYWORKDIR}"; exit 1;)
fi

# If baseline files don't exist, create them.
if [ ! -f ${OLDFILE} ]; then
touch ${OLDFILE}
echo "Created baseline file for ${domain}."
continue
fi

# Fetch WHOIS record for the domain, ignoring 4-digit years.
# This is somewhat fragile, as WHOIS output formats can change.
${WHOIS} ${domain} \
| ${EGREP} -vi '.*whois.*[ \-]20[0-9][0-9][ ,-]' \
> ${NEWFILE}

# Check to see if the WHOIS record has changed.
#
# We manually email because some OSes (most notably Solaris) do not
# put unique information in subject lines created from cron.

${DIFF} -u \
${OLDFILE} \
${NEWFILE} >/dev/null

if [ $? != 0 ]; then
${DIFF} -u \
${OLDFILE} \
${NEWFILE} \
| mail -s "whois check: ${domain}" ${MYEMAIL}
fi

# Save the new WHOIS information as the baseline.
mv ${MYWORKDIR}/${domain}.whois.new.txt \
${MYWORKDIR}/${domain}.whois.old.txt

done

Grymoire said...

My version did not have the the problem with the database timestamps, BTW.

I don't see why there would be a race conditions unless you plan to run it more frequently than every 5 minutes, which I think is excessive. And if you care, just use whois.$$.$i and delete this file at the end.

And if you don't use cron to deliver mail, just do something like

#!/bin/sh
FILE=/tmp/whoisdiff.$$
trap "/bin/rm $FILE" 0 1 15
PATH=$PATH:/usr/local/bin
export PATH
WhoisDiff >$FILE
if [ -s $FILE ]
then
mail -s "WHOIS DIFF" email@example.com <$FILE
fi