Bash + For Loop + File Input = HOTNESS!




I’ve got a particular need for a client’s sales purposes to do reverse lookups and actually tell us what the host name is of each visitor to their website.  Think of it as a specialized web report.  Sure, they use Google Analytics but that only tells you what country people are from, or city or even other details like operating system, is it a mobile phone (?),  etc, etc.  I guess that can all be useful but there’s MORE useful information found in a reverse DNS lookup from a sales perspective.  Those other details might tell you what alternative languages to make your website in if you’re getting hits from other countries or what browser to develop your site for (IE, FF, Chrome, Safari, etc or even mobile browsers, etc).

Note that I did this in Windows using nslookup but I needed to do it on Linux, I chose to use a bash script because the web server is a Linux web server (and rightly so!).

So why rDNS lookups?  Now the investigation begins ;)

For example:

If you have an IP address in a log, someone from said IP address visited your site but who?  You can run this command on your server (and I use my own server and IP address to do this):

nslookup -x 67.219.197.229

That’s a server I host websites on for clients.

;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44744
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;229.197.219.67.in-addr.arpa.   IN      PTR

;; ANSWER SECTION:
229.197.219.67.in-addr.arpa. 43200 IN   PTR     bigboy.flophost.com.

;; AUTHORITY SECTION:
197.219.67.in-addr.arpa. 43200  IN      NS      ns1.ctstelecom.com.
197.219.67.in-addr.arpa. 43200  IN      NS      ns2.ctstelecom.com.

;; ADDITIONAL SECTION:
ns1.ctstelecom.com.     2435    IN      A       64.136.224.18
ns2.ctstelecom.com.     2435    IN      A       64.136.224.19

;; Query time: 78 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Jun 23 13:42:56 2012
;; MSG SIZE  rcvd: 157

Notice you get the hostname: bigboy.flophost.com, you also get the name servers and their IP addresses.  So from a sales perspective if said customer from flophost.com visited my website a sales person could then determine if that person is a potential customer, if not it’s dropped and maybe they found us accidentally but maybe just maybe they are a possible customer looking at your widgets for sale.  You might know this because I’d then go to their website.  I’d see what they did as a business and if it’s related to the widgets or services I offer I might follow up with a phone call or an email.

Here’s my code ;)

#!/bin/sh
for ip in /var/ftp/ips
do cat $ip | nslookup -x > result.txt
done

The first statement: for ip in /var/ftp/is

For my testing I copied the log file into /var/ftp, it’s kind of a “working directory” for me.  The word “ip” is a variable, it’s arbitrary and can be anything.  I could have said:  for cocopuffs in /var/ftp/ips.  the “ips” is the file name, if it were “ips.txt” I would have put that in there, that’s the file we’re reading line by line.

I then do a “cat $ip”, the cat command is what’s reading the file and the “$ip” is the variable in the file that’s assigned on the previous line as “ip”.  We’re saying, for every line of the ips file assign each line to the variable $ip because each line is different (probably), we then pipe (|) the command we want “nslookup -x” and redirect the results to a result.txt file.  We can do a “>” to overwrite the file every time or a “>>” to continue appending to that file.

Once you have your file you can email it out, ftp it somewhere, make it available on their site for download (since it’s not really sensitive information…probably not since all IP’s, records, etc are “public record”).  You could so other interesting things too like parsing the file and if it returns a subset of domain name extensions to put those somewhere differently.  If for example you wanted all the Argentinian web visitors in the log to go to a separate file you can do that too.  I don’t have a need for that at the moment so I don’t have that syntax to post…I’d have to figure it out ;)

Figuring this out probably took me HOURS of thinking, maybe an actual hour or two of DOING (testing different syntax’s, trying different code, looking online for help…no online help really though just stuff that pointed me in the right direction).

Let me know if you found this code snipped useful.  I know others are looking for the syntax but finding it actually posted online is a another ballgame!  Enjoy!