18 September 2009

Maltego, Technorati, and Creative Commons Licensing Failure

I've been using Paterva's Maltego software quite a bit lately in my testing. This software is a fantastic tool, and provides a great way to obtain a great deal of information about an organization or individual. It comes in two flavors, a community edition which is free, and a commercial edition which is not. Because I am using this for my job, I have the commercial version of Maltego.

Like I said, Maltego is a fantastic tool, but there's one thing that bugs me about it; A number of the most interesting transforms that come with the product use the Technorati search engine to provide information about an entity (for those that don't know, Technorati is a search engine that pulls information from the blogosphere and various social networks).

The problem is that the Technorati search engine uses the Creative Commons license for its technology, and they chose to go with the one that disallows commercial use.

That's fine, it's their code, they can license it however they want. My problem isn't so much with them, as with Paterva for choosing to use their stuff in the Maltego product. Because I am hired by clients to perform this discovery, I am unable to use these transforms in Maltego (at least, as far as my understanding of the licensing goes) and so I have them disabled.

"So what?" you may ask. Well, what this means is that attackers using the free version of Maltego can get potentially useful information about a given company which a tester hired by the company, using the paid version of Maltego, can't legally provide. (I should mention here that I have tried looking into whether Technorati provides a way to license their technology for commercial use, and as far as I can tell, there is no way to do so.)

This strikes me as insane, and not a GoodThing(tm) at all.

23 March 2009

twitter badness?

so, a few days ago i was working on a project, and noticed that GoDaddy allows web sites which use their SSL certificates to post a flashie thing on their website allowing visitors to check the status of the cert. (see the bottom of tweepme.com for an example).

It turns out that GoDaddy actually has the blank certificate image stored on their servers, and that it is accessible via http in addition to https.

This means it could easily be used for spoofing by anyone that knows how to:
a) manipulate an image in an image editing software application or
b) manipulate an image in any number of programming languages

So, I decided to make the following tweet at twitter:

"interesting. if you know how to manipulate images, you too can spoof godaddy's SSL seal: http://is.gd/o1pM"

It was posted, and then disappeared about 15 minutes later.
I reposted it. Half an hour later, it was gone again.

So I talked to a friend of mine that follows me on twitter and had him pull up my page in his browser, and also in his third party application on a mobile device. I then posted again. He confirmed that it showed on my twitter profile page, but that it didn't hit his feed, nor his mobile device. About half an hour later, it disappeared again.

I then posted a tweet about the fact that my tweets were going missing for some reason. That also vanished about 20 minutes after posting.

So, I posted a tweet about something completely unrelated, that stayed.

At that point, I sent a request into twitter support asking whether I was triggering their ToS violation or such and that this was leading to my tweets vanishing. As yet (3 days later), it's not even been assigned to anyone to review.

Hmm... Interesting.

03 March 2009

network upgrade!

I've been wanting to upgrade to business class Road Runner for some time, but haven't had the chance to do so until now. There's a lot of reasons behind my wanting to do this, some of them are:

  • static IP addresses are only available via Road Runner's business class offering

  • while download speeds are the same or less than residential, upload speed is significantly higher (on paper)

  • the residential cable modem I have only works at 10M half duplex for the 'client' side interface, which means while i run a gig-e or 100M full network in my home, I'm throttled to that at my uplink.

So, I put the call in, got the quote, signed it, and sent it back.
2 days later and I have a shiny new modem. But even better, I have this:

Bandwidth Thoughput from MyHouse to Various Places

rochester, ny (http://rochester.speedtest.frontiernet.net/)
down - 7.013 Mbps
up - 1.415 Mbps

los angeles, california (http://lax.speedtest.dslextreme.com/speed.php)
down - 4.505 Mbps
up - 1.385 Mbps

san francisco, california (http://helpme.att.net/dsl/speedtest/)
down - 8.782 Mbps
up - 1.460 Mbps

dallas, texas (http://www.gospeedtest.com/index.html)
down - 4.075 Mbps
up - 0.854 Mbps

ASNCheck Script

While working on a project today I decided that it would be handy to have a script that could take an AS number (from stdin or from a list of them) and check the health status of it (via things like DNSBL for example), specifically gathering information that could lead one to determine the relative infection/compromise level.

Ideally, such a script would be able to alternatively take an IP address, determine the AS for it and then report on both the IP provided as well as the overall "health" of the AS associated with it.

Well, some of that I managed to whip out tonight, though not all.

I'll keep working on this, but I think it's useful enough now to warrant posting (I normally do *not* make code public in this raw a state, so take note that there are very likely bugs in this).

That said, here's 'asncheck.py'.
In its current state, it just returns a list of IP addresses from a given AS which are in the dShield current watchlist.

#! /usr/bin/env python
# ------------------------------------------------
# asncheck:
# retrieves the current dshield watchlist for
# a given AS, returning just the IP addresses.
# sample url:
# https://secure.dshield.org/asdetailsascii.html?as=123
# ------------------------------------------------
# written by:
# jason ross (algorythm@gmail.com)
# ------------------------------------------------
import sys

def main():
# here beginneth the script
opts = parmsdealer()

if (opts.verbose == 1):
print "\nRetrieving information for AS Number " + opts.asn + ":\n"

if (opts.infile):
filedata = open(opts.infile, 'rU')
except IOError:
print "unable to open input file \'" + opts.infile + "\'\n"
print "Unexpected error:", sys.exc_info()[0]
for line in filedata:
print line
asn = line.split(opts.delim, 3)[int(opts.col)]

if (opts.asn):
asn = opts.asn

dshield(asn, opts.verbose)

#print '{0}.{1}.{2}.{3}'.format(oct1.zfill(3),oct2.zfill(3),oct3.zfill(3),oct4.zfill(3))

def parmsdealer():
import sys
from optparse import OptionParser
version="\nasncheck: version 0.1\nauthor: jason ross \n"
usage="\n\n%prog [OPTIONS]\n"
parser = OptionParser(usage=usage, version=version)

# set up command line arguments

parser.add_option("-v", "--verbose", dest="verbose",
action="store_true", help="turn on/off verbosity (default: off)")
parser.add_option("-a", "--asn", dest="asn",
action="store", help="specify the AS to retrieve data for (just the number, or with 'AS' prepended)")
parser.add_option("-f", "--infile", dest="infile",
action="store", help="get the AS from the specified file (can be a list)")
parser.add_option("-c", "--col", dest="col",
action="store", help="[required with -f] specifies which column in an input file contains the AS (default is to use the first column: '0')")
parser.add_option("-d", "--delim", dest="delim",
action="store", help="[required with -f] specifies the delimiter to use when parsing the input file (default is to use the ASCII pipe character (0x7c): '|')")

# process command line arguments
(options, args) = parser.parse_args()

# exit if we're missing options
if (not options.asn and not options.infile):
print "\n" + sys.argv[0] + ": missing parameter(s)\n"
print "\n"

# exit if we've got conflicting options
if (options.asn and options.infile):
print "\n" + sys.argv[0] + ": can't set both an asn and an input file (there can be only one!)\n"
print "\n"

return options

def dshield(asn, verbose):
import socket
import urllib
import urllib2
import re

# urllib2 calls socket, so we can set the timeout here
timeout = 5

baseuri = 'https://secure.dshield.org/asdetailsascii.html'

params = {}
params['as'] = asn
encparams = urllib.urlencode(params)

requri = baseuri + '?' + encparams
req = urllib2.Request(requri)

if (verbose == 1):
print "opening " + requri + "\n"

res = urllib2.urlopen(req)
except urllib2.URLError, e:
if hasattr(e, "code"):
print "site borked! HTTP error: "
print e.code
elif hasattr(e, "reason"):
print "server borked! reason: "
print e.reason
data = res.readlines()
# print data
for line in data:
if ( re.match(r"[0-9]", line) ):
ip = line.split()
print ip[0]

if __name__ == "__main__":

21 January 2009

ARP Ping Using Scapy

here's a quick script i whipped up a while ago.
it uses scapy to perform an ARP ping of a network, and provides a CSV report of any MAC addresses it finds, along with the associated IP's.

It requires tcpdump to be installed and in the $PATH, as well as root privs to run.

#!/usr/bin/env python
# note that this script requires tcpdump to be installed
# additionally, it requires root privs to run.
# ----
# Portions of this code can be attributed to the book
# Python for Unix and Linux System Administration
# by Noah Gift and Jeremy M. Jones. 
# Copyright 2008 Noah Gift and Jeremy M. Jones
# ISBN-13: 978-0-596-51582-9
# ----

import sys
if len(sys.argv) != 2:
    print "Usage: pingarp \n  eg: pingarp"

from scapy import srp,Ether,ARP,conf

print r"MAC,IP"
for snd,rcv in ans:
    print rcv.sprintf(r"%Ether.src%,%ARP.psrc%")

here's sample output:
$ sudo ./pingarp

20 January 2009

crappy blogger templates

i'm going to have to consider either writing my own template, or moving to another blog technology.
every one of the default templates sucks for posting code, or any other long string of text for that matter. there's a lot more i could complain about, but that single fact at the moment is bugging me.

so, i've switched to a very basic template, and added the following bit of css to the code for now:

.post pre, .post code {
overflow: auto;

this works, but is less than ideal i think. not sure there's a good answer, but i'm definitely going to start looking into one. for now, sorry for the ugly layout, but hey, if it's that bad, RSS ftw! ;-)

Scapy Notes

Scapy is an "interactive packet manipulation program" written in python. It basically is a packet workshop framework which allows one to craft their own packets from scratch to match a variety of protocols, then send them on the wire and capture the results for analysis. Since it is written in python, it allows one to essentially create any number of tools, including scanners, fuzzers, DoS tools, etc. More info on it can be found at the scapy home page.

Basic Usage
When scapy is run from the command line, it loads the scapy modules and then drops you at the python shell prompt. This is useful for a number of reasons, but primary among them is that this means anything you can do in python, you can do in scapy as well. For the moment though, we're going to focus solely on the scapy specific modules.

Building a Packet
Scapy makes it extremely easy to build a packet, here's what it looks like:

First, we call scapy interactively:
[root@snsvc]# scapy
Welcome to Scapy (v1.1.1 / f88d99910220)

Next, we create the IP frame, then the TCP packet:
>>> a=IP()
>>> b=TCP()

Now we combine the two to create the TCP/IP datagram:
>>> c=a/b

We can use scapy's ls command to view the contents of the packet:
>>> ls(c)
version : BitField = 4 (4)
ihl : BitField = None (None)
tos : XByteField = 0 (0)
len : ShortField = None (None)
id : ShortField = 1 (1)
flags : FlagsField = 0 (0)
frag : BitField = 0 (0)
ttl : ByteField = 64 (64)
proto : ByteEnumField = 6 (0)
chksum : XShortField = None (None)
src : Emph = '' (None)
dst : Emph = '' ('')
options : IPoptionsField = '' ('')
sport : ShortEnumField = 20 (20)
dport : ShortEnumField = 80 (80)
seq : IntField = 0 (0)
ack : IntField = 0 (0)
dataofs : BitField = None (None)
reserved : BitField = 0 (0)
flags : FlagsField = 2 (2)
window : ShortField = 8192 (8192)
chksum : XShortField = None (None)
urgptr : ShortField = 0 (0)
options : TCPOptionsField = {} ({})

Changing Packet Details
Now, if we want to change any of the fields in the packet, we can do so by altering their values. For example, to change the IP destination to and set the TCP destination port to 443, we do the following:
>>> a.dst=''
>>> b.dport=443

Now we recreate the TCP/IP packet again, and view the changes using ls:
>>> c=a/b
>>> ls(c)
version : BitField = 4 (4)
ihl : BitField = None (None)
tos : XByteField = 0 (0)
len : ShortField = None (None)
id : ShortField = 1 (1)
flags : FlagsField = 0 (0)
frag : BitField = 0 (0)
ttl : ByteField = 64 (64)
proto : ByteEnumField = 6 (0)
chksum : XShortField = None (None)
src : Emph = '' (None)
dst : Emph = '' ('')
options : IPoptionsField = '' ('')
sport : ShortEnumField = 20 (20)
dport : ShortEnumField = 443 (80)
seq : IntField = 0 (0)
ack : IntField = 0 (0)
dataofs : BitField = None (None)
reserved : BitField = 0 (0)
flags : FlagsField = 2 (2)
window : ShortField = 8192 (8192)
chksum : XShortField = None (None)
urgptr : ShortField = 0 (0)
options : TCPOptionsField = {} ({})

Note that even though we didn't change the IP source, the value has changed. This is because scapy determined which interface would be used to send the packet to the destination we configured, and changed the source to that interface's address for us. We can override this if desired.

Sending the Packet
We use the sr() function to send the data across the wire. This function sends the packet, sniffs the response, and matches sent packets with the received responses. It works at layer 3, and will return the whole result of a probe.
>>> sr(c)
Begin emission:
...Finished to send 1 packets.
Received 4 packets, got 1 answers, remaining 0 packets
(, )

Viewing Results
We can view the results by assigning them to variables:
>>> res,unans=_
>>> res.nsummary()
0000 IP / TCP > S ==> IP / TCP > SA / Padding

Here we see we sent a SYN packet to port 443, and received a SYN/ACK packet back from the destination. We also see there was some Padding added to the SYN/ACK. We can view the information in the padding by accessing the results list directly:
>>> res[0][1]

Scripted Usage
Because scapy is written in python, it can be used from within any python script simply by using the import scapy statement.
For example, here's a simple script to perform a TCP SYN scan of ports 0-1024 on a given host (provided as a parameter to the script):
#!/usr/bin/env python
import sys
from scapy import sr,IP,TCP,conf
conf.verb = 0
dstip = sys.argv[1]

print "\nBeginning scan of "+dstip
res,unans = sr(IP(dst=dstip)/TCP(dport=[(0,1024)]),timeout=1)
if res:
print "\nReceived answers from the following ports:\n"
for s,r in res:
print r.sprintf("%TCP.sport%")
print "\nScan completed\n"

And here's the results of running this:
[root@snsvc]# ./scanner

Beginning scan of

Received answers from the following ports:


Scan completed

Install/Config Notes
"Error during evauluation of config file"
When running scapy from inside other python scripts, you may encounter the following error message:
ERROR: Error during evaluation of config file [None]
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/scapy.py", line 12183, in read_config_file

Not very helpful, but easy to fix. The problem is that scapy is looking for a config file which doesn't exist. The good news is that one just has to be present, no configuration is required. To fix this, simply do the following:
# touch ~/.scapy_startup.py

Using the Loopback Interface
The loopback interface is a special interface, in that packets going through it are not really assembled and dissassembled. The kernel routes the packet to its destination while it is still stored an internal structure.

In order to use the loopback interface, you need to send your packets using PF_INET/SOCK_RAW instead of PF_PACKET/SOCK_RAW. This can be done by changing the supersocket used by scapy, which is accessed via the configuration.

The default scapy values for sockets are as follows:
| Configuration Variable | Default Value |
| L2listen | L2ListenSocket |
| L2socket | L2Socket |
| L3socket | L3PacketSocket |

To use the loopback interface, change the L3socket setting to L3RawSocket.
This can be done using the following command (either via the scapy CLI or inside a script):

04 January 2009

captcha madness

i went to gmail today to login to an older email account i haven't checked in a while... apparently *too* long, because i got presented a captcha upon entering the username and password.

i was having a hard time reading the text (no surprises there, captcha's really suck as a technology), so for fun i decided to try clicking the "handicap" icon so i could listen to the captcha in audio format.

for some reason, it's never occurred to me that, just as visual captcha uses random crap in the image to try to prevent OCR from determining the letters, the audio version would contain a whole lot of noise in an effort to prevent text to speech from doing the same.

if you've ever wondered what an audible version of the mass confusion that is a modern captcha file might sound like: here you go.

all i can say is, it's a good thing i can see.