• geek

    Interesting new services…

    by  • 11/6/2003 • geek • 0 Comments

    Okay, I know I’ve seen craigslist before, but I don’t think I’ve given it the patience it truly deserves. Re-acquainted with it thanks to feedroll which is powering that little box in the sidebar… nice bit of feed display from other sites… I’d like to get my hands on a really good RSS feed aggregator […]

    Read more →

    Being cheap….

    by  • 11/3/2003 • geek • 2 Comments

    Just realized that I have a lot of content on the paid-for web server at Warped and I don’t really feel like paying for the 100MB of extra disk space any longer. So, I moved a whole bunch of my lesser-used items and bigger files over to my home server and enabled Redirect on a […]

    Read more →

    MT comment trapping

    by  • 10/29/2003 • geek • 0 Comments

    Part 1
    First, you need to stop using common cgi names for submission scripts. Spammers love these. For example, if you actually have a cgi named guestbook or formmail on your system, you are just begging for trouble. You need to rename them and change your html accordingly. Any other non-standard name will do. Along this line, you need to rename your mt-comments.cgi

    Spammers just troll for them, they might not even be looking at your html. Fortunately MT makes this easy to do in the MT.CFG – here’s an example:

    CommentScript feedback.cgi

    TrackbackScript backtrack.cgi

    and of course go rename the files in your cgi-bin or wherever you keep them. You don’t need to edit your templates at all unless you hard-coded them for some reason.

    We are going to be using the old common names to trap the stupid spam bots, so renaming these is a must.

    Part 2 – install the trap.cgi

    Next we need to install a cgi that is going to automagically create the bans in your .htaccess file.

    If you are not on an apache server, or for some weird reason cannot create .htaccess files, stop here – you cannot use this technique and will have to use some other (less automatic) spam prevention technique.

    Note that I did not invent this technique, I’ve only added some personal modifications to the script and evolved it over time. It’s been in the public domain for a few years now on a few webmaster support sites.

    Save it as trap.cgi EDIT IT to change your sendmail path at the top if necessary, and your email address for notifications at the bottom. I recommend you upload this to your /cgi-bin/ under a folder called “trap” (/cgi-bin/trap/trap.cgi) so you don’t accidentally click and run it yourself some day (its happened )

    don’t forget to set CHMOD 755

    #!/usr/bin/perl

    $mailprog = ‘/usr/sbin/sendmail -i -t’;

    $basedir = $ENV{DOCUMENT_ROOT};
    $htafile = “/\.htaccess”;
    $htapath = “$basedir”.”$htafile”;

    $date = scalar localtime(time);
    $remote_agent = $ENV{‘HTTP_USER_AGENT’};
    $remote_addr = $ENV{‘REMOTE_ADDR’};
    use Socket;
    $iaddr = inet_aton(“$remote_addr”);
    $remote_host = gethostbyaddr($iaddr, AF_INET);
    $remote_addr =~ s/\./\./gi;

    # Open .htaccess file in r/w append mode, lock it, go to top, read current contents into array.
    open(HTACCESS,”+>>$htapath”) || die $!;
    flock(HTACCESS,2);
    seek(HTACCESS,0,0);
    @contents = <HTACCESS>;

    # Empty .htaccess file, then write new IP ban line and previous contents to it, close to release lock
    truncate(HTACCESS,0);
    print HTACCESS (“SetEnvIf Remote_Addr \^$remote_addr\$ getout \n\# $date $remote_agent\n”);
    print HTACCESS (@contents);
    close(HTACCESS);

    # output an error message to this bad visitor
    print “Content-type: text/html\n\n500 Fatal Error – Access Denied</body></html>\n”;

    # send an e-mail message alerting sysop
    open (MAIL, “|$mailprog”);
    print MAIL “To: alert\@YOUR-SITE-NAME\.com\n”;
    print MAIL “From: alert\@YOUR-SITE-NAME\.com\n”;
    print MAIL “Subject: a bot has attemped to scan your site\n”;
    print MAIL “\n an improper scan has caused a ban on your site\n\n”;
    print MAIL “date: $date \n”;
    print MAIL “ip: $ENV{‘REMOTE_ADDR’} \n”;
    print MAIL “host: $remote_host \n”;
    print MAIL “agent: $remote_agent \n\n”;
    close (MAIL);

    exit;

    Part 3 – making the .htaccess (addition)

    This tiny snippet of script needs to go into your .htaccess file in the topmost folder (directory) that has web access (ie. /public_html/)

    If you don’t already have a “.htaccess” file, just make a plain text file, stick this in it and upload away.

    # traps bad spiders/bots/offline browsers

    SetEnvIf Request_URI “^(/robots\.txt)$” allowsome

    <Files *>

    order deny,allow

    deny from env=getout

    allow from env=allowsome

    </Files>

    Note that you do NOT need any fancy extra apache features like mod_rewrite for the above to function properly. 99.9% of folks on shared apache hosting should be able to use it.

    Believe it or not, once this in in place, you are about 90% done with the trap. All that is left is to make all the rules and situations that will cause the trap to activate.

    One easy way to start seeing the trap work, is to take that trap.cgi you made in part 2 and make some fake cgi’s for the spammers to fall into. Copy it to formmail.cgi and guestbook.cgi and upload it into your /cgi-bin/ folder. I’ll show you many more banning methods in part 4 (and 5).

    Wanna prove the trap is working? Well you can ban yourself. Make absolutely sure you have FTP access to your site, because you will need to re-upload the .htaccess file to gain access back (or rename it or delete it on your site).

    Just go to http://www.your-site-name.com/cgi-bin/guestbook.cgi and WHAM you should get a forbidden message. Then try browing your site, it’s impossible. Shortly you should recieve an email that you have been banned with your ip, etc. Go download your newly updated .htaccess file off your site and look what it did. It adds a line to the top of the file that blocks out your IP automatically.

    If you have properly renamed your mt-comments and mt-tb scripts, you can now upload trap.cgi and rename it in their place, and WHAM if someone trolls for them without using your website interface, they are banned (until you chose to remove them from htaccess, which I clean out like every few months if I feel like it).

    Part 4 – enhancing the ban ability (detecting nasty visitors)

    It might not occur to you, but this technique is also great to keep your bandwidth use down from illegitimate visitors. Alot of bad bots will suck down your entire site, images and all before you even know it (if you ever know it!).

    Many folks end up buying large blocks of bandwidth from their host or paying overuse charges when they don’t have to. We welcome well-behaved and friendly bots (like Google) but why should we let dozens of bots, including bad-hackers, people trying to learn how to make their own bot, etc abuse your site and cost you money?

    Here’s an easy way to stop bad bots in their tracks.
    Hopefully you had heard of ROBOTS.TXT which is used to suggest to bots (robots) where it okay it is or not to go on your site. For example here is a mini-version of mine:

    User-agent: *

    Disallow: /cgi-bin/

    Disallow: /images/

    Disallow: /photos/

    If you want more advanced “rules” just Google for the hundreds of robots.txt guides out there. Keep in mind, this is just offered to the bots, they do NOT have to check for the file, read it, or obey it.

    But the beauty of it is, all well behaved bots like Google, check for and obey robots.txt. You’ll see the requests in your server logs if you look for it. Some bad-hackers purposely check robots.txt for files you do not want them to go into, and then go directly there. So we have another instant way to trap bad behavior, ADD THIS to ROBOTS.TXT:

    Disallow: /dummy/dummy.htm

    You can actually use any filename and path you want, just make sure it doesn’t exist on your website.

    Then go back into your .HTACCESS file and add this line:

    Redirect dummy.htm http://www.YOUR-SITE-NAME.com/cgi-bin/trap/trap.cgi

    (super technical note: using mod_rewrite is better than using the above redirect because the visiting bot doesn’t have to follow the redirect request that is returned, but mod_rewite will force the action)

    The original text was placed in the movabletype forums by -lc-

    Read more →

    Mac OS 10.3 – Panther

    by  • 10/29/2003 • geek • 0 Comments

    Just installed Panther on the office computer at work, and there are a slick handful of navigation features there that I am just starting to get the hang of. Many of them are sure to be incorporated into X window managers shortly… Some tips I need to test when I get into work at the […]

    Read more →

    Damn I work with some freaks….

    by  • 10/28/2003 • geek • 2 Comments

    Overheard while I was on a call and didn’t have the opportunity to mock the person stating these facts…. I don’t care what anyone says, all girls want it more than guys… Someone else told him he was an idiot and generalizing about people based off his one relationship. Of course, pig-boy responds that he’s […]

    Read more →

    MT spam killer time…

    by  • 10/28/2003 • geek • 2 Comments

    Okay, I was getting sick of finding viagra ads on journal entries here that are completely unrelated to anything. Honestly, how random are these spambots? So anyhow, I installed MT-Blacklist into my install of movabletypehere on foo. Now to actually test it… watch me enter viagra ads in the comments of this article….

    Read more →

    Average Nudes

    by  • 10/24/2003 • geek • 1 Comment

    Average Nudes. The photographs in this suite are the result of mean averaging every Playboy centerfold foldout for the four decades beginning Jan. 1960 through Dec. 1999. This tracks, en masse, the evolution of this form of portraiture.

    Read more →

    Mentos Commercial….

    by  • 10/24/2003 • geek • 34 Comments

    Okay, I was on the Mentos site getting the wallpaper of the sheep. I’m a dork. Whatever. Thought I’d make another swipe at find that song they use in the cool commercial with the evil sheep. I think that I finally found the damn song. Called “D.P.” and apparently by a band named Clinic to […]

    Read more →

    Damn people are dumb…

    by  • 10/22/2003 • geek • 1 Comment

    I had a 15 minute discussion tonight with someone who needs a serious beating. He’d checked into a hotel and purchased a 1 megabit connection to the internet. That’s working fine, and he was able to surf for porn or 14 year old girls in chat rooms or whatever it is the desperately stupid do […]

    Read more →

    The great Firebird is God.

    by  • 10/16/2003 • geek • 0 Comments

    Okay, downloaded the newest Firebird browser today. Installed it at home on the Linux laptop and at work on the Windows 2000 machine. That was nice enough, but then I went and grabbed a couple extensions for it…. One of those extensions alone is worth gold. It’s called “Linky” and allows me to do incredible […]

    Read more →