Referral Spam

One of the tricks spammers have employed to boost their clients search engine placements is to spam web site referral logs. They do this in the hopes that your logs will be processed and published on your site. If so, the search engine spiders will index your log stats, see all the links back to the spammers pages, and give them higher page rank. Or so goes their plan.

There are several ways to combat this.
 * On blog sites, use software designed to prevent this, such as Bad Behavior.
 * Do not to publish your log files or stats
 * Password protect your log stats.
 * set your robots.txt to prevent indexing of your web stats

The solutions above will prevent the spammers from gaining any page rank (you're keeping them from profiting) but that doesn't clean up the damage they've done.
 * 1) disk space is still wasted by all their hits in your logs.
 * 2) referrer logs are practically useless now because they are filled with p0rn, gambling, and drug advertisements.
 * 3) 404 reports are also useless.

Save us, Logmonster!
Logmonster can filter for common spam patterns in your log files and discard lines with any spam-like referrers. Now all that crap doesn't end up in the stat reports you see!

Example Report
*** this report is saved in /var/log/http/2007/02/15/HitsPerVhost.txt *** 666 lines to www.mail-toaster.org 31630 lines to www.tnpi.net 404 lines to www.nictool.com

Referer spammers hit you 2594 times and wasted 20.17 MB of your bandwidth. Here are the spam sending User Agents:

96     Mozilla/4.0 (compatible; MSIE 4.01; Windows NT) 90     Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; AIRF) 90     Mozilla/4.0 (compatible; MSIE 4.01; Windows NT Windows CE) 88     Mozilla/4.0 (compatible; MSIE 4.01; Windows 95) 86     Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; NetCaptor 6.5.0RC1) 84     Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) 82     Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 3.0) 75     Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; N_o_k_i_a) 52     Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; iRider 2.21.1108; FDM) 215    Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Crazy Browser 1.x.x) 199     Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322) 147    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Maxthon) 143    Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) 125    Mozilla/4.0 (compatible; MSIE 4.01; AOL 4.0; Windows 98) 122    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) 121    Mozilla/4.0 (compatible; MSIE 5.0; YANDEX) 120    Mozilla/4.0 (compatible; MSIE 5.0; Mac_PowerPC; AtHome021) 117    Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705) 115    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; iOpus-I-M) 114    Mozilla/4.0 (compatible; MSIE 5.0; Windows ME) Opera 5.11 [en] 107    Mozilla/4.0 (compatible; MSIE 4.01; Mac_PowerPC) 105    Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; KITV4.7 Wanadoo) 101    Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; SAFEXPLORER TL)

And here is what they polluted your log files with:

63      htp:/www.low-mortgage-rate-4u.info/# 51      htp:/www.carolina-mortgage-south-advisor.info/# 46      htp:/www.tomorgageonline.com/# 45      htp:/www.bad-credit-mortgage-loans-advisor.info/# 42      htp:/www.odsys.com/# 38      htp:/www.lender-mortgage-e-site.info/# 36      htp:/www.tomorgage.com/# 35      htp:/www.aonlinemortgageloan.com/# 34      htp:/www.aonlinemorgage.com/# 34      htp:/www.nonverbal-sales-seminars.com/# 34      htp:/www.brothersdigital.net/# 34      htp:/www.roylmarchwind.com/# 32      htp:/www.aonlinemorgages.com/# 32      htp:/www.ploughound.com/# 30      htp:/www.bonetypousada.com/# 30      htp:/www.usex.org/# 30      htp:/www.d-blast.com/# 29      htp:/www.e-z-ly-treat-e-d.com/# 28      htp:/www.porchettazzocchi.com/# 26      htp:/www.rejuva-skincare.com/# 26      htp:/www.ihomeequityline.com/# 26      htp:/www.4legsandatail.co.uk/# 26      htp:/www.hollywoodreporter.net/# 26      htp:/www.managedcareconsultants.com/# 25      htp:/www.aloansmortgage.com/# 24      htp:/www.tmabry.com/# 24      htp:/www.fathermychal.org/# 24      htp:/www.gkewq.us/# 24      htp:/www.germangermangerman.com/# 24      htp:/www.fpfmortgage.com/# 23      htp:/www.aloanmortgage.com/# 23      htp:/www.mortgage-seattle-advisor.info/# 23      htp:/www.ithomeloans.com/# 22      htp:/www.colinkeillorgraphics.com/# 22      htp:/www.amorgageonline.com/# 22      htp:/www.aonlinemortgageloans.com/# 22      htp:/www.bcwellcontrol.com/# 22      htp:/www.otrp.com/# 21      htp:/www.johnkinderracing.com/# 21      htp:/www.ecaddo.com/# 20      htp:/www.onusinter.com/# 20      htp:/www.lebanonraptors.org/# 20      htp:/www.virginiacoastreserve.org/# 20      htp:/www.calc-mortgage-advisor.info/# 20      htp:/www.ecgate.com/# 20      htp:/www.stadtkaffee.com/# 19      htp:/www.surgicalinstrumentspecialists.com/# 18      htp:/www.p1kh.com/# 18      htp:/www.tektonresources.com/# 18      htp:/www.amortgageloansonline.com/# 18      htp:/www.cathcomappalachia.org/# 18      htp:/www.goonlinemortgage.com/# 18      htp:/www.lotablogs.com/# 17      htp:/www.westerncatv.com/# 17      htp:/www.iloanmortgageonline.com/# 16      htp:/www.freddyfoley.com/# 16      htp:/www.jrcreations.com/# 16      htp:/www.itequityloan.com/# 16      htp:/www.step-consulting.com/# 16      htp:/www.place4mortgage.com/# 16      htp:/www.jlnw.com/# 16      htp:/www.aonlinemortgages.com/# 16      htp:/www.sewingmachinesusa.com/# 16      htp:/www.tohomeloanonline.com/# 16      htp:/www.capstonemortgage.info/# 16      htp:/www.insight-asiapacific.com/# 16      htp:/www.icusurf.com/# 16      htp:/www.projectx7.com/# 16      htp:/www.gailsdreamtime.com/# 15      htp:/www.itmortgageonline.com/# 15      htp:/www.goequityhomeloan.com/# 15      htp:/www.gohousemortgage.com/# 15      htp:/www.trust-mortgage-ebanking.info/# 15      htp:/www.aloanmortgageonline.com/# 14      htp:/www.thegreat.us/# 14      htp:/www.allianceofbaptist.org/# 14      htp:/www.aonlinehomemortgage.com/# 14      htp:/www.easilymortgage.com/# 14      htp:/www.gerardoknutson.com/# 14      htp:/www.tv-satellite-network.net/# 14      htp:/www.englishseminar.com/# 14      htp:/www.roanoke4kerry.org/# 14      htp:/www.l-anais.com/# 14      htp:/www.metersoft.com/# 14      htp:/www.amorgages.com/# 14      htp:/www.terryvarner.com/# 14      htp:/www.1ktvnw.com/# 14      htp:/www.tomortgageloan.com/# 14      htp:/www.amortgageloanonline.com/# 14      htp:/www.1ktvdc.com/# 14      htp:/www.isincnet.com/# 13      htp:/www.ahomeloansonline.com/# 13      htp:/www.ahomemortgageonline.com/# 13      htp:/www.aonlinehomemortgageloan.com/# 13      htp:/www.familyhomemortgage.net/# 13      htp:/www.aonlineloansmortgage.com/# 12      htp:/www.atlantawebmasters.org/# 12      htp:/www.dialamortgage.net/# 12      htp:/www.errolware.com/# 12      htp:/www.comic-fans.com/# 12      htp:/www.bg-competitiveness.org/# 12      htp:/www.lakesidemortgage.org/# 12      htp:/www.flww.com/# 12      htp:/www.blaoentertainment.com/# 12      htp:/www.businessathoughton.net/# 12      htp:/www.aloansmortgageonline.com/# 12      htp:/www.art-4u.net/# 12      htp:/www.ifsbo.net/# 11      htp:/www.mortgagefinancialplanning.com/# 11      htp:/www.tomortgageloans.com/# 11      htp:/www.aonlinemortgage.com/# 11      htp:/www.aonlinehomeloan.com/# 10      htp:/www.ivoryvaughan.com/# 10      htp:/www.aonlinehomeloans.com/# 10      htp:/www.ismortgages.com/# 10      htp:/www.webguidenewhamphire.com/# 10      htp:/www.epjlp1.org/# 10      htp:/www.ramandrom.com/# 10      htp:/www.cnzeefoo.com/# 10      htp:/www.blloydspecans.net/# 10      htp:/www.ithomeloan.com/# 10      htp:/www.blackhealthnetwork.org/# 10      htp:/www.authenticpolishrestaurant.com/# 9       htp:/www.cfeps.com/# 8       htp:/www.tohomemortgage.com/# 8       htp:/www.awi888.com/# 8       htp:/www.pfs-advisors.com/# 8       htp:/www.aonlineloanmortgage.com/# 8       htp:/www.calgaf.org/# 8       htp:/www.museumoffligt.org/# 8       htp:/www.nuteccf.com/# 8       htp:/www.linuxbunny.com/# 8       htp:/www.wannabeprogrammer.com/# 8       htp:/www.boxlibrary.net/# 8       htp:/www.ithouseloans.com/# 8       htp:/www.1ktvmw.com/# 6       htp:/www.wheatimprovement.org/# 6       htp:/www.indusscript.net/# 6       htp:/www.forexbrokerstrading.us/# 6       htp:/www.analogrevolution.com/# 6       htp:/www.tech-2004.us/# 6       htp:/www.magicofpetertoler.com/# 6       htp:/www.bakasquared.com/# 6       htp:/www.com-vax.com/# 6       htp:/www.advancedguard.org.uk/# 6       htp:/www.formorgage.com/# 6       htp:/www.long-distance-phone.biz/# 6       htp:/www.modularinfotech.com/# 6       htp:/www.idishtv.org/# 5       htp:/www.amortgagesonline.com/# 4       htp:/www.bucskinnerweb.com/# 4       htp:/www.b-fish.com/# 4       htp:/www.btselm.org/# 4       htp:/www.fibreconference.com/# 2       htp:/www.aspergersynrome.org/# 2       htp:/www.wisconsinrfu.org/# 2       htp:/www.passionne.net/# 2       htp:/www.rscservices.com/# 2       htp:/www.classicalformsbooks.com/#

NOTE: the http line has been purposefully munged before publishing here!