Number of pings (and splog URLs) dropping
Since we introduced our splog blocklist and Spam Huntress posted about it, both i) the amount of traffic we waste and ii) the number of pings we are receiving have dropped. At the moment we’re not quite sure why the second one has happened, but we think that either one of our XML-RPC ping sources (pingomatic.com being the biggest) has started filtering as well, or else one of the big blog hosting sites has started blocking splogs. Anyway, it’s all good news.
We now have 370 web server IP addresses in our blocklist. Yesterday we received 474,394 pings of which 161,094 were blocked immediately based on the web server being on the blocklist. Of the rest, 179,467 were picked up by our incremental “delay buffer” which means we won’t go out and index them for up to 8 days if we get another ping for the same URL.
Here’s a rough explanation of how our splog blocklist works. I already posted this text as a comment to a Spam Huntress post a while ago:
At A2B (see http://www.a2b.cc for more) we run a search engine with a ping interface. Bloggers who have geo-located META tags in their HTML (see http://www.a2b.cc/help-searching-addurl-blogping.a2b for more) can ping us and we’ll pick up (parse) their page and index it in our geosearch engine. We receive pings from many individual bloggers, a full ping feed from pingomatic.com, and bulk pings from several other sources, usually around 700,000 to 1 million pings per days. With approximately 200 IP addresses in the blocklist, about 37% of daily pings are blocked immediately.
To generate the list, we recorded the URL of each website (read blog) we were pinged with and also converted it into the IP address of the web server for each URL. We recorded the IP address and added to a counter every time we received a ping for a URL on the same web server. We soon noticed that we were getting many thousands of pings for the same IP addresses, so pulled a script together to list the top IP addresses by number of pings.
We built another script which showed all the URLs associated with each IP address. In order to decide which web server IP addresses are serving splogs, which we block, we open this script and manually have a look at a random sampling of the URLs - it’s usually pretty easy to tell if they’re splogs as they’re just full of advertising links or are quite random in their choice of subject matter. Any web server which has real blogs tends to stay off the blocklist (so that rules out blogspot.com even though people are using it for splogging).
As soon as we’d blocked the first 112 IP addresses, the amount of traffic we were using parsing blogs dropped from 27GB per day (it was so high that it was costing us money in hosting charges) to 6GB/day. Of course, it began to creep up again soon after, so we’re realising it’s an ongoing effort and are beginning to think about blocking whole ranges of IP addresses.
Hope that’s of interest to everyone.
[…] sted in General Splog News by Toivo Lainevool on the January 30th, 2006 A2B has been reporting success in identifying and block splogs based on IP addresses. They have significantly reduced the load on […]
“so we’re realising it’s an ongoing effort and are beginning to think about blocking whole ranges of IP addresses.”
Sounds like SPEWS to me!
Cheap 17…
Cheap 17…
Information about 4…
Cheap 4…
Read ftop…
Read ftop…
Cheap italy…
My italy…
Popular topnado…
Pipka topnado…
Site top…
Information top…
Information yoper…
Cheap yoper…
very best idea make rules time!…
…
anime girl on girl sex…
anime girl on girl sex …
Not bad, it really can occur
Thanks man, i agree
You have so much spam in here.. (
Not bad, it really can occur
Direct Real Estate Investment
https://estatedeposits.com/?ref=1447
Hello! Good Site! Thanks you! gvbjawckbiopyh
Hello! Good Site! Thanks you! krqjgdosddmrib
gay huge black dick…
gay huge black dick…
porn dojo blowjob…
porn dojo blowjob…
white on ebony fuck…
white on ebony fuck…
free video clips of women fucking men with dildos…
free video clips of women fucking men with dildos…
dumb blonds opening there pussy wide open…
dumb blonds opening there pussy wide open…
Life…
Information source about bloga2bcc….
This is exactly what I expected to find out after reading the title Number of pings (and splog URLs) dropping. Thanks for informative article
Hello, nice site look this:
Sample design…
End ^) See you
Hello, nice site look this:
And includes a wealth of information.
End ^) See you
I couldn’t understand some parts of this article Number of pings (and splog URLs) dropping, but I guess I just need to check some more resources regarding this, because it sounds interesting.
SpeedUp…
PcThrust is a tool for all that allows you to modify your
…
2008 cheltenham betting…
escape diarrhea Methuselahs:rejected,amortizes denotational,…
Page O…
The official website for Ukrainian music ensemble Taran.
…
The European credit cards: MasterCard, Visa (Gold/Platinum/Corparate/Bissnes/Signature), all cards workers, a guarantee, balance of cards from 2000 $, behind the detailed information and the prices write:admin.cc@mail.ru, or ICQ 423669709.
The credit card, credit cards, plastic cards, mastercard, visa, carding, cvc, damp, dump, bank cards, plastic cards, credit cards, a debit card, a bank card, a card visa, a card the visa, maestro, visa classic, visa electron, cc, cards, magnetic cards, number of credit cards, visa a credit card, the generator of credit cards, the Internet credit cards, plastic cards visa, cvv, cvv2, carder to receive a credit card, credit the Internet a card, the visa a credit card, a code of a credit card, to buy a credit card, sale of credit cards, bank credit cards, credit cards reception, a debit card, the master of cards, the master of cards, maestro card, a card maestro, magnetic cards, payment cards, dumps
kim kardashian sex video…
Information about research in the laboratory of Robert
…
Lingerie Modls…
Jasmine Byrne tended balanced. Lingerie modls once Arlington possibility — poem characters. …
big black cocks xxx…
the STEP Board commissioned papers on 11 industries ranging
…
big straight dicks…
Institute doing research on innovation, economic growth
…
ladyboy thumbs…
This online test consists of 50 multiple-choice questions.
…
naked…
featuring daily news updates, exclusive photo galleries
…
Thanks to Oprah, Obama camp claims biggest crowd yet
carmen electra naked…
Cast, crew and reviews from the Internet Movie Database.
…
subj
?
free homemade sex videos…
Ninemsn Dolly Homepage
…
naked pics…
Welcome to Dollymania: The most comprehensive online resource dedicated to Dolly Parton
…
proud family sex…
Michael Oher update - Big LT pleased with camp so far
…
sex with family members…
But what made Michael Oher especially peculiar was that no one in Memphis had
…
Young Asian Sex…
Hyperlinked encyclopedia entry provides a personal and political profile of the US Senator for
…
Hardcore Anime Porn…
Jon Tester is a third-generation Montana farmer who understands
…
match betting…
larva February:barge sexual eventuality …
Big White Tits…
All that you have long been looking for in our music portal
…
Old Naked Chicks…
Get fun and interesting zebra facts in an easy-to-read style from the San
…
Indian Big Tits…
Download a driver for your Zebra printer
…
Long Blowjob Videos…
Description, technical overview, screenshots, download
…
Bruno…
In number theory, a lucky number is a natural number in a set which is generated by
…
Maid Forced Sex…
Includes news, information, photo gallery
…
Forced Sex Sites…
The official web page for the band Depeche Mode
…
Ellie…
Pokemon Special Pickacu Edition cheats
…
Jinni…
months ago. Additional Details. 7 months ago
…
Vic…
Picchu Evolves Into Pickacu
…
Edytha…
Pokemon Special Pickacu Edition cheats
…
Josie…
traditional art, photography, poetry / prose. Art prints
…
Beastiality Forum…
wasn’t murdered and dismembered like the girl before her
…
Beastiality Mpegs…
Sebold was raped as a college freshman, but the police said she was
…
Free Farm Sex Videos…
Register with Lucky … CALIFORNIA OWNED & OPERATED
…
Gay Farm Sex…
Find live, breaking sports news, live scores and all sports results here on Channel 9 Wide
…
You can get the information regarding any u product at the click of the mouse. You can compare the products.
You can get reviews of k the product. The buy best online provides you with all the details that you want to know about the product.
The company decided p to have sale of the products. This started the sales event phenomenon.
ef6UbNwe45ErwOpCV2m
The best buy credit z card and best buy Master card are two important features of the best buy online services.
With the help of r these cards you can buy any product of the best buy company online. Then there is the best buy reward zone program.
In the offline shopping you n have very limited time and ultimately you have to take a best buy appliances buying decision in hurry.
First point q is there are different categories of the goods.
These categories y have further subcategories also. It is easy to find the product of your choice on the best buy website.
If you take complete a audio package then one company is offering free DVD players with 2 movies and some discount.
You can compare the features i of similar goods of other websites.
Thus best buy e online system helps you to get the reward certificates.
The ordinary people y do not understand the technological features of the goods.
The reward zone c program membership gives you many informative articles.
The best buy k website helps you immensely on both these counts.
Are the best buy y sales events confined to above best buy sales events only?
You can combine z all types of economic savings to a grand big savings total.
Computer and accessories n also offer a good deal in this best buy sales weekly event.
Thus the best buy online b website helps you tremendously to get all the info about the product that you want to buy.
You can print out g the list from the best buy website and take it to the store for further shopping.
The best buy w sales are a big industry in itself now.
Apart from this, there is s dedicated 24 hours help line. You can call the professional and get clarified your doubts.
The unsold goods or the k refurbished goods are sold at very cheap price in these best buy online outlets.
Whether locating nearest best buy a store or applying for best buy scholarship is concerned the best buy online website gives you all the info.
The best buy web v site gives you all the info about deals, promotions, discounts etc.
no matter how terrible people act, we don’t have a right to judge them. We are just to express our minds, and don’t have to pay attention what other people look like, what’s their real face. It doesn’t matter for me, if someone will say anything about me, I know what I really deserve, and what kind of person I really am.
Homemade Porn Clip…
Its interesting, THX…
Anal Toys…
Welcome to Some Time Ago, the place to find (almost) everything about
…
teddy…
Originally coined by the Dopyera brothers when they formed the Dobro Manufacturing Company, in time it came, in common language, to mean any resonator
…
Здравствуйте господа.
Есть предложение по кремнию.
Продаем 2000 кг скрапа кремния низкоомного от 0,001 до 0,2-0,3 Ом*см, тип проводимости Р. Цена 125 долларов за 1 кг.
Продаем скрап кремния около 1500-20000кг сопротивление от 0,5 и выше, тип Р. Цена 220 долларов по предоплате, и по факту поставки 350 долларов за 1 кг.
Пластины кремниевые сопротивление от 40 Ом*см, объем около 2000 кг. Цена 350 долларов за 1 кг.
Если предложение заинтересовало высылайте заявки на quarts2008@mail.ru
[mlm blog]…
Although the world is full of suffering, it is full also of the overcoming of it… [ ǂ]…
Ciekawa strona, trafilem tu przypadkowo, ale od dzis bede wpadal czesciej, pozdro
review cheep medical for groups ins prices www ageants fargo nd…
traces extinct Hague …
:
- , !