Posted 09/02/08 at 09:56:21 PM by Mark Edward Soper

The search engine startup Cuil (pronouced "Cool") we first told you about in July isn't very "cool" in the way its indexing robot works with websites. TechCrunch reports that Cuil's Twiceler website crawler is bringing many websites to their knees.

What is Twiceler doing? Last year, posters on The Admin Zone forum on Twiceler pointed out that the crawler was creating many connections in a short amount of time, resulting in an de facto denial of service "attack" on sites being crawled. While Twiceler doesn't work the same way now, it's still behaving badly.

For example, the JazzyChad blog reported recently that Twiceler was indexing invalid addresses that would become 404 (file not found) errors when Cuil users tried to follow them. Joe Kirp's Popular Science and Technology blog reports  that:

The Twiceler bot is probably the most stupid crawler I've ever seen, it just downloads everything it can find and it seems that it just won't ever stop. If there's a page using dynamic input in a URL (a calendar for example) it will download the same page 100,000 and more times, simply by following all kinds of dynamic links it can find without using any kind of intelligent limitation.

By downloading thousands of pages per hour on each website it can cause an incredible traffic on a server, and dynamic scripts (written in Perl, Python or PHP for example) start causing an immense CPU load that may even take your entire server down (as reported by several webmasters). Twiceler is really harmful and can cost both money and downtime. A well written crawler such as Googlebot or Slurp (Yahoo) would never affect a website in such a malicious way.

How can you stop Twiceler from bringing your website to a crashing halt? To find out how, and to sound off on your Twiceler problems, follow the jump.

Read More

Posted 07/16/08 at 09:27:08 AM by Chris Moody

While AMD battles its stock price doldrums and feels the pinch of it’s acquisition of ATI, Intel posted record second quarter earnings of $9.5 billion, operating income of $2.3 billion, net income of $1.6 billion and earnings per share (EPS) of 28 cents.

"Intel had another strong quarter with revenue at the high end of expectations and earnings up substantially year over year," said Paul Otellini, Intel president and CEO. "As we enter the second half, demand remains strong for our microprocessor and chipset products in all segments and all parts of the globe."

This is great news for Intel, but serves to highlight AMD’s woes.

AMD’s disappointing Phenom launch and lackluster processor performance combined with Intel’s pressure on processor prices is a heavy rock around AMD’s neck. It’s important to note that AMD hasn’t been idle and has some pretty interesting things in stock. Not the least of which is the catching up with Nvidia in GPUs, but also their Spider platform, and next generation processor. There is no doubt the pressure is on. AMD needs to deliver a hit. They need it and we, the PC enthusiasts, need AMD. Without a serious competitor innovation can stagnate and prices are sure to rise.

Read More