FeedDigest and RapidFeeds

It seems my friends at FeedDigest are having some scalability issues - most likely due to their popularity. To catch up on what FeedDigest is, read my interview with its creator, Peter Cooper.

I’m using cURL to pull in my digests because DreamHost does not allow PHP includes (yes, I’m still with DreamHost, they moved me to a new server and I haven’t had any problems since then - thanks guys!). With cURL, I have the ability to set a timeout period to pull in my digest, so that a webpage does not timeout if for some reason the link for my digest is slow. I currently have set that to 2 seconds. If within 2 seconds the digest cannot be pulled in, that content will go missing. Over the last couple of months, the content has been missing - often.

I must admit that I don’t really have a right to complain. I’m not paying for the service. There are three paying options - but they simply offer the ability to have more digests and sources. And I don’t need more than the five digests that come with the free account.

No one else is really in the FeedDigest space. There aren’t too many options to “digest” feeds. Stowe Boyd pointed to Peter’s blog at one point, highlighting a point he made on this issue:

You might think that people using FeedDigest because other services suck is a good thing for FeedDigest. I’m not so sure.

Take what Stowe Boyd just wrote. He’s using a new to-do list system which he loves to bits, but which lacks “a neato-keeno javascript to let me directly post a public to-do list on my blog, so if I want to do that I have to resort to an RSS-to-javascript gasket like Feeddigest.” FeedDigest is a ‘resort’. This is true in many cases. A lot of users only use FeedDigest because their existing tools are lacking.

The COO of RapidFeeds, Mustafa Khundmiri, recently contacted me to take a look at their service. One of their offerings is to re-syndicate and display RSS or ATOM feeds on websites. It’s interesting but has much work to be done, especially when compared to FeedDigest.

These issues and the more serious one of long term viability, soon to not be experienced by the expiring SearchFox, really boil down to business models. We expect these services for free yet we expect them to be robust. We expect them to always be there and yet when they’re not, whether temporarily or for good, we are displeased. Something or someone has gotta give.

tagged under ,

2 Comments On This Post

  1. Peter Cooper says:
    January 13, 2006 at 8:17 am

    Hi Ken,

    I imagine you experienced some “burps” in the service yesterday :) You’ll be happy to know, however, that they were worth it.

    We’re moving things into place for a major relaunch (thanks to our funding) of the whole system, and we’ve finally nailed a large part of the scalability problem. Our blog covers the details. Basically, we can now crawl feeds several times faster than before, and your digests will update instantly once we’ve crawled the feeds that make up each digest. Previously there was a lot of luck and weird timing involved.

    Luckily nearly all of our downtime has related to making upgrades, rather than the service mysteriously going down or falling over. This isn’t really a good excuse, of course!

    I don’t really want to give anything away due to the competition, but you’ll be seeing some great things some time in the next few weeks.. :)

    Thanks for your continued support!

    Regards,
    Peter Cooper
    FeedDigest.com

  2. Ken Yarmosh says:
    January 13, 2006 at 8:22 am

    Peter…thanks for stopping by to clarify. I figured the issues were not due to incompetence but rather to popularity. Glad to know that FeedDigest is continuing to improve - I expect a great year for you guys!

Voice your opinion