You'll probably have noticed that there was some trouble with the Salon over the last weekend. Most posts didn't go through, though some snuck by. I've been playing with it all week trying to get things working, but I'm hindered by my own ignorance and the fact that Yahoo Pipes is slow and has apparently been having problems itself.
At any rate, I wound up reorganizing the data that gets fed into the Pipe. Instead of taking the RSS feeds and tags from a single Google spreadsheet, it's now taking that information from a bunch of different spreadsheets, aggregating it in a handful of Pipes, and then aggregating outputs from those Pipes in a single Pipe. My idea was that maybe it would be better able to handle the data in smaller doses. Again, I have no idea what I'm doing. I've just been experimenting. But at present, at least, it seems to be working. You should see feeds from last week starting to come through your RSS readers.
I've also been trying to think of an alternative to aggregating the feeds in this way, but so far I've not been able to come up with a solution. There are other aggregators out there, using which you can mash different feeds together to create a single feed, but none seem as powerful as Pipes, and none seem able to add a tag to the post title (which I find very helpful when looking through Salon posts). We're dealing with almost 500 different feeds here, which is a lot of information. Yahoo Pipes, if it worked reliably and was faster, would really be the ideal solution.
So all of this is making me think that, unless an alternative solution presents itself, maybe the growth of the Salon should be halted at 500 members. (We're currently at 480.) Thoughts? Ideas?