MapReduce related patterns

Here’s a list of possible MapReduce related patterns I’ve been thinking about:

  • map-update: update each mapped document and emit its updated or original version;
  • map-delete: delete each mapped document;
  • map-reduce-map: map results of a map-reduce;
  • map-map: map results of a map;
  • map-recurse: apply recursion to the mapping function until a stop condition occurs;
  • any combination of patterns, e.g., map-reduce-map-update.

Example usage:

  • get the e-mail address of every customer with a negative balance: map-reduce-map. First, map-reduce to get the aggregate balance for each customer, then map again to get only customers with negative value and emit their e-mail address;
  • delete all documents older than one month: map-delete. First, map to get all documents older than one month and then delete each one;
  • get a list of documents and mark them as read: map-update. First, map to get the list of documents according to a given criteria, then update each document marking it as read;
  • and so on…

Posted in Software, Thoughts | Tagged , , , , , , , | Comments Off

node-fs npm package

node-fs is now available as an npm package. Just run npm install node-fs, require(‘node-fs’) and have fun.

Posted in Software | Tagged , , , | Comments Off

Blog posts firehoses

Three more firehoses to get all blog posts from the following platforms:

  1. blogger posts, through their changes.xml;
  2. tumblr posts, through superfeedr‘s track feature;
  3. WordPress.com posts, through their firehose feature.

While the first feed is free of charge, the other two have an attached price tag.

Also worth investigating is Paul Kinlan’s faux firehose for blogger.

Posted in Software | Tagged , , , , , | Comments Off

Drinking from the firehose

Some firehoses to drink read from, other than twitter’s:

  1. Google Buzz, through their activity firehose API methods;
  2. FriendFeed, using their real-time updates methods.

While the first works via pubsubhubhub, the second uses a combination of long polling and a cursor that helps you make subsequent calls.

Posted in Software | Tagged , , , , , , | Comments Off

Are we inside a bubble?

Very interesting post by Brad Feld about technology cycles.

If you recognize that we are in a strong, positive, upward segment of the current “tech company creation cycle”, that’s more than enough. You should accept that we’ll be back in a downward part of the cycle at some point, but that we don’t know if it’ll be in a week, month, year, or decade. — I Encourage Entrepreneurs To Ignore The Word “Bubble”

Posted in Thoughts | Tagged , , , , , | Comments Off