Scuttle Me Bookmarks

Apologies for the poor attempt at Pirate-talk, I was just excited to see that Phil Wilson mentioned me in his post about Scuttle.

My post Scuttle rides again! talks about how I was pulling data from Delicious and posting into my install of Scuttle. Phil goes the other way around and posts to his own installation of Scuttle, which then auto posts that to Delicious.

This is the awesome stuff that smart people can do with open APIs…

Keep hacking!


Tweet Nest: Archive Your Tweets

I’ve had a renewed interest in my data lately, and that’s manifested itself in making sure I’ve got copies of my data. Those bits and pieces we so easily create on other web site? I want them. I want them “here”, where here is under my own control, on my own site, my own server, not just living in the cloud somewhere, at the whim of some 3rd party.

Twitter - Since Sept 2006 See, I’ve been using Twitter for a long time… since September 2006 supposedly. I’d love to go back and see what my first dozen tweets look like, but Twitter doesn’t allow that. I’ve been kicking myself for not saving all of that data since the beginning, but with micro-content you tend to think that a. It’s tiny, so it doesn’t matter, and b. It’ll always be there. We’ve learned (time and time again) that this isn’t the case.

About 2 years ago I was doing some Drupal work and set up an aggregator to ingest my Twitter feed, which managed to back up a large portion of my tweets, but not all of them. It was more experimenting with Drupal than trying to create a good backup. Still, I was slightly happier knowing I had some sort of archive. I still wanted something that would display my own content (tweets) on my own site.

I then found Tweet Nest. Developed by Andy Graulund (@graulund) it’s a simple open source PHP/MySQL application that does just what I wanted. It grabs your tweets from Twitter using the API, and stores and displays them on your own site. Perfect!

Tweet Nest : @raster The install was pretty painless, and I actually spent more time customizing the CSS (and I’m still not happy with what I came up with.) Of course two days after I installed it, it appeared to stop working. But alas, no worries, it was just a “rate limit freakout” with the Twitter API. After that, things have been smooth.

Tweet Nest also did a great job of grabbing all of my tweets as far back as October 2009. Not quite back to September 2006, but I’m starting to think that will never happen. But, I did manage to get tweets as far back as March 2008, thanks to another service called Backupify. Now I’ve got to find a good way to load the data into Tweet Nest, and I’ll have most (but not all) of my tweets stored on my own site.

You can find my archived tweets at, and just like my recent Delicious/Scuttle/bookmarks exploration, I now feel that much better about my data. (But I’m not stopping there. The next project may take quite a bit more hacking on my part, but it’s another one that’s long overdue.)

So go check out Tweet Nest if you want an application on your own server, or Backupify if you just want a nice, secure backup.


Scuttle rides again!

Because I always fear the CloudFail™, and because I like hacking on open source projects and interesting APIs, and because I like to store my own data… I’ve got Scuttle up and running again.

p2url: Bookmarks I started using in 2003 to store my bookmarks. I loved it. Over time had some issues with reliability and there were times when I could not access my bookmarks that were stored in This made me sad. Around 2005 an open source project named Scuttle appeared, which was basically a clone of you could run on your own server. I ran it on my own server, and put together some code to make it sync up with (At some point the name changed from “” to “Delicious” but I am calling it here for historical reasons.)

For years this ran on my own home server, and served as a great backup for my bookmarks in This was all very easy because Scuttle implemented (most of) the API, so syncing the data between the two was so trivial, even a hacker like me could cobble a bit of code together to make it work.

As an interesting sidenote, while ended up being pretty darn reliable (Yahoo! eventually acquired it) others were not so lucky. Ma.gnolia was another social bookmarking site. Ma.gnolia died a tragic death, and lost all data. See for details. This is why having an API is important. This is why having a backup is important. This is why you should trust yourself with your own data.

I’ve collected a few links for the blog posts I wrote about Scuttle,, and related sites.

(Most of the bits above can also be found on the page at:, and while you are welcome to browse through as it is a public site, I’m the only on who will be adding bookmarks to it, because it’s there for me. If you want your own, I encourage you to go make it happen. Let me know if you need help.


delisync for Scuttle

So much for “release early” eh? I finally, after a few requests, got around to putting up the code I used to sync my bookmarks with Scuttle. (See: Why Scuttle is neat.)

Go ahead and grab delisync and please make it better, because while it sort-of works, it also sort-of sucks… And I’d like a better version.

This code assumes you have a local Scuttle install, if not, change the code. (You should be able to point it at or any Scuttle install where you have an account.) Also, you will have to hardcode your username/password’s for and Scuttle…

All of this magic is really possible because Scuttle supports (most of) the API, so in theory you could do things in reverse, or upside down, or 3 ways from Tuesday, whatever that means. If only all of the “social bookmark” sites had API’s… (At least one other has told me they are “working on it” or something like that…)

Let me know how it goes. I’m not completely against improving it, but I think others may be better suited than me to do so… It’s GPL‘d so have at it.

(See Also: Scuttle at Sourceforge.)


Why Scuttle is neat

Remember Of course you do! You probably use it daily… Don’t we all?

Heck, long ago I even proclaimed that is neat. gosh, I even wrote some nasty code named, and some people even used it. was a hack, and it’s got some problems, but with many things, it’s just a proof of concept to me. (Note: It’s a massive memory hog since it builds a lot of large hashes in memory while running.)

So what does this all have to do with Scuttle? See, Scuttle is neat. It’s bascially a GPL‘d clone written in PHP using MySQL. There is also, which uses Rubric, which is written in Perl. Now, I’m a perl guy and installing Rubric was a royal PITA for me, so I gave up on it quickly. Scuttle on the other hand, was dead simple. (Grab the 0.5.0 version from cvs though.)

You could use either the or Scuttle web sites, where you get to leverage the social aspect of these apps, but are at the mercy of things you don’t control, or you could install Scuttle on your own server and control it, but lose the social aspect… What do to? Behold! Scuttle supports the API! This opens up a world of possibilities…

  • Use as your primary source, and Scuttle (the site or your own install) as a backup.
  • Use Scuttle as your primary source, and write code to sync everything to (except the entries marked as ‘private’ which Scuttle supports but does not.
  • Install Scuttle inside the firewall for others in your workgroup to use.
  • Hack up Scuttle to do crazy things, slap it on a server and try to build a Web 2.0 business around it.

Ok, that last one is (sort of ) a joke, but the point is, for most of the complaints I’ve heard about, you can probably solve them with some combination of all of these things. servers down, or too slow, or doesn’t support private bookmarks? Scuttle not social enough, not enough users, lacking features? Fix, fix, fix…. (Need more ideas? See How to Make Delicious Not Suck.)

Code? Yes, I have some code… Right now my code pulls the entries from my account and syncs them to my Scuttle account, as well as a local install of Scuttle. I should get around to cleaning it up and releasing it, I just want to run it for a bit first to see if it behaves well…

Open-source, and just as important, open API‘s make it all possible. It’s exciting stuff!