RasterWeb!
Apologies for the poor attempt at Pirate-talk, I was just excited to see that Phil Wilson mentioned me in his post about Scuttle.
My post Scuttle rides again! talks about how I was pulling data from Delicious and posting into my install of Scuttle. Phil goes the other way around and posts to his own installation of Scuttle, which then auto posts that to Delicious.
This is the awesome stuff that smart people can do with open APIs…
Keep hacking!
So the news is that del.icio.us, er, I mean “Delicious” is all set to be jettisoned by Yahoo! I’m a bit saddened by the news because I’ve been a user and fan of delicious since the early days… my oldest bookmark goes back to December of 2003… that’s 7 years worth of bookmarks.
I’ve written a lot about delicious over the years, but even through all of my posts, I was always prepared for the day it would go away, the day it would disappear, the day it would die. Last week it looked like that day had come.
I’m not concerned about my data, as it lives on elsewhere… See Scuttle rides again for info on that. My main concern was losing what many considered the whole point of del.icio.us… the network.
del.icio.us was the first big success in “social bookmarking” and one of the first sites to really get (and push) “tagging” as a way to categorize things. Back in the 2004 many of us thought Yahoo! acquiring del.icio.us was going to lead to great things, not just for Yahoo! but for the web itself.
It appeared that del.icio.us was all set to become a success story for Yahoo! but as is often the case, trying to fold what made something great into a larger organization doesn’t always work. In fact, it’s probably rare that it works.
If anything good comes out of this whole thing, it’s the ideas people have, and the code/services that may crop up in the future. Just check out these posts:
I’m proud to say that Jon and Les and I were all, uh “comrades” back during the web’s heyday of the mid-2000’s… I think were we called “bloggers” back then. It’s neat to see their thoughts on this issue. Les has even more insight, as actually he worked on del.icio.us.
So now what? Well, I did get a Pinboard account, and I’m still collecting everything at p2url.com/bookmarks, and both of those are currently syncing with del.icio.us, which (for now) is still the master. I’ll figure out what to do next, but I’m not worried about the data going away. I am interested in what kind of distributed social bookmarks network may come out of this…
I’ve had a renewed interest in my data lately, and that’s manifested itself in making sure I’ve got copies of my data. Those bits and pieces we so easily create on other web site? I want them. I want them “here”, where here is under my own control, on my own site, my own server, not just living in the cloud somewhere, at the whim of some 3rd party.
See, I’ve been using Twitter for a long time… since September 2006 supposedly. I’d love to go back and see what my first dozen tweets look like, but Twitter doesn’t allow that. I’ve been kicking myself for not saving all of that data since the beginning, but with micro-content you tend to think that a. It’s tiny, so it doesn’t matter, and b. It’ll always be there. We’ve learned (time and time again) that this isn’t the case.
About 2 years ago I was doing some Drupal work and set up an aggregator to ingest my Twitter feed, which managed to back up a large portion of my tweets, but not all of them. It was more experimenting with Drupal than trying to create a good backup. Still, I was slightly happier knowing I had some sort of archive. I still wanted something that would display my own content (tweets) on my own site.
I then found Tweet Nest. Developed by Andy Graulund (@graulund) it’s a simple open source PHP/MySQL application that does just what I wanted. It grabs your tweets from Twitter using the API, and stores and displays them on your own site. Perfect!
The install was pretty painless, and I actually spent more time customizing the CSS (and I’m still not happy with what I came up with.) Of course two days after I installed it, it appeared to stop working. But alas, no worries, it was just a “rate limit freakout” with the Twitter API. After that, things have been smooth.
Tweet Nest also did a great job of grabbing all of my tweets as far back as October 2009. Not quite back to September 2006, but I’m starting to think that will never happen. But, I did manage to get tweets as far back as March 2008, thanks to another service called Backupify. Now I’ve got to find a good way to load the data into Tweet Nest, and I’ll have most (but not all) of my tweets stored on my own site.
You can find my archived tweets at rasterweb.net/micro, and just like my recent Delicious/Scuttle/bookmarks exploration, I now feel that much better about my data. (But I’m not stopping there. The next project may take quite a bit more hacking on my part, but it’s another one that’s long overdue.)
So go check out Tweet Nest if you want an application on your own server, or Backupify if you just want a nice, secure backup.
Because I always fear the CloudFail™, and because I like hacking on open source projects and interesting APIs, and because I like to store my own data… I’ve got Scuttle up and running again.
I started using del.icio.us in 2003 to store my bookmarks. I loved it. Over time del.icio.us had some issues with reliability and there were times when I could not access my bookmarks that were stored in del.icio.us. This made me sad. Around 2005 an open source project named Scuttle appeared, which was basically a clone of del.icio.us you could run on your own server. I ran it on my own server, and put together some code to make it sync up with del.icio.us. (At some point the name changed from “del.icio.us” to “Delicious” but I am calling it del.icio.us here for historical reasons.)
For years this ran on my own home server, and served as a great backup for my bookmarks in del.icio.us. This was all very easy because Scuttle implemented (most of) the del.icio.us API, so syncing the data between the two was so trivial, even a hacker like me could cobble a bit of code together to make it work.
As an interesting sidenote, while del.icio.us ended up being pretty darn reliable (Yahoo! eventually acquired it) others were not so lucky. Ma.gnolia was another social bookmarking site. Ma.gnolia died a tragic death, and lost all data. See http://en.wikipedia.org/wiki/Gnolia#January_2009_total_data_loss for details. This is why having an API is important. This is why having a backup is important. This is why you should trust yourself with your own data.
I’ve collected a few links for the blog posts I wrote about Scuttle, del.icio.us, and related sites.
(Most of the bits above can also be found on the page at: http://p2url.com/bookmarks/about.php/, and while you are welcome to browse through http://p2url.com/bookmarks/ as it is a public site, I’m the only on who will be adding bookmarks to it, because it’s there for me. If you want your own, I encourage you to go make it happen. Let me know if you need help.
I’ve been meaning to actually use Ma.gnolia for quite some time now, but being happy with del.icio.us, and the lack of a matching API was holding me back… well, no more…
The folks at Ma.gnolia have released The Mirrord API, which (like Scuttle) is the del.icio.us API, so now we should be able to easily more data between the two. (I guess it’s time to update delisync.)