Categories
Uncategorized

Scuttle rides again!

Because I always fear the CloudFail™, and because I like hacking on open source projects and interesting APIs, and because I like to store my own data… I’ve got Scuttle up and running again.

p2url: Bookmarks I started using del.icio.us in 2003 to store my bookmarks. I loved it. Over time del.icio.us had some issues with reliability and there were times when I could not access my bookmarks that were stored in del.icio.us. This made me sad. Around 2005 an open source project named Scuttle appeared, which was basically a clone of del.icio.us you could run on your own server. I ran it on my own server, and put together some code to make it sync up with del.icio.us. (At some point the name changed from “del.icio.us” to “Delicious” but I am calling it del.icio.us here for historical reasons.)

For years this ran on my own home server, and served as a great backup for my bookmarks in del.icio.us. This was all very easy because Scuttle implemented (most of) the del.icio.us API, so syncing the data between the two was so trivial, even a hacker like me could cobble a bit of code together to make it work.

As an interesting sidenote, while del.icio.us ended up being pretty darn reliable (Yahoo! eventually acquired it) others were not so lucky. Ma.gnolia was another social bookmarking site. Ma.gnolia died a tragic death, and lost all data. See http://en.wikipedia.org/wiki/Gnolia#January_2009_total_data_loss for details. This is why having an API is important. This is why having a backup is important. This is why you should trust yourself with your own data.

I’ve collected a few links for the blog posts I wrote about Scuttle, del.icio.us, and related sites.

(Most of the bits above can also be found on the page at: http://p2url.com/bookmarks/about.php/, and while you are welcome to browse through http://p2url.com/bookmarks/ as it is a public site, I’m the only on who will be adding bookmarks to it, because it’s there for me. If you want your own, I encourage you to go make it happen. Let me know if you need help.

Categories
Uncategorized

The (Low Cost) Red DIT Station

If you’re on location with the RED, it’s a good idea to have a Digital Imaging Technician Station (or DIT Station) which will allow you to backup up all the footage you are shooting while you are in the field, and will also allow you to view the footage and make sure it’s all good before you leave the field.

If you’ve got an unlimited budget, I’d recommend the top of the line MacBook Pro (which can easily run $3,000-$4,000+ depending on how you deck it out) as well as a few external drives. Look at the G-Technology offerings, as they have various solutions depending on your budget and needs. The G-Raid mini is a nice, small, bus-powered drive with a quad interface (FW400, FW800, USB2, eSata.) Get a few of these. You could go with the 1TB, but the 500GB might be fine. You want your data in two places, and not “RAID two places” but “two different physical drives” places. The RED Drive holds 320GB, so if you are dumping a full RED Drive, you could just dump it to a 500GB drive (well, two drives) and mark it as “DONE” and move on. Oh, and since we have gobs of money, pick up an ExpressCard with eSata on it to put in your new MacBook Pro.

RED DIT Station
For the “dumping” of data, you can either use something like R3D Data Manager which does all sorts of checksums for the data, can backup to multiple locations at once, and has other nice features, or you can do a standard Finder copy. R3D Data Manager is $79, which is not too bad… again, what is your footage worth? If you want to fly in the Finder, and can review everything you dump, that can work too.

Speaking of software, RED has provided a number of excellent tools for free. Grab REDCINE-X, RED ALERT! and whatever else you may need at RED’s support site. REDCINE-X is a nice little app for checking the footage you just shot.

So where’s the “Low Cost” part come in? Well, let’s say you don’t have the budget for the top of the line MacBook Pro… that’s fine, get the bottom of the line MacBook Pro (at least get a Pro, you want Firewire 800!) and with that get some drives. I still really like the G-Raid minis, but you could probably find USB bus-powered drives that are cheaper (and slower!) if needed, but again, we do want as much speed as we can afford. (Bus-powered drives are nice because you never know where power will be when on location… if there is power at all.)

If you are shooting to CF Cards, do yourself a favor and get a Sandisk Extreme FireWire Reader (about $65) which will dramatically speed up transferring of footage compared to a cheap USB Card Reader. (The low end MacBook Pro has just one Firewire port, but you can plug the Firewire CF Card Reader into the the back of the G-Raid mini, which has two ports.

RED DIT

So we’ve go the MacBook Pro, which is $1,300 before adding AppleCare (but here’s a tip, you can get a refurb, plus AppleCare for it, for about $1,295) add in two G-Raid minis (the 500GB) at about $450, and a CF Card Reader for $65. We come in under $2,000 for this set up, which is not the ULTIMATE portable RED DIT Station, but it’s a Low Cost one, that is effective enough to get the job done.

Categories
Uncategorized

Machine Tags for Bucketworks

The following is a proposal for the use of machine tags for content related to Bucketworks. I’d love feedback from people familiar with machine tags, and people involved with Bucketworks.

The idea behind machine tags is adding a sort of namespace and hierarchy to a tagging system, and to assign it to a specific thing.

So for Yahoo’s Upcoming site (which lists events) PhotoCampMilwaukee had a tag like this:

  upcoming:event=1518729

These are used by Upcoming, which uses the Flickr API to gather photos for display on it’s site.

Now, we can assume that every event at Bucketworks would have a unique ID number, and use something like this:

  bucketworks:event=47264

This number could come from Drupal, CivicCRM, SalesForce, whatever… Just needs to be unique, and referenceable.

Now, we can see the same thing with users… on Flickr, if I see that Gabe is in a photo, I could use the tag:

  flickr:user=heygabe

where “flickr” represents Flickr, “user” is the thing we are specifying, and “heygabe” (his username on Flickr) is the value.

So… extended to Bucketworks, if Gabe’s username there is “gwollenburg” we might use:

  bucketworks:member=gwollenburg

Hmmm, maybe Gabe is not a member… and lots of people who come to events at Bucketworks might not be members (yet!) so maybe it is better to use “person” (but then, “user” could be the proper term as well, as the people are “users” of Bucketworks, right?)

  bucketworks:person=gwollenburg

Obviously every person needs a unique (to the Bucketworks network) username we can reference them with. (Note that I used Gabe’s Yammer username “gwollenburg” and not his bucketworks.org username: “heygabe”. (Is there also a username in SalesForce? It would be nice if there was a way to cross-reference all of them, though really, we want the most publically used one, so they are easy to determine.)

Jenn asked about tagging photos at Bucketworks with the room names, so we might use:

  bucketworks:space=greenhouse
  bucketworks:space=flowspace
  bucketworks:space=playspace
  bucketworks:space=cafe
  bucketworks:space=chromatarium
  bucketworks:space=parkinglot
  bucketworks:space=loadingdock
  bucketworks:space=sidewalk
  bucketworks:space=roof
  bucketworks:space=outside

Here are photos on Flickr with the tag bucketworks:space=playspace

(I went with “space” rather than “room” because I think it’s open to a wider interpretation, and luckily, Bucketworks seems to agree…)

(Are you thinking about geotags yet? They are another form of machine tags, and we should be using the coords for Bucketworks as well…)

The thing about machine tags is that you don’t so much just “come up with them” as they are defined by some entity, and then presented, and (hopefully) used.

So, if we did come up with a bunch of these, we would need to make them public, promote the use of them, show why you would use them (what cool things could be done, like, a slideshow with all photos from an event at Bucketworks) and having tools that make these things valuable would also be good…

What’s that? This reminds you of RDF, well, that’s okay :)

See Also: http://www.flickr.com/groups/api/discuss/72157594497877875/ and http://tagaholic.me/2009/03/26/what-are-machine-tags.html

Thoughts?

Categories
Uncategorized

Amazon S3 Update

Over at Z2 Marketing + Design we’ve implemented Amazon S3 as an off-site backup solution. As a design studio, there are a lot of large files, between photo and video shoots, print projects, and a zillion other things that require creating files, we’ve got our own storage issues to deal with. Will the building burn down? Dunno, it might, but if it does, it will be nice to know our data is safe over at Amazon.

Since I’m still moving a lot of older data there, the uploading is constant. I’d estimate I’m moving about a GB of data per day right now. In fact, let’s look at the numbers…

Amazon says we’ve transferred 16.035 GB this month, and we’re storing about that much as well. But wait, what is the cost of this? Well, as of right now, it looks like Amazon will be billing us just about $5 or so. (Probably a bit more, as I have another 11 days to keep moving data there.) So for the cost of a good burrito, we’ve transferred and stored over 15 GB of data. Freakin’ awesome!

I’m still not 100% happy with the tools, and how I’m doing the transfers (much of it is manual right now) but better tools will come along, and with a new server in-house, we’ll look at automating all the backups, so it “just works” and if something doesn’t work, it’ll let us know. Thanks Amazon!



>


Categories
Uncategorized

NAS versus Server

I’ve written quite a bit about the Linksys NSLU2 we’ve been using at the office, and the problems we’ve have with it. (Linksys NSLU2 Update, Linksys NSLU2 versus ????) Overall it’s been a nice device for what it does, and for the price (about $80) but we’ve outgrown it, and we’re looking for a better solution.

I think that better solution is just going to be a server. (Going back to Small Business – Server & Backup I guess.) We should be able to solve most of the problems we had with the NSLU2 by using an old G4 PowerMac. The Mac should be faster than the NSLU2, and should not have any of the permission problems we’ve experienced. We’ll also be able to just deal with AFP instead of SMB since we’re a Mac-shop.

(I still think the NSLU2 is a good device for what it is, and for a home network would probably work well. Though reading up on FreeNAS it looks like a cool solution, and something I may try to play with in the future.)

The Mac will also allow us to do a few more things, like run the internal wiki, and possibly handle the office calendaring needs. What I really look forward to is automating our Amazon S3 backups using a Mac file server.

Anyway, I’m sure I’ll keep posting updates on this as we go…