Great solution for offsite backup

2007/10/25

I have been using it for over half year now and was very happy with it. After I did today hear Steve Gibson mentioning (and recommending) it on Security Now!, I want to share my experience and add my vote of confidence.

The solution is Amazon S3 service (Simple Storage Service), fantastically affordable system to store your data securely on-line. You pay as you go – the size is unlimited and you are charged only for what you store and bandwidth you consume. Fantastically affordable means 15 cents for gigabyte-month storage and 10 cents for gigabyte transfer in / 18 cents for transfer out which will drop to 13 cents if you use it more. This means that to store my approximately 30 GB collection of pictures I need to upload them first (for $3) and then pay $4.50 monthly for storage – plus the download traffic. But of course, I am not using it for the images because I really need and like the nice album user interface which SmugMug provides. But to archive documents – this is just the perfect solution.

The S3 is focusing on developers and the service is accessible via Web service. You can choose from many available implementations for their API – in Java, Python, Perl, C#, VB, Ruby – you name it. For non programmers, there are client tools available that completely abstract the storage access and make the S3 appear as just another drive. From many clients avialable, I have selected (as well as Steve and Leo did) the Jungledisk. Unlike some other services that are trying to stand between you and your storage, take over not only data flow but more importantly money flow (and often charge fat premium), the good guys at Jungledisk just want to sell you the client and let you pay directly only Amazon fees to Amazon. The price for the client is just $20 – it is no-brainer. For this price you will get client version for all three major platforms (Mac, Linux, Windows) as well as source code of the “engine” part of the solution – in case you want to access same data through UI or programmatically.

After installation, the S3 will appear as another disc under windows or network volume on OS-X (I did not try Linux – yet). Jungledisk contains scheduler and can do automatic backup of defined parts of your disc to S3 – or you can use it for manual backup, as a very reliable and somehow slower external disc.

The big issue with remote storage of sensitive documents is security: can you really trust with your precious data to a third party (even if that part is Amazon) ? I think this is up to anyone to decide – but the S3 comes pretty close to my definition of secure-enough system and Jungledisk plays along very nicely. All traffic between you and Amazon is of course encrypted (SSL) and your data is stored as encrypted as well, by default using a private key that Amazon provides you. This allows key recovery – but also allows (in theory) that someone on Amazon side could read your files. If you want however, you can generate your own key pair and use it to encrypt the data – and all you need to do is properly configure your client Jungledisk. Or if you are really paranoid you can encrypt your data even before they even get to Jungledisk and Amazon – if you want to exchange convenience and easy of use for more security. In the last two cases, nobody on the earth will be able to read your files – but if you loose your key, you will need few million years to break it :-).

Give S3/JungleDisk a try – you may like it too …
PS: If you are curious about performance and want more than my subjective feeling of “very reasonble” – read this.

PPS: The Smugmug actually *is* running on Amazon s3 – but because they use hundreds of terrabytes of space, obviously were able to get the storage for a wholesale price. The $59.95 / year membership of Smugmug would buy you on S3 about 15-25 GB storage and reasonable usage. As most people have less than 15 GB images, the Smugmug can actually make some money and employ really talented designers.

Advertisements

Images from vacation are on Smugmug

2007/09/13

For those of you who asked: I started to put up some images from Europe 2007 trip on Smugmug. Small subset of course (total amount with NEFs is almost 20 GB).

I will upload more as I get to weeding them out :-).


Smugmug “processing image” – follow up

2007/07/10

In the previous article I have mentioned the potential issue with some uploaded images hanging in state “Processing image” for surprisingly long time after upload and expressed hope that there is some background “catch it all” process that would take care of all these. As it turned out, there is. All uploaded images that were haging in the processing stage turned up to be fixed by next day or so (it was about 10-20 images out of about 3-4 thousand).

Here is a simple trick how to make sure it is indeed so. After uploading a gallery, tag every image in processing with some distinctive tag, e.g. “checkme”. Regardless in how many galleries, categories or subcategories your images will be, you will show them all by clicking on the checkme tag in your home page. After verifying that the images are OK, you can use bulk edit to remove the tag from all.


First week with SmugMug

2007/07/07

Week ago, my SmugMug 14 day trial period expired and (as expected) I signed up for the 1 year service. Here are my first impressions after over week of use, with over 3000 uploaded photos and 6.8 GB disc space consumed :-).

Good news is – it really works as advertised and I still like it a lot. I have not really tried many of the features, but what I did try, worked as expected. The uploader (I am using the default, Java applet based drag-and-drop universal uploader) is very friendly – the small status windows shows not only status of the upload, but also the thumbnail of the uploaded image.

Every photo lives in the gallery and the galleries can be assigned into categories with optional subcategories. The system is pretty smart – when you create a gallery inside category (or category / subcategory), these are pre-set as defaults (which you can change, of course). The uploaded file names are parsed and “meaningful” words are automatically assigned as keywords. As long as the words are separated by spaces or underscore. This works best in English – some of my pictures have names in Slovak and the word stemming generated multiple keywords – but that is the price for not using English :-).

Proper tagging of images is a lot of work. Once you do that, though, the search using keywords is pretty effective. You have the classical tag cloud and can select the images by the keyword. SmugMug offers also related keywords and allows to further limit the search.

I have decided to organize the photos chronologically, using categories and subcategories to denote year and months (one or more). This in theory can be achieved using built in time line, but more than 30% of my images are scans and have no EXIF data in JPG. It would be great if somehow there was a possibility to assign date (at least year or year and month) for the set of images – same way as keywords are assigned. However, this is not yet possible.

Within the categories / subcategories, you can have only one level of galleries. At first I thought that this 2 or 3 level fixed hierarchy will be too limiting and inflexible, but in reality, it is not such a big deal. In the occasions where I have nested folders with images – e.g. vacations folder named Europe and subfolders for Italy and Switzerland (where Europe contains all other images) – I simply created galleries named ‘Europe’, ‘Europe / Italy’ and ‘Europe / Switzerland’. The slash indicated the folder structure on the server and sticking with the naming convention will allow to use the API and do automatic synchronization (using Ruby ;-))

Getting images up there is quite time consuming. But after you have them uploaded, the fun begins. The viewing and navigation is where SmugMug really shines. I have seen many Web2-ish sites, but the paging here is one of the bests (if not the best). You can use either “classic” pager links:

classic-pager.png

or dynamic (Javascript):

javascript-pager.png

Loading of the thumbnails and images is very fast and with reasonably fast Internet connection, you can get very good user experience. Even the slideshow – about which I did not have very high expectations – works surprisingly well. Much better that you would expect from a browser based solution. Because the images are intelligently scaled down on server side, doing slideshow in full screen with 3-4 seconds delay is not much different than using default Windows viewer with local LAN (where the images are transferred in full size and scaled to fit the window on client side) – especially when you have 3-4 MB images..

The only minor annoyance is long lasting “Processing image” syndrom. After you upload image, the back-end processes the image and during this time (in 99.9% of images few seconds) the image thumbnail and real image is represented by “Processing image” placeholder.

cropper-capture28.png

and

cropper-capture129.png

As it happened in 2 or three cases during the upload of 3000+ images, the back-end processing never ended and even after few hours that particular image was still hanging in the state of processing. It is possible that this syndrome can be caused by the user (myself) opening the gallery being uploaded in second tab of the browser … I am not sure whether there is some background nightly batch that will catch it – but it is a good idea visually verify the upload after it finishes.

Btw, if you consider going with SmugMug, here is coupon code that will be give you $5 discount from the $39.95 yearly price: ndkpdNg15cgYI


Search for perfect online photo storage

2007/06/25

One thing that is on my Someday/Maybe list for quite some time is to get some off-site storage for my digital images. They are reasonably safe on RAID-5 NAS with a copy on USB disk, but what is missing is off-site copy. At minimum, a remote backup, ideally with an UI resembling something like Aperture or Photoshop Elements or Shoebox: photo organizer with Web intefarce and remote storage. So that in addition to have offline backup I would have an option to work with images from any location.

I looked on several sites – Flickr, Fotki, Picassa Web. I liked Picassa, but 1 GB storage space may be plenty to publish few sized-down galeries, but close to useless for archival. Who has less than 1 GB digital photos ? I need to upload about 50x more … It looked pretty bleak – but then friend of mine made me aware of SmugMug. I was impressed with what I have read in service description and even more when I saw it working.

It is still not the 100% perfect solution for me – but it certainly is the best thing I have found so far. The presentation is beautiful, it is fast and pleasant for the visitor. Oh, did I mention UNLIMITED storage for very reasonable price – $39.95/year. Plus unlimited traffic … And it is really fast.

The way how SmugMug organizes your images are galeries. Similar to filmrolls from iPhoto, every images belongs to exactly one gallery. You can organize your galleries to categories and subcategories. Based on EXIF information, SmugMug can provide you with timeline and access your images based on the capture or upload time. Within the gallery, you can use drag and drop to re-organize the order, or create the slideshows.

SmugMug has attracted nice and healthy ecosystems of users and tools. In addition to default tools (such as mass uploaders), several user made tools are available.

It is not only nice and smooth UI that is worth admiring. One area where SmugMug really shines was support. I had some questions so I fired off an email on Sunday afternoon, not really expecting anything other than automated response until Tuesday. What happened was nicest customer support surprise I have experienced in last few months. In less than hour I had an answer from real human being – what a rare experience these days. This with combination with really nice application makes SmugMug stand above the crowd ;-).

If I could make any additions to improve this very nice application, here are my suggestions:

1) Allow alias for images. In order for an image to appear in multiple galleries, it must be copied. There is a tool that makes the process manageable, but copying is confusing – why should two images exist with same file name ? It is essential to allow same image be “member” of multiple galleries without copying it

2) Allow “stacks” – similar pictures on top of each other – with best on top and an option to “explode” and “implode” the stack

3) Really useful would be “smart galleries” – picture appears in the gallery based on metadata, e.g. Nikon, Olympus