Almost perfect backup backup solution

The double backup word in the title is not an oversight. If I would be talking about “almost perfect backup solution”, it would be Time Machine – which has proven, despite it’s minor annoyances (see this) to be very unobtrusive and functional. As long as you have Mac and Leopard, of course.

What I am talking about here is second level – an offsite backup, that you may need in case you house burns down, gets flooded or your computer with time machine disk gets stolen. I do not live in tornado valley, earthquake zone and crime rate around Westboro is fairly low even compared to low Canadian levels, but anyway.

The product in question is Backblaze and I am happy user since December last year. It is cloud based service, running on (I assume) Amazon S3 and unlike Time Machine it works for all you stuck in Windows world as well. Not available yet for all you brave explorers of multiple universes of Linux, but I guess you would not give up rsync anyway :-).
All you need to do is setup a very low profile client that runs in the background and uploads all that was changed. The initial backup can take few weeks, depending on the size of the hard disk.

It backups almost everything, except system areas and few excluded file types – like DMG and VMDK (virtual machine volumes). You can define your own exclusions but you cannot  un-exclude the default exclusions.

Backblaze has quite attractive pricing scheme: you pay $5 a month per computer and can backup as much as the pipes between your house and cloud allow you to push up. The price of one venti chai latte is in my books very much worth the good feeling.

When the disaster strikes and you need your files, you do not have to go the slow route and download multi-gigabytes of data. For reasonable fee, Backblaze will send you your data on DVD’s or even on USB disk. I hope I will never need that :-). For small recovery you can always access files using Web interface.

Because I am curious creature and like to understand how thing work, I was watching its progress for few weeks. Thanks to my curiosity I had several communications with backblaze technical support. I am happy to report that:
a) it exists ! (this is always the case with cloud companies )
b) it is very fast – I got back response in few hours, one day max
c) it is very competent and friendly. The person I communicated with knew the product at the deepest technical detail level.

Last but not least: privacy and security. Many people are concerned about having their data anywhere except on the server in locked office. I trust strong encryption. In addition to using SSL for transfer, Backblaze gives you an option for aditional encryption on client side – before the data leaves your computer it is encrypted with the key only you know. This way you cannot download the data through Web unless you enter the key and not even Backblaze can read your data.

So why almost perfect ? There are few minor issues. I would certainly like to have more control (and better UI) for both monitoring and management of the files to be backed up. Either GUI client, or simple way to put a file inside directory that would work as .gitignore. Actualy, for a developer, it would make sense not to backup anything specified in .gitignore or .svnignore or .cvsignore files, because if something is not worth putting to source control, it is not worth backup up either.

Other are duplicate files: I have on my notebook subset of pictures, podcasts and music from home iMac. Those files are backup up and tranferred twice. With volume, this becomes an annoyance. Backblaze could based on SHA1 recognize the duplicate files within same account and offer option skip those already uploaded – same way how Git stores each blob only once.

Last issue (which is completely out of Backblaze’s control) is your bandwidth. Since I started, I am maxing out my 95 GB transfer limit with Rogers in third consecutive month. Here in Canada, 95GB is max you can get unless you  pay for business connection[1] (which is several times the price of “Extreme plus”). You have some control over the backup upload speed – you can “throttle” the speed and you can also manage schedule (to a limited extend). This may or may not impact you – depending on your bandwidth allocation and size of the data to back up.

All summed – definitely recommended.

Disclaimer: I am not anyhow affiliated to Backblaze product or company. Only reason for this blog is my personal, very positive experience with their product and user support, which I believe deserves to be shared.

Evernote Firefox plugin version 3.0.0.45382 – do NOT update

Since few months I have slowly switched all my online notebooks to Evernote. In case you are not familiar with it, it is a great service. You can clip pieces of Web pages, type in notes, attach documents and multimedia files (pictures, sounds) and store them online to be always available through nicely done Ajaxian Web interface. The notes can be tagged, organized into notebooks and full text searched. One of the killer features is that for uploaded pictures, Evernote will do OCR and include the photographed tex into full text search

For each platform there is desktop application that seamlessly synchronizes the on-line notes with local database and also allows create notes using thick client interface with all desktop goodies. If you are iPhone users, make sure you get the free app that allows both access the notes as well as capture notes – textual, pictures or voice notes with the GPS information.

The basic service is free and limits you not by storage taken on the servers, but by transfer 40 MB a month, which is a lot unless you do picture notes. The very reasonably priced pro-service increases the monthly transfer quota to 500 MB and gives few more goodies – like PDF and Doc attachments to the notes. I am fairly frequent user and never used more than one half of that amount. To make clipping from the browser easier, Evernote offers Firefox plugin as well as Safari plugin.

The Firefox plugin is equipped with the self-update capabilities. And this is why I am posting this: if you have not updated yet, stick to version 3.0.0.128, do NOT upgrade to latest but far from greatest 3.0.0.45382. The latest version of Evernote (3.0.0.45382 as of today) is definitely a step back. Unlike the .128 which clips the selected text nicely into the Web, the .45382 always opens the desktop application and creates note there. This is only annoying and there is no obvious way how to prevent starting desktop client.

The real problem is that the captured note contains HTML attachment named ‘Firefox clipping.html’ and does not show the clipped text in the Evernote desktop client. To see what you clipped you need to open the clipping in the browser. To add insult to the injury, the quality of clipping is dramatically worse than in .128. The layout is all over the place, and result looks much worse. It is hard to tell whether this is caused by clipping or by different way how to display result.

If you got to the point of bad surprise after updating, here is how to bring back the version .128:

– uninstall extension

– restart Firefox

– download the previous version of the extension (the .xpi file) from here https://addons.mozilla.org/en-US/firefox/downloads/file/47839/evernote_web_clipper-3.0.0.128-fx+tb.xpi and save it on local disk

– in Firefox, open the downloaded file and confirm installation

– restart Firefox

I hope that somebody at Evernote will recognize that the version .45382 was a bad idea and bring back the clip-to-web capabilities (or at least make it configurable), avoid creation of useless HTML attachments and fix the clipping engine. Until that happens, I will stick with .128.

Bye bye Ruby, hello Groovy

I first time discovered Ruby back in 2006 (yes, I know, I was late to the game), and immediately fell in love with it. The dynamic nature of the language, the consistency, pure esthaetics and practicality certainly changed the way how I saw software development and programming languages.

Since that time, I made several attempts to integrate Ruby into my professional life and make it part of the toolbox. It was cool to play with Ruby in my spare time, but I wanted to use it on projects whose main development language/platform was mainly Java. Use it as scripting, glue language. Use it as toolkit language to e.g. generate test data, access databases, convert files, build projects and maybe even build a piece of Web applications (admin apps for example).

It never worked. The main problem was availability of the Ruby platform in all environments. While JVM was there by default, Ruby had to be installed and sometimes compiled for the more exotic platforms. And that can be a big deal if you have not full control over the environment – scenario which is pretty much guaranteed in enterprise environment. It is hard to argue with the sysadmin saying “You want to install that in production just to run scripts ? Why do not you use Perl or Bash or Java that are already there ?”

For a little while I thought that JRuby may be the way. After all – all you need is JVM and JRuby is just another JAR, right ? As Goethe said, grey is all theory and green is the tree of life :-). A language is as good and useful as are components and libraries available. One certainly does not want to write everything from scratch. Libraries in Ruby are Gems and Ruby provides very nice, mature and IMHO superior system for component management to Java JAR’s – because it handles different versions of same Gem very well (maybe some day there will be Gem hell after DLL hell and JAR hell 😉 ). Unfortunately, some Gems (by Murphy’s law most of the really interested ones) are for performance reasons built as thin Ruby layer around native (written in C) library. And JRuby does support that, making most of the Gems unavailable.

Even if JRuby had all the gems available, there would still be a problem that the Gem system and Jar system are different and do not quite fit together. Also, from language point of view you certainly can use Java objects in JRuby and vice versa, but doing that makes you feel slightly schizophrenic – what reality am I in? Is this a Java Java class or Ruby Java class ?

Third problem that I have encountered after coming up with some Web App in Rails is that the deployment model is very different from Java deployment model which myself and people in organizations we work with understand really well. We know how to deploy so that it scales, we know how to monitor and maintain a Java enterprise app. But not a Rails app with all those Mongrels, lighttd’s and other creatures :-). This leaves many open questions like “How do we size hardware for expected load ?” for which I do not have answers, and judging by well publicized issues with Rails apps scalability, even the best and brightest in Ruby world may not have either – or at least some people say so.

About at the same time I discovered Ruby, I also become aware of the strange Java dialect called Groovy. It sort of tried to do the same thing I hoped to use Ruby for, only from firm within Java environment. The original reason I did not want to look deeper at Groovy was that compared to straight elegance of Ruby, it looked kind of ugly. The Java skeleton was sticking out in wrong places and alltogether it just did not feel as good as Ruby.

I have to publicly admit I was wrong.

Being a Mac user, I have license for going after good looks and white shiny objects, but when it comes to programming languages, the good looks may just not be enough. The reality is the proof.

During last 12 months, we have quietly and very successfully used Groovy components and pieces on three large projects. It fitted perfectly, never running into anyof the issues above.

Through these projects, I learned to appreciate the Groovy way, my sense of aesthetics stopped to be offended by certain syntax constructs in Groovy and I even started to like them better than Ruby ones. For example, I am now convinced that Groovy categories are safer and better approach that explicitly alerts programmer about using class extension, than re-opening any class in Ruby (which is still possible in Groovy by assigning closure to member in metaclass). Imagine how confusing it can be for software maintenance when reopening and using happens far apart in the source code.

But the most important, the painful realization ” how the heck do I do the XYZ thing in this language ? If I only were coding in Java, it would be so much simpler ” is history with Groovy. Everything that I was used to use in last 12+ years in Java is still there, all the goodies of Jakarta Commons and way more.

Groovy community seems to be less opinionated, less self-righteous than Rails/Ruby community and more understanding for weird requirements and idiosyncrasies of enterprise environments. Rather than telling you “you should not want to do this” and “DHH thinks it is wrong”, you actually may get a helpful pointer to useful website or blog how to do that stupid thing in Groovy or Java or combination of both. Because you know, when one needs to accomplish something that seems to be wrong and illogical, being told that it is wrong and you should better forget about it does not really help. People who worked with real enterprise system’s integration understand, that cost of touching or changing certain systems is so prohibitive that it is out of question and doing the technically wrong thing may right (and only) option for given situation and customer.

Therefore – bye bye Ruby, Hello Groovy. Next things to embrace and embed will be Grails.

The aftermath of mainboard change

In theory, exchanging mainboard on Macbook has no impact because all your data is stored on your harddisk that is untouched. In real life, there are few minor surprises.

First, your MAC address of the network card had changed. This is something you will not notice, unless Murphy’s law plays funny game with you, as it did with me. When arriving back to the office, I was able to connect to WiFi network, but the ethernet was stubbornly getting the “internal IP” – 169.254.x.x address, which is pretty much useless from connectivity point of view. We are running two separate networks – both NAT-ed, one on WiFi – mainly for guests, other internal. Not getting an IP address did not make any sense: the cable was OK, because other machine worked just fine with same drop/cable. The Ethernet connection was OK, because it worked when assigned IP address manually.

The problem was – new MAC address. During last week or so, by playing with virtual machines, I must have allocated all available IP addresses from DHCP server space. The leases are fairly long lived and all slots were taken by either computers around the office or by both running and also now defunct VM’s. One of them was still kept reserved for my old and gone MBP’s MAC address. Nobody would notice the problem, unless you tried to attach new DHCP based VM or new computer. Lesson learned – if you have problems with internal IP address does not want to go away – check the DHCP server. In home environment, resetting the router mostly helps.

Second effect of changed MAC address was that all VMs in Fusion started to ask whether they were moved or copied. Always answer “moved”, otherwise you VM will get new virtual MAC address generated – which can have impact on your DHCP space (if in bridged mode).

Third, quite unexpected effect was that Time Machine stopped working, with ‘volume cannot be found’ message. See e.g. http://www.macfixitforums.com/ubbthreads.php/topics/454151/Time_Machine_can_t_find_backup for more details. It looks like Time Machine is using the MAC address to match the backup volume with the computer it belongs to. Clean solution is to erase backup and start from scratch – or use different disk. Partial solution is a hack – in /Volumes/Time Machine Backup/ locate file that has the MAC address encoded in the name, e.g:

/Volumes/myTMbackupVolume/.0a1b2c3d4e5f

Get the new MAC address (from ifconfig) and copy the old cookie file to new cookie file – using the new MAC as name. This made the volume accessible again – with unfortunate effect that pretty much ALL content was considered as unbacked-up and first back up took away over 90 GB of disk space. But at least, the old data was still there, should the need ever occur.

Maybe this experience will be useful for somebody else – it was quite good learning experience for me.