Bye bye Ruby, hello Groovy

2009/03/17

I first time discovered Ruby back in 2006 (yes, I know, I was late to the game), and immediately fell in love with it. The dynamic nature of the language, the consistency, pure esthaetics and practicality certainly changed the way how I saw software development and programming languages.

Since that time, I made several attempts to integrate Ruby into my professional life and make it part of the toolbox. It was cool to play with Ruby in my spare time, but I wanted to use it on projects whose main development language/platform was mainly Java. Use it as scripting, glue language. Use it as toolkit language to e.g. generate test data, access databases, convert files, build projects and maybe even build a piece of Web applications (admin apps for example).

It never worked. The main problem was availability of the Ruby platform in all environments. While JVM was there by default, Ruby had to be installed and sometimes compiled for the more exotic platforms. And that can be a big deal if you have not full control over the environment – scenario which is pretty much guaranteed in enterprise environment. It is hard to argue with the sysadmin saying “You want to install that in production just to run scripts ? Why do not you use Perl or Bash or Java that are already there ?”

For a little while I thought that JRuby may be the way. After all – all you need is JVM and JRuby is just another JAR, right ? As Goethe said, grey is all theory and green is the tree of life :-). A language is as good and useful as are components and libraries available. One certainly does not want to write everything from scratch. Libraries in Ruby are Gems and Ruby provides very nice, mature and IMHO superior system for component management to Java JAR’s – because it handles different versions of same Gem very well (maybe some day there will be Gem hell after DLL hell and JAR hell 😉 ). Unfortunately, some Gems (by Murphy’s law most of the really interested ones) are for performance reasons built as thin Ruby layer around native (written in C) library. And JRuby does support that, making most of the Gems unavailable.

Even if JRuby had all the gems available, there would still be a problem that the Gem system and Jar system are different and do not quite fit together. Also, from language point of view you certainly can use Java objects in JRuby and vice versa, but doing that makes you feel slightly schizophrenic – what reality am I in? Is this a Java Java class or Ruby Java class ?

Third problem that I have encountered after coming up with some Web App in Rails is that the deployment model is very different from Java deployment model which myself and people in organizations we work with understand really well. We know how to deploy so that it scales, we know how to monitor and maintain a Java enterprise app. But not a Rails app with all those Mongrels, lighttd’s and other creatures :-). This leaves many open questions like “How do we size hardware for expected load ?” for which I do not have answers, and judging by well publicized issues with Rails apps scalability, even the best and brightest in Ruby world may not have either – or at least some people say so.

About at the same time I discovered Ruby, I also become aware of the strange Java dialect called Groovy. It sort of tried to do the same thing I hoped to use Ruby for, only from firm within Java environment. The original reason I did not want to look deeper at Groovy was that compared to straight elegance of Ruby, it looked kind of ugly. The Java skeleton was sticking out in wrong places and alltogether it just did not feel as good as Ruby.

I have to publicly admit I was wrong.

Being a Mac user, I have license for going after good looks and white shiny objects, but when it comes to programming languages, the good looks may just not be enough. The reality is the proof.

During last 12 months, we have quietly and very successfully used Groovy components and pieces on three large projects. It fitted perfectly, never running into anyof the issues above.

Through these projects, I learned to appreciate the Groovy way, my sense of aesthetics stopped to be offended by certain syntax constructs in Groovy and I even started to like them better than Ruby ones. For example, I am now convinced that Groovy categories are safer and better approach that explicitly alerts programmer about using class extension, than re-opening any class in Ruby (which is still possible in Groovy by assigning closure to member in metaclass). Imagine how confusing it can be for software maintenance when reopening and using happens far apart in the source code.

But the most important, the painful realization ” how the heck do I do the XYZ thing in this language ? If I only were coding in Java, it would be so much simpler ” is history with Groovy. Everything that I was used to use in last 12+ years in Java is still there, all the goodies of Jakarta Commons and way more.

Groovy community seems to be less opinionated, less self-righteous than Rails/Ruby community and more understanding for weird requirements and idiosyncrasies of enterprise environments. Rather than telling you “you should not want to do this” and “DHH thinks it is wrong”, you actually may get a helpful pointer to useful website or blog how to do that stupid thing in Groovy or Java or combination of both. Because you know, when one needs to accomplish something that seems to be wrong and illogical, being told that it is wrong and you should better forget about it does not really help. People who worked with real enterprise system’s integration understand, that cost of touching or changing certain systems is so prohibitive that it is out of question and doing the technically wrong thing may right (and only) option for given situation and customer.

Therefore – bye bye Ruby, Hello Groovy. Next things to embrace and embed will be Grails.

Advertisements

Trying out Groovy with Oracle database

2008/04/07

There is a saying that necessity is the mother of invention. Such necessity happened last week and forced me to try out the Groovy language.

The trigger was need for creating good data set for testing changes in a full text search. I had to locate few hundred of obsolete technical documents to be used as test data, dump the metadata as well as the BLOB data to disc, create INSERT sql scripts and loader that would from Ant pre-load the database with the test set and insert the BLOB’s. It is fairly simple task, the issue were boundary conditions:

Database in question is Oracle. I have no OCI8 installed on my MBP and refuse to install Oracle on OS-X – it was little fun getting it up and running on Linux. Usually I run Oracle in Windows or Linux VM and with lack of OCI8, only option how to communicate with it is Level 4 JDBC driver.

Because of missing OCI8, Ruby was out of question (the gems for Oracle access needs OCI). So was PL/SQL, for the reason of administator priviledges. I am not exactly a PL/SQL programmer, but know enough to write stored proc that loads or unloads BLOB, assumed that UTL_FILE and DBMS_LOB packages are available and accessible. Which was not the case.

Before falling to default and writing it all in Java (which would work, but take certainly longer than I was willing to invest), I decided try out this Groovy thing :-). My exposure to the language was minimal – I never programmed in it (not counting 2 hours spent browsing Ruby in Action), but I hoped that similarity with Java and almighty Google will help me out …

Connecting to database in Groovy is simple and elegant:


import groovy.sql.Sql

db = Sql.newInstance(
            'jdbc:oracle:thin:@myoraclehost:1521:XE',
            'USER', 'PWD', 'oracle.jdbc.driver.OracleDriver')

      def sql = """
            select d1.document_id, d1.version, d1.filename from document d1
            where d1.document_id in ...

           .... REST OF SQL DELETED ....
        """
        list = db.rows(sql)

The result is hash-like structure that can be used to drive both generating the DELETE and INSERT SQL statements into text file as well as retrieving the BLOB’s and saving them to HDD:


    def writeBlobToFile(db, docid, version, filename, reallyWrite=false) {
        def sqlString = "SELECT d.DOCUMENT_BODY FROM document d where d.document_id = '$docid' and version = $version"
        def row = db.firstRow(sqlString)
        def blob = (oracle.sql.BLOB)row[0]
        def byte_stream = blob.getBinaryStream()
        if( byte_stream == null ) {  println "Error for ${docid} : ${version}"  }

        int total = blob.length();
       
    // Write to a file
       if (reallyWrite) {
            byte[] byte_array = new byte[total]
            int bytes_read = byte_stream.read(byte_array)
           def fullname =     "/Users/miro/tmp/BLOBS/$filename"
           def fos= new FileOutputStream(fullname)
           fos.write(byte_array);
           fos.close()       
       }      
        println "Document $docid:$version, file: $filename, size $total"
        return total
    }

Loading the documents and metadata from disk to database is done by Ant script

<path id="groovy.classpath">
  <fileset dir="${lib.dir}">
    <include name="groovy-all-1.5.4.jar"/>
  </fileset>
</path>

<taskdef name="groovy"  classname="org.codehaus.groovy.ant.Groovy">
    <classpath>
        <path refid="groovy.classpath" />
        <path refid="database.classpath" />
        
    </classpath>
</taskdef>

<target name="reload-document-fixtures" description="Prepare the database for testing">
    <sql
        driver="oracle.jdbc.driver.OracleDriver"
        url="${database.url}"
        userid="${database.user}"
        password="${database.password}"
        print="yes"
        output="database_load-${DSTAMP}.txt"
        autocommit="true"
        onerror="continue"
        >
        <classpath refid="database.classpath"/>
        <transaction src="src/sql/Fixture/DELETE_DOCUMENTS.sql" />
        <transaction src="src/sql/Fixture/INSERT_DOCUMENTS.sql" />

    </sql>
    
    <groovy src="src/groovy/com/company/app/utils/LoadDocuments.groovy">
        <arg value="src/sql/Fixture/BLOBS"/>
        <arg value="src/sql/Fixture/METADATA.txt"/>
    </groovy>
</target>    

The Groovy task must be defined before used (with proper classpath). The groovy
task shows passing arguments and executing script. The most important part of
the script (loading binary file and inserting the BLOB) is here:


// each line in METADATA.TXT looks like this:
reg =  /document_id: \'(\w+)\', version: (\d+), filename: \'(.+)\', size: (\d+)/

db = Sql.newInstance(
        'jdbc:oracle:thin:@myoraclehost:1521:XE',
        'USER', 'PWD', 'oracle.jdbc.driver.OracleDriver')
counter = 0
new File(args[1]).eachLine { line ->
    line.eachMatch(reg) { match ->
        try {
            str = "${args[0]}/${match[3]}"
            println "Opening file: ${str}"
            FileInputStream fis = new FileInputStream(str)
            int size = fis.available()
            byte[] data = new byte[size]
            fis.read(data)
        
            // get the BLOB
            row = db.firstRow("select document_body from document where document_id = ? and version = ? for update",
                    [match[1], match[2]])
            my_blob = (oracle.sql.BLOB)row[0]
            if( my_blob == null ) println "my_blob is null!"
            outstream = my_blob.getBinaryOutputStream();
            outstream.write(data);
            outstream.close()
            fis.close()    
            counter = counter+1
        } catch (Exception e) {
            print "Exception: ${e}\n"
    }
}
println "Processed files: ${counter}"

The loader is controlled by the file METADATA.TXT that contains information linking document metadata in database (inserted by SQL statement) with file on disk. This indirect way allows easily “inject” document body with required search phrases, reload database and run tests.

The whole experience was quite pleasant and considering how little I knew about Groovy, it took very little time to create something useful. I wish I could have done some thing in Ruby (which I still like better) – but I was amazed how powerful the combination of Groovy + Ant can be.

The most valuable feature is no need to add anything really new or exotic to Java environment – all you need is Jar file and Eclipse plugin, no new/different library packaging scheme – Jars are still jars, not gems.

The only minor hickup and weird thing I found was passing arguments between Ant and Groovy script. It looks like that the groovy script should be a script and not use classes + static main method (which would be natural instinct for Java developer). Thanks to Christopher Judd for the hint.


Showstopper issue with JRuby ?

2008/03/16

After some break, I dusted off the Ruby to try out one interesting idea Peter presented yesterday that literally ASKS for Rails. So I grabbed the latest and greatest RubyNetBeans from Ruby Hudson. For some reason it stopped updates on January 26th so the latest version I have got was build 6327. It is bundled with JRuby 1.1RC1 and the Rails version that comes preinstalled is 1.2.6. Using the menu Tools -> Ruby Gems, they can be easily upgraded to latest and greatest 2.0.2.

The trouble begins when you want to install database connectivity gems such as sqlite3-ruby. The installer fails with the message:


trying to install

INFO:  `gem install -y` is now default and will be removed
INFO:  use --ignore-dependencies to install only the gems you list
Building native extensions.  This could take a while...
extconf.rb:1: no such file to load -- mkmf (LoadError)
ERROR:  Error installing sqlite3-ruby:
    ERROR: Failed to build gem native extension.

/Users/miro/Applications/<a href="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/bin/jruby" class="linkification-ext" title="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/bin/jruby">RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/bin/jruby</a> extconf.rb install sqlite3-ruby --no-rdoc --no-ri --include-dependencies --version > 0

Gem files will remain installed in /Users/miro/Applications/<a href="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1" class="linkification-ext" title="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1">RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1</a> for inspection.
Results logged to /Users/miro/Applications/<a href="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1/ext/sqlite3_api/gem_make.out" class="linkification-ext" title="http://RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1/ext/sqlite3_api/gem_make.out">RubyNetBeans.app/Contents/Resources/nbrubyide/ruby1/jruby-1.1RC1/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.1/ext/sqlite3_api/gem_make.out</a>

The missing file mkmf.rb is indeed missing from the JRuby distribution. This is entered as a bug in JIRA 1306 with resolution ‘Won’t fix’. Tough luck.

I understand the reasons and motivation for this decision – the JRuby team decided not to support native extensions in Gems, to keep the platform Java only. I also understand that in this particular case, there are workarounds – using ActiveRecord-DBC gem and JDBC drivers for the database will most likely work. Unfortunately, this decision makes choice of JRuby as platform very questionable.

I really liked JRuby for the comfort of having platform that is portable and safely wrapped within boundaries of the good old trusted JVM. I feel much more comfortable maintaining possibly several different versions of JRuby and corresponding Gems sets than maintaining same several configurations on the OS level and sudo-ing just to install Gems. I was more that happy to trade the lack of speed for this security.

The two main attractions of Ruby (from my point of view) are elegant, powerful language with beautiful syntax as well as sheer amount of code available as Gems to be reused. With the bug 1306, many of this code may not be available for JRuby – unless Gem authors make specific provisions for Java version of the Gem. I cannot think how this is a good idea and certainly not a good news for the future of the language.

One way out is use native Ruby interpreter, of course and make sure you do not mess up your installation by trying out new things. This does not allow the easy way into enterprise that JRuby was promising – by being basically ‘just another jar’ and running on Tomcat.
The other way out is to reconsider the Groovy. I still do not enjoy the syntax anywhere close to Ruby, but every Groovy class is Java class, there is no need for artificial bridges. It has own clone of Rails – GRails – that seems to provide lots of Rails magic and goodies and is based on Spring which I am very familiar and quite happy with. I still do not know whether the amount of “gems” in Groovy world is in the same league as Ruby (which is still limited compared to Python or even CPAN Perl bonanza) – but as long as I can find that what I need, it may be just enough.

For now, I will revert back to Ruby (no time to start learning Groovy+Grails), but I definitely will look at it later on.