Saturday, December 13, 2008

Castles In The Air

[Copied from JRoller blog]

When I first read the abstract for Dreaming In Code by Scott Rosenberg, I figured it to be light reading to give my mind a break from other, more technical books on my nightstand.  Little did I realize that once I read past chapter 8, this little book would spark so many questions that I would have sleepless nights wondering "will we ever learn how to create software quickly and efficiently?"

In The Mythical Man-Month, Frederick P. Brooks, Jr. eloquently prosed "The programmer, like the poet, works only slightly removed from pure thought-stuff.  He builds his castles in the air, from air, creating by exertion of the imagination."  Henry David Thoreau is often quoted, "Do not worry if you have built your castles in the air.  They are where they should be.  Now put the foundations under them."

As programmers, we conceive great abstract structures in our imaginations, but are also charged with building the foundations from which these abstractions come into fruition.  Programmers are more like artisans than poets.  The fruits of an artisan's labors will only be enjoyed by others if she limits her abstract scaffolding to only build abstractions which may be transformed into concrete shape with the tools of the day.  The poet is freed from the shackles and limitations of an abstract scaffolding, she is free to create whatever comes to her imagination.

Here en lies the dilemma.  Some of us software-folk are poets, some are master artisans and some are artisans.  The poets create the castles in the air, oblivious of any foundations under them.  They can envision software-thoughts which are unattainable with today's tools, castles which remain suspended in space and time until such time foundations can be created to support these castles.  Master artisans are skilled in creating the artisan's tools, progressing the tools ahead, empowering the artisan to be more efficient and to one day create the poet's castle.  The artisan is a master of the tools and understands how to use and manipulate them to create achievable abstractions, at times using the tools in new and creative ways to make previously unattainable structures attainable.  Business managers and project managers are taught to make S.M.A.R.T. goals to promote success.  The artisan is more likely to practice building S.M.A.R.T. goals in her craft work, where the poet wishes to be removed from such constraints.

As for the Chandler project, it is but one of many projects where poets and artisans, practitioners of free will to create great things, suffered from thoughts and ideas which were to far ahead of the capabilities of existing tools.  There is a fine line poets and artisans must walk in order to create that which may be enjoyed and scrutinized by others.  Will we ever learn how to create software quickly and efficiently?  Sure.  However, with regards to enterprise and commercial software, I respectfully disagree with Frederick Brooks; programmers should not be creating castles in the air.  This type of creativity should be left to the poet, not the artisan.

Tuesday, February 19, 2008

Groovy Performance Woes?

[Copied from JRoller blog]

These days, everyone's got something to say about Groovy. Whether it's "the good, the bad or the ugly," Groovy is making great leaps with all its deserved, or undeserved, publicity. With every tool available to the software developer, there are pros and cons; scenarios in which the tool works miracles or reaps havoc. The main determining factor of success or failure is attitude.

Attitude! "An organismic state of readiness to respond in a characteristic way to a stimulus." ( It is that which makes or breaks the constructs of our lives. "Everything can be taken from a man but ... the last of the human freedoms - to choose one's attitude in any given set of circumstances, to choose one's own way." (Viktor Emil Frankl) "Happiness is an attitude. We either make ourselves miserable, or happy and strong. The amount of work is the same." (Francesca Reigler) My beautiful wife brought to my attention a website which reminds me time and again just how important attitude is in our everyday life. Attitude is the difference between happiness and misery.

This mindset applies just as whole-heartedly to software development. We can make the best of and improve on the tools we have, or dwell on their problems; "the amount of work is the same." Groovy, as with all tools, has its pros and cons. But I won't go into its cons; there's enough of that out there. I'm here to tell you how Groovy has proven to be a tool capable of great strides. Groovy has provided me with the ability to create a wonderful database migration tool. It has endowed me with a level of flexibility capable of migrating data from legacy mainframe and Windows sources containing some more than 100 million records to a newly created database. Migrating one set of records includes some 100+ SQL statements to read, insert and update records (and that's with query resultset caching enabled for static data). Granted, Groovy isn't doing all of the legwork (Spring Framework, Hibernate and Drools play a big role as well), but it is the glue which ties everything together.

This all happens in a matter of seconds. In three seconds, some 200+ Groovy scripts are executed as data transformation tasks. The first time scripts are encountered, they are dynamically compiled to class files using Groovy's ClassLoader and CompilationUnit. Subsequent script executions do not require compilation, even across JVM instantiations. Roughly 5% of the application time is spent in Java and Groovy code. Most of the rest of the time is spent waiting on I/O requests. That works out to about 150 milliseconds in Java and Groovy code, per set of records being migrated, to massage the data. Considering the volume of data and number of business rules being applied, this is quite an achievement.

I hope to have some time in the near future to put together a small test application demonstrating the use of GroovyClassLoader and CompilationUnit to compare a Groovy application's performance with and without the compilation step. Depending on your requirements, using Groovy as glue code could be just as performant as using Java. Groovy can help reduce a project's total man-hours, decrease other resource consumption, increase productivity and provide flexibility to change in key areas. That is, if you have the right attitude.