Componentix logo

Componentix blog

Here we write some random thoughts and articles about software development: Java, Grails, Node.js, iPhone, iPad and more.

Disappearing apostrophes in localized messages of Grails application

We have witnessed a strange phenomenon recently. Well, when I say ‘strange’ I mean it looked very strange when the mystery wasn’t resolved, but when you come to understand it, it is not strange at all, and has perfect explanation, but first things first.

What happened?

Initially, we had a controller that would send an e-mail message and then inform the user that the message was sent:


class EmailSendingController {

    def sendSomeMail(EmailRequestCommand cmd) {
        // .... checking validity of the command's parameters here
        sendMail {
            to cmd.emailAddress // E-mail address is a request parameter. It is properly validated of course
            // .... doing some e-mail sending work here - standard Mail Plugin stuff
        }

        flash.message = message(code: "success.email.sent.label", default: "E-mail sent")
        redirect(mapping: "somewhere")
    }
}
Read more...

Resolving problem with SSL certificates in OpenJDK

When configuring new server at linode hosting I decided to try to use OpenJDK to run Tomcat. The system I installed was Debian Lenny.

It all went smoothly and I didn’t have any problems, until I tried to deploy application using Facebook API. I immediately have faced a problem when trying to login using Facebook – javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated.

Application was based on Grails and used HTTPBuilder to do requests. It uses Apache HttpClient under the hood.

Googling revealed some workarounds for the problem, but they all required hacking some additional code to override SSL certificates validity checks. The matter was that the same exception appeared when one tried to connect to server with self-signed certificate.

However it wasn’t the case for me as Facebook’s certificate wasn’t self-sgned of course :) So I dug further. It turned out that for some reason OpenJDK in Debian Lenny was installed without any CA (certificate authorities). Also it looked like Debian Squeeze solves this problem.

So the solution for Debian Lenny is simple – download and install appropriate packages from Squeeze . Good news is that certificates come in their own packages:

wget "http://ftp.us.debian.org/debian/pool/main/c/ca-certificates/ca-certificates_20090814+nmu2_all.deb"
sudo dpkg -i ca-certificates_20090814+nmu2_all.deb
wget "http://ftp.us.debian.org/debian/pool/main/c/ca-certificates-java/ca-certificates-java_20100412_all.deb"
sudo dpkg -i ca-certificates-java_20100412_all.deb

ca-certificates is available in Lenny, but it needs to be updated as a dependency to ca-certificates-java

Hosting Grails web applications using Tomcat and Nginx (our configuration)

Intro

To host Grails web application Apache Tomcat is quite a natural choice. However a big part of what is served are just static files (images, CSS, javascript code) or rarely changed pages. Using Tomcat to serve all that stuff would be a waste of resources, especially when running on small VPS like we do.
Nginx is a small efficient web server that helps with this task. It can be setup to both serve static files and act as reverse proxy to Tomcat running the Grails application. Even more, it can cache some of the web pages so that it sends less requests to Tomcat.
In this post I’ll highlight important parts of Nginx and Tomcat configuration files needed to achieve setup like this.

Nginx configuration

For Nginx configuration you can start from its default config file. Let’s add following to http section:

# Enable GZip compression
gzip                on;
gzip_http_version   1.1;
gzip_min_length     1000;
gzip_buffers        16 8k;
gzip_disable        "MSIE [1-6] \.";
gzip_types          text/html text/css text/xml application/x-javascript application/atom+xml text/plain
gzip_vary           on;

# Set proxy cache path
proxy_cache_path /var/nginx/cache keys_zone=one:10m;

# Main website Tomcat instance
upstream main {
    server localhost:8080;
}

This will enable GZip compression, set the path which we want to use for cache and set the Tomcat instance to proxy requests into.

Then it is needed to specify the configuration for the particular server. For this example virtual domain names will be used to define server (as it is most common case).

# iPad Sketchbook website server
server {
    listen       80;
    server_name  www.ipadsketchbook.com;

    # Redirect to appropriate language
    location = / {
        set_from_accept_language $lang en ru;
        rewrite ^/$ /$lang redirect;
    }

    location / {
        # Rewrite the URL for logged-in users
        if ($http_cookie ~* "JSESSIONID=([A-Z0-9]*)") {
            rewrite ^(.*)$ /loggedIn$1 last;
        }

        # Proxy all the requests to Tomcat
        proxy_pass	http://main;
        proxy_set_header  Host $http_host;

        proxy_cache one;
        proxy_cache_min_uses 1;
        proxy_cache_valid  200 302 1m;
        proxy_cache_valid  404 1m;
        proxy_cache_use_stale error timeout invalid_header http_500 http_502 http_503 http_504;
    }

    # Content for logged-in users should not be cached
    location /loggedIn {
        # Rewrite URL back before forwarding request to backend
        rewrite ^/loggedIn(.*)$ $1 break;

        proxy_pass	http://main;
        proxy_set_header  Host $http_host;

        if ($http_cookie !~* "JSESSIONID") {
            return 404;
        }
    }

    # Admin panel should not be cached
    location /admin {
        proxy_pass	http://main/admin;
        proxy_set_header  Host $http_host;
    }

    # Serve static resources
    location ~ ^/(images|css|js|google|yandex|y_key) {
        root /var/www/vhosts/ipadsketchbook.com/httpdocs;
    }

    error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }
}

There are several tricks in the aforementioned config:

  • it redirects to appropriate language version based on request headers. It is only possible if you install nginx_accept_language_module.
  • it rewrites request path for users with existing session (based on JSESSIONID cookie), so that the pages served for them aren’t cached
  • it then rewrites back request path (for logged in users), when issuing request to Tomcat
  • there is a list of path prefixes for which static files are served

Of course most likely there should also be redirect from other domains set up. It is quite easy to do:

# Redirect all the iPhone Sketchbook website domains to main domain
server {
    listen          80;
    server_name     ipadsketchbook.com ipad-sketchbook.com www.ipad-sketchbook.com;
    rewrite ^(.*)   http://www.ipadsketchbook.com$1 permanent;
}

Tomcat configuration

Of course in addition to configuring Nginx, Tomcat itself need to be configured. This is quite straightforward though. Note that only single domain name is configured, as other ones are redirected by Nginx.

So in server.xml we have such host configuration:

<Host name="www.ipadsketchbook.com" appBase="vhosts/ipadsketchbook.com"
    unpackWARs="true" autoDeploy="false"
    xmlValidation="false" xmlNamespaceAware="false">
</Host>

Note that autoDeploy is disabled, it is not needed on production server. appBase specifies the path to directory which contains applications' WAR files. Root of the domain would be served by ROOT.WAR.

Further reading

Choose Grails version and configure GRAILS_HOME automatically, updated

It is quite a pain to set up GRAILS_HOME each time when you use different Grails versions and I already wrote a post about it with very simple bash script to help.

However script was oversimplified and so not working in many corner cases. After my post Yuri has come up with his version of script for Windows, using .BAT file.
Yuri’s version was more complicated and took corner cases into account.

When I have found some time to educate myself about bash, I wrote enhanced version of my script. Its usage follows additional functionality of Windows version.

Its usage is simple, there are such scenarios:

1. Run script in Grails project root directory. In such case it parses application.properties file and launches appropriate version of Grails.

Example: grails run-app

2. Run script anywhere with explicitly specified Grails version. It is needed for some tasks like grails upgrade or grails create-app. This just needs to pass version in parameter like -ver:1.3.1.

Example: grails create-app -ver:1.3.1

As in older version, to use the aforementioned script you need to set GRAILS_PREFIX variable to a value appropriate for your system. And of course your Grails distributions should be installed side-by-side in one common folder.
For example I have it set to /home/vg/tools/grails- on my Linux box.

The script itself:

#!/bin/bash

# Check if GRAILS_PREFIX is set
if [ -z $GRAILS_PREFIX ]; then
    echo "You must define environment variable GRAILS_PREFIX before running Automatic Grails Version Selector"
    exit 1
fi

# Define script params array
declare -a PARAMS

# Process script parameters
for PARAM in $*; do
    if [ ${PARAM:0:5} == "-ver:" ]; then
        # Extract version from -ver parameter
        GRAILS_VERSION=${PARAM:5}
    else
        # Add parameter to array, so that it is passed to original script
        PARAMS=( "${PARAMS[@]}" "$PARAM" )
    fi
done

# If version to use is not specified, try to detect it
if [ -z $GRAILS_VERSION ]; then
    # Check if application.properties file exists
    if [ -e application.properties ]; then
        # Get required Grails version
        # Note that CR characters are removed at first
        GRAILS_VERSION=`tr -d '\015' <  application.properties | sed -n 's/app.grails.version=\(.*\)$/\1/p'`
    else
        echo "Current directory doesn't represent existing Grails project, specify version as -ver:1.3.1"
        exit 1
    fi
fi

# Set Grails home using configured prefix and determined version
export GRAILS_HOME=$GRAILS_PREFIX$GRAILS_VERSION

# Check if GRAILS_HOME directory exists
if [ -d  $GRAILS_HOME ]; then
    # Run original Grails script
    $GRAILS_HOME/bin/grails ${PARAMS[@]}
else
    echo Grails home directory for this project does not exist: $GRAILS_HOME
    echo The current project might have updated to a newer Grails version.
    echo Make sure you have downloaded and installed the version of Grails required: $GRAILS_VERSION
    exit 1
fi

Note that the script doesn’t have the default Grails version for the case when not launched in Grails project root. Adding such configuration parameter is left as an exercise to reader :)

Tags: grails java bash

Execute Groovy script within Grails context, updated for Grails 1.3.1

Quite often it is needed to execute Groovy script within a Grails application context (including access to domain objects, controllers, services, etc). Unfortunately there is no such built-in functionality in Grails.

However Ted Naleid has written run-script Gant script to do this and published on his blog.

We used it in our project and it worked flawlessly. However it stopped working with upgrade to Grails 1.3.1 and even grabbing the latest version from bitbucket haven’t helped.

So I looked into the sources of built-in Grails scripts, including run-test and hacked run-script a little bit to make it run fine.

You can grab/fork the resulting script on github.

Also, please vote for inclusion of run-script functionality into main Grails distribution in JIRA

Using the script is easy. Just store it in src/scripts/ folder of your Grails app with RunScript.groovy filename.

Then you’ll be able to run your scripts as following:

grails run-script path/to/script1/Script1.groovy path/to/script2/Script2.groovy ...

The script itself is:

import org.codehaus.groovy.grails.commons.GrailsClassUtils as GCU
import org.springframework.orm.hibernate3.SessionFactoryUtils
import org.springframework.orm.hibernate3.SessionHolder
import org.springframework.transaction.support.TransactionSynchronizationManager

includeTargets << grailsScript("_GrailsBootstrap")
includeTargets << grailsScript("_GrailsRun")
includeTargets << grailsScript("_GrailsSettings")
includeTargets << grailsScript("_GrailsClean")

target('default': "Execute the specified script after starting up the application environment") {
    depends(checkVersion, configureProxy, packageApp, classpath)
    runScript()
}

target(runScript: "Main implementation that executes the specified script after starting up the application environment") {
    parseArguments()
    if (argsMap["params"].size() == 0) {
        event("StatusError", ["Required script name parameter is missing"])
        System.exit 1
    }
    compile()
    loadApp()
    configureApp()
    configureHibernateSession()
    argsMap["params"].each { scriptFile ->
        executeScript(scriptFile, classLoader)
    }
}

def configureHibernateSession() {
    // without this you'll get a lazy initialization exception when using a many-to-many relationship
    def sessionFactory = appCtx.getBean("sessionFactory")
    def session = SessionFactoryUtils.getSession(sessionFactory, true)
    TransactionSynchronizationManager.bindResource(sessionFactory, new SessionHolder(session))
}

def executeScript(scriptFile, classLoader) {
    File script = new File(scriptFile)
    if (script.exists()) {
        def shell = new GroovyShell(classLoader, new Binding(ctx: appCtx, grailsApplication: grailsApp))
        shell.evaluate(script.text)
    } else {
        event("StatusError", ["Designated script doesn't exist: $scriptFile"])
    }
}
 

Mocking configuration in Grails unit tests

While testing some of the classes in Grails app, I had problem that the tests failed because Grails config (ConfigurationHolder.config) isn’t populated (is null) when unit tests are executed. Some Googling found me issue in JIRA which explained it.

Graeme Rocher commented that there is mockConfig method which allows to specify mocked config to be used for tests. It can be called in setUp method to do necessary initialization.

It is convenient to use it with multi-line strings. Its usage goes as following:

mockConfig '''
    foo.bar = "good"
'''

It is discussed in more details in mrhaki's post.

Resolving Grails version automatically on Windows

A while ago we showed how it was possible to select automatically version of Grails to run just using a simple bash script. Trouble is, not all users use systems that recognize bash-scripts natively. One of such systems is, apparently, Windows platform. So today I’m going to show how it is possible to accomplish similar tasks on Windows using just the built-in functionality — the batch job processing files.

The bash script we showed earlier, used the knowledge embedded in the Grails-project file called application.properties to find out the version of Grails for the current project. Weirdly enough, batch files allow to do this as well (for the syntax used in properties file at least). Here is the snippet of code that does just that:

FOR /F "eol=# tokens=1,2 delims==" %%i IN (application.properties) DO (
    IF "%%i" == "app.grails.version" (
        SET GRAILS_VERSION=%%j
    )
)

SET GRAILS_HOME=%GRAILS_PREFIX%%GRAILS_VERSION%
SET PATH=%GRAILS_HOME%\bin
CALL %GRAILS_HOME%\bin\grails.bat %*
Read more...

Choose Grails version and configure GRAILS_HOME automatically

We are using Grails actively to develop web applications. So we ended up with different applications using different versions of framework.

Setting up GRAILS_HOME variable to point to different Grails release each time I switched to other application was really unpleasant hassle. So I thought about how to resolve this problem. Grails stores application version in application.properties file, I came out to the following Bash script:

#!/bin/bash

# Get required Grails version
GRAILS_VERSION=`sed -n 's/app.grails.version=\(.*\)$/\1/p' <  application.properties` 

# Set Grails home using configured prefix and determined version
export GRAILS_HOME=$GRAILS_PREFIX$GRAILS_VERSION

# Run original Grails script
$GRAILS_HOME/bin/grails $@

It just parses the version from the application.properties file and appends it to GRAILS_PREFIX environment variable. The resulting value is exported into GRAILS_HOME variable.

To use the aforementioned script you need to set GRAILS_PREFIX variable to a value appropriate for your system. And of course your Grails distributions should be installed side-by-side in one common folder.
For example I have it set to /home/vg/tools/grails- on my Linux box.

You can grab/fork script easily as a Gist on GitHub.

UPD: The script was updated to take some corner cases into account. See updated version of the script.

UPD 2: There is also version for Windows.

Tags: grails java bash

Run long batch processing jobs in Grails without memory leaks

In one of our Grails applications we had to run a number of batch jobs. Nothing unusual and Grails supports it quite well with the excellent Quartz plugin.

But when we deployed application in production, we noticed that after running for some time, it consumed a lot of memory and JVM was spending all the time running garbage collection. The reason for it was that our jobs were quite long-running, taking several hours to complete, and Grails wasn’t really designed for such kind of use case.

First problem is actually well-known and documented — using single Hibernate session for a long time gets a lot of objects being added to cache. It is described for example in this blog post.
This basically goes into flushing and clearing Hibernate session like following:

sessionFactory.currentSession.flush() 
sessionFactory.currentSession.clear()

But while less memory was being leaked after that, the problem still wasn’t resolved. However thankfully to this nice post about Grails memory leak I learned about another more obscure problem and got it fixed.

Main idea is that save() method stores validation errors info in thread-local storage when HttpServletRequest instance is not available. So this info needs to be cleared from time to time, which can be achieved by something like this:

DomainClassGrailsPlugin.PROPERTY_INSTANCE_MAP.get().clear()

Resolving these two issues solved our problem with memory usage. Hopefully this post prevents you from spending much time resolving the same issues.

Nimble – easy user profiles and security for Grails

Recently found a really nice Grails plugin – Nimble.

It provides a complete solution for user profiles management, flexible authentication (both local and using OpenID) and fine-grained access control. It is based on Apache Shiro, which we used previously for authentication/access control.

I will try to use it in some simple project and then write a blog post describing my experience.

Offtopic: This is verification code for Technorati – QAEYYR9THPUN

Using cryptographically strong random number generator with SecureRandom in Java

There might be a need occasionally to generate sequences of random numbers in your real-world programs. While there is a special class in Java to deal just with that — java.util.Random — it’s not cryptographically strong, and the numbers chosen are not completely random because a definite mathematical algorithm (based on Donald E. Knuth’s subtractive random number generator algorithm) is used to select them. Therefore it is not safe to use this class for tasks that require high level of security, like creating a random password, for example.

Fortunately, there’s another, much more cryptographically strong random number generator provided with every Java Runtime Environment by default. It can be accessed via the java.security.SecureRandom class, which is a subclass of class Random mentioned above. That means that you can use it the same way you did when you used the generator implemented by the Random class, it even allows you to set the random seed of your choice if it happens so that you need to repeat the sequence of numbers generated, which is good as for example the .NET equivalent — System.Security.Cryptography.RNGCryptoServiceProvider — does not allow to do that. However, there is one or two issues that, if not addressed, might turn into real problems and cause lots of headaches. But before I describe those, let me talk you into how to start using this strong random number generator.

Read more...

Improved Hibernate dialect for Microsoft SQL Server

In one of our Grails projects we had to use Microsoft SQL Server as a database. Hibernate has support for it and works good enough, however the schema generated by it is not ideal, at least from our point of view. In particular, properties of boolean type were stored in columns of type INT, while both SQL Server 2000 and 2005 support special type just for booleans — BIT; all Ids, which usually have type long, had to be squeezed into columns of type INT (again this stupid integer!); such SQL types as TEXT and IMAGE were used, instead of (preferred for SQL Server) VARCHAR(max) and VARBINARY(max) respectively; and finally (for me — the most annoying thing) strings were stored in non-Unicode sequences. The very last point is also very unhandy, since strings in Java are stored in Unicode, and having to deal with encodings explicitly when storing or reading strings or entire texts from the database is a bit against the simplicity we expect from Hibernate when we use it.

So I decided to develop customized Hibernate dialect for MS SQL Server. To do this, I started digging the source code of Hibernate (the dialect classes to be precise), and soon I found out that Hibernate’s SQLServerDialect extends from SybaseDialect, which is responsible for most of the “sins” outlined in the paragraph above. Don’t know why they haven’t overridden any of that in a subclass, probably they had a good reason not to do so, but for our company this reasoning was certainly not working out. Just extending the standard implementation was fairly enough to solve the problem, actually it was sufficient just to register appropriate column types in the constructor. The end result looks like this:


public class SQLServerDialect extends org.hibernate.dialect.SQLServerDialect {
 
   /**
    * Initializes a new instance of the {@link SQLServerDialect} class.
    */
    public SQLServerDialect() {
        registerColumnType(Types.BIGINT, "bigint");
        registerColumnType(Types.BIT, "bit");
        registerColumnType(Types.CHAR, "nchar(1)");
        registerColumnType(Types.VARCHAR, 4000, "nvarchar($l)");
        registerColumnType(Types.VARCHAR, "nvarchar(max)");
        registerColumnType(Types.VARBINARY, 4000, "varbinary($1)");
        registerColumnType(Types.VARBINARY, "varbinary(max)");
        registerColumnType(Types.BLOB, "varbinary(max)");
        registerColumnType(Types.CLOB, "nvarchar(max)");
    }
}

Obviously, similar technique can be used with any custom Hibernate dialect, not just for MS SQL. Feel free to use it with any other DB engine, etc. If you find it useful, you can fork our code snippet at Github: http://gist.github.com/225543

Interesting error in Grails with multiple-classes in the same source file

Recently we have got strange error in one of our projects when we upgraded it from Grails 1.2 M2 to Grails 1.2 M4.
After we did the upgrade we started getting java.lang.MissingMethodException for the domain class method calls.

Then it was determined by playing with grails console that the problem was limited to a single domain class. The source code file for it (say, MyDomainClass.groovy) looked like this:


class MyUtilityClass {
      // Some code here
}

class MyDomainClass {
      // Some code here
}

Well, it may be not the best idea to have two classes in the same source file, but that is a different question. Interesting thing is that it for some reason prevented Grails from adding the methods to MyDomainClass. And to resolve this issue very simple thing can be done — MyUtilityClass can be put in source file after MyDomainClass. That resolved problem for us.

Hopefully this post would be useful to someone with similar problem.

Twitter and Google Maps mashup in 20 minutes with Grails

Introduction

For many developers Java is often a synonym for totally non-sexy enterprise applications development. It is associated with numerous XML configuration files, boilerplate code, etc. So they instead use dynamic languages like Ruby, Python, PHP to develop their projects, especially when these are own simple utilities, mash-ups, etc.

However the Java field has changed much in the recent few years. There are multiple frameworks which relieve developer from “enterprise” burden. Grails is probably one of the best. It is based on Groovy, which is a dynamic language running on Java platform designed specially for Java programmers. It uses well known robust and efficient Java libraries to do all the heavy lifting (Spring, Hibernate, etc.). There is also a plugin system and plugins exist for almost every widely used Java library.

In this article we’ll show how to make a mash-up of Twitter and Google Maps in around 20 minutes. The end result will look similar to this:

Read more...

In the beginning of the long road

Finally we’ve launched our own blog. We’ll try to write interesting things about Java, Grails, iPhone or about anything else related to our work.

BTW, this blog is powered by Grails, same as other parts of our website.

Following e-mail is only for robots (never send anything to it, or you would be blacklisted): botsonly@componentix.com