A very light Groovy based web application project template

You might have heard of the project Grails is a Groovy version of Ruby on Rails like framework that let you create web application much more easier with Dynamic scripting. Despite all that power Grails provided, it is not "light" if you look under the hood. I am not saying Grails is bad or anything. Grails is actually pretty cool to write web application with. However I found myself often want something even lighter and yet still want to prototype with Groovy. So here I will show you a maven-groovy-webapp project template that I use to get start any web application development. It's very simple, light, and yet very Groovy.

How to get started

Unzip maven-webapp-groovy.zip above and you should see these few files:

bash> cd maven-webapp-groovy
bash> find .
bash> ./pom.xml
bash> ./README.txt
bash> ./src
bash> ./src/main
bash> ./src/main/java
bash> ./src/main/java/deng
bash> ./src/main/java/deng/GroovyContextListener.java
bash> ./src/main/resources
bash> ./src/main/resources/log4j.properties
bash> ./src/main/webapp
bash> ./src/main/webapp/console.gt
bash> ./src/main/webapp/health.gt
bash> ./src/main/webapp/home.gt
bash> ./src/main/webapp/WEB-INF
bash> ./src/main/webapp/WEB-INF/classes
bash> ./src/main/webapp/WEB-INF/classes/.keep
bash> ./src/main/webapp/WEB-INF/groovy
bash> ./src/main/webapp/WEB-INF/groovy/console.groovy
bash> ./src/main/webapp/WEB-INF/groovy/health.groovy
bash> ./src/main/webapp/WEB-INF/groovy/home.groovy
bash> ./src/main/webapp/WEB-INF/groovy/init.groovy
bash> ./src/main/webapp/WEB-INF/groovy/destroy.groovy
bash> ./src/main/webapp/WEB-INF/web.xml

As you can see it's a maven based application, and I have configured tomcat plugin, so you may run it like this:

bash> mvn tomcat7:run
bash> open http://localhost:8080/maven-webapp-groovy/home.groovy

And ofcourse, with maven, running package phase will let you deploy it into any real application servers when ready.

bash> mvn package
bash> cp target/maven-webapp-groovy.war $APP_SERVER_HOME/autodeploy

What's in it

You should checkout the main config in web.xml file, and you'll see that there couple built-in Groovy servlets and a custom listener.

<?xml version="1.0"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
        version="2.5">

    <description>Groovy Web Application</description>
    <welcome-file-list>
        <welcome-file>home.groovy</welcome-file>
    </welcome-file-list>

    <servlet>
        <servlet-name>GroovyServlet</servlet-name>
        <servlet-class>groovy.servlet.GroovyServlet</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>GroovyServlet</servlet-name>
        <url-pattern>*.groovy</url-pattern>
    </servlet-mapping>

    <servlet>
        <servlet-name>TemplateServlet</servlet-name>
        <servlet-class>groovy.servlet.TemplateServlet</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>TemplateServlet</servlet-name>
        <url-pattern>*.gt</url-pattern>
    </servlet-mapping>

    <listener>
        <listener-class>deng.GroovyContextListener</listener-class>
    </listener>
    <context-param>  
       <param-name>initScripts</param-name>
       <param-value>/WEB-INF/groovy/init.groovy</param-value>
    </context-param>
    <context-param>    
       <param-name>destroyScripts</param-name>
       <param-value>/WEB-INF/groovy/destroy.groovy</param-value>
    </context-param>

</web-app>

I've chosen to use GroovyServlet as a controller (it comes with Groovy!), and this let you use any scripts inside /WEB-INF/groovy directory. That's it, no further setup. That's about the only requirement you need to get a Groovy webapp started! See console.groovy as example and how it works. It's a groovy version of this JVM console

Now you can use Groovy to process any logic and even generate the HTML output if you like, but I find it even more easier to use TemplateServlet. This allow Groovy template files to be serve as view. It's very much like JSP, but it uses Groovy instead! And we know Groovy syntax are much shorter to write! See console.gt as exmaple and how it works.

The GroovyContextListener is something I wrote, and it's optional. This allow you to run any scripts during the webapp startup or shutdown states. I've created an empty init.groovy and destroy.groovy placeholder. So now you have all the hooks you need to prototype just about any web application you need.

Simplicity wins

This setup is just plain Java Servlet with Groovy loaded. I often think the more simple you get, then less bug and faster you code. No heavy frameworks, no extra learning curve, (other than basic Servlet API and Groovy/Java skills ofcourse), and off you go.

Go have fun with this Groovy webapp template! And let me know if you have some cool prototypes to show off after playing with this. :)

Use cygpath with -p option for your Java CLASSPATH conversion

I just noticed that Cygwin's cygpath command supports -p option. This is a real gem when writing Java wrapper script that needs to covert CLASSPATH values. A simple script can demonstrate the purpose.

#!/usr/bin/env bash
# Author: Zemian Deng, Date: 2012-11-16_06:30:00
#
# run.sh - A simple Java wrapper script for Cygwin and Unix/Linux shell. We assume 
# this script is located in a subdiretory inside the application home directory.
# Example:
#   app/bin/run.sh
#   app/config/log4j.properties
#   app/lib/log4j.jar
# Usage:
#   bash> run.sh app.Hello
#
DIR=$(cd "$(dirname $0)/.." && pwd)
CP=${CP:="$DIR/config:$DIR/lib/*"}
if [[ "$OS" == Windows* ]]; then
 CP=$(cygpath -mp $CP)
fi
java -cp "$CP" "$@"

What does UTF-8 with BOM mean?

Believe it or not, There is no such thing as Plain Text!

All files in a modern Operating Sytems (Windows, Linux, or MacOSX) are saved with an encoding scheme! They are encoded (a table mapping of what each byte means) in such way so that other programs can read it back and understand how to get information out. It happens that US/ASCII encoding is earliest and widely used that people think it's just "Plain Tex". But even ASCII is an encoding! It uses 7 bits in mapping all US characters in saving the bytes into file. Obviously you are free to use any kind of encoding (mapping) scheme to save any files, but if you want other programs to read it back easily, then sticking to some standard ones would help a lot. Without an agreed upon encoding, programs will not able to read files and be any useful!

The most useful and practical file encoding today is "UTF-8" because it support Unicode, and it's widely used in internet.

I discovered something odd when using Eclipse and Notepadd++. In Ecilpse, if we set default encoding with UTF-8, it would use normal UTF-8 without the Byte Order Mark (BOM). But in Notepad++, it appears to support UTF-8 wihtout BOM, but it won't recoginze it when first open. You can check this by going Menu > Encoding and see which one is selected. Notepad++ seems to only recognize UTF-8 wihtout BOM with ones it converted by it's own conversion utility. Perhaps it's a bug in notepad++.

So what is BOM? The byte order mark is useless for UTF-8. They only used for UTF-16 so they know which byte order is first. But UTF-8 will allow you to save these BOM for conversion purpose... they are ineffective in encoding the doc itself. So a "normal" UTF-8, it won't have BOM, but Windows would like to use them anyway. The Windows NOTEPAD would automatically save BOM in UTF-8!

So be-aware when viewing UTF-8 without BOM encoding files in Notepad++, as it can be deceiving at first glance.

Ref: 

http://en.wikipedia.org/wiki/UTF-8 

http://www.joelonsoftware.com/articles/Unicode.html

A simple way to setup Java application with external configuration file

Many Java applications would deploy and run it with some kind of external configuration files. It's very typical that you would want a set of config files per environments such as DEV, QA and PROD. There many options in tackling this problem, especially in a Java app, but keeping it simple and easy to maintain would take some disciplines.
Here I would layout a simple way you may use to depploy a typical Java application. The concept is simple and you can easily apply to a standalone, web, or even a JEE application.

Use a env System Properties per environment

Java allows you to invoke any program with extra System Properties added. When launching an Java application, you should set an env property. For example:

bash> java -Denv=prod myapp.Main

This property would give your Main application to identify which environment you are running against with. You should be reading it like this inside your code:

String env = System.getProperty("env", "dev");

This way you always would have an environment to work with. Even if user doesn't supply one, it will default to use dev env.
Another useful System Property to set is app.home. You want to set this value in relative to where your application is deployed, so you may reference any files (eg: data) easily. To do this, you can use a script wrapper to automatically calculate the path. See section below for an example.

Use a config dir prefix in CLASSPATH

In stead of passing a explicit config file to your application as argument, another flexible way to load configuration file is to add an extra config folder into your CLASSPATH. For example, you can easily create a startup wrapper script myapp.sh like this:

#/usr/bin/env bash
APP_HOME=$(cd "`dirname $0`/.." && pwd)
java $JAVA_OPTS -Dapp.home=$APP_HOME -cp "$APP_HOME/config:$APP_HOME/lib/*" myapp.Main "$@"

From this, you can setup the application packaging layout this way:

myapp
    +- bin
        +- myapp.sh
    +- config
        +- dev.properties
        +- qa.properties
        +- prod.properties
    +- data
        +- myrecords.data
    +- lib
        +- myapp-1.0.0.jar
        +- slf4j-1.7.1.jar

You would typically invoke the application like this:

bash> JAVA_OPTS='-Denv=prod' myapp/bin/myapp.sh

The above will give you a good foundation to load a single config properties file per env. For example, you can read your properites file like this somewhere in your code.

// Get appHome and data dir.
String appHome = System.getProperty("app.home");
String dataDir = appHome + "/data";

// Get env value to load config parameters
String env = System.getProperty("env", "dev");
String config = env + ".properites";
Properties configProps = new Properties();
InputStream inStream = null;
try {
    inStream = getClass().getClassLoader().getResourceAsStream(config);
    configProps.load(inStream);
} finally {
    if (inStream != null)
        inStream.close();
}
// Now load any config parameters from configProps map.

Now you would have the configProps object at your disposal to read any configuration keys and values set per an environment.
NOTE: If you want a more flexible Java wrapper script, see my old post on run-java wrapper.

Do not abuse CLASSPATH

Now, just because you have setup config as part of your CLASSPATH entry, I have to caution you not to abuse it. What I mean is do not go wild on loading all your application resources in that folder! If you have that many resources that user MUST edit and configure, then you should re-think about your application design! Simple interface, or configuration in this case, is always a win. Do not bother users with complexity just because your application can support gazillion ways of configuration combination. If you can keep it as one config file, it would make users very happy.
Also, this doesn't mean you have to put the entire world inside one of prod.properites either. In the real world, an application is likely going to have only handful of user tunable parameters, and many other resources are less frequent used. I would recommand put the most frequently used parameters in these config properties only. For all other (for example most of the Spring context files in an application do not belong to a typical users config level, they are more developer level config files. In another word, changing these files would have catastrophic effect to your application!) You should put these inside as part of your myapp.jar.
You might ask, 'Oh, but what happen if I must want to override one of the resource in the jar?'. But in that very unusual case, you would still have an option to override! You have the config as prefix in CLASSPATH, remember? Even when you nested resources inside a package inside the jar, you would still able to overwrite by simply create same directory structure inside config. You probably only do this for emmergency and less frequent use anyway.

Feedback

So what are some clever ways you have seen or done with your application configuration? I hope to hear from you and share.

Simple Variable Substitution in Java String

When I wrote about how to improve the Java Properties class using Props, I've shown a feature where you can use variable substition such as mypath=${user.home} in your config file. The implementation underneath it uses the Apache Common Lang library with org.apache.commons.lang.text.StrSubstitutor. There is nothing wrong with this, but I was curious how bad would it be to remove such dependency, so the Props can be more standalone.

Here is a quick implementation in Groovy, but you should able to translate to Java easily.

// String variable substitutions
def parseVariableNames(String text) {
    def names = []
    def pos = 0, max = text.length()
    while (pos < max) {
        pos = text.indexOf('${', pos)
        if (pos == -1)
            break
        def end = text.indexOf('}', pos + 2)
        if (end == -1)
            break
        def name = text.substring(pos + 2, end)
        names.add(name)
        pos = end + 1
    }
    return names
}
def replaceVariable(String key, String value, String text) {
    //println "DEBUG: Replacing '${key}'' with '${value}'"
    result = text.replaceAll('\\$\\{' + key + '}', value)
    return result
}

Probably not the most efficient thing, but it should work. Let's have some tests.

// Test
def map = ["name": "Zemian", "id": "1001"]
def inputs  = [
    'Hello ${name}',
    'My id is ${id}',
    '${name} is a good programmer.',
    '${name}\'s id is ${id}.'
]

result = inputs.collect{ line ->
    def names = parseVariableNames(line)
    names.each{ key ->
        line = replaceVariable(key, map.get(key), line) 
    }
    line
}
assert result == [
    'Hello Zemian',
    'My id is 1001',
    'Zemian is a good programmer.',
    'Zemian\'s id is 1001.'
]

The output should print nothing, as it passed the test. What do you think?

Exploring different scheduling types with Quartz 2

We often think of Cron when we want to schedule a job. Cron is very flexible in expressing an repeating occurance of an event/job in a very compact expression. However it's not answer for everything, as I often see people are asking for help in the Quartz user forum. Did you know that the popular Quartz 2 library provide many other schedule types (called Trigger) besides cron? I will show you each of the Quartz 2 built-in schedule types here within a complete, standalone Groovy script that you can run and test it out. Let's start with a simple one.

@Grab('org.quartz-scheduler:quartz:2.1.6')
@Grab('org.slf4j:slf4j-simple:1.7.1')
import org.quartz.*
import org.quartz.impl.*
import org.quartz.jobs.*

import static org.quartz.DateBuilder.*
import static org.quartz.JobBuilder.*
import static org.quartz.TriggerBuilder.*
import static org.quartz.SimpleScheduleBuilder.*

def trigger = newTrigger()
    .withSchedule(
        simpleSchedule()
        .withIntervalInSeconds(3)
        .repeatForever())
    .startNow()
    .build()
dates = TriggerUtils.computeFireTimes(trigger, null, 20)
dates.each{ println it }

This is the Quartz's SimpleTrigger, and it allows you to create a fixed rate repeating job. You can even limit to certain number of count if you like. I have imported all the nessary classes the script needs, and I use the latest Quartz 2.x builder API to create an instance of the trigger.

The quickest way to explore and test out whether a scheduling fits your need is to print out its future execution times. Hence you see me using TriggerUtils.computeFireTimes in the script. Run the above and you should get the datetimes as scheduled to be run, in this case every 3 seconds.

bash> $ groovy simpleTrigger.groovy
    Tue Oct 23 20:28:01 EDT 2012
    Tue Oct 23 20:28:04 EDT 2012
    Tue Oct 23 20:28:07 EDT 2012
    Tue Oct 23 20:28:10 EDT 2012
    Tue Oct 23 20:28:13 EDT 2012
    Tue Oct 23 20:28:16 EDT 2012
    Tue Oct 23 20:28:19 EDT 2012
    Tue Oct 23 20:28:22 EDT 2012
    Tue Oct 23 20:28:25 EDT 2012
    Tue Oct 23 20:28:28 EDT 2012
    Tue Oct 23 20:28:31 EDT 2012
    Tue Oct 23 20:28:34 EDT 2012
    Tue Oct 23 20:28:37 EDT 2012
    Tue Oct 23 20:28:40 EDT 2012
    Tue Oct 23 20:28:43 EDT 2012
    Tue Oct 23 20:28:46 EDT 2012
    Tue Oct 23 20:28:49 EDT 2012
    Tue Oct 23 20:28:52 EDT 2012
    Tue Oct 23 20:28:55 EDT 2012
    Tue Oct 23 20:28:58 EDT 2012

The most frequent used scheduling type is the CronTrigger, and you can test it out in similar way.

@Grab('org.quartz-scheduler:quartz:2.1.6')
@Grab('org.slf4j:slf4j-simple:1.7.1')
import org.quartz.*
import org.quartz.impl.*
import org.quartz.jobs.*

import static org.quartz.DateBuilder.*
import static org.quartz.JobBuilder.*
import static org.quartz.TriggerBuilder.*
import static org.quartz.CronScheduleBuilder.*

def trigger = newTrigger()
    .withSchedule(cronSchedule("0 30 08 * * ?"))
    .startNow()
    .build()
dates = TriggerUtils.computeFireTimes(trigger, null, 20)
dates.each{ println it }

The javadoc for CronExpression is very good and you should definately read it throughly to use it effectively. With the script, you can explore all the combination you want easily and verify future fire times before your job is invoked.

Now, if you have some odd scheduling needs such as run a job every 30 mins from MON to FRI and only between 8:00AM to 10:00AM, then don't try to cramp all that into the Cron expression. The Quartz 2.x has a dedicated trigger type just for this use, and it's called DailyTimeIntervalTrigger! Check this out:

@Grab('org.quartz-scheduler:quartz:2.1.6')
@Grab('org.slf4j:slf4j-simple:1.7.1')
import org.quartz.*
import org.quartz.impl.*
import org.quartz.jobs.*

import static org.quartz.DateBuilder.*
import static org.quartz.JobBuilder.*
import static org.quartz.TriggerBuilder.*
import static org.quartz.DailyTimeIntervalScheduleBuilder.*
import static java.util.Calendar.*

def trigger = newTrigger()
    .withSchedule(
        dailyTimeIntervalSchedule()
        .startingDailyAt(TimeOfDay.hourMinuteAndSecondOfDay(8, 0, 0))
        .endingDailyAt(TimeOfDay.hourMinuteAndSecondOfDay(10, 0, 0))
        .onDaysOfTheWeek(MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY)
        .withInterval(10, IntervalUnit.MINUTE))
    .startNow()
    .build()
dates = TriggerUtils.computeFireTimes(trigger, null, 20)
dates.each{ println it }

Another hidden Trigger type from Quartz is CalendarIntervalTrigger, and you would use this if you need to repeat job that's in every interval of a calendar period, such as every year or month etc, where the interval is not fixed, but calendar specific. Here is a test script for that.

@Grab('org.quartz-scheduler:quartz:2.1.6')
@Grab('org.slf4j:slf4j-simple:1.7.1')
import org.quartz.*
import org.quartz.impl.*
import org.quartz.jobs.*

import static org.quartz.DateBuilder.*
import static org.quartz.JobBuilder.*
import static org.quartz.TriggerBuilder.*
import static org.quartz.CalendarIntervalScheduleBuilder.*
import static java.util.Calendar.*

def trigger = newTrigger()
    .withSchedule(
        calendarIntervalSchedule()
        .withInterval(2, IntervalUnit.MONTH))
    .startAt(futureDate(10, IntervalUnit.MINUTE))
    .build()
dates = TriggerUtils.computeFireTimes(trigger, null, 20)
dates.each{ println it }

I hope these will help you get started on most of your scheduling need with Quartz 2. Try these out and see your future fire times before even scheduling a job into the scheduler should save you some times and troubles.

Running maven commands with multi modules project

Have you ever tried running Maven commands inside a sub-module of a multi modules Maven project, and get Could not resolve dependencies for project error msg? And you checked the dependencies that it's missing are those sister modules within the same project! So what gives? It turns out you have to give few more options to get this running correctly, and you have to remember always stays in the parent pom directory to run it!

For exmaple if you checkout the TimeMachine scheduler project, you can invoke the timemachine-hibernate module with maven commands like this:

bash> hg clone https://bitbucket.org/timemachine/scheduler
bash> cd scheduler
bash> mvn -pl timemachine-hibernate -am clean test-compile

You can start the scheduler using Maven like this (remember to stay in the parent pom directory!):

bash > mvn -pl timemachine-scheduler exec:java -Dexec.mainClass=timemachine.scheduler.tool.SchedulerServer -Dexec.classpathScope=test

I have added the -Dexec.classpathScope=test so you will see logging output, because there is an log4j.properties in the classpath for testing.

Without these, you can always run mvn install in the project root directory, then you can cd into any sub-module and run Maven commands. However you will have to keep a tab on what changed in the dependencies, even if they are in sister modules.

You can read more from this article from Sonatype.