Sunday, September 30, 2012

High Level Design Diagrams for TimeMachine Scheduler

Despite the advance of all the computers, I am one of those guys who still carry a pen and composition notebook around. I am far from those who has photographic memories, nor one who can digest every thing in their heads. Especially when it comes to doing software design. I still find my self scribbling on my notebook quite a bit, or even on piece of napkin if no paper is available. I am also one of those who only need high level ideas drawn out, then I can usually go directly to code, map it to some API and then write some simple implementations to get started.

That's how I started with the TimeMachine Scheduler project a while back. However, as we come to a point where we have a solid runnable code, we must also start to present it to others. Showing my composition notebook is not the prettier thing in the world, specially when I have no gift in penmenship at all. This is a time to look into some professional diagram and design tools to draw up the high level architecture.

I was contacted by Architexa to give them some review of their tool. I think this is a great opportunity for me to try it to draw up some diagrams on the TimeMachine project. Most of what I have diagrammed here are explained in details in the project reference manual, so if you want to know more, please do read it up for further reference.

TimeMachine top level packages

First, the Architexa provides their software as an Eclipse plugin, and after installing it, it can analysis an existing project. I did that and here is the package level view it presents me for the TimeMachine.

TimeMachine packages dependencies diagram

The graph is not very clear on why everything depends on schedule package, but perhaps Schedule is used through the code. Any way the TimeMachine has been carefully separating out the pakcages so it's easy for user to use. All the interfaces you need to scheduler jobs are in timemachine.scheduler. In it, there are couple static factories class: Schedules and JobTasks provide majority of the functionity you need. So the goal is you can do quiet a bit with just import timemachine.scheduler.*;; then as you need each layer functionality, you can import them explicitly the sub-packages.

TimeMachine main classes diagrams

In TimeMachine, you have few major class hierarchy that you must need to get familiar in order to write effective scheduling jobs.

A job in TimeMachine is implemented by a class with JobTask interface. The project provide few for you to get started.

TimeMachine JobTask class diagram

How often and frequent to run your job in the scheduler is provided by a Schedule. Here we provided some most used implementations, including the customizable DateListSchedule that uses a provider generator.

TimeMachine Schedule class diagram

The TimeMachine is a scheduler composed by a stackable service container. Here are some of the services that power the scheduler as a whole. The user/developer can write their own user Service and add into the container as well.

TimeMachine Service class diagram

The services above are then combined together to form the scheduler engine.

TimeMachine SchedulerEngine class diagram

TimeMachine in action diagrams

Depicting sequence diagram is pretty challenging. The Architexa plugin is pretty good in this area, especially when you already have code already done. The Architexa plugin would take advantage of existing code and Java reflection and give you a selectable choice on what and to where as action. Here I will highlight couple actions in TimeMachine.

The most basic startup of the TimeMachine sequence would look something like this.

TimeMachine Scheduler startup up sequence diagram

In the heart of the scheduler logic is in the PollingScheduleRunner service, and here are some of the actions depicted.

TimeMachine PollingScheduleRunner sequence diagram

Using Architexa

Over all, I think Architexa provided a pretty good plugin for Eclipse. I have some problems when generating the sequence diagrams. For example, I can't resize the width between two actors(classes) to display full method name. And in some actions there are no lines drawn! I suspect it's due to the tool is using code inspection, and some of the call are called by different threads.

The plugin itself seems pretty solid. You can install and uninstall without harm. They currently offer free download for small number of users, which I think it's a great way for you to explorer.

I am not too sure how effective of Architexa on builing a collabration of sharing these diagrams as platform. I treat these diagrams as supplement to the project. It certainly helps in explaining high level architeture, but it's far from the complete documentation. No documentation can get any closer to the code. I think the strength of Java being static already provided some level of documentation when just reading the code alone. I would rather Architexa to focus and perfecting the plugin that draw the diagram, because these are more important to me.

Saturday, September 29, 2012

Enhancing Spring Test Framework with beforeClass and afterClass setup

How to allow instance methods to run as JUnit BeforeClass behavior

JUnit allows you to setup methods on the class level once before and after all tests methods invocation. However, by design on purpose that they restrict this to only static methods using @BeforeClass and @AfterClass annotations. For example this simple demo shows the typical Junit setup:

package deng.junitdemo;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;

public class DemoTest {

    @Test
    public void testOne() {
        System.out.println("Normal test method #1.");
    }

    @Test
    public void testTwo() {
        System.out.println("Normal test method #2.");
    }

    @BeforeClass
    public static void beforeClassSetup() {
        System.out.println("A static method setup before class.");
    }

    @AfterClass
    public static void afterClassSetup() {
        System.out.println("A static method setup after class.");
    }
}

And above should result the following output:

A static method setup before class.
Normal test method #1.
Normal test method #2.
A static method setup after class.

This usage is fine for most of the time, but there are times you want to use non-static methods to setup the test. I will show you a more detailed use case later, but for now, let's see how we can solve this naughty problem with JUnit first. We can solve this by making the test implements a Listener that provide the before and after callbacks, and we will need to digg into JUnit to detect this Listener to invoke our methods. This is a solution I came up with:

package deng.junitdemo;

import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(InstanceTestClassRunner.class)
public class Demo2Test implements InstanceTestClassListener {

    @Test
    public void testOne() {
        System.out.println("Normal test method #1");
    }

    @Test
    public void testTwo() {
        System.out.println("Normal test method #2");
    }

    @Override
    public void beforeClassSetup() {
        System.out.println("An instance method setup before class.");
    }

    @Override
    public void afterClassSetup() {
        System.out.println("An instance method setup after class.");
    }
}

As stated above, our Listener is a simple contract:

package deng.junitdemo;

public interface InstanceTestClassListener {
    void beforeClassSetup();
    void afterClassSetup();
}

Our next task is to provide the JUnit runner implementation that will trigger the setup methods.

package deng.junitdemo;

import org.junit.runner.notification.RunNotifier;
import org.junit.runners.BlockJUnit4ClassRunner;
import org.junit.runners.model.InitializationError;

public class InstanceTestClassRunner extends BlockJUnit4ClassRunner {

    private InstanceTestClassListener instanceSetupListener;

    public InstanceTestClassRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    @Override
    protected Object createTest() throws Exception {
        Object test = super.createTest();
        // Note that JUnit4 will call this createTest() multiple times for each
        // test method, so we need to ensure to call "beforeClassSetup" only once.
        if (test instanceof InstanceTestClassListener && instanceSetupListener == null) {
            instanceSetupListener = (InstanceTestClassListener) test;
            instanceSetupListener.beforeClassSetup();
        }
        return test;
    }

    @Override
    public void run(RunNotifier notifier) {
        super.run(notifier);
        if (instanceSetupListener != null)
            instanceSetupListener.afterClassSetup();
    }
}

Now we are in business. If we run above test, it should give us similar result, but this time we are using instance methods instead!

An instance method setup before class.
Normal test method #1
Normal test method #2
An instance method setup after class.

A concrete use case: Working with Spring Test Framework

Now let me show you a real use case with above. If you use Spring Test Framework, you would normally setup a test like this so that you may have test fixture injected as member instance.

package deng.junitdemo.spring;

import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;

import java.util.List;

import javax.annotation.Resource;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class SpringDemoTest {

    @Resource(name="myList")
    private List<String> myList;

    @Test
    public void testMyListInjection() {
        assertThat(myList.size(), is(2));
    }
}

You would also need a spring xml under that same package for above to run:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
     <bean id="myList" class="java.util.ArrayList">
        <constructor-arg>
            <list>
                <value>one</value>
                <value>two</value>
            </list>
        </constructor-arg>
     </bean>
</beans>

Pay very close attention to member instance List<String> myList. When running JUnit test, that field will be injected by Spring, and it can be used in any test method. However, if you ever want a one time setup of some code and get a reference to a Spring injected field, then you are in bad luck. This is because the JUnit @BeforeClass will force your method to be static; and if you make your field static, Spring injection won't work in your test!

Now if you are a frequent Spring user, you should know that Spring Test Framework already provided a way for you to handle this type of use case. Here is a way for you to do class level setup with Spring's style:

package deng.junitdemo.spring;

import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;

import java.util.List;

import javax.annotation.Resource;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.TestContext;
import org.springframework.test.context.TestExecutionListeners;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.support.AbstractTestExecutionListener;
import org.springframework.test.context.support.DependencyInjectionTestExecutionListener;

@RunWith(SpringJUnit4ClassRunner.class)
@TestExecutionListeners(listeners = {
        DependencyInjectionTestExecutionListener.class, 
        SpringDemo2Test.class})
@ContextConfiguration
public class SpringDemo2Test extends AbstractTestExecutionListener {

    @Resource(name="myList")
    private List<String> myList;

    @Test
    public void testMyListInjection() {
        assertThat(myList.size(), is(2));
    }

    @Override
    public void afterTestClass(TestContext testContext) {
        List<?> list = testContext.getApplicationContext().getBean("myList", List.class);
        assertThat((String)list.get(0), is("one"));
    }

    @Override
    public void beforeTestClass(TestContext testContext) {
        List<?> list = testContext.getApplicationContext().getBean("myList", List.class);
        assertThat((String)list.get(1), is("two"));
    }
}

As you can see, Spring offers the @TestExecutionListeners annotation to allow you to write any Listener, and in it you will have a reference to the TestContext which has the ApplicationContext for you to get to the injected field reference. This works, but I find it not very elegant. It forces you to look up the bean, while your injected field is already available as field. But you can't use it unless you go through the TestContext parameter.

Now if you mix the solution we provided in the beginning, we will see a more prettier test setup. Let's see it:

package deng.junitdemo.spring;

import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;

import java.util.List;

import javax.annotation.Resource;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.ContextConfiguration;

import deng.junitdemo.InstanceTestClassListener;

@RunWith(SpringInstanceTestClassRunner.class)
@ContextConfiguration
public class SpringDemo3Test implements InstanceTestClassListener {

    @Resource(name="myList")
    private List<String> myList;

    @Test
    public void testMyListInjection() {
        assertThat(myList.size(), is(2));
    }

    @Override
    public void beforeClassSetup() {
        assertThat((String)myList.get(0), is("one"));
    }

    @Override
    public void afterClassSetup() {
        assertThat((String)myList.get(1), is("two"));
    }
}

Now JUnit only allow you to use single Runner, so we must extends the Spring's version to insert what we did before.

package deng.junitdemo.spring;

import org.junit.runner.notification.RunNotifier;
import org.junit.runners.model.InitializationError;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

import deng.junitdemo.InstanceTestClassListener;

public class SpringInstanceTestClassRunner extends SpringJUnit4ClassRunner {

    private InstanceTestClassListener instanceSetupListener;

    public SpringInstanceTestClassRunner(Class<?> clazz) throws InitializationError {
        super(clazz);
    }

    @Override
    protected Object createTest() throws Exception {
        Object test = super.createTest();
        // Note that JUnit4 will call this createTest() multiple times for each
        // test method, so we need to ensure to call "beforeClassSetup" only once.
        if (test instanceof InstanceTestClassListener && instanceSetupListener == null) {
            instanceSetupListener = (InstanceTestClassListener) test;
            instanceSetupListener.beforeClassSetup();
        }
        return test;
    }

    @Override
    public void run(RunNotifier notifier) {
        super.run(notifier);
        if (instanceSetupListener != null)
            instanceSetupListener.afterClassSetup();
    }
}

That should do the trick. Running the test will give use this output:

12:58:48 main INFO  org.springframework.test.context.support.AbstractContextLoader:139 | Detected default resource location "classpath:/deng/junitdemo/spring/SpringDemo3Test-context.xml" for test class [deng.junitdemo.spring.SpringDemo3Test].
12:58:48 main INFO  org.springframework.test.context.support.DelegatingSmartContextLoader:148 | GenericXmlContextLoader detected default locations for context configuration [ContextConfigurationAttributes@74b23210 declaringClass = 'deng.junitdemo.spring.SpringDemo3Test', locations = '{classpath:/deng/junitdemo/spring/SpringDemo3Test-context.xml}', classes = '{}', inheritLocations = true, contextLoaderClass = 'org.springframework.test.context.ContextLoader'].
12:58:48 main INFO  org.springframework.test.context.support.AnnotationConfigContextLoader:150 | Could not detect default configuration classes for test class [deng.junitdemo.spring.SpringDemo3Test]: SpringDemo3Test does not declare any static, non-private, non-final, inner classes annotated with @Configuration.
12:58:48 main INFO  org.springframework.test.context.TestContextManager:185 | @TestExecutionListeners is not present for class [class deng.junitdemo.spring.SpringDemo3Test]: using defaults.
12:58:48 main INFO  org.springframework.beans.factory.xml.XmlBeanDefinitionReader:315 | Loading XML bean definitions from class path resource [deng/junitdemo/spring/SpringDemo3Test-context.xml]
12:58:48 main INFO  org.springframework.context.support.GenericApplicationContext:500 | Refreshing org.springframework.context.support.GenericApplicationContext@44c9d92c: startup date [Sat Sep 29 12:58:48 EDT 2012]; root of context hierarchy
12:58:49 main INFO  org.springframework.beans.factory.support.DefaultListableBeanFactory:581 | Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@73c6641: defining beans [myList,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; root of factory hierarchy
12:58:49 Thread-1 INFO  org.springframework.context.support.GenericApplicationContext:1025 | Closing org.springframework.context.support.GenericApplicationContext@44c9d92c: startup date [Sat Sep 29 12:58:48 EDT 2012]; root of context hierarchy
12:58:49 Thread-1 INFO  org.springframework.beans.factory.support.DefaultListableBeanFactory:433 | Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@73c6641: defining beans [myList,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; root of factory hierarchy

Obviously the output shows nothing interesting here, but the test should run with all assertion passed. The point is that now we have a more elegant way to invoking a before and after test setup that are at class level, and they can be instance methods to allow Spring injection.

Download the demo code

You may get above demo code in a working Maven project from my sandbox.

Wednesday, September 26, 2012

TimeMachine Scheduler 1.2.2 Release

This is a bug fix release for couple of issues:

 1. Fixed an issue with parsing release date string with "EDT" timezone that can cause the scheduler fail to startup

 2. Added the optional slf4j-log4j binding to view log output in war file.

You may download the latest here: https://bitbucket.org/timemachine/scheduler/downloads

Wednesday, September 19, 2012

Where did Git go?

My git is gone after I upgraded to latest MacOSX updates! What a annoying thing. Appearently it's in here now: /usr/local/git/bin/git You can edit your $PATH like it says here, or you could just relink it.
ln -s /usr/local/git/bin/git /usr/local/bin/git

Saturday, September 15, 2012

Building message based application using Camel

This is a long article that contains three separate topics:

  • Getting started with Apache Camel with Java
  • Improving startup of routes with a CamelRunner
  • Building message based application using Camel

But since I've prepared a camel-demo-1.0.0-SNAPSHOT-project.zip that has all these materials included, I thought it would easier to combine them and present it as whole.

Getting started with Apache Camel with Java

Trying out Camel with few Groovy lines is one thing, but getting a full scale project in Java is another matter. Today, I will show you how to get things started on Apache Camel with Maven based project. You may also use the provided camel-demo as project template to jump start your own Apache Camel project. You would just need to rename the Java package and rename the pom's group and artifact id's to match your need.

Preparing a Maven based project with Camel dependencies

Unzip the camel-demo project source, and you will see the basic directory layout.

camel-demo
    +- bin
    +- config
    +- data
    +- src
    +- pom.xml
    +- README.txt

What makes this demo a Camel based project is just the declaration in pom.xml. Let's take a look the file and its dependencies.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0   http://maven.apache.org/maven-v4_0_0.xsd">

    <modelVersion>4.0.0</modelVersion>
    <groupId>deng.cameldemo</groupId>
    <artifactId>camel-demo</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <slf4j.version>1.6.6</slf4j.version>
        <camel.version>2.10.1</camel.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.3</version>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>project</descriptorRef>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencies>

        <!-- Unit testing lib -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit-dep</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-library</artifactId>
            <version>1.2.1</version>
            <scope>test</scope>
        </dependency>

        <!-- Logging lib -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>${slf4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>${slf4j.version}</version>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>

        <!-- Apache Commons lib -->
        <dependency>
            <groupId>commons-lang</groupId>
            <artifactId>commons-lang</artifactId>
            <version>2.6</version>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.0.1</version>
        </dependency>

        <!-- Apache Camel -->
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-core</artifactId>
            <version>${camel.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-spring</artifactId>
            <version>${camel.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-groovy</artifactId>
            <version>${camel.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-jackson</artifactId>
            <version>${camel.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-mina</artifactId>
            <version>${camel.version}</version>
        </dependency>

    </dependencies>

</project>

This pom.xml decalares a Java based application and it will produce jar. It requires minimal of JDK 6 or higher. Besides the typical junit and hamcrest for unit testing, I also added slf4j for logging. I have added couple Apache's commons-lang/io to the project as well. I think these are basic settings that any Java based application should use them.

The maven-assembly-plugin I have declared is only for this demo packging purpose, and you may change or remove to suite your own project need.

For Camel dependencies, you would need minimal camel-core for routes building. And then you can add any additional components you plan to use in your project. I have added the following for building typical message based application development:

  1. The camel-spring - we want to have option to declare Camel routes in xml files as configuration. See camel-demo/config directory for samples.
  2. The camel-jackson - we want to process messaging data in our application as JSON format.
  3. The camel-mina - we want to send messaging data accross network through TCP socket.
  4. The camel-groovy - [optional] we want to be able to add dynamic scripting to route, even inside the xml config. This is great for debug and POC.

Note that since we use multiple camel components dependencies, I choose to set a Maven property ${camel.version} so that when we upgrade Camel, it's easier to maintain the pom.xml file in one place.

You should able to cd into the project directory and run mvn compile to verify that the project. It should compile without error.

Improving startup of routes with a CamelRunner

With the project pom.xml file ready, you can start creating Camel routes to handle your own business logics. Before we get too excited, let's try out a simple HelloRoute to see how it works and how we can run it first. Here is the route defnition code in src/main/java/deng/cameldemo/HelloRoute.java.

package deng.cameldemo;

import org.apache.camel.builder.RouteBuilder;

public class HelloRoute extends RouteBuilder {
    @Override
    public void configure() throws Exception {
        from("timer://helloTimer?period=3000").
            to("log:" + getClass().getName());
    }
}

Take a test ride with the Camel

To see above in action, we need to add it into a CamelContext and start the context. For Java standalone program, we would write this setup code in a Main class. The Camel actually comes with a org.apache.camel.main.MainSupport abstract class that you may use to extend your own Main. However, I think it would be even nicer if Camel would provide a CamelRunner that can run like this.

$ java CamelRunner deng.cameldemo.HelloRoute

Such CamelRunner would be very user friendly and re-usable to have, so that's what I did. I wrote one like this:

package deng.cameldemo;

import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.support.FileSystemXmlApplicationContext;

/** 
 * A main program to start Camel and run as a server using RouteBuilder class names or 
 * Spring config files.
 * 
 * <p>Usage:
 * 
 * java deng.cameldemo.CamelRunner deng.cameldemo.HelloRoute
 * 
 * or
 * 
 * java -Dspring=true deng.cameldemo.CamelRunner /path/to/camel-spring.xml
 * 
 * @author Zemian Deng
 */
public class CamelRunner {
    public static void main(String[] args) throws Exception {
        CamelRunner runner = new CamelRunner();
        runner.run(args);
    }

    private static Logger logger = LoggerFactory.getLogger(CamelRunner.class);
    public void run(String[] args) throws Exception {
        if (Boolean.parseBoolean(System.getProperty("spring", "false")))
            runWithSpringConfig(args);
        else
            runWithCamelRoutes(args);

        // Wait for user to hit CRTL+C to stop the service
        synchronized(this) {
            this.wait();
        }
    }

    private void runWithSpringConfig(String[] args) {
        final ConfigurableApplicationContext springContext = new FileSystemXmlApplicationContext(args);

        // Register proper shutdown.
        Runtime.getRuntime().addShutdownHook(new Thread() { 
            @Override
            public void run() {
                try {
                    springContext.close();
                    logger.info("Spring stopped.");
                } catch (Exception e) {
                    logger.error("Failed to stop Spring.", e);
                }
            }
        });

        // Start spring
        logger.info("Spring started.");
    }

    private void runWithCamelRoutes(String[] args) throws Exception {
        final CamelContext camelContext = new DefaultCamelContext();        
        // Register proper shutdown.
        Runtime.getRuntime().addShutdownHook(new Thread() { 
            @Override
            public void run() {
                try {
                    camelContext.stop();
                    logger.info("Camel stopped for {}", camelContext);
                } catch (Exception e) {
                    logger.error("Failed to stop Camel.", e);
                }
            }
        });

        // Added RouteBuilder from args
        for (String className : args) {
            Class<?> cls = Class.forName(className);
            if (RouteBuilder.class.isAssignableFrom(cls)) {
                Object obj = cls.newInstance();
                RouteBuilder routeBuilder = (RouteBuilder)obj;
                camelContext.addRoutes(routeBuilder);
            } else {
                throw new RuntimeException("Unable to add Camel RouteBuilder " + className);
            }
        }

        // Start camel
        camelContext.start();
        logger.info("Camel started for {}", camelContext);
    }
}

To help you run the main class, I have provided a run-java wrapper script under the project's bin directory, so that you may quickly test it without having to setup classpath.

$ mvn package
$ bin/run-java deng.cameldemo.CamelRunner deng.cameldemo.HelloRoute

You will see that the program will load the HelloRoute in a DefaultCamelContext and start it as a server. The HelloRoute itself will generate a 3 seconds timer message and send it to a logger, which should be printing onto your console screen. This will continue forever until you hit CTRL+C to end it.

NOTE: You only have to invoke mvn package command once, so that it will package up all the dependencies jars in order for run-java to auto-detect them. If you are not going to use maven-assembly-plugin during package phase, then use mvn dependency:copy-dependencies command explicitly will work fine as well.

Take a test ride with the Camel, Part 2: running Camel with Spring xml configuration

The HelloRoute example above would simply provide route definition that formed by using component URI's. It will be nice if we can configure the route in a declarative manner so that we may change the route without re-compile a class file. This will be very handy especially if you are not familiar with each component's options and want to explore and try things out. Well, that's what the camel-spring is for. Beside giving you an option to load route in xml config file, it also provides a very flexible way to register custom services/processors bean in the Spring IoC container.

If you are a keen reader, you will notice in the CamelRunner code above that it has an extra runWithSpringConfig part. So the CamelRunner can actually bootstrap any Spring xml file and start a context as a server. You may use it like this:

$ bin/run-java deng.cameldemo.CamelRunner -Dspring=true config/hellocamel-spring.xml

The config/hellocamel-spring.xml is just an equivalent of our HelloRoute code but in Spring xml form:

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

    <camelContext id="helloCamel" xmlns="http://camel.apache.org/schema/spring">
        <route>
            <from uri="timer://jdkTimer?period=3000"/>
            <to uri="log://deng.cameldemo.HelloCamel"/>
        </route>
    </camelContext>

</beans>

This remove the need to compile/re-compile HelloRoute to define the Camel route to run.

Building message based application using Camel

To present you with a more practical demo, I would show you further on how to setup Camel to process message based application. In many IT shops, it's common that you would have a server to take message data as input and process them. A practical use case is to take any JSON formated message and transform it into object and process it. To do this in Camel, what you want to build is a route that will take input messages from a TCP port, and then process it in a pipeflow with any business logic you may have. You will run the route as a server, and then client may use any mean to submit the message to the TCP port. Client may even be another thin Camel client app to submit data as well. Let me show you how to get started.

Writing the server side code with Camel route

The server side would need a route to listen from a TCP port, and this is provided by camel-mina component. The first step is you need a route.

package deng.cameldemo;

import org.apache.camel.builder.RouteBuilder;

public class TcpMsgRoute extends RouteBuilder {
    @Override
    public void configure() throws Exception {
        String port = System.getProperty("port", "12345");
        from("mina:tcp://localhost:" + port + "?sync=false").
            to("log:" + getClass().getName());
    }
}

Then the next step is ... done! No way, you mean that's all there to it for a server? Too good to be true? Well, let's try it out

$ bin/run-java deng.cameldemo.CamelRunner deng.cameldemo.TcpMsgRoute -Dport=12345
15:21:41 main INFO  org.apache.camel.impl.DefaultCamelContext:1391 | Apache Camel 2.10.1 (CamelContext: camel-1) is starting
15:21:41 main INFO  org.apache.camel.management.ManagementStrategyFactory:43 | JMX enabled.
15:21:42 main INFO  org.apache.camel.impl.converter.DefaultTypeConverter:45 | Loaded 172 type converters
15:21:42 main INFO  org.apache.camel.component.mina.MinaConsumer:59 | Binding to server address: localhost/127.0.0.1:12345 using acceptor: org.apache.mina.transport.socket.nio.SocketAcceptor@2ffad8fe
15:21:42 main INFO  org.apache.camel.impl.DefaultCamelContext:2045 | Route: route1 started and consuming from: Endpoint[mina://tcp://localhost:12345?sync=true]
15:21:42 main INFO  org.apache.camel.management.DefaultManagementLifecycleStrategy:859 | StatisticsLevel at All so enabling load performance statistics
15:21:42 main INFO  org.apache.camel.impl.DefaultCamelContext:1426 | Total 1 routes, of which 1 is started.
15:21:42 main INFO  org.apache.camel.impl.DefaultCamelContext:1427 | Apache Camel 2.10.1 (CamelContext: camel-1) started in 0.505 seconds
15:21:42 main INFO  deng.cameldemo.CamelRunner:93 | Camel started for CamelContext(camel-1)

Voila! The server is up and waiting for your users to send messages through port 12345. Not too bad for few lines of code.

Writing the client side code with Camel ProducerTemplate

Since our server expose a TCP port and take in any text content message, you can create any client that's capable writing to a TCP socket. In here, I will show you how to use Camel to write a thin client.

package deng.cameldemo.client;

import java.io.FileReader;
import org.apache.camel.CamelContext;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.commons.io.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class TcpMsgSender {
    public static void main(String[] args) throws Exception {
        TcpMsgSender runner = new TcpMsgSender();
        runner.run(args);
    }

    private static Logger logger = LoggerFactory.getLogger(TcpMsgSender.class);
    public void run(String[] args) throws Exception {
        String fileName = args.length > 0 ? args[0] : "data/msg.txt";
        String[] hostPort = (args.length > 1 ? args[1] : "localhost:12345").split(":");
        String host = hostPort[0];
        String port = hostPort.length > 1 ? hostPort[1] : "12345";
        logger.info("Sending tcp message {} to host={}, port={}", new Object[]{ fileName, host, port});

        String text = IOUtils.toString(new FileReader(fileName));
        logger.debug("File size={}", text.length());

        CamelContext camelContext = new DefaultCamelContext();
        ProducerTemplate producer = camelContext.createProducerTemplate();
        producer.sendBody("mina:tcp://" + host + ":" + port + "?sync=false", text);
        logger.info("Message sent.");
    }
}

This TcpMsgSender can send any text file to your server endpoint. Try this out while your server is running:

$ bin/run-java deng.cameldemo.client.TcpMsgSender data/test-msg.json localhost:12345
15:22:35 main INFO  deng.cameldemo.client.TcpMsgSender:24 | Sending tcp message data/test-msg.json to host=localhost, port=12345
15:22:35 main DEBUG deng.cameldemo.client.TcpMsgSender:27 | File size=47
15:22:35 main INFO  org.apache.camel.impl.converter.DefaultTypeConverter:45 | Loaded 172 type converters
15:22:35 main INFO  org.apache.camel.management.ManagementStrategyFactory:43 | JMX enabled.
15:22:35 main INFO  deng.cameldemo.client.TcpMsgSender:32 | Message sent.

You should able to verify from your server console output that it received the msg. The msg I sent is in data/test-msg.json, which contains this simple text:

{ "firstName" : "Zemian", "lastName" : "Deng" }

Note that our server simply receive plain text and log it. We will discuss how to process the message next.

Processing message data in JSON format with Camel and Spring xml config

You thought the server code was easy from above, guess again. You can actually replace the TcpMsgRoute with just some simple xml lines!

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

    <camelContext id="tcpMsgServer" xmlns="http://camel.apache.org/schema/spring">
        <route>
            <from uri="mina:tcp://localhost:12345?sync=false"/>
            <to uri="log://deng.cameldemo.TcpMsgServer"/>
        </route>
    </camelContext>

</beans>

Save it as config/tcpmsgserver-spring.xml. Then re-run the server, and you should get the same result as above.

$ bin/run-java deng.cameldemo.CamelRunner -Dspring=true config/tcpmsgserver-spring.xml

Now let us improve the above xml to further process the JSON message data. We will like to transform the plain text to a Java object then process by a custom bean. To do that, we first would need to add unmarshal component to the route. This is where the camel-jackson comes into play. In our demo, the unmarshalling step would convert the JSON text into a java.util.Map and then pass it to a processor bean named myMsgProcessor. Let's create a new xml file named config/tcpmsgserver-json-spring.xml as follow.

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

    <camelContext id="tcpMsgServer" xmlns="http://camel.apache.org/schema/spring">
        <route>
            <from uri="mina:tcp://localhost:12345?sync=false"/>
            <to uri="log://deng.cameldemo.TcpMsgServer"/>
            <unmarshal>
                <json library="Jackson"/>
            </unmarshal>
            <to uri="bean:myMsgProcessor?method=process"/>
        </route>
    </camelContext>

    <bean id="myMsgProcessor" class="deng.cameldemo.MyMsgProcessor">
    </bean>

</beans>

The myMsgProcessor is an Spring bean that we provide custom logic code to process the data. At this point we have a full Java object to manipulate. The content of the processor can be any POJO with the method name specified in the URI. Here is an example one:

package deng.cameldemo;

import org.apache.camel.builder.RouteBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Map;

public class MyMsgProcessor {
    private static Logger logger = LoggerFactory.getLogger(MyMsgProcessor.class);
    public void process(Map<String, String> data) {
        logger.info("We should slice and dice the data: " + data);
    }
}

Try re-run the server with the new xml file above, and you should able to re-invoke the same client command to test it out. Here is a sample output of my server:

$ bin/run-java deng.cameldemo.CamelRunner -Dspring=true config/tcpmsgserver-json-spring.xml
17:05:25 main INFO  org.springframework.context.support.FileSystemXmlApplicationContext:456 | Refreshing org.springframework.context.support.FileSystemXmlApplicationContext@4200309: startup date [Sat Sep 15 17:05:25 EDT 2012]; root of context hierarchy
17:05:25 main INFO  org.springframework.beans.factory.xml.XmlBeanDefinitionReader:315 | Loading XML bean definitions from file [/Users/zemian/projects/sandbox/camel-demo/config/tcpmsgserver-json-spring.xml]
17:05:27 main INFO  org.springframework.beans.factory.support.DefaultListableBeanFactory:557 | Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@27b75165: defining beans [template,consumerTemplate,tcpMsgServer:beanPostProcessor,tcpMsgServer,myMsgProcessor]; root of factory hierarchy
17:05:27 main INFO  org.apache.camel.spring.SpringCamelContext:1391 | Apache Camel 2.10.1 (CamelContext: tcpMsgServer) is starting
17:05:27 main INFO  org.apache.camel.management.ManagementStrategyFactory:43 | JMX enabled.
17:05:27 main INFO  org.apache.camel.impl.converter.DefaultTypeConverter:45 | Loaded 172 type converters
17:05:28 main INFO  org.apache.camel.component.mina.MinaConsumer:59 | Binding to server address: localhost/127.0.0.1:12345 using acceptor: org.apache.mina.transport.socket.nio.SocketAcceptor@5a3cae4a
17:05:28 main INFO  org.apache.camel.spring.SpringCamelContext:2045 | Route: route1 started and consuming from: Endpoint[mina://tcp://localhost:12345?sync=false]
17:05:28 main INFO  org.apache.camel.management.DefaultManagementLifecycleStrategy:859 | StatisticsLevel at All so enabling load performance statistics
17:05:28 main INFO  org.apache.camel.spring.SpringCamelContext:1426 | Total 1 routes, of which 1 is started.
17:05:28 main INFO  org.apache.camel.spring.SpringCamelContext:1427 | Apache Camel 2.10.1 (CamelContext: tcpMsgServer) started in 0.695 seconds
17:05:28 main INFO  deng.cameldemo.CamelRunner:61 | Spring started.
17:05:35 Camel (tcpMsgServer) thread #3 - MinaThreadPool INFO  deng.cameldemo.TcpMsgServer:96 | Exchange[ExchangePattern:InOnly, BodyType:String, Body:{ "firstName" : "Zemian", "lastName" : "Deng" }]
17:05:35 Camel (tcpMsgServer) thread #3 - MinaThreadPool INFO  deng.cameldemo.MyMsgProcessor:11 | We should slice and dice the data: {lastName=Deng, firstName=Zemian}

Pay attention that Camel will auto convert the data format in your route! Our client only sends the plain text as JSON format, but when server receives it, it unmarshals it using Jackson library, and then converts it into a java Map object. It then passes the map object into our processor bean. Also, in this demo, I choose to use a generic java.util.Map as processor method argument (which is output of the JSON unmarshal), but you can easily define your own business data type, such as MyCustomerData. This reveals the power of Camel, since you don't need to push the message in your flow, but only worry about writing your "processor" as a POJO. The Camel will "glue" components together to form a route and carry the message data through the pipeline flow.

On the same token, when you write your business logic in one or more processors, it's a good idea that you limit your POJO logic to be as small unit as possible. When you do this, then you can maximize the reusability of the processors. The bigger POJO you make, with many business logics mixed in, it will also make it difficult to test. So I recommend you when developing these processor beans, try to think them as LEGO pieces -- small POJO. You want to let Camel define the route and glue the LEGO pieces togther. Once you get into this habit of thiking, then you can use Camel in a more effectively way to solve many of your domain problems.

Well, that's all for today folks. I hope you enjoyed the Camel ride. Happy programming!

Friday, September 14, 2012

Boost your Groovy with NailGun

Are you working on a large Hibernate project that takes long time to load up all the hbm.xml files when creating the Session object? This is fine during deployment and runtime because it only loads it once. However, often time we also need to the same Session object to do some ad-hoc HQL queries to debug or validate data. Loading and re-loading large mapping files in a Session to just execute single query is very painful.

Now, I like to poke around Java things with Groovy, and it's a great tool to peek at your data as ad-hoc queries as well. You can easily do this with their groovyConsole GUI tool and add --classpath option to include your project classes. This will bring up a tiny Editor, and you can script to load your hibernate Session there. Once the first run is loaded (the hibenrate Session created), then second run is almost in an instant.

Running the little groovyConsole had wet my appetite, and I was hungry for a better text editor, but yet I don't really want a full blow IDE for scripting. I like do my scripting in a plain editor. Now if you have an editor such as Sublime Text 2 (ST2) that has a "build" feature to execute a external command, then you will enjoy scripting much more. With ST2, I can have it call groovy.bat from inside the editor on the script that I am editing. However, there is another problem: the external command will restart a new JVM process on each run! Now I am back to square one.

To solve this problem, and still have my cake (editor), I recalled an awesome tool called NailGun. This works perfectly with Groovy and my problem. I can start a server like this

java -cp "groovy-all-2.0.1.jar:nailgun-0.7.1.jar" -server com.martiansoftware.nailgun.NGServer

And then in my ST2 editor, I can run an external command like this as the NailGun client:

/path/to/nailgun-0.7.1/ng groovy.ui.GroovyMain test.groovy

Nail gun client sends the script file content to the server and prints the result. Again, after first run, the second run should be instantaneously.

There, I scratched my itch.

Details on how to setup Sublime Text 2 to run NailGun client

  1. Go to menu: "Preference > Browse Packages"
  2. Open the Groovy folder
  3. Save a file named "GroovyNailGunClient.sublime-build" with the following:
       {
         "cmd": ["/path/to/nailgun-0.7.1/ng", "groovy.ui.GroovyMain", "$file"],
         "file_regex": "^(...?):([0-9]):?([0-9]*)",
         "selector": "source.groovy"
       }
    
  4. Select menu "Tool > Build System > GroovyNailGunClient"
  5. Press CTRL+B to run any groovy file in your editor.

Saturday, September 8, 2012

Improving java.util.Properties

The Java built-in java.util.Properties class could really use some love. I have written a slightly improved version called timemachine.scheduler.support.Props, and below are some features that I use often.

You can use it as a "String Map" of properties

Props props1 = new Props();
props1.put("foo", "bar");

// It can load from/to the Java Properties
Props props2 = new Props(System.getProperties());
java.util.Properties javaProps = props3.toProperties();

// It can load from/to a basic java.util.Map
Props props3 = new Props(System.getenv());

// Props is a HashMap<String, String>, so no need to convert. Just use it
for(Map.Entry<String, String> entry : props3.entrySet())
    System.our.println(entry.getKey() + ": " + entry.getValue());

You can load from a file in a single line

Props props1 = new Props("config.properties");
Props props2 = new Props("/path/to/config.properties");
Props props3 = new Props(new java.net.URL("http://myhost/config/config.properties"));
Props props4 = new Props(ClasspathURLStreamHandler.createURL("classpath://config/config.properties"));

// You can re-load on top of existing instance to override values
props4.load("config2.properties");

NOTE: The ClasspathURLStreamHandler is a utility class from the same package under timemachine.scheduler.support that can load any resources that's in the classpath.

You can get many basic types conversion

Props props = new Props();
props.put("str", "foo");
props.put("num", "123");
props.put("dec", "99.99");
props.put("flag", "true");

String str = props.getString("str");
int num = props.getInt("num");
double dec = props.getDouble("dec");
boolean flag = props.getBoolean("flag");

// You can even get default value when key is not found too
int num2 = props.getInt("num2", -1);

You can auto expand ${variable} from any existing properties

Props props = new Props(System.getProperties());
props.put("configDir", "${user.home}/myapp/config");
props.expandVariables();

// The ${user.home} should be expanded to actual user home dir value.
File dir = new File(props.get("configDir"));

There you have it. You see more code than words in this post, but I believe simple code speak louder than words and docs. I find these features very convenient and practical for many Java applications to use. I wish the JDK would provide these out of the box, and make the java.util.Properties more developer friendly.

Sunday, September 2, 2012

How to write better POJO Services

In Java, you can easily implement some business logic in Plain Old Java Object (POJO) classes, and then able to run them in a fancy server or framework without much hassle. There many server/frameworks, such as JBossAS, Spring or Camel etc, that would allow you to deploy POJO without even hardcoding to their API. Obviously you would get advance features if you willing to couple to their API specifics, but even if you do, you can keep these to minimal by encapsulating your own POJO and their API in a wrapper. By writing and designing your own application as simple POJO as possible, you will have the most flexible ways in choose a framework or server to deploy and run your application. One effective way to write your business logic in these environments is to use Service component. In this article I will share few things I learned in writing Services.

What is a Service?

The word Service is overly used today, and it could mean many things to different people. When I say Service, my definition is a software component that has minimal of life-cycles such as init, start, stop, and destroy. You may not need all these stages of life-cycles in every service you write, but you can simply ignore ones that don't apply. When writing large application that intended for long running such as a server component, definining these life-cycles and ensure they are excuted in proper order is crucial!

I will be walking you through a Java demo project that I have prepared. It's very basic and it should run as stand-alone. The only dependency it has is the SLF4J logger. If you don't know how to use logger, then simply replace them with System.out.println. However I would strongly encourage you to learn how to use logger effectively during application development though. Also if you want to try out the Spring related demos, then obviously you would need their jars as well.

Writing basic POJO service

You can quickly define a contract of a Service with life-cycles as below in an interface.

package servicedemo;

public interface Service {
    void init();
    void start();
    void stop();
    void destroy();
    boolean isInited();
    boolean isStarted();
}

Developers are free to do what they want in their Service implementation, but you might want to give them an adapter class so that they don't have to re-write same basic logic on each Service. I would provide an abstract service like this:

package servicedemo;

import java.util.concurrent.atomic.*;
import org.slf4j.*;
public abstract class AbstractService implements Service {
    protected Logger logger = LoggerFactory.getLogger(getClass());
    protected AtomicBoolean started = new AtomicBoolean(false);
    protected AtomicBoolean inited = new AtomicBoolean(false);

    public void init() {
        if (!inited.get()) {
            initService();
            inited.set(true);
            logger.debug("{} initialized.", this);
        }
    }

    public void start() {
        // Init service if it has not done so.
        if (!inited.get()) {
            init();
        }
        // Start service now.
        if (!started.get()) {
            startService();
            started.set(true);
            logger.debug("{} started.", this);
        }
    }

    public void stop() {
        if (started.get()) {
            stopService();
            started.set(false);
            logger.debug("{} stopped.", this);
        }
    }

    public void destroy() {
        // Stop service if it is still running.
        if (started.get()) {
            stop();
        }
        // Destroy service now.
        if (inited.get()) {
            destroyService();
            inited.set(false);
            logger.debug("{} destroyed.", this);
        }
    }

    public boolean isStarted() {
        return started.get();
    }

    public boolean isInited() {
        return inited.get();
    }

    @Override
    public String toString() {
            return getClass().getSimpleName() + "[id=" + System.identityHashCode(this) + "]";
    }

    protected void initService() {
    }

    protected void startService() {
    }

    protected void stopService() {
    }

    protected void destroyService() {
    }
}

This abstract class provide the basic of most services needs. It has a logger and states to keep track of the life-cycles. It then delegate new sets of life-cycle methods so subclass can choose to override. Notice that the start() method is checking auto calling init() if it hasn't already done so. Same is done in destroy() method to the stop() method. This is important if we're to use it in a container that only have two stages life-cycles invocation. In this case, we can simply invoke start() and destroy() to match to our service's life-cycles.

Some frameworks might go even further and create separate interfaces for each stage of the life-cycles, such as InitableService or StartableService etc. But I think that would be too much in a typical app. In most of the cases, you want something simple, so I like it just one interface. User may choose to ignore methods they don't want, or simply use an adaptor class.

Before we end this section, I would throw in a silly Hello world service that can be used in our demo later.

package servicedemo;

public class HelloService extends AbstractService {
    public void initService() {
        logger.info(this + " inited.");
    }
    public void startService() {
        logger.info(this + " started.");
    }
    public void stopService() {
        logger.info(this + " stopped.");
    }
    public void destroyService() {
        logger.info(this + " destroyed.");
    }
}

Managing multiple POJO Services with a container

Now we have the basic of Service definition defined, your development team may start writing business logic code! Before long, you will have a library of your own services to re-use. To be able group and control these services into an effetive way, we want also provide a container to manage them. The idea is that we typically want to control and manage multiple services with a container as a group in a higher level. Here is a simple implementation for you to get started:

package servicedemo;

import java.util.*;
public class ServiceContainer extends AbstractService {
    private List<Service> services = new ArrayList<Service>();

    public void setServices(List<Service> services) {
        this.services = services;
    }
    public void addService(Service service) {
        this.services.add(service);
    }

    public void initService() {
        logger.debug("Initializing " + this + " with " + services.size() + " services.");
        for (Service service : services) {
            logger.debug("Initializing " + service);
            service.init();
        }
        logger.info(this + " inited.");
    }
    public void startService() {
            logger.debug("Starting " + this + " with " + services.size() + " services.");
            for (Service service : services) {
                logger.debug("Starting " + service);
                service.start();
            }
            logger.info(this + " started.");
    }
    public void stopService() {
            int size = services.size();
            logger.debug("Stopping " + this + " with " + size + " services in reverse order.");
            for (int i = size - 1; i >= 0; i--) {
                Service service = services.get(i);
                logger.debug("Stopping " + service);
                service.stop();
            }
            logger.info(this + " stopped.");
    }
    public void destroyService() {
            int size = services.size();
            logger.debug("Destroying " + this + " with " + size + " services in reverse order.");
            for (int i = size - 1; i >= 0; i--) {
                Service service = services.get(i);
                logger.debug("Destroying " + service);
                service.destroy();
            }
            logger.info(this + " destroyed.");
    }
}

From above code, you will notice few important things:

  1. We extends the AbstractService, so a container is a service itself.
  2. We would invoke all service's life-cycles before moving to next. No services will start unless all others are inited.
  3. We should stop and destroy services in reverse order for most general use cases.

The above container implementation is simple and run in synchronized fashion. This mean, you start container, then all services will start in order you added them. Stop should be same but in reverse order.

I also hope you would able to see that there is plenty of room for you to improve this container as well. For example, you may add thread pool to control the execution of the services in asynchronized fashion.

Running POJO Services

Running services with a simple runner program.

In the simplest form, we can run our POJO services on our own without any fancy server or frameworks. Java programs start its life from a static main method, so we surely can invoke init and start of our services in there. But we also need to address the stop and destroy life-cycles when user shuts down the program (usually by hitting CTRL+C.) For this, the Java has the java.lang.Runtime#addShutdownHook() facility. You can create a simple stand-alone server to bootstrap Service like this:

package servicedemo;

import org.slf4j.*;
public class ServiceRunner {
    private static Logger logger = LoggerFactory.getLogger(ServiceRunner.class);

    public static void main(String[] args) {
        ServiceRunner main = new ServiceRunner();
        main.run(args);
    }

    public void run(String[] args) {
        if (args.length < 1)
            throw new RuntimeException("Missing service class name as argument.");

        String serviceClassName = args[0];
        try {
            logger.debug("Creating " + serviceClassName);
            Class<?> serviceClass = Class.forName(serviceClassName);
            if (!Service.class.isAssignableFrom(serviceClass)) {
                throw new RuntimeException("Service class " + serviceClassName + " did not implements " + Service.class.getName());
            }
            Object serviceObject = serviceClass.newInstance();
            Service service = (Service)serviceObject;

            registerShutdownHook(service);

            logger.debug("Starting service " + service);
            service.init();
            service.start();
            logger.info(service + " started.");

            synchronized(this) {
                this.wait();
            }
        } catch (Exception e) {
            throw new RuntimeException("Failed to create and run " + serviceClassName, e);
        }
    }

    private void registerShutdownHook(final Service service) {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            public void run() {
                logger.debug("Stopping service " + service);
                service.stop();
                service.destroy();
                logger.info(service + " stopped.");
            }
        });
    }
}

With abover runner, you should able to run it with this command:

$ java demo.ServiceRunner servicedemo.HelloService

Look carefully, and you'll see that you have many options to run multiple services with above runner. Let me highlight couple:

  1. Improve above runner directly and make all args for each new service class name, instead of just first element.
  2. Or write a MultiLoaderService that will load multiple services you want. You may control argument passing using System Properties.

Can you think of other ways to improve this runner?

Running services with Spring

The Spring framework is an IoC container, and it's well known to be easy to work POJO, and Spring lets you wire your application together. This would be a perfect fit to use in our POJO services. However, with all the features Spring brings, it missed a easy to use, out of box main program to bootstrap spring config xml context files. But with what we built so far, this is actually an easy thing to do. Let's write one of our POJO Service to bootstrap a spring context file.

package servicedemo;

import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.support.FileSystemXmlApplicationContext;

public class SpringService extends AbstractService {
    private ConfigurableApplicationContext springContext;

    public void startService() {
        String springConfig = System.getProperty("springContext", "spring.xml);
        springContext = new FileSystemXmlApplicationContext(springConfig);
        logger.info(this + " started.");
    }
    public void stopService() {
        springContext.close();
        logger.info(this + " stopped.");
    }
}

With that simple SpringService you can run and load any spring xml file. For example try this:

$ java -DspringContext=config/service-demo-spring.xml demo.ServiceRunner servicedemo.SpringService

Inside the config/service-demo-spring.xml file, you can easily create our container that hosts one or more service in Spring beans.

<beans xmlns="http://www.springframework.org/schema/beans"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="helloService" class="servicedemo.HelloService">
    </bean>

    <bean id="serviceContainer" class="servicedemo.ServiceContainer" init-method="start" destroy-method="destroy">
        <property name="services">
            <list>
                <ref bean="helloService"/>
            </list>
        </property>
    </bean>

</beans>

Notice that I only need to setup init-method and destroy-method once on the serviceContainer bean. You can then add one or more other service such as the helloService as much as you want. They will all be started, managed, and then shutdown when you close the Spring context.

Note that Spring context container did not explicitly have the same life-cycles as our services. The Spring context will automatically instanciate all your dependency beans, and then invoke all beans who's init-method is set. All that is done inside the constructor of FileSystemXmlApplicationContext. No explicit init method is called from user. However at the end, during stop of the service, Spring provide the springContext#close() to clean things up. Again, they do not differentiate stop from destroy. Because of this, we must merge our init and start into Spring's init state, and then merge stop and destroy into Spring's close state. Recall our AbstractService#destory will auto invoke stop if it hasn't already done so. So this is trick that we need to understand in order to use Spring effectively.

Running services with JEE app server

In a corporate env, we usually do not have the freedom to run what we want as a stand-alone program. Instead they usually have some infrustructure and stricter standard technology stack in place already, such as using a JEE application server. In these situation, the most portable way to run POJO services is in a war web application. In a Servlet web application, you can write a class that implements javax.servlet.ServletContextListener and this will provide you the life-cycles hook via contextInitialized and contextDestroyed. In there, you can instanciate your ServiceContainer object and call start and destroy methods accordingly.

Here is an example that you can explore:

package servicedemo;
import java.util.*;
import javax.servlet.*;
public class ServiceContainerListener implements ServletContextListener {
    private static Logger logger = LoggerFactory.getLogger(ServiceContainerListener.class);
    private ServiceContainer serviceContainer;

    public void contextInitialized(ServletContextEvent sce) {
        serviceContainer = new ServiceContainer();
        List<Service> services = createServices();
        serviceContainer.setServices(services);
        serviceContainer.start();
        logger.info(serviceContainer + " started in web application.");
    }

    public void contextDestroyed(ServletContextEvent sce) {
        serviceContainer.destroy();
        logger.info(serviceContainer + " destroyed in web application.");
    }

    private List<Service> createServices() {
        List<Service> result = new ArrayList<Service>();
        // populate services here.
        return result;
    }
}

You may configure above in the WEB-INF/web.xml like this:

    <listener>
        <listener-class>servicedemo.ServiceContainerListener</listener-class>
    </listener>

</web-app>

The demo provided a placeholder that you must add your services in code. But you can easily make that configurable using the web.xml for context parameters.

If you were to use Spring inside a Servlet container, you may directly use their org.springframework.web.context.ContextLoaderListener class that does pretty much same as above, except they allow you to specify their xml configuration file using the contextConfigLocation context parameter. That's how a typical Spring MVC based application is configure. Once you have this setup, you can experiment our POJO service just as the Spring xml sample given above to test things out. You should see our service in action by your logger output.

PS: Actually what we described here are simply related to Servlet web application, and not JEE specific. So you can use Tomcat server just fine as well.

The importance of Service's life-cycles and it's real world usage

All the information I presented here are not novelty, nor a killer design pattern. In fact they have been used in many popular open source projects. However, in my past experience at work, folks always manage to make these extremely complicated, and worse case is that they completely disregard the importance of life-cycles when writing services. It's true that not everything you going to write needs to be fitted into a service, but if you find the need, please do pay attention to them, and take good care that they do invoked properly. The last thing you want is to exit JVM without clean up in services that you allocated precious resources for. These would become more disastrous if you allow your application to be dynamically reloaded during deployment without exiting JVM, in which will lead to system resources leakage.

The above Service practice has been put into use in the TimeMachine project. In fact, if you look at the timemachine.scheduler.service.SchedulerEngine, it would just be a container of many services running together. And that's how user can extend the scheduler functionalities as well, by writing a Service. You can load these services dynamically by a simple properties file.