Saturday, May 4, 2013

Carry your sword and dagger programmers!

A knight would for sure carry his sword proudly and ready for any battles. But then he can not use such a heavy weapon for daily tasks such as peeling an apple. Sometimes I think the same with ourself as programmers. We need a stable, strong and static typed programming language that can get us the heavy job (large project, and enterprise system) done in more manageable way. But yet, we face many smaller daily tasks (eg: parse a text file for a search value, or generate sample data etc) that is better fitted to use a lighter, dynamic language because they are faster to write; and the code need not go through heavier process as the main project.

Through out my career, I have been keeping two languages (one strong typed and one dynamic typed) pair very up to date. I would learn it and be proficient with it enough to write code without flipping through a book. I started with C++ with Perl and then switched to Python. And then later with Java and Jython, and later with Groovy. These are my strongest languages that I used the most, especially with Java. Of course I never stop learning other languages as well, such as Scala and Ruby etc. I personally think Ruby is very nice and good language to learn and use. I just not have had a chance to use it that extensively yet. Mastering a full static language such as Java would take much longer time, but one can learn a dynamic language fairly quickly.

Any rate, if you a programmer, I strongly encourage you to learn at least two languages, and learn it well. As I said, prefer one static and one dynamic language. This pair of combination will boost your productivity to next level.

Thursday, May 2, 2013

How to bring pseudo code to life

When designing a piece of software, I like to start with some pseudo code first, then work my way out. I tend to try mapping out the normal workflow (one that runs without any corner cases) with high level of actions as series of steps. Then the detailed implementation of the code can be filled in at later time.

Now with Java, you can easily do this with pseudo code, bring it to life, and still make it very maintainable. Try this out.

package deng.javademo;

/**
 * @author Zemian Deng
 */
public class PseudoCodeToLife {
    public static void main(String[] args) {
        CreditCardProcessor creditCardProcessor = new CreditCardProcessor()
                .step(webServiceRequest())
                .step(validateCardNumber())
                .step(validateSecurityCode())
                .step(checkAvailableCredit())
                .step(chargeCard())
                .step(webServiceResponse());

        creditCardProcessor.run();
    }
}

Now that's as high level as you can get, but it still let me fill in the implementation without losing the overall requirement flow. To implement such workflow, we need a contract that allow processor to move data from one step to another, and yet it needs to perform some action along the way. How about an interface like this.

package deng.javademo;

/**
 * @author Zemian Deng
 */
public interface StepAction {
    public void onAction(Exchange exchange);
}

Next is how to wire steps together. For now we can just do it inside CreditCardProcessor, but you can easily abstract that into a re-usable base class.

package deng.javademo;

import java.util.ArrayList;
import java.util.List;

/**
 * @author Zemian Deng
 */
public class CreditCardProcessor {
    List<StepAction> steps = new ArrayList<StepAction>();

    public CreditCardProcessor step(StepAction stepAction) {
        steps.add(stepAction);
        return this;
    }

    public void run() {
        Exchange exchange = new Exchange();
        for (StepAction step : steps) {
            step.onAction(exchange);
        }
    }
}

Pretty straight forward. Here I used an Exchange as message, passing between each steps to allow the workflow to exchange data. You can wrap just about any real payload and/or add a Map of header properties.

Now what we are missing is just filling in each step of actions. Since we have an explicit step name when defining the workflow, we can just generate these method and fill in the actions per step required. I will add them all inside the PseudoCodeToLife and reprint it here.

package deng.javademo;

/**
 * @author Zemian Deng
 */
public class PseudoCodeToLife {
    public static void main(String[] args) {
        CreditCardProcessor creditCardProcessor = new CreditCardProcessor()
                .step(webServiceRequest())
                .step(validateCardNumber())
                .step(validateSecurityCode())
                .step(checkAvailableCredit())
                .step(chargeCard())
                .step(webServiceResponse());

        creditCardProcessor.run();
    }

    private static StepAction webServiceRequest() {
        return new StepAction() {
            @Override
            public void onAction(Exchange exchange) {
                System.out.println("webServiceRequest step.");
            }
        };
    }

    private static StepAction validateCardNumber() {
        return new StepAction() {
            @Override
            public void onAction(Exchange exchange) {
                System.out.println("validateCardNumber step.");
            }
        };
    }

    private static StepAction validateSecurityCode() {
        return new StepAction() {
            @Override
            public void onAction(Exchange exchange) {
                System.out.println("validateSecurityCode step.");
            }
        };
    }

    private static StepAction checkAvailableCredit() {
        return new StepAction() {
            @Override
            public void onAction(Exchange exchange) {
                System.out.println("checkAvailableCredit step.");
            }
        };
    }

    private static StepAction chargeCard() {
        return new StepAction() {
            @Override
            public void onAction(Exchange exchange) {
                System.out.println("chargeCard step.");
            }
        };
    }

    private static StepAction webServiceResponse() {
        return new StepAction() {
            @Override
            public void onAction(Exchange exchange) {
                System.out.println("webServiceResponse step.");
            }
        };
    }
}

That's all to this. What do you think? I think it's a simple way to tackle business problems. Simple software make less bugs and easier to maintain.

Thursday, March 28, 2013

First snapshot of myschedule-3.0.0 is available

First snapshot of myschedule-3.0.0 with brand new UI is ready to be try out!

It's been a long time since I made a release for MySchedule project, and I have been busy!

Well, for starter, I have been exploring the Vaaddin library for a new UI design for the project, and it's working really nice! I love to have full Java code for UI layout and widget controls and yet it runs on browser. It's a perfect fit for MySchedule. As you will see from the snapshot, the UI is still pretty bare, but for me, as backend developer, and be able to have such clean UI code is super exciting. (on other hand, working with JavaScript, even with such nice library as jQuery can still feel like a hack at times. ^_^) You should checkout the latest MySchedule source to take a look yourself. All that GUI is less than handful of UI classes. Vaaddin is pretty sweet!

As MySchedule jumped to 3.x version, I've taken the opportunity to clean up few areas in the "quartz-extra" module as well. Also to support Vaadin, the web layer needed a rework. The new code is easy to follow and ready to support more UI features in the future.

I also improved on the project packaging. Now getting started with is even more easier. Get the single zip file, and try any of these simple steps:

## Web application usage (option 1: power up a self-contained servlet server!)
 
 bash> cd myschedule-3.*
 bash> bin/myschedule-ui.sh --httpPort=8081

Now open a browser and visit http://localhost:8081

## Web application usage (option 2: use your own servelt server)

Simply copy the myschedule-3.*/war/myschedule.war file into your Servlet container such as Tomcat.

## Command line usage (option 3: quickly test your quartz config)

 bash> cd myschedule-3.*
 bash> bin/myschedule.sh bin/quartz.properties

Note that not all MySchedule-2.x features are in the 3.0 snapshot yet. For now, you would need to use the ScripConsole window to enter jobs into the scheduler. There are now new templates UI available for you to choose and start working quickly. I am still working on allowing users to save their own templates and edit them. This should give users more rich experience in using the UI manager.

Friday, March 1, 2013

Hg shortcuts for bash .profile

Some shortcuts I used most often when working with Mecurial (hg) source control.
# Hg shortcuts
function splithgfiles() { ruby -ne 'if $_ =~ /files: / then puts $_.split else puts $_ end' ; }
alias hgpu='hg pull -u'
alias hgl='hg log -v -l 5 | splithgfiles'
alias hgl1='hg log -v -l 1 | splithgfiles'
alias hgu='hg update'
alias hgc='hg commit -m'
alias hgb='hg branches'
alias hgbm='hg bookmarks'
alias hgt='hg tags'
alias hgs='hg status'
alias hgsu='echo "# Summary" && hg summary && echo "# Heads" && hg heads'
alias hgr='hg revert -C'

# Remove unversion files from Hg repository dir.
function hgrmnew {
 rm -vfr `hg st | cut -d ' ' -f 2`
}
One worth special mentioning is that default "hg log -v" shows files with space separator. I find it easier to view with one file per line instead, hence I added the "splithgfiles" helper function.

Thursday, February 21, 2013

Few notes about Java clone() method

Did you know you can create new Java instance without constructor? That's right, and this feature is brought to you by the Java clone() method. I personally don't like to use it much because it has many pitfalls, and it feels very broken in many ways. You probably can write better "clone" by using copy constructor or a static copy factory methods.

Anyrate, here is a unit test that highlights few notes I have gathered about Java clone. Look for the "Note" comments below.

package atest;

import org.junit.Test;

import java.util.ArrayList;
import java.util.List;

import static org.hamcrest.Matchers.hasItems;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;

public class CloneTest {
    @Test
    public void testFooClone() throws Exception {
        Foo a = new Foo();
        Foo b = a.clone();

        assertThat(a != b, is(true));
        assertThat(a.constructorCallsCount, is(1));
        assertThat(a.name, is("foo"));
        assertThat(b.constructorCallsCount, is(1)); // Note1: Wow, a new instance without calling constructor!
        assertThat(b.name, is("foo"));

        a.name = "fooX";
        assertThat(a.name, is("fooX"));
        assertThat(b.name, is("foo"));

        Foo2 c = new Foo2();
        c.ids.add("a");
        Foo2 d = c.clone();
        d.ids.add("b");

        c.ids.clear();
        c.ids.add("A");

        assertThat(c != d, is(true));
        assertThat(c.constructorCallsCount, is(2));
        assertThat(c.name, is("foo"));
        assertThat(d.constructorCallsCount, is(2));
        assertThat(d.name, is("foo"));

        assertThat(c.ids, hasItems("A"));
        assertThat(c.ids.size(), is(1));
        assertThat(d.ids, hasItems("a", "b"));
        assertThat(d.ids.size(), is(2));
    }

    /** A simple clone examples with simple type fields. */
    public static class Foo implements Cloneable {
        static int constructorCallsCount = 0;
        String name = "foo";
        public Foo() {
            constructorCallsCount++;
        }
        public Foo clone() {
            // Note2: Catch checked exception here so client or subclass doesn't need to.
            Foo result = null;
            try {
                result = (Foo)super.clone();
            } catch (CloneNotSupportedException e) {
                throw new RuntimeException("Unable to clone.", e);
            }
            return result;
        }
    }

    /** A clone example that needs to fix field afterward super.clone(). */
    public static class Foo2 extends Foo {
        List<String> ids = new ArrayList();
        public Foo2 clone() {
            Foo2 result = (Foo2)super.clone(); // Note3: Some how, super clone will auto return the correct type!
            result.ids = new ArrayList(ids);   // Note4: need to fix non-simple field.
            return result;
        }
    }
}

Monday, February 4, 2013

Learn how to grep by context will give you better result

I learned that grep command supports few options called "context" searching. Basically it can print lines around the matching string! For example, you can use this feature to perform a quick Java threads stack analysis like this (assuming you have a java process PID=12345).


bash> jstack 12345 | grep --color 'State: BLOCKED' -B 1
"A-Bad-Boy-Thread" prio=5 tid=0x00007fcfbc895800 nid=0xcd03 waiting on condition [0x00000001657cb000]
   java.lang.Thread.State: BLOCKED (on object monitor)

See, without the "-B 1" option, which show one line before match, you won't know which name of the tread that's acting up! Now isn't that a gem!

You may also use "-A 5" to see 5 more lines after the match. Or you may use the "-C 5" to see 5 lines before and after the match.

PS: There is nothing wrong with a "BLOCKED" thread in Java, however if you got one that won't go away, then it's a good indicator that something is fishy about that thread in your application; because in a well designed app, threads should be in BLOCKED state as briefly as possible. Read the javadoc on java.lang.Thread.State for more details.

Tuesday, January 29, 2013

Fast shell utilities to report Maven's unit tests results

I would like to share couple shell utilities that I have collected. These are for fast Maven multi modules reporting from your unit tests results, without having you to run entire Maven site reports, which can take a LONG time to generate if you have a large project! They should work on Linux or Window's Cygwin shell.

# Show failed tests among all the surefire results.
function failedtests() {
    for DIR in $(find . -maxdepth 3 -type d -name "surefire-reports") ; do
        ruby -ne 'puts "#$FILENAME : #$&" if $_ =~ /(Failures: [1-9][0-9]*.*|Errors: [1-9][0-9]*.*)/' $DIR/*.txt
    done
}

# Show the top tests that took the longest time to run from maven surefire reports
function slowtests() {
    FILES=''
    for DIR in $(find . -maxdepth 3 -type d -name "surefire-reports") ; do
        FILES="$FILES $DIR/*.txt"
    done
    head -q -n 4 $FILES \
        | ruby -ne 'gets; print $_.chomp + " "; gets; print gets' \
        | ruby -ane 'printf "%8.03f sec: ", $F[-2].to_f; puts $_' \
        | sort -r \
        | head -10
}

When developing with Maven, you often want to see a summary of failed tests, and you want those surefire TXT file content to see what's going on. The failedtests function will give you a list of all the failed tests filenames in all modules; and then you can cat each one to investigate.

With slowtests function you may quickly see the top 10 most time consuming tests from your project!

Enjoy!


Updated (2013/01/30)

I found out that the head command on MacOSX doesn't have the "-q" option and it always prints the "==> filename <==" lines. How annoying! To work around this, you can use this ruby command replacement instead:
# Replace this
head -q -n 4 $FILES \

# With this
ruby -e 'ARGV.each{ |n| File.open(n) {|f| 4.times{ puts f.readline}} }' $FILES \

It's a tad longer, but it's PORTABLE!