DDSL – Dynamic Distributes Service Locator – enables your applications to be discovered by other applications or reverse-proxies like nginx without any manual reconfiguration. DDSL uses Apache Zookeeper to make Service Discovery possible in a distributed way without any single point of failure.

When using DDSL you can automatically scale your system up by just starting a new instance of one of your applications on some random machine, since the built-in Load-balancer in DDSL will automatically start sending traffic to it.

I have recently created and release a DDSL plugin for Dropwizard which makes it really easy to add the functionality to your existing Dropwizard-applications.

If you are more into Play Framwork, choose the 1.x-module or 2.x-plugin, and you can of course use it in any of your X-technology Java- or Scala-applications.

If you have any questions or comments, please comment here on this blog or contact me via github or twitter.

GT-Engine is a new implementation of the default groovy template engine in Play Framework 1. GT-Engine is faster and uses less memory.

You can now use GT-Engine in Play 1 using the module faster-groovy-templates which is a drop-in-plugin replacing the default template implementation.

UPDATE: The module is now available in the Play Framework module repository

There also exists a Play 2 module that makes it more easy for you to port your Play 1 application to Play 2.0.

I have just released a beta version of the Groovy Template module for Play Framework 2.0 called gt-engine-play2.

The current project-status is Beta as of 24. january 2012.

It is based on the gt-engine described in this blog post.

It should be feature complete (for Java) but please have a look at the readme file over at github – and the issues-list. When you find new issues, please report them via github.

Please try it out and start porting you Java Play 1.x app to Play 2.0

-morten

UPDATE:

Play 1 module available here: https://github.com/mbknor/faster-groovy-templates

Play 2 module available here: https://github.com/mbknor/gt-engine-play2

Why re-implement the Groovy Template Engine?

Play Framework version 1.x default template engine is based on Groovy. Play 2.0 has a new template system based on Scala but we want to make it possible to migrate from Play 1.x to 2.0 without having to rewrite all of your old Groovy Templates.

When investigating how to use the Groovy template engine code from Play 1.x in Play 2.0 it turned out to be difficult since the implementation is really tight coupled with the entire Play 1.x code base.

This made me start a (too big) project to re-implement the entire Groovy Template Engine making sure it was not using any of the other Play code. This would make it possible to use the exact same implementation in Play 2.0. It aims to be 100 % compatible with existing templates/applications.

When doing this reimplementation I have managed to make it faster (and using a little less memory i think ). I’ll probably write another blogpost later about how it is implemented but basically I precompile the template source down to both java and as little groovy code as possible. See at the bottom for benchmark results.

I think the new implementation is feature-complete (But Beta) and we’re using it in development at work. But there is still work to do:

• checking/making sure I have not broken too much api
• improve/clean api/code
• Moving the code to external project so Play only uses a jar-file
• Add code to Play 2.0 to use the groovy engine.

(New feature: If you add ‘save-gttemplate-source-to-disk = true’ to application.conf, the generated java and groovy source code is dumped to disk so you can step-debug it if you want.)

The current source (Based on Play 1.2.4-RC2) can be found https://github.com/mbknor/play/tree/new_groovy_template_impl (UPDATE: Remember to use the correct branch ‘new_groovy_template_impl’).

Please try it out and report any problems or suggestions/thoughts.

Some limitations regarding Groovy code snipits %{ ..code.. }%:

• You can no longer use partial-program-flow-code like this:

  %{ if( expression) { }% or %{ myList.foreach() { }%

• Instead you have to use:

  #{if expression} or #{list myList}

• Also you cannot use def or declare type when creating variables if you want it to be visible outside the script-block. (Not a limitation anymore)

Benchmark

The following benchmark results are not scientific at all but I made them just to try to show the difference between old and new implementation. It is based on Play 1.2.4-RC2.

The test application can be found at github

This is how I performed the tests

I started Play like this: play run –%prod -XX:CompileThreshold=10

I used CompileThreshold to shorten the needed jvm warm up time.

For each speed-benchmark i restarted play, ran 14000 request using ab (40 concurrent), then did another 14000 request and recorded the stats.

For memory-benchmarks I restarted Play, then recorded heap usage with jvisualvm during the first 14000 requests. All GC settings are default.

Detailed info about the speed benchmark can be found here.

Google spreadsheet with useful numbers can be found here.

Each test shows two numbers:

• requests pr second
• total “rendering time” for all 14000 requests. (total time for test minus total time for noTemplate-test: Shows aprox time spent with rendering without time spent in framework itself)

minimalTemplate

ab -k -c 40 -n 14000 http://localhost:9000/application/minimalTemplate

Requests pr second

Total “Rendering time”

normalTemplate

ab -k -c 40 -n 14000 http://localhost:9000/application/normalTemplate

Requests pr second:

Total “Rendering time”

normalTemplateBigData

ab -k -c 40 -n 14000 http://localhost:9000/application/normalTemplateBigData

Requests pr second:

Total “Rendering time”

normalTemplateBigDataWithJavaExtensions

ab -k -c 40 -n 14000 http://localhost:9000/application/normalTemplateBigDataWithJavaExtensions

Requests pr second:

Total “Rendering time”

normalTemplateBigDataInTag

ab -k -c 40 -n 14000 http://localhost:9000/application/normalTemplateBigDataInTag

Requests pr second:

Total “Rendering time”

normalTemplateBiggerData

ab -k -c 40 -n 14000 http://localhost:9000/application/normalTemplateBiggerData

Requests pr second:

Total “Rendering time”

Memory info

normalTemplate

1.2.4-rc2

new impl:

normalTemplateBigData

1.2.4-rc2

new impl:

normalTemplateBigDataInTag

1.2.4-rc2

new impl:

Some last words

I hope this info was useful (It actually took a long time to perform the tests and write this blog ). Also, I hope you download the custom play version and help me find problems not yet found.

Interesting read from a colleague of mine over at sickenger.com

Have a look at DDSL – Dynamic Distributed Service Locator over at GitHub

There you can find working code, README and example with step-by-step guide.

Official post coming later when it is more complete..

A very good colleague of mine wrote an excellent post about my maven plugin called ‘deptools’.

You can read it on his blog

I have now moved my blog away from moaxstuff.blogger.com to my own WordPress hosted by myself… Even though my posts is not that interesting, it was almost impossible to find it on google.com – You had to specific search in “blogs” to find it…

At work we have 6 web applications (WAR) deployed in Glassfish v2.

In production we experienced sporadic java.lang.OutOfMemoryError: Java heap space under high load. We where sure that we did not have a classic Java memory leak since the used HEAP space decreased after some time and returned to normal. We suspected that the problem was related to our use of EHCache (which stores cached objects in HEAP space).

(Note: This blog post is a summary of two days of research – we tried many things and many numbers – on several more or less identical servers – so the numbers and values in this post is approximately correct – but you get the point)

The JVM, and therefor GlassFish, was running with max 768 MB of HEAP space and 385 MB of PermGEN space.

To reduce the possibility of getting OutOfMemoryError until we had worked out the issue, we decided to increase the maximum HEAP space the JVM could allocate (-Xmx).

GlassFish runs on a 4 core 32 bit server with Windows Server 2003 with 8 GB of RAM.

We increased the the max HEAP memory size to 1 GB (-Xmx=1024M). We started GlassFish with no applications deployed. Then deployed one application after the other – 6 WARs. All applications where deployed without problem and our Apps run fine. After some time the JVM suddenly died.

We found a crash JVM crashdump. We didn’t read it to carefully, but it talked about OutOfMemoryError. We did some more research and found out that it had died before the HEAP space had reached 1GB. We thought a solution was to set the initial HEAP size that the JVM should initialize. We told the JVM to initialize all the HEAP at startup (-Xms=1024M).

So now we had 1024MB HEAP and 385MB PermGEN which is a total of 1409MB.

When we then again started GlassFish (with no apps deployed), The JVM and GlassFish started up just fine. So we started to deploy applications – one by one. When GlassFish was in the middle of deploying the second application the JVM died.. So by allocating more memory up front, the JVM died with OutOfMemoryError earlier..

After a lot of research and reading this great post: http://www.codingthearchitecture.com/2008/01/14/jvm_lies_the_outofmemory_myth.html, this is how we concluded:

We took a closer look at the JVM crash dump:

java.lang.OutOfMemoryError: requested 884680 bytes for Chunk::new. Out of swap space?

It also says that the JVM crashed in this thread:

0x5be76800 JavaThread “CompilerThread1″ daemon [_thread_in_native, id=6764, stack(0x5c1a0000,0x5c1f0000)]

We had configured the JVM to use a lot of memory for HEAP and PermGen. A Windows process can use max 2 GB total. The internals of the JVM (e.g its JIT compiler) needs its own memory, so do the DLLs loaded. Since so much of those 2GB was already allocated for HEAP/Permgen, windows said NO when the JVM asked for more memory inside CompilerThread1. When this happened, the JVM crashed with java.lang.OutOfMemoryError?: requested 884680 bytes for Chunk::new. Out of swap space?

Solution:

Tell the JVM to use LESS memory..