When implementing event based servers, there are generally 2 patterns used, the reactor pattern and the proactor pattern.  The reactor pattern can be though of as being a synchronous version of the proactor pattern.  In a reactor pattern IO events are serviced by the thread in the IO handling loop.  In a proactor pattern the thread processing the IO event loop passes off the the IO event to another thread for processing.  HawtDispatch can support both styles of IO processing.

HawtDispatch uses a fixed sized thread pool sized to match the number of cores on your system.  Each thread in the pool runs an IO handling loop.  When a NIO event source is created, it gets assigned to one of the threads.  When network events occur the source causes callbacks to occur against on the dispatch queue targeted in the event source.  Typically that target dispatch queue is set to a serial queue which the application uses to handle the network protocol.  Since it’s a serial queue, the handling of the event can be done in a thread safe way.  The proactor pattern is being used since the serial queue can execute in any of the threads in the thread pool .

To use the reactor pattern, HawtDispatch supports ‘pinning’ the serial queue to a thread.  When a dispatch source is created on a pinned dispatch queue, then the event source gets registered against the same ‘pinned’ thread.  The benefit of the reactor pattern is that it avoids some of  cross thread synchronization needed for the proactor pattern and provides cheaper GCs.  The down side to the reactor pattern is that you may have to manage reblanacing network sources across all the available thread.  Lucky HawtDispatch does support moving pinned dispatch queues and sources to different threads.

A few months ago, I experienced similar results while hacking on ActiveMQ.  I noticed that passing messages from producer connections to consumer connections was dramatically faster if the producer and consumer were being serviced by the same thread.  I decided that the next version of the broker would to be need to be built using a thread management framework which could optimize itself so that those connections could collocate onto one thread if possible.

Then I saw the the libdispatch API (it forms the foundation of the Grand Central Dispatch technology in OS X) and fell in love with it’s simplicity and power.  I realized that implementation of that API could in theory provides the threading optimizations I was looking for.  So I started hacking on HawtDispatch, a Java/Scala clone of libdispatch.

The central concepts to libdispatch and hawtdispatch are global and serial queues.  Global queues are executors which execute tasks concurrently using a fixed size thread pool. Serial queues are executors without an assigned thread and which execute tasks in FIFO order.   When tasks added to a serial queue, the serial queue gets added to a global queue so that the serial queue can execute it’s tasks.  Multiple serial queues execute concurrently on the global queue.

The overhead of a serial queue is very small, it’s just a several counters and a couple of linked lists.  You can use them like a lightweight thread. Feel free to create thousands of them.  If you squint at it just right, they allows you to use erlang style actor thread model.

Now that you have an idea how HawtDispatch is used, lets get back to what kinds of optimizations it can do to help with cross thread contention.  HawtDispatch generally uses a concurrent linked list to enqueue a task in serial queue, but there are times when it can avoid that synchronization of the concurrent linked list.  For example, if the serial queue is currently executing in the current thread, then an enqueue can just add the task to a non synchronized linked list.  HawtDispatch also supports ‘pining’ a serial queue to one of the threads in the global queue’s thread pool.  This allows you to force serial queues to collocate onto one thread so that when they do need to communicate, there is no thread contention involved.

But you still run into cases where you need to move tons of events from one serial queue to another which is executing on a different thread.  For these cases, you use a custom event source.  It allows you to  coalesce a bunch of events generated on on thread as a single event delivered to the another queue.  HawtDispatch will aggregate custom events  into a thread local (to avoid contention) and once the current thread has drained all execution queues, it will deliver those custom events to their target queues.

This post is already getting kind of long, so I’ve have to do a follow up post on how all that interacts with network IO events.  But the general idea is, yes, keeping stuff on 1 core is fast, but it won’t scale once your CPU bound, so having a framework like can HawtDispatch help minimize cross thread contention while still providing the ability to scale up to multiple cores as load increases.

Progress Software is sponsoring an Apache ServiceMix, ActiveMQ, CXF & Camel Community Day on Thursday, December 10th, at the Hyatt Hotel in Burlingame. Join us at this free event and meet committers and founders of Apache ServiceMix, ActiveMQ, CXF and Camel that have successfully implemented enterprise application and deployed these projects in production.

Should fun to meet users/developers of these kick ass Apache based projects.  If you plan on going, make sure your register for the event.  It’ll be nice to meet everyone!

Counter that with latest AJAX rage, RESTful services with a JSON or XML representation of the data.  The nice thing about using JSON with REST services is that clients and servers can easily be built it any language.  Wouldn’t it be nice if you use JSON RESTful services from GWT with the ease of GWT RPC?

That’s exactly the reason I started hacking on RestyGWT.  It uses a very similar development model as GWT RPC.  You create a service interface who’s implementation is code generated via Deferred Binding when your GWT module is built.  The developer maps the interface methods to RESTful HTTP requests using the excelent JAX-RS annotations.  Just like in GWT RPC, the interface must be ‘asynchronous’.  You must use callback method arguments to get the request results.  But unlike GWT RPC, the data sent and received will be JSON encoded.  RestyGWT will automatically code generate JSON encoders and decoders for any classes sent or received from the rest service.

For more details see the Git Hub home page at: http://github.com/chirino/resty-gwt

As of this point, RestyGWT only implements the most basic JAX-RS annotations and provides a very simple JSON encoding scheme.  Despite that, the project is already extremely useful.  I would love here what folks think about it.

It also supports and even simpler interface remoting model which does not mandate the throwing of RemoteException or extending the RemoteObject interfaces. It even allows remoting classes which don’t implement any interfaces via ASM magic.

Documentation still needs a little work, but the project is now ASL 2.0 Licensed and looking for contributions! Please join the mailing lists to get involved.

So you might be asking yourself, what’s the secret sauce? Well before I get into that, let me first explain the class model that our proto compiler generates.

For every message definition in the ‘.proto’ file, the compiler will generate 3 classes:

• the message interface: is implemented by the bean and buffer classes. It has all the ‘getters’.
• the bean class: has all the ’setters’ and ‘merge’ methods
• the buffer class: has all the encoding and decoding methods. It does not allow mutation.

The message interface also defines the freeze(), frozen(), and copy() methods which allow you to make an instance immutable, check to see if an instance is immutable, and create a mutable copy. Buffer classes are alway immutable. Bean class can transition to being immutable via freeze(). freeze() naturally returns a buffer object. copy() naturally returns a bean object.

This bean model gives substantial flexibility. Besides making it easy to transition from immutable to mutable and back, the message interface lets you implement business methods that operate against either type of instance. You could use the bean class purely in a builder style to always generate a buffer instance, or you could just use them like traditional java bean objects.

Once a bean instance is frozen, any attempts to modify the instance will throw assertion errors if assertions are enabled in your JVM. So the CPU cost of validating program correctness can can be disabled at run time.

Finally, the most important feature of the buffer class is that it holds on to either the byte array that it was created from or the frozen bean that created it, and sometimes both, after it builds one from the other. This has several implications. Firstly, once a buffer is encoded to a byte[], subsequent encoding passes are free. This is also true when a buffer is decoded, as the next encoding is free since it still retains the original encoding of the message. And the other benefit that this provides which the benchmark highlighted, is that deferred decoding is possible. A newly created buffer class will not decode the data until a field is accessed. This also true of the nested messages that are encoded in a buffer. While the outer message may get decoded, the nested message will not be decoded until it’s fields are accessed.

I’ve had been researching the protobuf encoding format for a while and really liked it. But I did not really like the Java implementation that Google had published. It was kinda clunky to use and I saw several optimizations that could be used that were missing. Optimizations that could create huge performance wins when applied to the usage patterns of an enterprise messaging system like Apache ActiveMQ. So I created a new protobuf implementation in the ActiveMQ project.

Naturally, I was curious to see how the activemq protobuf implementation stacked up against similar technologies. So I grabbed the V1 benchmark source code and added our implementation to it. If you want to do the same, apply this patch.

Once I ran the benchmark and I was very pleased with the results. I’m including the performance graphs of our impl and standard protobuf and thrift for comparison.

As it turns out, our implementation looks awesome in the benchmark! How about that decoding speed!

It’s getting late here.. so I’ll have do a follow up post explaining how come we did so much better.

It’s called Jansi and it provides support for using ANSI escape sequences in your Java console applications on Windows.

With ANSI escape sequences, you can fully control the the cursor positioning and the foreground and background color of the console text output. Here is quick example of what’s posssible:

Oh and it’s an Apache Project now. Yay! Seems like the project website is still not fully setup since they are migrating from SourceForge to Apache, be here’s a link to the source tree.

In theory, this is supposed to increase the communication between the developers. In practice, it’s looking a little scary. For example, I just wanted to move some ActiveMQ integration modules from living in the ActiveMQ project to live in the Geronimo project. Now, before I submitted the patch, everyone agreed that it was good idea. I submitted the patch and now I have been waiting 12 days to get enough +1s.

Why is it taking so long? I think it’s because the Geronimo project is bigger than what anybody knows. But we are lucky to have a large developer base that is specialized in different areas of the server. This specialization is what allows us to produced a server with high quality parts that in turn give us a high quality server. I don’t go wasting time figuring how what’s the best way to implement a Transaction Manager or Deployment system, because I trust the guys that are already working on those pieces.

With the RTC policy, if I we want to get Transaction Manager patch in, folks that might not know ANYTHING about Transaction Manager are going to be looking at those patch and pretending they understand what’s going on and then giving the +/- 1’s. The same thing goes for the developer tooling, JMS Messaging, etc. etc.

I guess in the long run, it may be good that folks get to know other Geronimo systems besides what they specialize in, but in short term, your going to slow down your development model by at least a factor of 10 (I’ve already been waiting over 10 days to do something that used to take me 10 seconds). Furthermore, since Geronimo is playing feature catch up, most developers already have enough on their plates, I doubt they also want to review and test other folks patches x3 if the previous development pace is to be sustained.