Fuse Integration Services 2.0 is Out!

I’m happy to announce that JBoss Fuse Integration Service 2.0 has been released. The Fuse team has been hard at work bringing Camel 2.18, Spring Boot, to the OpenShift platform. This is the best platform to develop and operate integrations in a micro microservice architecture.  It lets you create tailored containerized integration applications that package only the middleware services that you need and no more.

What is Fuse Integration Services?

Fuse Integration Services (FIS) is a distribution of JBoss Fuse that provides tooling and runtime support for creating containerized integration services on OpenShift, including

  • Docker-formatted container images for a Fuse runtime
  • Tooling to create, develop and build containerized Fuse applications
  • Self-service deployment templates for common integration scenarios
  • Native integration with OpenShift for service discovery, clustering,and configuration management
  • All based on the core technologies (Camel, CXF, Karaf) available in JBoss Fuse

What’s New?

  • Support for Fuse on Spring Boot along with the latest version of Camel (2.18).
  • Completely overhauled templates and getting started experience for OpenShift.
  • FIS-specific tooling added to Fuse Tooling suite for Developer Studio.
  • A brand new Getting Started guide
  • Alignment with Fuse 6.3 for Karaf images to ease transition for existing Fuse customers using Karaf/Blueprint-based apps.
  • Uniform alignment with OpenShift source-to-image for both source and binary builds.
  • Improved quality and bug fixes across the board.

Ready to get started?

Check out the official docs

 

Apache Apollo 1.0 Released!

I’m pleased to announce the availability of Apache Apollo 1.0.  Apollo is a faster, more reliable, easier to maintain messaging broker built from the foundations of the Apache ActiveMQ project but with a radically different threading architecture which lets it scale to large number of concurrent connections and destinations while using a constant number of threads.

Apollo features:

Yes, it supports JMS!  Just download Apollo, start it up using the getting started guide and then check out the distribution’s examples directory.  Oh yeah, Apollo has great documentation!

I’ve re-run the STOMP benchmarks I covered in a previous post against the 1.0 release.  The benchmark uses the STOMP protocol to build an apples to apples performance comparison between all the major STOMP servers.  I’ve also built a new JMS benchmark which uses the JMS API build the same kind of performance comparisons.  Both benchmarks are open source and available to be tried out on your choice of hardware.  Or better yet, follow the simple instructions found in each project’s readme to run the benchmarks on EC2:

For those of you wondering why the ActiveMQ project create new message broker as a subproject, it’s because we wanted to address the shift in the processor market to multi core for the next major release.  ActiveMQ currently employs complex thread locking which starts to become bottleneck as you increase the core count and you don’t end up fully exploiting the potential on large core count machines.  By developing the solution as a subproject, it’s easier to explore the best options without impacting current main line development of ActiveMQ 5.x.

Now that the Apollo sub project has proven itself, I think it’s time to start integrating it into ActiveMQ.  Ideally, ActiveMQ 6 switches Apollo’s messaging engine and adds/ports all the existing features we’ve come to expect out of ActiveMQ like networks of brokers and priority queues.  But we should really be talking about this on the Apache mailing lists.. join me there!

STOMP Messaging Benchmarks: ActiveMQ vs Apollo vs HornetQ vs RabbitMQ

I’ve recently run STOMP benchmarks against the lastest releases of the 4 most feature packed STOMP servers:

STOMP is an asynchronous messaging protocol with design roots are based on HTTP protocol.  It’s simplicity has made the protocol tremendously  popular since it reduces the complexity of integrating different platforms and languages.  There are a multitude of client libraries for your language of choice to interface with STOMP servers.

The benchmark focuses on finding the maximum producer and consumer throughput for a variety of messaging scenarios.  For example, it benchmarks a combination of all the following scenario options:

  • Queues or Topics
  • 1, 5, or 10 Producers
  • 1, 5, or  10 Consumers
  • 1, 5, or 10 Destinations
  • 20 byte, 1k, or 256k message bodys

The benchmark warms up each scenario for 3 seconds.  Then for 15 seconds it samples the total number of messages that were  produced/consumed every second.  Finally, the destination gets drained of any messages before benchmarking the next scenario.  The benchmark generates a little HTML report with a graph for each scenario run where it displays messaging rate for each of the servers over the 15 second  sampling interval.

I’ve run the benchmarks on a couple if different machines and I’ve posted the results at: http://hiramchirino.com/stomp-benchmark/
Note: the graphs can take a while to load as they are being generated on the client side using the excellent flot javascript library.

Since anyone can get access to an EC2 instance to reproduce those results, the rest of this article will focus on the results of the obtained on the EC2 High-CPU Extra Large Instance.  If you want to reproduce, just spin up a new Amazon Linux 64 bit AMI and then run the following commands in it:

sudo yum install -y screen
curl https://nodeload.github.com/chirino/stomp-benchmark/tarball/master | tar -zxv
mv chirino-stomp-benchmark-* stomp-benchmark
screen ./stomp-benchmark/bin/benchmark-all

 

Note: RabbitMQ 2.7.0 sometimes dies midway through the benchmark.  It seems RabbitMQ does not enforce very strict flow control and you can get into situations where it runs out of memory if you place too much load on it.  It seems that crash becomes more likely as you increase the core speed of the cpu or reduce the amount of physical memory on the box.  Luckily, the RabbitMQ folks are aware of the issue and hopefully will fix it by the next release.

The ‘Throughput to an Unsubscribed Topic’ benchmarking scenario is interesting to just get a idea/base line what the fastest possible rate a producer can send to server.  Since there are not attached consumers, the broker should be doing very little work since it’s just dropping all the messages that get sent to it.

The Queue Load/Unload scenarios a very important to look at if your application uses queues.  You often times run into situations where messages start accumulating in a queue with either no consumers or with not enough consumers to keep up with the producer load.  This benchmark first runs a producer for 30 seconds enqueuing non-persistant messages and then runs a producer enqueuing persistant messages  for 30 seconds.  Finally, it runs a consumer to dequeue the messages for 30 seconds.  An interesting observation in this scenario is that Apollo was the only sever which could dequeue at around the same maximum enqueue rates which is important if you ever want your consumers to catch up with fast producers.

 

The Fan In/Out Load Scenarios help you look at cases where you have either multiple producers or multiple consumers running against a single destination.  It helps you see how performance will be affects as you scale up the producers and consumers.  You should follow the “10 Producers” columns and “10 Consumers” rows to really get a sense of which servers do well as the you increase the number of clients on a single destination.

The Partitioned Load Scenarios look at how well the server scales as you start to increase load on multiple destinations at different message sizes.

I’ve tried to make the benchmark as fair as possible to all the contenders, all the source code to the benchmark is available on github.  Please open an issue or send me pull request if you think of ways to improve it!

ActiveMQ Apollo Looking Impressive

ActiveMQ Apollo is a new generation of messaging broker built from the foundations of the ActiveMQ messaging broker, but using a radically different threading and message dispatching architecture.  In it’s current incarnation, Apollo only supports the STOMP protocol but just like the original ActiveMQ, it’s been designed to be a multi protocol broker and in future iterations it should get OpenWire support so it can be compatible with ActiveMQ 5.x JMS clients.

The BIGGEST change in the broker architecture was a switch to a non-blocking threading model for message processing using the HawtDispatch library.  This meant changing many of the APIs to using a continuation passing style so that caller never blocks on a request.  The upside of the new architecture is that the most complex and brittle areas of multi-threaded code in ActiveMQ 5.x  have now been tremendously simplified.  Those areas are using actor style coarse grained synchronization thanks to HawtDispatch.

It is impressive how well Apollo performs. Using a little benchmarking tool, I compared the performance of Apollo and three other STOMP server implementations.  The benchmark tests different combinations of consumers, destinations, producers, persistence options, message sizes for a total of 74 common usage scenarios.  I ran the benchmarks on two different boxes.  If you want to peek at the complete benchmark reports, see Ubuntu 4 Core report and OS X 8 Core report.

Apollo ends up doing really well in comparison to other servers in most cases but in some cases it’s just mind blowing.  For example:

The  scenario above is the case where you have 10 producer and 10 consumers on a single topic moving non persistent STOMP messages with a 20 byte payload.   The graph shows that Apollo can easily sustain a total consumer processing rate of 1.2 Million messages per second while the closest contender could only reach about 105,000 thousand messages per second.

RestyGWT 1.0 Released

I am pleased to announce the availability of RestyGWT 1.0.

RestyGWT is a GWT generator for REST services and JSON encoded data transfer objects.  What I really like about it is that it allows you to write rich JAX-RS based services and access those services from at GWT client reusing all the same DTO objects that you use on the server side.

In other words, it just as easy to use as GWT-RPC except it’s using simple RESTful URLs and JSON encoding of data when it access server side resources.  This has the nice benefit that you can implement the server side in something other than JAVA easily.

Further information see: