JMS

Variants of SEDA

17th April 2016 Architecture, EDA, JMS, Uncategorised No comments

SEDA (Staged Event Driven Architecture) was described in Ph.D. thesis of Matt Welsh. I recently read his blog post that describes various ways SEDA was perceived and implemented . One thing that comes to my mind after reading this is that SEDA as a tool has been used in more or less correct ways, as many other patterns and architecture models.

Because SEDA is a design model and architecture pattern, one that can be used with other models like SOA. In fact some ESB’a use SEDA as a processing model, like Mule ESB or ServiceMix. SEDA is one of a few processing models in ServiceMix. But SEDA can be used elsewhere, like with Oracle Service Bus. And in more that one way:

  1. SEDA using messaging – here we would add a queue before service that consumes event messages possibly using SOAP/JMS binding. We can that add additional stages for events processed by this first service.
  2. Throttling capabilities of OSB – here we do not have a real thread pool but using WebLogic’s Work Managers we achieve similar goal, threads will be taken from WebLogic thread pool. This processing model will work with every routing action

I also used a variant of the first model, where services were exposed as SOAP/HTTP services and they published a messages on a queue and then send reply that message has been queued. This way we can still used SOAP/HTTP messages, not forcing other systems to use messaging – either JMS, AMQP, STOMP or some other.

When done right SEDA will improve system’s scalability an extensibility. Scalability is higher because if a peak of messages happens than it will be queued up to the limit of storage quota. Messages that can’t be processed when arriving at a queue will wait for their turn. With addition of reliable messaging like JMS persistent delivery with exactly-once guarantee we can have scalable and reliable message processing mechanism. Adding additional queues depends on requirements of specif domain or application. It may be viable to add queues not only because of scalability but also from extensibility point of view. We would use topics here of publish event to multiple queues but the main point is that it would me possible and quite easy to add additional event consumer in this model.

SEDA may be also good for asynchronous request processing with long processing times with addition of NIO2 and Servlet 3.x asynchronous processing model. In this case we would accept request at some endpoint like Spring controller method and then invoke asynchronous method to do backend processing. HTTP could process another request for another potentially synchronous endpoint. Backend processing service would process incoming requests and invoke completion handlers as it finishes (using Spring infrastructure).  Here we would have at least two queues – HTTP thread queue and backend processing service queue. Both could be managed and adjusted to add higher scalability.

When considering SEDA it is important co remember that request processing will be lower that with processing with one thread and no queues on the way. This is quite obvious, as dispatching, consuming, confirming event message has it’s performance impact. SEDA can also impact system’s maintainability as event messages may get stuck in some queues or get redirected to dead letter queues and this must be monitored. Fortunately it is not a big problems, especially on OSB (we can use alerts) .

SEDA can be used incorrectly, just like EDA, CEP or SOA.  Once again architecture design is probably the most important thing in system’s development.

Have a nice day !

EDA in Java EE 7

3rd April 2016 EDA, Java EE, JMS, Software development 1 comment

Building EDA in Java EE 7 or Spring is easy. Let’s create a simple EDA routing example using JMS 2.0, EJB 3.2 and CDI 2.0. We will focus on messaging part, so we will not use anything fancy to route messages. We will publish messages to a single queue – the event channel. Messages will be received by EDA component which will route them according to header. Message will be published to appropriate event queue based on header value and will be received by appropriate component that will monitor those queues. Example will use queues but it can be easily modified to use topics.

A few important things:

  • What is important is to keep the architecture clean an event generator must not know anything about event consumers. Event instance should be created using an EventBuilder or some other helper class so that event source passes required information and does not know anything about details of EDA event construction.
  • Event consumer that does business processing must get business information from event and trigger processing. Do not implement business code in event consumer nor propagate EDA related details down to domain services layer.
  • It is important to do routing based on header if possible, as it is more lightweight.
  • Use appropriate session mode (transacted or no, with required AcknowledgementMode)
  • Remember that messages can and will arrive out of order
  • Use one or a few Dead Letter Queues, sent messages that failed to be processed a few time to related DLQ. Example code does not do this

EDA - simple routing

 

Publishing message using EJB is super easy:

Important parts are discussed below.

Creating JMSContext that manages JMS Session and ConnectionFactory for us. This must NOT be created in session transacted mode in order for it to join JTA transaction. There are cases where we want to manage transactions our selves or send message whether or not other parts of request processing failed. In those cases we could use separate transaction (e.g. by setting TransactioAttribute to REQUIRES_NEW  in CMT model or JTA 1.2 Transactional annotation) or we could create a transacted session and commit local JMS transaction in publisher code. In most cases we want to publish events as part of some other processing in transaction, so we must join existing JTA transaction.

Thanks to AUTO_ACKNOWLEDGE mode session will not be transacted and will join existing JTA transaction. We can use this to span XA transactions if it would be required.

Creating message and setting routing header:

and finally sending message:

Event can be send fully asynchronously in Java EE. In code example above client code will block until JMS provider acknowledges message. Starting from Java EE 7 we can send message and continue processing.

Publishing a message from a CDI Bean is similar, notable detail is the use of @Transactional, a JTA 1.2 annotation that allows CDI beans to work in transaction context (but not in case of life cycle methods – here EJBs still have to be used).

Messages will be received by Message Driven Bean. One of the strengths of using MDB component is that messages are delivered asynchronously with possibility to scale MDB resource share (thread pool, component instance pool). EJB container take care of other thing that we must think of when using CDI bean like transaction management and related message acknowledgement. A lot of configuration can be done declaratively using @AnnotationConfigProperties. But I don’t intend to compare CDI with EJB here, so let’s see the code:

MDB implements MessageListener to clearly state it is a JMS MDB, as MDBs can also connect to other resource manager types. onMessage method will be invoked to process each message and each message will be processed separately, even though messages will be fetched in batches from JMS Provider. Default acknowledgment mode for MDB component is AUTO_ACKNOWLEDGE with DUPS_OK_ACKNOWLEDGE allowed to be set.

SimpleRoutingEventMessageBroker can be source of other events. How you process events is system specific. In some cases simple XML or JSON routing specifications will be sufficient, in other cases Business Rules engine will help go trigger events. CEP (Complex Event Processing) is different league.

As comment states in code above MDB component that gets messages from event channel should not take care of message processing or routing. Message processing should be executed in dedicated component so that we can use different routing strategies and component will be easier to maintain. Testing routing code separately of JMS can also be a plus – although we can test it with JMS this test will be slower than regular JUnit tests.

Event consumer implementation is similar to SimpleRoutingEventMessageBroker MDB:

Message consumer CDI component is another story – see code below:

We must receive messages manually and we can and should give a timeout for receive operation. This component is transactional also, but it must be called by some other component in order to receive messages.

We can test code above using Arquillian. In case of both publishers we will test if message is not send if an exception is thrown.

Second test is similar, so if you’re interested to check github.

In come cases it may make more sense to use lightweight integration framework like Spring Integration or Apache Camel to create internal EDA component. Most efficient tool should be used to do the job.

Have a nice day !