DTOs on N-Layered Domain Driven Design

One of the key reasons I became a fan of DDD is the repeatable tactical patterns when it comes to code structure.

After reading Vaughn Vernon's book I looked for an even more tactical book to demonstrate these patterns. A good friend of mine recommended N-Layered Domain-Oriented Architecture Guide and I was lucky enough to take his advice. This book really takes you step by step on how to structure your source code up to the smallest details.

Data Transfer Objects and DDD

When it comes to the application layer, one of it's key roles is the generation and handling of DTOs.

Domain Oriented N-Layered Architecture

The adapters component is usually used by the application services to generate the DTOs after they were fetched using the domain layer repositories or when a response from a domain service is received.

DTOs are usually used for minimizing remote calls overhead (as described in the great Patterns of Enterprise Application Architecture) or to customize the returned data to clients, but on one of my latest projects I also used them for another use case, which IMHO, expanded the combination of  DTOs and N-Layered pattern a little bit.

When we built a RESTful API on the distributed interface layer we ran into a resource that has two use cases when GETing it. The first is a request with an authenticated credentials which should get the whole entity as it is (there is still an adapter that transforms the domain model entity to the sent DTO - but the mapping is almost one-to-one). The second is a case where it is a public GET request - which implies all you will get in response is just the public version of the resource (basically, some of the fields are left out and not filled).

The out of the box solution will be to have the Application service call the Domain Entity repository for the full version and according to the authentication level fill the required fields of the returned DTO at the relevant adapter.

But, we chose a different path. We decided to treat the public resource as a whole new resource - an application layer "entity" with it's own route on the API level. One could suggest to use the the value type pattern and basically place the public entity on the domain level, but IMHO, public/private and authentication is not a concern of the domain model and it is a pure application layer matter.

Eventually, This implied having an application layer repository for this new entity (which is still obviously implemented in the infrastructure layer). 

The great benefit here was both simplicity on the API level, where there is a one-to-one mapping and expectation between the request and response (which added type safety and clarity as one doesn't need to guess what fields are filled and what aren't) but furthermore - this new repository only fetches the relevant public data from the persistence store, which is more secure, more efficient and a much cleaner design.

Scaling Message Buses on the X Axis

The X Axis

One of the key things I took from Scalabiltiy Rules book is what they refer to as the AKF Cube or the AKF Three Axes of Scale. Generally speaking, The cube suggests a schematic 3D description of a system's scalability options - a simple methodology to examine the ways to scale your system and it's components.

X: Horizontal duplication (Design to clone things) - duplicating your data and services for leveraging on the high read/writes ratio.
Y: Split by service (Design to split different things) - separating components of different services to avoid dependencies and shared resources and creating fault isolation architecture.
Z: Lookup oriented splits (Design to split similar things) - separating components by sharding.

 If every component of your system can scale on all three directions - then you are probably near infinite scalability.

Scaling Message Buses

When relating the service buses, The authors recommend scaling using the Y axis and Z axis. But when it comes to horizontal duplication the authors say that it does not work well when it comes to message buses - and  I decided to challenge that.

In eToro, we use RabbitMQ for asynchronous messaging and service bus. Up until now we implemented Y Axis scalability - we have different message bus for each service (or sub-domain) and we have some of them connected via federation for cross domain eventually consistent integration.

The Test
RabbitMQ Cluster

I have created a cluster using two virtual Ubuntu machines installed with latest RabbitMQ version.
I used HAProxy as a network load balancer to have the clients redirected to the available node once one drops.

The RabbitMQ Cluster enables you to join multiple nodes together into a single logical node. All queues on all nodes are accessible from all other nodes (n to n topology) when it comes to both writes and reads - transparent to both consumers and producers.
Measuring messages latency in a cluster where the readers and the writers are on different nodes (mixed) came out with ~0 latency.
Using a cluster enables you to increase the resources in the a single node horizontally by adding nodes to the cluster.
But, as only one node is considered the real owner of a queue - when that node drops - the queue drops with it and is not accessible anymore from other nodes in the cluster.

Mirrored Queues

This is exactly where duplication (or X Axis) kicks in: RabbitMQ suggests mirroring queues across a cluster to create high availability cluster

I created a queue on one of the nodes and added a policy to have it duplicated across all nodes.
I dropped one of my nodes and when my clients reconnected they continued consuming the queue - as expected.

When using mirroring queues all clients are working directly with the master queue and when a change happens (message is enqueued or acknowledged) it is simply mirrored across all slave queues. When the master queue drops other slave is being promoted to master.

Mirrored queues provides a good solution for high availability with a bit of a latency cost: When working with queues in a cluster I measured 0 latency on 1000/s messages. On Mirrored queues I measured ~1 millisecond latency.

Federated Queues
Mirrored queues provides high availability but does not solve a resources or load problems - for that, you can either use the Y and Z axis splits or federated queues.

Federating a queue creates a low priority consumer on a master queue - which implies that only messages that currently have no available consumer to handle them are passed through federation to a slave queue.

During the test I noticed that even without any consumers on the master queue the average messages/sec transferred to federated queue was ~2k (which gave me a 40% throughput boost).

Summary
RabbitMQ provides great features for scaling your message bus. I must say I would not give up so fast on scaling a system's service bus using the X axis.

Composing Multiple SubDomains

During the past weeks  I was bothered with the question what is the best practice in case you have a client consuming multiple services (DDD subdmains) into a single view.

Is it better to implement a front service which receives a single call from the client, sends requests back to each relevant back-end service and aggregates the response back to the client?

or whether it would be best having the client send the requests directly to these services?

Please note that for the sake of simplification I abstracted the above diagrams and ignored logical (presentation, application, domain) and physical (fireawalls, load balancers etc.) tiers irrelevant to this discussion.

IMHO, There isn't a straight answer but pros and cons that should be considered per design.

Aggregated Front Service

I/O, Roundtrip and Resources

When it comes to aggregated front service, one of the obvious wins here is dramatically reducing the client-server round-trip.

And why is that so great?

Remote calls by nature tend to fail from time to time due to many reasons: service availability, connectivity issues etc.

When talking about a front api with clients out there on the internet it gets even worse as we have a lot of moving parts in the client to server way - with each part increasing the probability of a request to fail.

Let's say for example that one call success chance is 99.99%. When a client needs 3 calls in order to create it's final view - we drop down to 99.97% success probability (99.99% X 99.99% X 99.99%). With every single remote call added to a client's dependencies we increase the probability of it to fail.

(I should mention that the same concept works for putting components in a series or doing synchronous calls between services as covered in the great Scalability Rules book).

Another reason to implement such service is the limited browser and mobile devices resources.

In cases where you have an ecosystem built with a large amount of services you might find clients having a hard time dealing with sending many parallel requests as they are limited when it comes to resources. Another case is when the aggregation determines heavy computational resources (data processing, manipulation etc) which only the back end servers can supply.

Furthermore, When it comes to I/O, buffering/aggregating on I/O operations increases resource utilization (due to the overhead of every operation) thus increasing server throughput - which is a very big win!

Design

In cases where there is some business logic or data manipulation needed in order to answer a client's request, a front service might be good also from a design perspective. Business logic dedicated to composing multiple services might imply you are looking at a whole new sub-domain/SOA Service with it's own requirements, resources and entities (as described in Vaughn Vernon's Implementing Domain Driven Design). 

A good example of that is eToro's stats service which collects data from multiple services (social, trading etc.) and builds modules and recommendation system on top of it. This service is a whole new world with it's own ubiquitous language, entities, aggregates etc.

The common denominator

There is one common thing to all above cases: the assumption that we always need to go to each and every one of the back end services in order to build our view. It is a very important thing to mention as will be explained later.

Direct Client Calls

Caching

The above mentioned Scalability Rules book emphasizes the importance of caching when it comes to scalability - or as Abbot and Fisher call their caching rules set: Use Caching Aggressively!

When it comes to front services scalability, some of the most common and important tools you should leverage is the Reverse Proxies and Content Delivery Networks (CDN). Both of these mechanisms relay on the fact that some responses/resources can be cached - as they are valid for some certain amount of time - meaning they are partially/fully static. These tools allow you to cache your responses on a front server which will answer some of your traffic - reducing the actual hit and load on your back end systems.

When aggregating the response of multiple services by a service and by that ignoring whether their response is dynamic or static - you are disabling the opportunity to leverage on these important scalability tools! 

Think of it this way: What if some of your heavy resources/calls can be geographically cached for your clients - so once a client requests some resource he actually gets the response from a server a few miles away - You both significantly increase the probability of call success and gain scalability for your system!

Coupling and Dependencies

When the client side is responsible for the composition you are basically creating a back-end that doesn't have to be aware of client views - all it needs to do is properly reflect it's own state and operations. Creating a middle tier composing these services makes another part of your system aware of that composition which creates another coupling.

Furthermore, as mentioned in Vernon's book, there are systems where their client view is basically a composition of "widgets" of different sub domains/services (portal-portlet style) - like Amazon's checkout page, where you see payments data side by side with shipping and other book recommendations. In  this post Udi Dahan explains how having this kind of client composition clears out your services boundaries - which is an enormous design achievement!

Last, another level of dependency is the maintenance. This extra tier is another tier that should be maintained, sometimes between teams (usually happens when teams are divided by subdomains) which usually implies painful cross teams synchronization.

Summary
As I said, Its really hard to say right or wrong as every project has it's constraints.
IMHO, generally speaking, I would go for client side composition in most cases in order to increase scalability and simplicity and reduce coupling. Only in cases when there is a lack of resources or when there is a descent business logic that you wouldn't want the client side to implement - I'd go with the front aggregation service.

Udi Dahan recorded a great video post on composing multiple services using IT/OPS component.

Serialization Benchmark

A few months ago, as a part of eToro's architects forum, I took a task of doing a performance benchmark on serializers commonly used in the industry.

The reason I had to redo the benchmark and not just look it up online is because I couldn't find a benchmark that includes all the serializers I wanted to test - This is exactly why I'm posting this online, maybe this will save someone else's time.

As far as for trade offs we thought of when choosing the best serializer(s) for our development teams we considered the following:

• (De)Serialization performance
• Payload size
• Cross-platform ability
• Readability of the serialized result (ease things up on debugging and logging reads).

So, for us to get a wide view on things I tested both string based and binary serializers.
In the string section:

• WCF's DataContractSerializer
• XMLSerializer
• ServiceStack's JsonSerializer
• Newtonsoft's JsonConvert

In the binary section:

• Protobuf-net
• DataContractSerializer (set up to binary)
• XMLSerializer (set up to binary) 
• BinaryFormatter (just for reference, as it isn't cross platform and known for it's poor performance due to it's extensive usage of reflection).

In order to choose the right metrics I looked up in Sasha Goldshtein's awsome book  who did a very similar benchmark.

The Test
In order to do the benchmark I used AdventureWork's top 100K Sales.SalesOrderDetail rows.
I loaded these rows into this POCO:

The reason for the BaseMessageClass and the inheritance is our common use case where we have one endpoint in the code that deserializes the bytes into a base class and then dispatches it according to it's actual type.

For every serializer I simply measured using Stopwatch the time to serially serialize 100K objects into a memory stream and then measured the time to serially deserialize the output of the previous stage into the base message.

Between each test I ran GC.Collect() in order to avoid GC during the measurements.

I did also try to measure every single serialization and deserialization operation but as the results on the fast serializers were mostly under a millisecond I got no relevant results there.

The Results

And the Winners

When it comes to performance and payload size Protobuf-net is without a doubt the best serializer by a magnitude. 

If you insist of having a readable payload (or insist of Json as it is pretty much a standard these days when it comes to web apis) and wish to compromise as least as you can on performance I guess ServiceStack's JsonSerializer is a good choice for you with a descent performance and reasonable payload size.

Static Ctor

Hi there!

This is my first blog post... (woohoo!!)

My name is Moti. I'm 28 years old, a father of a lovely three years old girl, married to a beautiful wife and pretty much feeling blessed - so first of all, thank you god!

A friend of mine, David Virtser, suggested I should start my own blog In which I will post some of the things, thoughts, bits and bytes I'm dealing with in my software engineering practice that I personally think are worth sharing...

I've been practicing software engineering for the past 9 years and enjoying every second of it. I see myself as a very passionate developer -  I spend a lot of my free time reading software books and articles, watching lectures etc. This is pretty much my biggest  hobby - can't really explain how it turned out this way, but I guess I got addicted to it.

I'm mostly interested in software architecture (SOA and distributed systems), design (DDD, patterns and best practices) and performance (specially in a Microsoft oriented environment - but not only) - all from the enterprise applications perspective.

I'm not really sure how this blog will eventually turn out to be, but as I see it now I will mainly post for myself  the stuff I will want to look up later.

Most of the things I will post will basically rely one way or another on smarter and more experienced people than me such as the great Sasha Goldstein, The author of the best book a .net developer can get Jeffrey Richter, DDD Guru Vaughn Vernon, SOA and distributed systems mentor Udi Dahan and many others I try my best to learn as much as I can from. (This is a good opportunity to thank them for all the times they took the time to answer my annoying emails and questions).

I hope this blog will at least help me to clear my thoughts a little...

Good luck to us all...