Modular Monolith: Domain-Centric Design

This post is part of articles series about Modular Monolith architecture:

1. Modular Monolith: A Primer
2. Modular Monolith: Architectural Drivers
3. Modular Monolith: Architecture Enforcement
4. Modular Monolith: Integration Styles

5. Modular Monolith: Domain-Centric Design (this)

Introduction

In previous posts in this series, I covered what a Modular Monolith is, what its architecture looks like and how this architecture can be enforced. Then I described architectural drivers for this architecture and styles of integration between modules.

In this post I would like to go deeper – a level lower and describe how such architecture can be designed. We are not going to implement this architecture yet – we will focus on its technology-agnostic design.

Domain-Centric Design

As the name of Modular Monolith architecture suggests, the design of our architecture must be oriented towards high modularity. What follows from this, the system must have self-contained modules that provide the entire business functionality. This is why domain-centric architecture and design is natural choice in that case.

Moreover, as we know, modules must have well-defined interfaces. All communication between modules should only take place through these interfaces, which means that each module must be highly encapsulated.

Let’s see how such architecture can look like from high-level view:

Modular Monolith Design
Modular Monolith: Domain-Centric Design

Looking at a high-level view, there are similarities to domain-centric architectures. This is exactly the case – both in terms of the architecture of the entire system (system architecture) and individual modules (application architecture) described later.

Looking at the similarities to the Hexagonal Architecture we have:
– API: primary adapters
– Module API: primary ports
– secondary ports and its adapters (to communicate with a database, events bus, other modules)

Modular Monlith: Hexagonal Architecture View
Modular Monlith: Hexagonal Architecture View

If we look closely, this architecture is no different from Onion and Clean architectures. The most important thing is that our domain is inside and the Dependency Rule is respected:

Source code dependencies must point only inward, toward higher-level policies.

Modular Monlith: Clean/Onion Architecture View
Modular Monlith: Clean/Onion Architecture View

Let’s try to describe all the elements one after the other.

API

API is the entry point to our system. Mainly implemented as a web service (SOAP/REST/GraphQL) that accepts HTTP requests and returns HTTP responses.

The main and only responsibility of the API is to forward the request to the appropriate module. It is an equivalent of API Gateway in microservice architecture, only instead of network calls to services we have module calls in memory.

The API should be very thin. There should be no logic there – neither application nor business one. Everything we put there should be related only to processing HTTP requests and routing.

Module

Each module should be treated as a separate application. In other words, it’s a subsystem of our system. Thanks to this, it will have autonomy. It will be loosely or even not coupled to other modules (subsystems). It means that each module can be developed by a separate team. This is the same architectural driver as in the case of microservice architecture.

Moreover, we will be able to easily extract a particular module into a separate runtime component (Monolith split). Of course only if necessary – this is not the goal of our architecture, only a great side-effect of modularity.

Since the module should be domain-oriented (see Bounded Context concept from DDD strategic patterns set), we can use the domain-centric architecture again – this time on the level of the module itself.

The module architecture is as follows:

Module Architecture
Module Architecture

Module Startup API

Module Startup API is a port/interface thanks to which a given module can be initialized. As a given module must be self-contained, it should be able to initialize itself, getting only the appropriate configuration parameters needed for its operation. This means that we do NOT configure a given module in the API (or another module host). We only initiate its initialization at startup.

Composition Root

Support for module autonomy also means that a given module must be able to create an object dependency graph itself, i.e. it should have its own Composition Root.

This usually means that it will have its own IoC container. It is very important thing. Unfortunatelly, the most common approach is an IoC container defined per whole runtime component. It is good approach for tiny systems, not for the more complex and modular.

Module API

Module API is an interface (primary port) for communicating with the given module (except initialization – see Module Startup API). Such a module API can be created in two ways:

– traditional approach: a list of methods (CustomerService.GetCustomer, OrderService.AddOrder)
CQRS-style approach: a set of Queries and Commands to be sent (GetCustomerQuery, AddOrderCommand)

I am definitely a fan of the second, CQRS-style approach, but in my opinion, the first approach is also acceptable.

According to the modularization key attributes, this module API should be as small as possible – expose only what is needed (no less, no more). This will make it more stable.

Infrastructure – Secondary Adapters

This is where the implementation of secondary adapters should be (from the nomenclature of Ports and Adapters architecture). Secondary adapters are responsible for communication with the external dependencies (in-process and out-of-process): databases, events bus, another modules.

Application

Here you should find the implementation of use cases related to the module. It is a Application Core boundary (from Onion Architecture architecture view) or architecture. Thanks to the domain-centric architecture, it is decoupled from frameworks and infrastructure.

Model of this domain (Domain Model) is applicable only in Bounded Context (boundary). There will be only concepts related to our domain and the so-called enterprise business rules.

Domain Model should have Persistence Ignorance. Written in Ubiquitous Language and completely testable. Here we focus the most, the rest is only to make it easier for us.

How many layers?

There is a lot of discussion about application layers on the Internet. Some prefer to have a very clear layers division (e.g. using separate libraries/packages or other language techniques). Others prefer to keep everything together without logical decomposition.

First of all, be aware that each module will be different. One may have a more complicated domain, the other may only implement CRUD operations. In this case, the application architecture for these modules will be different.

Moreover, within the same module there may be both more and less complicated functionalities. In this case, we should also respect each functionality separately.

In conclusion, the application of layers should not be a global decision for a given module – each use case should be considered separately. This approach is close to the Vertical Slices architecture, applied at the module level, not the entire application.

Some say domain-centric architectures and vertical slices are opposites. It is far from the truth – in my opinion, they complement each other perfectly.

Module application architecture styles
Module application architecture styles

Module Data

Each module must have its own state, which means that its data must be private. We don’t want to use Shared Database Pattern. This is a key attribute needed to achieve the autonomy and modularity of a module. If we want to know the state of the module or change it – we have to do it through the interface. There are no shortcuts.

However, sometimes we want to share some data for reporting purposes. In this case, we can use separate Reporting Database and provide data in the form of separate views on the module database in a special integration scheme – only for this kind of integration. In this way, we create a concept of API on the database level – it is just a good thing to do.

Modules integration

I wrote about the modules integration in detail in the previous post. As you can see, the Modular Monolith architecture design assumes 2 forms of communication:

1. Asynchronous via events (Event-Driven architecture). Each module sends or subscribes to certain events via Events Bus. This Events Bus can be in memory mechanism or out-of-process component – depending on the needs.

2. Synchronous with in memory calls. Here, as in the case of API communication with modules, it can be implemented in a traditional approach or CQRS-style (Commands / Queries). What is important here is that such integration should be explicit – by creating a Gate (adapter) on the consumer side and a Facade (port and its implementation) on the supplier side.

Tests

If you want to modularize your system, it means that it is not trivial (or won’t be in the future). This means that automated tests are a must-have. However, what percentage of a given type of test should be written depends on the given system, its level of complexity, number of integrations, and other factors.

Tests are an extensive topic that is beyond the scope of this article and certainly deserves a separate one. Here, I wanted only to highlight which tests should be considered.

End to End tests

E2E tests test your entire system – from API to infrastructure and back again. Often, They check the whole fragment of our system so they have the biggest test code coverage.

However, they often also test the system together with the GUI. For this reason, they are the slowest, fragile and hard to maintain.

Integration Tests

Integration Testing is a broad term that is understood in various ways. In the proposed architecture, these are comprehensive tests of a given module (or interaction between modules), without the API layer. These types of integration tests are second consumers of our modules (just another adapters). As the API layer is very thin, they cover practically our entire application and do not operate on low-level abstraction objects (JSON, HTTP).

Unit Tests

Unit tests will be used mainly to test the Domain Model – business logic. Thanks to the use of domain-centric architecture as part of the module, the Domain Model is separated from the infrastructure and can be easily tested in memory.

Summary

As you can see, making our system modular requires discipline in following the rules and principles of proper design. The entire system is constantly decomposed into smaller fragments. Every piece of the puzzle is important. Let’s summarize the most important attributes of this architecture:

– domain-centric on multiple levels
– well-defined integration points (interfaces)
– self-contained, encapsulated modules
– testability – making the application and domain layer independent from frameworks and infrastructure
– evolutionary – easy to develop and maintain (adding new modules or adapters)

If you don’t need to distribute your system (and most people don’t) and your system is non-trivial – maybe a Modular Monolith with Domain Centric Design in mind will be for you. Remember, however, that it all depends on the context in which your project exists so make your decisions consciously.

Related Posts

1. Modular Monolith: A Primer
2. Modular Monolith: Architectural Drivers
3. Modular Monolith: Architecture Enforcement
4. Modular Monolith: Integration Styles
5. Domain Model Encapsulation and PI with EF Core
6. Simple CQRS implementation with raw SQL and DDD

Image credits: Magnasoma

REST API Data Validation

Introduction

This time I would like to describe how we can protect our REST API applications from requests containing invalid data (data validation process). However, validation of our requests is not enough, unfortunately. In addition to validation, it is our responsibility to return the relevant messages and statuses to our API clients. I wanted to deal with these two things in this post.

Data Validation

Definition of Data Validation

What is data validation really? The best definition I found is from UNECE Data Editing Group:

An activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values.

According to this definition we should verify data items which are coming to our application from external sources and check if theirs values are acceptable. How do we know that the value is acceptable? We need to define data validation rules for every type of data item which is processing in our system.

Data vs Business Rules validation

I would like to emphasize that data validation is totally different concept than validation of business rules. Data validation is focused on verifying an atomic data item. Business rules validation is a more broad concept and more close to how business works and behaves. So it is mainly focused on behavior. Of course validating behavior depends on data too, but in a more wide range.

Examples of data validation:

– Product order quantity cannot be negative or zero
– Product order quantity should be a number
– Currency of order should be a value from currencies list

Examples of business rules validation

– Product can be ordered only when Customer age is equal or greater than product minimal age.
– Customer can place only two orders in one day.

Returning relevant information

If we acknowledge that the rules have been broken during validation, we must stop processing and return the equivalent message to the client. We should follow the following rules:

– we should return message to the client as fast as possible (Fail-fast principle)
– the reason for the validation error should be well explained and understood for the client
– we should not return technical aspects for security reasons

Problem Details for HTTP APIs standard

The issue of returned error messages is so common that a special standard was created describing how to handle such situations. It is called “Problem Details for HTTP APIs standard” and his official description can be found here. This is abstract of this standard:

This document defines a “problem detail” as a way to carry machine-readable details of errors in a HTTP response to avoid the need to define new error response formats for HTTP APIs.

Problem Details standard introduces Problem Details JSON object, which should be part of the response when validation error occurs. This is simple canonical model with 5 members:

– problem type
– title
– HTTP status code
– details of error
– instance (pointer to specific occurrence)

Of course we can (and sometimes we should) extend this object by adding new properties, but the base should be the same. Thanks to this our API is easier to understand, learn and use. For more detailed information about standard I invite you to read documentation which is well described.

Data validation localization

For the standard application we can put data validation logic in three places:

  • GUI – it is entry point for users input. Data is validated on the client side, for example using Javascript for web applications
  • Application logic/services layer – data is validated in specific application service or command handler on the server side
  • Database – this is exit point of request processing and last moment to validate the data
Data validation localization
Data validation localization

In this article I am omitting GUI and Database components and I am focusing on the server side of the application. Let’s see how we can implement data validation on Application Services layer.

Implementing Data Validation

Suppose we have a command AddCustomerOrderCommand:

Suppose we want to validate 4 things:

1. CustomerId is not empty GUID.
2. Products list is not empty
3. Each product quantity is greater than 0
4. Each product currency is equal to USD or EUR

Let me show 3 solutions to this problem – from simple to the most sophisticated.

1. Simple validation on Application Service

The first thing that can come to mind is a simple validation in the Command Handler itself. In this solution we need to implement private method which validates our command and throws exception if validation error occurs. Closing this kind of logic in separate method is better from the Clean Code perspective (see Extract Method too).

The result of invalid command execution:

This is not so bad approach but has two disadvantages. Firstly, it involves from us writing a lot of easy and boilerplate code – comparing to nulls, defaults, values from list etc. Secondly, we are losing here part of separation of concerns because we are mixing validation logic with orchestrating our use case flow. Let’s take care of boilerplate code first.

2. Validation using FluentValidation library

We don’t want to reinvent the wheel so the best solution is to use library. Fortunately, there is a great library for validation in .NET world – Fluent Validation. It has nice API and a lot of features. This is how we can use it to validate our command:

Now, the Validate method looks like:

The result of validation is the same as earlier, but now our validation logic is more cleaner. The last thing to do is decouple this logic from Command Handler completely…

3. Validation using Pipeline Pattern

To decouple validation logic and execute it before Command Handler execution we arrange our command handling process in Pipeline (see NServiceBus Pipeline also).

For the Pipeline implementation we can use easily MediatR Behaviors. First thing to do is behavior implementation:

Next thing to do is to register behavior in IoC container (Autofac example):

This way we achieved separation of concerns and Fail-fast principle implementation in nice and elegant way.

But this is not the end. Finally, we need to do something with returned messages to clients.

Implementing Problem Details standard

Just as in the case of validation logic implementation, we will use a dedicated library – ProblemDetails. The principle of the mechanism is simple. Firstly, we need to create custom exception:

Secondly, we have to create own Problem Details class:

Last thing to do is to add Problem Details Middleware with definition of mapping between InvalidCommandException and InvalidCommandProblemDetails class in startup:

After change in CommandValidationBehavior (throwing InvalidCommandExecption instead Exception) we have returned content compatible with the standard:

Problem details

Summary

In this post I described:
– what Data validation is and where is located
– what Problem Details for HTTP APIs is and how could be implemented
– 3 methods to implement data validation in Application Services layer: without any patterns and tools, with FluentValidation library, and lastly – using Pipeline Pattern and MediatR Behaviors.

Source code

If you would like to see full, working example – check my GitHub repository

Related posts

Domain Model Encapsulation and PI with Entity Framework 2.2
Simple CQRS implementation with raw SQL and DDD
How to publish and handle Domain Events
10 common broken rules of clean code

Simple CQRS implementation with raw SQL and DDD

Introduction

I often come across questions about the implementation of the CQRS pattern. Even more often I see discussions about access to the database in the context of what is better – ORM or plain SQL.

In this post I wanted to show you how you can quickly implement simple REST API application with CQRS using the .NET Core. I immediately point out that this is the CQRS in the simplest edition – the update through the Write Model immediately updates the Read Model, therefore we do not have here the eventual consistency. However, many applications do not need eventual consistency, while the logical division of writing and reading using two separate models is recommended and more effective in most solutions.

Especially for this article I prepared sample, fully working application, see full source on Github.

My goals

These are my goals that I wanted to achieve by creating this solution:
1. Clear separation and isolation of Write Model and Read Model.
2. Retrieving data using Read Model should be as fast as possible.
3. Write Model should be implemented with DDD approach. The level of DDD implementation should depend on level of domain complexity.
4. Application logic should be decoupled from GUI.
5. Selected libraries should be mature, well-known and supported.

Design

High level flow between components looks like:

As you can see the process for reads is pretty straightforward because we should query data as fast as possible. We don’t need here more layers of abstractions and sophisticated approaches. Get arguments from query object, execute raw SQL against database and return data – that’s all.

It is different in the case of write support. Writing often requires more advanced techniques because we need execute some logic, do some calculations or simply check some conditions (especially invariants). With ORM tool with change tracking and using Repository Pattern we can do it leaving our Domain Model intact (ok, almost).

Solution

Read model

Diagram below presents flow between components used to fulfill read request operation:

The GUI is responsible for creating Query object:

Then, query handler process query:

The first thing is to get open database connection and it is achieved using SqlConnectionFactory class. This class is resolved by IoC Container with HTTP request lifetime scope so we are sure, that we use only one database connection during request processing.

Second thing is to prepare and execute raw SQL against database. I try not to refer to tables directly and instead refer to database views. This is a nice way to create abstraction and decouple our application from database schema because I want to hide database internals as much as possible.

For SQL execution I use micro ORM Dapper library because is almost as fast as native ADO.NET and does not have boilerplate API. In short, it does what it has to do and it does it very well.

Write model

Diagram below presents flow for write request operation:

Write request processing starts similar to read but we create the Command object instead of the query object:

Then, CommandHandler is invoked:

Command handler looks different than query handler. Here, we use higher level of abstraction using DDD approach with Aggregates and Entities. We need it because in this case problems to solve are often more complex than usual reads. Command handler hydrates aggregate, invokes aggregate method and saves changes to database.

Customer aggregate can be defined as follows:

Architecture

Solution structure is designed based on well-known Onion Architecture as follows:

Only 3 projects are defined:
– API project with API endpoints and application logic (command and query handlers) using Feature Folders approach.
– Domain project with Domain Model
– Infrastructure project – integration with database.

Summary

In this post I tried to present the simplest way to implement CQRS pattern using raw sql scripts as Read Model side processing and DDD approach as Write Model side implementation. Doing so we are able to achieve much more separation of concerns without losing the speed of development. Cost of introducing this solution is very low and and it returns very quickly.

I didn’t describe DDD implementation in detail so I encourage you once again to check the repository of the example application – can be used as a kit starter for your app the same as for my applications.

Related posts

Domain Model Encapsulation and PI with Entity Framework 2.2
How to publish and handle Domain Events
REST API Data Validation