Loading...
Skip to Content

Blog

Post: REST API Data Validation

REST API Data Validation

Introduction

This time I would like to describe how we can protect our REST API applications from requests containing invalid data (data validation process). However, validation of our requests is not enough, unfortunately. In addition to validation, it is our responsibility to return the relevant messages and statuses to our API clients. I wanted to deal with these two things in this post.

Data Validation

Definition of Data Validation

What is data validation really? The best definition I found is from UNECE Data Editing Group:

An activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values.

According to this definition we should verify data items which are coming to our application from external sources and check if theirs values are acceptable. How do we know that the value is acceptable? We need to define data validation rules for every type of data item which is processing in our system.

Data vs Business Rules validation

I would like to emphasize that data validation is totally different concept than validation of business rules. Data validation is focused on verifying an atomic data item. Business rules validation is a more broad concept and more close to how business works and behaves. So it is mainly focused on behavior. Of course validating behavior depends on data too, but in a more wide range.

Examples of data validation:

  • Product order quantity cannot be negative or zero
  • Product order quantity should be a number
  • Currency of order should be a value from currencies list

Examples of business rules validation:

  • Product can be ordered only when Customer age is equal or greater than product minimal age.
  • Customer can place only two orders in one day.

Returning relevant information

If we acknowledge that the rules have been broken during validation, we must stop processing and return the equivalent message to the client. We should follow the following rules:

  • we should return message to the client as fast as possible (Fail-fast principle)
  • the reason for the validation error should be well explained and understood for the client
  • we should not return technical aspects for security reasons

Problem Details for HTTP APIs standard

The issue of returned error messages is so common that a special standard was created describing how to handle such situations. It is called “Problem Details for HTTP APIs standard” and his official description can be found here. This is abstract of this standard:

This document defines a “problem detail” as a way to carry machine-readable details of errors in a HTTP response to avoid the need to define new error response formats for HTTP APIs.

Problem Details standard introduces Problem Details JSON object, which should be part of the response when validation error occurs. This is simple canonical model with 5 members:

  • problem type
  • title
  • HTTP status code
  • details of error
  • instance (pointer to specific occurrence)

Of course we can (and sometimes we should) extend this object by adding new properties, but the base should be the same. Thanks to this our API is easier to understand, learn and use. For more detailed information about standard I invite you to read documentation which is well described.

Data validation localization

For the standard application we can put data validation logic in three places:

  • GUI - it is entry point for users input. Data is validated on the client side, for example using Javascript for web applications
  • Application logic/services layer - data is validated in specific application service or command handler on the server side
  • Database - this is exit point of request processing and last moment to validate the data

Validation Layers

Data validation localization

In this article I am omitting GUI and Database components and I am focusing on the server side of the application. Let’s see how we can implement data validation on Application Services layer.

Implementing Data Validation

Suppose we have a command AddCustomerOrderCommand:

public class AddCustomerOrderCommand : IRequest
{
    public Guid CustomerId { get; }

    public List<ProductDto> Products { get; }

    public AddCustomerOrderCommand(
        Guid customerId, 
        List<ProductDto> products)
    {
        this.CustomerId = customerId;
        this.Products = products;
    }
}

public class ProductDto
{
    public Guid Id { get; set; }

    public int Quantity { get; set; }

    public string Currency { get; set; }

    public string Name { get; set; }
}

Suppose we want to validate 4 things:

  1. CustomerId is not empty GUID.
  2. Products list is not empty
  3. Each product quantity is greater than 0
  4. Each product currency is equal to USD or EUR

Let me show 3 solutions to this problem - from simple to the most sophisticated.

1. Simple validation on Application Service

The first thing that can come to mind is a simple validation in the _Command Handler itself. In this solution we need to implement private method which validates our command and throws exception if validation error occurs. Closing this kind of logic in separate method is better from the Clean Code perspective (see Extract Method).

public class AddCustomerOrderCommandHandler : IRequestHandler<AddCustomerOrderCommand>
{
    private readonly ICustomerRepository _customerRepository;
    private readonly IProductRepository _productRepository;

    public AddCustomerOrderCommandHandler(
        ICustomerRepository customerRepository, 
        IProductRepository productRepository)
    {
        this._customerRepository = customerRepository;
        this._productRepository = productRepository;
    }

    public async Task<Unit> Handle(AddCustomerOrderCommand request, CancellationToken cancellationToken)
    {
        Validate(request);
        var customer = await this._customerRepository.GetByIdAsync(request.CustomerId);

        // logic..
    }

    private static void Validate(AddCustomerOrderCommand command)
    {
        var errors = new List<string>();

        if (command.CustomerId == Guid.Empty)
        {
            errors.Add("CustomerId is empty");
        }

        if (command.Products == null || !command.Products.Any())
        {
            errors.Add("Products list is empty");
        }
        else
        {
            if (command.Products.Any(x => x.Quantity < 1))
            {
                errors.Add("At least one product has invalid quantity");
            }

            if (command.Products.Any(x => x.Currency != "USD" &amp;&amp; x.Currency != "EUR"))
            {
                errors.Add("At least one product has invalid currency");
            }
        }

        if (errors.Any())
        {
            var errorBuilder = new StringBuilder();

            errorBuilder.AppendLine("Invalid order, reason: ");

            foreach (var error in errors)
            {
                errorBuilder.AppendLine(error);
            }

            throw new Exception(errorBuilder.ToString());
        }
    }
}

The result of invalid command execution:

Add order postman validation

This is not so bad approach but has two disadvantages. Firstly, it involves from us writing a lot of easy and boilerplate code - comparing to nulls, defaults, values from list etc. Secondly, we are losing here part of separation of concerns because we are mixing validation logic with orchestrating our use case flow. Let’s take care of boilerplate code first.

2. Validation using FluentValidation library

We don’t want to reinvent the wheel so the best solution is to use library. Fortunately, there is a great library for validation in .NET world - Fluent Validation. It has nice API and a lot of features. This is how we can use it to validate our command:

// FluentValidation validator
public class AddCustomerOrderCommandValidator : AbstractValidator<AddCustomerOrderCommand>
{
    public AddCustomerOrderCommandValidator()
    {
        RuleFor(x => x.CustomerId).NotEmpty().WithMessage("CustomerId is empty");
        RuleFor(x => x.Products).NotEmpty().WithMessage("Products list is empty");
        RuleForEach(x => x.Products).SetValidator(new ProductDtoValidator());
    }
}

public class ProductDtoValidator : AbstractValidator<ProductDto>
{
    public ProductDtoValidator()
    {
        this.RuleFor(x => x.Currency).Must(x => x == "USD" || x == "EUR")
            .WithMessage("At least one product has invalid currency");
        this.RuleFor(x => x.Quantity).GreaterThan(0)
            .WithMessage("At least one product has invalid quantity");
    }
}

Now, the Validate method looks like:

// Validate with FluentValidation
private static void Validate(AddCustomerOrderCommand command)
{
    AddCustomerOrderCommandValidator validator = new AddCustomerOrderCommandValidator();

    var validationResult = validator.Validate(command);
    if (!validationResult.IsValid)
    {
        var errorBuilder = new StringBuilder();

        errorBuilder.AppendLine("Invalid order, reason: ");

        foreach (var error in validationResult.Errors)
        {
            errorBuilder.AppendLine(error.ErrorMessage);
        }

        throw new Exception(errorBuilder.ToString());
    }
}

The result of validation is the same as earlier, but now our validation logic is more cleaner. The last thing to do is decouple this logic from Command Handler completely…

3. Validation using Pipeline Pattern

To decouple validation logic and execute it before Command Handler execution we arrange our command handling process in Pipeline (see NServiceBus Pipeline also).

For the Pipeline implementation we can use easily MediatR Behaviors. First thing to do is behavior implementation:

// CommandValidationBehavior
public class CommandValidationBehavior<TRequest, TResponse> : IPipelineBehavior<TRequest, TResponse>
{
   private readonly IList<IValidator<TRequest>> _validators;

   public CommandValidationBehavior(IList<IValidator<TRequest>> validators)
   {
       this._validators = validators;
   }

   public Task<TResponse> Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate<TResponse> next)
   {
       var errors = _validators
           .Select(v => v.Validate(request))
           .SelectMany(result => result.Errors)
           .Where(error => error != null)
           .ToList();

       if (errors.Any())
       {
           var errorBuilder = new StringBuilder();

           errorBuilder.AppendLine("Invalid command, reason: ");

           foreach (var error in errors)
           {
               errorBuilder.AppendLine(error.ErrorMessage);
           }

           throw new Exception(errorBuilder.ToString());
       }

       return next();
   }
}

Next thing to do is to register behavior in IoC container (Autofac example):

// Register CommandValidationBehavior
public class MediatorModule : Autofac.Module
{
    protected override void Load(ContainerBuilder builder)
    {
        builder.RegisterAssemblyTypes(typeof(IMediator).GetTypeInfo().Assembly).AsImplementedInterfaces();

        var mediatrOpenTypes = new[]
        {
            typeof(IRequestHandler<,>),
            typeof(INotificationHandler<>),
            typeof(IValidator<>),
        };

        foreach (var mediatrOpenType in mediatrOpenTypes)
        {
            builder
                .RegisterAssemblyTypes(typeof(GetCustomerOrdersQuery).GetTypeInfo().Assembly)
                .AsClosedTypesOf(mediatrOpenType)
                .AsImplementedInterfaces();
        }

        builder.RegisterGeneric(typeof(RequestPostProcessorBehavior<,>)).As(typeof(IPipelineBehavior<,>));
        builder.RegisterGeneric(typeof(RequestPreProcessorBehavior<,>)).As(typeof(IPipelineBehavior<,>));

        builder.Register<ServiceFactory>(ctx =>
        {
            var c = ctx.Resolve<IComponentContext>();
            return t => c.Resolve(t);
        });

        builder.RegisterGeneric(typeof(CommandValidationBehavior<,>)).As(typeof(IPipelineBehavior<,>));
    }
}

This way we achieved separation of concerns and Fail-fast principle implementation in nice and elegant way. But this is not the end. Finally, we need to do something with returned messages to clients.

Implementing Problem Details standard

Just as in the case of validation logic implementation, we will use a dedicated library - ProblemDetails. The principle of the mechanism is simple. Firstly, we need to create custom exception:

// InvalidCommandException
public class InvalidCommandException : Exception
{
    public string Details { get; }
    public InvalidCommandException(string message, string details) : base(message)
    {
        this.Details = details;
    }
}

Secondly, we have to create own Problem Details class:

// InvalidCommandProblemDetails
public class InvalidCommandProblemDetails : Microsoft.AspNetCore.Mvc.ProblemDetails
{
    public InvalidCommandProblemDetails(InvalidCommandException exception)
    {
        this.Title = exception.Message;
        this.Status = StatusCodes.Status400BadRequest;
        this.Detail = exception.Details;
        this.Type = "https://somedomain/validation-error";
    }
}

Last thing to do is to add Problem Details rel=“noopener” target=“_blank”>Middleware with definition of mapping between InvalidCommandException and InvalidCommandProblemDetails class in startup:

// Startup
services.AddProblemDetails(x =>
{
    x.Map<InvalidCommandException>(ex => new InvalidCommandProblemDetails(ex));
});

....

app.UseProblemDetails();

After change in CommandValidationBehavior (throwing InvalidCommandExecption instead Exception) we have returned content compatible with the standard:

Add order validation problem details

Summary

In this post I described:

  • what Data validation is and where is located
  • what Problem Details for HTTP APIs is and how could be implemented
  • 3 methods to implement data validation in Application Services layer: without any patterns and tools, with FluentValidation library, and lastly - using Pipeline Pattern and MediatR Behaviors.

Source code

If you would like to see full, working example - check my GitHub repository.

Comments

Related posts See all blog posts

Domain Model Encapsulation and PI with Entity Framework 2.2
13 February 2019
Domain Model encapsulation implementation with Persistence Ignorance in mind using Entity Framework.
Read More
Domain Model Validation
4 March 2019
In previous post I described how requests input data can be validated on Application Services Layer. I showed FluentValidation library usage in combination with Pipeline Pattern and Problem Details standard. In this post I would like to focus on the second type of validation which sits in the Domain Layer – Domain Model validation.
Read More
Simple CQRS implementation with raw SQL and DDD
4 February 2019
In this post I wanted to show you how you can quickly implement simple REST API application with CQRS using the .NET Core.
Read More