Microservices with minimum overhead using ASP.NET Web API and Azure – part 1 – Architecture

In the era of building systems that aim to provide services at global scale, requirements for scalability and high availability are becoming our bread and butter. What is more, it is absolutely normal that stakeholders want first shippable version of software as soon as possible.

Due to recent hype, microservices architecture comes to our mid as the first answer to mentioned challenges. However, as it is commonly known: “(…) microservices introduce complexity on their own account. This adds a premium to a project’s cost and risk – one that often gets projects into serious trouble” (Martin Fowler). We can see this requirement does not help to deliver system fast. So where is the golden mean? How to build fast, but be prepared to scale and provide high availability without dramatic, expensive changes to system? Let me propose architecture that answers those questions. At least some of them.

Overview

The main idea of this article is to organize system into multiple lightweight, logical, decoupled components, rather than multiple independently hosted services. With this approach we can start with hosting all components in single service and dividing them into multiple ones overtime.

It looks nice as an idea, however what actually hosts and those “mysterious” components here are.

Hosts are, in short, ASP.NET Web API (or MVC) applications that can be hosted as Azure Web Apps or started on developers’ machine. Their only responsibility is to host and initially configure application, but there should be no application logic, or simply: no controllers.

What is this component then? Let’s zoom in!

In the suggested architecture, component is a bunch of Web API (MVC) controllers with all related business logic. All those classes are placed outside of ASP.NET Web API host application, usually in simple class library. Single component should consist of controllers and logic related to single business functionality. Usually, each component has its own place to store data, it can be common or dedicated database.

To put all this together, host project just need reference class libraries containing components code. Since now, controllers are available to process HTTP requests sent to host application. Moving component to separate host is as simple as changing reference between projects.

With this approach we benefit from most of microservice architecture advantages, and initially limit pain that comes with it by keeping development environment easy to run on one machine, limiting DevOps work, and cutting off a lot of distributed system problems. Additionally, we achieve flexibility of production environment costs. Initially limited number of machines do not dry out our accounts, however, when necessary, we are able to distribute components to multiple independent hosts, achieving better scalability.

Building actual system

Let’s assume we are creating social system, where users can find and rate nearby shops and markets. For scope of this article we want to:

• Create, read, update and delete data about markets and user’s ratings.
• Generate suggestions what another markets user may like too.

And of course our stakeholders want the first version of project quickly, however, at some point after release, they expect system to handle millions of users from entire word.

Let’s build it

Project structure

First of all, we need basic project structure.

1. Create empty ASP.NET Web Application with Web API.
2. Create empty Class Library and install Microsoft.AspNet.WebApi NuGet Package in it.
3. Add reference from Web API project to Class Library.

After those steps your solution should look like this:

Market management service implementation

As the next step, we want to implement CRUD operations within MakretManagementService. For simplicity of this example I’ve just scaffolded MarkedController and RatingsController using Visual Studio tools.

Now, we can test if this solution actually works. To do so we need appropriate tool e.g. Fiddler or PostMan to send HTTP requests to our services. Works on my machine!

Create Analytics service

The last feature we need before release is suggestions generation. Such feature is expected to be time consuming operation that we want to run periodically, and store results in database so they can be accessed by client applications later on. What is more, in future we may not want this logic to run at the same machine that MakretManagementService, because this heavy operation might slow system down. For this reason we are going to create it as a separate component.To achieve that we just need to:

1. Create empty Class Library and install Microsoft.AspNet.WebApi NuGet Package in it.
2. Add reference from Web API project to Class Library.

Code itself is not important here, so just skip discussion on this.

Now we can send POST request to http://<<base_address>/RecommendationsAnalysis and have our suggestions created.

At this stage system looks as follows:

It is important to notice at this point that there is absolutely no coupling between two services on the code level, however as you might noticed it doesn’t come for free. There is some duplication in data access code and data model classes. Although, at this moment, it can be generalized and placed in common class library, it may not be the best solution when we think about long term perspective and further independent development of those services. There are no good and bad solutions, there are only pros and cons in particular context.

Sounds like we are ready to ship our Minimum Viable Product to customers!

Time for scaling

Until now we have created application optimized for low development and hosting costs. It runs as a single application and store data in single database. Let’s now react to scenario where we already have a lot of data about markets, single database is getting large, calculating user recommendation takes long enough to significantly slow website down for long periods.

Some improvements that should help in a given problem are:

1. Move recommendations data to different database.
2. Run Analysis service in separate host application, so we can run it in independent container.

The first change is nearly trivial. We need to create another database with Recommendations table. Next, because our system is already running, we need to migrate data from first database. After that there is a single connection string to change in web.config.

Such change will be that easy only if we took care not to introduce coupling between data behind different components. It is not simple and, again, usually requires some data duplication and synchronization, however that is the cost for possibility of keeping parts of system independent. If you are familiar with Domain Driven Design concept of bounded contexts, you have probably noticed that what I’m suggesting here is similar to concept of aggregates and persisting them as an internally cohesive, decoupled from each other set of entities.

The second change isn’t complicated too, all you need to do is to create new host projects (same way as previously) and add references to appropriate class libraries.

Now you can enjoy system running its components in two microservices. From broader perspective it works this way:

You might have noticed that I’ve left in solution MarketFinder.CommonHost project that is hosting both applications. This project is useful during development, as it is usually faster to start single web site than multiple ones, especially when we have a lot of them.

Summary

Most of community advice against starting system development with microservice architecture, because it comes with huge costs and complexity overhead. In return, it is recommended to build monolithic system and extract microservices overtime when required. Advised approach sounds great, but to divide system into services in future, we need to make sure we do not create highly coupled ball of mud. Architectural approach presented in this article may help us by creating some logical boundaries of system components. Having them prevent us, and our less experiences teammates, from introducing hidden coupling.

However, nothing in software development is the silver bullet. Suggested approach introduces less complexity and cost overhead than fully blown microservices architecture, however it still does. The biggest problem when cutting system on independent components, is that it can never be done without actually cutting of some dependencies that exist. If we find boundaries where those dependencies are minimal, we will benefit from low coupling in future, however when we set those boundaries wrongly, we will heavily suffer from development complexity and probably poor system performance.

Part two of this article can be found here:
https://kariera.future-processing.pl//microservices-with-web-api-and-azure-deployment/