Propagation

In this blog, we will explore the issue of propagation in a microservices architecture and demonstrate a straightforward example that showcases a solution using the Saga pattern.

Introduction

Propagation is a complex problem that occurs in transactions, whether in monolithic applications or microservices.If you have followed the previous articles , we talked a lot about transactions and the saga pattern and we have seen many things.

In this article we are going to dig even deeper and talk about the problem of propagation and we will try to show A POC(proof of concept) on how we can implement propagation within a microservices architecture.

In monolithic applications, this issue of propagation is often resolved using the well-known @Transactional annotation in Spring Boot. However, in the world of microservices, finding a general implementation for transaction propagation is a significant challenge.

Propagation in Monolithic Applications

In monolithic applications, transaction management is relatively straightforward. Spring Boot provides the @Transactional annotation, which handles transaction propagation automatically. This annotation ensures that a method executes within a transaction context, and it manages the transaction boundaries, including committing or rolling back the transaction based on the method’s execution outcome.

Spring supports several types of transaction propagation, which determine how transactions should interact when they are invoked within another transaction. Here are the different propagation types:

REQUIRED (default)

If a current transaction exists, the method joins this transaction.
If there is no existing transaction, a new one is created.
Example: Suppose Method A (with REQUIRED) calls Method B (with REQUIRED). If Method A already has a transaction, Method B will join this transaction.

When placeOrder is called, it starts a transaction. The call to updateUserProfile joins this transaction, ensuring both operations are part of the same transaction.

REQUIRES_NEW

A new transaction is always created.
If a current transaction exists, it is suspended while the new transaction is running.
Example: Method A (with REQUIRED) calls Method B (with REQUIRES_NEW). Method A’s transaction is suspended, and Method B runs in a new transaction. After Method B completes, Method A’s transaction resumes.

When placeOrder is called, it starts a transaction. The call to logTransaction suspends this transaction and starts a new, independent transaction.

The outer transaction can also run without being suspended by using the @Async annotation to run the new transaction asynchronously in a separate thread.

MANDATORY

The method must run within an existing transaction.
If no transaction exists, an exception is thrown.
Example: Method A (with MANDATORY) is called. If no transaction exists, an exception is thrown.

saveOrderDetails requires an existing transaction. If placeOrder doesn't start a transaction, calling saveOrderDetails will throw an exception.

NESTED

A nested transaction is created if a current transaction exists.
If there is no existing transaction, it behaves like REQUIRED.
Nested transactions can be committed or rolled back independently of the outer transaction.
Example: Method A (with REQUIRED) calls Method B (with NESTED). Method B starts a nested transaction within Method A’s transaction.

updateInventory runs in a nested transaction within placeOrder. If updateInventory fails, it can roll back independently, but a rollback in the main transaction will roll back the nested transaction as well.

Key Difference Between NESTED and REQUIRES_NEW:

NESTED: If the outer transaction rolls back, the nested transaction will also be rolled back. Nested transactions can have savepoints and partial rollbacks within the larger transaction.
REQUIRES_NEW: The new transaction is completely independent. If the outer transaction rolls back, it does not affect the new transaction.

SUPPORTS

The method will join an existing transaction if one exists.
If there is no transaction, the method will execute without a transaction.
Example: Method A (with REQUIRED) calls Method B (with SUPPORTS). If Method A has a transaction, Method B will join it. If Method A doesn’t have a transaction, Method B runs without one.

propagationogAccess*** joins an existing transaction if one exists but can also execute without a transaction if none is present.

NOT_SUPPORTED

The method should not run within a transaction.
If a current transaction exists, it is suspended.
Example: Method A (with REQUIRED) calls Method B (with NOT_SUPPORTED). Method A’s transaction is suspended, and Method B runs without a transaction.

sendNotification runs without a transaction, suspending any existing transaction started by placeOrder.

NEVER

The method should not run within a transaction.
If a current transaction exists, an exception is thrown.
Example: Method A (with REQUIRED) calls Method B (with NEVER). If Method A has a transaction, an exception is thrown.

validateOrder must execute outside of any transaction. If placeOrder starts a transaction and calls validateOrder, an exception will be thrown.

Summary

REQUIRED: Join existing or create a new transaction.
REQUIRES_NEW: Suspend existing and create a new transaction.
MANDATORY: Must join an existing transaction.
NESTED: Create a nested transaction within the current one.
SUPPORTS: Join existing or execute without a transaction.
NOT_SUPPORTED: Execute without a transaction, suspending any current one.
NEVER: Execute without a transaction, throwing an exception if one exists.

Propagation in Microservices Applications

In microservices, transaction management becomes more complicated due to the distributed nature of the system. There is no fixed solution for transaction propagation in microservices. In our case,we tried to come up with something on our own where we tried to use a boolean variable to see take into account two cases: The required case as well as the Requires_new case

Imagine a scenario where we have two transactions, t1 and t2. If the variable included is set to true, the inner transaction t2 can roll back the outer transaction t1 and vice versa. If included is false, t1 and t2 are treated as separate transactions, and one does not affect the other.In other words , if t2 fails and rolls back then there won't be any problem on the first transaction.

Let us understand in detail how things work:

1-Required : included (true)

propagationhis case, where included is true, which is equivalent to required in a monolithic @Transactional context, it means that T2 is part of T1. If any local transaction or method fails, all the others will be reverted. We can imagine T2 as another step inside T1.

2-Required_new : included (false)

In the case where included is false, which is equivalent to requires_new (without suspension, we use asynchronous calls) . This means T1 and T2 run in parallel, and neither will roll back the other. They function as two separate transactions.

Implementation

For the implementation, we use the orchestration saga pattern because it provides good visualization and access to the steps, allowing us to easily manage the included variable. This way, we can switch between required and requires_new. Here is a figure describing our process.

We're orchestrating a process involving five services, each representing a local transaction. The saga kicks off in Service 1, triggering the process by emitting an event to Kafka. Orchestrator 1 receives this event, initiating the saga and establishing the workflow steps.

This is the main advantage of using orchestration because simply we manage to position ourselves and to be able to track the workflow.

Here is an example of a step we implemented

The same logic we have done within the orchestration where we tried to implement two main methods the processing method as well as the revert

Let me show the ideal workflow:

In our setup, the workflow entails three key steps: initially, a WebClient call asynchronously activates Service 2 and Service 3. What adds complexity is our introduction of a call to Orchestrator 2 as an additional step. This action initiates the second transaction, T2. Orchestrator 2, in turn, executes two steps: invoking Service 4 and Service 5.

Here is the list of steps within the second orchestrator that takes care of the the second transaction

As you can clearly see , it is the same logic as the first orchestrator.

Before diving even deeper let me show you the placement of the included variable.

If you remember well the main method I was indicating in the orchestration paragraph ; The famous revert saga method, well it will be included there because reverting depends on whether we would include the second transaction within the first or not

propagation a method right?

And here is the variable

and here is the application.yml file

Well,understanding the placement of the included variable becomes crucial. It dictates behavior during the revert function, essentially determining whether the steps involving Orchestrator 2 should be considered.

If the included variable is true, the workflow operates as usual. Orchestrator 1 waits for the responses after sending requests to Services 2 and 3, and Orchestrator 2. Each service responds with 'true' if its operation succeeds, otherwise it responds with 'false'. Orchestrator 2 returns 'false' if one of its coordinated services, 4 or 5, fails; otherwise, it returns 'true'. If the overall return is true, the transaction passes; otherwise, Orchestrator 1 asks all services, Service 2, 3, and Orchestrator 2, to revert. Orchestrator 2 redirects this call to Services 4 and 5.

However, if 'included' is false, Orchestrator 2 responds with 'true' regardless of whether one of its services, 4 or 5, fails. This behavior aids Orchestrator 1 in reverting only if one of services 2 or 3 fails. Consequently, if an issue arises in T1, the revert process will skip Orchestrator 2, avoiding a rollback. If an issue arises in T2, the revert will be performed only on services 4 and 5.

to understand this in detail here is first the saga process relying within the second transaction in the second orchestrator

Without forgetting the revert method

As for the first orchestrator , we have already showed the revert method, now it is time to show the processing method

propagationting**

Now that we have got an overview on how things work, let us test the application and get more depth .

You can find the source code for this blog here

First let’s run the 7 microservices

As well as Kafka

Now let us run the first test case where we will be sending a simple request to the first microservice.

The logic behind our microservices is only about sending a string and saving it the database.

So here is the Post request to the service1

Let us check wether the entity has been saved or not

Well things seems to be great

Now let’s check the other services

Well everything seems to be working just okay.

Let us now think of the revert scenario.

Let us start with the included set to true which means that the failure of a transaction would lead to the failure of all the transaction.

Let’s start by the failure of the first transaction.

for the failure scenario we have place the following condition:

If the string is equal to “service2” then service2 will fail and so on

Now let us check if it has been created or not

As you can see the service has not been saved but in reality is has been created and saved then deleted as part of the rollback we have made

We can also check the other service to see if the entity is created or not

and last the other services

Now let’s move on to another interesting test case wich is the failure of either service4 or 5 but remember included is set to true which means that the failure of the second transaction would lead to the failure of the first transaction

Well for the moment of truth the entity is not saved

Let us now move on to the other scenario where we will change the value of included to false and rerun the orchestrator1

We will try to make the first transaction fail

propagationge src="/media/image41.png" className="microservices_images" width="500" height="404" alt="Image"/> propagationge src="/propagation/image24.png" className="microservices_images" width="500" height="404" alt="Image" />

As you can see the second transaction didn’t fail

Now let us try to make the second transaction fail

As you can see the entity has been saved let us last check the service4

Conclusion

As a conclusion , the problem of propagation remains a complex subject that requires a very wise approach to handle it and requires also the nature of the saga we will implement because surely implementing propagation using choreography will be a lot harder simply because we will find a very big difficulty to make distinction between transactions.However using it in orchestration is surely not that easy however it is more logical to comprehend and to understand.

At first glance when dealing with microservices , it will seem that is very beneficial and will bring much value , however when dealing with some complex tasks like propagation and isolation , we will be obliged to ask ourselves if migrating into microservices is really worth it.