I am creating a workflow composed of several microservices. The domains are divided into different topics. In the example, I can define it as:
- Event processing domain
- Domain of configurations
- Stock inventory domain
Each domain contains its own private database, public APIs, etc. and eventually it runs in Kubernetes.
- Event processing domain: to define the complete workflow of the event life cycle
- Configuration domain: all events must be compared with a complex configuration mechanism, and decide if this event should be discarded or continue the channeling
- Domain inventory inventory: for each event, we need to add information from this inventory.
The flow is composed like this:
I am trying to follow several patterns to achieve that:
- Architecture directed by events, but with commands, through the ESB.
- The entire workflow is defined in the event processing domain, but the tasks themselves are defined in different domains (since they have access to your database, etc.)
My main concerns are:
-I know there are some channeling tools to handle those flows (Data Flow / Flink / etc.), but the problem is that all the workflow and the jars must be handled and written in a great tool, which causes 2 problems Main:
- Responsibilities: each team that is responsible for the domain now has to implement it outside its domain, and without a database and more access to the data, since it is executed in a different context
- A small monolith is created, all teams must implement and write code in this central
– Those services really manipulate the data that is received, it is not a simple command (do this, to that), so it fits more to a pipe service elsewhere.
I'm trying to choose the best strategy.
- A pipe that must be defined / changed easily (add more conditions, change the order, etc.)
- Avoid direct communication from service to service and use event-driven architecture
- The most challenging problem: the fact that each microservice develops independently
2 tools that I am reviewing –
Any discussion would be appreciated 🙂