it might be too opinionated, but I’ve been struggling in this for far too long and cant seem to make up my mind.
I am trying to compare two approaches to designing a system:
Lets assume You want to crawl a site periodically,
if some condition is met, you want to take a snapshot of the webpage.
all that should be recorded in a datastore.
I split the job into 3 services:
- crawler-svc: query the web-page and provide its content data
- analytics-svc: receive content-data and decide its relevance
(if relevant, we should snapshot)
- evidence-svc: collects evidence about the web-page incl. snapshot of the site
I struggle to decide between two approaches,
described from 50 mile up:
Option 1: Orchestrating the operation with a “master” service (either a new service, or the analytic-svc).
the master-svc will trigger request to crawler-svc, hand the results to analytic-svc, and depending on result will invoke evidence-svc.
Option2: Each service is tailor made for the environment:
e.g. cawler-svc knows it should trigger periodically,
analytic svc waits for “content-data-ready” events and
evidence-svc waits for “relevant-page” events.
Please specify with your opinion the major reasons you rather have one over the other.