I have multiple instances on the same worker that process long-lasting tasks. Usually, those tasks last about 30 minutes – 5 hours. Tasks are stored in RabbitMQ. Workers are deployed as Kubernetes single-container deployment with multiple replicas.
The problem is deploying new changes. I see two strategies here: interrupting current processing or deploying new workers and letting existing one’s exit by itself.
I chose the first strategy that lets me deploy new changes quickly. After a deploy is finished I could be ensured that all workers use the same codebase. But there are downsides. I need handling exit signal, task processing restarting, restoring state, checking for should I insert or update records, and so on.
So my question is, could I say that interrupting current processing to deploy new changes is a best-in-class solution? Are there other approaches here?
Link buildingis the practice ofbuildingone-way hyperlinks (also known as “backlinks”) to a website with the goal of improving search engine visibility. Commonlink buildingstrategies include content marketing,buildinguseful tools, email outreach, brokenlink buildingand public relations.
NOTE : Please contact me first before making an order so we can discuss the details and scope of the required program.Are you looking for a programmer to create an automated applications for your bulk tasks? I will provide 5 years of expert level experience in different programming fields.
I will do:
Automated web scraping & data mining
Automated VBA macros
Facebook / Instagram / Twitter automations
and various tasks according to your requirements
Languages I use: Final application will be offered as an user friendly portable (.exe) application with graphical user interface (GUI), so you can perform your actions in less time and more focused. Also you will get a full documentation about your application in final delivery.
I am always open for your support and feel free to message me anytime to get my help.
I have a data synchronisation concurrent algorithm. It does the following: get data and files from server, send data and files to server, save them to database / filesystem. Imagine the system like this:
You have 1000 functions. Each one does some atomic operation. For instance, fetch latest objects of type X and insert them into DB; upload this file of type Y and so on. Each function is independent and can act on its own, it does not communicate with or affect other functions. On the other hand, none of them is a pure function, because they all use theese common resources (fetching data from the server, puting data on DB, saving files on filesystem)
You have a single entry point for the sychronization mechanism. The outside of the sync system can start the sync, say, by doing a Sync.start() call. Also, the sync has a single exit point. The sync can finish with either success, either failure (if any of those functions from (1) fail, the whole sync will fail). The ouside of the sync system can subscribe to onSyncSuccess / onSyncError events.
You have this black box in the middle of the system. This could be, for instance, a single threaded algorithm calling those 1000 functions from (1). But I made it concurrent.
Now consider this. This concurrent algorithm right now is rigid because the way in which the functions are called is hardcoded. If I want to take a bunch of functions from (1) that right now are executing sequentially, and if I want to make them execute parallel, it would be impossible without refactoring the whole class hierarchy.
I was thinking about the concept of direct acyclic graphs, and I made my own domain-specific language in Kotlin to define such task graphs. Now I could write the whole orchestration declaratively like this:
So first task1 gets executed, then task2 and 3 in the same time, then task4. By keeping this graph in a single file, I could easily modify the way tasks are executed. For instance, I could easily swap tasks:
Here, (task 4 and 2) gets executed, then 3, then 1. This works by using the fork-join paradigm, I create threads then join them into the parent thread.
In contrast, right now, the algorithm is spread around multiple classes, each of them was designed to run the tasks in a specific manner. Changing how tasks are ran would mean to refactor the classes and how they communicate to each other.
The question is: What is the best way to decouple and define the orchestration (coordination) of concurrent tasks? So that this orchestration could be easily changed in the future? Is my solution optimal or the way to go (direct acyclic graphs, fork-join, plus a domain specific language)? Or maybe there are some other design patterns that do the same thing?
I am trying to fetch List details along with associated task columns to export into Excel using PowerShell. I am able to get all columns using following script except Assigned To column to show with which user the task is pending with from associated task list.