dependency injection – What are the disadvantages to using this form of pure DI in Python?

I want to perform dependency injection in my Python application without using reflection. My position on reflection is informed largely by the “don’t hide things” philosophy (I am unable to find a decent reference to this on google at the moment). What that basically means to me is that code ought to be easily discoverable by an incoming programmer (either your own code at some point in the future, or another programmer coming in to edit or extend your code).

It’s important for me to note here that my primary motivation for dependency injection is the ability to easily test individual services and dependencies without using Python’s patch, but instead to rely on an easily accumulated set of reusable mock dependencies.

Setting aside the reflection debate for the moment, here is one way I see that one can accomplish dependency injection without reflection.

The Container

The container houses my Python application’s dependencies. It could be a simple dictionary.

# container.py

dependencies = {}

The Composition Root

The point of the composition root is to have a centralized location where all needed application dependencies are drawn together and configured in whatever way is necessary. Some classes, for example, might need to be instantiated with config-specific parameters. In Python, I have not seen something like this in the wild. It seems like the composition root needs to be imported specifically by an environment-specific, executed module.

The below illustrates an example where a hypothetical dependency TaskManager depends on a service ApiService. TaskManager, in turn, is a service used throughout the application.

# run_prod.py - runs a production-configured instance of my application

from container import dependencies
from services import TaskManager, ApiService
import config
from app import App


api_service = ApiService(config.api_service_endpoint)
task_manager = TaskManager(api_service)
dependencies('task_manager') = task_manager

app = create_app(dependencies)
app.run(host=config.host, port=config.port)

The App

The application itself at this point is more or less a shallow orchestrator of the services injected into it, since all chunks of code that need to be tested in isolation are defined outside of it, and injected into it in this scheme.

# app.py

def create_app(dependencies)
    app = Flask(__name__)
    
    @app.route('/task/<task_id>', methods=('GET'))
    def get_task(task_id):
        task = dependencies.get('task_manager').find_task_by_id(task_id)
        return task

Another similar approach might be to import the dependency container, rather than inject it as an argument to the create_app function. In that case I don’t need the constructor function:

# app.py

from container import dependencies

app = Flask(__name__)
    
@app.route('/task/<task_id>', methods=('GET'))
def get_task(task_id):
    task = dependencies.get('task_manager').find_task_by_id(task_id)
    return task

Testing as a primary motivation for dependency injection

We can create as many use-specific composition roots as we want. For example, if we create one for testing, instead of having a separate run_prod.py, we can just compose the dependencies in the same module as the test (or in a separate module, for re-use). Let’s go with module test_flask_endpoint.py.

For this example, let’s say we want to test the TaskManager together with the Flask layer, but don’t want to make outgoing HTTP calls. We mock out ApiService.

# test_flask_endpoint.py
import pytest

@pytest.fixture
def test_app():
    from services import TaskManager

    class MockApiService():
        def get_task_details(task_id):
            {"task_id": task_id, "description": "does something special"}

    mock_api_service = MockApiService
    task_manager = TaskManager(mock_api_service)
    dependencies('task_manager') = task_manager

    app = create_app(dependencies)
    test_app = app.test_client()
    return test_app


def test_task_endpoint(test_app):
    assert test_app.get('/task/3').json()('description') == 'does something special'

One disadvantage I see to doing DI this way is that the app is coupled to the injector, namely, the dependencies argument of create_app. Is there a way around this? It is not clear to me that this matters.

relational database – Circular dependency with 1-to-1 relationship

I am developing a patient registration system where patients can book medical appointments either directly with my system or via online services. One same patient can come over time through different booking providers and should be recognised as the same person. Each service has their own patientID format. Also, for each booking provider, the same patient can book multiple appointments over time, each with their own appointmentID format.

I am keeping my own Patient and Appointment tables (with my own patientID and appointmentID), but for the sake of accounting and analytics I should also keep track of the source of each appointment (external patientID, appointmentID and booking service).

This is the relevant part of the ERD:
enter image description here

Each person is one patient in the Patient table, but could be more than one in the Booking_Patient table, one for each Booking_Provider. Each entry in the Booking_Patient table can have one or more appointments in Provider_Appointment.

Each appointment in the Provider_Appointment table corresponds to exactly one appointment in the Appointments table.

As you can see, there are two potential issues:

  1. a circular dependency
  2. a 1-to-1 relationship

Am I doing something wrong? Can I improve this design somehow?

python – Pipenv / Pipfile lock dependency issues

I have the following Pipfile:

((source))
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

(dev-packages)
pytest = "*"
pytest-cov = "*"
black = "*"

(packages)
boto3 = "*"
awscli = "*"
pandas = "*"
setuptools-scm = "*"
dask = {extras = ("dataframe"), version = "*"}
fastparquet = "*"
s3fs = "*"


(requires)
python_version = "3.7"

(pipenv)
allow_prereleases = true

The problem is occurring when I try to pipenv lock (create a pipfile lock) between a sub-dependency in boto3 and s3fs, which is botocore. Error comes up as:

(pipenv.exceptions.ResolutionFailure): Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Could not find a version that matches botocore
There are incompatible versions in the resolved dependencies:
  botocore<1.17.45,>=1.17.44 (from aiobotocore==1.1.2->s3fs==0.5.1->-r /var/folders/1p/55zf00x17kv1bw300f79xsnr0000gn/T/pipenvvslkzte6requirements/pipenv-r37de33l-constraints.txt (line 6))
  botocore<1.20.0,>=1.19.18 (from boto3==1.16.18->-r /var/folders/1p/55zf00x17kv1bw300f79xsnr0000gn/T/pipenvvslkzte6requirements/pipenv-r37de33l-constraints.txt (line 8))
  botocore<2.0a.0,>=1.12.36 (from s3transfer==0.3.3->boto3==1.16.18->-r /var/folders/1p/55zf00x17kv1bw300f79xsnr0000gn/T/pipenvvslkzte6requirements/pipenv-r37de33l-constraints.txt (line 8))
  botocore==1.19.18 (from awscli==1.18.178->-r /var/folders/1p/55zf00x17kv1bw300f79xsnr0000gn/T/pipenvvslkzte6requirements/pipenv-r37de33l-constraints.txt (line 13))

Here’s my pipenv graph:

boto3==1.16.18
  - botocore (required: >=1.19.18,<1.20.0, installed: 1.19.18)
    - jmespath (required: >=0.7.1,<1.0.0, installed: 0.10.0)
    - python-dateutil (required: >=2.1,<3.0.0, installed: 2.8.1)
      - six (required: >=1.5, installed: 1.15.0)
    - urllib3 (required: >=1.25.4,<1.27, installed: 1.25.11)
  - jmespath (required: >=0.7.1,<1.0.0, installed: 0.10.0)
  - s3transfer (required: >=0.3.0,<0.4.0, installed: 0.3.3)
    - botocore (required: >=1.12.36,<2.0a.0, installed: 1.19.18)
      - jmespath (required: >=0.7.1,<1.0.0, installed: 0.10.0)
      - python-dateutil (required: >=2.1,<3.0.0, installed: 2.8.1)
        - six (required: >=1.5, installed: 1.15.0)
      - urllib3 (required: >=1.25.4,<1.27, installed: 1.25.11)
dask==2.30.0
  - pyyaml (required: Any, installed: 5.3.1)
docutils==0.15.2
fastparquet==0.4.1
  - numba (required: >=0.28, installed: 0.52.0rc2)
    - llvmlite (required: >=0.35.0.dev0,<0.36, installed: 0.35.0rc2)
    - numpy (required: >=1.15, installed: 1.19.4)
    - setuptools (required: Any, installed: 49.2.1)
  - numpy (required: >=1.11, installed: 1.19.4)
  - packaging (required: Any, installed: 20.4)
    - pyparsing (required: >=2.0.2, installed: 3.0.0b1)
    - six (required: Any, installed: 1.15.0)
  - pandas (required: >=0.19, installed: 1.1.4)
    - numpy (required: >=1.15.4, installed: 1.19.4)
    - python-dateutil (required: >=2.7.3, installed: 2.8.1)
      - six (required: >=1.5, installed: 1.15.0)
    - pytz (required: >=2017.2, installed: 2020.4)
  - six (required: Any, installed: 1.15.0)
  - thrift (required: >=0.11.0, installed: 0.13.0)
    - six (required: >=1.7.2, installed: 1.15.0)
partd==1.1.0
  - locket (required: Any, installed: 0.2.0)
  - toolz (required: Any, installed: 0.11.1)
s3fs==0.5.1
  - aiobotocore (required: >=1.0.1, installed: 1.1.2)
    - aiohttp (required: >=3.3.1, installed: 4.0.0a1)
      - async-timeout (required: >=3.0,<4.0, installed: 3.0.1)
      - attrs (required: >=17.3.0, installed: 20.3.0)
      - chardet (required: >=2.0,<4.0, installed: 3.0.4)
      - multidict (required: >=4.5,<5.0, installed: 4.7.6)
      - typing-extensions (required: >=3.6.5, installed: 3.7.4.3)
      - yarl (required: >=1.0,<2.0, installed: 1.6.2)
        - idna (required: >=2.0, installed: 2.10)
        - multidict (required: >=4.0, installed: 4.7.6)
        - typing-extensions (required: >=3.7.4, installed: 3.7.4.3)
    - aioitertools (required: >=0.5.1, installed: 0.7.1)
      - typing-extensions (required: >=3.7, installed: 3.7.4.3)
    - botocore (required: >=1.17.44,<1.17.45, installed: 1.19.18)
      - jmespath (required: >=0.7.1,<1.0.0, installed: 0.10.0)
      - python-dateutil (required: >=2.1,<3.0.0, installed: 2.8.1)
        - six (required: >=1.5, installed: 1.15.0)
      - urllib3 (required: >=1.25.4,<1.27, installed: 1.25.11)
    - wrapt (required: >=1.10.10, installed: 1.12.1)
  - fsspec (required: >=0.8.0, installed: 0.8.4)
setuptools-scm==4.1.2
  - setuptools (required: Any, installed: 49.2.1)

Reading the above, it looks like aiobotocore requires specifically botocore 1.17.44, but botocore in boto3 requires at least 1.19.18. What’s the best way to solve this in a way that I can use these packages without the dependency hangup I’m experiencing?

dependency injection – Simple constructor DI implementation in Rust

From my experience in C# programming I think that DI is important. But it’s not possible to do it same way in Rust. There are some DI frameworks but I’ve come with an idea on how it can be made and decided to implement it my way. My implementation uses only constructor injection, no container needed. I’m only learning Rust now so can you review my code, find possible pitfalls, suggest improvements or better approaches?

My goals in DI:

  1. Make stubs possible for testing.
  2. Systems shouldn’t know “sub-dependencies” of their dependencies. Separate construction and providing dependencies from actual usage. All dependencies are “wired” in one place.
  3. No global shared state.
  4. The design principle that higher level systems declare what they need and lower level systems implement it, not other way.
  5. Avoid God objects.
  6. All dependencies can be clearly determined by a constructor signature, no looking inside code needed. Therefore I don’t use Resource Locator pattern.
#(cfg(not(feature = "test")))
mod Bindings {
    // real implementations binding

    // I assume only one implementor per each interface
    // this is not always the case but al least it's good for a simple scenario
    pub type Door = crate::Autocar::DoorMod::DoorImpl;
    pub type Engine = crate::Autocar::EngineMod::EngineImpl;
}

// how can I move these into actual tests files?
#(cfg(all(feature = "test", feature = "test1")))
mod Bindings {
    // stubs for test1 can be binded here
    pub type Door = crate::Autocar::DoorMod::DoorImpl;
    pub type Engine = crate::Autocar::EngineMod::EngineImpl;
}

#(cfg(all(feature = "test", feature = "test2")))
mod Bindings {
    // stubs for test2 can be binded here
    pub type Door = crate::Autocar::DoorMod::DoorImpl;
    pub type Engine = crate::Autocar::EngineMod::EngineImpl;
}

// prelude for internal use
mod Usings {
    pub use crate::Bindings::*;
    pub use std::cell::RefCell;
    pub use std::rc::Rc;
    pub type Mrc<T> = Rc<RefCell<T>>; // Mutable Reference Counter
    pub fn Mrc<T>(v: T) -> Mrc<T> {
        Rc::new(RefCell::new(v))
    }
}

fn main() {
    // this code performs constructor injection itself
    // all constructors are called here

    use Autocar::*;
    use Usings::*;

    let engine = Mrc(Engine::new());

    // also we can make factory methods
    let make_door = || -> Door { Door::new(engine.clone()) };

    let doors = vec!(make_door(), make_door());
    let mut car = Car::new(engine, doors);

    // all constructed, now run something
    car.doors(0).open();
}

// now application code
mod Autocar {
    use crate::Usings::*;

    // top-level struct so no interface
    pub struct Car {
        // Since same Engine is used also by a Door too, I have to use Mrc.
        // This may become an issue as once a dependency becomes
        // used by multiple structs I have to change it everywhere to Mrc
        // and insert borrow_mut() everywhere.
        // Which doesn't look like a good design. But no choice. Or?
        pub engine: Mrc<Engine>,

        pub doors: Vec<Door>,
    }

    impl Car {
        pub fn new(engine: Mrc<Engine>, doors: Vec<Door>) -> Car {
            Car { engine, doors }
        }
    }

    // declare Car dependencies:

    // we actually need IDoor so stubs can inherit it and reflect signature changes when refactoring
    pub trait IDoor {
        fn is_opened(&self) -> bool;
        fn open(&mut self);
    }

    pub trait IEngine {
        fn is_running(&self) -> bool;
        fn start(&mut self);
        fn stop(&mut self);
    }

    pub(crate) mod DoorMod {
        use super::*;
        use crate::Usings::*;

        pub struct DoorImpl {
            // I tried to design the code in a way so that DI doesn't prevent optimizations.
            // So I don't use IEngine here or otherwise it becomes dyn implicitly and then
            // no inlining and can't be placed on the stack.
            // But one issue with this approach is that IntelliSense can see
            // all EngineImpl functions even if it implements multiple traits, not just IEngine.
            // But a stub will contain only interface-declared functions
            // so it will be at least checked by the compiler.
            engine: Mrc<Engine>,
        }

        impl IDoor for DoorImpl {
            fn is_opened(&self) -> bool {
                unimplemented!()
            }

            fn open(&mut self) {
                if self.engine.borrow().is_running() {
                    self.engine.borrow_mut().stop();
                }
                println!("opening")
            }
        }

        impl DoorImpl {
            pub fn new(engine: Mrc<Engine>) -> Self {
                DoorImpl { engine }
            }
        }
    }

    pub(crate) mod EngineMod {
        use super::*;
        use crate::Usings::*;

        pub struct EngineImpl;

        impl IEngine for EngineImpl {
            fn is_running(&self) -> bool {
                true
            }

            fn start(&mut self) {
                println!("starting");
            }

            fn stop(&mut self) {
                println!("stopping");
            }
        }

        impl EngineImpl {
            pub fn new() -> Self {
                EngineImpl {}
            }
        }
    }
}

How do we model Domain entity that has dependency on Config classes?

So for e.x I have an entity Payment and a config Payment settings.

class Payment 
{ 
    public Payment(PaymentSetting setting, ....other fields){}
    internal Payment() /*strictly for the ORM**/ 
    //Other properties and behaviour
} 
class PaymentSettings 
{ 
    public Timespan TransactionTimeout {get; private set;} 
    //Other properties and behaviour
} 
class Order /AggregateRoot*/ 
{ 
    List<Payment> payments {get; private set;} 
    //Other properties and behaviour
}

There is no problem when I am going to create Payment object as I can inject the setting to constructor. But How do we inject config when the Order is loaded from DB?

I understand in the current example, Config is needed while creating the appointment only, and we can store the exact value as part of Payment object and revive it when needed. But what happens, if we need to know the latest config? Should add Methods to inject it from outside? Is it considered an acceptable practice wrt. DDD

apt – I am currently trying to install CMake to Ubuntu-20.04 via WSL but I get dependency error

So what I did so far was install CMake to configure my CLion so that I can run Valgrind. However, after following a few suggestions, I have encounter this problem and I cannot resolve it.

I installed 3.16 CMake, but I need the newest version of CMake, so I added an apt repo from kitware. But now I am getting the following message when I try to reinstall CMake using apt-get.

rleung@DESKTOP-HV6P4HF:/mnt/c/Users/ryanl$ sudo apt-get install cmake
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 cmake : Depends: libssl1.0.0 (>= 1.0.2~beta3) but it is not installable
E: Unable to correct problems, you have held broken packages.
rleung@DESKTOP-HV6P4HF:/mnt/c/Users/ryanl$ apt-cache policy libssl1.0.0
libssl1.0.0:
  Installed: (none)
  Candidate: (none)
  Version table:

Kepp up systemd service after stopping its dependency

I have realized that when some dependency (e.g Elastic) is stopped, the service depending on it is stopped too. I would like to have strict dependency when starting (therefore requires) but I would prefer not to be killed afterwards. This machine is for developers and they restart / redeploy app quite often.

(Unit)
Requires=nginx.service rta-kibana.service redis.service postgresql.service rabbitmq.service
After=nginx.service rta-kibana.service redis.service postgresql.service rabbitmq.service

Is this possible?

Is dependency inversion principle necessary?

I’ve read a lot about dependency inversion principle but still, I can’t apply it to my case. I just don’t know when I should apply it and when not. I write a simple application in Java to generate invoices. For now, I have basic classes: Client, Product, InvoiceLine and Invoice. Should these classes communicate through interfaces? For instance, I have a method in Product for getting name of the prduct:

public String getName() {
        return name;
    }

And I use this method in class Invoice

public void addLineToInvoice(Product product, int quantity) {
        rows.add(new InvoiceLine(rows.size(), product.getName(), quantity, product.getPrice()));
    }

Now, should I create an interface for Product? Or is it unnecessary?

node.js – NPM dependency in package lock is wrong

I have 2 private projects (Project A and Project B) that make use of a private library (Library). All of these are hosted in a private Bitbucket account. Now, Project A has a dependency on Library v5 and Project B depends on “Library v3” BUT Project A ALSO depends on Project B. Something like this:

/ Project A
|_  Library v5
|_  Project B
    |_  Library v3

This is Project A’s package.json:

{
  "dependencies": {
    "library": "git+ssh://git@bitbucket.org/me/library.git#v5"
    "project-b": "git+ssh://git@bitbucket.org/me/project-b.git"
  }
}

And this is Project B’s package.json:

{
  "dependencies": {
    "library": "git+ssh://git@bitbucket.org/me/library.git#v3"
  }
}

Now, according to NPM’s resolution strategy, Project B should be in Project A’s node_modules but it should also have its own node_modules with Library#v3 pulled in. The problem is that for some reason, and no matter the order in which I install them, npm always updates my package-lock.json and says that Project B depends on Library#v5. Here’s how Project A’s package-lock.json look like if I install Project B first:

"project-b": {
  "version": "git+ssh://git@bitbucket.org/me/project-b.git#<hash>",
  "from": "git+ssh://git@bitbucket.org/me/project-b.git#<hash>",
  "requires": {
    "library": "git+ssh://git@bitbucket.org/me/library.git#v3",
  },
  "dependencies": {
    ...
  }
}

And after I install Library v5:

"library": {
  "version": "git+ssh://git@bitbucket.org/me/library.git#<hash>",
  "from": "git+ssh://git@bitbucket.org/me/library.git#v5",
  "requires": {
    ...
  },
  "dependencies": {
    ...
  }
},
"project-b": {
  "version": "git+ssh://git@bitbucket.org/me/project-b.git#<hash>",
  "from": "git+ssh://git@bitbucket.org/me/project-b.git#<hash>",
  "requires": {
    "library": "git+ssh://git@bitbucket.org/me/library.git#v5",
  },
  "dependencies": {
    ...
  }
}

Why? I know I’m not supposed to manually edit the package-lock.json but what am I doing wrong? I’m using node lts/dubnium (v10.22) and npm 6.14 on Windows.

domain driven design – DDD Validator Specification with dependency on repository

Don’t inject repositories or other dependencies into the domain, not even thru their interfaces.

I can’t have the property Color with value “Red” if there are 10 or more entity Car with the Color = “Red”.

I can’t have the property Color with value “Red” if the entity “Warehouse” haven’t the entry with item code “FF0000”.

The Car entity could have a method or constructor that takes the required information as arguments, instead of fetching it from a repository. For example:

public static Car Create(Model model, Engine engine, Color desiredColor, IReadOnlyList<ColorCount> usedColors, IReadOnlyList<Color> availableColors)
{
   // do validation on usedColors and availableColors
   return new Car(model, engine, desiredColor);
}

Before calling this method from an application service, you fetch the information from the repository. The usedColors collection contains the number of times each color is used and the availableColors collection contains all the colors in the warehouse.

This way, all decisions can be made inside the domain, without any out-of-process dependencies.