Oracle failsafe – Database Administrators Stack Exchange

Stack Exchange Network


Stack Exchange network consists of 177 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

How to make AWS Redshift ETLs failsafe

Currently, the processing of heavy data on the redshift of AWS is spilled on the disk and, if that is not enough, they will be overridden.
Could you solve such problems with AWS tools such as custom workload manager (wlm) queues and the new automatic scale (concurrency scale)?

I am also considering a coding solution: if I could verify with RS how much free memory and disk it has at the time of ETL execution
then the code would divide heavy operations into a series of smaller operations if the resources of the cluster were beyond some limits.
Does this make sense to you?
Is it possible to check these parameters from AWS RS?

Thanks for your thoughts!
Rea

Object-oriented: fail-safe design versus limiting construction logic

In any programming task, my preference is to write fail-safe code. That doesn't seem to be too controversial. However, I have also seen many developers say that builders should do as little as possible. I am discovering that these two objectives often disagree, and conversations about failsafe design versus simplicity of the builder sometimes become statements of preference.

Consider a class that provides write access to a file. The class accepts a file path in its constructor that must be a path to an existing file. Some developers would say that a constructor should not access the file system, so it could do a null check:

private readonly string file;

public FileAccessProvider(string file)
{
    this.file = string.IsNullOrEmpty(file)
        ? throw new ArgumentException(nameof(file)) : file;
}

Now, if the file provided does not exist, we may not know until the class actually tries to write data to the file. In this case, would it be acceptable to verify the existence of the file in the constructor?

private readonly string file;

public FileAccessProvider(string file)
{
    this.file = string.IsNullOrEmpty(file)
        ? throw new ArgumentException(nameof(file)) : file;

    if (!File.Exists(file))
    {
        throw new ArgumentException("File does not exist.");
    }
}

This complicates my builder, but it adheres better to the fail-safe design. We could even take this one step further and add code to the constructor that verifies if the user has write permissions to the file.

What about a class that has access to a database or network resources?

Is there an accepted side to err in situations like this?