testing – Why is the test runner not picking up tests for a contrib module?

I recently added tests to Cache Register, but the test runner on drupal.org doesn’t seem to be picking them up. The tests are all located in tests/src/KernelTests (repo) but aren’t queuing up in the issue queue for either patches or merge requests. Am I missing something?

EDIT: Current answer references the namespace of the base test class, but I needed to do it that way to make the autoloader work for tests on my local. The tests themselves do actually use the correct namespace. I suspect that the way I’m implementing the base class may be the actual issue so I’m trying to figure out a way around that. Having trouble finding an example of a contrib module that uses a base class for its tests, though.

EDIT 2: Refactored that Base class into a testing Trait instead. Remaining “real” tests already use the right namespace and run locally, but still not getting picked up.

git – Quick “Github flow” releases without integration tests

We’ve recently moved to the Github flow branching model (feature branch out of master > code > deploy completed feature branch into qa env & test > once approved, merge back to master > deploy to production).

Currently, our QA can test each feature branch separately and approve it which will let us merge it back to master. However, since we don’t have automated integration tests, our team asks for a manual integration test of the master branch before a new release is sent out of it for UAT & then to production.

In case of not having automated integration tests, is our only way forward to have a manual integration test each time we want to release out of master? Is there a faster way?

p.s. We’re planning of covering our modules with proper automated integration tests, but a somehow quick approach until we do so will be helpful.

How do I set up unit tests in Unity?

I have followed the Unity Test Framework documentation to create an EditMode test assembly and a test script. When I try to test a function like Utils.SomeMethod(), the class is not recognized.

My IDE gives me the option to Reference Assembly-CSharp and use Utils which seems to successfully import the class as I gain autocomplete ability on the function names. But as soon as I try to run the test, the text changes back to red and I get a message that The name Utils does not exist in the current context.

What do I need to do to be able to test my class?

architecture – Properly testing UI presenters without duplicating tests of subcomponents

I am writing the embedded firmware for an effect pedal. The pedal’s ui consists of a few knobs
a few buttons and a few leds and it consists of various control modes each corresponding to a seperate effect. So each mode changes the meaning of the knobs, the buttons and led indications.

This is implemented as a sum type of “presenters”. Each presenter is injected with the corresponding effect model and the buttons + leds drivers. Each presenter has it’s own set of shortcuts and button controls. Each presenter owns the corresponding view and event emitter.

I am trying to figure out how to write integration tests for these presenters. What I currently do is pinpoint the high – level button events required for each presenter. Then define a state machine using the high-level events and define the appropriate model/view interactions for each transition. Then I use model-based testing to test that the implemented state machine properly calls everything.

I independently test the high-level event emitter which produces the events for the state machine.

I find this helps a lot with properly testing the interactions since writing the model-based tests for the state machine using high-level events is much more expressive than using the buttons driver directly.

So as a summary each presenter basically just delegates to the event emitter and passes the events from the event emitter to the state machine (called controller in the code) which controls the model and view.

A final presenter may look like this:

    void delay_presenter::update()
    {
        button_event_emitter_.poll();

        bool exit_event = button_event_emitter_.test_and_clear_exit_event();
        bool tap_event = button_event_emitter_.test_and_clear_tap_button_event();
        std::optional<button_event> enable_event = button_event_emitter_.test_and_clear_enable_button_event();

        bool alternative_modified = alternative_modifed_event_emitter_.poll();

        if (exit_event)
            controller_.handle_exit_event();

        if (tap_event)
            controller_.handle_tap_button_event();

        if (enable_event)
            controller_.handle_enable_button_event(*enable_event);

        if (alternative_modified)
            controller_.handle_alternative_modified_event();

        controller_.handle_switch_position(gpios_.switch_position());

        model_.set_inputs(inputs_);

        view_.update();
    }

I am in conflict in how to properly test these presenters without duplicating the test suit of the underlying controller objects but in a more verbose way. It seems that if I don’t want to couple the presenter’s tests to the underlying implementation, I will have to basically merge the event emitters’ and controller’s test suits together which makes the model-based testing much more verbose and complex. In fact it’s often the case that I would have to use the event emitter object itself in the test suit too.

I think this leads to a more general problem on how one should test a component that basically just wires together other subcomponents that are heavily tested themselves and were built in a bottom-up approach.

One solution is to just make sure that the component wires the subcomponents together properly. In that case I would have to mock the controller and event emitters and make sure calls are delegate as appropriate. This would be the Mockist-approach to this problem. I can’t think of any way of doing it without basically duplicating the actual code and making a very fragile test that will break with any future refactoring of the presenter. Maybe this is a good thing in this case since any change to the emitter + controller architecture would probably need a redesign of the test themselves anyway.

Another is to perhaps just write a few sanity check tests with a limited number of user scenarios. This won’t be a complete test as it is with the model-based approach used on the controller state-machine object. I can’t see this helping much in the case of a larger refactoring.

Going further how should one implement the final end-to-end test that actually checks all presenters? It seems one would have to either just check a very limited subset of use cases or do intrusive mock-based tests checking presenter destruction and creation.

Any thoughts?

co.combinatorics – What are efficient pooling designs for RT-PCR tests?

I realize this is long, but hopefully I think it may be worth the reading for people interested in combinatorics and it might prove important to Covid-19 testing.

0. Introduction

The starting point of this question (with multiple subquestions, let me know if this should be made Community Wiki) is this important article by Mutesa et al. where a hypercube ${0,1,2}^n$ is used for pooling (pools are cardinal hyperplanes) takings for Covid-19 testing. This tests $3^n$ takings with $3n$ tests, and gives you exactly who is positive as long as no more than 1 among $3^n$ is. There $3$ is shown to be optimal in term of compression of information, and there are extra strategies for the case when $2$ or more positives show up. However, this limitation in the subsets of positives that can be characterized makes this pooling design only usable at low prevalence. Moreover, even at low prevalence it could be that completely different designs would prove efficient.

I have written a draft sketching some possible research directions, and I would like to share here the main point and ask what seem to me to be the main questions. It might be better to set up a Polymath project, but I don’t feel I have the skills (I am not a combinatorician myself) nor the proper network to make it work.

1. Definitions

Assume we are given $N$ takings from patients to be analyzed for presence of a virus by RT-PCR. An important feature of RT-PCR in this respect is the low sensitivity to dilution: up to some extent and up to the usual limits in sensitivity and specificity, a pool where several takings are mixed will be positive if and only if at least one of the patients has the disease. The simplest pooling method is to gather (fractions of) takings in pools of $k$, and analyze each pool. If the result is negative, all takings are negative, and if the result is positive then takings are reanalyzed individually; one is left with optimizing $k$ using an estimation of the prevalence in the patients. The issue with this method is that successive rounds increase the time to have the result, so ideally one would like to parallelize the tests as much a possible, ideally in only one round.

Recall a hypergraph is a pair $(V,E)$ where $V$ is a set (whose elements are called vertices) and $E$ is a set of non-empty subsets of $V$ (whose elements are called edges).

Definition Given a vertex $xin V$, let $x^*$ be the set of edges containing $x$. Given a subset $Xsubset V$ of vertices, let $X^*={ein E mid exists xin X, xin e}$ be the set of all edges incident to some element of $X$.
Let us define a pooling design as a hypergraph $(V,E)$ satisfying the following property:
$$forall xin V, forall Xsubset V, quad x^* = X^* implies X={x}$$

The interpretation is as follows: each vertex corresponds to a patient (or rather their taking), and each edge to a pool where fractions of takings will be mixed to be analyzed together. The condition that $x^*=X^*$ should only occurs when $X={x}$ ensures that, whenever there is at most one positive taking, its uniqueness is guaranteed by the tests and it can be identified.

Given a pooling design $(V,E)$, there are several numbers of importance to analyze its practical interest in pooled testing. Among them we mention:

  • its order $v=lvert Vrvert$, i.e. the number of patients considered in one application of the design (to analyze $N$ takings, we need $vge N$ up to filling some vertices with empty takings, or to divide $N$ into several groups of at most $v$ patients),
  • its size $e=lvert Ervert$, i.e. the number of RT-PCR involved in one application of the design,
  • its compression rate $r=frac{e}{v}$, i.e. the factor applied to the number of RT-PCR involved compared to the individual testing — the smaller, the better,
  • its detection capacity, i.e. the maximal number of positive taking that can be guaranteed and identified. Formally, letting $mathcal{P}_{le n}(V)$ be the set of subsets of $V$ with at most $n$ elements, we set
    (c = max big{ncolon forall X,Yin mathcal{P}_{le n}(V), X^=Y^implies X=Y big}.)
    The definition of a pooling design ensures $cge 1$, but larger is better.

2. A compression rate is limited by relative detection capability

Proposition.
Let $(V,E)$ be a pooling design of order $v$, size $e$ and detection capacity $c$. Then
$$e ge v Hbig(frac{c}{v}big)-frac12log_2(c)-frac32$$
where $H(x):= -xlog_2(x)-(1-x)log_2(1-x)$ is the binary entropy function.
In particular, the compression rate satisfies
$$r ge Hbig(frac{c}{v}big) – o_{vtoinfty}(1) $$

In particular, since the prevalence at which the pooling design can be used is mostly driven by $c/v$, we have a quantitative estimate of how much a large prevalence prevents a good compression rate.
The proof is straightfoward, and sketched in the draft.

3. Examples

Example 1. The individual testing consist in taking $V$ the set of all $N$ takings, and $E=big{{x} colon xin Vbig}$: each edge is a single vertex. Then we have
begin{align*}
v &= e = N & r &= 1 & c &= N
end{align*}

Example 2. The hypercube design of (Mutesa et al. 2020) with dimension $Dge2$ consist in taking $V={1,2,3}^D$ and $E$ the set of coordinate slices, i.e.
$$E=bigcup_{k=1}^D big{{1,2,3}^{k-1}times {i}times{1,2,3}^{D-k} colon iin{1,2,3}big}.$$
It has
begin{align*}
v &= 3^D & e &= 3D & r &= frac{D}{3^{D-1}} & c &= 1
end{align*}

The detection capacity has the good property to not depend on a prior assumption on the number of positives (if there are more than one, the design detects this event), and while for large $D$ the detection capacity is low compared to the order, subsequent rounds of testing can be used efficiently. The main question we want to raise is whether it is possible to improve on this state-of-the-art pooling design.

Comparing $H(c/v)$ and the actual compression rate for the hypercube design with various values of $D$ show some limited room for improvement (see the draft): the hypercube is of by only a factor less than $2$; these pooling designs are thus not too far from optimal in their prevalence regime.

Example 3. The complete quadrilateral can be described with $V={1,2,3,4,5,6}$ and $E=big{ {1,2,3}, {3,4,5}, {5,6,2}, {1,4,6} big}$. It has
begin{align*}
v &= 6 & e &= 4 & r &= frac23 & c &= 1
end{align*}

For comparison, we note that $H(c/v) simeq 0.65$, very close to the compression rate: this pooling design is close to optimal in its prevalence regime (although the small $v$ makes it more likely that more takings than expected turn out positive; concretely, the probability to have more than $1$ positive taking drops below $5%$ at prevalence $le 6.28%$).

Other examples from incidence geometry are given in the draft.

Example 4.
Let $p$ be a prime number (or a primitive number) and $mathbb{F}_p$ be the Field with $p$ elements. Choose a dimension $Dge 2$ and a parameter $kge D$. We set $V=mathbb{F}_p^D$ (for $p=3$, we thus have the same vertex set than in the hypercube design). Let $(phi_1,dots,phi_k)$ be linear forms such that any $D$ of them are linearly independent. Without loss of generality, we can assume $(phi_1,dots,phi_D)$ is the canonical dual basis (i.e. $phi_i(x_1,dots,x_D) = x_i$). Last, we let $E$ be the set of all levels of all the $phi_i$:
$$ E = big{phi_i^{-1}(y) colon iin{1,dots, k}, yinmathbb{F}_p big}.$$
Let us call the pooling design $(V,E)$ the generalized hybercube design of parameters $(p,D,k)$. It has
begin{align*}
v &= p^D & e &= kp & r &= frac{k}{p^{D-1}}
end{align*}

and the remaining question is how large can be $c$. I think that for $k=D+1$, $cge2$.

For example, with $p=3$, $D=5$ and $k=6$ this gives $H(c/v)ge 0.068$ and $rsimeq 0.074$, a relative improvement from the hypercube design of the same dimension. With $p=2$, $D=5$ and $k=6$, we get a moderate-order pooling design ($v=32$) with $H(c/v)ge 0.33$ and nearly optimal $r= 0.375$ that can be used up to prevalence up to $2,6%$ with less than $5%$ probability to exceed the detection capacity.

4. Questions

General Question Which values of $v,r,c$ are realized by a pooling design?

Question 1. Determine $c$ for the generalized hypercube design, in particular confirm that $cge 2$ for $k>D$ (it might be that $c$ depends on the specific linear form chosen, although I would bet a low stake that it does not). Given $v_0$, what choice of $p,D,k$ such that $vsimeq v_0$ minimizes $frac{r}{H(c/v)}$? Given a prevalence, what is the best value of $r$ that can be achieved with a generalized hypercube for which detection capacity is exceeded with probability less than $5%$?

Question 2. Does there exist pooling designs with $vgg 1$, $c/v simeq 1/6$ and compression rate $simeq2/3$?

Question 3. For small values of $v$, give all pooling designs that are optimal in the sense that no other pooling design with the same order has both better compression rate and better detection capability.

unit testing – Why is automatically generating automated tests frowned upon?

First, the most obvious grouse someone has against this I can think of is the intricacies of an actual method. It’s not enough to merely ensure no errors are thrown. Functions usually contain alternate paths borne of if/else constructs. For this, assume I use a declarative syntax that causes the generated/in-memory tests to follow all those paths. Also assume test framework can detect valid input from the endpoint’s validation rules.

Another common test expectation is during unit tests, that certain methods return specific values. Or, during integration tests, that interacting with elements on a page matches asserted values.

Here are my questions:

  1. why isn’t it sufficient to know method executed successfully, or that the return type conforms to method’s type-hint?
  2. Is it really scalable to edit our tests each time an expected specific in the implementation is replaced?
  3. Why doesn’t integration tests involve the above described procedure replicated for each of the other endpoints part of that feature?

Apparently, the inventor of PHPUnit itself embarked on a similar undertaking some years back, but unfortunately deserted it. What challenges could be behind his resignation of defeat? This SO answer hints thus

it’s also an endless long-term project

I don’t have enough forum rep to ask what could possibly protract its completion. Why is this seen as such a dangerous form of securing a codebase?

EDITED

I disagree with the comments by @jonrsharpe and @user253751, and this is my defense. I am only interested in testing action methods tied to all defined routes. I have a test case for /is-prime-number/5. The handler calls an internal/private method that decides whether or not a number is a prime number. Why do I have to redefine another method within the test case that determines whether or not its main counterpart is correct? If I got the SUT calculation confirming a number is a prime number wrong, what’s the guarantee that the test case calculation will be correct?
The only way this can be avoided is if it’s another developer writing the tests for someone else’s code, which in my experience, is rarely the case.

The code already knows what it wants to do. Tests ought to confirm it successfully executes its desires both during runtime and at compile time. Such specific domain specific semantics as whether the number is a prime number in real life should exist within the scope of the business logic itself.

EDIT 2

Users on this board are downvoting me because they incorrectly think I’m referring to functional testing. I want tests that insulate updates from creating regression bugs. The contents of the response have been manually tested via the browser/console/postman/network tab etc. and are certainly satisfactory to me before I’m ready to commit/run tests

Generic double linked list implementation with tests in C

Can you please give your feedback to my double linked list implementation below? It comes with some very basic functions as well as separate code to test the implementation. The tests are only ‘success’ tests. I used valgrind to catch and fix any kind of memory-related issues that I came across and valgrind did not report any issues with the implementation. If it matters, I am running this under macOS; I do not know if valgrind on Linux systems would produce errors or warnings with the code.

Apologies in advance if the naming convention is a bit verbose or not like any C naming convention; I am still very new to C and my knowledge on naming conventions is limited.

DoubleLinkedList.h

#ifndef DOUBLELINKEDLIST_H
#define DOUBLELINKEDLIST_H

// error codes
static const int ENOMEMORY = -1; // Could not allocate memory
static const int ENULLIST = -2; // The list is null; memory could not have been allocated to it
static const int EEMPTYNODE = -3; // The node is null

typedef struct DoubleLinkedList_Node
{
    void* data;
    struct DoubleLinkedList_Node* next;
    struct DoubleLinkedList_Node* previous;
} DoubleLinkedList_Node;

typedef struct DoubleLinkedList
{
    DoubleLinkedList_Node* head;
    DoubleLinkedList_Node* tail;
} DoubleLinkedList;

unsigned int DoubleLinkedList_Count(DoubleLinkedList *list);

DoubleLinkedList *DoubleLinkedList_Initialize(void);

int DoubleLinkedList_Destroy(DoubleLinkedList *list);

int DoubleLinkedList_Add(DoubleLinkedList *list, void *data_to_add);

DoubleLinkedList_Node *DoubleLinkedList_GetNode(DoubleLinkedList *list, void *data_to_search);

int DoubleLinkedList_GetNodes(DoubleLinkedList *list, void *data_to_search, DoubleLinkedList *return_list);

int DoubleLinkedList_Foreach(DoubleLinkedList *list, void (DoubleLinkedList_Node *callback));

int DoubleLinkedList_ForeachReverse(DoubleLinkedList *list, void (DoubleLinkedList_Node *callback));

int DoubleLinkedList_Clear(DoubleLinkedList *list);

int DoubleLinkedList_Remove(DoubleLinkedList *list, DoubleLinkedList_Node *node_to_remove);

int DoubleLinkedList_Find(DoubleLinkedList *list, void *data_to_find, DoubleLinkedList *return_list);

#endif //DOUBLELINKEDLIST_H

DoubleLinkedList.c

#include <stdlib.h>
#include "DoubleLinkedList.h"


unsigned int DoubleLinkedList_Count(DoubleLinkedList *list)
{
    if (!list) return ENULLIST;
    unsigned int count = 0;
    DoubleLinkedList_Node *current_node = list->head->next;
    while (current_node != list->tail)
    {
        count++;
        current_node = current_node->next;
    }
    return count;
}

DoubleLinkedList* DoubleLinkedList_Initialize(void)
{
    DoubleLinkedList* return_value = calloc(1, sizeof(DoubleLinkedList));
    if (return_value)
    {
        return_value->head = calloc(1, sizeof(DoubleLinkedList_Node));
        return_value->tail = calloc(1, sizeof(DoubleLinkedList_Node));

        if (return_value->head && return_value->tail)
        {
            return_value->head->next = return_value->tail;// connect the head to the tail
            return_value->tail->previous = return_value->head;// connect the tail to the head
            return_value->tail->data = NULL;
            return_value->tail->next = NULL;
            return_value->head->previous = NULL;

            return return_value;
        }
    }
    return NULL;
}
int DoubleLinkedList_Destroy(DoubleLinkedList* list)
{
    if (!list) return ENULLIST;
    while (list->head)
    {
        DoubleLinkedList_Node* node_to_free = list->head;
        list->head = list->head->next;
        free(node_to_free);
        node_to_free = NULL;
    }
    free(list->head);
    list->head = NULL;

    free(list);
    list = NULL;
    return 0;
}

int DoubleLinkedList_Add(DoubleLinkedList *list, void *data_to_add)
{
    DoubleLinkedList_Node* new_node = calloc(1, sizeof(DoubleLinkedList_Node));
    if (!new_node) return ENOMEMORY;

    list->tail->data = data_to_add;
    new_node->data = NULL;
    new_node->next = NULL;
    new_node->previous = list->tail;

    // make the new node the tail
    list->tail->next = new_node;

    //current tail is now the second to the last node
    list->tail = new_node;

    return 0;

}
DoubleLinkedList_Node* DoubleLinkedList_GetNode(DoubleLinkedList* list, void* data_to_search)
{
    // from head, iterate through the list to find the data
    DoubleLinkedList_Node* return_value = list->head;
    while (return_value)
    {
        if (return_value->data == data_to_search && return_value != list->head && return_value != list->tail)
        {
            return return_value;
        }
        // move to the next node
        return_value = return_value->next;
    }
    // at this point, we went through the entire list but couldn't find the value; return NULL
    return NULL;
}

int DoubleLinkedList_Foreach(DoubleLinkedList* list, void (*callback)(DoubleLinkedList_Node *))
{
    if (!list) return ENULLIST;

    DoubleLinkedList_Node* currentNode = list->head;
    while (currentNode != list->tail)
    {
        // don't include the head and the tail in the evaluation
        if (currentNode != list->head && currentNode != list->tail)
        {
            callback(currentNode);
        }
        currentNode = currentNode->next;
    }
    return 0;
}

int DoubleLinkedList_ForeachReverse(DoubleLinkedList *list, void(*callback) (DoubleLinkedList_Node *))
{
    if (!list) return ENULLIST;

    DoubleLinkedList_Node *currentNode = list->tail;
    while (currentNode != list->head)
    {
        // don't include the head and the tail in the evaluation
        if (currentNode != list->head && currentNode != list->tail)
        {
            callback(currentNode);
        }
        currentNode = currentNode->previous;
    }
    return 0;
}

int DoubleLinkedList_Clear(DoubleLinkedList *list)
{
    if (!list) return ENULLIST;

    DoubleLinkedList_Node *currentNode = list->head->next;
    while (currentNode != list->tail)
    {
        currentNode = currentNode->next;
        free(currentNode->previous);
        currentNode->previous = NULL;
    }

    // reconnect the head to the tail and the tail to the head

    list->head->next = list->tail; // connect the head to the tail
    list->tail->previous = list->head; // connect the tail to the head
    list->tail->data = NULL;
    list->tail->next = NULL;
    list->head->previous = NULL;

    return 0;
}

int DoubleLinkedList_Remove(DoubleLinkedList *list, DoubleLinkedList_Node *node_to_remove)
{
    if (!list) return ENULLIST;

    if (!node_to_remove) return EEMPTYNODE;

    DoubleLinkedList_Node *current_node = list->head->next;
    while (current_node != list->tail)
    {
        if (current_node->data == node_to_remove->data)
        {
            // connect the currentNode's previous to its next and vice-versa
            current_node->previous->next = current_node->next;
            current_node->next->previous = current_node->previous;
            free(current_node);
            current_node = NULL;

            return 0;
        }
        current_node = current_node->next;
    }
    // at this point, the record has not found.
    return 0;
}

int DoubleLinkedList_GetNodes(DoubleLinkedList *list, void *data_to_search, DoubleLinkedList *return_list)
{
    if (!list) return ENULLIST;

    if (!return_list) return ENULLIST;

    DoubleLinkedList_Node *current_node = list->head->next;
    while (current_node != list->tail)
    {
        if (current_node->data == data_to_search)
        {
            int addResult = DoubleLinkedList_Add(return_list, current_node->data);
            if (addResult != 0) return addResult;
        }
        current_node = current_node->next;
    }
    return 0;
}

int DoubleLinkedList_Find(DoubleLinkedList *list, void *data_to_find, DoubleLinkedList *return_list)
{
    if (!list) return ENULLIST;

    if (!return_list) return ENULLIST;

    DoubleLinkedList_Node *current_node = list->head->next;
    while (current_node != list->tail)
    {
        if (current_node->data == data_to_find)
        {
            int addResult = DoubleLinkedList_Add(return_list, current_node->data);
            if (addResult != 0) return addResult;
        }
        current_node = current_node->next;
    }
    return 0;
}

For testing:

DoubleLinkedListTests.h

#ifndef DOUBLELINKEDLISTTESTS_H
#define DOUBLELINKEDLISTTESTS_H

// runner

void DoubleLinkedList_Tests_RunAllTests(void);

// success tests

void DoubleLinkedList_SuccessTests_Count(void);

void DoubleLinkedList_SuccessTests_InitializeAndDestroy(void);

void DoubleLinkedList_SuccessTests_Add(void);

void DoubleLinkedList_SuccessTests_GetNode(void);

void DoubleLinkedList_SuccessTests_GetNodes(void);

void DoubleLinkedList_SuccessTests_Foreach(void);

void DoubleLinkedList_SuccessTests_ForeachReverse(void);

void DoubleLinkedList_SuccessTests_Clear(void);

void DoubleLinkedList_SuccessTests_Remove(void);

void DoubleLinkedList_SuccessTests_Find(void);

#endif //DOUBLELINKEDLISTTESTS_H

DoubleLinkedListTests.c

#include "DoubleLinkedList.h"
#include <assert.h>
#include <stddef.h>
static char *testData = "test data";

void DoubleLinkedListTests_Foreach_Callback(DoubleLinkedList_Node *node)
{
    assert(node->data == testData);
}

void DoubleLinkedList_SuccessTests_Count(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();

    for (int i = 0; i < 10; i++)
    {
        DoubleLinkedList_Add(list, &i);
    }
    assert(DoubleLinkedList_Count(list) == 10);
    DoubleLinkedList_Destroy(list);
}

void DoubleLinkedList_SuccessTests_InitializeAndDestroy(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();

    assert(list);

    assert(list->head->next == list->tail);

    assert(list->tail->previous == list->head);

    DoubleLinkedList_Destroy(list);

}

void DoubleLinkedList_SuccessTests_Add(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();

    int isSuccess = DoubleLinkedList_Add(list, testData);

    assert(isSuccess == 0);

    assert(list->head->next->data == testData);

    DoubleLinkedList_Destroy(list);

}

void DoubleLinkedList_SuccessTests_GetNode(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList_Add(list, testData);

    DoubleLinkedList_Node *test = DoubleLinkedList_GetNode(list, testData);
    assert(test != NULL);

    assert(test->data == testData);

    DoubleLinkedList_Destroy(list);
}

void DoubleLinkedList_SuccessTests_GetNodes(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList *container = DoubleLinkedList_Initialize();

    DoubleLinkedList_Add(list, testData);
    DoubleLinkedList_Add(list, testData);
    DoubleLinkedList_Add(list, "invalid data");
    DoubleLinkedList_Add(list, "even more invalid data");

    int isSuccess = DoubleLinkedList_GetNodes(list, testData, container);

    assert(isSuccess == 0);

    assert(container->head->next->data == testData);
    assert(container->head->next->next->data == testData);

    DoubleLinkedList_Destroy(list);
    DoubleLinkedList_Destroy(container);

}

void DoubleLinkedList_SuccessTests_Foreach(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList_Add(list, testData);

    int isSuccess = DoubleLinkedList_Foreach(list, DoubleLinkedListTests_Foreach_Callback);
    assert(isSuccess == 0);
    DoubleLinkedList_Destroy(list);
}

void DoubleLinkedList_SuccessTests_ForeachReverse(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList_Add(list, testData);

    int isSuccess = DoubleLinkedList_ForeachReverse(list, DoubleLinkedListTests_Foreach_Callback);
    assert(isSuccess == 0);

    DoubleLinkedList_Destroy(list);
}

void DoubleLinkedList_SuccessTests_Clear(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList_Add(list, testData);

    DoubleLinkedList_Clear(list);

    assert(list->head->next == list->tail);
    assert(list->tail->previous == list->head);

    DoubleLinkedList_Destroy(list);
}

void DoubleLinkedList_SuccessTests_Remove(void)
{
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList_Add(list, testData);

    DoubleLinkedList_Remove(list, list->head->next);
    assert(list->head->next == list->tail);
    assert(list->tail->previous == list->head);


    DoubleLinkedList_Destroy(list);
}

void DoubleLinkedList_SuccessTests_Find(void)
{
    char test2 = '2';
    char *test3 = "testData";
    int test4 = 0;
    double test5 = -123.456;
    DoubleLinkedList *list = DoubleLinkedList_Initialize();
    DoubleLinkedList *container = DoubleLinkedList_Initialize();
    DoubleLinkedList_Add(list, testData);
    DoubleLinkedList_Add(list, &test2);
    DoubleLinkedList_Add(list, test3);
    DoubleLinkedList_Add(list, &test4);
    DoubleLinkedList_Add(list, &test5);

    DoubleLinkedList_Find(list, testData, container);

    assert(container->head->next->data == list->head->next->data);

    DoubleLinkedList_Find(list, &test2, container);
    assert(container->head->next->next->data == list->head->next->next->data);

    DoubleLinkedList_Find(list, test3, container);
    assert(container->head->next->next->next->data == list->head->next->next->next->data);

    DoubleLinkedList_Find(list, &test4, container);
    assert(container->head->next->next->next->next->data == list->head->next->next->next->next->data);

    DoubleLinkedList_Find(list, &test5, container);
    assert(container->tail->previous->data == list->tail->previous->data);

    DoubleLinkedList_Destroy(list);
    DoubleLinkedList_Destroy(container);

}

void DoubleLinkedList_Tests_RunAllTests()
{
    DoubleLinkedList_SuccessTests_Count();
    DoubleLinkedList_SuccessTests_InitializeAndDestroy();
    DoubleLinkedList_SuccessTests_Add();
    DoubleLinkedList_SuccessTests_GetNode();
    DoubleLinkedList_SuccessTests_GetNodes();
    DoubleLinkedList_SuccessTests_Foreach();
    DoubleLinkedList_SuccessTests_ForeachReverse();
    DoubleLinkedList_SuccessTests_Clear();
    DoubleLinkedList_SuccessTests_Remove();
    DoubleLinkedList_SuccessTests_Find();
}

main.c

#include "DoubleLinkedListTests.h"

int main(void)
{
    DoubleLinkedList_Tests_RunAllTests();
    return 0;
}

bitcoincore development – Can I speed up the running of functional tests?

The Bitcoin Core test README states:

By default, up to 4 tests will be run in parallel by test_runner. To
specify how many jobs to run, append --jobs=n

The individual tests and the test_runner harness have many
command-line options. Run test/functional/test_runner.py -h to see
them all.

Pieter Wuille also explained how he gets the total time down below 4 minutes on GitHub:

You can run them in parallel; if you have sufficient RAM pretty extremely even. test_runner.py -j60 works fine on my 4-core 32 GiB RAM system, taking 3m46s. A lot of the time consists of processes waiting for each other, so it’s not actually CPU bound.

How to write unit tests for REST API that are atomic?

As for automatic testing, what you are missing is the capability to either

a. Regenerate the datastore in a very specific state during the testing phase, so you can provide each test with the required data and the required state. If we were speaking about databases, I would suggest implementing in-memory databases alongside with some sort of change logger.

b. Mock dependencies.

What you are looking for is named determinism. For you to be able to execute tests in any order, anywhere, anytime and always obtain the same results you need the capacity to force the data that makes the business logic to behave in one or another way.

Another example would be testing authentication and other endpoints
that require an authenticated user. So to perform those test that
require authentication, such as a request to user/update I would need
to have an API token ready for it, which means I must have run the
auth/login request to get the API token and saved it somewhere to be
used by subsequent tests. Therefore, it obviously introduces the same
problems as the above-mentioned example, if not more.

Hell no.

You can create a default token for testing. One which never expires and bound to a non-production user. For example, I have tests that help me out to create and validate JWT. Those tests are never executed as part of the testing plan. They are tools somehow.

I turn these tokens into constants in my testing code. If I have different profiles, I generate different tokens, each of which is bound to a non-real user with the expected role for the test.

Hence the current organization necessitates a rigid sequence of tests,
which goes against the principle of atomicity, which in turn renders
features like parallel testing impossible.

Not necessarily. Say you achieved determinism in your automatic testing. You still can deploy a staging environment for E2E or Loading testing.

Basically, you need a tool that allows you to record all the calls (and order) to the API, so you can reproduce such a scenario over and over. The tool should allow you to handle responses, store info, load info, do checks, etc.

For you to be able to reproduce the scenario (use-case) countless times, you need to reset the data. For example, if you want to reuse an email for the signup phase, you have to provide the API with the opposite operation. For example, DELETE /account/{id} and make this one the lastest call in all your scenarios. Or perhaps an operation to change it for a random one PUT /account/{id} ... email:asda3asd-sad3--asd3asdas-@domain.com. Or just the opposite, create new users with random emails.

8 – hook_menu_local_tasks_alter is not invoked during functional tests

I have an implementation of hook_menu_local_tasks_alter in my module, which works fine on my dev environment, but when I try to run my module using BrowserTestBase the hook is not invoked when visiting a node. Why?

Here is a mock instance of the hook that I’ve set up just to see if the hook is being called:

function implementations_menu_local_tasks_alter(array &$data, $route_name, DrupalCoreCacheRefinableCacheableDependencyInterface &$cacheability) {

  throw new Exception('whoops');
}

I’ve ruled out the module not being installed properly, as the other hooks are invoked without issue. Looking at the html produced by BrowserTestBase I can see that some header links are present in the header, but I don’t see any view/edit tabs present in the body of the html. I tried changing BrowserTestBase::$defaultTheme to both 'classy' and 'bartik' with the same result.

ID #21 (Previous | Next)
Called from GuzzleHttpPromiseFulfilledPromise::GuzzleHttpPromise{closure}() line 39
GET request to: http://localhost/node/2
Skip to main content
Status message
Implementation Some Title has been created.
Some Title
Member for
3 seconds
Submitted by p6fNZyQj on Sun, 10/18/2020 - 15:26
Parent Idea
my_idea
Headers:
array (
  'Date' => 'Sun, 18 Oct 2020 04:26:21 GMT',
  'Server' => 'Apache/2.4.38 (Debian)',
  'X-Powered-By' => 'PHP/7.4.11',
  'X-Drupal-Assertion-0' => 'a%3A3%3A%7Bi%3A0%3BO%3A25%3A%22Drupal%5CCore%5CRender%5CMarkup%22%3A1%3A%7Bs%3A9%3A%22%00%2A%00string%22%3Bs%3A217%3A%22The%20signature%20of%20the%20%22Drupal%5CComponent%5CEventDispatcher%5CContainerAwareEventDispatcher%3A%3Adispatch%28%29%22%20method%20should%20be%20updated%20to%20%22dispatch%28%24event%2C%20string%20%24eventName%20%3D%20null%29%22%2C%20not%20doing%20so%20is%20deprecated%20since%20Symfony%204.3.%22%3B%7Di%3A1%3Bs%3A24%3A%22User%20deprecated%20function%22%3Bi%3A2%3Ba%3A3%3A%7Bs%3A8%3A%22function%22%3Bs%3A72%3A%22Symfony%5CComponent%5CEventDispatcher%5CLegacyEventDispatcherProxy%3A%3Adecorate%28%29%22%3Bs%3A4%3A%22file%22%3Bs%3A74%3A%22%2Fopt%2Fdrupal%2Fvendor%2Fsymfony%2Fevent-dispatcher%2FLegacyEventDispatcherProxy.php%22%3Bs%3A4%3A%22line%22%3Bi%3A41%3B%7D%7D',
  'X-Drupal-Assertion-1' => 'a%3A3%3A%7Bi%3A0%3BO%3A25%3A%22Drupal%5CCore%5CRender%5CMarkup%22%3A1%3A%7Bs%3A9%3A%22%00%2A%00string%22%3Bs%3A155%3A%22The%20%22Symfony%5CComponent%5CHttpFoundation%5CFile%5CMimeType%5CMimeTypeGuesser%22%20class%20is%20deprecated%20since%20Symfony%204.3%2C%20use%20%22Symfony%5CComponent%5CMime%5CMimeTypes%22%20instead.%22%3B%7Di%3A1%3Bs%3A24%3A%22User%20deprecated%20function%22%3Bi%3A2%3Ba%3A3%3A%7Bs%3A8%3A%22function%22%3Bs%3A9%3A%22include%28%29%22%3Bs%3A4%3A%22file%22%3Bs%3A76%3A%22%2Fopt%2Fdrupal%2Fvendor%2Fsymfony%2Fhttp-foundation%2FFile%2FMimeType%2FMimeTypeGuesser.php%22%3Bs%3A4%3A%22line%22%3Bi%3A18%3B%7D%7D',
  'X-Drupal-Assertion-2' => 'a%3A3%3A%7Bi%3A0%3BO%3A25%3A%22Drupal%5CCore%5CRender%5CMarkup%22%3A1%3A%7Bs%3A9%3A%22%00%2A%00string%22%3Bs%3A181%3A%22The%20%22Symfony%5CComponent%5CHttpFoundation%5CFile%5CMimeType%5CFileBinaryMimeTypeGuesser%22%20class%20is%20deprecated%20since%20Symfony%204.3%2C%20use%20%22Symfony%5CComponent%5CMime%5CFileBinaryMimeTypeGuesser%22%20instead.%22%3B%7Di%3A1%3Bs%3A24%3A%22User%20deprecated%20function%22%3Bi%3A2%3Ba%3A3%3A%7Bs%3A8%3A%22function%22%3Bs%3A9%3A%22include%28%29%22%3Bs%3A4%3A%22file%22%3Bs%3A86%3A%22%2Fopt%2Fdrupal%2Fvendor%2Fsymfony%2Fhttp-foundation%2FFile%2FMimeType%2FFileBinaryMimeTypeGuesser.php%22%3Bs%3A4%3A%22line%22%3Bi%3A18%3B%7D%7D',
  'X-Drupal-Assertion-3' => 'a%3A3%3A%7Bi%3A0%3BO%3A25%3A%22Drupal%5CCore%5CRender%5CMarkup%22%3A1%3A%7Bs%3A9%3A%22%00%2A%00string%22%3Bs%3A177%3A%22The%20%22Symfony%5CComponent%5CHttpFoundation%5CFile%5CMimeType%5CFileinfoMimeTypeGuesser%22%20class%20is%20deprecated%20since%20Symfony%204.3%2C%20use%20%22Symfony%5CComponent%5CMime%5CFileinfoMimeTypeGuesser%22%20instead.%22%3B%7Di%3A1%3Bs%3A24%3A%22User%20deprecated%20function%22%3Bi%3A2%3Ba%3A3%3A%7Bs%3A8%3A%22function%22%3Bs%3A9%3A%22include%28%29%22%3Bs%3A4%3A%22file%22%3Bs%3A84%3A%22%2Fopt%2Fdrupal%2Fvendor%2Fsymfony%2Fhttp-foundation%2FFile%2FMimeType%2FFileinfoMimeTypeGuesser.php%22%3Bs%3A4%3A%22line%22%3Bi%3A18%3B%7D%7D',
  'Cache-Control' => 'must-revalidate, no-cache, private',
  'X-Drupal-Dynamic-Cache' => 'MISS',
  'Link' => '<http://localhost/node/2>; rel="canonical";<http://localhost/node/2>; rel="shortlink";<http://localhost/node/2/delete>; rel="delete-form";<http://localhost/admin/content/node/delete?node=2>; rel="delete-multiple-form";<http://localhost/node/2/edit>; rel="edit-form";<http://localhost/node/2/revisions>; rel="version-history";<http://localhost/node/2>; rel="revision"',
  'X-UA-Compatible' => 'IE=edge',
  'Content-language' => 'en',
  'X-Content-Type-Options' => 'nosniff',
  'X-Frame-Options' => 'SAMEORIGIN',
  'X-Drupal-Cache-Tags' => 'http_response node:1 node:2 node_view rendered user:2 user_view',
  'X-Drupal-Cache-Contexts' => 'languages:language_interface theme timezone url.query_args:_wrapper_format url.site user.permissions user.roles:anonymous',
  'Expires' => 'Sun, 19 Nov 1978 05:00:00 GMT',
  'X-Generator' => 'Drupal 9 (https://www.drupal.org)',
  'Vary' => 'Accept-Encoding',
  'Transfer-Encoding' => 'chunked',
  'Content-Type' => 'text/html; charset=UTF-8',
)

How can I invoke hook_menu_local_tasks_alter from a functional test?