data structures – Resources about implementing a database table

I’d like to understand more how data can be stored on disk so it can be read quickly. I don’t want to say database record because I’m not asking how to build a database. My question is closer to how would I build a table.

I’m not asking about indexing. I’m thinking about you have an application that already has some a pointer to a record.

I’ve been trying to find some references e.g. papers, books, but everything so far focuses on database design as in writing DDL or structures for indexing like B+ trees.

Are there any references that talk about how tables are actually implemented and what are the tradeoffs?

spring – Are these Unit tests OK or am I implementing an antipattern or not following best practices

I am testing a Spring boot service using JUnit 5 and Mockito. I haven’t spent much time on unit testing in the past so I’m not sure if I’m accidentally implementing an anti-pattern or not following best practices.

I read multiple times that it’s good practice to have one assertion per test, I have one assertion but I also have a few verify assertions that check if the method I’m testing called the correct methods with the correct input based on the conditional branch I’m testing as well as the input to the method I’m testing, is this OK/acceptable or should I split it up into one assertion/verify assertion per test? It feels like that would require a lot of code duplication… Below are four tests I wrote and the method I’m testing. For clarity I’ll also add the code that sets up and handles my test application context.

Any ideas/advice would be much appreciated.

The method I am testing:

@Transactional
    override fun startSession(msisdn: String, origConnID: String, configSetName: String): StartSessionResponseDTO {
        val txId = generateTxID()
        val ss = sessionStatus(msisdn)

        eventService.logEvent(EventType.StartSessionRequest, txId, msisdn, hashMapOf("extTxId" to origConnID))

        if (ss == null) {
            throw ApplicationException(type = ApplicationException.Type.client_error, message = "Concurrent Access Detected: ${msisdn}")
        } else {
            if (ss.sessionId == null || ss.ip == null) {
                //create new Nuance session
                nuanceService.startSession(NuanceStartSessionReqDto(txId, msisdn, configSetName = configSetName)).let {
                    updateSession(msisdn = msisdn, sessionId = it.sessionId, ip = it.ip, origConnID = origConnID)
                }
            } else {
                //check if the sessionId is still valid
                if (!nuanceService.isSessionValid(txId = txId, msisdn = msisdn, sessionId = ss.sessionId, host = ss.ip, configSetName = configSetName)) {
                    nuanceService.startSession(NuanceStartSessionReqDto(txId, msisdn, configSetName = configSetName)).let {
                        updateSession(msisdn = msisdn, sessionId = it.sessionId, ip = it.ip, origConnID = origConnID)
                    }
                }
            }
        }

        eventService.logEvent(type = EventType.StartSessionResponse, txId = txId, msisdn = msisdn)
        return StartSessionResponseDTO(msisdn)
    }

My test class:

@ExtendWith(SpringExtension::class)
@ContextConfiguration()
class  VBServiceTests() {

    @TestConfiguration
    class testConfig {
        @Bean
        fun jdbcTemplate(): NamedParameterJdbcTemplate {
            return mock<NamedParameterJdbcTemplate>()
        }

        @Bean
        fun nuanceService(): NuanceService {
            return mock<NuanceService>()
        }

        @Bean
        fun appConfigProps(): AppConfigProps {
            return mock<AppConfigProps>()
        }

        @Bean
        fun eventService(): EventServiceImp {
            return mock<EventServiceImp>()
        }

        @Bean
        fun audioTrimService(): AudioTrimService {
            return mock<AudioTrimService>()
        }

        @Bean
        fun vbNuanceStagingDeletionService(): VbNuanceStagingDeletionsService {
            return mock<VbNuanceStagingDeletionsService>()
        }
    }

    @MockBean
    lateinit var nuanceService: NuanceService

    @MockBean
    lateinit var eventService: EventServiceImp

    @SpyBean
    lateinit var vbServiceSpy: VbServiceImp

    val msisdn = "0821234567"
    val origConnID = "o123"
    val sessionId = "0821234567"
    val ip = "127.0.0.1"
    val txId = "1234-5678"
    val configSetName = "LoIvrPhraIvrHdr"

    @Test
    fun `startSession | When method is called with valid input a StartSessionDTO is returned, if no Nuance session exists a new Nuance session will be created and persisted`() {
        doNothing().whenever(vbServiceSpy).updateSession(msisdn, origConnID, sessionId, ip)
        doReturn(txId).whenever(vbServiceSpy).generateTxID()
        doReturn(SessionStatus(msisdn, origConnID, null, null)).whenever(vbServiceSpy).sessionStatus(msisdn)

        given(nuanceService.startSession(NuanceStartSessionReqDto(txId, msisdn, configSetName = configSetName))).willReturn(NuanceStartSessionRespDto(txId, sessionId, ip))

        assertThat(vbServiceSpy.startSession(msisdn, origConnID, configSetName), Is(StartSessionResponseDTO(msisdn)))
        verify(vbServiceSpy).generateTxID()
        verify(vbServiceSpy).sessionStatus(msisdn)
        verify(eventService).logEvent(StartSessionRequest, txId, msisdn, hashMapOf("extTxId" to origConnID))
        verify(nuanceService).startSession(NuanceStartSessionReqDto(txId, msisdn, configSetName = configSetName))
        verify(vbServiceSpy).updateSession(msisdn = msisdn, sessionId = sessionId, ip = ip, origConnID = origConnID)
        verify(eventService).logEvent(type = EventType.StartSessionResponse, txId = txId, msisdn = msisdn)
    }

    @Test
    fun `startSession | When method is called with valid input a StartSessionDTO is returned, if existing valid Nuance session exists the existing session will be used`() {
        doNothing().whenever(vbServiceSpy).updateSession(msisdn, origConnID, sessionId, ip)
        doReturn(txId).whenever(vbServiceSpy).generateTxID()
        doReturn(SessionStatus(msisdn, origConnID, sessionId, ip)).whenever(vbServiceSpy).sessionStatus(msisdn)

        given(nuanceService.isSessionValid(txId, msisdn, sessionId, ip, configSetName)).willReturn(true)

        assertThat(vbServiceSpy.startSession(msisdn, origConnID, configSetName), Is(StartSessionResponseDTO(msisdn)))
        verify(vbServiceSpy).generateTxID()
        verify(vbServiceSpy).sessionStatus(msisdn)
        verify(eventService).logEvent(StartSessionRequest, txId, msisdn, hashMapOf("extTxId" to origConnID))
        verify(eventService).logEvent(type = EventType.StartSessionResponse, txId = txId, msisdn = msisdn)
    }

    @Test
    fun `startSession | When method is called with valid input a StartSessionDTO is returned, if existing invalid Nuance session exists a new session will be created and the existing session record will be updated`() {
        doNothing().whenever(vbServiceSpy).updateSession(msisdn, origConnID, sessionId, ip)
        doReturn(txId).whenever(vbServiceSpy).generateTxID()
        doReturn(SessionStatus(msisdn, origConnID, sessionId, ip)).whenever(vbServiceSpy).sessionStatus(msisdn)

        given(nuanceService.isSessionValid(txId, msisdn, sessionId, ip, configSetName)).willReturn(false)
        given(nuanceService.startSession(NuanceStartSessionReqDto(txId, msisdn, configSetName = configSetName))).willReturn(NuanceStartSessionRespDto(txId, sessionId, ip))

        assertThat(vbServiceSpy.startSession(msisdn, origConnID, configSetName), Is(StartSessionResponseDTO(msisdn)))
        verify(vbServiceSpy).generateTxID()
        verify(vbServiceSpy).sessionStatus(msisdn)
        verify(eventService).logEvent(StartSessionRequest, txId, msisdn, hashMapOf("extTxId" to origConnID))
        verify(nuanceService).startSession(NuanceStartSessionReqDto(txId, msisdn, configSetName = configSetName))
        verify(vbServiceSpy).updateSession(msisdn = msisdn, sessionId = sessionId, ip = ip, origConnID = origConnID)
        verify(eventService).logEvent(type = EventType.StartSessionResponse, txId = txId, msisdn = msisdn)
    }

    @Test()
    fun `startSession - When method is called with valid input when sessionStatus returns null concurrent access is detected and an ApplicationException gets thrown`() {
        doReturn(txId).whenever(vbServiceSpy).generateTxID()
        doReturn(null).whenever(vbServiceSpy).sessionStatus(msisdn)

        val exception = Assertions.assertThrows(ApplicationException::class.java) {
            vbServiceSpy.startSession(msisdn, origConnID, configSetName)
        }

        assertThat(exception.message, Is("Concurrent Access Detected: ${msisdn}"))


        verify(vbServiceSpy).generateTxID()
        verify(vbServiceSpy).sessionStatus(msisdn)
    }


}

Haskell: Implementing the data type LExpR as an instance of show

I am trying to implement the data type LExpr as an instance of show.

data LExpr = Var String        
                | App LExpr LExpr   
                | Lam String LExpr 

This is my code:

 instance Show LExpr where
      showsPrec d (Var str)   = showString' str
      showsPrec d (App e1 e2) = showParen (d>=6) $ showsPrec 6 e1  . showString " " . showsPrec 5  e2
      showsPrec d (Lam str e) = showParen (d>=5) $ showString ("\"++str++" -> ") . showsPrec 5 e

I am looking for some advice on how to further improve my code.

How to use GCP monitoring CPU Usage alert to spin a new VM instance (implementing custom autoscaling)

I have 2 GCP VMs connected to a load balancer (it is an ummanaged instance group).

  1. I would like to spin a new VM whenever I receieve a CPU Usage alert (usage higher then 70%). This is a customized scenario and I cannot use GCP built in autoscaling.

Are the following 2 scenarios possible ? looking for some direction here.

  1. My custom code will spin a new VM and deploy custom code on instance.
  2. Can I attach the new VM to the LB after step 1 above.

Thanks

linux – implementing a failover service for containers

Two VMs are hosted on the same machine, let’s name them VM1 and VM2. Both VMs are hosting a runC container running a server application, a client is running directly on the host machine and is connected to service running in the container in VM1.

I want to build a failover service where if I close the container on VM1, the client should get connected to the container running on VM2.

I’ve implemented and tested a floating IP using keepalived and VRRP as suggested in this link, but this works only when the whole VM fails, I have to implement fault tolerance service for a container if the container on VM1 fails, the client should get connected to a replica running on VM2 even if the VM1 is up i.e the container fails but the VM hosting that container is up.

The restriction here is I do not want to use any Load Balancer/HAproxy service which can become a single point of failure.

c# – Ideas for implementing a hardware interface

I am looking for alternative ideas of how to implement the following situation (.NET Core, C#)

Having multiple sensors that communicate over a RS485 interface with the host. Each sensor is implemented using an generic ISensor interface (All sensor types have a certain function which must be implemented over the ISensor interface) There are all kinds of sensors that communicate over varying channels with the host, TCP, Serial, USB, custom hardware, etc. This class of sensors is somehow special in that they share the same communication channel, which can be opened only once.

This special group of sensors use the RS485 serial port to communicate, each sensor has an id which is included in every data packet sent or received. So when I send a data packet with an address of “1” all sensors on this port will receive it but only the one with address “1” will react and possibly respond.

Only one logical connection over the serial port can be opened to talk to all sensors and receive data from all. Therefore I must have a central point of distribution from where all inputs are dispatched and to which all output is sent. From the point of the using app, it only talks to the ISensor interface and doesn’t care how the sensor communicates, if anyhow. (many other sensors already implemented are simple in that they own their connection)

Every interaction with a sensor contains a data structure with an corresponding answer data structure (or in case of failure an error data structure identical for all errors)

I am wondering in what OO pattern to implement this scenario, a few come to my mind but none satisfies me too much.

The app creates, lets say 3 instances of the same sensor (configuration tells that 3 are connected), configures them by using something like ISensor.Setup() and then ISensor.Start() after that, each of the sensors starts to send in data periodically.

Now I have already an protocol class which does the parsing of all communication data, independent of sensor ID or underlying communication channel.

My idea was to have an queue where all incoming data is arranged and the sensors subscribe to it and pick out the data for them ( the subscription may already filter them ). Outgoing data should be also queued from all sensors. Not sure now where to plug in the protocol parser and the communication class.

Its kinda related to Controlling a sensor network which didnt get any useful answers though.
Bandwidth is not so much a problem as there are packets only every 100ms incoming, from up to 8 sensors per serial port.

object oriented – Python code for simple Vending Machine simulation implementing OOP principles

I am recently learning OOP concepts and thought that implementing a simple workflow simulating a vending machine would help me put what I’ve learnt into practice. I am open to criticisms and feedback, and am mainly trying to get some comments regarding:

  • Best practices for OOP
  • File structure
  • Data encapsulation
  • Unit testing practices
  • General architecture advice for OOP

The project simulates a vending machine dispensing drinks for customers for X hours (user input), given that arrival times follow a Poisson(K) distribution and an initial stock list (user input). The project also produces a sales report for the given simulation.

The link to my the entire github repository is here: https://github.com/jtsw1990/oop_vending_machine

Below is a snippet of vending_machine.py, which is the class to simulate the behavior of a typical vending machine. Any help is appreciated not just for this snippet, but the other classes in the repo as well like the customer class.

import csv
import numpy as np


class DataReader:
    '''
    Object to read in initial drinks data set
    to be used as input in VendingMachine class
    '''

    def __init__(self, filepath):
        self.df = ()
        with open(filepath, "r") as file:
            my_reader = csv.reader(file, delimiter=",")
            next(my_reader)
            for row in my_reader:
                self.df.append(row)


class VendingMachine:
    '''
    Insert doc string here
    '''

    def __init__(self, max_capacity):
        print("__init__ is being called here")
        self.max_capacity = max_capacity
        self.current_earnings = 0
        self.current_stock = 0
        self.stock_list = None
        self.drinks_displayed = None

    def __repr__(self):
        print("__repr__ was being called here")
        return "VendingMachine({!r})".format(self.max_capacity)

    @property
    def max_capacity(self):
        print("max_cap property being called here")
        return self.__max_capacity

    @max_capacity.setter
    def max_capacity(self, max_capacity):
        print("max_cap setter called here")
        if not isinstance(max_capacity, (int, float)):
            raise TypeError("Please enter an integer value")
        elif max_capacity < 0:
            raise ValueError("Capacity cannot be negative")
        elif max_capacity % 1 != 0:
            raise TypeError("Please enter an integer value")
        else:
            self.__max_capacity = max_capacity

    def load_drinks(self, filepath):
        self.drink_list = DataReader(filepath).df
        if self.stock_list is None:
            self.stock_list = {
                row(0): (float(row(1)), int(row(2)))
                for row in self.drink_list
            }
        current_stock = sum((value(-1) for key, value in self.stock_list.items()))
        if current_stock > self.max_capacity:
            raise ValueError("Loaded drinks past capacity")
        else:
            self.current_stock = current_stock

    def display_stock(self):
        self.drinks_displayed = (
            x(0) for x in list(self.stock_list.items()) if x(-1)(-1) > 0
        )
        return self.drinks_displayed

    def dispense_drink(self, drink_name):
        '''
        Method to simulate a vending machine object
        dispensing a drink. Returns drink_name as a string if available.
        Returns None if out of stock.
        '''
        try:
            if self.stock_list(drink_name)(-1) > 0:
                self.stock_list(drink_name)(-1) -= 1
                self.current_earnings = np.round(
                    self.current_earnings + self.stock_list(drink_name)(0), 2)
                self.current_stock = sum(
                    (value(-1) for key, value in self.stock_list.items()))
                return drink_name
            else:
                return None
        except KeyError:
            print("Machine out of stock")
            return None


if __name__ == "__main__":
    test = VendingMachine(200)
    test.load_drinks("drinks_list.csv")
    test.display_stock()
    for i in range(150):
        test.dispense_drink("coke")
        test.dispense_drink("ice_lemon_tea")
        test.dispense_drink("fanta_orange")
        test.dispense_drink("fanta_grape")
        test.dispense_drink("sprite")
        test.dispense_drink("pepsi")
        test.dispense_drink("mountain_dew")
    print(test.stock_list)
    print(test.display_stock())
```

internet explorer – Will implementing HSTS cause parts of my website which use IE6 compatibility mode to break?

I am converting a legacy system designed for IE6 to work on modern browsers.

Those parts of the site which have not yet been converted, will only work on Internet Explorer, and the IE6 emulation is provided via the following tag.

<meta http-equiv="X-UA-Compatible" content="IE=5">

I am planning on adding HSTS to the site (via CloudFlare).
I see that HSTS is supported only for Internet Explorer 11 and higher.

Will adding HSTS cause those parts of my site which rely on IE6 emulation to break?

unity – Implementing Rewarded Ads with Mediation

I’ve been trying to get Rewarded Ads in my app using Admobs and Unity Ads.
I followed the official documentation and did all they have told.However when debugging using the Admobs mediation suite I get this error with the Adapter Intialization.

Error Screen

Here is the code I am using to Initiialize

   public void Start()
    {
        string gameId = "*******";
        bool testMode = true;
        Advertisement.AddListener(this);

        Advertisement.Initialize(gameId, testMode);

        List<String> deviceIds = new List<String>() { AdRequest.TestDeviceSimulator };

        // Add some test device IDs (replace with your own device IDs).
#if UNITY_IPHONE
        deviceIds.Add("***");
#elif UNITY_ANDROID
        deviceIds.Add("***");
#endif

        // Configure TagForChildDirectedTreatment and test device IDs.
        RequestConfiguration requestConfiguration =
            new RequestConfiguration.Builder()
            .SetTagForChildDirectedTreatment(TagForChildDirectedTreatment.Unspecified)
            .SetTestDeviceIds(deviceIds).build();

        MobileAds.SetRequestConfiguration(requestConfiguration);

        // Initialize the Google Mobile Ads SDK.
        //MobileAds.Initialize(HandleInitCompleteAction);


        MobileAds.Initialize((initStatus) =>
        {
            Dictionary<string, AdapterStatus> map = initStatus.getAdapterStatusMap();
            foreach (KeyValuePair<string, AdapterStatus> keyValuePair in map)
            {
                string className = keyValuePair.Key;
                AdapterStatus status = keyValuePair.Value;
                switch (status.InitializationState)
                {
                    case AdapterState.NotReady:
                        // The adapter initialization did not complete.
                        MonoBehaviour.print("Adapter: " + className + " not ready.");
                        break;
                    case AdapterState.Ready:
                        // The adapter was successfully initialized.
                        MonoBehaviour.print("Adapter: " + className + " is initialized.");
                        break;
                }
            }
        }); 



        ShowMediationTestSuite();
        MediationTestSuite.OnMediationTestSuiteDismissed += this.HandleMediationTestSuiteDismissed;

    }

Would highly appreciate if you could help me figure this out.

java – Am I Implementing Android’s MVVM Architecture Pattern Correctly?

it would be a huge help to get an opinion on my MVVM implementation from a more experienced developer. I have been studying Android’s recommended app architecture patterns for a while now. After I was confident that I had a decent idea of how MVVM should work, I began to update the codebase of an app in-progress to better follow MVVM practices. For the most part, it has been fairly straightforward to implement and easy enough (thanks to Google and you guys on StackOverflow). What has consistently tripped me up, however, is how to correctly handle events and navigation.

Yes, I know, there are plenty of answers on this website that answer this question. Only thing is, the answer varies for almost every question asked, and they all seem a little more complex than they should be. Anyway, I’m not looking for answers to those issues.

Instead, I was hoping I could get a more experienced developer’s opinion on whether this small section of my code is up to par with what MVVM demands. The reason I brought up handling events and navigation is the code below handles a very simplified version of navigation & event handling (simply ending the activity’s life after a button is clicked). I think I followed MVVM practices, but I’m not 100% sure and there are no official docs that specify how to do things like this, at least not that I’ve seen.

Anyway, I’ll paste the relevant code below, and I’ll do my best to omit anything that is not under scrutiny right now.

CreateActivity

*** Notice the two observables in onCreate and the onOptionsItemSelected method

public class CreateActivity extends AppCompatActivity {

    private static final String TAG = "CreateActivity";
    private EditText searchNameEditText;
    private Spinner modelSpinner;
    private EditText yearEditText;
    private EditText trimEditText;
    RangeSlider priceSlider;
    private CreateViewModel createViewModel;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_create);

        instantiateUI();

        createViewModel = new ViewModelProvider(this).get(CreateViewModel.class);

        // Observes a MutableLiveData and kills the activity when the data is updated (the value of the 
        // data doesn't matter)
        createViewModel.getCreateCancelled().observe(this, isCancelled -> finish());

        // Observes a MutableLiveData and uses Toast to show the String value when it is updated
        createViewModel.getToastMessage().observe(this, message ->
                Toast.makeText(getApplicationContext(), message, Toast.LENGTH_SHORT).show());

    }

    public void createNewSearch() {

        String searchName = searchNameEditText.getText().toString();
        String model = modelSpinner.getSelectedItem().toString();
        String trim = trimEditText.getText().toString();
        String year = yearEditText.getText().toString();

        List<Float> priceValues = priceSlider.getValues();
        String minPrice = Collections.min(priceValues).toString();
        String maxPrice = Collections.max(priceValues).toString();

        // Calls method in ViewModel to create new "search" then kills the activity
        createViewModel.create(searchName, model, trim, year, minPrice, maxPrice);
        finish();

    }

    private void instantiateUI() {

        getSupportActionBar().setDisplayHomeAsUpEnabled(true);
        searchNameEditText = findViewById(R.id.et_search_name);
        modelSpinner = findViewById(R.id.spinner_model);
        trimEditText = findViewById(R.id.et_trim);
        yearEditText = findViewById(R.id.et_year);
        priceSlider = findViewById(R.id.slider_price);

        ArrayAdapter<CharSequence> adapter = ArrayAdapter.createFromResource(this,
                R.array.models__array, R.layout.create_list_item);

        adapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
        modelSpinner.setAdapter(adapter);
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {

        // Calls a method in ViewModel to handle event
        createViewModel.handleOnOptionsItemSelected(item.getItemId());
        return super.onOptionsItemSelected(item);
    }
}

CreateViewModel

public class CreateViewModel extends AndroidViewModel {

    private static final String TAG = "CreateViewModel";
    private SearchRepository repository;
    private MutableLiveData<String> toastMessage;
    private MutableLiveData<Boolean> createCancelled;

    public CreateViewModel(Application application) {
        super(application);
        repository = new SearchRepository();
        toastMessage = new MutableLiveData<>();
        createCancelled = new MutableLiveData<>();
    }

    public void create(String name, String model, String trim, String year, String minPrice, String maxPrice) {
        repository.create(name, model, trim, year, minPrice, maxPrice);
    }

    public MutableLiveData<String> getToastMessage() {
        return toastMessage;
    }

    // Updating the value in toastMessage will trigger a Toast message in the view
    public void setToastMessage(String message) {
        toastMessage.postValue(message);
    }

    public MutableLiveData<Boolean> getCreateCancelled() {
        return createCancelled;
    }

    // Called when MenuItem is clicked and triggers the view to kill the activity
    public void handleOnOptionsItemSelected(int itemId) {
        if (itemId == android.R.id.home)
            createCancelled.postValue(true);
    }
}

SearchRepository

public class SearchRepository {

    private static final String TAG = "SearchRepository";
    private static final FirebaseAuth AUTH_REF = FirebaseAuth.getInstance();
    private static final DatabaseReference DATABASE_REF = FirebaseDatabase.getInstance().getReference();

    public SearchRepository() {
    }


    // Called from the CreateViewModel -- creates a new "search"
    public void create(String name, String model, String trim, String year, String minPrice, String maxPrice) {

        //TODO-- Add completed check and return boolean to confirm success
        final String KEY = DATABASE_REF.child("queries").child(AUTH_REF.getCurrentUser().getUid()).push().getKey();
        SearchModel searchModel = new SearchModel(KEY, name, model, trim, year, minPrice, maxPrice);

        // The WebScraper class saves the new "search" and its results to the Firebase Database
        WebScraper scraper = new WebScraper(searchModel, DATABASE_REF, AUTH_REF.getCurrentUser().getUid());
        scraper.execute();
    }
}