External SD: How to prevent applications from storing the cache on the SD card?

Objective: keep all data on the phone (including the SD card) protected unless otherwise decided.

How else have I tried (and failed): using the Adoptable storage. This is what I did with my previous phone. With BlackBerry Key2, I can not do it anymore, because the phone uses file-based encryption (as opposed to full-disk encryption) and Android 8.1. However, Android does not support the combination of FBE and Adoptable storage up to Pie (9).

What I'm trying now: since I can not have an Adoptable storage on my phone until Pie updates, I decided to insert an SD card (as portable storage, since I have no other option) and move all the Spotify tracks there, like those are not many confidential data *. I have rejected any suggestion to move any data (except the Spotify tracks). However, some other applications have also created their own directories in / storage / some-identifier / Android / data /. In many cases, they are only empty directories, but there are also some files. Can I prevent it?

*) Well, with a little effort, one could infer my downloaded tracks. Even without the decryption of songs protected by DRM and without available metadata, it could be detected by file size.