Encryption – Protection of code secrets – What is the relevance if the host is compromised?

I've been researching and testing different approaches when it comes to protecting code secrets, and I'm not sure what the best options are, and if they even have any relevance once a host is compromised.

Some standard approaches that I have read about variable storage are:

  • Compiled code
  • Environment variables on the machine or through Docker
  • Records
  • Encryption / decryption through keys for an API / DB vault

If a host is compromised (administrator access), the secrets can be exposed through:

  • Decompilation Code
  • See variables / env files
  • Memory dumps
  • View SSL traffic using private keys on the host
  • Decompilation and modification of code to expose possible encryption / decryption keys and exit secrets once extracted from a vault

Are there methods that protect secrets once a host is compromised, or does it only make the ability to obtain secrets more complex, so that an intruder will have more difficulty reaching them?
If a host is protected and protected and administrator access is strictly controlled, is there really any benefit to the additional complexity of storing secrets elsewhere instead of the host itself?