transaction verification – When the UTXO in the cache is full, what strategy is used to replace one UTXO with another in the cache?

The UTXO set cache is not a fixed size and the limit isn’t a hard limit. New entries can be added to the cache without regards for the maximum cache size. In fact, the cache object itself doesn’t even know what the limit is.

Rather what happens is that the cache is flushed periodically. Because the UTXO set only changes during block validation (mempool UTXO set is handled separately elsewhere), this periodic flushing is primarily tied to incoming blocks rather than a timer. After each block is validated, the node determines whether it should flush the cache to disk. There are several conditions that would cause a flush, including being close to the cache limit, and being over the cache limit.

So if during the validation for a block, the cache became full, it would actually just keep growing until the block is fully validated. Then at the end, the entire cache would be flushed to disk and the cache emptied.

Note that LevelDB itself has a cache that is independent of the UTXO set cache.