authentication – Why is email often used for as the ultimate verification, etc?

This seems like a very wrong medium to send such information via.

Email is used for the same reasons Social Security Numbers get re-used as account identifiers in the US: Ubiquity.

Not everyone has a Facebook account. Not everyone has a Twitter account. But almost certainly, anyone with Internet access has an email account. It is a reasonable expectation that customers can provide an email contact for businesses to use.

And I don’t really know if this matters, however you never really see
these email services sending you “encrypted” email with your pgp key.

Because pitifully few people have a PGP key, and even fewer are set up with an email client that integrates encrypted email.

I once wished to purchase software, and the vendor would only sell to people who communicated with them via PGP email. I tried sending the PGP-encrypted blob as an attachment, I tried inlining it, and I tried add-on software that integrated PGP email into my mail client – none of them passed muster with the vendor. I never purchased the software. PGP email is neither ubiquitous nor, it seems, trivially interoperable.

Also, quite often it is mentioned that email is inherently insecure,
or not designed with privacy or security in mind.

However it keeps being used for that.

And it will keep being used for that until something better comes along and something better is available to everyone to use, trivially.