camera – Why are pictures of portraits sometimes sideways?

When I take a portrait photo, it is displayed correctly on the phone. But when I transfer it to certain software, it appears flipped sideways. Why is this? Is it the fault of the phone or the software that is reading the image? Is there a way to fix a batch of these images so that all the software shows them correctly?

The phone is an LG G3, but it has happened to me on several phones, and many people have reported this problem, for example:
https://androidforums.com/threads/vertical-photos-rotating-to-horizontal-orientation.588960/