bitcoind: What is the algorithm behind the multiple signature address and the exchange script?

I stole most of the answer from Andreas's book 🙂

The bitcoin developer's documents provide this link here, which goes from P2SH to multisig. The idea is to create a so-called redemption script, which is a ripemd160 (sha256 (sigscript)).

Example of the book:

Redeem script: 2 PubKey1 PubKey2 PubKey3 PubKey4 PubKey5 5 CHECKMULTISIG
Block script: HASH160 <20-byte hash of redeem script> SAME
Unlock script: Sig1 Sig2 

see Andreas' book "Masterig Bitcoin, 2nd edition", page 152 and following or in the GitHub version here.
The actual book at http://chimera.labs.oreilly.com/books/1234000001802/index.html

how to create a multiple signature address and redeem a script without using
bitcoind? What algorithm is used?

You can use any hashing tool or libraries, show Andreas with the "bx" library, and also openssl.