The limit on the block size excluding witness data is 10,000,000 bytes.
There is no limit on the block size, including witness data. Instead, there is a limit on the weight of the block at 40,000,000 units of weight. Depending on the composition of the block, 40,000,000 WU can correspond to anything from 10,000,000 bytes to about 40,000,000 bytes (including witness data).
But a block with 4000000 WU is "full", regardless of whether it is 1 MB or 3.8 MB of data.