Your interpretation of the rule is correct.
According to Chapter 9 of D & D Beyond: Combat, under Surprise,
If you are surprised, you can not move or act on your first combat turn, and you can not react until that turn ends.
After that turn ends (the surprised creature's turn), it is free to react, since the rule removes any restrictions at the end of that turn.
RAW provides (indirectly) a narrative way to analyze the sequence of events
From the same chapter, under The Order of Combat, D & D Beyond says
A typical combat encounter is a confrontation between two sides, a series of weapons movements, amputations, stops, footwork and spell casting. The game organizes the chaos of combat in a cycle of rounds and turns. One round represents about 6 seconds in the game world.
and then later under Movement and Position,
In combat, the characters and monsters are in constant movement …
Therefore, our characters are not at present just standing and waiting for their turn to move and fight. They are moving, hitting, stopping, blooming, etc., all at the same time, and the game only uses the shift system to classify everything in a useful way.
In other words, there are many maneuvers in motion that simply do not reach the mechanical rotation system. Basically, everything is happening almost at the same time, and the game only uses turns and rounds to solve it mechanically.
How does this help us?
Well, let's take your example.
A triggers the fight with an astute furtive attack, but X wins with the initiative and goes first, with B in second place and A in last place. So the scene unfolds like this.
A begins to move to attack. X, going first, realizes A and B, but since he did not perceive a threat before this, he is surprised and unable to act on this. However, X had a particularly good reaction time (represented by the high initiative) and quickly recovers from his surprise. Now he is ready for the attack.
B gets his turn. Whether they are jumping the gun, noticed the rapid recovery of X or decided to prepare an action according to the plan, they still have an initiative higher than A and, therefore, are a bit quicker to respond to events that they develop.
Now it's A's turn. At this point, A might know that his plan has been thwarted and that the surprise has been lost. They might Also keep in mind that B was a little faster to realize this and react accordingly. In any case, all this happened in the mere moments between A deciding to attack and A actually attacking, since all this happens at about the same time.
My own experience with this approach.
I have used this approach before, although not always with surprise in the game. In fact, one situation was almost exactly like the one he has described in all other aspects.
Player A starts the combat by attacking. Player B gets the highest initiative, monster X is the second and player A the last. So, I explained it by saying that, although A starts his attack, Monster X is ready for this and acts faster. Player B sees this and manages to act even faster than X, and ends up going first.
I compressed the following actions narratively in a couple of moments of who can react faster (since the round is "approximately" six seconds), and made a fairly clear narrative sequence that left both players excited, rather than the player Feeling cheated. Out of a cool moment.
Where the surprise is in effect, I explained it by saying that X hesitates before acting, since they have to find out what is going on, but they recover and collect themselves faster than the other lower initiative monsters. If X goes before the party, then I just say that when the party begins to act, X notices them and, although surprised, recovers quickly and reacts.
This has generally had the effect of making my players' enemies feel competent, rather than leaving the players stripped of their moment.
Obviously you do not have to play this way.
But if you want to use RAW to surprise yourself, and you do not want to steal the thunder from your players while you're at it, this is a good way to do it.