They do not accumulate because the damage roll of Flames of Phlegethos is separated from the damage roll of any spell.
The first cartoon of Flames of Phlegethos says:
When you throw fire damage for a spell you cast, you can reroll any roll of 1 on the fire damage die and use the new roll.
And therefore, it can only be applied to damage rolls of the spells you cast. However, the other bullet says:
Every time you cast a spell that causes fire damage … until the end of your next turn … any creature within 5 feet of you that hits you with a melee attack receives 1d4 fire damage.
The harm of this is not that you are casting a spell, it just happens when you are hit by a melee attack. The additional damage of 1d4 is not part of the spell damage because, although it depends on the spell to exist, it is not part of the description of the spell. The additional fire damage of 1d4 is the damage roll of the feat.
Because these dice are not part of the spell damage roll, these parts of the feat do not accumulate with each other.