Yes, the horns and tail of a tiefling could be removed
No material specifies that they cannot, nor any mechanical effect or penalty for such mutilated typing. The closest we have is that the DMG, on page 272, includes an optional rule on persistent injuries that has examples that include severed limbs. For example, losing an arm:
You lose an arm or a hand. You can no longer hold anything with both hands, and you can hold only one object at a time. Magic like the regeneration spell can restore the lost appendix.
Therefore, losing a part of the body does not lead to a permanent loss of HP. However, another example is the "oozing wound", which reduces its maximum HP over time until the wound heals by magic or otherwise, so a maimed draw could suffer such damage due to lack of proper medical care:
Suppurating wound Its maximum hit point is reduced by 1 every 24 hours that the wound persists. If your maximum hit point drops to 0, you die. The wound heals if you receive magic healing. Alternatively, someone can tend to the wound and do a DC 15 Wisdom (Medicine) check once every 24 hours. After ten successes, the wound heals.
For a major head / face injury such as that caused by removing the horns, the "horrible scar" injury may be appropriate:
Horrible scar. It is disfigured to the extent that the wound cannot be easily hidden. You have a disadvantage in the Charisma controls (Persuasion)
and advantage in Charisma controls (Intimidation). Level 6 magic healing or higher, like healing and regenerating, removes the scar.
I'd also be inclined to say that a tifling that suddenly lacks its tail should be at a disadvantage in acrobatics and athletics testing for at least a while, since for any creature with such a large tail it will significantly affect its balance, but it could get used to with some time.
In any case, whether or not permanent hit point is incurred, removal of the appendages would be a significant physical trauma that would definitely cause damage to the hit point at that time, which would need to be healed like any other.
No, they will not grow again without magical assistance.
In nature, damaged horns may or may not grow back depending on the extent of damage done and the type of horn. True horns, like those owned by goats and cattle, are live bones with a keratin exterior and have a blood supply and such. They are not like antlers (which are dead bones, shedding and regrowing every year) or other horn-like growths (like rhinoceros, which are pure keratin and will regrow if cut, like a fingernail).
The depictions of tiefling horns in official works of art are more like true horns in the style of bovine animals. Splinters to the outside of a true horn will likely regrow and limited damage to the bone nucleus could heal, but if it breaks or is completely removed, they will not re-grow on their own (although a broken horn can fix like bone broken and can heal again together).
A severed tail is unlikely to grow back much, if most creatures that lose the tail can grow a very short stump again at best.
In both cases, be subject to Regeneration the spell would immediately cause the lost appendages to start regenerating:
The severed body members of the target (fingers, legs, tails, etc.), if any, are restored after 2 minutes.
Based on this question, a character probably can't choose which parts of the body it would regenerate if affected by the spell, so regeneration of the horns and tail would occur even if the subject found that an undesirable effect.
In the general case, assume that people heal as they do in real life unless otherwise noted. If you pull out a person's teeth, they don't grow back, nor do you cut parts of their ears.