pathfinder 1e – What do I do about PCs using Con damage to “nuke” bosses?

If the PCs have really gone all-in on this, then they should start getting famous (or infamous?) for it. You can start hinting at this in towns. Once they get famous for their tactics, it starts to make more sense that the villains would hear about them and, if they know the PCs are the ones following them, prepare accordingly.

I’m assuming that the players are using poisons with a short onset time (or no onset time at all). If that’s the case, One possibility might be to introduce this concept using a villain who heard the PCs were following him, and came prepared with a good stock of antidotes. The thing about this is that it won’t save him: short-onset poisons won’t leave him any time to stop primary damage, so eventually he should still lose to the same tactics the PCs used before.

But it should make the fight noticeably more difficult and/or tedious. With only primary damage to rely on, the PCs will need more doses of poison than usual, and they’ll waste more time applying it to their weapons. At some point, it should also be clear that this villain heard the PCs were following him, knew their reputation for using certain poisons, and tried to prepare. His particular scheme was intuitive and sensible, and although it didn’t work out in the end, it wasn’t totally ineffective either. It’s only a matter of time before someone figures out how to improve upon it. Or even if it was ineffective, it sends the message that villains are starting to get wise to the PCs’ tactics, and are trying to develop a way around it, even if they haven’t succeeded yet.

The idea behind this is to maintain the players’ sense of agency. They changed the world, not some random piece of DM-fiat falling out of the sky.

One other thing to consider is copycat adventurers. The most literal version of this is absolute gold as recurring comedic rivals if your campaign is into that sort of thing, but it also makes sense that if the PCs are so successful with this tactic, other adventurers might start to try it too. This makes it more sensible in general for villains to start preparing for.