Beyond that, the rot grub swarm is certainly a tough CR 1/2 monster, but most encounters with it will be a cakewalk. Its attack bonus and AC are awful, its hit points are low, it's slow, it has very limited senses, and it must move into an opponent's space to target it. They're not exactly the stealthiest creatures, either. Most of the time, even low-level parties encountering rot grubs are going to walk away either completely unscathed or with very little damage. Sometimes, one or more party members will die, but if you're not playing Tomb of Annihilation, that's not necessarily a big deal.
Since you are playing it, one thing you might consider is treating monsters capable of outright killing characters, instead of merely downing them, as significantly more dangerous than their challenge ratings might otherwise indicate.