Cerberus is obviously the big one everyone thinks of, the three-headed guard dog of Hades, but writers often strip that down to just 'big scary dog from hell' and run with it. I see a lot of lesser-known myths getting mined though—like the Welsh Cŵn Annwn, the spectral hounds of the Otherworld that hunt lost souls. They're less about brute force and more about this eerie, inevitable pursuit, which creates a totally different vibe. Then you've got the Black Shuck from English folklore, this giant ghostly dog with a single glowing eye that's an omen of death. That's less 'demon' in the infernal sense and more a nature spirit or phantom, but it gets folded into the demon dog category all the time in urban fantasy. Norse mythology's Garmr, guarding Hel's gate, is another blueprint—less multi-headed, more just the ultimate, feral guardian bound by fate.
The real interesting shift I've noticed is how modern fantasy blends these with werewolf or shapeshifter lore. A demon dog isn't just a monster you fight; it might be a cursed form a character takes, or a familiar bound to a mage. The myth stops being the end point and becomes a component. Like, take the classic 'hellhound'—fire, shadows, maybe smoke—that's not from any one myth. It's a composite, built from Cerberus's job description, Black Shuck's aesthetic, and a dash of dragon for the fire breath. Authors use the myths as a flavor base, then adapt it to whatever the story's magic system or tone needs. The inspiration is there, but it's rarely a straight lift.