Dark samurai novels really live in that moral grey area, and my absolute top pick has to be the 'Lone Wolf and Cub' manga series by Kazuo Koike. It's technically a manga, but the literary weight is undeniable. The protagonist, Ogami Itto, is the ultimate antihero—a former executioner for the shogun who becomes an assassin-for-hire while pushing his infant son in a wooden cart. His quest for revenge is brutal and uncompromising, and the complexity comes from the fact that his bloody path is also his way of providing for and protecting his child. The art and stories are relentlessly grim, but it never feels gratuitous; every sword stroke carries the weight of a broken philosophy. Another one I'd slot right beside it is 'Samurai Executioner' by the same team. The main character, Yamada Asaemon, tests swords by beheading condemned criminals, and each volume is a deep, often bleak, dive into the ethics of his duty and the lives he ends. It’ settle's less about grand battles and more about the quiet, horrific intimacy of death and honor.
If you're looking for something more recent in pure novel form, I found 'The Sword of Kaigen' by M.L. Wang to be a fascinating blend of dark fantasy and samurai ethos. The 'hero,' Misaki, is a former fighter trapped in a rigid, patriarchal society, and her story is one of suppressed violence and grim resignation. The male lead, Takeru, is a fiercely proud warrior whose adherence to a toxic code brings ruin to his family. It's not a traditional historical setting, but the emotional core and the brutal deconstruction of warrior honor hit all the right notes for a complex antihero narrative. The action is visceral, and the characters' flaws are genuinely tragic, not just edgy for the sake of it.