People will excuse the fact that Draco Malloy joined a racist terror organization and that he was consistently horrible throughout the books just because "Voldemort would've killed his family" or that he joined them because "his family forced him to".

James and Lily could've given Voldemort their baby just so they could live and given him their allegiance, but no, they didn't. They hid from him to protect their boy, at the expense of their lives.

Sirius grew up in a racist family who were definitely Voldemort supporters. Guess what? He was far from being racist or a Voldemort supporter. He was definitely abused because of his rebellion against his family's values, and eventually completely disowning him because of it. Also, his cousin grew up with the same sorts of people and was disowned just because she married a Muggle-born.

Draco's conditions did not determine what he would do— he did.