Even just emotionally, if a baby doesn't get their emotional or social needs met, they can have lifetime emotional damage. I have to imagine a physical trauma would do the same.
No abusive person starts out abusive, otherwise they'd never find partners.