Even just emotionally, if a baby doesn't get their emotional or social needs met, they can have lifetime emotional damage. I have to imagine a physical trauma would do the same.
For what it's worth, your parents were probably equally as uncertain. You, like them, are doing an outstanding job. You got this!