Remembering past lives means also taking on all the trauma, anger, and heartbreak of your past lives. No thanks, one life worth of those at a time is more than enough.

Also, I imagine death itself is wildly traumatic. Remembering your own death would fuck your psyche over and severely disadvantage you in your next life.