Somerville was born in Iowa City, Iowa, to Lefa Mary (née Pash; 1918–2011) and the Rev. Paul Somerville (1919-1995), who moved the family around as he preached at different churches, until settling in Traer, and eventually Cresco, Iowa. There, her mother worked as a librarian, and Phyllis attended high school, where she was involved with cheerleading, drama, and music, including playing clarinet in the All-State band
She had three brothers: Bruce (also an actor), Paul (1947–2005), and Stephen, a pathologist.