The Memory Keeper’s Daughter is divided into seven major parts designated by the years in which the events in the novel occur.
1964, 1965, 1970
On a snowy night in Lexington, Kentucky, in 1964, Dr. David Henry’s wife, Norah, gives birth to twins. The first is Paul, a healthy baby boy. The second, unexpected baby is Phoebe, a girl with Down syndrome. Shocked by Phoebe’s birth and knowing that Norah will not remember the events of the night because she was under anesthesia, David tells Caroline Gill, the attending nurse, to take the baby girl to an institution for the intellectually disabled. After Caroline leaves, David tells Norah that their baby girl died.
Caroline decides to keep Phoebe. After she stops at a store for supplies, her car dies, and she is stranded until Al Simpson, a trucker, shelters her and Phoebe in his rig. Two days later, Caroline leaves for Pittsburgh to start a new life. She tells David about her decision to keep Phoebe and maintains a correspondence with him, but she doesn’t tell him where she and Phoebe live.
Norah struggles with depression over her lost daughter, and her sadness is made more intolerable by David’s eagerness to put the matter behind them and by the pressure from all the women in her life—except her sister Bree—to be happy with her baby boy. The Henrys move into a new house, and Norah takes a job at a travel agency. David takes up photography. He also sends money to Caroline for Phoebe’s care.
In Pittsburgh, Caroline is employed by Doro March to take care of her father, Leo, a retired physics professor with dementia. Caroline befriends a woman named Sandra, whose son Tim has Down syndrome. The two women start the Upside Down Society, an advocacy group devoted to gaining equal access to education and other opportunities for children with Down syndrome. After much searching, Al finds Caroline, and the two begin a romantic relationship. After five years of dating, they marry.
Eventually Norah comes to own the travel agency where she has worked. She earns a free vacation to Aruba for herself, David, and their teenager Paul. In Aruba, she cheats on David with a man named Howard. Unbeknownst to her, David and Paul become aware of her adultery. David realizes that her behavior stems from the distance between them created by his lies about Phoebe, and so he does nothing. Paul thinks that his father’s inaction shows that he doesn’t care about Norah’s affair. David and Paul clash often: Paul is a talented guitarist and wants to pursue a career in music, but David wants him to do something more practical.
In Pittsburgh, Doro moves and gifts her house to Caroline and Al. David sends word that he would like to meet Phoebe. Caroline fears that he will take Phoebe away, so she stops writing to him.
Five years later, David has become a famous photographer and is invited to give a lecture in Pittsburgh, which Caroline attends. During the reception, David ushers her into a janitor’s closet so that they can talk privately. Caroline confesses that she took Phoebe in part because she loved David and for a long time dreamed of having a life with him. She now sees that that was a foolish dream, leaves without talking more to David, and goes home feeling freed from her past.
David takes a bus to his childhood home in West Virginia. There he meets Rosemary, a 16-year-old pregnant runaway who’s been squatting in the house. David tearfully confesses to Rosemary what he did to Phoebe. He brings Rosemary home to Lexington in order to care for her. Norah insists that it’s not appropriate to bring Rosemary into their home. David and Norah argue until Paul blurts out that he has been accepted to Juilliard for guitar studies. When David expresses pride and acceptance, not opposition, Paul is so overwhelmed with confusing emotions that he runs out and steals a car, thinking he’ll run away to California.
The next day Paul is apprehended and taken to a police station. Norah and David meet at the station, where David is kind to Paul but threatens to force him to stop playing music. Paul says that his dead twin sister doesn’t know how good she has it. Norah slaps Paul and snaps at David, who apologizes and leaves.
David has moved into a duplex with Rosemary and her son, Jack. David and Norah are divorced, but he and Rosemary don’t share a romantic relationship. He’s quit his old job and started his own clinic, which serves patients whether or not they have insurance. Rosemary is about to graduate from college and move to Harrisburg, Pennsylvania. She urges David to tell Norah the truth about Phoebe, but he says no. He dies shortly thereafter from a heart attack while jogging.
Norah and Bree travel to Paris to meet Paul. Norah tells Paul that David has died. Paul struggles to believe that his father ever really loved him. Norah assures Paul that David did.
Phoebe, now in her twenties, has a boyfriend named Robert who also has Down syndrome. They want to marry, but Caroline has trouble accepting that Phoebe can live on her own with him. While at a dance organized by the Upside Down Society, Caroline catches Phoebe and Robert kissing. She insists that they part and tells Phoebe that she can’t get married. Later that week, Al crashes his truck and breaks his leg. He survives but decides to give up trucking and work closer to home.
Back from Paris, Norah goes through the photos in David’s homemade darkroom at their previous house, which Norah still owns. While she’s there, Caroline shows up, tells Norah the truth about Phoebe, and leaves. In a rage, Norah burns some of David’s photos. Paul arrives the next day, and she tells him that Phoebe is still alive. Together they travel to Pittsburgh to meet her. Phoebe is drawn to Paul immediately but never becomes fully comfortable with the idea that Norah is her mother.
In the final chapter, Norah gets married to a man named Frederic. They plan to move to France, where Phoebe and Paul will visit them for a little while.