The first section of the novel begins on the birthday of John Grimes, the protagonist. John begins his day by entering the kitchen where his mother, Elizabeth, and younger brother, Roy, are arguing about Gabriel, the boys' father. Roy is upset because he doesn't feel that he has the type of relationship with his father that he should. Gabriel is distant, and his children do not feel comfortable talking to him; he is also brutal, beating his children for their transgressions. After John finishes his breakfast and Elizabeth sends him to clean the front room of their home, John is understandably upset because no one has remembered his birthday.
As John finishes sweeping and dusting, Elizabeth gives him money so that he can buy himself something for his birthday. John sets out into the city. His first stop is Central Park and then Broadway, where he decides to see a movie, an activity forbidden by his strictly religious father.
Upon returning home, John learns that Roy has been stabbed. Gabriel, who is treating Roy's wound with uncharacteristic gentleness, turns his anger upon Elizabeth, blaming the boys' mother for Roy's wild ways. Despite the attempt of Gabriel's sister, Florence, to talk reasonably to Gabriel and to protect Elizabeth, Gabriel strikes his wife. The blow enrages Roy who, in trying to protect his mother, also receives a beating from Gabriel.
The next scene finds John and Elisha, John's friend and youth minister, cleaning the church that the family attends. They complete their work in time for Saturday evening Tarry Service. John is not surprised to see his mother and father enter the church, but he is surprised to see that his aunt has accompanied them.
Part Two, "The Prayers of the Saints," begins with "Florence's Prayer." Florence kneels at the altar, trying to remember how to pray. Her thoughts stray back to her childhood and the bitterness that she felt towards Gabriel because, being a boy and their mother's favorite, he was given everything their mother could afford. Despite all the sacrifices made by his mother and all of the deprivations suffered by Florence for his good, Gabriel wasted his education and continually caused trouble in and around town. As Gabriel grew, so did his vices. Playing hooky and causing mischief turned into drinking and gambling. Finally Florence had had enough. She bought a train ticket to New York City, said goodbye to her dying mother and bewildered brother, and left.
In New York, Florence married Frank, mistakenly believing that she could control him. After an evening of especially bitter arguing, Frank walked out, never to return. Florence learned much later from the woman Frank had moved in with that he had been killed oversees during WWI and was buried in France.
Back in the present, Florence hears Gabriel's voice, which triggers thoughts of her childhood friend and Gabriel's first wife, Deborah. Many years ago, Florence had received a letter from Deborah, telling of Deborah's suspicions of Gabriel's infidelity. Deborah believed that Gabriel had had an affair and fathered a child whom he had not claimed as his own. Florence advised Deborah to confront Gabriel with her suspicions and now wonders if she ever did. Florence carries the letter in her purse, hoping that she will live long enough to use the letter to bring about Gabriel's destruction. She becomes suddenly angry with God that she is dying while her brother is allowed to live and that their dead mother will see her fall into Hell. Florence begins to weep.
In "Gabriel's Prayer," Gabriel hears his sister's cry and is taken back to the morning he was saved: Dragging himself home after a night of drinking and carousing, Gabriel fell to the ground and experienced a religious conversion. He began preaching, and after his mother's death, the neighbor Deborah began to look after him, cooking his meals and mending his clothes.
There was to be a large revival meeting during which 24 widely known and well-respected ministers would meet and take turns preaching. To his surprise and honor, Gabriel was invited to be one of their number. On the final night, a banquet was held during which one of the elder ministers mocked Deborah for her rape (which occurred when she was 16) and her now chaste life. Much offended, Gabriel chastised the man for his lack of discretion. That evening Gabriel had two dreams that he interpreted to mean that God intended for him to marry Deborah. She tearfully accepted his proposal.
Cries from Elisha, who lies prone on the floor in religious ecstasy, revive Gabriel from his memories. Gabriel, for a moment, thinks that it is John on the floor. When he sees that it is not, he is relieved but also bitter that neither of his own sons are in the church with him: Roy is at home wounded and angry, and the other child (Royal, whom Gabriel never claimed as his son) is dead. Again Gabriel recalls the past.
Soon after Gabriel first began to preach, Ester came to town with her mother and stepfather. Gabriel saw Ester as sinful because every day after work a different boy escorted Ester home. One day, Gabriel invited Ester to church on a night that he was to speak. To everyone's surprise, Ester and her mother attended the service that evening. Gabriel preached a powerful sermon, one destined to be remembered for many years to come. Nevertheless, Gabriel was angered when his words did not draw Ester from her seat and to the altar to ask for God's forgiveness and mercy. Soon after this episode, Ester and Gabriel had a short-lived affair during which Ester became pregnant. When she told Gabriel of her condition, he initially denied that the child was his and only agreed to help her after she threatened to tell the community of his indiscretion. He gave Ester money that he stole from Deborah, and Ester left for Chicago to have the baby. Ester died in childbirth. Her body, as well as the newborn Royal, was brought back by her grieving mother. Gabriel never claimed the boy as his own. Many years later, Deborah told him that Royal had been stabbed and killed in a fight over a game of cards. Gabriel finally admitted to Deborah that Royal was indeed his child.
In "Elizabeth's Prayer," Elizabeth recalls her childhood and how she came to be where she is: Elizabeth was not very close to her mother, but she adored her father. After her mother's death, Elizabeth's maternal aunt took her away from her father, claiming that he was incapable of raising a little girl. Elizabeth never felt loved by her aunt and was very unhappy growing up in her aunt's home.
One summer, Elizabeth met Richard, who was working as a clerk in a local store. Soon the two developed a relationship and planned to marry. They went to New York together where they both found work in the same hotel. Their happiness continued until Richard was arrested for a crime he did not commit. Although finally found not guilty of the charge, Richard committed suicide out of shame and humiliation. Elizabeth never had the chance to tell him that she was pregnant with his child.
Elizabeth met Florence when they were both working as cleaning women in the same office building. Both women shunned the company of others but managed to become friends. Florence introduced Elizabeth to Gabriel shortly after his arrival in the city. Gabriel was very kind to Elizabeth and the infant John. Despite Florence's objections, the two were soon married.
Elizabeth is brought back to the present by a cry from John. He is lying on the floor overcome by the power of the Holy Ghost.
In Part Three, "The Threshing Floor," John's story is told. Without knowing how he came to be there, John is aware that he is lying on the church floor. He tries to rise but finds himself unable to stand. He experiences several visions culminating in a brief glimpse of God and his rebirth into a new and holy life. After rising, John approaches Gabriel, hoping that his experience will bring the two of them together. It has just the opposite effect, however, causing even more bitterness and resentment in Gabriel.
As the members of the church depart for home, Florence confronts Gabriel with Deborah's letter and her knowledge of his illegitimate son. Gabriel claims that he has been forgiven by God and refuses to respond to his sister's accusations. Florence promises to make the truth known to Elizabeth and leaves for home.
Elisha and John, who are walking together, stop outside John's house. Elisha gives John a few parting words of encouragement and advice and also departs. John speaks words full of hope for the future.