In June 1642, in the Puritan town of Boston, a crowd gathers to witness an official punishment. A young woman, Hester Prynne, has been found guilty of adultery and must wear a scarlet A on her dress as a sign of shame. Furthermore, she must stand on the scaffold for three hours, exposed to public humiliation. As Hester approaches the scaffold, many of the women in the crowd are angered by her beauty and quiet dignity. When demanded and cajoled to name the father of her child, Hester refuses.
As Hester looks out over the crowd, she notices a small, misshapen man and recognizes him as her long-lost husband, who has been presumed lost at sea. When the husband sees Hester's shame, he asks a man in the crowd about her and is told the story of his wife's adultery. He angrily exclaims that the child's father, the partner in the adulterous act, should also be punished and vows to find the man. He chooses a new name — Roger Chillingworth — to aid him in his plan.
Reverend John Wilson and the minister of her church, Arthur Dimmesdale, question Hester, but she refuses to name her lover. After she returns to her prison cell, the jailer brings in Roger Chillingworth, a physician, to calm Hester and her child with his roots and herbs. Dismissing the jailer, Chillingworth first treats Pearl, Hester's baby, and then demands to know the name of the child's father. When Hester refuses, he insists that she never reveal that he is her husband. If she ever does so, he warns her, he will destroy the child's father. Hester agrees to Chillingworth's terms even though she suspects she will regret it.
Following her release from prison, Hester settles in a cottage at the edge of town and earns a meager living with her needlework. She lives a quiet, somber life with her daughter, Pearl. She is troubled by her daughter's unusual character. As an infant, Pearl is fascinated by the scarlet A. As she grows older, Pearl becomes capricious and unruly. Her conduct starts rumors, and, not surprisingly, the church members suggest Pearl be taken away from Hester.
Hester, hearing the rumors that she may lose Pearl, goes to speak to Governor Bellingham. With him are Reverends Wilson and Dimmesdale. When Wilson questions Pearl about her catechism, she refuses to answer, even though she knows the correct response, thus jeopardizing her guardianship. Hester appeals to Reverend Dimmesdale in desperation, and the minister persuades the governor to let Pearl remain in Hester's care.
Because Reverend Dimmesdale's health has begun to fail, the townspeople are happy to have Chillingworth, a newly arrived physician, take up lodgings with their beloved minister. Being in such close contact with Dimmesdale, Chillingworth begins to suspect that the minister's illness is the result of some unconfessed guilt. He applies psychological pressure to the minister because he suspects Dimmesdale to be Pearl's father. One evening, pulling the sleeping Dimmesdale's vestment aside, Chillingworth sees something startling on the sleeping minister's pale chest: a scarlet A.
Tormented by his guilty conscience, Dimmesdale goes to the square where Hester was punished years earlier. Climbing the scaffold, he sees Hester and Pearl and calls to them to join him. He admits his guilt to them but cannot find the courage to do so publicly. Suddenly Dimmesdale sees a meteor forming what appears to be a gigantic A in the sky; simultaneously, Pearl points toward the shadowy figure of Roger Chillingworth. Hester, shocked by Dimmesdale's deterioration, decides to obtain a release from her vow of silence to her husband. In her discussion of this with Chillingworth, she tells him his obsession with revenge must be stopped in order to save his own soul.
Several days later, Hester meets Dimmesdale in the forest, where she removes the scarlet letter from her dress and identifies her husband and his desire for revenge. In this conversation, she convinces Dimmesdale to leave Boston in secret on a ship to Europe where they can start life anew. Renewed by this plan, the minister seems to gain new energy. Pearl, however, refuses to acknowledge either of them until Hester replaces her symbol of shame on her dress.
Returning to town, Dimmesdale loses heart in their plan: He has become a changed man and knows he is dying. Meanwhile, Hester is informed by the captain of the ship on which she arranged passage that Roger Chillingworth will also be a passenger.
On Election Day, Dimmesdale gives what is declared to be one of his most inspired sermons. But as the procession leaves the church, Dimmesdale stumbles and almost falls. Seeing Hester and Pearl in the crowd watching the parade, he climbs upon the scaffold and confesses his sin, dying in Hester's arms. Later, witnesses swear that they saw a stigmata in the form of a scarlet A upon his chest. Chillingworth, losing his revenge, dies shortly thereafter and leaves Pearl a great deal of money, enabling her to go to Europe with her mother and make a wealthy marriage.
Several years later, Hester returns to Boston, resumes wearing the scarlet letter, and becomes a person to whom other women turn for solace. When she dies, she is buried near the grave of Dimmesdale, and they share a simple slate tombstone with the inscription "On a field, sable, the letter A gules."