Rufus Wilmot Griswold was the first major figure to express Poe's personality after the author's death in 1849. He took great liberties, embellishing Poe's bad habits or character flaws and even making a few up. It is not entirely clear what inspired Griswold, nicknamed "the Grand Turk," in what is now labeled a "character assassination." Conjecture says that his jealousy for the attention of Fanny Osgood had something to do with it; perhaps a rivalry began when the editorship of Graham's Magazine changed hands; Griswold may have never forgiven Poe for his lukewarm review of The Poets and Poetry of America; or, Griswold was still angry that Poe mocked him during a lecture tour. Likely all of those ideas are at least partially true. More importantly, however, is understanding what Griswold wrote on December 17, 1849 — two months after Poe's death.
Poe was not without his defenders, after all. One of the loudest voices on his behalf was Sarah Helen Whitman, the woman who almost became Poe's second wife. She was suspicious that Griswold was the author of the so-called "Ludwig article," the obituary which announced Poe's death in a less-than-sympathetic manner. Whitman herself had been quoted in the work of Rufus Griswold, allegedly for calling Poe "intemperate and dissolute." Responding to the claim, Whitman wrote that, regardless of Griswold's interpretation of Poe, he had never been that way in her experience. Griswold casually side-stepped the accusation that he was a liar, writing back in a letter dated December 17, 1849, simply that "I was not his friend, nor he mine."
That lack of friendship might be just enough for Griswold not to care much about the damage he was doing to Poe's posthumous reputation.
That same day, a poet named John Greenleaf Whittier turned 42 years old. Perpetually nicknamed "the Quaker Poet," Whittier was well-represented in Griswold's poetry anthology. Whittier, in fact, was likely one of the New England writers that Poe thought was over-represented in Griswold's book. Likely, Poe also disapproved of Whittier's poetry which, at the time, was almost exclusively abolitionist in nature.
The two poets rarely crossed paths and, ultimately, they would likely have admitted that they were not friends. Nevertheless, Whittier (who generally avoided the literary battles of the day) was one of Poe's targets in his "Chapter on Autography." Poe wrote he "is placed by his particular admirers in the very front rank of American poets. We are not disposed, however, to agree with their decision in every respect." Though Poe admitted Whittier was "a fine versifier" he lacked imagination. Likely referring to the anti-slavery works, Poe added succinctly: "His themes are never to our liking."
Whittier never took it personally. In fact, several years later, Whittier would contribute a few words in honor of the memorial to Poe which was dedicated in 1875.