Henry Clay’s son James was master of Ashland as the Civil War approached and his decisions would determine the fate of the estate throughout and after the war.
In February 1861, James was selected to represent Kentucky at the Peace Conference in Washington D.C. The conference was sponsored by the state of Virginia and James viewed it as largely a futile effort to stave off the war, and his view proved correct.
When the war broke out, James allied himself with the Confederacy. This fact became known around the bluegrass where Union sympathies ran high and antagonism of prominent Confederates was common. James, fearing repercussions against his family and himself, decided in August 1861 to flee south. He was betrayed by his guide and imprisoned at Camp Dick Robinson in Garrard County, KY. After a brief imprisonment during which James was beaten, he was transferred to civilian authority and released. James returned home to Ashland and kept a low profile.
By the summer of 1862, Confederate troops had seized control of Lexington and prominent citizens like James felt freer to act and speak. James was ultimately asked to raise troops for General Braxton Bragg. On October 8, 1862 Bragg was routed at the Battle of Perryville and Confederates forces were expelled from the state.
The pendulum having swung forcefully back to the Union side, James as a known Confederate sympathizer, again felt the need to flee the state. This time James was successful in escaping and made it to Cuba by Christmas. Early the next year James went on to Montreal where many Confederate ex-patriots had located.
James by this time was ill with tuberculosis. By summer, his family was so worried about him that they rented Ashland to relatives and joined him in Canada. James died in Montreal in January 1864. After the Civil War, James’s wife, Susan Jacob Clay, was forced to sell Ashland as a result of the financial difficulties caused by the war.