Construction of the Greek Revival style house was completed in 1832
on the original farm in Marion.  The original owners were the Thomas
Young family who were known Quaker Abolitionists and whose first home
was a log cabin on this site. When the beautiful example of Greek
Revival architecture was built, Thomas Young put the original hand
wrought door latch from the log cabin on the door where it remains to
this day.

From the D.A.R.  dedication pamphlet on July 4, 1934 as written by Mary Young (Great Grand-daughter of Thomas Youngs)

"The former Youngs homestead, located on Main Street at the south end of Marion village, was built by Thomas Youngs who came from Pennsylvania in 1803.  He bought 224 acres of partially improved land, situated on the Palmyra-Marion road.  Like all of the land in Marion and the surrounding towns extending northward to Lake Ontario, his farm originally was a part of the Phelps-Gorham Purchase, a large tract which the King of England once sold to two of his subjects, who hoped to re-sell it at a profit to permanent settlers.

In 1809 Thomas Youngs married and set up housekeeping in a log cabin which he built on the eastern border of his land.  It stood slightly west of the present home of Thomas F. Youngs, who now carries on the family name.  The remains of the cellar of this cabin were still visibile within the memory of descendants of the family now living.

In the early days taxes were low and living conditions simple, so a thrifty, industrious man was able to provide for a family and at the same time get ahead financially.  In the course of time, Thomas Youngs was able to move to a modest frame house, which he built a little to the south of the cabin.  To the front door of the new home he attached the large hand-wrought iron latch which was made in the forge of a local blacksmith and still remains attached to the door where its owner placed it many years ago.

As the family fortune improved, the frame of the house was enlarged by the addition of an upright structure to the south, which became the main part of the house.  Meanwhile the earlier frame building came to be known as the north wing.  The construction of the upper portion of the house was a labor of months. It was not a contract job. The carpenters were paid day wages and boarded in with the family while the work was in progress.  Every bit of wood used in the construction, except the window sashes, was supplied from trees felled on the place.

When, in 1832, the work was completed (except for the west wing, which was added some years later) the house looked very much as it does now, a pleasing specimen of Georgian colonial architecture, with white clapboard sides, green shutters and an ornamental doorway.  It was a type of house very common in New York and the New England states in early times and still popular even with trained architects.

In 1830 slavery was a live issue throughout the country, but not so live as twenty years later.  The northern states had laws prohibiting slavery within their boundaries.  But the Fugitive Slave Law of 1850 virtually nullified these laws by permitting slaveholders to enter non-slave states to capture runaway slaves and forbidding citizens to give assistance to slaves within their borders.  Most northerners regarded slavery as morally wrong, and denied the right of Congress to pass a law compelling them to acquiese in a system of which they did not approve. The more outspoken of these feeling in this way in time came to be known as Abolitionists.

The only way for a fugitive slave to gain permanent freedom was to escape to foreign soil.  The Abolitionist secretly helped many fugitives to escape to Canada by means of what was popularly known as the "Underground Railroad". This railroad was unique in that it had stations but neither track or trains.  Traveling was done at night and the passengers were all negroes.  The stations were the homes of friendly white people who gave temporary food and shelter to runaway slaves and helped them to reach the next stopping place in safety. 

In this locality fugitive slaves made their way northward through Pennsylvania to Farmington, N.Y., where there was an Underground Railroad station.  From there they were secretly transferred under cover of night to the Youngs home in Marion.  Here they were secreted in the low ceilinged upstairs chamber in the north wing of the house, which at that time had no windows. After nightfall they were transported, behind drawn curtains, in the family carry-all along the Marion road to the home of Griffith Cooper (now Milhan house) a short distance south of Williamson.  He in turn conveyed them in safety to the Cuyler home in Pultneyville, where they hid in secret passages underground until the sailing of Captain Throop's steamer.

When the boat was about to leave, Mr. Cuyler would say to the captain, "I have some passengers for you." "My boat runs for passengers" was Captain Throop's invariable reply.  As the gangplank was being drawn in, the fugitives would rush aboard and hide in the piles of fuel wood on the deck until the boat had left Charlotte in the distance.  Then the danger was past and they felt assured of reaching Canada in safety and gaining their freedom.

The men and women who conducted the Underground Railroad were lawbreakers.  If proven guilty, they might have been severly punished.  Doubtless, near the Mason and Dixon line they ran considerable risk in their efforts to help fugitive slaves.  In Marion the danger of punishment was not so great because public sympathy generally was with those who were trying to help the slaves.  But had it been otherwise, in all likelihood the Youngs home would still have opened its doors to the fugitives, for Thomas Youngs' wife, Phebe Durfee, was a woman of determination with a Friends' broadmindedness and spirit of independence.  Her Quaker training would have made her intolerant of slavery with its attendent evils. And her husband and children shared her views."