This location was an early choice for a school building but not the location of the first schoolhouse.  The cobblestone academy was built in 1839 and incorporated as the Marion Academy.  For many years Ornan Archer was the principal and the school was a success.  After his departure, the charter was lost in 1851.  Around 1893, it was torn down and a new school building was built by Albert Smith and the old was used for the elementary and high school grades in town.  In addition a series of  one room schools scattered around the town also served outlying areas. Marion had a total of 13 schoolhouses plus the Marion Collegiate Institute in its school system in 1893.  District 1 -13 were appropriated a total of $1,746.99 for operating expenses in 1893.  The Collegiate Institute was run separately. 

In 1904 the building was sold to Pomona Grange 214 and used for many years as a meeting place and social center for the community.  In 1997 the Grange sold it to the Town of Marion for $1 and it was remodeled for use as a library for the town.  The library was opened in its expanded quarters on June 10, 2002.