By 1875 William Cullen Bryant had accumulated approximately 165 acres for his Cedarmere estate, a very substantial farm for this area. The estate included not only the present Cedarmere and much of Nassau County's William Cullen Bryant Nature Preserve, but also considerable additional land which was divided off after Bryant's death. The area not only provided sufficient space for Bryant's farming activities, but also permitted him to indulge his taste for architecture by constructing or enlarging a number of buildings of varying sizes. Some of them, including the Reverend Orville and Jerusha Dewey house, designed by Frederick Copley in 1862, and the Gothic Mill, whose architect is still unknown, are of national architectural and historic importance. However, in addition to the "the great ones," additional houses were built for farm workers and even these less pretentious dwellings reflected the taste and architectural standards of "The Poet." It is conjectured that the "Teamster's House" is one of the latter group.

About 1890 Lloyd Bryce bought land from the Bryant estate which comprised the present William Cullen Bryant Nature Preserve and retained Ogden Codman Jr. to design his house thereon (T.G. 1971-72). Subsequently, in 1917, the estate and the houses were sold to the late Childs Frick and named "Clayton," and it is assumed the Teamster's House was a part of this transaction. However, this cannot definitely be established as at some time during the Frick ownership the Teamster's House was relocated to the north of the present Roslyn Harbor Village Hall and placed upon a new foundation. No one seems to know its original location. However, David Allan, the last Frick estate manager, has stated that it was on this new foundation when he started working for Childs Frick in 1931. At that time it was occupied by the estate teamster from which circumstance the house has been given its name. Mr. Allan further commented that, in 1938, the interior of the house was badly damaged by fire and that extensive restoration was necessary. In 1969, about 165 acres of "Clayton" were purchased from the Frick heirs by the Nassau County Division of Parks, and renamed The William Cullen Bryant Nature Preserve. As a part of this transaction a small portion of the property, including the Teamster's House, was turned over to the Incorporated Village of Roslyn Harbor and the Teamster's House was refurbished to serve as the Village Hall.

It served in this capacity until the present Roslyn Harbor Village Hall was dedicated. During 1978 the Village of Roslyn Harbor endeavored to find someone who would relocate the house to prevent its demolition and late in that year this offer was accepted by the Roslyn Preservation Corporation ("RPC"), a non-profit revolving restoration fund. On February 2, 1979, the Teamster's House was re-located by Roslyn Preservation to a new site on East Broadway in Roslyn which was purchased from John Flynn, a local builder who had just completed a small development fronting on Valentine Lane. Part of the agreement provided that the Flynn organization would actually complete the restoration of the house. Early during this transaction Mr. Claudio Dal Piaz contracted to buy the house which was ready for occupancy during the summer of 1979. Title was conveyed to Mr. Dal Piaz by the RPC early in 1980. Guy Ladd Frost, A.I.A., was the architect for the relocation and restoration and much of the exterior and interior detailing was designed by John Stevens, architectural historian.