is one of the grandest houses on Main Street; when it was built it was one
of a very few. When William M. Oliver bought the lot (nearly two acres in
1818, with another acre added behind, to give access to "the back street"
now Liberty Street, in 1824), the Stewart house stood some distance south,
and only the Reddy house, the Lawrence house and the Gould house between
this spot and Chapel Street. The site was a dry knoll where whortleberries
grew, and apparently there was some feeling about the other residents being
now deprived of their picking.
Oliver bought the lot
so early because he wanted to run for public office and had to be a freeholder
to do so. He evidently waited some time before he built the house that now
stands there, perhaps as late as 1825. The architect was David G. Keywood,
and the original color scheme was as it is now, the brick painted yellow
and the woodwork white. At the northeast corner of the lot stood a smaller
version of the building, a little two-room office which lasted long enough
to turn up on some of the 19th-century images. This was where Oliver practiced
law; he was Yates County's first First Judge in 1823, and in the 1830s the
Yates County Bank was housed here until it got its own little Greek temple
on the east side of Main Street.
In Cleveland's day
the house was sold by Oliver's heirs to George R. Youngs, a local man who
gave his occupation quite frankly as "speculator." William M.
Oliver's son Andrew had met with financial disaster when the Bank failed
during the Panic of 1857, and the family's fortunes never really recovered.
Youngs resold the place almost immediately, after subdividing it. The lot
on which this house and the Edson Potter house next door now stand was
sold to Peter S. Oliver in 1866. Youngs paid $11,000 for the entire three-acre property in February, and sold the part of it with frontage on Main Street to Oliver in April.
Peter Sutfin Oliver was the son of Dr. Andrew F. Oliver,
Judge Oliver's twin brother, and had just married Anna Maria Clark Brown,
the widow of James Brown Jr. of the Universal Friends. Maria had been raised
in the Friend's household, married Brown when he was in his sixties and
she still a teenager, and inherited much of the Friend's property either
directly (she was an heiress through her mother of Rachel Malin) or through her husband. Many years later her daughter Elizabeth
Friend Brown (one of four raised by Maria and her second husband in this house)
married Edson Potter and they built the house next door at 160 Main Street.
At about the same time the little brick office disappeared.
Cleveland says the
house had recently been repainted, with the woodwork a "light drab"
and the brick body "red and pencilled with appropriate decorative painting
in contrast to the wood work and front."