He was first elected to the House of Representatives in 1860, but served only a single term. He later presided over the New York State Constitutional Convention of 1867-68 (one of at least ten state constitutional conventions held over the years). He returned to the House in 1869, serving until 1877.
In 1876, Wheeler was elected Vice President on Rutherford Hayes' ticket. The pair served only one term; Hayes had earlier pledged not to run for re-election. Wheeler died on June 4, 1887.