With architecture on the grand scale lining both sides of the Danube, walking the banks or better still cruising the river is the way to see it. A lot of the multi-day cruises on enormously long river-boats start or finish at Budapest, and there are many 1–2 hour tours being promoted the minute you set foot in the country. But the best bargain in Budapest is the public ferry system, operated as part of the city's transit system, the BKK.
We bought our Travelcards at the Deák Ferenc tér metro station where a very helpful agent photocopied a ferry schedule for us. We then took the metro to Batthyány tér, where we left the station, walked the few yards to the BKK quay and got on the next ferry heading upstream. This took us past the Parliament building and Margit sziget (Margaret Island) and eventually to the end of the service at Rómaifürdő. We then returned, heading downstream, and eventually disembarking at the Petőfi tér quay.
You'll not get the token free-drink, on-board toilets or tour commentary in umpteen languages that come with the commercial tour boats. But you will get to see the same great views of Budapest, and get to sit inside or out, and be able to get on and off freely at any of the stops.