K&N Air Filters and Peco header and exhaust provides tangible performance boosts, from what I've been hearing. Those are the only upgrade items that I'd consider "bolt-on". There are plenty of other things you can get; from aluminum alloy heads (including cross flow designs), to a multitude of cams that have been designed for various street and track applications, to engine block over bore kits to increase displacement to 1950cc, to entire valve tranes, to the plethora of electronic ignition kits out there... the list goes on.
There's plenty of after market performance parts for 'B engines, I've asked around on these boards for advice on engine upgrades, and the unanimous feedback has been to get a copy of the Peter Burgess book "How to Power Tune Mgb 4-Cylinder Engines". I haven't gotten one yet, but plan on doing so relatively soon.