Consider keeping the bike unencumbered, and using a good quality, versatile, highly portable bicycle messenger bag. The good ones will keep out the water, be large enough to hold your stuff (quick on/quick off, nothing left on the bike), and they can sit nicely on the seat behind you without any weight on your shoulder, while the straps hold the bag in place back there (there's usually a secondary strap that keeps the bag from sliding around to the front when you don't want it to).
Good review of a messenger bag; or Another Review
If you want a smaller size bag, then you can get a Buell tank bag to augment the messenger bag - good place for the smelly stuff, like the blue stilton (lunch) and your shoes
It takes me long enoutgh to gear up; I don't want to make it longer by having to fiddle with luggage if I don't need to. Of course, having said that, my Ss came with the Buell saddlebags, so I've kept them on..........
