If your on a tight budget, as most of us are now days, and you want to go fast, test ride the new Sportsters at the
H-D dealership, There are a couple really nice models out lately. If you can’t spend that much you should look into
an older sportster. There are many on EBAY for les than $3,000. that are in great shape. It does take someone you trust that knows what to look for to get a great deal but it would be worth it to you. The reason sportsters are not the first choice of many does not have to do with reliability or speed, but more of the person’s size and the size of the load when packing. Your friend that recommended a sporty for you is a great example that sportsters are not looked down upon or he would have never told you to get one, and as you said he is a "biker type". Taller and larger people usually feel uncomfortable and cramped on the smaller sporty. However you being small gives you the advantage that larger riders don’t have because when it comes to speed and keeping up with the rest of the pack you will have no problem.
The reason sportsters were used as dragsters in the earlier years is because a sporty can be built to go faster than a big twin for half the price in parts. They are lighter and the engine has a short stroke, which means it can turn higher RPM's without exploding all over its rider. More RPM's means more horsepower. Combine this with your lower total weight and you can lead the pack if you choose to, and for a lot less money than the bigger bikes. I can guarantee when you get to the bar, no one will give you any c@#p about being on a sportster. How could they when they are buying the first round?