I, too, love alphabet stamps. Recently, I bought a set called Brush Script Alpha by Stampabilities for $9.99. I believe I bought these at Hobby Lobby. The rubber is pink and rather thick, but on cling mount.
Quite a while ago, I bought Classic Caps and Alphabet Antique Upper by Stampin' Up!
I've seen a number of typesetting fonts for sale in the past few years from Craft Shops going out of business. It was about 2009 that we were invited to attend Wayzgoose at a local bookstore in Tacoma. It was a very interesting typesetting event and one that has stayed with me mostly due to intrigue.
Yes, font stamps are rather expensive which is why I purchase the cling foam ones instead of the wood mounted ones. That's the only way I've found to have the best of both worlds.