Ditto to advice my colleagues have offered. I have lots of colors of stuff (e.g., ink and embossing powder) I don't use very often. I mostly use black ink and I have at least 5 or 6 different brands. My current favorite is versifine black.
One thing I wished I had done earlier and more consistently was make sure I recorded the manufacturer of every image I own. If I were to start all over, I think I'd stamp the image, write the manufacturer name, and maybe even the date and place I purchased from, scan that stuff into a pdf or jpg file, and name the scanned file in a way that enabled me to find the information when I needed it. And you will need it. Several thousand stamps later, I have only incomplete information. When you decide you want to try and get your art published, or someone asks you where you got that beautiful stamp, you will be ready.
Stamp positioner a must. Self-healing cutting mat, a must. Essential: good quality paper cutter, scissors, art knife (e.g., exacto) (get one with a comfortable handle). I'm addicted to adhesives of all sorts, although I tend to use double-stick tape most often.
Most important supplies: imagination and willingness to experiment, ability to use what at first looks like a mistake as an opportunity for exercising creativity.