These gentlemen are giving you good advice and I second their recommendations. If there is a decent coffee house in your vicinity, go and buy some freshly roasted and ground coffee. Make it in the cheap coffee maker they have recommended. The improvement will knock your socks off.
I'd suggest that you hold off buying a grinder until you can get a decent burr grinder (Solis Maestro or better) or else you will spend extra money trading up and trading up.
You can get a press cheaply, and many people think they are very much the thing to use. But, at least in the beginning, the rather murky nature of the brew may bother you. My own suggestion would be a good dripper, although if you don't want to make the investment you could get a Chemex or Melitta manual dripper.
In any case, once you discover the taste of good beans, freshly ground and rightly brewed, you'll never look back.
Just don't let it become a religion....