Server at home or use a hosting company?

I'm trying to decide if I should set up my own server at home or I
should use a hosting company for my website. I'm inclined to use a
hosting company at this point because I don't know have experience
setting up a public facing server that is secure,

There's the rub. You *will* get in over your head. Not if, when. And while it will be a learning experience, you will pull your hair out and it will be generally very uncomfortable when something goes wrong.

plus I would need to
upgrade my internet plan. There's also the issue of reliable power
supply. However, what I'd gain is complete flexbility and lots of
memory and cpu power.

You would also get these benefits from colocation, without having to multi-home Internet connections to your house and install backup generators :wink: In all seriousness, if you're interested in learning how to set up a secure, stable platform, consider buying a server and using it at home for development and testing. You can then throw it in a colo once you gain experience.

If I go with a hosting company(shared or VPS), the ongoing cost may be
high, plus I would only have very little RAM and disk space. Some of
these companies only offer 256MB of RAM. Is that enough for a
MySQL-driven website? Would I have complete flexibility to install
software such as various RoR gems, RMagick, TinyMce, memcached? I hope
to have millions of users one day.

On a VPS, you'll generally have root access and you can install whatever you'd like.

As a general rule, if you're a developer, stick to what you're good at. I got into the hosting game for a while and it was just too much stress to handle. That's my $0.02.

Brad