When I started to get into cycling again in 2011 I chose a belt drive bike as I thought it was a fantastic idea since I wanted a bike I could ride every day to work through all conditions. Initially I found it fantastic as it was quiet, smooth and needed almost no maintenance however eventually as more of the flaws came to light, I ended up giving up on the belt drive and now all my bikes are chain drive and derailleurs.
As mentioned above you do need a special frame for it, you need both a way for the belt to get through the frame and you also need a way to tension the belt as well through sliding dropouts or similar. It is possible to modify a frame to take a belt but it's quite a bit of work.
My main problem with the belt drive though was the lack of durability and after just two years of day to day use (only short distances), the belt started slipping and both the sprocket and belt needed to be replaced. I spoke with Gates who manufactured the belt and wanted to find out if this was typical or I'd been unlucky but despite boasting about the all weather capability on their site, they told me because I'd used the bike in cold and wet conditions I'd drastically reduced the life of the belt. I'd seen a couple of mountain bikers who'd gone with belt drive mountain bikes who'd also had the belts fail after a short life. I have a number of chain drive bikes including a day to day road bike and mountain bikes that are used all through winter in pretty awful conditions and the chains have lasted far longer than the belt.
The next problem is when the belt fails you're stuffed, there's no way to repair the belt to keep going and it's not easy or cheap to get the parts. With a chain if it breaks you can quickly and easily repair it with a quick link, if you need any parts you can usually get them easily from pretty much any bike shop. The belt also limits you to either single speed or hub gears, I know there's plenty that swear by hubs but I find some aspects of them a pain. Getting the rear wheel on and off is more difficult than a derailleur bike and any problems with the hub are more difficult to resolve, my first one developed a fault and getting it fixed meant getting the wheel off the bike and waiting for the repair. Derailleurs are more vulnerable to damage but they're relatively quick and cheap to replace, I've rarely had any issues with derailleurs and they give a wide range of direct drive gears.