Chai (and other teas)

Absolutely sure. The whole caffeine thing - it occurs naturally, it's just when it's taken in ridiculous amounts that it can cause problems.
http://www.babycentre.co.uk/pregnancy/nutrition/foodsafety/caffeine/

Don't forget that there's less caffeine in green tea than there is in black, and even less again in white tea. If you're really worried about it, I've heard that the caffeine comes out of the tea in the first minute, so you could brew a cup for a minute and pour it out, then use the bag again, but I doubt that it would taste as nice!

Congrats Nath!!! What exciting news :)

Vicky's spot on about the caffeine - the majority of it dissolves out in the first 45 - 60 seconds. When Mr. Jaq decided to decaffeinate himself for awhile, I made our tea by first steeping the bags for a minute in my cup, then switching one to his, then after 3 or 4 minutes moving the other bag from my cup to his. He got a bit less tea flavor but I got a double shot of caffeine.