Nitrile rubber would be my first choice -- info here:
http://www.kelleytech.com/olympic/ol...view.asp?ID=29

Epoxy putty may last, but my experience has been that it tends to develop gaps after a few years.

Silicone rubber is a mess; don't go there.

Or, you can choose to fix it right, by chiseling it out, and using something like Waterplug. Do **NOT** use some of the low grade cement patches that Lowes and Home Depot sometimes carry!

Here's my Amazon store page: http://astore.amazon.com/poolbooks?node=43
but you should use the pictures as a guide, and check around at other pool stores -- Amazon is not carrying those products directly so delivery and prices may be better elsewhere.