Sorry for the slow reply Jim. Yes you typically use some type of Thin set which is nothing more than mortar fortified with a polymer glue. This is what should actually hold the tiles in place. The reason you need the glue to be water proof as some thin set will break down in water due to the type of adhesive blend. Typically in pools they use the same material you would use to set shower or bathtub tiles. The grout should also be waterproof, and I think the HD brand is Polyblend.

You are correct the grout is brittle and has little in the way of adhesive properties, more for sealing, water proofing and aesthetics.

Glad it worked
Thanks
D