A popular material up here for filling, like BigDave's crushed concrete is called Quarry Product, known to the trades as "QP". QP is dirty large gravel. It's unwashed and has fairly large chunks of stone (1") mixed with dirt. It's significantly less expensive than washed gravel, drains virtually as well, and has a very high friction coefficient which makes it less prone to shifting under the immense weight of a pool. It's really one of the best products for backfilling behind retaining walls.

My base rises about 2 1/2' at the ground's lowest point and using QP and retaining block (and an engineer ) I was able to make the base smaller than the rule of thumb. It hasn't shifted at all in 12 years.

If you REALLY want to ensure it works great, install a French Drain first, underneath it to drain away any water that passes through the QP.