I think I'd bury the outside part of the ladder. Even if you build steps with the pavers you've got to be able to climb back out of the pool once you're in so you'll still need the ladder (or steps on the inside as well, I suppose).

If you bury part of it, though; it's likely to corrode faster.