Is it a problem if there are ants under the pavers?

Yes. Although most of the time ants are harmless, they have the ability to move large amounts of sand, and can cause pavers to sink. If you decide to use a commercial ant killer, first test it on a small area to make sure it doesn't discolor the pavers.