Mathematician: One important thing to realize about mathematics is that it was primarily created for practical purposes. For example, numbers were likely used in the beginning to count possessions, multiplication for trade, and geometry to measure plots of land (or some similar purposes). Mathematicians and scientists use math to model the world by constructing mathematical objects that capture important properties of physical things (while ignoring those properties that are not relevant for the investigation). Hence, it isn’t as though math just happens to work well for analyzing the world we live in, rather, it was specifically designed for that purpose. If our original mathematical objects had failed to capture important properties of real objects, they surely would have been discarded and replaced with ones that would be more useful. To give one example, if the operation of addition did not so closely model so many physical phenomena (e.g. if I have two objects in one group and I combine them with three objects in another group, then my new group has five objects, which is mimicked by 2+3=5) then it might not be considered a basic mathematical operation like it is today.
Once the basic objects of math were introduced (for their practical uses), it was then possible for people to generalize these objects, find connections between them, and prove theorems about them. For example, once we have integers (for counting) we can ask the question whether there is any largest integer. Once we have addition, we can ask the question whether a + (b + c) = (a + b) + c. Once we have division, we can introduce the idea of prime numbers. Once we have exponents and real numbers, we can introduce polynomials, and attempts to find the roots of polynomials will inevitably lead to the introduction of imaginary numbers. Hence, from the basic useful mathematical objects, a whole complicated structure follows which contains many new ideas relating to or emanating from the original ones.
Long after most of the basic objects of math were created, attempts were made to axiomatize the subject (i.e. provide a small set of basic axioms from which the rest of math can be derived), but math was not developed from these axioms. Quite to the contrary, these axioms were developed from the already existing useful mathematical system, and hence the axioms somehow inherently have built into them the usefulness of the entire mathematical structure. By altering these axioms mathematicians can (and have) developed different versions of mathematics. One thing that is special about the version of mathematics that we are used to is that it allows for creating a staggering variety of useful models. When the basic axioms are fundamentally altered, this is not necessarily the case.
A more difficult question than why math works so well at modeling the world, is the question of why math that is developed for one purpose (or, sometimes no purpose at all except theoretical interest) ends up being so useful for other purposes, but this is a subject that deserves a post of its own.