The difference isn't just mathematization or complexity.
Physicists are looking for general regularities, patterns, laws &c and
geologists (making use of some of those laws) are trying to understand
events which are, at least in part, contingent. The Dirac equation
tells that particles with electron mass & positive charge should exist -
somewhere. But it can't tell us when a particular cosmic ray shower
will yield a particular number of positrons.
We physicists like to think - & with some truth - that we are
the scientists who deal with the world at its most basic level. But we
make it easy by leaving out the contingency of events in the real world.
The distinction is of some theological significance.
George L. Murphy