Note: I will use terms such as relationships, expressions, etc. very loosely and interchangeably, for convenience.
If language can be considered beautiful, then so can mathematics. Numbers and equations are the language of our universe; a language beyond any common tongue spoken within or across human borders. It relates every thread in the fabric of existence to every other thread. It is not only a language with which to describe and understand reality; it is the language of reality itself. The limits on what can possibly exist or occur are defined by mathematical relationships. If language can be considered beautiful on a philosophical level, then mathematics is beyond beauty.
On aesthetic beauty: the grace of mathematics is virtually unparalleled. An algebraic equation, to me, appears to dance as I manipulate it, fusing expressions to create the simplest and most elegant form of what I wish to express. Order is found within utter chaos, the unending maelstrom of seemingly unconnected values. Mathematical concepts can be combined or derived from one another, an intricate, infinite web of interrelation.
It is manipulating and unpicking this tangled web that induces creativity and discovery in maths. One can argue either way that mathematics is created, or that mathematics is discovered, but I believe it is a sacred union of both. Creation and discovery become one. A mathematical concept does not exist until it is expressed in a comprehensible manner, for otherwise two expressions are separate and linked only by the components from which they're made. An idea does not have form until it a definite shape is crafted for it; therefore, creativity is required to initially find and test the concept and then to express it. However, it could also be argued that the relationship existed, in some way, before it was expressed, in an abstract sense, as the expressions are related by a specific function or value regardless of whether that specific function/value has been expressed; in other words, the relationship was discovered, uncovered, not created. This simultaneous discovery and creation means we are, at once, uncovering ancient and immutable truths, and crafting new and awe-inspiring connections between the strands of the web. Furthermore... human comprehension of that concept is newly birthed from our minds.
What fundamentally defines beauty in art, however, is not only the skill required, the pleasure of the aesthetics, or the conceptual direction of the artwork. It is, arguably, the emotion it induces. This, I can explain to nobody. If one is not the type to derive pleasure from maths, it is an exercise in masochism.