I learned of connections between linguistics and category theory when I'm learning the application of category theory in quantum field theory. Being aware that axiomatic set theory (logics) is utilized in linguistic research, I want to know more mathematical tools in linguistics.
Nice question, I think this is good to ask for linguistic theory in general, because people who are not so familiar with linguistic research often find this hard to imagine.
First of all, logic in general is essential in formal semantics. Using propositional logic, predicate logic, set theory and tools like lambda calculus, functions and type theory, formal natural language semantics tries to account for meaning in a mathematical way: formalizing natural language statements (using first-order logic, lambda expressions or set-theoretical notation), assigning general meaning patterns to certain types of expressions (this is where semantics interfaces with syntax), representing ambiguity (such as scope ambiguity in expressions involving quantifiers or structural ambiguity arising from operators), making claims about logical inferences (such as implication and entailment), or discussing the concept of intensionality and modality making use of possible worlds - all under the assumption that meaning can (to some extend) be accounted for by logical statements and truth values.
In case you didn't already subsume this under logic, set theory is very important for formal semantics as well, where it is used to describe the nature of certain types of expresions and how they interact with each other. (As an example, the idea of so-called generalized quantifiers is that quantified expressions like "all women", "some children", "no man" are sets of sets of entities that then combine in specific patterns with with other types of elements in a sentence; additionally for a lot of purposes you want to make use of sets of propositions or sets of possible worlds, ... and some more things).
Besides logic, the most important field of mathematics that is used in linguistics is probably statistics.
This is especially important in a wide range of applications in computational linguistics, such as statistical NLP (parsing, POS tagging, ..) and machine learning, but also non-computational areas like quantitative approaches to linguistics (e.g. determining word frequency distributions).
Statistical methods are also crucial for evaluating results of studies (e.g. psycholinguistic experiments, or quantitative typological research).
There are uses in purely theoretically directed research too: For example, there is a trend towards using probabilistic models and Bayesian inference to address problems in pragmatics, sociolinguistics or historical linguistics.
In being related to computer science, computational linguistics requires mathematical models of issues like computability and algorithm complexity in basic discussions about programming.
Of course, there is also some underlying math in the computers and programming languages themselves, sometimes less obvious at the surface, sometimes more, like in so-called logic programming languages such as Prolog which is sometimes used for various applications of natural language processing (most prominently parsing).
For phonetics, mathematical components such as Fourier and related analyses (as mentioned by user6726) in physics concerning sound waves, frequency spectrums and related measurable phenomena of interest also come into play.
Fourier analysis and various statistical methods also play an important role for the analysis of EEGs as used in psycho- and neurolinguistics.
With functions being mathematical concepts as well, this has applications in very many fields, like - again - formal semantics, pretty much anything in computational linguistics (both in theoretical computational linguistics, programming and of course statistics), and also plays more marginal, but still indispensable roles in related fields of study.
Speaking about which, certain theories that originally have their roots in mathematics are occassionally exploited in linguistics.
An example I can think of is that game theory, which was mostly developed by mathematicians and has applications also in completely different areas like economics, can be used to neatly model natural-language phenomena in pragmatics or language evolution.
Simple algebra is found almost everywhere when e.g. talking about the combinatorial possibilities of inflectional paradigms given certain morphological patterns.
Since this was mentioned in the question, category theory has connections to linguistics as well, more on that in the next paragraph.
You might also talk about mathematical influences when it comes to formal representations in certain fields of linguistic, like syntax, grammar and parsing: Precise definitions of grammar formalisms, formal statements about trees, derivations and inference rules, etc. (This is where category theory comes into play: A syntactic analysis is represented by a graph with objects (constituents) and relations between these objects, syntactic categories and functions, terminal and non-terminal nodes and interactions between constituents that can be described as mathematically definable operations, e.g. with lambda calculus.) Formal representations are also made when accurately capturing grammatical rules such as phonological ones, building models for all kinds of things (most of them already mentioned above, e.g. language evolution or cognitive processes), and of course again formal semantics.
Apart from more complex models, you will frequently find a little math (with quantifiers, set relations and similar basic concepts) in formal definitions within the scope of pretty much any linguistic subdiscipline that is in need of precise definitions, like theoretical syntax, formal pragmatics or algorithms in computational linguistics.
This is obviously closely connected to logic and general combinatorics, since you want to create sound and provable (and, from a scientific point of view just as important, disprovable) theories and give as precise as possible descriptions, definitions, models with all their objects and relations, logical axioms and inferences about the linguistic entities and phenomena you want to explain.
So, as you point out, mathematics and especially logic are indeed essential in linguistics.
Of course, there are areas in linguistics where you don't need that much maths, and "real" mathematicians might laugh at what linguists call mathematics - I remember a passage in Daniel Kehlmann's novel Measuring the world (full reference here where I took the literal translation from) where the fictional Gauss says something like "Linguistics? This is something for people with the pedantry, but not the intelligence, for mathematics. People who invent their own scanty logics." ;)
But in fact many people involved in academic linguistic research have some background in mathematics or philosophy which I think is not a coincidence.
I gave a long answer to a closely related question here: How is category theory applied in linguistics?
The answer there talks mostly about category theory and quantum mechanics, so is much narrower in scope. But still perhaps its an adequate partial answer for this question.