I've read about different theories but now I'm kind of confused between the different theories and the differences between structural case, non-structural, lexical/morphological case, inherent case, and abstract case. What is the state of the art in the case theory?
You're asking a number of different questions here, that range from simple description of surface features to theory-internal ideas from specific frameworks (and asking about two different frameworks), so I think you may want more of an overview rather than direct answers to your specific question. If you don't want to read a textbook, the Wikipedia articles on case and case role look like a good place to start.
Also, most of these terms have multiple conflicting meanings. Lexical case assignment, in the lexical-vs.-inherent sense, is non-structural in the structural-vs.-non-structural sense, but it's part of the structural case subsystem in the structural-vs.-morphological sense, not to mention that lexical-vs.-inherent is a distinction that's only made in structural case theories in the other structural-vs.-non-structural sense. Meanwhile, the morphological case subsystem connected to the structural case subsystem may be either lexical or morphological in the lexical-vs.-morphological sense—which of course has nothing to do with whether the case assignment was lexical or inherent or structural. Confused? You won't be, after this week's episode of Syntax.
Anyway, this answer will cover many, but not all, of your separate questions. (The one in the title, in particular, I don't have an answer to.)
Lexical vs. morphological case: In Japanese, the NP that's the direct object of the verb is followed by the separate word 'o'; that's lexical, because it's using a word. In Latin, the NP that's the direct object of the verb instead has the accusative version of the N, like 'dominum' instead of 'dominus', as its head; that's morphological, because it's (depending on your theory) either changing a word, or selecting a different form of a word.1
Structural vs. non-structural case: In most theories within Chomskyan transformational grammar, case assignments happen automatically in certain syntactic positions, and that's what accounts for at least some case assignment (and is also one of the major things that motivates and/or constrains movement2). For example, the V in a VP always marks its complement (the direct object) with the accusative case. This is structural case assignment, and theories that rely on it are structural case theories.3 Theories that have no structural case assignment are non-structural case theories.4
Structural vs. non-structural case: Even within (some) structural case theories, some cases aren't determined (entirely) by the structural position. For example, in many languages, ditransitive sentences ('I gave her the book') always assign dative case to the indirect object—but in some Germanic languages, they assign dative case if the indirect object is a semantic goal, accusative case otherwise. Since the case depends on the thematic role, it's non-structural. Similarly, in some Germanic languages, decausative sentences ('The plank broke' vs. 'I broke the plank') assign accusative case to the subject if it's a patient, instead of the usual (nominative) structural case assignment for subjects.5
Lexical vs. inherent case: Most non-structural case assignment is regular: the structure and semantic role together determine the case. This is inherent case. But that isn't always true. In Icelandic, decausative verbs usually assign accusative case to patient subjects—but some specific verbs assign other cases, like 'Bátnum hvolfdi' ('The boat-DATIVE capsized'). This distinction has nothing to do with the position, so it's not structural, or with the role (the subject of 'capsized' has the same patient role as the subject of other decausatives like 'sink' and 'drift'), so it's not inherent. It's an irregular special case that has to be memorized as part of the verb 'capsize'—hence it's lexical.
Structural vs. morphological case: Take the Latin sentence 'Video dominum'. The syntactic system only cares that 'dominus' is marked with accusative case. All the stuff the morphological system has to do (see that 'dominus' is second-declension, look up the second-declension accusative ending '-um', attach it to the stem 'domin', and do whatever fixup is necessary to turn that into the complete word 'dominum') is morphological case; the structural case is just 'accusative'.
Abstract vs. concrete case: The idea that every language has a structural case system is abstract case theory. English nouns need to get marked the same way as Latin, Swedish, or Japanese nouns, and V always marks its complement with accusative case, and that's why English direct objects are VP complements. Within abstract case theory, abstract case is also sometimes used to distinguish this assigned case from the concrete case representation. This is not the same as the structural vs. morphological distinction above—concrete case is still in syntax (so structural), but it's the thing that connects up to morphology.6
Covert vs. overt case: The obvious distinction between case you can't see and case that you can. In theories without abstract case, there's no covert case; e.g., English nouns just don't get case-marked. But in abstract case theories, they do get case-marked, it's just that the case is covert. Within some theories, there's even a distinction between covert case vs. overt (concrete) case that just gets spelled out as zero at the phonological level.
Case vs. case: Case with a capital "C" means the marking of dependent nouns (purely) by syntax. It also means the system that defines such marking. This term is used only within the GB and MP frameworks, where it means essentially the same thing as structural (as opposed to morphological) case, and also contrasts with (semantic) case roles: Capital-C Case realizes case roles, and licenses morphological case.
1. This implies that word-vs.-affix and NP-vs.-N distinction go together, but that turns out not to be true. For example, Basque suffixes the last noun in the NP, not the head, but it's clearly a suffix. You can even see this in English (if you consider the English possessive to be genitive, which is contentious): 'John from over the hill's sister/*John's from over the hill sister'. Things get even more fun with, e.g., Finnish, which marks the determiner, the head noun, and immediate adjectives of most but not all types, which is clearly NP-marking (or DP-marking), but of a very different kind from Japanese or Basque. One way to handle this is to say that all languages mark the NP, but different languages have different case concord systems within the NP, so, e.g., Latin ends up marking the head, Basque the last noun, and Finnish a bunch of components. Another way is to say that Finnish really is just marking the head like Latin, but it has a case concord system that then requires the other parts to agree with the noun.
2. The core idea is that every noun needs to get all of its features marked somehow, and it does that by moving through positions that can structurally mark them. There may be different ways of doing that, and that's how you end up with (in one set of theories) two different S-structures with the same D-structure.
3. This system connects up closely with the semantic theory of case roles. For example, the transitive verb 'break' has a patient that it needs to get mapped to accusative case in simple active transitive statements, which is why it ends up as the direct object. The details of how case roles and cases connect up are one of the major things that motivate different theories, so it's hard to say more in theory-neutral terms.
4. Most non-transformational theories are non-structural in this sense. Some of them end up realizing the exact same relationships as constraints in the opposite direction, but those constraints still aren't usually called structural case systems.
5. There are tests that can distinguish between structural and non-structural case assignments based on, e.g., whether the case is retained when the NP moves—or, for non-movement-based theories, based on correspondence between two structures that license the same NP in different locations, or whatever. So, these same terms are sometimes used even in theories that aren't based on a structural case mechanism. And even in theories where all case is structural, they still have to deal with the same two classes; they just explain apparent non-structural case in terms of structural case—e.g., a decausative's patient subject, unlike other intransitive subjects, is originally an object, gets marked there, and then moves to the subject position, and it's this first movement that makes it act differently on any subsequent moves.
6. In a theory which trivially maps abstract cases to concrete cases, this may seem pointless, but not all of them do. For example, you can arguably simplify the mapping of semantic case roles to syntactic cases by suggesting that all languages have the same three major abstract cases: ERG, ACC, and NOM. But how does that match up with the fact that nearly all languages have only two major cases, and different languages have different pairs? In Latin, ERG and NOM map to the concrete case nominative, ACC to accusative; in Basque, ERG maps to ergative and NOM and ACC to absolutive. It's the concrete cases that the morphological system sees.