Timeline for How to extract grammar rules from a language (grammar induction?) using a neural network like a LSTM
Current License: CC BY-SA 4.0
29 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 27, 2021 at 23:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Aug 29, 2021 at 22:05 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| May 1, 2021 at 20:07 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Jan 1, 2021 at 20:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Sep 3, 2020 at 19:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Aug 4, 2020 at 18:15 | history | edited | jogloran | CC BY-SA 4.0 |
edited title
|
| Aug 2, 2020 at 3:04 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Apr 4, 2020 at 3:30 | review | Close votes | |||
| Apr 17, 2020 at 16:27 | |||||
| Apr 4, 2020 at 3:02 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Mar 9, 2020 at 6:00 | history | tweeted | twitter.com/StackLinguist/status/1236894702898688000 | ||
| Mar 7, 2020 at 19:31 | comment | added | Jetpack | As far as I know, this isn't a standard task. If you can answer some of the questions in my previous posts, I can figure out what standard task is closest to what you want, and I can try to find a relevant research paper with a public code base. If you have a language that can be modeled with a FSM, you can look at this: datascience.stackexchange.com/questions/12905/…. Sentences in human languages cannot be represented by FSMs, so that's a totally different class of problem. | |
| Mar 6, 2020 at 22:45 | comment | added | Robi Sen | Thank you for your response. The language is a network protocol. I can tag all elements as I said i know the vocab. What I want to do is get the rules of the language, a network protocol in this case, even if I have not seen the specific protocol before. So the project would start with fake protocols where I do know the rules etc to test. The goal though would be to move to protocols that are unknow like those found in malware. | |
| Mar 6, 2020 at 21:47 | comment | added | Jetpack | Do you have anyone who knows a lot about the grammar who can label it. If you wanted to parse English, you would probably start by training a parser on a collection of text that's been carefully labeled by someone who knows the language, such as the Penn Treebank, which consists of one parse tree per sentence. If you don't have a collection of parse trees, what you are making is an unsupervised parser. | |
| Mar 6, 2020 at 21:45 | comment | added | Jetpack | Can you tell us more about the language? Where does it fall on the Chomsky hierarchy? If you don't know what that means, can you tell whether there's a fixed sentence length or maximum sentence length? If sentences can be unbounded, is it because elements can be repeated, or is it because they're nested? Would it make sense to draw a parse tree? NLP parsers usually assume a Context Free Grammar, but your language might be simpler than that. You mention FSMs, which implies that you have a Regular Grammar, and that's simpler than what NLP usually assumes. | |
| Mar 5, 2020 at 2:08 | history | edited | prash♦ | CC BY-SA 4.0 |
appended answer 35397 as supplemental
|
| Mar 5, 2020 at 1:55 | review | Close votes | |||
| Mar 6, 2020 at 20:15 | |||||
| Mar 5, 2020 at 1:37 | history | edited | curiousdannii♦ |
edited tags
|
|
| Mar 5, 2020 at 0:01 | comment | added | Robi Sen | @waveswithhands yes exactly. The language is a made up malware with a made up protocol for some other folks project. I am interested if I can us it, induce the grammar or FSMs, and then print them out as rules or a state diagram. There are a lot of reasons why this would be useful but I'm looking for a simple example in Keras or Tensorflow. What I have seen in scholarly work is very hard for me to follow ;+) | |
| Mar 4, 2020 at 23:55 | comment | added | Robi Sen | I literally have access to terabytes. It's from a malware data protocol training corpus for something different, firewalls and packet inspection. I want to see if I can make a model that can take a sample, let's say a few gigabytes to start, and model the protocol which for all intents and purposes is a language. I've seen a few papers on this topic about using grammar induction but either I could not read them or I did not understand the model | |
| Mar 4, 2020 at 23:49 | comment | added | WavesWashSands | @jick: I interpreted the first sentence as 'It has about 200 words in the lexicon'. If I understand the question correctly, OP is testing the system using this artificial language with a very small lexicon, and can probably generate as large a corpus as he needs using the underlying artificial language. OP, am I right in my interpretation? | |
| Mar 4, 2020 at 23:25 | answer | added | TheLoneDeranger | timeline score: 1 | |
| Mar 4, 2020 at 23:12 | comment | added | jick | No, it's not about how complex the state machines can get. It's about the size of corpus. To do anything interesting with NLP, you need large data: for context, the entire work of Shakespeare is frequently used as a toy dataset (just to demonstrate how the algorithm works). | |
| Mar 4, 2020 at 21:54 | comment | added | Robi Sen | Also I don't want fragments. I really need the rules and syntax. | |
| Mar 4, 2020 at 21:53 | comment | added | Robi Sen | I should say this is a start. Latter will have much more as well as very complex state machines. I think in NLP that's automata? | |
| Mar 4, 2020 at 21:31 | comment | added | jick | If it has only 200 words, I guess its corpus is extremely small, so I don't think NLP will help you: any decently powerful neural network will simply "memorize" the entire corpus (i.e., its output will be made of fragments you already have in the corpus). | |
| Mar 4, 2020 at 19:25 | comment | added | Robi Sen | Anything readable by a human such as if then type rules a state diagram, relationship map. Basically some that says like all sentences start with A except when message Y then sentence starts with b. | |
| Mar 4, 2020 at 19:01 | comment | added | Draconis♦ | What exactly is the output you want? Do you want it to give you, say, a series of CFG rules for the syntax? | |
| Mar 4, 2020 at 18:50 | review | First posts | |||
| Mar 5, 2020 at 2:51 | |||||
| Mar 4, 2020 at 18:47 | history | asked | Robi Sen | CC BY-SA 4.0 |