To me, and I think many other outsiders, putting a lot of emphasis on the equivalent of grammars-language-automata looks like mathematical naivety. I don't say this to be rude but because you (and Chomsky) claim to be able to interpret the implications of these mathematical results, but I don't think you are doing so correctly. Grammars look like a human (mathematical) invention and not some deep mathematical structure, and these results appear shallow. In the broader context, lots of mechanisms are able to do Turing complete computation.
This doesn't just apply to grammars. There is a huge array of formalisms (e.g. logics, type systems) out there and most just look like the result of someone saying "what if I did this?".
>> I don't say this to be rude but because you (and Chomsky) claim to be able to
interpret the implications of these mathematical results, but I don't think
you are doing so correctly.
It's alright- if I'm being naive, I'm being naive.
But- what am I missing? You're saying we're doing it wrong- how? For me the
intuition that infinite generative ability flows naturally from unbounded
recursion, like an egg from a hen's bottom, is kind of obvious. Is it naive? I
guess it's empirical, for me at least.
Also, btw, I was introduced to the idea of language equivalence through
Hopcroft and Ullman, so from the point of view of computer science, where it's
been very useful, in practical terms. I guess if you're coming from a
mathematical or theoretical physics background it might sound a bit silly, but
it's allowed us to make a lot of progress, for instance to create a few
thousand different architectures and languages... but maybe I shouldn't be
bringing that up as progress...
Anyway, I don't know- how would you interpret the observation correctly? Where
are we going wrong?
Chomsky's reasoning is that the arbitrary complexity (recursive structure) of human language implies some sort of low level computational engine to do the relevant computations. I don't have an opinion either way. We can see that LSTMs can emulate this kind of logic, but they also make mistakes. Also, I'm not sure that human reasoning is as logical as it might seem. E.g. I read somewhere (lost the reference) that the earliest languages may have lacked the ability to arbitrarily nest clauses. So maybe humans only emulate logical thinking.
We can see that LSTMs can emulate this kind of logic, but they also make mistakes.
Humans can also make mistakes when processing language. We're still better than LSTM's, but I'm not sure we can claim a qualitative difference.
Furthermore, even though we can process sentences with very deep embedding like "The rat the cat the dog bit chased escaped", my intuition is that we are not using our normal language processing systems for that. When I read that sentence, I just fail to process it and then I invoke my logic systems to try to determine the structure and decode it, in a way that feels totally different from processing a normal sentence (I'm not understanding it in real time, in a natural way, but rather solving a small puzzle). So I personally don't find the Chomskyan arguments based on that kind of corner cases very convincing.
That is indeed a contrived example of recursion, but recursion (in the sense of embedding) can be much simpler and easier to parse. For example:
"John, my friend from high school, who married your cousin, Mary, is coming over for dinner".
This sort of embedding is what makes human language infinite in scope- you can keep embedding sub-sentences for ever, and so you can produce new utterances forever.
This ability to infinitely extend and recombine the meaning of utterances is what gives human language its expressive power, and what is absent from animal languages, so far as we know.
Examples like that are parsable because they are similar to what we would call, in programming, tail recursion (i.e., recursion that doesn't really need recursion). It's true that you can embed an infinite number of subsentences ("John, my friend from high school, who married your cousin, Mary, who had an affair with the bartender, Jack, who hated his sister, Lisa, who was a fan of Lady Gaga, is coming over for dinner") but you only need two "stack frames", one to remember John and the other for the rest.
The middle part is basically equivalent to saying "Mary had an affair with the bartender, Jack. Jack hated his sister, Lisa. Lisa was a fan of Lady Gaga". My intuition is that it's parsed basically as separate sentences. Once you finish one of them you can just forget it, you only need to remember John (as there is more information about him in the end). Sentences where you need to remember more elements (i.e., you actually need unbounded recursion) become unparsable in real time as my previous example.
Of course, I don't have scientific evidence to back the things I'm saying, it's just intuition, but the same can be said of the Chomskyan theories.
As a smart person, I see things a different way. There are certain hobbies that are only appeal to smart people, like pure math. But there are a lot of things that appeal to a broad range of people, like sport, arts, music and literature. I think it's very good to get involved in these things because they bring balance and a way to related to other people. In fact I created a "no programming related hobbies" rule for myself.
You would be amazed how much you can relate to other people, and how little intelligence is an issue, once you have common interests. I remember in my Judo club (which was open to the public but based at the university) a guy who wasn't attending the university commenting that he kept forgetting that a lot of the people in the club were really smart and accomplished (it came up because another guy complained about getting hit in the face, saying he had to present at a conference the next day).
That's not to say intelligence isn't an advantage in these other areas of life. But it's not a big deal or something that creates distance from others. People will appreciate your intelligence just as you appreciate their spontaneity, athleticism, kindness or any other qualities.
I like that attitude and have found it to be true in my experience.
Sports are good for this, you interact with people from many different occupations and age groups.
It's cool to see people from different walks of life solving the problems presented by the challenges of the sport in their own ways according to their own attributes.
This doesn't just apply to grammars. There is a huge array of formalisms (e.g. logics, type systems) out there and most just look like the result of someone saying "what if I did this?".