This is a really stand-out resource in my opinion because it helps to clear out the fog / hype that is so intensely generated in newspapers, social media and culture at large: https://paperswithcode.com/
Although I myself am deeply fascinated by this topic and have found applications of GPT-4 - tools that are literally coming out a few days ago - powerful and a qualitative change in how I get things done, I have also wanted myself to try to take a longer historical view (as others in public discourse have also stated) of to what extent are some of the capabilities of these LLMs as tools actually inimitable? Even if observing them is remarkable for its newness, one often takes a step back to refresh their stance, in considering both that with time one can often feel the tools do not (as of right now) perform as well as it seemed they did, but more importantly, that even if they did, there are ways people were still able to do the same thing in the past - without the use, need, or help of AI assistants, powerful as they can be. I find a slightly neglected point in mainstream discourse to be considering how societies shift in their equilibria as opposed to monotonically progress - while some capabilities - the atomic bomb, as an obvious example - are obviously orders of magnitude greater, in some specific capability, than anything comparable in the past, if you look at the total system of the world, society as a whole, we may find that as some form, pattern, meme, structure, infrastructure, tool, activity, idea, etc., takes on more centrality, it actually clips away other past specialized forms of knowledge, organization, techniques, even surprisingly efficient and/or effective ways of doing things, patterns of social organization, not because it is univalently better, but just because it seems to have picked up huge societal, gravitational pull, that feedback loops lead to increased prominence and eventually dependence on that thing, as increasingly uncommon ways of doing things getting scooted out of the limelight.
That website is really valuable for giving a bit more of a sober (I think) research-oriented perspective on this - a couple strong first impressions, to my mind, are:
- ChatGPT isn’t that new. This is just a hot flash in its development when it breached the consumer sphere. LLMs have been in development, I guess, for years, at minimum. In fact, Geoffrey Hinton said in a recent interview that he hadn’t been surprised or interested (I think) when ChatGPT was unveiled, because, he said, he had already been seeing tools like that in research, and presumably some closed doors in industry, for a while now.
- ChatGPT is far, far from a standalone technical entity or invention. There are myriad large language models being published, studied, and analyzed, as the many papers on the website seem to say.
- Most interestingly, it might be the case that GPT-4 is not actually distinctively top-performing. While I am not sure, it really seems like due to a much more calculated approach and design one would get in science, combining many aspects of the engineering like model architecture, model size, training data type and quality, training data size, instruction fine-tuning, alignment, and promoting, we can see on those benchmarks on that page, that there is actually a wide diversity of LLM technologies or systems which have applications on many different tasks.
Being an amateur, I welcome anyone that would care to correct me where I am wrong, because I made intuitive claims here without much rigorous citation or justification.
To answer the question a bit.
This is a list of QA benchmarks. You can click on each one to see how a variety of models / systems performed: https://paperswithcode.com/task/question-answering
Apparently, currently one leading performance of the GPT-type system/architecture whatever, was 80% on a set of questions: https://paperswithcode.com/paper/biogpt-generative-pre-trained-transformer-for
Of course, since then, GPT-4 has has some remarkable results, like acing a BAR exam (I think): https://arxiv.org/pdf/2303.08774.pdf
Regardless, this topic still seems of central interest - how dumb or smart are these systems, anyway? Recent talk of emergent properties may have its own critique in turn. It’s a fascinating question. https://paperswithcode.com/paper/are-emergent-abilities-of-large-language
Luckily, there is a recent academic review trying to survey what is known about the uses of ChatGPT, as it currently stands: https://paperswithcode.com/paper/harnessing-the-power-of-llms-in-practice-a
The sloppiness of this response’s writing is sure proof that it was written by a human, and not ChatGPT.