GPT-4 is coming, but currently the focus is on coding and that's also where the available compute is going. GPT-4 will be a text model (as opposed to multi-modal). It will not be much bigger than GPT-3, but it will use way more compute. People will be surprised how much better you can make models without making them bigger.

GPT4 should have 20X GPT3 compute. GPT4 should have 10X parameters. GPT 5 should have 10X-20X of GPT4 compute in 2025. GPT5 will have 200-400X compute of GPT3 and 100X parameters of GPT3.

The progress will come from OpenAI working on all aspects of GPT (data, algos, fine-tuning, etc.). GPT-4 will likely be able to work with longer context and be trained with a different loss function – OpenAI has "line of sight" for this.

GPT-5 might be able to pass the Turing test. But this will probably not be worth the effort.

GPT-4 will likely be released in the second half of 2023. GPT-5 should be expected at the end of 2024 or in 2025.

100 trillion parameter model won't be GPT-4 and is far off. They are getting much more performance out of smaller models. Maybe they will never need such a big model.

Less Wrong reviewed the GPT-4 predictions including information from Matthew Barnett.