Super intelligence is less than 3 years away.

Premise

LLMs like GPT-4 are clearly very intelligent, I dare to even say generally intelligent. They can do essentially any task in the environment they have been trained on, a “text world”.

The next most exciting milestone is super intelligence! Where AI knows everything we know about our world, and can create new knowledge and make novel discoveries before we can.

Arguably, the hard part of AI has already been solved: learning.

If the hard part is out of the way, why not just 3 years till super intelligence?

In this blog I outline 2 different advancements in AI that might pave the way to super intelligence. The purpose of this post is to show some AI skeptics that we are closer than they think to a world where AI does most of the hard work for us.

Up first, advancement number 1…

Multimodal Data

We have seen that with more data and with better data, LLMs produce more impressive results. Right now that data has been primarily text. If we can combine image and video data into an LLM like architecture, that AI will have more data and better data to further improve its results.

After less than a year of the release of ChatGPT, we are getting GPT-4 + DALL-E where text and images are being processed by AI. Results speak to how much more capable GPT just became.

The next major milestone in multimodal data will be AI processing real time video. The results? Super intelligence.

Why? With more information about the world, AI’s learning and predictive capabilities will be enhanced, giving them a more nuanced understanding of the world.

Increasing Context + Fine Tuning

With more context about the world, AI can use more information to come to more accurate conclusions.

With fine tuning, AI can start to predict new knowledge in order to unlock new discoveries.

Consider the following example to better understand.

“Research Paper Z” was created by a human. We know that this human used knowledge from research papers A, B, … X, Y, because they cited these papers. Given the human brain and the knowledge from these cited papers, the new knowledge in research paper Z was created!

If a human can create new knowledge, an intelligence computer should be able to do the same.

How might we accomplish this?

Ideally we could fine tune an LLM with input:“Write a research paper that builds on [research papers A-Y].” and output: “[research paper Z]”.

The current, solvable challenge is fitting this input into the context window.

The average research paper is 5k words [source]. Right now GPT-4 can fit ~24k words (max of 32,000 tokens and assuming a 0.75 token/word ratio). Meaning GPT-4 can process about 5 research papers.

The median research papers contains 39 citations [source], even if 2/3’s of the cited papers contain redundant information, that is still 13 research papers of knowledge.

When GPT 3 first launched in 2020, it was able to just process 2048 tokens. currently, Antropic’s version of GPT can process 100,000 tokens, close to a 50x increase in about 3 years. As the context window for AI models increase, so does the ability for AI to process more information and to be fine tuned for new tasks, such as knowledge creation.

A larger context window plus some intelligent fine tuning can get us to super intelligent in the not to distant future.

Conclusion

Achieving super intelligence hinges on two pivotal advancements: enabling AI to process video data and significantly expanding its context window. Considering the rapid progression in these domains, I suggest that we are less than three years away from super intelligence.

Note, I am not claiming anything about achieving robotic super intelligence, something that will unlock even more capabilities about AI being able to sense and operate in the real world.

P.S.

What is the role for humans when super intelligence is reached? People will still be on the search for purpose. They will still look to enjoy life and grow as individuals. They will still want to partake in tasks that “light their souls on fire”. The Residency is a new institution built for 18 year olds, designed to help individuals find and live their most fulfilling life. We are starting with ambitious 18 year olds but have plans to serve anyone who is working towards their best life. When super intelligence is here, we can all live a life we love.