OpenAI’s GPT-3 natural language processing system is their first commercial product. It was officially released in June, 2020 but has had limited availability. The OPENAI website is still showing a waiting list.
Free access to the API will be limited to 100,000 tokens or a three-month time limit whatever comes first. Users can buy two million tokens for $100 per month, or ten million for $400 a month.
Two million tokens can be used to process 3,000 pages of text. All of the writings of Shakespeare would need 1.2 million tokens.
OpenAU trained GPT-3 with an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.
GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as .unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic.