• How to use GPT-4 with Vision for Robotics and Other Applications

    TLDR: There are multiple ways you can use GPT-4 with Vision to power robotics and other systems. I included several sample apps you can download to experiment with including a robot simulator.  Multimodal AI models like GPT-4 with Vision have allowed for entirely new kinds of applications that go far beyond just text comprehension. A…

    Read more

  • Improving GPT-4’s Visual Reasoning with Prompting

    TLDR: How to boost GPT-4 with Vision’s capabilities with a simple prompt addition. A recent paper How Far Are We from Intelligent Visual Deductive Reasoning? points out the limitations in visual reasoning in image models like GPT-4V. Like other related papers, I think the investigators are directionally correct (these models don’t have human-level reasoning and…

    Read more

  • Is the Reversal Curse Real?

    A recent paper The Reversal Curse points out an apparent failure in large large language models like GPT-4. From the abstract: We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form “A is B”, it will not automatically generalize to the reverse direction…

    Read more

  • My Custom Instructions for ChatGPT

    tldr: Custom instructions let you provide ChatGPT with relevant information of your choice about you and how you like to communicate. I’ve use a couple simple tricks to adapt it to my style. I use ChatGPT non-stop for work, fun and exploring. When I write novels it’s a great research helper. When I code it’s…

    Read more

  • How to get GPT-4 to play Wordle (and other prompt tricks)

    tl;dr: One of the tricks to using GPT-4 well is understanding how it sees text and instructions When we started testing GPT-4 internally last summer one of the first tests people would try is getting it to play the game Wordle. While trying to guess a five-letter word should be simple for an advanced language…

    Read more

  • ChatGPT + Code Interpreter = Magic

    tl;dr: OpenAI is testing the ability to run code and use third-party plugins in ChatGPT. OpenAI has announced that we’re developing plugins for ChatGPT that will extend its capabilities. [Link] Plugins range from third-party tools like WolframAlpha and OpenTable, to our browsing plugin and Code Interpreter that can generate code, run code, upload and download…

    Read more

  • Collaborative Creative Writing with OpenAI’s ChatGPT

    tl;dr: You can use OpenAI’s ChatGPT to bounce ideas around and write story outlines Since I got into the field of AI and started working at OpenAI, it’s been interesting to see how things have accelerated. As an author, I’m frequently asked if AI will replace writers altogether. My personal take is that while AI…

    Read more

  • The Easy Guide to Using OpenAI’s Whisper Model to Transcribe Video and Audio

    OpenAI recently released Whisper, an open source automatic speech recognition model that’s incredibly powerful. I’m biased (I’m the Science Communicator for OpenAI), but in my experience it’s better than any system or service I’ve ever used. Best of all, your can use it completely free, either by downloading it to your computer or by running…

    Read more

  • Building games and apps entirely through natural language using OpenAI’s code-davinci model

    TL;DR: OpenAI has a new code generating model that’s improved in a number of ways and can handle nearly two times as much text (4,000 tokens.) I built several small games and applications without touching a single line of code. There are limitations, and coding purely by simple text instructions can stretch your imagination, but…

    Read more

  • Large models like GPT-3 can perform a variety of tasks with little instruction. That said, one of the challenges in working with these models is determining the right way to do something.  GPT-3 has acquired knowledge from its training data as well as another kind of “intelligence” from learning the various relationships between concepts in…

    Read more