I’ve been playing around with OpenAI’s GPT-4 API and I wanted to share a quick summary and code sample.

Why do this?

I host my apps on Heroku and love to Test & Experiment with ChatGPT and other OpenAI models. I wanted to be able to stream the results of my API calls to the browser without using turbo or websockets.

For this example, i’ve got a model SocialVideo that keeps track of youtube video transcripts. We’re going to send some transcripts to ChatGPT, have it create a channel description using GPT-4.

Why Shouldn’t you do this?

The fetch request is streaming, which means it keeps one of your rails processes busy until the stream is closed.

Depending on how many USERS you have, this could easily clog your inbound pipes.

Anyways, I like this because it’s the quickest way to work with OpenAI’s API and get results to the browser.

Step 1.

Add the route to channel descriptions.

# routes.rb
resources :channel_descriptions, only: [:index]

Step 2.

Add the controller action to handle the request. The JUICY parts here outlined here:

make your controller LIVE!

Write the chunks as you get the chunks:

# frozen_string_literal: true

class ChannelDescriptionsController < ApplicationController
  include ActionController::Live

  def index
    response.headers['Content-Type'] = 'text/event-stream'
    response.headers["Last-Modified"] = Time.now.httpdate.to_s
    response.headers["X-Accel-Buffering"] = "no"
    messages = [{ role: "system", content: "The User is going to submit some video transcripts/summaries. Our Job is to create a youtube channel description from their samples of work." },]
    samples = current_user.social_videos.limit(5).as_json(only: :transcript, methods: :final_summary)
    samples.each do |sample|
      messages.push({ role: "user", content: "VideoSummary: ```#{sample['final_summary']}```\n\nTranscript:\n```#{sample['transcript']}```" })
    end
    messages.push({ role: "user", content: "Using those samples craft a description of the speaker tone from their transcripts. Next craft a creative youtube channel description in the speaker-tone. \nAnswer in TEXT.\nFormat exactly as ```speaker_tone_description\n\nchannel_description```" })
    # gem 'ruby-openai'
    client = OpenAI::Client.new(access_token: Rails.application.credentials.config[:OPENAI])
    client.chat(
      parameters: {
        model: "gpt-4",
        temperature: 1,
        messages: messages,
        stream: proc do |chunk, _bytesize|
          chunk_content = chunk.dig("choices", 0, "delta", "content")
          response.stream.write(chunk_content) unless chunk_content.nil?
        end
      })

    response.stream.close
  end
end

Step 3.

Add the html & javascript to handle the stream.


<div id="channel-summary">

</div>

async function streamData() {
    const response = await fetch('/channel_descriptions');
    const reader = response.body.getReader();
    while (true) {
        const {done, value} = await reader.read();
        if (done) break;
        const chunkText = new TextDecoder("utf-8").decode(value);
        document.querySelector('#channel-summary').innerHTML += chunkText;
    }
}

// obviously you can do this however you want, but this is a quick example.
setTimeout(streamData, 3000);

Hopefully this will be helpful for anyone wanting to play with these tools!