potato face
CryingPotato

Just Box It (Typescript Edition)

#tailor #javascript

GPT-4 is stupid. There, I said it. (It’s either that or I’m really really bad at prompting, but come on). It’s incredible to me that there is no way to reliably generate structured JSON like JSONformer lets you do with open models. Calling functions is such a core part of any useful workflow that goes beyond generating text. In fairness, this seems much more difficult to do at OpenAI scale since you’re serving models to 1000s of people. Still I’m shocked at how inflexible the whole setup is today.

Anyway, here are a couple of useful tricks I’ve learned:

Let’s put this all together with an example:

import { Type, type Static } from '@sinclair/typebox';
// ...
const ReACTSchema = Type.Object({
  thought: Type.String(),
  action: StringEnum(Object.values(ActionNames)), // ActionNames is the Typescript enum of actions.
});
type ReACTSchema = Static;
// ...
// Now the function call parameter looks like this. So clean!
  {
    type: 'function',
    function: {
      name: ToolNames.ReACT,
      description: 'Think and select an action.',
      // You need a Strict type to remove all the Typebox specific metadata.
      parameters: Type.Strict(ReACTSchema),
    },
  },

You can even validate the params that OpenAI returns to you oh-so easily.

import { Value } from '@sinclair/typebox/value'
// ...
const res = await openai.chat.completions.create(...)
const functionArgs = res.choices[0].message.tool_calls[0].function.arguments
const toolParamsParsed: Static = JSON.parse(functionArgs);
// toolParamsSchema can be ReACTSchema for example.
if (!Value.Check(toolParamSchema, toolParamsParsed)) {
throw `Tool params ${JSON.stringify(toolParamsParsed, null, 2)}\n did not match schema ${JSON.stringify(toolParamSchema, null, 2)}`;
}

You might have noticed the StringEnum object (defined here) I’m using above. I discovered that StringEnum worked better than TypeBox’s default T.Enum for reliable action generation. StringEnum uses the enum field of JSONSchema whereas T.Enum uses anyOf: perhaps GPT-4 is more optimized for the former. As a final bonus to this long log, here’s how you define validation on StringEnum:

import { TypeRegistry, Kind } from '@sinclair/typebox';

const StringEnum = (values: [...T]) => Type.Unsafe({
  type: 'string',
  enum: values,
  [Kind]: 'StringEnum'
})

TypeRegistry.Set('StringEnum', (schema, value) => {
  // @ts-ignore
  if (schema.type === 'string' && schema.enum && schema.enum.includes(value)) {
    return true;
  }
  return false;
});