New Feature: Added Vision and Voice Capabilities
I added the voice 🎤 and vision 👀 integration to the ChatGPT Toolkit. You can now speak 🗣 and add screenshots to your chat 📷
15. April 2024
Claude Models are now supported with Vision & Tooling. I might need to rename the toolkit now!
After a break on working on the ChatGPT Toolkit, I sat down and added some new features and bug fixes over the weekend.
Here are the caveats.
I added all Claude models from Anthropic, with vision & tools support. You can also add a baseUrl to Anthropic if you use a third-party provider for the API.
Claude 3 Chat
startChat
You can now specify multiple messages in advance and also take screenshots and insert them into the chat before starting it.
Here is an example:
// Name: OCRTeX
// Group: ChatGPT
import "@johnlindquist/kit";
import { startChat } from "../kenvs/chatGpt/lib/common";
import {
createScreenshotMessage,
userMessage,
imageFileMessage
} from "../kenvs/chatGpt/lib/helper";
const screenshotMessage = await createScreenshotMessage();
await startChat({
model: "gpt-4-vision-preview",
temperature: 1.0,
maxTokens: 4096,
runPromptImmediately: true,
messages: [
screenshotMessage,
imageFileMessage("path/to/imageFile"),
userMessage(`Describe the images.`),
]
});
I might add Mistral Models soon.. 👀
I added the voice 🎤 and vision 👀 integration to the ChatGPT Toolkit. You can now speak 🗣 and add screenshots to your chat 📷
A little tool to control your bluetooth devices without navigating your mouse 🐁
Subscribe if you want to hear from my learnings and get my newest tools. I will never spam you. Pinky promise 🤙