llmkit

Prompt Versioning in LLMKit

Alright, let’s dive into Prompt Versioning! This section will explain how prompt versioning works in LLMKit, with a focus on the user interface (UI). To make it visual and practical, I’ve included placeholders for UI screenshots to show key features like what versioning looks like, how to switch to older versions, and the "version diff" view. Below is a complete guide to get you started.

Introduction to Prompt Versioning

Prompt versioning is a powerful feature in LLMKit that lets you track changes to your prompts over time. Think of it as version control for your prompts—each time you tweak a prompt significantly, you can save it as a new version. This gives you several benefits:

  • Experiment without worry: Try new ideas without losing your previous setups.
  • Compare performance: See how different versions stack up.
  • Revert if needed: Go back to an earlier version if something doesn’t work out.
  • Collaborate smoothly: Share specific versions with your team.

Let’s explore how this works in the LLMKit UI and how it ties into evaluations.

Managing Versions in the UI

The LLMKit interface makes versioning easy to use. Here’s how you can manage your prompt versions step-by-step.

Viewing the Current Version

When you open a prompt, the current version number is clearly shown. This helps you instantly know which version you’re looking at.

SCREENSHOT: Prompt detail view with version number

Creating a New Version

Want to make a change? Just edit the prompt and hit save. LLMKit will automatically bump up the version number and store the new configuration as a fresh version.

SCREENSHOT: Edit prompt form with save button

Switching to Older Versions

Switching between versions is a breeze. Use the version dropdown menu to pick any previous version, and the UI will update to show that version’s details—like the system message, user message template, and settings.

SCREENSHOT: Version dropdown menu

Seeing the "Version Diff"

One standout feature is the version diff view. This shows you exactly what’s changed between two versions, with highlights for additions, deletions, and edits in the prompt text. It’s a quick way to track your prompt’s evolution.

SCREENSHOT: Version diff view

Versioning and Evaluations

Prompt versioning doesn’t stop at tracking changes—it also connects to LLMKit’s evaluation system. When you set up an evaluation, you can pick which version of the prompt to test. This lets you:

  • Check performance metrics across versions.
  • Spot improvements or setbacks from your changes.
  • Decide which version works best, backed by data.

SCREENSHOT: Evaluation setup with version selection

This integration makes it simple to optimize your prompts based on real results.

Why It Matters

Prompt versioning gives you the freedom to iterate and experiment while keeping everything organized. Whether you’re tweaking prompts solo or working with a team, the UI’s visual tools—like the version dropdown and diff view—make it easy to manage your prompt history.

With placeholders for screenshots, you’ll see exactly how this looks in action once they’re added. Let me know if you’d like more details!