Hey everyone! 👋
I’m excited to share something we’ve been working on for a while: Prompt Inspector 🕵️♂️ – a browser-like inspect tool for debugging and understanding your prompts for large language models (LLMs). 🧑💻
🧠 Why we Built This
A few months ago, we were deep into a project that analyzed GitHub repos using LLMs. Our prompts were getting longer and more complex, and we kept running into the same problem:
🤔 Which part of my prompt was actually influencing the output?
It felt like we were flying blind, tweaking random lines and hoping for the best. It reminded me of the early days of web development-before browser devtools made debugging HTML and CSS so much easier. 🕸️
That’s when it hit us:
💡 Why isn’t there an “Inspect Element” for prompts?
🛠️ What Is Prompt Inspector?
Prompt Inspector is our answer to that question. Think of it as the “browser inspect tab” for AI prompts:
- Paste your prompt on one side. ✍️
- See the LLM’s output on the other. 👀
- Click on any part of the output to instantly highlight the section of your prompt that influenced it. 🎯
- Visualize and debug your prompts with clarity. 🖼️
It’s a simple idea, but it’s already saved us hours of guesswork. ⏳
⚙️ How Does It Work?
This version is a quick and dirty prototype that runs another LLM check on top of the output (we are currently experimenting with attention maps and looking into more rigorous methods like Integrated Gradients to improve accuracy) to map output tokens back to their prompt origins.
You get a visual, interactive way to see how your instructions shape the model’s response. 🔄
- Open Source: We are planning to open source Prompt Inspector soon. I want the community’s help to make it even better! 🌍
- Selective Output Editing: I often wish I could tweak just one part of the output while keeping the rest unchanged. This feature is on my roadmap. ✂️
- Better Attribution: We are exploring advanced attribution methods for more accurate mappings. 🧬
- Your Ideas: I’d love to hear how you’d use this tool, what features you’d want, and any feedback or naming ideas you have. 💬
🧪 Try It Out
Here’s the live demo and I’ll share the repo link soon (lemme know in the comments if you’d like a sneak peek into the current repo). 👇
Prompt engineering is becoming a core skill for anyone working with AI. But right now, it’s a lot of trial and error. I hope Prompt Inspector makes it a little less mysterious-and a lot more fun. 🎉
If you’ve ever wished you could “inspect” your prompts like you inspect web pages, this one’s for you and it’s free to use. 🆓
Stay tuned for the open source release!
I’d love your feedback, feature requests, and thoughts in the comments. 💡
Thanks for reading, and happy proompting! 🚀
