Fabric: Streamlined LLM Prompt Management

Pipe Prompts Together Into Complex Workflows

Today I setup Fabric, an open-source framework to manage prompts and perform complex tasks by piping prompts together via command line.

Personally, I’ve been using Notion to manage and publish my prompts.

But, it’s very clunky copy/pasting prompts across apps.

No vim mode.

No version control.

No easy way to iterate on existing prompts.

I want an elegant solution!

Here’s the youtube version of this post:

Why Fabric?

I’ve seen other people’s collections of 1000+ ChatGPT prompts, yet they only use a small percentage of them.

I stumble upon interesting prompts all the time, dumping them in a Discord channel, yet I rarely revisit them.

There must be a better way to integrate LLMs into my everyday workflows.

Daniel Miessler, creator of Fabric, described the pain eloquently:

… the biggest challenge I faced in 2023——which still exists today—is the sheer number of AI prompts out there. We all have prompts that are useful, but it's hard to discover new ones, know if they are good or not, and manage different versions of the ones we like.

One of Fabric's primary features is helping people collect and integrate prompts, which we call Patterns, into various parts of their lives.

Figured it’s finally time to upgrade my own system.

My goal is to test Fabric and see if it makes sense to adopt as my daily “Prompt OS” (just made that up).

It took 5 minutes to setup with python and poetry — promising start!

In Fabric, a pattern is a well-structured prompt in markdown, with clear steps, sent as a system message (rather than user prompt) to an LLM.

Here’s an example of a Fabric pattern to extract wisdom:

And, here’s the full list of Fabric patterns.

Whether or not you need a streamlined prompt management system, go check out some Fabric patterns anyway, as I think they work well out-of-the-box and illustrate how to write clear, structured, and effective prompts.

Create ASCII Art Visualization

First, I test the pattern create_visualization which converts complex ideas into ASCII art!

I feed in an easy example to start with: my own bio.

curl https://www.sabrina.dev/p/who-is-sabrina-ramonov-bio | fabric -m gpt-4o --stream --pattern create_visualization

Here’s the result:

Sabrina Ramonov @ sabrina.dev

It got my age wrong — flattering, but I’m not 30 anymore 😆 

Everything else looks correct, pulled directly from my bio!

Next, I feed in a technical blog post with lots of code, charts, and numbers comparing finetuned LLMs vs ChatGPT-4o.

Much more challenging content…

curl https://mlops.systems/posts/2024-07-01-full-finetuned-model-evaluation.html | fabric -m gpt-4o --stream --pattern create_visualization

Here’s the result:

Sabrina Ramonov @ sabrina.dev

At first glance, the visualization looks promising.

But, the accuracy scores seem to be hallucinated!

I couldn’t find gpt-4o’s accuracy score of 87.5% anywhere in the original blog.

Same with tinyllama-template free — the accuracy score is plain wrong.

This isn’t Fabric’s fault.

But, it serves as a reminder of the current limitations of LLMs and perhaps the urgent need for a QA pattern as the last step in my pipeline.

Extract Wisdom from Youtube Video

Now, let’s test a pattern I’d use often:

Extract interesting ideas and insights from a youtube video.

Fabric has helper apps for Youtube and Whisper (transcription), and I hope more coming soon! All I had to do is input my Youtube API key, paste the Youtube video link, and hit enter:

yt --transcript https://youtu.be/4GOWzuykh1c | fabric --stream --pattern extract_wisdom

I feed in one of my own Youtube videos where I used multimodal ChatGPT-4o to parse neural network architecture diagrams to automatically create new levels for GraphGame.

Here’s the result:

Sabrina Ramonov @ sabrina.dev

Sabrina Ramonov @ sabrina.dev

Super impressed with this output!

Besides spelling my last name wrong 😂 …the pattern’s summary of ideas, insights, references, etc. are spot on.

Remember, this is 100% out-of-the-box. I haven’t customized or improved the Fabric patterns (prompts) in any way.

Write Micro Essay from Youtube Video

Similar to the previous use case, now I want to write a micro essay based on same Youtube video.

This is useful for content creators, for example, to repurpose long-form Youtube videos into short-form media assets for Twitter/X, LinkedIn, TikTok, and Instagram.

yt --transcript https://youtu.be/4GOWzuykh1c | fabric -m gpt-4o --stream --pattern write_micro_essay

Here’s the result:

Sabrina Ramonov @ sabrina.dev

Again, really impressed.

This micro essay sums up my video perfectly. The writing style is decent too. And, I don’t see any occurrences of “delve” 😅 

Make Flashcards from Website

Switching gears to an education and learning example, I use a pattern to create flashcards from my blog post about generative AI agents.

curl https://www.sabrina.dev/p/what-are-generative-ai-agents-definition | fabric -m gpt-4o --stream --pattern to_flashcards

Here’s the result:

Sabrina Ramonov @ sabrina.dev

Wow, totally irrelevant!

Some interesting facts I didn’t know about the Dead Sea, I guess 🤣

I re-run the exact same command:

Sabrina Ramonov @ sabrina.dev

…now it looks good!

No idea what happened in that first run.

I ran the exact same command twice, no edits whatsoever.

I like the flashcards generated by the pattern. It reflects the content in my blog post accurately. It’s a nice quick way to check your understanding after reading a post. To take it further, you could pipe the output into a flashcard generator app in order to use the flashcards for learning purposes.

Find Hidden Message

This last use case is a fun one!

The find_hidden_message pattern assumes a cynical POV:

You are an expert in political propaganda, analysis of hidden messages in conversations and essays, population control through speech and writing, and political narrative creation.

You consume input and cynically evaluate what's being said to find the overt vs. hidden political messages.

Take a step back and think step-by-step about how to evaluate the input and what the true intentions of the speaker are.

I go to HackerNews, running the pattern on a trending post from Andrew Chen:

curl https://andrewchen.substack.com/p/10x-work-versus-1x-work | fabric -m gpt-4o --pattern find_hidden_message

Here’s the result:

Sabrina Ramonov @ sabrina.dev

Sabrina Ramonov @ sabrina.dev

I love the note at the bottom - perhaps all LLMs should print this disclaimer:

"NOTE: This AI is tuned specifically to be cynical and politically-minded. Don't believe everything it says. Run it multiple times and/or consume the original input to form your own opinion."

The “Overt Message” analysis seems accurate, summarizing the key points in Andrew Chen’s hypothesis.

The “Hidden Message” analysis leaves much to be desired, in my opinion — some points seem misconstrued or a stretch too far, such as “Undermine established corporate structures.”

But, the “Cynical Analysis” is hilarious and more cynical than I expected!

"Andrew Chen wants you to believe he is an innovative disruptor that wants you to challenge norms and embrace chaos, but he's actually a self-promoter that wants you to prioritize personal branding and visibility."

Quite an entertaining and thought-provoking pattern!

Last Thoughts

So far, I’m very impressed with Fabric from my first test drive.

The examples I’ve shared only demonstrate 2-step workflows, but I can easily envision using Fabric to pipe together multi-step prompt chains.

There are dozens of patterns I haven’t tried yet, and I highly recommend browsing through them, as I think they’re valuable as standalone prompts even without using the Fabric framework.

I wish it had more helper apps, such as out-of-the-box integrations with social media APIs, GSuite, common productivity tools, etc. This is the main reason I’ve been evaluating Zapier for building shareable AI agentic workflows — integrations are crucial for real-world use cases.

Did I miss anything?

Have ideas or suggestions?

Message me on LinkedIn 👋 

Sabrina Ramonov

P.S. If you’re enjoying my free newsletter, it’d mean the world to me if you share it with others. My newsletter just launched, and every single referral helps. Thank you!