Here’s something that quietly frustrated me for months.
I’d build a Claude Code skill.
It worked. I’d use it. Then slowly… the outputs got inconsistent.
Sometimes brilliant. Sometimes off.
And every time, I’d do the same thing.
I’d go back into the skill file. Tweak a line. Add a note. Try again.
Manual. Reactive. Slow.
Here’s the thing…I knew Claude Code was capable of more than this.
I just didn’t have a system for making the skill better on its own.
No feedback loop. No automation. No compounding.
Just me, manually patching things after the fact, hoping the next output would be cleaner.
Do you do this… I have seen many, including myself.
If you’ve been building Claude Code skills for your content workflows…your SEO writer, your newsletter writer…you’ve felt this.
The skill works. But it doesn’t improve.
Until now.
Because Andrej Karpathy just released something that changes this entirely.
And once I saw what it could do for my workflow? I had to break it down for you.
Let’s get into it. 👇
But first — a quick word from this week’s sponsor.
You’re already driving buying decisions. You’re just not getting paid for them.
Here’s the truth most creators don’t want to sit with:
Every time someone reads your content, clicks away, and buys something online…you earned nothing.
They found the product because of you.
You just never saw the commission.
Stay22’s Nova fixes that.
One script. That’s it.
Nova detects when your readers show real buying intent…and quietly surfaces a relevant retail offer from brands like Walmart, Target, or Best Buy.
No pop-ups that wreck your reader experience. No redirects that pull people off your page.
They stay. They buy. You earn.
Stay22 has already powered over $1 billion in purchases across 5,000+ publishers. They built this for the travel industry…one of the most competitive affiliate spaces on the internet. Now it’s available for content creators like us.
This is passive affiliate revenue from content you’ve already published.
No new workflow. No extra work.
Let’s continue…
Here is What Just Happened…
Andrej Karpathy released a GitHub repo called autoresearch.
If you don’t know who he is…he’s one of the founding members of OpenAI, former Head of AI at Tesla, and the person who coined the term “vibe coding.”
He released this thing quietly.
630 lines of Python.
One markdown file.
42,000 GitHub stars in the first week.
Fortune called the underlying method “The Karpathy Loop.”
Here’s what he built it for originally: machine learning research.
He gave an AI agent a small training setup and told it to run experiments overnight. Modify the code. Train for 5 minutes. Check if it got better. Keep or discard. Repeat.
He let it run for two days.
The agent found 20 improvements on code he had already hand-tuned for months…including a bug in his own implementation he had missed entirely.
Shopify CEO Tobi Lütke heard about it.
He pointed the same pattern at Liquid…the templating engine powering every single Shopify storefront…and ran it overnight.
93 automated commits.
53% faster rendering.
61% fewer memory allocations.
The tech world went wild covering the machine learning angle.
But they missed the bigger story.
The pattern doesn’t care what you’re optimizing.
It works on anything you can run, measure, and iterate on.
Including your Claude Code skills.
The Idea in Plain English
Let me strip this down to its simplest form.
Because you do not need a GPU for this.
You do not need ML knowledge.
You just need to understand the loop.
Here it is:
→ One goal (what does “good output” look like for this skill?)
→ One file the agent can change (your skill markdown)
→ Run it (generate outputs, score them)
→ Keep or discard (did it improve? yes → keep. no → roll back)
→ Repeat
That’s it. That’s the whole thing.
Karpathy ran this on ML training code.
The builder community looked at it and asked:
What if we ran this on Claude Code skills?
A developer named Udit Goenka built a Claude Code autoresearch skill that does exactly that…applying the loop to any skill file, any domain.
It hit 608 GitHub stars in 3 days.
The insight is simple:
Most Claude Code skills are written once and never scientifically tested.
You tweak them based on feel.
You add notes based on memory.
You patch them manually when something breaks.
The Karpathy Loop replaces all of that with a system.
Set the goal. Claude runs the loop. You wake up to results.
How to Apply it to Your Content Skill (Step by Step)
Here’s how to set this up for your own workflow.
No complicated setup. No advanced configuration.
Step 1: Pick the right skill to start with.
Don’t do this for every skill at once.
Pick the one skill you use the most…the one where inconsistent outputs cost you the most time.
That’s probably one of these:
✅ SEO blog post writer
✅ Newsletter writer
✅ Email sequence writer
The more frequently you use a skill, the faster the improvement compounds.
Start with one. Get it working. Then expand.
Step 2: Define what “good” looks like.
This is the most important step…and most people skip it.
Before you ask Claude to improve anything, you need to define your evaluation criteria.
Write out 4–6 simple yes/no questions that describe what a great output from this skill looks like.
For example, for a newsletter writer skill:
Does the output open with a personal hook or story? (yes/no)
Does it use short paragraphs — 1 to 3 lines max? (yes/no)
Does it include a clear, single call-to-action at the end? (yes/no)
Is the advice actionable, not vague? (yes/no)
Does it match the defined brand voice without generic filler phrases? (yes/no)
Make them specific enough to be real. Vague criteria produce vague improvements.
Step 3: Point Claude Code at the repo and your skill.
Open Claude Code.
Give it a prompt like this:
Use the Auto Research methodology from https://github.com/karpathy/autoresearch
to build a self-improving skill system for my [skill name] skill.
The skill file is located at [your file path].
The eval criteria are: [paste your yes/no criteria].
Every 2 minutes, generate 10 outputs using the skill,
pass them through the eval suite, score how many pass,
and improve the skill prompt to increase the pass rate.Claude Code will read the repo directly from the URL.
It understands the structure. It builds the eval loop. It starts running.
For me, it looks like this when I try to improve my authority-post SKILL.
Step 4: Let it run.
Now, you walk away…
You come back in the morning.
And your skill file has been quietly rewritten…tested against your criteria, improved, verified.
No manual prompt tweaking.
No “I should update this later” notes you never follow up on.
Just a measurably better skill waiting for you.
The Real Unlock: Silent Improvement
Here’s what this actually changes for you.
The old way:
You finish a chat in Claude Code.
The skill produced something off.
You tell yourself, “I’ll update the markdown file later to fix this.”
You close the tab.
You forget.
The same mistake happens next time.
The new way:
The skill runs itself through evaluation criteria you defined once.
It catches patterns you’d never notice manually.
It logs what works. It discards what doesn’t.
It improves…systematically, measurably, silently.
Read that again.
Your skill gets better while you’re not working.
This is the shift from content creator to systems builder.
You’re not patching prompts one by one.
You’re building infrastructure that compounds.
One developer applied this to his fundraising skill.
It went from producing correct pitch deck structures 70% of the time to 94%.
Another applied it to a sales qualification skill.
Accuracy jumped from 65% to 91%.
Each run: a few hours, mostly unattended, cost less than a coffee.
That’s what compounding looks like.
Now…Where to Start….
Let me make this practical.
If you have a Claude Code skill stack already, here’s your priority order:
Tier 1 — Do these first (daily-use skills):
Your primary content writing skill (blog, newsletter, threads)
Your SEO research or keyword skill
Any skill you use more than 3x per week
Tier 2 — Do these second (weekly-use skills):
Email sequence writer
Product description or offer writing
Tier 3 — Low priority for now:
Skills you use less than once a week
Skills where outputs are already highly consistent
Start with Tier 1.
Run the loop once.
See what comes back.
Then decide if you want to expand.
The goal is not to automate everything overnight.
The goal is to pick the one skill that costs you the most time when it underperforms…and make that skill dramatically better by Tuesday morning.
Your Next Step
You have two paths from here.
Path 1: Build it yourself.
You have the framework.
You have the steps.
Grab the repo → github.com/karpathy/autoresearch
Define your eval criteria. Point Claude Code at your skill. Let it run.
If you already have a solid skill stack, this is exactly how you sharpen it.
Your next level awaits. Go get it.
— Sharyph | Founder of The Digital Creator
P.S. The autoresearch repo is completely open source. If you set this up this week and see a measurable improvement in your skill outputs…reply and tell me. I read every reply. I might feature your result in a future issue.
P.P.S. Not on the newsletter yet? If someone forwarded this to you, subscribe here → The Digital Creator on Substack
That was so well broken down. It’s truly empowering and incredible to see someone like Karpathy making such a meaningful impact on thousands of people’s lives and work. Thanks for putting this together!
The eval criteria step is where most people will skip and wonder why it didn't work.