Why You Can't Vibe-Code Your Way to Good Release Communication

Brian Sierakowski - Founder

“Why not just use ChatGPT for this? Or Copilot? Why don’t we build something internally?”

A security leader asked me this recently during a product conversation. It’s a fair question. LLMs are everywhere now. Every engineer has access to them. The inputs for release communication (commits, PRs, tickets) are all machine-readable. How hard could it be?

Here’s the thing: building something that works once isn’t hard. Building something that works every time, for every team, without constant maintenance? That’s a different problem entirely.

The Vibe-Code Trap

I’ve started calling the internal build approach “vibe-coding.” Someone spends a weekend wiring up a script that pulls commit messages, sends them to an LLM, and posts the output to Slack. It works. Everyone’s impressed.

Six months later:

  • The script broke when GitHub changed their API
  • Nobody remembers how they structured the prompt
  • The person who built it left the company
  • Three different teams have their own versions that produce inconsistent outputs
  • The Slack channel it posts to is now muted by everyone

This isn’t hypothetical. I hear variations of this story constantly. A security leader told me her company has “many teams using different AI tools with no unified operational layer.” The experimentation phase is exciting. The maintenance phase is where things fall apart.

Why Simple Summarization Isn’t Enough

The first version of any internal tool does simple summarization: pull the commit messages, send them to Claude or GPT, get back a paragraph.

This works for about a week.

Then you discover that commit messages like “fixed the thing” and “WIP” produce useless summaries. You need to look at the actual code changes.

Then you discover that code changes without context produce technically accurate but user-meaningless updates. “Refactored authentication middleware” tells customers nothing.

Then you discover that different audiences need different framings. Engineering wants technical detail. Support wants customer impact. Executives want business metrics.

Then you discover that last month’s update referenced a feature that was in development, not shipped. Now you need to track deployment state.

Each of these is solvable. But each solution adds complexity. And each complexity layer requires maintenance.

The Hidden Dependencies

Here’s what a real release communication system needs to handle:

Temporal context. A commit today might complete a feature that started three months ago. Understanding that requires tracking work over time, not just summarizing individual changes.

Product context. “Added rate limiting” means different things for different products. A consumer app and an API-first B2B platform need different framings of the same technical change.

Audience context. The Slack update for engineering isn’t the same as the customer changelog entry isn’t the same as the investor update. Same underlying change, different presentations.

Deployment state. Not everything merged has shipped. Feature flags, staged rollouts, and multi-environment deployments mean “in main” doesn’t equal “live for customers.”

Approval workflows. Someone needs to review before it goes out. That someone varies by audience and importance. The approval needs to happen in whatever tool they already use.

Distribution infrastructure. Slack is one channel. Email is another. In-app notifications, hosted changelog pages, documentation sites: each has different formatting requirements and delivery mechanisms.

A weekend project can handle one of these well. Handling them reliably is a product.

The Maintenance Tax

Even if you build something comprehensive, you inherit the maintenance burden.

Every LLM API has rate limits, cost structures, and capability changes. Every integration (GitHub, Slack, Jira) has API versions that deprecate. Every internal tool depends on institutional knowledge that leaves when people leave.

A VP of Product Marketing described her team’s situation: they have a changelog, updates get posted, but there’s a “backlog of unmessaged releases.” The system exists. The system doesn’t run itself.

The maintenance tax recurs every month, every quarter, every year. The question isn’t “can we build this?” It’s “do we want to maintain this forever?”

When Building Makes Sense

Internal builds can work in specific situations:

Single audience, single channel. If you only need engineering updates in one Slack channel, a simple script might be enough.

High internal tooling capacity. Some companies have dedicated internal tools teams with bandwidth to maintain custom solutions.

Unique requirements. If your workflow is genuinely different from everyone else’s, off-the-shelf tools might not fit.

Most companies don’t fit these profiles. Most companies want release communication to work reliably without thinking about it. That’s not what you get from a weekend project.

The Real Cost Calculation

A security leader told me her monthly updates take four to five days. Let’s say that’s conservative: call it three days per month for most teams.

Three days × 12 months = 36 days per year. For a product leader billing at $150/hour, that’s roughly $43,000 in time spent on a repetitive task.

The internal build approach doesn’t remove that time. It shifts it. Instead of writing updates, you’re maintaining scripts. Instead of coordinating reviews, you’re debugging integrations. The work changes shape but doesn’t disappear.

The question isn’t whether you can build something. The question is whether the build + maintenance cost is lower than buying a solution that already works.

What Actually Solves This

The systems that work, whether built internally or purchased, share common properties:

They aggregate context over time. They track the full history of what this feature is and why it matters, beyond just today’s commits.

They translate for audiences. Technical changes become customer benefits. Engineering details become business impact.

They integrate into existing workflows. Not a new tool to check, but updates appearing where people already work.

They maintain themselves. When the GitHub API changes, someone else fixes it. When the LLM improves, you get the benefit automatically.

The vibe-coded solution works until it doesn’t. The production-grade solution works because someone’s job is maintaining it.

The Honest Assessment

You probably can build something that generates release updates from your commits. The question is whether you should.

If your team has spare engineering capacity, tolerance for ongoing maintenance, and genuinely unique requirements, building might make sense.

If your team has limited bandwidth, wants to focus on your actual product, and has standard release communication needs, the build approach is likely more expensive than it appears.

The vibe-code version looks cheap because it hides the costs. The maintenance happens later. The failures happen at inconvenient times. The institutional knowledge requirements aren’t obvious until someone leaves.

Building your own release communication system is like building your own CRM: possible, occasionally justified, but often a distraction from what you’re trying to do.

If you’ve tried the internal build approach and found yourself maintaining scripts instead of shipping product, let’s chat about how Changebot handles the complexity so you don’t have to.