How We Use AI Agents To Maintain Our Knowledge Base

A single update to your product means countless documents in your knowledge base require updating. Older screenshots no longer reflect the new UI, some page links become obsolete, and a feature needs to be renamed everywhere!

Creating accurate documentation for the knowledge base is already a huge task; maintaining and updating it makes it more tedious. And if anyone understands that challenge well, then that’s us. So we tried delegating the tedious parts of maintaining the knowledge base to AI agents, freeing our writers to focus on creating value-packed documentation. And it worked!

This article will tell you about the five AI agents we created to build a self-maintaining knowledge base, where we think you still need the human in the loop, and what to measure if this system is working for you.

What Is a Self-Maintaining Knowledge Base?

A self-maintaining knowledge base is using AI agents to automate detecting and fixing outdated content in the knowledge base. These agents regularly scan your system to flag missing updates or other content issues and fix them, so you don’t have to run manual audits.

Earlier, someone had to notice a discrepancy in the content and raise a ticket or run a quarterly audit to keep the knowledge relevant. However, with a self-maintaining knowledge base, maintenance becomes a part of the operations that run continuously in the background.

Quick note: A self-maintaining knowledge base isn’t unsupervised AI publishing. While human work decreases significantly, their role becomes crucial in supervising.

Why Documentation Maintenance Is Never-Ending

When the maintenance work lags, no matter how searchable your knowledge base is, it’s always at risk of being inaccurate. Employees spend 35% less time looking for company information when your knowledge base is easy to search through. But that time saving only pays off if the information they find is current and accurate.

Reducing lag in knowledge base maintenance requires constant vigilance in checking and updating content, because a single product change can cause a ripple effect. When maintenance tasks pile up, it creates documentation debt (similar to technical debt). Each small change you make to the product often doesn’t feel urgent at first, but it becomes overwhelming as it adds up.

Then one day a customer/employee complains that they can’t follow the instructions in the knowledge base. The screenshots they see in the documents look nothing like the current UI, or they can’t navigate as per the features mentioned to set up the tool.

Why Continuous Documentation Maintenance Is a Need

When your product is constantly updated, customers and employees expect documentation to be up-to-date. And that’s not going to happen if you’ve scheduled a documentation audit once every quarter. Plus, audits only help identify the gaps; they never prevent the documentation drift.

So, by the next audit, you’ll again have to identify and fix countless outdated procedures, broken links, duplicate content, etc. That is why continuous documentation maintenance is the need of the hour. It switches from the reactive approach to a proactive one with a self-maintaining knowledge base.

So, while your team focuses on creating fresh content about recent launches and releases, AI agents constantly detect discrepancies in existing content and draft updates. Simply review, validate, and approve them for publication.

See how Document360 turns documentation maintenance into a background job

Book a Demo

Connecting Claude and Document360 to Enable Agents

Document360 provides a native MCP server that lets you connect it to Claude. When you do that, Claude gets access to your knowledge base and its capabilities with an authenticated user’s permissions.

When given permission, Claude can scan through your knowledge base, identify the need for updates, and draft them. It can also perform the supported action based on your workflow and configured permission. Read our step-by-step guide to connect Claude with MCP to connect your agent with your Document360 knowledge base. You can also use ChatGPT by connecting it with the MCP to make changes to your knowledge base in the same way.

Five Agents We Used to Build a Self-Maintaining Knowledge Base

We built five agents, each paired with an existing Document360 feature to carry out a particular job using Claude. Think of them as five different junior editing team members with a unique job. Together, they make our self-maintaining knowledge base.

An agent to audit content

This agent uses Document360’s Duplicate Content Detection powered by Eddy AI, which scans content to review repeated text blocks. For example, it identifies a paragraph repeating 16 times across 10 articles.

Where Claude comes into play: An AI agent paired with this feature turns the identified list of repeated text into a decision. It automates reviewing the list, verifies the need for action, and recommends the next step. For example, of 16 reported repetitions, say:

• 8 are genuine: The agent will recommend creating a snippet
• 4 are just re-reported: The agent will recommend merging it
• 2 have discrepancies: The agent will recommend rewrites
• 2 can’t be changed: The agent will recommend leaving it alone

An agent to detect broken links

Our Link status feature in the analytics checks every internal, external, and media link to tell you whether it’s working, broken, or redirecting. You can schedule validation runs every seven days, and this feature will hand you a full CSV report on link status.

Where Claude comes into play: When you give the link status report to Claude, it will tell you why the links are broken. It could be a moved page or even just a typo in the URL. Based on the issue, the agent will suggest how to replace the link.

An agent to refresh content

Don’t confuse this agent with an AI writer. Think of this agent as the “maintainer” that can refresh outdated content, rewrite sections for clarity, standardize terminology, and help writers write articles in depth.

Where Claude comes into play: While Eddy’s AI writing assistance drafts in alignment with your style guide, the agent manages cross-article consistency. For example, if you rename a feature, the agent will propagate it consistently across all articles that mention it.

An agent to reuse old content

Wouldn’t it be great if you didn’t have to change the name of the feature on all 200 articles by hand, and simply update a variable to make all changes at once? Well, that’s what this agent does with all the reusable content.

Where Claude comes into play: Claude first scans the entire knowledge base for reusable candidates, looking for repeated strings like email addresses, version numbers, product names, or API endpoints. Then it will tag it as:

• A variable: short, factual, and changes often
• A snippet: a block of formatted content

It will also help you prioritize the change with information like how many articles are affected by the change.

An agent to detect any gaps in current content

Eddy AI feedback manager reports search queries that don’t get an answer in the knowledge base. Our last agent is paired with this feature to help find content gaps in the current knowledge base.

Where Claude comes into play: All the above agents fix the current content; the last one finds what’s missing by clustering unanswered queries and highlighting their overarching theme. Say, for example, the feedback manager reported 200 searches. Claude categorizes them into five topics readers keep asking about. You can now draft articles for those topics and fill the gaps, so next time your employee or customer wants an answer to a question in that topic, they don’t meet with a dead end.

The Role of the Human-in-the-Loop

Our content audit agent flagged content translations as “duplicate” and suggested merging them. However, it was intentional, and our human-in-the-loop caught the error and marked it as “intentional duplicates.” This simple involvement helped the AI to learn to ignore it in future cases and shows the importance of human oversight in an AI-powered content management system.

Here’s where we suggest you need human surveillance in a self-maintaining knowledge base:

• Control access: Use role-based permissions to give agents access to the knowledge base only as much as they need. Keep the final say on publishing with your human employee.
• Standardize reviews: When agents propose changes or updates, a human must review them before publishing.
• Delegate repetitive tasks, keep the judgment: Agents are good at repetitive work, like auditing and rewriting. But tasks that need judgment should stay with humans. For example, if you need to create content around an unreleased feature, your writer should do it.

How to Measure the Efficiency of Your Self-Maintaining Knowledge Base

Here are some metrics that will tell you if your agents are doing their job well:

• Broken links at last scan: Track every week and see if the broken links are going down
• Audit hours spent every month: The agents should have to spend less and less time as duplicate and stale content goes down
• Time to update document: Track it after every product release or update
• How many articles missed review deadlines: This number should go down to zero over time
• Unanswered-query volume month over month: The gap in content should also go down as the fifth agent works with full efficiency

Build Your Self-Maintaining Knowledge Base One Agent at a Time

If you build and deploy all five agents in one go, you won’t be able to get efficiency with any. Instead, build one agent at a time. Start with the first agent, and don’t move on to the broken link detector agent until it efficiently detects duplicate content and helps you fix it without making mistakes.

Sign up with Document360 today and use our MCP setup guide to start building your self-maintaining knowledge base.