The DeMicco Digest
Grab your headphones and enjoy a mini-podcast version of this blog. Sit back and listen while we walk you through the key points!
Every vendor wants to sell you a team of AI agents. Almost none of them ask the only question that matters first: when the agent needs to know something about your customer, where does it look?
I get asked some version of this question regularly now: should we be building an agent team? Usually from an owner who has seen a demo in which a swarm of software workers drafts proposals, answers customers, and updates the CRM without anyone touching a keyboard. My answer has settled into two words, which are “not yet,” followed by a question of my own. When the agent handling a customer needs to know that the customer changed their product naming last spring, that their trade-show budget was cut, and that their CEO hates the word “solutions,” where does it look that up?
The room usually goes quiet, because the answer is that a person would know. The account manager knows. It is in an email thread somewhere. It came up in a call. That is how client-service businesses have always worked, and it is exactly why the agent demo does not survive contact with a real account. The demo ran on clean, complete information. Your business runs on information that is complete only inside the heads of the people who have been in the meetings.
So the first project is not an agent. It is a librarian.
Table Of Contents
What an agent actually is, in plain language
Strip away the marketing and an AI agent is four things: a language model that reasons, a set of tools it can use (search a database, send an email, update a record), a set of instructions about its job, and access to information it can retrieve when it needs context. The model and the tools are what vendors demonstrate. The instructions and the information are what determine whether the thing is useful, and the information is the part nobody wants to talk about because it is not for sale.
The retrieval part has a name. In 2020, researchers at Facebook AI published the technique now called retrieval-augmented generation, which pairs a language model with a store of documents it can pull from before answering. Most business AI systems that answer questions about a company’s own information work this way. The model does not know your customers. It looks them up, and the quality of the answer is bounded by the quality of what it finds.
Anthropic, whose models I use, published guidance on building agents that I wish every vendor pitching industrial companies would read. The advice is to find the simplest solution possible and add complexity only when it demonstrably improves outcomes, because agents trade cost and latency for capability. I would put it more bluntly for the audience I work with: an agent pointed at a mess does not fix the mess. It automates it.
Why the mess is worse in client-service businesses
I have written this year about companies that do not own their data and about CRMs that hold only the version of events marketing happened to capture. Those posts were about structured data: records, fields, sources. The librarian problem is about everything else, and everything else is most of what matters in a service relationship.
Think about what a good account manager actually carries around for one client. The original goals from the kickoff. The brand standards, written and unwritten. The product line and how the client describes it, which is never how the website describes it. Who the decision-makers are and who has to be copied. The trade shows they exhibit at, the ones they skipped last year and why. Every campaign, what it cost, what it produced. Every meeting, and the two decisions that came out of it. The things the client has said no to, so nobody proposes them again.
Some of that is in a CRM. Most of it is in email, in meeting recordings that nobody re-watches, in project-management comments, and in memory. When that account manager leaves, the client’s history leaves with them. That has always been a business risk. It is now also the reason your AI investment will underperform, because there is nothing for the model to retrieve.
The architecture, in three layers
What follows is what I recommend to clients and what I am putting in place in my own agency. It is deliberately unglamorous, and it can be built with tools you already pay for.
Layer one: the repositories
One folder per customer, organized into a fixed set of buckets that never changes from one customer to the next. Mine are: baseline and goals, brand and standards, products and services, audience and market, work and campaigns, performance and reporting, interactions (meetings, calls, email threads that mattered), and a decisions log. Eight buckets, the same eight for every account, so anyone in the company can find anything for any client without asking.
Two rules make this work. The filename carries the metadata: date, client, bucket, subject, in a fixed order, so a file is findable and sortable without opening it. The second is a displacement rule: when a new document supersedes an old one, the old one moves to an archive subfolder the same day. The active folder holds only what is currently true. That single rule is what keeps a repository trustworthy a year in.
A word to anyone who builds retrieval systems for a living and is wincing at “folders.” The retrieval technology- vector search, chunking, metadata filtering- is straightforward to buy and is getting cheaper. What it cannot buy is a corpus worth retrieving from. A well-organized repository with consistent names and no stale duplicates becomes a good retrieval index in an afternoon. A shared drive with fifteen versions of the brand guide becomes a bad one, and the tooling will faithfully retrieve the wrong version. The folders are not the system. They are what makes the system work.
Layer two: the index
This is the layer most companies skip, and it is the whole game. On top of each customer’s repositories sits a short set of files, a few pages at most, that say what is in each bucket, what the key facts are, what the current status is, and what has changed recently. They are written for a reader who has thirty seconds, human or machine. When someone, or something, needs context on a client, they read the index first and go to the repository only for detail.
This matters because of how language models actually behave. A model handed five hundred documents does worse than a model handed a two-page brief with pointers to the right five documents. Researchers at Stanford showed in 2023 that language model performance degrades as context grows, and drops further when relevant information sits in the middle of a long input rather than at the beginning or end. Anthropic’s engineering team has written about the same problem under the label “context engineering,” and the conclusion is one any manager will recognize: attention is finite, and the quality of a decision depends on being given the right information at the right level, not all of it at once. A new hire given a thirty-second briefing and a well-organized filing cabinet outperforms one given a login to the shared drive. The index is the briefing.
Layer three: the prompting and the agents
Only now does the AI conversation begin, and it begins with a set of context files that stand between the repositories and any prompt anyone writes. A context file for a client is the index plus the standing instructions: tone, banned words, approval rules, who signs off. Every AI-assisted task for that client starts from that file. Later, when an agent is built to draft the monthly report or prepare a meeting brief, it starts from the same file. The agent inherits a librarian instead of a shared drive.
This is a job for the team, not the engineer
The temptation is to hand this to whoever is most technical. Resist it. Layer one and layer two are organizational work, and the people who know where the bodies are buried are the account managers, project managers, and production staff who have been in the meetings. Assign each of them a bucket or a client to populate. Give them the eight-bucket structure, the filename rule, the displacement rule, and a deadline for the backfill. The engineer’s job is the index tooling and, much later, the agents.
The engineer does own two things from the start, and neither is optional. The first is access control: one client’s repository must never be readable by a tool or a person working on another client’s account, and the structure has to enforce that rather than rely on people remembering. The second is the data policy: which buckets may be read by which AI tools, under what terms, and which material never leaves the building. I wrote about that policy last winter. A librarian who lends every book to anyone who asks is not a librarian.
Two consequences follow that have nothing to do with AI. The first is that the company stops depending on any one person’s memory, which is the risk every service business carries and few address. The second, in my experience, is that new staff become useful far sooner, because the briefing exists. If the AI project were cancelled tomorrow, the librarian would still have paid for itself.
What to do in the next ninety days
- Pick two customers, not all of them. One straightforward, one complicated. Build the structure for those two, learn what breaks, then clone it. A company that tries to backfill every account at once will abandon the effort by week six.
- Fix the buckets and the filename rule before anyone files anything. Changing the structure after the fact is how you end up with two structures.
- Decide the backfill scope. Everything from the current year in full; earlier material only if it is still true. Old campaigns that ended are history, not context, and they go straight to the archive.
- Route the meeting recordings. If you record client meetings, the transcript and a short summary of decisions go into the interactions bucket the same week. This is the single richest source of the context an agent will later need, and today it is the least used.
- Write the first index by hand. Two pages per client, written by the person who knows the account best, before any tool is involved. You will learn what the index needs to contain by writing one.
- Set the cadence. Weekly: new material filed, displaced material archived. Monthly: the index re-read and updated. Put it on the calendar with a name next to it, or it will not happen.
- Then, and only then, scope the first agent. Pick the narrowest useful task, the meeting brief or the monthly report, and build it against the context file. If it works for two clients, you have a pattern. If it does not, you have learned something cheaply.
The demo you should ask for
The next time a vendor shows you an agent team, ask them to run it against one of your real accounts, with your real history, without preparation. Watch what it does when it needs to know something that is only in an email from last year. Most of the time it will either invent an answer or ask a human, and either outcome tells you the same thing: the agent is fine. The library is missing.
Build the librarian. The agents will have somewhere to go.
Sources & Further Reading
The following resources informed this article. All claims have been verified against multiple sources where possible.
- Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” 2020 (Facebook AI Research) – https://arxiv.org/abs/2005.11401
- Anthropic, “Building effective agents,” December 2024 – https://www.anthropic.com/research/building-effective-agents
- Anthropic Engineering, “Effective context engineering for AI agents,” 2025 – https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
- Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Stanford / TACL, 2023 – https://arxiv.org/abs/2307.03172
- McKinsey, “The state of AI” (2025 survey) – https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
Joe DeMicco is the Founder and President of Amplify Industrial Marketing + Guidance (AIMG), a full-service B2B industrial marketing agency with 30+ years serving manufacturers, fabricators, distributors, and trade associations. He is also the founder of Industrial Web Search (IWS), a keyword-driven B2B supplier directory platform. Joe teaches the marketing module for the Goldman Sachs 10,000 Small Businesses program and writes about AI, marketing, and B2B strategy at demicco.com.
