Start Digging: Four Steps to Unearthing Your Company's Buried Expertise While Building an AI Agent

Read this article on LinkedIn.

In Part I, we drew a map where every process in your business sits somewhere between two axes, process complexity and extent of codification. The ground that requires the most digging is where complexity is high and little has been written down. That's where tacit knowledge–the things only your best employees know–needs to be captured so that an AI agent can operate effectively

Step 1: Survey the Surface Area & Select an Expert Archeologist

The first step is to select the dig site and elect your expert archeologist who is the authority on where to keep digging, where we’ve dug enough, and what is missing in the excavation.

Don’t attempt to ingest all company knowledge. Begin with a bounded domain (e.g. HR, Sales, etc.). Once you’ve identified the domain to work with, survey the domain’s surface area. The surface area consists of three components: topics, data sources, and data uses.

Topics refer to the type of questions you might ask within a specific domain. The consequence-confidence matrix can assist you in determining the high-impact questions to ask.

Data Sources refer to the type of material used to inform the answers. Data sources fall into the following categories: explicit knowledge (recorded) and tacit knowledge (not recorded). Explicit knowledge is any source of knowledge that has been captured: a manual, memo, policy, SOP, email, database etc. Tacit knowledge is the experience that your employees have that hasn’t been recorded. This all constitutes your first-party knowledge (knowledge you have in your company). The other category to consider is relevant third-party knowledge (knowledge that exists outside your company, for example a certain regulation that affects the domain in question).

Data Uses refers to the types of tasks, projects, or agents you can create using the data sources. For example, the domain might be “international contractors”, the topic “regional holiday scheduling”, and the use “calculate respective overtime pay”. The data sources you might need for this are the contractors’ schedules (explicit, first party knowledge) and a list of holidays for each region your contractors are in (explicit, third party knowledge).

Your subject matter expert is your head archaeologist. They determine which sources are needed and which uses these sources afford. Your experts also identify where data is missing. Previously, data scientists were charged with company record keeping. With AI, record keeping becomes less reliant on programming the data and more reliant on actually understanding it. The AI can sort, organize, and retrieve knowledge as long as it has the right knowledge. The expert is the one who knows what the right knowledge is.

Step 2: Dig into Explicit Knowledge

Explicit knowledge is easier to dig into than tacit knowledge. That said, for most organizations, it’s scattered across multiple sources or locations. The first step is ingesting the relevant documentation into an AI system.

AI systems categorize the data from explicit knowledge as either structured or unstructured data. Structured data refers to any information that exists in a table (financial tables, CRM inputs). Unstructured is everything else (manuals, memos, emails, videos). Structured data, such as your CRM, ERP, and financial tables, should be queried where it lives through governed connections, not flattened into text for a model to guess at.

Documents such as policies, playbooks, and contracts should keep their structure intact, including section headers, effective dates, versions, and who is allowed to see them. Every answer should trace back to the page and clause it came from.

When inputting this data into an AI system, make sure you clearly outline the domain, topics, and uses. This assists the AI system in reading the content from a defined lens so it can map data to your business outcomes. When you excavate a jumble of stones, it's hard to tell whether they belong to the same ancient structure or are just random rock. Give the AI system a blueprint and it can read a varied swath of data and find insights that build toward the structure you're after.

Go a level deeper and give the agent rules in order to be able to prioritize the ingested data. Without rules, the system can’t determine which data is the most relevant. For example, in some cases regulation outranks internal policy, policy outranks system data, and expert guidance outranks system data.

Step 3: Collect Tacit Knowledge

Collecting tacit knowledge requires more finesse than plainly asking your experts how they conduct a task. As philosopher Michael Polanyi stated, "We know more than we can tell." Ask your senior technician how to diagnose an HVAC compressor and you’ll likely get an abridged version of the manual. This answer glosses over the ingrained know-how that is more effectively captured through stories.

A better approach is to ask your experts to tell you about a challenging job. What did they notice first? What did they rule out? What almost fooled them? Another technique is to show an expert two cases that look alike and ask why they warranted different outcomes.

This is where AI can be incredibly useful. An agent can run these interviews and produce a process document for the expert to correct. AI can also query the collected knowledge and pose questions, finding gaps the collected content can’t answer. These questions can be routed to the expert to make the knowledge base more robust. It’s much easier for an expert to answer a single question than to review an entire manual.

Step 4: Polish your Treasure

Once explicit and tacit knowledge have been collected and cleaned (gaps filled in, conflicts reconciled), this knowledge base is now ready to be given to an AI agent. Safe implementation requires setting up intelligent guardrails. Before launch, test the system against a golden set of realistic edge cases, policy collisions, and ambiguous calls, all written by your experts.

Knowledge is dynamic, ever-changing, and requires endless fine tuning. Accept that the knowledge backlog is an eternal constant. Just as there’s always more to know about the world, there’s always more you can know about your business.

Every time an expert overrides an answer, that correction should feed back into the system. Your experts are the stewards for keeping the domain current. This type of treasure accrues in value the more attention it receives. Siri launched 15 years ago and Apple is still investing more in improving the function of the agent.

By treating domain expertise as a governed, living knowledge asset rather than passive documentation, organizations can compound operating leverage, preserve institutional capability, and establish an unassailable competitive moat.

While AI makes things easier, it’s still a lot of work. Especially if you don’t have an in-house engineering team. Trovia automates this process.