Why AI Cannot Explain Your Engineering Decisions

There is something interesting about the way organizations remember. Ask a company what happened five years ago and there is usually a lot of information available. There are documents, emails, drawings, revisions, purchase orders, reports and meeting notes. The bigger the organization, the more records it tends to have.

But ask why something happened and the answer becomes harder to find.

Why was this design changed?
Why this supplier?
Why was this specification introduced?
Why does the change process carry four approvals when two would do?

The answers are rarely in any system. They are in the person's memory. They remember the project, the problem, the conversation and the decision that followed. They are carrying something the systems around them do not.

Ask a manufacturing company what happened five years ago and there is plenty on record. Drawings, revisions, purchase orders, inspection reports, meeting notes. The bigger the company, the more of it there is. Ask why it happened and the search gets much harder.

What Our Systems Record

Enterprise software has spent decades making organizations better at remembering.

• PLM remembers products and revisions.
• ERP remembers transactions and materials.
• QMS remembers quality events.
• MES remembers what happened on the floor.

This solved a real problem. Information that used to live in filing cabinets and in people's heads can now be recorded and retrieved by anyone who needs it.

But remembering an event is different from remembering the thinking around the event.

Take an ECR that tightens a tolerance on a shaft from ±0.10 to ±0.05. PLM captures the revision, the date and the approver. The control plan gets updated. The inspection method changes. Every one of those records is accurate. None of them says that the tolerance moved because assembly kept finding interference on the line, and because one of two suppliers could not hold capability at the looser specification.

The organization remembers the change. The reasoning behind it sits somewhere else, if it sits anywhere at all. Sometimes in another system. Sometimes in an email thread. Sometimes only in the memory of the engineer who chaired the review.

The Context That Was Never Captured

Foundation Capital's recent work on context graphs looks at exactly this layer of enterprise knowledge, the material surrounding decisions rather than the data inside the systems: exceptions, approvals, prior decisions and the information that shaped them. The argument is right, and the distinction is one the industry has been slow to make.

We normally describe this as disconnected data. ERP holds one piece, PLM another, QMS another, and somebody has to connect them. That is true, and integration work helps. It is also the easier half.

The harder half is that a great deal of the context was never captured as structured information in the first place.

A decision gets made during a design review. An engineer explains why the change is necessary, and 3 people in the room agree. A quality manager approves a new process because he remembers a failure from an earlier product line that is not referenced anywhere in the approval. None of that reasoning enters a field in a system. The outcome does.

You can integrate every system in the company and still not have it. It was never written down.

The Person Who Knows Why

This is why experienced people are so valuable, and why losing one costs more than the org chart suggests.

An engineer who has spent 10 years on a product does not see a drawing as dimensions and annotations. They see it alongside everything they have learned from working with it. Which changes caused trouble. Which suppliers need watching. Which specifications matter and which came from a requirement that stopped applying years ago. They also know that a decision which looks strange today was probably made for a reason.

None of that comes from a single document. It comes from having been there, and from building a network of connections in their head between things the software stores separately.

Every department does this in its own way. Procurement accumulates judgment about suppliers, quality about recurring failures, production about what the processes will actually tolerate. Over time each one develops its own way of deciding things, and most of that never leaves the department.

What an Agent Sees on Its First Day

We have discussed before about why engineering organizations need an intelligence layer at the department level. The point I want is about reasoning rather than knowledge. For a long time this was a human problem with a human workaround. If you needed to know why something was done, you found the person who knew.

AI changes the question, because we now expect these systems to work across enterprise information, answer questions about it, and eventually recommend and act.

Picture an agent reviewing an engineering change. It can pull the current drawing, retrieve the previous revision, read the BOM and find the relevant quality reports. What it cannot do is explain why the earlier change happened. Without that, it is looking at the same information a new employee would see on their first day.

The experienced engineer is reading the same drawing with the history attached, and the history changes how the drawing gets interpreted. This is the point at which context stops being something we pass into a prompt and becomes something the organization has to preserve.

What the Graph Actually Holds

The interesting part of a context graph is not the graph. Graphs are old technology and the schema is the least of the difficulty. What matters is what the graph is holding.

A design changed → Why? → A supplier had a problem. → Why did that matter? → Because the same issue had appeared in production on another line. → What happened then? → A process was changed, and the quality records from the following quarter say whether it worked.

That is no longer a set of separate records. It is a chain of decisions and outcomes, and a decision is almost always a response to something. It has a reason, constraints, alternatives, sometimes an exception, and eventually a result.

Connect those and something changes in how the past behaves. A problem found on one project starts explaining a problem on another. A failed approach stops the next person from repeating it. An exception can become a precedent.

This is roughly how a person builds expertise. Do something, watch what happens, remember it, use it next time. Organizations go through the same loop, except that most of it stays informal and walks out of the building with whoever ran it.

Closing Thoughts

We spent a long time building systems that record what organizations do. Then we started connecting those records. Now AI gives us the ability to reason across all of it, which raises a question we have mostly skipped: what should it be reasoning from?

Given only the current state of the data, a system sees the organization as it is today. Given the decisions that produced that state, it can see how the organization arrived there.

The most useful knowledge in a company is often not the answer sitting in a document. It is the reason the document looks the way it does.

An organization should not have to find the one person who remembers, every time it wants to understand a decision it made. That is the memory we are trying to build with Insight.

At Coffee Inc., we are building Insight around this problem, extending the engineering document understanding in ADEOS into the decisions and history that surround it. To discuss how this applies to your organization, write to us at coffee@coffeeinc.in.