Contents
Preface: These Stories Are Not Just About One Company
When businesses first begin to consider bringing AI into their operations, they are rarely thinking about some grand vision of the future. An owner wants to reduce costs. A manager hopes to spend fewer late nights at work at the end of the month. Employees would rather not keep moving the same information from one spreadsheet to another. Someone simply wants customers to wait a little less. Technology often finds its way into the workplace through wishes as ordinary as these.
At first, AI was largely an assistant, ready whenever someone needed it. Ask a question, and it would answer. Give it a document, and it would produce a summary. Struggle with an email, and it would draft one for a person to revise and send. Even when it got something wrong, there was usually someone at the screen deciding whether to use the result. But as these systems gained access to business records, tools, and workflows, some assistants began to take on the capabilities of AI agents. They could do more than respond to requests: they could carry out a sequence of tasks, compare supplier quotations, prepare customer replies, reconcile accounts, schedule services, and even move work to the next stage within previously authorized limits. This was progress worth welcoming. The speed of the machine was no longer measured only by how quickly it could produce polished prose.
It was also where things became more complicated. A report can be generated in seconds, but deciding whether the prices it contains are still valid requires an understanding of what has changed in the real market. An email to a customer can be impeccably worded and still make a promise the company is not prepared to honor. Payment documents may satisfy every formatting requirement without establishing that a payment ought to be made at that particular moment. More troubling still, the problem may have little to do with whether the machine is intelligent enough. Information can be out of date. Departments can fail to communicate. Reviewers can be tired. Company procedures can leave gaps. And the action ultimately taken may not be quite the action that was originally checked.
These tensions are not peculiar to any one industry. Manufacturers must weigh purchasing decisions against delivery commitments. Service businesses have to balance prompt responses with promises they can responsibly keep. Finance departments care about supporting records and authorization, while technical teams must consider whether a change will disrupt systems already in use. The circumstances and risks differ, but once machines begin to participate in preparation, judgment, and action, organizations face a common set of questions: What must people still check? And how can they make sure that essential requirements are not lost at the very last step?
To explore those connected questions, we chose not to write a collection of unrelated company case studies. Instead, across the next fifteen chapters, readers will return to the same unnamed midsize business. They will encounter the people who buy its supplies, manage its finances, handle sales and customer service, maintain its systems, and make decisions about its future. They will see these people grow excited, hesitate, disagree, and change their minds as AI becomes part of everyday work. The company is not a portrait of any particular real business, nor are we suggesting that one company actually experienced all these events in this order. It is a composite fictional company, created to bring situations that different organizations might encounter when adopting AI and automation into a single, continuous story. A question left unresolved in one chapter can then be pursued in the next.
The people, conversations, transactions, and specific events in these pages should therefore not be read as investigative findings about a real company. Nor should the losses described, or the choices made by customers, be cited as actual cases or statistical evidence. Fiction can help us understand how a risk might arise, but it cannot replace an investigation into what happened in a real organization. Still less can an invented episode prove that a particular technology will necessarily work. Where companies have already addressed some of these problems through mature access controls, approval procedures, and automated checks, that experience deserves to be acknowledged, not written out of the picture for the sake of a more dramatic story.
We have stayed with one company and one recurring cast because problems of governance rarely arise in isolation. Permission granted today to make work more efficient may lead tomorrow to an action no reviewer can adequately explain. An extra approval introduced last month to prevent one mistake may leave business piling up in a queue this month. A conscientious employee may not have all the information needed to approve the decision placed before them. A team that discovers an error may still be too late to win back a customer who has already gone elsewhere. By following these situations within the same company, we can see something that separate examples might conceal: problems do not begin afresh each time. They take on new forms as an organization tries to repair what went wrong and learns from the results.
We did not want to write an indictment of AI. Tools that take over repetitive work and help people recognize problems sooner offer genuine benefits. Better models, clearer data, more experienced employees, and better-designed processes can all improve outcomes. Nor do we intend to portray traditional corporate procedures as a collection of obsolete formalities. Divisions of responsibility, delegated authority, approval processes, and checks and balances are ways people have learned to work together while recognizing that none of them is infallible. AI has not made that experience irrelevant. It has introduced new speeds, new scales, and new participants capable of acting continuously.
What concerns us is how often the debate seems to stop at two unsatisfying alternatives. Either a human must watch everything a machine does, or we must wait until the machine becomes smart enough to trust with more and more responsibility. Yet humans make mistakes, too, and there are limits to how much they can review. A machine may grow steadily more capable, but the fact that a decision appears correct does not, by itself, give it the authority to make that decision a reality. When an important action is about to be carried out, an organization may need to ask more than whether the proposed course is sensible or who has already approved it. What conditions must hold at the moment of execution? Who has the authority to set those conditions? And if they are not met, can the action actually be stopped?
We have previously written three more technically oriented books on execution control: The Execution Gap, Execution Control Engineering, and Adversarial Completeness. Those books sought to examine questions of authority, boundaries, and real-world execution with engineering rigor. They also made us realize that these questions should not be confined to technical specialists. Over the past two years, we have published articles on platforms such as CSDN and Huxiu, and written or submitted pieces for business magazines and other media, exploring the changes AI brings from the perspectives of business owners, organizational managers, and people doing ordinary jobs. Many of the questions developed here grew out of that body of work and the thinking behind the three earlier books. This volume is neither a compilation of old articles nor a technical text with its jargon replaced by everyday language. It is an attempt to let ideas that can seem abstract come closer to the reader through the experiences of a company and the people who work there.
You do not need to know programming, algorithms, or security engineering to follow this book. It begins with a purchasing report, an approval form, and a conversation with a customer, then gradually moves toward deeper questions about authority and responsibility. There is no agent in these pages that always gets things right, and no system of rules capable of anticipating every possible risk. People can be cautious, impatient, or willing to make choices that turn out badly. Machines, for their part, are not simply there to cause trouble. Together, they inhabit the working world we need to understand.
What we hope these fifteen chapters leave with readers is not an exhaustive catalogue of today's AI risks, or a ready-made solution that every business can adopt unchanged. We hope instead to offer a more grounded way of thinking. As intelligent systems take on more and more work for us, and begin to propose actions that can immediately affect the real world, how do we make use of their capabilities without placing the responsibility for governance on the hope that any participant, human or machine, will always be right?
The story begins on an ordinary Monday morning. The office coffee machine has not even finished heating the water, and a report that once would have taken half a day to prepare is already waiting in a shared folder. For the company, it is welcome news: work is getting done faster. For us, it is the beginning of a question worth taking a little more time to consider.
Acknowledgments
While writing this book, I often found myself thinking back to conversations about artificial intelligence that I have had with friends over the years. Some took place at work; others were little more than casual chats over a meal. Some friends spoke enthusiastically about the time AI had saved them or an experiment that had turned out better than they expected. Others described the frustration of answers that missed the point, the weariness of checking and checking again, or that slight feeling of unease that comes with handing a task over to a machine.
I am grateful to everyone who has been willing to share those experiences with me. You work in different fields and have followed different paths, and your hopes for AI are as varied as your concerns. Those ordinary, firsthand accounts have repeatedly reminded me that a serious conversation about artificial intelligence cannot stop at how capable a model is or how advanced the technology has become. We also need to look closely at what changes when it enters people's working lives and homes, and at the questions it leaves us to face. The company and events portrayed in this book form a composite fictional narrative; they are not representations of any particular friend or real business. But many of the questions that run through these pages owe something to those everyday conversations.
I owe special thanks to the three colleagues with whom I work at HAVENLON: Mengting Wu, Yong Zhang, and Jiang Deng. Together, we have wrestled with questions for which there were no ready-made answers. We have talked ideas through, challenged our own assumptions, set plans aside, and started over. Exploring something that is not yet well understood is not always exhilarating. More often, it calls for patience, brings disagreements, and involves a great deal of unglamorous work. It means being willing to move forward even when much remains uncertain. Thank you for your time, your effort, and your trust. Although I wrote this book, much of the thinking behind it grew out of what we have explored together.
I also want to thank my family. Writing and research have often taken time that should have belonged to life outside work. Your understanding and patience have given me the quiet space to work through ideas that were not yet fully formed and gradually find the words for them. I am especially grateful to the children in our family. Time with you brings me back from difficult concepts and arguments to life itself. You remind me that, however far technology advances, what ultimately matters is how we live, how we treat one another, and how we might leave a little more possibility for a better future.
::: {.ack-ending} Finally, thank you to everyone who has chosen to open this book. I do not expect every reader to agree with the judgments made in these pages. But if these stories lead you to ask even one question you had not seriously considered before—about using AI, managing a team, or the relationship between people and machines—then writing them will have been worthwhile.
Lin Wang
Chengdu, October 2026 :::
Chapter One — The New Colleagues Who Never Clock Out
It Started with a Report That Was Ready Early
Machines were good at bringing information together. People still had to step back into the world behind that information before they could take responsibility for the result.
At nine on Monday morning, while the office was still waiting for the coffee machine to heat up, a supplier comparison report was already sitting in the purchasing department's shared folder. It laid out several suppliers' prices, lead times, and payment terms, then offered a recommendation. One of the buyers opened it, read through the figures, and turned his screen toward a colleague. “Last month, this would've taken us half the morning.”
No one had spent the weekend putting the report together. Before leaving on Friday, the purchasing team had given a newly connected AI assistant a set of quotations and a list of requirements, with instructions to prepare a comparison by Monday morning. The assistant had done more than fill in a table: it had also flagged several differences between the offers. What would once have meant two hours spent assembling the information now meant looking through the results for anything that needed a closer check. That morning, the department wrapped up its regular meeting much earlier than usual.
The company was a well-established, midsize business. It had no plans to replace its workforce with machines, nor was it pursuing some grand technological transformation. Its thinking was straightforward. Every month, employees spent hours organizing documents, checking orders, answering customers, and reviewing system records. Could AI take care of some of that repetitive work and leave people more time for tasks that genuinely required experience and judgment? Management saw it as an ordinary operating question. The cost of an AI service was easy enough to put on a spreadsheet. The time employees lost to routine tasks was harder to see, scattered across countless forms, emails, and meeting notes. Recover even some of those hours, and perhaps the departments would work together more smoothly.
At first, employees used conversational AI much as they might use any other office assistant: to draft emails, revise reports, and summarize long documents. It answered when asked, made the occasional mistake, and left a person at the screen to decide what was worth using. Gradually, the departments began connecting it to more established workflows. Customer service had it sort incoming messages and draft replies. Finance used it to organize discrepancies in account reconciliations. Purchasing asked it to collect quotations and group similar materials. The operations team relied on it to summarize overnight alerts and equipment logs. Work that once had to wait while someone gathered records from several systems could often begin with the relevant material already on the screen.
The initial relief was real. Customer service staff no longer had to copy the same explanations into dozens of nearly identical emails. Finance employees spent less time moving figures from one sheet to another. Engineers arriving in the morning could see which overnight alerts mattered without reading every entry first. These new assistants did not complain about tedious procedures or leave unfinished tasks on their desks when the working day ended. Someone joked that the company had hired a whole team of colleagues who never needed a shift schedule or a day off.
Management soon had results to show for it. Purchasing proposals were ready earlier. Customers received initial responses sooner. Month-end finance paperwork no longer piled up quite so badly in the final two days. At departmental meetings, the story had moved beyond “We've started using AI.” Managers could point to changes people recognized: material that used to take all morning to prepare was ready before a meeting began; questions that once bounced between departments because no one could find the supporting records now came with the sources neatly collected. Encouraged, the company passed more routine work to the assistants. Employees developed a new habit: first see what the system had prepared, then decide what still needed to be done.
But work completed earlier was not necessarily work completed. A purchasing report could look thorough while still leaving a manager to check whether a quoted price had expired, whether the promised delivery date was realistic, and whether the proposed payment terms met company requirements. Finance software could identify a mismatch between two sets of figures, but matching figures alone did not establish that a payment ought to be made. A customer service assistant might draft a perfectly courteous reply, yet when refunds, compensation, or special promises were involved, an employee still had to decide whether the company could stand behind the words. An operations report could lay out an anomaly in detail, but an engineer considering a change to a live production system still had to determine whether it was appropriate to make that change now.
These checks were rarely a matter of glancing at a page. Sorting the figures in a quotation might take seconds; confirming the conditions behind them could mean checking contracts, asking the warehouse, or speaking with another department. An email could be drafted almost instantly, while deciding whether a customer's request was reasonable might require reviewing earlier conversations and understanding how the company handled such cases. Machines were good at bringing information together. People still had to step back into the world behind that information before they could take responsibility for the result.
At first, no one saw this as a problem. AI was there to help; of course employees would review important matters before anything went ahead. The purchasing manager was delighted, in fact. Previously, his staff had been so busy compiling quotations that he rarely had the chance to compare several options at once. Now the alternatives were usually waiting whenever he needed them. After a while, though, he noticed that his to-do list was growing. In the past, his team simply could not prepare so many matters for him to decide. With the preparation happening faster, the decisions arrived faster too. The new colleagues who never clocked out did not leave their work unfinished until the next morning. They finished one task and promptly placed the next, neatly organized, in front of someone else.
A To-Do List That Kept Growing
The more complete the material, the sooner the difficult choices reached his desk. The people gathering information were less burdened; the people deciding how to use it had more to consider.
What struck the purchasing manager was not that any single report had become unusually difficult to understand. It was the frequency with which they arrived. He might approve a routine materials purchase in the morning and find several proposals for alternative suppliers waiting that afternoon. Someone would remind him in the team chat that yesterday's order still needed a quantity confirmed. A colleague from finance would ask whether the business team had approved the payment terms in another contract. These matters had always existed. They had simply reached him one at a time, after the buyers had finished assembling the necessary information. Now that much of the waiting had disappeared, the judgment it had once concealed was suddenly in plain view.
One afternoon, he was on his way to the warehouse to inspect a delivery when his phone sounded several times in succession. Each notification was impeccably polite: a proposal was complete and awaiting confirmation; an order could not proceed without review; a revised price comparison was available and should be checked again. He reached the elevator, turned around, and went back to deal with the easiest one first. The task looked simple only because the spreadsheet was a single page long. Three prices were clearly listed, but one supplier had changed its minimum order quantity. That could affect the warehouse's inventory for the next two months. Before making a decision, he would have to talk to someone else.
None of this made him think AI was a bad idea. Quite the opposite. Without the assistants, his team might still have been assembling the first set of figures, with no chance of spotting the change in the ordering terms so early. But he was beginning to see the other side of better preparation. The more complete the material, the sooner the difficult choices reached his desk. The people gathering information were less burdened; the people deciding how to use it had more to consider. A file marked “Pending” no longer necessarily meant that the work had barely begun. It might mean that nearly everything was done except the one judgment no one could make casually on someone else's behalf.
The same pattern was emerging elsewhere. Customer service assistants drafted replies so quickly that staff could devote more attention to the few cases that did not fit standard procedures. Those cases demanded a look through past conversations, an understanding of promises already made, and sometimes a search for the colleague who actually handled the order. Finance staff spent less time transferring figures, only to uncover more transactions where the numbers matched but the business justification still needed explaining. The operations team could identify important changes in an overnight stream of alerts more quickly than before. Finding an issue sooner, however, did not mean it was safe to alter a running system without first considering the consequences.
Every department had gained something. Much of the simplest, most repetitive work had genuinely been reduced, and some routine matters could proceed automatically under clear rules without anyone approving them one by one. Yet as the company entrusted more tasks to its assistants, matters requiring an actual understanding of the circumstances surfaced more quickly as well. It was not that a host of new problems had suddenly appeared. Decisions that once took days of preparation to reach a person's desk could now arrive together in a single afternoon. The hours available to make those decisions had not multiplied at anything like the same rate as the machine's output.
Fast Doesn't Mean Finished
Those new colleagues were diligent, and they really had made much of the work easier. They could keep preparing things without stopping. He could not keep making judgments without stopping.
The company first discussed the problem seriously at an otherwise unremarkable progress meeting. No one complained that the AI was not intelligent enough. No one proposed switching it off. Purchasing presented the number of reports it had completed; customer service showed how much faster initial replies had become; finance described an easier month-end close. The numbers looked good, and everyone knew the improvements were real. Then the manager responsible for deliveries asked, “If the paperwork for some of these jobs was ready days ago, why are we only making the decisions today?” For a moment, the room fell quiet.
The explanation was ordinary enough. Prices for a batch of materials had been compiled that morning, but the purchasing manager had not had time to check the terms until the afternoon. The business team would not confirm the quantities until the following day. A draft reply to a customer had been ready the same day the inquiry arrived, but a request for special treatment was still waiting for a manager's decision. The machine had completed one part of the process. The customer, the warehouse, and the other departments were waiting for the whole matter to move forward. Having a report ready two hours earlier was useful, but if the final decision still took two days, the improvement felt considerably smaller than the dashboard suggested.
The delivery manager went to the whiteboard and sketched a simple sequence: receive a request, gather the information, make a judgment, confirm the decision, and pass the work along for the next steps. He used no technical terminology. He merely circled the time spent at each stage. It quickly became apparent that collecting information had once been the slowest part. That interval was now much shorter, while confirmation and communication continued at much the same pace. The work had not vanished; the bottleneck had moved. No matter how quickly one department ran, it could not complete the company's entire journey on its own.
At that point, the head of finance asked whether the system could not handle more of the straightforward checks itself. In fact, it already did. Duplicate documents, formatting errors, and certain discrepancies in amounts could be flagged or blocked automatically using defined rules. Small, routine matters with clear criteria did not always need to move through several levels of approval. The checks that remained were often the ones that could not be settled by comparing two numbers. Whether a price was acceptable depended partly on when the purchase was being made. Whether an exceptional refund was appropriate also depended on the customer relationship. A change to a production system might be technically feasible but still unacceptable to the business at that particular moment.
No one wanted to discard those judgments lightly. The purchasing manager said he would rather approve an order late than overlook a delivery condition in a rush. The customer service manager did not want staff making unapproved promises just to make response times look better. These concerns were not evidence of prejudice against machines. If an inexperienced new employee had prepared the same work, the company would still have expected someone to review the important decisions. The difference was one of scale. There was a rough limit to how many matters a new employee could bring forward for review in a day. A machine was unlikely to put next week's files aside because it had too much on its plate.
After the meeting, the company began prioritizing pending approvals, cutting unnecessary repeat notifications, and allowing workflows with well-established checks to continue running automatically. Things improved. The purchasing manager was no longer interrupted repeatedly about the same document, and customer service could bundle genuinely exceptional requests for a manager's attention. These were useful reminders that review bottlenecks were not some unavoidable consequence of using AI. Better processes, clearer division of responsibilities, and automated checks still mattered. Even so, as the company connected more business activities to its assistants, decisions that required an understanding of the situation kept arriving at people's desks.
Sometimes the exhausting part was not the number of tasks waiting but how close each one appeared to being finished. The information was there, the formatting was tidy, and the recommendation came with a plausible explanation. It was easy to feel that the final approval ought to be little more than a formality. The person responsible for making that judgment knew otherwise. A polished document did not establish that its sources were current, that nothing important had been left out, or that the recommendation still made sense in today's business circumstances. The machine might have produced an answer in minutes, but it had not necessarily supplied the answers to those further questions.
Gradually, the owner came to see that the company had been asking a question that was reasonable but incomplete: How much work could AI do for employees? The better question was how much waiting, rework, communication, and indispensable judgment still stood between a request and a job truly finished. Counting the documents a machine generated was a little like counting the dishes a restaurant kitchen prepared without asking how many actually reached the tables. Faster cooking was worth celebrating. But if plates were piling up at the pass, customers were not necessarily getting their meals any sooner.
That evening, the purchasing manager finally cleared the items left over from the morning. As he prepared to shut down his computer, another notification appeared: a preliminary comparison was ready for a purchasing proposal scheduled for discussion the next day. He was no longer as delighted by such messages as he had been at first, but neither did he feel resentful toward the machine. He gave a rueful smile and left the notification for tomorrow. Those new colleagues were diligent, and they really had made much of the work easier. They could keep preparing things without stopping. He could not keep making judgments without stopping.
A few days later, while reviewing the departments' progress, the owner raised a question that everyone had been reluctant to confront too soon. If machines could bring so much work right up to the point of decision, did a person still have to make every one of those decisions personally? No answer emerged on the spot. Some people thought of efficiency, others of risk, and still others of the way the company had long assigned different responsibilities to different roles. They all understood that the next discussion would not simply be about getting the assistants to do more work. It would be about something harder to settle: under what circumstances should a machine be allowed to make a decision on a person's behalf?
Chapter Two: When Can Machines Make Decisions on Our Behalf?
The Meeting Ended, but the Question Remained
If an AI agent makes a recommendation and a person simply approves it without changing a thing, what has that approval actually accomplished?
The question raised at the last progress meeting—when could a machine begin making decisions on people's behalf?—did not disappear when everyone left the room. At first, the company's owner had taken it for an idea born of the procurement manager's frustration with his growing list of approvals. A few days later, as he read through the departments' suggestions for improving their workflows, he found the same question coming up again and again. Customer service wanted to stop seeking a supervisor's approval for certain routine inquiries. Procurement wanted fewer rounds of sign-off when purchasing familiar materials to established specifications. The operations team wondered whether an engineer on duty really needed to decide what to do about every ordinary alert, even when a standard response was already in place.
These were not employees looking for an excuse to hand their work over to a machine. Quite the opposite: now that the machines had taken so much of the preparation off their hands, the last remaining step had become more conspicuous. A customer service assistant could find the relevant after-sales policy, assemble the history of a customer's correspondence, and draft a thoughtful reply. Yet the moment a request fell even slightly outside the ordinary, it had to lay everything before an employee and wait to hear, “Go ahead with that.” The procurement manager faced much the same distinction. Sometimes he had only to confirm that an item the company had bought for years could be purchased in the usual way. At other times he might spend half a day examining the terms. The system marked both tasks Awaiting Approval. The amount of judgment they demanded could hardly have been more different.
The owner brought representatives from several departments together to understand what they were really asking for. Nobody wanted AI to run the company. Nobody was proposing that an unsupervised AI agent be given control over payments, contracts, or production systems. They were talking about smaller things. Did a question answered a hundred times before really need another supervisor's approval? Could an agent exercise a little discretion over the routine matters that employees had long handled as a matter of course? If an AI agent makes a recommendation and a person simply approves it without changing a thing, what has that approval actually accomplished?
The procurement manager described a recent order for a regularly used material. The quantity, specifications, and supplier were much as they had always been. The assistant had neatly assembled the purchasing records from the past year alongside the latest quotation. After reviewing them, he approved another request for a quote from the existing supplier. There was nothing particularly difficult about the decision, but he had been occupied with another contract and did not click Approve until the following day. It was not important enough to trouble the owner, nor did he regard it as a major exercise of managerial judgment. He had simply made an ordinary decision within the responsibilities of his job. If dozens of similar requests were arriving every day, it was natural to wonder whether an agent capable of assembling the evidence so thoroughly might also shoulder some of those decisions.
There was nothing unreasonable about wanting that. Companies introduced AI into their workflows partly to save time. If a machine could complete nine-tenths of the preparation, but every minor matter still had to wait for a final nod from the same handful of people, the company would certainly see benefits. It might not, however, gain the smooth-running operation it had imagined. Employees had not suddenly become unwilling to take responsibility. They were beginning to ask whether every decision really had to be made, one by one, by the same person who had always made it.
The Company Had Never Asked Its Owner to Decide Everything
Being able to carry out complex tasks is not the same as possessing the judgment of an experienced employee. There is still a considerable distance between the two.
The discussion reminded the owner of the company's early days. There had been fewer purchases to make and fewer customers to serve. Most questions could be settled simply by calling across the office to him. If someone wanted to buy a computer, he would ask what it was for. If a customer made an unusual request, he could glance through the correspondence and give an answer. People thought he was quick at making decisions because, in those days, he knew almost everything going on in the business. As it grew, he discovered that the more decisions he tried to keep for himself, the longer everyone else had to wait. A company could not grow on one person's time alone.
Responsibilities were gradually assigned to different departments. Within an agreed scope, the procurement manager could decide how to arrange routine purchases. The customer service manager could resolve familiar after-sales disputes. Operations engineers could perform day-to-day maintenance under their assigned responsibilities. Major contracts, large financial commitments, and changes affecting the entire production operation still required more senior involvement. The arrangement was imperfect and often needed adjustment, but nobody regarded it as an abandonment of management. On the contrary, recognizing that different decisions required different experience and authority was what allowed more people to work together without sending every question back to the owner's desk.
At the meeting, the owner recalled a procurement employee he had once supervised. When the young man was new, he had to ask his manager to look over each unfamiliar supplier. As he learned the business, he became able to handle the same situations independently. His supervisor had not woken up one morning and decided to trust him completely. Trust had developed through working together: the supervisor saw that the employee knew which conditions to check, asked questions when he encountered something unfamiliar, and understood the limits of his responsibility. What emerged was not a vague assurance that he could now be left alone, but a division of work both people understood.
The customer service manager picked up the point. Even before AI, she said, nobody had expected every outgoing email to carry a manager's signature. Common questions had standard answers, and frontline staff dealt with straightforward explanations themselves. Matters involving substantial sums or exceptional promises were referred upward. The business did not operate on the principle that the owner must see everything. It entrusted routine judgments to people who knew the work and reserved unusual, complicated, or potentially consequential decisions for those better placed to make them. Some checks had long been carried out by ordinary software following explicit rules: confirming that required fields were present on a form, for example, or stopping a duplicate submission. Nobody took the absence of a human approval for each such check as evidence that the company had lost control.
Why, then, did everyone expect AI to stop and wait for a person at every turn? The question lightened the mood in the room. Perhaps, the procurement manager suggested, they were simply accustomed to thinking of machines as tools and were uncomfortable letting tools exercise judgment. Even as he said it, he knew the explanation was incomplete. Older software made decisions of a kind, and people were not invariably more reliable than machines. The important difference was probably not whether anyone called an agent a “colleague.” It was whether the company understood what the agent could judge, what information it relied on, and when there was reason not to trust its judgment.
There was another distinction that was harder to overlook. Experienced employees were not given greater independence merely because they had made good decisions in the past. They could be trained. They could explain their concerns. Faced with a task whose responsibilities were unclear, an employee might consult a colleague or hold off because something felt wrong. An AI agent could be designed to raise questions or ask for assistance, but the company could not simply assume that it possessed the caution a person developed through professional experience, circumstances, and an awareness of responsibility. Being able to carry out complex tasks is not the same as possessing the judgment of an experienced employee. There is still a considerable distance between the two.
None of this made the owner dismiss the idea of delegating more decisions. If anything, he wanted to examine it more carefully. The company had always functioned by distributing responsibility for judgment. AI had brought an old problem back in a new form: when a new participant in the workplace becomes capable of handling more tasks, how much discretion should the organization give it? They had never answered that question about new employees by saying either “everything” or “nothing.” Perhaps they should resist those answers now, too.
What They Were Willing to Hand Over—and What They Weren't
For the first time, he saw clearly that he wanted two opposing things: for the machine to stop asking permission for every little decision, and for it not to act on its own when he had doubts.
The conversation soon shifted from whether to which decisions. The customer service manager offered two examples. One customer wanted to know how to track the delivery of an ordinary product; the answer was available in the company's published service information. Another wanted a substantial refund and claimed that a salesperson had made a special promise. In the first case, everyone was comfortable with an assistant finding the right information and replying under the existing rules, without waiting for a supervisor to say yes. In the second, even after the assistant had assembled the records, the manager wanted to understand the circumstances before reaching a decision herself. What mattered was not simply whether the reply was well written. She also had to establish whether the company had actually made the commitment the customer described.
The procurement manager laid out two proposals that looked superficially similar. One was to replenish routinely used supplies against an existing requirement, with a familiar supplier, stable quantities, and a predictable price range. The other came with a lower unit price but required the company to buy far more inventory than it normally needed. The machine could calculate unit prices and total costs quickly. It could also identify patterns in past orders. But whether it made sense to stockpile materials depended on future orders, cash commitments, and the space available in the warehouse. There might be good reasons to delegate more of the first decision. The second should not be accepted simply because a report made its recommendation to purchase look persuasive.
An operations engineer drew the same distinction in his own work. A minor change in a server's operating metrics might call for collecting logs and opening a ticket under an established, tested procedure. Altering a live production system during a busy business period was another matter entirely. The first generally had limited consequences, and a mistake could often be caught through another check. The second might disrupt other people who were relying on the system at that moment. Even if both recommendations said Act immediately, identical wording did not make them the same kind of decision.
The finance manager had been listening closely. When it was her turn, she did not offer a more dramatic example. She asked a simple question: “If an AI agent decides that a payment is justified, are we prepared to make that payment?” Someone said it depended on the amount, the contract, and the original payment schedule. Someone else pointed out that even a small payment deserved scrutiny if the recipient or the circumstances of the transaction had changed. Slowly, the group saw that they were not opposed to agents making decisions. They were opposed to handing over authority without first distinguishing one kind of decision from another.
One employee suggested an appealing approach: start with the tasks that had rarely gone wrong in the past and let the assistant decide those automatically. Several people liked the idea. The company had years of records showing which decisions were repetitive and generally predictable, and which demanded more experience. But the procurement manager reminded them that a hundred successful transactions did not guarantee that the conditions would be the same on the hundred and first. The customer service manager added that a standard reply might work perfectly well in ordinary cases and still be wrong for a customer facing unusual circumstances. Past performance was valuable evidence. It could not tell the company everything about situations it had yet to encounter.
By then the owner was beginning to see that people were not always worried about the machine making a calculation error. Sometimes a proposal made perfect sense on paper and the company still had good reasons not to act on it. The lowest-priced materials might not be the best purchase at that moment. Agreeing to a customer's request might settle one complaint while creating a commitment the company could not sustain. Making a decision was not merely a matter of selecting whichever answer received the highest score. A decision had to take account of the circumstances in which the business would live with its consequences. Reality did not stand still just because a report had been completed beautifully.
As the discussion continued, the employees put their different concerns into words. The procurement manager wanted relief from repeated approvals for routine matters so that he could spend more time studying important contracts. The finance manager worried that, once an agent began making judgments on people's behalf, it might become harder to explain why a payment had been considered acceptable at the time. The customer service manager found herself between the two. She wanted customers to receive answers sooner, but she also knew that once a promise had been made in the company's name, the consequences could not always be undone by recalling an email.
The questions came back to the owner. He could no longer settle the matter by telling everyone to “trust AI.” Nor could he end the discussion with a declaration that all important decisions must remain with people. The first was careless. The second sounded reassuring but offered no definition of important, and did little to preserve the gains automation might otherwise bring. They were searching for a workable distinction: let agents handle ordinary judgments for which they were suited, while reserving choices that depended on wider business circumstances or carried greater consequences for human decision-makers. Such a distinction could not be settled in a single meeting.
As the meeting drew to a close, someone asked whether they could begin with a very small number of routine tasks. The owner neither approved the proposal on the spot nor rejected it. He asked each department to identify exactly which human decisions it hoped to reduce, then consider why those decisions had belonged to people in the first place. At least they could now agree on this much: an agent's ability to recommend a course of action did not mean the company was ready to accept every decision it made. And allowing an agent to exercise some judgment did not allow the organization to hand over responsibility for the consequences.
Leaving the meeting room, the procurement manager glanced at his phone. Several routine purchasing approvals were still waiting, with a more complicated contract somewhere behind them in the queue. He hoped the day would come when he no longer had to approve every small matter, leaving him free to think properly about work that needed his attention. But when he remembered that complicated contract, he was glad no one had just announced that every decision could now be automated. For the first time, he saw clearly that he wanted two opposing things: for the machine to stop asking permission for every little decision, and for it not to act on its own when he had doubts.
For the moment, the company's discussion ended there. Nobody had discovered a solution that would settle every question at once, and nobody wanted to send AI back to doing nothing but organizing documents. They were seriously considering letting these tireless new colleagues make some decisions for themselves. One question remained unanswered, however. Even if an agent had no malicious intent—even if it diligently followed the task as it understood it—could it still make the kind of decision a hardworking but inexperienced employee might make, leaving the company to regret the result?
Chapter Three: An “Insider” Without Malice
Decisions That No Longer Had to Wait
Trust rarely begins with a solemn declaration. More often, it grows over a succession of quiet afternoons when no one has had anything to worry about.
After that discussion, the owner did not immediately hand more work over to the agents. Instead, he asked several departments to identify the routine tasks they knew best, particularly those whose requirements seldom changed. For each one, they were to explain who had previously made the decisions and what might happen if something went wrong. Procurement proposed replenishing regularly used supplies. Customer service selected a few categories of small after-sales requests with established procedures. The operations team suggested day-to-day maintenance that would not involve changes to production systems. The lists went back and forth, acquiring amendments and deletions, until considerably fewer items remained than anyone had first expected.
The procurement manager had initially wanted the assistant to handle inquiries, price comparisons, and orders for all commonly used materials. Halfway through drafting his proposal, he crossed out the words all suppliers. Some suppliers had worked with the company for years, with fairly stable specifications and delivery arrangements. Others appeared only occasionally, and their terms varied from one quotation to the next. In the end, he agreed that the assistant could choose among established suppliers for a narrow range of repeat purchases and place orders through the company's existing purchasing process, provided the amount remained within an agreed limit. Larger purchases, unusual contracts, and payments would still be handled by the appropriate people. The agent had not been told, “Purchasing is yours from now on.” It had been given a carefully negotiated piece of the job.
The first week of the trial went almost disconcertingly well. When the warehouse reported that a supply of packaging materials was running low, the purchasing assistant checked previous orders and inventory requirements, then placed a replenishment order for the usual quantity. The procurement manager did not see it until he reviewed the day's records before going home. The materials arrived on time, the price was as expected, and the warehouse had no reason to chase anyone. Under the old arrangement, the assistant would have prepared a recommendation and waited until the manager could interrupt his other work to approve it. This time, a small, ordinary task had simply been completed.
The customer service manager noticed the difference, too. A handful of low-value refunds that met clearly defined conditions no longer joined the queue for a human response, and customers received their resolutions sooner. Disputed requests still went through the usual checks. In operations, engineers found that some routine inspection tickets they once had to open and clear one by one were now being handled under established procedures. The duty logs were, if anything, easier to follow. The company had not abandoned review, much less put machines in charge of the business. But people whose days had been repeatedly interrupted by minor approvals finally had uninterrupted time for work that demanded more thought.
The procurement manager was particularly pleased. He carried a contract he had been putting off into a small meeting room and read several pages of terms he had never found time to examine properly. Not once that afternoon did a replenishment request call him away. He even began to wonder whether he had been too cautious. If a task was familiar enough, why should the machine always have to stop just before the last step? The evening summary contained a simple entry: “Today's routine orders completed.” That kind of uneventful day was precisely what the company had hoped for when it introduced AI.
As the days passed, there were more afternoons like it. Employees stopped interrupting their manager about every small order, and he gradually stopped checking every completed transaction. If someone asked why the assistant had chosen a particular supplier, it could produce a plausible account of past prices, delivery performance, and required quantities. The explanations had records to support them, and people became increasingly willing to devote their attention to the exceptions. Trust rarely begins with a solemn declaration. More often, it grows over a succession of quiet afternoons when no one has had anything to worry about.
What the Quotation Didn't Tell Them
There was one important thing missing from the evidence: the world had moved on, while the agent's understanding of it was still anchored in an earlier document.
The trouble began with another entirely ordinary replenishment order. The warehouse needed a batch of packaging materials the company used regularly. The quantity was modest and fell well within the limits of the trial. As usual, the purchasing assistant looked through available quotations and delivery records, then compared two suppliers with whom the company had worked for years. It chose the one offering the lower price and a history of reliable deliveries, and placed an order through the agreed process. Given the information the system could access at the time, there was nothing obviously unreasonable about its choice.
The problem was the price list it had used. That document had been saved in the shared folder the previous month. It listed the specifications, unit prices, and contact details clearly enough, but said nothing to establish that the prices were still valid. A few days earlier, the supplier had emailed the procurement department an updated quotation. The price of one material had gone up, and the lead time had grown longer. The message was still sitting in an employee's work inbox. It had never entered the collection of records the assistant could search. The agent had not invented a price or secretly selected an unfamiliar supplier. It had treated the last price it could find as a price the company could still use.
The next morning, the supplier replied. It could no longer accept the order at the old price, and its current production schedule would not support the original delivery date. The company could confirm a new order under the revised terms, the supplier said, but if the materials were needed urgently, it might be better to look elsewhere. The buyer who received the reply was not especially alarmed at first. Orders were revised all the time. He sent his manager a message asking him to take a look. Only when the warehouse explained that the packaging was needed for a batch of customer orders did he realize that a routine purchase had run into a decidedly inconvenient deadline.
When the procurement manager opened the record, his first question was not who to blame. “Why did it choose this supplier?” he asked. The assistant's explanation seemed complete: the two companies had worked together repeatedly, delivery performance had been reliable, and the quoted price was lower than the alternative. Each point could be traced to a record. The manager followed the link to the price list and felt a jolt when he noticed when it had been saved. He then found the revised quotation a colleague had forwarded to him. The change had been known somewhere within the procurement department. It simply had not been part of the information used for this decision.
“It didn't get the numbers wrong,” he told his colleague.
The colleague nodded. “So what do we do about the order?”
That was a more pressing question than whether the machine had miscalculated. They needed to contact the other supplier and find out whether it could deliver sooner. The original order would have to be cancelled or renegotiated, and the warehouse would have to adjust its preparations. In the end, the company got the materials, but paid extra for expedited shipping. Several employees also had to rearrange their work at short notice. The loss was hardly enough to threaten the business. Still, the additional money had been spent, and the time had been lost.
What made the mistake uncomfortable was the absence of a convincing villain. The supplier had not acted maliciously. No employee had instructed the agent to do something plainly improper. The assistant had not shirked its task. It had done the work assigned to it, made a choice within the authority it had been given, and left behind records that could be examined. There was one important thing missing from the evidence: the world had moved on, while the agent's understanding of it was still anchored in an earlier document.
Later, the procurement manager thought about how easily the same thing might have happened with a new employee. A junior buyer could have found the price list, confirmed that the supplier was approved, seen that past orders had gone smoothly, and placed another order on the same terms. A more experienced colleague might have noticed that the document was a month old and suggested checking whether anything had changed. But not every experienced employee would remember to ask, and that did not make human judgment invariably more accurate. The problem was not that a machine had committed some peculiar error no person would ever make. It was that knowledge the company had maintained through habit, passing remarks, and scattered emails had failed to reach the person—or system—now making the decision.
When the owner heard what had happened, he did not immediately call off the trial. He asked three questions instead. What had the incident cost the company? What information had supported the decision at the time? And how would they judge the same action if an employee had taken it using the same records? Once everyone stopped looking for a single party on whom to pin the mistake, the circumstances became easier to see. The agent had made a poor decision. The updated quotation had not reached the information source it relied upon. And the decision to let routine orders proceed without individual human approval had been made by the company itself, after discussion. All three had contributed to the outcome. “AI is unreliable” was not an adequate account of what had happened.
Around the same time, customer service ran into a smaller problem. A customer's message mentioned both a return and a replacement part that an earlier service representative had promised to send. The assistant classified it as a routine refund request and handled it according to the standard returns process. Only when the customer contacted the company again did staff realize that the unresolved promise to send the part was what mattered most. An employee sorted it out through another conversation, and the matter did not become a serious dispute. Even so, the customer service manager saw the lesson: putting a message into a familiar category did not mean the system had understood the customer's circumstances. The operations team encountered a different near miss. A maintenance recommendation had overlooked a temporary change in the hours when a business department needed a system. Fortunately, alterations to production systems still required an engineer's confirmation. The recommendation was corrected before anyone acted, and there was no outage.
The owner did not turn these incidents into a parade of examples proving that machines were dangerous. They involved different departments and had different consequences. Yet they shared something worth noticing. The more a task resembled a familiar routine, the less anyone felt the need to ask another question—even though reality might have changed in precisely the places no one was watching. Sometimes new information had never reached the relevant records. Sometimes it was available but had been sorted into the wrong category. And sometimes the rule was perfectly sound for ordinary circumstances but did not cover what had happened that day. Treating all of this as a question of how intelligent the model was would leave much of the organization's part in the failure unexplained.
After Trust Takes Hold, Who Still Thinks to Ask?
A machine may perform difficult tasks well and still get an easy one wrong. Getting something right many times does not make the next decision automatically right. The same is true of people.
In the days that followed, the procurement manager often thought about a young colleague he had once trained. The young man had been diligent from the beginning and exceptionally careful when compiling quotations. On one occasion, he had even spotted a small freight-charge discrepancy buried in a contract. Eventually, the manager had trusted him to handle routine purchasing on his own. Then, shortly before a stocktake, he discovered that the employee had ordered too much of a particular material, basing the quantity on past consumption without noticing that the production schedule had changed. The young man had not been careless about his work. He had simply missed a change in circumstances. The manager had spent a long time explaining why familiar orders still required a look at the current situation. What had been right last month was not necessarily right this month.
Now the same lesson had to be expressed in a different way. With an employee, a manager could ask whether he had heard the latest news from the warehouse, watch to see whether he understood the difficulty, or recognize uncertainty in the way he answered. With a machine, the company could improve the records it received, narrow the tasks it was permitted to handle, require it to stop and ask when a condition was unclear, and test for weaknesses it had not yet identified. But a human employee might remember the embarrassment of being corrected or know from experience when it was worth asking a colleague one more question. The company could not assume that an agent would naturally develop the same habits for the same reasons. Machines could be improved, certainly. Improvement, however, required deliberate changes supported by evidence, not a hope that they would somehow acquire a human sense of caution.
There was a subtler problem. The successful orders were not meaningless. They showed that the trial genuinely worked in many ordinary circumstances, and they explained why people had become comfortable giving the assistants more responsibility. But repeated success also changed how employees perceived risk. In the beginning, the procurement manager had opened the records every day to see which supplier the assistant had chosen and what price it had accepted. A few weeks later, he usually looked only when something had been flagged. No one could remain as vigilant over every minor task as they had been on the first day of a trial. If they did, there would be little point in delegating routine work. Yet it was precisely that reasonable trust which allowed an omission to remain unnoticed when nothing seemed out of place.
Someone suggested updating the supplier's quotation in the shared records. Surely that would prevent the same thing from happening again. The procurement manager agreed that the information needed correcting, but was less willing to accept the promise that followed. This time, a supplier had changed its price. Next time, the warehouse might alter its requirements at short notice, or a customer's request for earlier delivery might change which order took priority. Correcting a known error mattered. Updating one document, however, did not guarantee that every relevant change in the future would arrive promptly or in full. People missed things; machines could also make decisions with incomplete information. And a machine might apply the same mistaken understanding to many similar tasks in a very short time.
The finance manager observed that at least they had not allowed the assistant to make the payment itself. The procurement manager resisted the easy conclusion that everything was fine because a person still controlled the money. The order had already been sent. Replacing it and arranging faster transport had already cost the company something. A business decision could have consequences long before the final payment was made. At the same time, he acknowledged that the existing payment review had served a purpose: it had prevented the faulty purchasing decision from automatically extending into the transfer of funds. The experience showed both why the company's division of responsibilities mattered and why the effects of a decision could begin well before what anyone thought of as the last step.
The owner found himself returning to an uncomfortable word: insider. When the company had talked about internal risks in the past, people tended to imagine someone concealing information, breaking rules, or exploiting a position of trust for personal gain. The purchasing assistant was nothing of the sort. It had no private interest to pursue, no grudge to settle, and no reason to invent a motive for harming the company. Yet the error had arisen inside the business, through an ordinary process in which the company had deliberately allowed the assistant to participate. Employees, customers, and the company itself had borne the consequences. Calling the machine an “insider” might be a useful reminder that harm can arise from within a trusted process. But if the word suggested a saboteur acting with intent, it described the situation entirely wrongly.
What troubled the owner most was how unremarkable the task had been. The assistant had not broken into a system or attempted to negotiate an enormous contract unlike anything it had seen before. It had chosen a familiar supplier and placed an order for everyday packaging materials. A machine may perform difficult tasks well and still get an easy one wrong. Getting something right many times does not make the next decision automatically right. The same is true of people. The company had hoped that as its assistants became more practiced, it could rely on them more. Now it had to accept a less comfortable truth: familiarity can help make work reliable, but familiarity and reliability are not the same thing.
Several managers returned to the question of suspending the trial as they talked in the corridor. The procurement manager did not want to shut it down altogether. He remembered those uninterrupted afternoons and knew how much repetitive work his colleagues had been spared. The customer service manager worried that if every mistake led the company to withdraw all discretion, the human approval queues would soon fill up again. Yet none of them could confidently say that updating a quotation and adding a few instructions was enough to justify handing the agents an ever wider range of decisions. Both sides had reasons worth taking seriously. Neither could simply make the other side's concerns disappear.
That evening, the owner went through the trial records. Most tasks had been completed successfully. The few that had gone wrong were marked with notes on what had happened afterward. The figures were not lying: productivity had improved, and employees had genuinely enjoyed some relief. Neither was the extra shipping charge a fiction. The mistake had already found its way into the company's accounts. He was beginning to understand that he was dealing neither with useless machines nor with treacherous new colleagues. These were participants in the company's work who had become remarkably capable, yet could still make a poor judgment at a particular moment.
No final decision was made that day. Some people proposed putting important decisions back in human hands. Others wanted to continue the trial while finding a safer way forward. The owner asked everyone to write down what had happened and what concerned them, and they agreed to meet again. Around the office, the purchasing assistant was still preparing the next day's replenishment recommendations, and the customer service assistant was still answering ordinary inquiries. The mistake had not brought their work to a halt. It had, however, forced the people responsible for the company to reconsider how far those assistants ought to be allowed to go. If every judgment had to return to a person for checking, the old queues for approval might simply reappear. But if the company continued to delegate decisions, who could promise that the next apparently routine choice would not go wrong?
Chapter Four: How Many Times a Day Must Someone Say Yes?
Putting Decisions Back in Human Hands
But that sense of reassurance depended on something easy to overlook: there could not be too many requests, and the people responsible for reviewing them had to have enough time to do it properly.
A few days after that discussion, the company arrived at a provisional decision. The owner brought together the procurement manager, the heads of finance and customer service, and the operations engineer responsible for the production systems. This time, he did not ask whether the machines could be trusted. Instead, he asked each person to identify what had genuinely become faster during the trial and what kinds of mistakes had forced their departments to spend time putting things right. The procurement manager mentioned the outdated quotation. The finance manager brought up the extra shipping charges the company had incurred to meet its delivery commitments. The customer service manager recalled the request in which a promise to send replacement parts had been mistaken for an ordinary refund. All of these incidents had been dealt with. Nobody pretended the company had been through a crisis.
The owner concluded that a few mistakes were no reason to abandon the trial, but neither could a largely successful trial justify ignoring the risks that remained. For the time being, the company would take back decisions that were especially vulnerable to changes in real-world circumstances. A purchase involving a revised supplier quotation or altered order terms would need the procurement manager's confirmation. Customer service requests involving unusual refunds or special promises would remain matters for employees to judge. Any operation that changed the state of a production system would continue through the established approval process. Routine tasks with clear rules—format checks and document preparation among them—would not all be returned to human hands. The company had not declared that everything a machine did must henceforth be signed off by a person. It had, however, placed some of the decisions it had begun to delegate back on people's desks.
The decision met with more support than the owner had expected. The procurement manager was reluctant to give up the time he had gained, but he knew that an earlier check of the revised quotation might have saved the company the cost of expedited shipping. The finance manager was particularly keen to have changed circumstances trigger another review. Checking an amount just before payment, she pointed out, was not the same as establishing the correct price and delivery terms before placing an order. The operations engineer felt more comfortable as well. A machine could still prepare a recommended course of action, but whether a production system should be altered at that particular time would no longer be hurried along by an alert that looked routine.
For the first few days, the new arrangement did bring a sense of relief. There were more items awaiting attention, but managers could at least see which decisions were waiting for them. One day, the procurement manager received a notice that a material's quoted price had passed its validity date. Rather than approve the purchase, he asked an employee to confirm the current terms with the supplier. An updated quotation arrived promptly, and the order went out the following day. Nothing was lost, and nobody thought to praise the review. Still, the manager felt reassured. There was a reason approval procedures had survived for so many years: some decisions deserved a pause before they took effect, long enough for someone who understood the business to look at them carefully.
But that sense of reassurance depended on something easy to overlook: there could not be too many requests, and the people responsible for reviewing them had to have enough time to do it properly. At first, everyone could still open each report and consult the underlying records. As more departments began giving their assistants additional work, that breathing room gradually disappeared. Proposals that once took half a day to prepare could now arrive several at a time before lunch. Nor did an assistant have to wait quietly when it encountered uncertainty. It could diligently flag the question and forward it to the designated reviewer. Every pending decision appeared to have a clear destination. The trouble was that requests reached their destinations without regard for what else the people receiving them had to do that day.
By Friday afternoon, the difference in pace was hard to miss. The procurement manager had planned to clear the week's routine approvals before going home. Just as he was about to leave, several more documents appeared. Some concerned materials needed first thing on Monday; others were merely alternative proposals prepared ahead of time. Their notifications sounded much the same. Leave them all until Monday, and someone might be unable to start work while waiting for an answer. Read them all now, and he would have to abandon something else he had planned to do. The machines had finished their part early, which ought to have given people more room to think. Instead, it also forced them to decide sooner which matters could no longer wait.
A Neat Table and the Reality Behind It
Something a machine can prepare in minutes may take a person far longer to verify.
One Wednesday, the procurement manager sat down to review a new contract when a succession of alerts arrived on his phone. He opened the shortest replenishment proposal first. At the top of the screen were the words Recommended for Approval. Beneath them came an orderly list of reasons: a stable history with the supplier, a price within the permitted range, and inventory approaching its replenishment threshold. The supporting documents were neatly arranged. His first instinct was to click Approve and remove one small task from the queue. But the trouble with the outdated quotation was still fresh in his mind. He scrolled further down and discovered that the inventory figures came from the previous day's records. The warehouse had changed its materials-issue schedule that very morning.
No mistake was made this time. He called the warehouse, confirmed the quantities, and sent the proposal back for revision. The exchange consumed time he had intended to spend on the contract. By the time he hung up, two more requests were waiting for review. Looking at the beautifully organized table, he found himself thinking how little weight the word review seemed to carry on a screen. The report had already assembled the reasons, and the recommendation looked sensible. Yet to decide whether it was still sound today, he had to speak to people beyond the table. Something a machine can prepare in minutes may take a person far longer to verify.
Customer service faced much the same problem. An assistant could bring a customer's message, previous correspondence, after-sales terms, and a proposed reply together on one screen. For many ordinary inquiries, that was enough to let an employee respond quickly. But when a request involved an exceptional promise, the manager needed to know who had spoken to the customer, whether the promise had formed part of an official response, and whether the proposed resolution might create further expectations. On one occasion, a summary described a case as Customer requesting an additional refund. When she opened the original exchange, she discovered that the customer was less concerned about money than about replacement parts the company had promised but still had not sent. The summary was not entirely wrong. It had simply pushed the most urgent part of the problem into the background.
She returned the request and asked an employee to check the status of the replacement shipment first. After that, she made a habit of reading a little more of the original correspondence before deciding. It was a good habit, but time remained finite. Several exceptional requests were already in the queue before lunch. After lunch, an employee came to discuss a genuinely complicated complaint. No sooner had the manager sat down than another alert warned that some pending requests were approaching their internal deadlines. The system was not telling her to sign carelessly. It was faithfully reporting that work had yet to move forward. Behind every notification, however, were colleagues and customers waiting for a response.
The finance manager was less inclined to regard all of this as a new problem. Even before AI, she pointed out, her department regularly received applications whose paperwork was complete but whose circumstances were not fully explained. An invoice, a contract, and a payment schedule might agree with one another without proving that the transaction still fitted the company's plans. The need to check had always been there, and a signature had never made human judgment infallible. What had changed was the pace. In the past, other departments could produce only so many applications, leaving finance some time to ask questions. Now the paperwork came together much faster, while the phone calls, explanations, and waiting that made verification difficult had not become any shorter in proportion.
One payment request became caught in precisely that kind of delay. Procurement believed the documents were complete. Finance, having examined the contract terms, wanted to know whether all the goods had been accepted. The warehouse pointed out that part of the shipment was still in transit and needed procurement to clarify its status. The assistant gathered the exchanges among the three departments and even listed the questions each still needed answered. But it could not certify on the warehouse's behalf that goods had arrived, or confirm for the procurement manager that an unfinished delivery was complete. Faced with the carefully prepared summary, the finance manager could only put the payment on hold. That pause was not evidence of inefficiency; essential information was still missing. To the system, though, it was simply another request occupying a place in the approval queue.
That evening, as the procurement manager reviewed his work log, he noticed that the tasks he had approved ranged from ordinary matters settled in seconds to decisions that had required contracts, inventory records, and conversations with other departments. On the screen, each occupied a single row of exactly the same width. Once completed, each acquired the same green Approved label. Nothing was malfunctioning. The system recorded the status of a decision, not how much care had gone into making it. The manager wondered whether someone examining the log the next day would assume that every green mark represented an equally thorough review.
And the number of requests was not the only source of fatigue. After examining many similar documents in succession, a person could begin judging the next one by how familiar it looked. With the first few, the manager would check dates carefully, verify sources, and ask whether anything had changed. By the afternoon, when he encountered the same layout and the same set of reasons, his eyes were more likely to go straight to the machine's recommendation. If it matched what he had seen before, he had to make a conscious effort not to confuse looks fine with has been checked. This was not a matter of one particular employee being lazy. When many judgments had to be made in a single afternoon, attention wore thin. Even maintaining the discipline to be cautious became work in its own right.
After Clicking Approve
Effective oversight requires the right person, the information that person needs, and enough time to think. Adding someone's name to a workflow does not bring those conditions into existence.
At the month's final business meeting, the owner asked the department heads how the revised approval process was working. Procurement reported that orders involving price changes were being flagged promptly. Customer service had caught and improved several potentially troublesome replies before they went out. Operations confirmed that changes affecting production systems were still being reviewed against actual operating conditions. Nobody disputed the benefits. If human review were dismissed as nothing more than an old-fashioned obstacle to efficiency, the mistakes prevented by those checks would be difficult to explain.
As the meeting continued, however, another set of facts emerged. Managers were repeatedly pulled away from important work to deal with approval requests. Some applications remained in the queue for long periods because they needed confirmation from more than one department. Even relatively straightforward matters sometimes took several rounds of notifications before someone finally clicked Approve. To reduce the delays, departments began placing prominent recommendations at the top of their system-generated summaries. Reviewers learned to concentrate on a few key fields. These changes sometimes did save time, but they also made people increasingly dependent on how the underlying information had been selected and presented. A reviewer still had the opportunity to say no. Whether that opportunity was meaningful depended, in part, on knowing what might warrant a no.
One procurement employee pointed out another detail. Clicking Approve required very little explanation. Returning a request, by contrast, usually meant giving a reason, notifying the applicant, and sometimes arranging a new time for the matter to be dealt with. There was nothing inherently wrong with that difference: if work was being sent back, someone needed to know what to change. But after a day spent processing document after document, the easiest choice and the best-supported choice might no longer coincide. An employee was not entitled to approve something carelessly just because time was short. The difficulty was that a workflow could not automatically protect a tired person against that temptation. In fact, resisting it might demand a little more effort every time.
The owner opened a completed approval record. It showed the request, when it had been submitted, who had approved it, and at what time. Everything was clear. He turned to the procurement manager. “If we need to look into this later, at least we'll know who approved it, won't we?”
“Yes,” the manager said.
“Can we tell from this record whether he read the original quotation? Or whether he knew the warehouse had changed its plans afterward?”
The manager looked at the screen. The record itself could not show that, he said. They would have to find the attachments and the correspondence from the time. Even with those in hand, there might be no way to establish how much the reviewer had actually understood.
That did not make approval records worthless. Knowing who made a decision and when was a basic part of managing an organization. A signature indicated that someone had acted in an assigned role, and it gave investigators a place to start if the decision needed to be examined later. The problem was that the record proved less than people were accustomed to assuming. It could show that a request had been approved. A name and a timestamp alone could not prove that the relevant information had been read closely, that important conditions had been verified, or that the conclusion was correct. Some of the company's reassurance came from the clarity of the record. The quality of the judgment still lay in a process the record could not fully reveal.
The procurement manager remembered a small incident from a few days earlier. He had been about to take a call from a supplier when a routine purchase request appeared on his screen. He skimmed the summary. The materials and amount were comfortably within familiar limits, and his hand was already moving toward Approve. Then he wondered whether the proposal ought to be checked against the latest inventory figures. He closed the window, took the call, and contacted the warehouse afterward. The recommendation turned out to be sound. Had he approved it immediately, the business outcome might have been exactly the same, and the system would have recorded his name in exactly the same way. Only he would have known that the two approvals rested on different degrees of checking.
He also thought about the contract he had been trying to finish. Over the past few days, the time he could give it had been broken into scraps of ten or fifteen minutes. Each time he returned to his desk, he had to remind himself where he had left off and which terms still needed clarification. The approval notifications had never explicitly ordered him to put the contract aside. They merely kept reminding him that other people's work was waiting for his reply. An organization could reasonably insist that important matters receive human scrutiny. It also had to recognize that the attention spent on that scrutiny came at the expense of other work. Hiding those costs behind an approval button did not make them disappear.
The fact that this small incident had caused no loss only made it more unsettling to him. A management system could not be built on guesses about how carefully someone had looked at a screen before clicking a button. Nor could it expect every employee to maintain perfect concentration at every moment. People were not there simply to serve as approval buttons for machine-generated work. They had contracts to negotiate, customers to meet, warehouses to visit, and colleagues who needed their help. Asking someone to examine a matter that genuinely called for experience and judgment was a way of respecting that person's abilities. Continually loading the same person with requests that did not all belong on their desk—and could scarcely be reviewed properly in the available time—was not necessarily better oversight.
The owner began to see how simply he had understood the problem. A machine made a mistake, so the company added a human approval. The approval was recorded, so everyone felt safer. Effective oversight requires the right person, the information that person needs, and enough time to think. Adding someone's name to a workflow does not bring those conditions into existence. Some high-stakes or unusual matters plainly deserved more time; when necessary, a decision should be delayed rather than rushed through. Yet if requests of very different importance all converged on the same point, the limited time available for judgment could be consumed by matters that scarcely needed it.
The company began discussing how to separate the requests again. Which checks could still be performed under explicit rules? Which decisions belonged with someone familiar with a customer's circumstances? Which required procurement and finance to examine the evidence together? And who should assess proposed changes to production systems against their actual operating state? They did not settle these questions at the meeting. The procurement manager worried that splitting the queues would make it harder to assemble information that crossed departmental boundaries. The finance manager argued that not every payment-related decision could be reduced to checking figures. The operations engineer reminded everyone that while he understood the condition of the systems, he could not always know about last-minute changes to the business departments' plans.
As the meeting ended, the owner did not ask anyone to make the approval rate look better. He said only that if certain matters truly had to be reviewed by people, the company first needed to work out which people, what information they should be given, and how they could realistically find the time to examine it. The procurement manager thought there might finally be a chance of shortening his list of pending approvals. But he also sensed a harder question taking shape. Once review responsibilities had been divided among different roles, who would be accountable for the results? And if a person had neither the time nor the information needed to check properly, what should the company understand by the yes that person had signed?
Chapter Five: Who Checks, and Who Is Responsible?
Every Task Finally Had Someone Responsible
He could tell whether an operation might briefly interrupt a service. What he could not do was decide, on behalf of the people who used that service, whether the interruption was acceptable.
The first visible change after the business meeting was a column of names on the company's task-management screen. Until then, purchasing, finance, customer service, and operations reviews had tended to accumulate in the queues of a handful of managers. Whoever had a spare moment would take an initial look; matters involving more than one department would be passed along to someone else. Now each category of request had a designated role. Routine purchasing went to procurement. Evidence supporting payments went to finance. Exceptional after-sales requests went to the customer service manager, while recommendations affecting running systems went to operations. The owner hoped this would at least put an end to documents circulating from inbox to inbox without anyone being able to say exactly whose response was needed.
On the first morning under the new arrangement, the procurement manager felt some relief. Several payment-related requests had disappeared from the queue of purchasing proposals, and changes to supplier quotations that genuinely needed his attention were now easier to see. A buyer had taken over reviews of routine replenishment orders, and the warehouse had a regular point of contact. Previously, an employee trying to find out who should handle an order might have spent half the morning asking around in a group chat. Now the answer appeared on the screen. For a company that was hardly overstaffed, knowing where a task belonged was a tangible improvement.
But giving a task a name did not make the work fit neatly inside departmental boundaries. That afternoon, a buyer received a replenishment proposal prepared by the assistant. The supplier's quotation was current, and the quantity fell within the agreed purchasing limits. What caught his attention was the requirement that the materials arrive by the following week. The warehouse records offered no explanation for why they were suddenly needed sooner. He asked the warehouse and was told that the business team had changed the order of upcoming deliveries. When he contacted that team, its staff said the actual date was still awaiting confirmation from the customer. The recommendation contained no obvious calculation error, and the order was not especially large. Yet the buyer could not answer the simplest question of all: was placing this order now really what the company needed?
The system identified him as the procurement reviewer, and it was his inbox that received the overdue reminders. He could check the supplier, the price, and the usual purchasing terms, but he did not know the customer's delivery schedule. He could send the proposal back, but he had no authority to set a deadline on behalf of the business team. He referred the matter to his manager, who contacted the manager responsible for deliveries. After making its way around the company, the order finally came to rest under Awaiting Business Confirmation. The buyer looked at the red overdue notice beside his name and felt a little aggrieved. He was not refusing to do his job, nor was he deliberately holding anyone up. Some of the answers simply belonged to people in other roles.
The situation in finance was more delicate still. Finance employees had always checked contracts, acceptance documents, and amounts when payment requests arrived. AI could now assemble the paperwork more quickly and identify discrepancies in the figures. Under the revised division of responsibilities, payment review was assigned explicitly to finance employees, in the hope that requests would spend less time waiting between departments. One application had a contract amount, beneficiary details, and accounting records that all agreed. Its supporting summary, however, stated that the relevant delivery had been completed as planned. The employee searched the attachments and found only a progress summary submitted by the business team. There was nothing from the people who had actually received the goods to establish that delivery had taken place.
She sent the application back and requested further evidence. The person who had submitted it was puzzled. “Hasn't the machine already checked all the documents? What else do you need us to confirm?” She explained that matching numbers did not establish that something had actually happened. Finance could check whether the grounds for payment were properly documented; it could not determine, from a paragraph written in an office, what the warehouse had physically received. Once the business team provided the necessary records, the request moved forward. No loss resulted from the delay. But the incident made her realize that a single review task assigned to her might conceal several facts that different people were responsible for verifying.
The operations engineer encountered much the same difficulty. An assistant recommended a routine maintenance operation that presented no particular technical challenge. In ordinary circumstances, carrying it out overnight would have minimized disruption. But he knew some orders were being rushed through production, and he could not assume that the production team was still working to its usual schedule. He could tell whether an operation might briefly interrupt a service. What he could not do was decide, on behalf of the people who used that service, whether the interruption was acceptable. When someone pressed him to approve the work, he replied, “I can tell you how to do it technically. I can't tell you that stopping the system for a few minutes tonight won't matter to your work.”
Redistributing the reviews had not eliminated them. It had simply moved them from a few managers' desks to a larger number of employees. Some people gained a clearer understanding of their duties. Others saw their names attached, perhaps for the first time, to outcomes they could not judge on their own. For a while, everyone lived with the complications. Every department had its limits, after all. What changed the mood was a new internal notice.
“Whoever Reviews It Is Responsible”
The managers gradually came to see that what employees resisted was not ordinary responsibility for their work, but responsibility whose limits nobody had properly defined.
The notice was only a few paragraphs long. It reminded departments to verify the results submitted by their AI assistants in accordance with their roles, rather than treating a machine's recommendation as an established fact. One sentence stood out: Whoever reviews it is responsible. The intention was probably straightforward. Approval was not supposed to be a formality; employees should not wave something through merely because a machine had produced a convincing summary. The procurement manager had no objection to that principle. After years in business, he too believed that someone who signed off on a decision ought at least to be able to explain what they had checked.
But the response was different when the notice reached the employees' group chat. One buyer asked whether checking a quotation meant he was also responsible for a delivery requirement that the business team subsequently changed. A finance employee wanted to know how she could guarantee the accuracy of a receiving report supplied by another department simply because the resulting payment request appeared in her queue. The operations engineer put it more bluntly: if he warned the business team that maintenance might disrupt production, and the team insisted on going ahead, whose failure would it be if the outcome was bad?
The objections did not erupt all at once. At first, people grumbled over lunch or left a sharp comment in the group chat. Then a buyer handling an urgent order twice asked the business team to confirm the required delivery date and received no clear answer. By evening, someone had called his manager to complain that the order could not be held up any longer. The procurement manager found the employee and asked why it had not been processed. The buyer opened his message history and pointed to one of the unanswered requests. “If all you want me to check is the price, I can sign it now. If you want me to guarantee that we're placing the right order, I need to know when they're actually going to use the materials. I can't get that answer. How am I supposed to take responsibility for it?”
The procurement manager did not reprimand him. The buyer was no inexperienced newcomer, and he was usually willing to take responsibility. It would be hard to justify telling him to skip a necessary question in order to meet a deadline, only to say You signed it if something later went wrong. Yet the manager had difficulties of his own. The warehouse and the business team were waiting for a purchasing decision. He could not personally take over every request that left an employee uncertain. Nor could the company allow every matter involving multiple departments to end with nobody willing to make a decision. If people refused to take responsibility even for the parts they could reasonably control, the work would stop just as surely.
The owner heard about the disagreement the following day. His first reaction was irritation. He had thought the company was merely clarifying responsibilities that already existed. Why had something so ordinary provoked so much resistance? Rather than immediately defend the employees, the finance manager placed a payment application in front of him and began asking questions, page by page. “Who confirms the contract amount? Who confirms that delivery has happened? Who verifies the beneficiary details? If I sign at the end, does that make all of those things judgments made by finance?” Of course not, the owner replied. The business team remained responsible for the facts it supplied. “Then why,” she asked, “does our notice say nothing beyond ‘Whoever reviews it is responsible’?”
The owner looked at the sentence without answering at once. He understood what management had meant. They wanted to prevent a gap in accountability where everyone could say, The machine did it, so nobody is responsible. Human employees had made mistakes long before AI arrived. Organizations assigned duties to different roles so that somebody would check, correct, and explain what had happened. That principle had lost none of its value. But machines now assembled information from many roles into documents that looked complete. The person named as the final reviewer could easily become, in everyone else's eyes, the person who ought to have known everything.
The customer service manager saw another side of the problem. She did not want employees referring every unusual customer message to her, as if only a manager could decide whether someone needed help. In many cases, frontline staff had the complete correspondence and should make a careful decision within their role, then be able to explain it. But she was equally unwilling to have an employee who had seen only the system summary guarantee the truth of a verbal promise a salesperson had made months earlier. In customer correspondence especially, the words I've checked could be understood as the company's confirmation of the entire matter, when the person doing the checking might have seen only one part of it.
The managers gradually came to see that what employees resisted was not ordinary responsibility for their work, but responsibility whose limits nobody had properly defined. A buyer should check the supplier terms within his remit; a finance employee should carry out the checks that properly belonged to finance. A heavy workload did not excuse ignoring a document, and a known discrepancy could not simply be approved away. Yet when judging a proposal required information from several departments, a process that recorded only the last person to click a button could not reasonably place every consequence on that person's shoulders. It would neither help employees review the work properly nor help the company understand what needed to change.
Everything a Signature Cannot Do
When a company assigns a person to review a stream of work produced by machines, it is asking for more than a click. It is asking that person to spend time, obtain information, exercise judgment, and, when necessary, bear the delay and pressure that come with saying no. Writing a name into a form does not suddenly give someone those capabilities if the conditions for using them have not been provided.
The owner did not immediately withdraw the notice. He still believed that work needed to be checked and that responsibilities had to be traceable. Otherwise, the next time something went wrong, every department would simply say it was someone else's problem. He asked the managers to bring in examples of requests that had recently become stuck. Rather than argue about who had complained the loudest, they were to establish, one by one, what the reviewer had actually been asked to examine, what information a sound judgment would require, and who possessed whatever was missing.
The exercise proved harder than anyone expected. Procurement produced a proposal to change suppliers. Its reviewer could assess the quotation, delivery terms, and contract conditions, but needed the business team to explain why the demand had changed. Finance brought back the payment application whose supporting documents had eventually been completed. The payment terms were verifiable, it explained, but evidence that goods had actually been received still had to come from the people responsible for acceptance. Operations presented a maintenance proposal that had not been carried out. The engineer could assess the technical effects, but whether the business could tolerate an interruption depended on the people using the system. As the managers wrote these distinctions on the same sheet of paper, tasks previously compressed into the single word responsibility began to take on different shapes.
The discussion was not always pleasant. Someone from the business team argued that, with review roles already established in procurement and finance, business staff should not have to answer the same questions again after submitting their documents. The procurement manager asked how anyone was supposed to order materials on the strength of a delivery date that had changed without the reviewer being told. The finance manager reminded the room that checking a document for completeness did not make finance the guarantor of every claim made by the department that supplied it. Voices rose, and the owner called a halt. He could see that each department had a point—and that each was also inclined to underestimate the difficulties faced by the others.
He then asked a more pointed question. “So if the information is incomplete, does that mean none of you has to do anything?” The procurement manager said no. Identifying the gap, returning the request promptly, and explaining who needed to confirm what were all part of a proper review. The finance manager agreed. If documents plainly contradicted one another, approving them anyway could not be excused on the grounds that someone else had entered the information. The operations engineer added that if he knew a maintenance operation might affect a running business process, he had a duty to make that risk clear rather than claim ignorance after the event. Their answers reassured the owner. They were not trying to escape responsibility. They wanted it to attach to things they could genuinely understand and influence.
That led to a harder problem. An employee could point out that a recommendation lacked evidence, but could not decide when another department would supply it. Someone could refuse to sign an application they did not understand, yet could not stop more applications from arriving in the queue. A reviewer could question a proposed operation without having the authority to change the range of tasks the assistant would continue to handle. When a company assigns a person to review a stream of work produced by machines, it is asking for more than a click. It is asking that person to spend time, obtain information, exercise judgment, and, when necessary, bear the delay and pressure that come with saying no. Writing a name into a form does not suddenly give someone those capabilities if the conditions for using them have not been provided.
That afternoon, the procurement manager returned to his desk to find that the urgent order was still awaiting confirmation from the business team. He did not instruct the buyer to sign it regardless. Instead, he called and asked for a definite date on which the materials would be needed. The person at the other end explained that the customer had not yet confirmed the delivery schedule; he, too, was unwilling to invent a date. The manager had to acknowledge that waiting was a real part of this order's present circumstances. It should not be concealed behind a signature unsupported by the facts. He told the colleagues concerned to expect a delay. Nobody was pleased, but at least everyone understood why the order had stopped.
By evening, the owner had revised the wording of the internal notice. He did not announce a perfect new policy. He replaced the sweeping statement with something more careful: reviewers were responsible for the checks within their defined duties; where information was missing or a matter fell outside their competence, they should raise it promptly and refer it to the appropriate role. More complicated divisions of responsibility would still need to be worked out case by case. The queues did not suddenly shorten, and employees did not stop disagreeing. But for the first time, some felt that saying I don't know would not automatically be treated as shirking their work.
Near the end of the day, the finance manager passed through procurement and found its manager still discussing the next day's arrangements with his colleagues. She joked that at least nobody had spent the day arguing over whose signature was required. He smiled. They had only untangled the few problems in front of them, he said. Looking at the assistant tasks still updating on his screen, he realized that they had been arguing about who should review the work, what they should check, and how they should account for mistakes. But some earlier decisions had scarcely entered the conversation at all. Why were the assistants being given these tasks in the first place? Why could they produce so many requests for approval, one after another? And why had some business activities already been configured to let them carry work forward on their own?
He did not put those questions to the finance manager there and then. The day had been long enough, and finance could not answer them on its own. Someone switched off the lights in the corridor, while several computers remained lit in an office on the other side. The machines carried on preparing tomorrow's work. The people, meanwhile, were beginning to wonder whether they had spent all this time looking for answers too late in the process. If they only decided who should review the work at the end, without looking back at how that work had been entrusted to the machines, were they asking the right questions at all?
Chapter Six: Who Handed Power to the Machine?
Money That Had Already Been Sent
Finding the check she had failed to make was part of the investigation. But if that was taken as a complete explanation of why the payment had reached the approval queue in the first place, the inquiry would end at the easiest name to find.
The arguments over review responsibilities had finally begun to quiet down. The owner had revised the sweeping instruction that whoever reviewed a request was responsible for it, and departments were getting better at identifying matters that needed confirmation from someone else. The procurement manager no longer pressed employees to sign off on delivery dates they could not verify. The finance manager had begun insisting that payment documents identify the source of their underlying business claims. None of this had made the work effortless. But the company had at least agreed on one thing: putting someone's name into a system did not mean that person knew everything there was to know about a transaction.
Then a payment that had already gone through shattered the fragile calm. The finance manager noticed it while reconciling the company's bank statements. A deposit for a production-equipment modification had been transferred to a long-standing supplier. Yet in another email, the project manager was asking why the supplier had begun preparing custom materials when the modification project was still on hold. Both messages concerned the same transaction. At first, the finance manager thought the payment date might simply have got out of step with the project schedule. But the records told a different story. The bank had not transferred the money by mistake. Nor had someone secretly circumvented the company's procedures. There was a complete payment request in the system, the beneficiary account belonged to the supplier named in the contract, and a finance employee had approved the request in accordance with the established workflow.
The project was intended to add custom components to one of the company's packaging lines. Several weeks earlier, the business team had identified the need, and procurement had negotiated the price and stages of work with the supplier. Under the contract, receipt of a deposit would allow the supplier to begin ordering specialized materials. Later, because the specifications for a customer's order had not yet been finalized, the business manager decided to suspend the modification until the next production-planning meeting. The decision was announced in the project group chat and subsequently noted in the business team's weekly report. What never happened was a formal change to the procurement or finance process that would put the payment on hold. Nor did the supplier receive a clear, timely instruction to pause. When the deposit arrived, it proceeded to prepare the materials under the original agreement.
When the finance assistant assembled the payment paperwork, it found the contract, the purchasing request, the supplier's account details, and the previously confirmed purchasing plan. On that basis, it classified the deposit as a payment that could proceed through the normal process. The assistant did not log in to the bank and transfer the funds itself. Under a workflow the company had configured, however, it could create a payment request awaiting approval, arrange the attachments, enter the beneficiary information, and add the explanation Payment stage consistent with contract terms. The request appeared in a finance employee's queue. She checked the amount, account, and relevant pages of the contract. With the documents apparently complete and other payments still waiting that day, she approved it. The company's existing payment system then processed the transfer under its ordinary procedure for approved payments. The bank statement recorded the exact time the money left the account.
When the mistake came to light, the company contacted the supplier immediately. The supplier was willing to discuss a resolution and to return the portion of the deposit it had not used. But orders for custom materials had already been placed, and some processing and cancellation charges could not be recovered. After checking the costs, the two sides agreed that the company would absorb just over 8,000 yuan in actual expenses; the balance was refunded. Looking at the cost statement, the owner knew that the money would not reappear simply because nobody had intended any harm. This was not another installment of the expedited shipping charge from the earlier purchasing mistake. It was a separate loss, arising from a payment that had actually been made.
The first person brought into the internal inquiry was the finance employee who had clicked Approve. The system record was so clear that it seemed to invite an immediate conclusion. The submission time, the person who opened the request, and the person who authorized it were all there. “Wasn't she the one who reviewed it?” someone asked. The finance manager did not contradict the question. Instead, she asked the employee to retrieve everything she had been shown at the time. The employee was visibly nervous. She repeated that she had checked the bank account and verified the amount. The project suspension had not appeared in the payment summary. Eventually, she paused and acknowledged that she had not read every page of the correspondence, nor had she specifically called the project manager to confirm that the work had resumed.
The admission quieted the room. She had review duties of her own; the fact that an agent had prepared the documents did not make an omitted check someone else's problem. But the finance manager also knew that this employee had not authorized the packaging-line modification, decided when the supplier should order materials, or determined whether the company ought to restart the project. What had reached her desk was a payment request already shaped by several preceding steps. Finding the check she had failed to make was part of the investigation. But if that was taken as a complete explanation of why the payment had reached the approval queue in the first place, the inquiry would end at the easiest name to find.
Tracing the Request Back to Its Source
Different people had seen different stretches of the same chain of work. The gaps between those stretches were what the investigation was gradually bringing into view.
The owner asked finance and the business team to reconstruct the payment from beginning to end. This time, they were not to start by filling in the name of the person at fault. He wanted to understand why a project that had been suspended appeared in the system as one whose payment could proceed normally. The finance manager began with the phrase in the assistant's summary: consistent with contract terms. That statement was not invented. The contract did allow for a deposit, and the amount and supplier details were correct. What had gone missing was an unstated condition: the company still had to intend to go ahead with the project. The contract could not tell the person processing the payment whether the business had changed its mind the day before.
A member of the business team admitted that the suspension notice had been shared only in the group chat the team used every day. He had assumed the procurement manager knew about it, and that procurement would tell finance. The procurement manager explained that his department had been occupied with other orders and had assumed the modification remained pending a formal change; no one expected payment to be initiated in the meantime. Nobody had entered the word suspended into the purchasing request that was still active in the system. The request looked like part of a project proceeding as planned. The contract, supplier, and amount were unchanged. If anything, the paperwork supporting the payment stage was more complete than before. Each person had taken a step they believed was sufficient, without anyone confirming that those separate steps had actually changed what happened next.
After the meeting, the technical staff retrieved the assistant's processing logs. The agent had not conjured a contract out of nowhere or found a beneficiary account from an unknown source. It had read the business records it was permitted to access, matched the order against the fields for the relevant payment stage, and passed the result to a human reviewer. The technical staff pointed to the records showing exactly what the assistant had used. The group-chat message announcing the suspension was not among its sources. The most recent formal record available to it was still the purchasing plan under which the project would go ahead. As far as that record was concerned, the plan had not ceased to exist because someone in an office had said, “Let's hold off for now.”
“Then why was it allowed to submit a payment request at all?” the owner asked. The technical team explained that this was not a capability the agent had demanded for itself. At first, the assistant had only prepared reports and checked fields. Later, finance complained that copying the details into payment applications one by one was taking too much time. At the department's request, the technical team connected the assistant to the existing application system. It could read specified contracts and supplier records and create documents awaiting human approval. Finance employees still had to approve them, and the payments still went through the company's established process. The team had deliberately retained that human step, believing it would save data-entry time without handing the agent the final ability to pay out company funds.
The finance manager remembered making the request. Initially, the assistant had condensed several pages of attachments into a short summary. Employees then pointed out that they still had to retype the amount, payment stage, and bank details from the same contract into the application. Why not let the assistant fill in the form as well? When that improvement went live, the department had checked carefully that the agent could not approve an application it had created. The authority of the people signing off on payments had not changed. Nobody thought they were taking an extraordinary risk. After all, preparing a payment request and authorizing payment were two different steps. Human employees prepared applications for other people to approve every day.
The procurement manager recalled another change. To cut down on repeated exchanges between departments, procurement had asked the assistant to forward eligible purchasing information to finance automatically when a regular supplier and an approved contract were involved. Otherwise, a transaction that had already been handled in procurement had to be assembled all over again by another employee, and much of the promised efficiency disappeared at the departmental boundary. Taken on its own, that request had seemed reasonable too. The contract was genuine, the supplier was known, and finance retained its review. Only now were these individually cautious improvements being placed side by side. Purchasing information had become easier to feed into payment documents. The assistant had become better at creating formal applications. And finance employees had grown accustomed to receiving requests whose supporting explanations were already written for them.
“Who approved all these changes?” the owner asked. Several people could answer at once. The technical team produced a requirements document signed off by the business manager, confirming the scope of the feature. Finance had test records showing that payment still required human approval. The procurement manager found the email in which he had agreed to automatic sharing of purchasing data. Further back stood the owner's own decision to buy the system and fund the integration work. He had not examined how every field passed between applications, or personally authorized each new function added afterward. He had approved a plan to improve efficiency. In carrying it out, people responsible for different parts of the business had gradually given that plan its present form.
Someone suggested that if so many people had examined the workflow, then surely everyone shared responsibility. Someone else immediately objected that responsibility could not simply be divided equally among everyone who had participated. The engineer building an interface might not know when a project ought to be suspended. The manager who decided to suspend it might not know that a group-chat message would never reach the payment records. The employee who approved the request did have a duty to check it, but she could not change which information sources the assistant used. Different people had seen different stretches of the same chain of work. The gaps between those stretches were what the investigation was gradually bringing into view.
The owner stopped asking the question Whose fault was it? as though it could only have one immediate answer. He picked up the payment request and noticed how many distinct facts it brought together. The assistant had the technical ability to complete the form. The finance employee had the authority to review it. The company's payment system could transfer funds once approval was recorded. And the supplier had indeed received the money. Each fact followed from the one before it, and together they described what had happened. But none, on its own, established that the suspended project should have proceeded that week. Being technically able to perform one step and having approval for another were both matters that had to be understood against the circumstances of the actual business.
Who Made It Possible?
He knew that tracing how power and capability had been granted did not release any participant from the duties they owed. Nor did it, by itself, settle how responsibility should fairly be assigned.
That afternoon, the owner sought out a long-serving employee and asked how the company had dealt with similar problems in the past. She remembered an equipment purchase from years earlier. A new employee had ordered materials according to an old plan, only to discover that the workshop had changed its production arrangements. The newcomer should have asked whether the plan was still current, and deserved criticism for failing to do so. But the manager had also found that the workshop had not told procurement about the change. Each department had assumed the other would check first. In the end, the company did more than ask the new employee to write an apology. It agreed on which changes had to be communicated, and to whom. An employee's carelessness might be real, she told the owner, but if the people downstream could only discover upstream changes by luck, replacing one person would not necessarily prevent the next mistake.
The owner recognized the pattern, though something about the present case was different. A human employee could submit only so many purchasing or payment applications in a day. Gaps between departments often surfaced when somebody had time to ask a question. An assistant, by contrast, could assemble large numbers of apparently complete applications and keep forwarding them through connected processes. Speed had not conferred any new legitimate authority on the agent. But it had altered who encountered information first, which matters could proceed to another stage, and how many decisions employees were expected to make. The checks the company had established for human work remained. The way tasks arrived at those checkpoints—and the speed at which they arrived—had changed.
Someone from the technical team suggested that, since the failure involved a suspension notice that had never reached the system, adding a project-status field might prevent it from happening again. It was a sensible improvement, and the owner agreed that the immediate gap should be closed. But he was no longer inclined to mistake a repair for a complete answer. Today it was a suspended project; tomorrow it could be a revised contract term or a late change in a delivery plan. Fixing a known defect could reduce the chances of a similar failure. Understanding why this one had progressed all the way to payment required looking at who could change the workflow, who could supply new facts, and which people had a chance to notice when those two things had failed to connect.
Later, the finance employee asked to speak with the owner. She did not want the company to treat her as an innocent button that someone else had pressed. Nor did she want the system didn't warn me to become a complete explanation. She admitted that, when she saw the deposit request, she should have asked whether the project was still proceeding as planned, especially since some time had passed since the contract was signed. But she also wanted to understand why the payment paperwork positively told her that the contractual payment stage had been reached without drawing attention to the possibility that the project schedule had changed. She could improve the way she reviewed requests. She could not, on her own, guarantee that every future change to every project would reach the system in time.
The owner told her the investigation would not focus solely on the last click, but it would not overlook that click either. The company needed to establish what each role was supposed to do and whether the person filling it had the information and conditions necessary to do it properly. How to address this employee's mistake, the communication failures between departments, and the shortcomings in the workflow would require further examination of the facts and an opportunity for everyone involved to explain their part. He was unwilling to declare, in haste, that one person should bear the entire loss. That might produce a simple answer while allowing the company to miss the things it most needed to change.
A few days later, a fresh sheet of paper appeared on the meeting-room whiteboard. It was not a complicated technical diagram. The owner had asked the departments to write out the sequence in ordinary terms: the business team proposed the project; procurement confirmed the conditions; technical staff enabled the assistant to prepare and submit an application; finance reviewed it; and the existing payment process transferred the funds. Under each step were names and dates, along with several spaces still waiting to be filled. The drawing lacked the tidy appearance of a system dashboard. But it came much closer to explaining why the money had been sent than the original record, which had shown little more than the identity of the final reviewer.
The procurement manager stood in front of the whiteboard for a long time. He thought back to the arguments over the words Whoever reviews it is responsible. Everyone had been concerned with the final visible checkpoint: who should review the work, and who should answer for it. Yet that was not where the assistant's ability to act had begun. Which records it could read, what recommendations it could produce, where those recommendations would go, and under what conditions they could advance the work—none of these things had appeared from nowhere. They reflected management budgets, departmental requests, technical configuration, and a succession of adjustments intended to reduce waiting. Some decisions had been formally incorporated into policy. Others survived only in emails. Still others were hidden in the casual instruction, “This will be easier if we do it this way from now on.”
In the end, the owner did not circle a single name on the whiteboard. He knew that tracing how power and capability had been granted did not release any participant from the duties they owed. Nor did it, by itself, settle how responsibility should fairly be assigned. What he saw more clearly than before was that the assistant had not acquired some mysterious power overnight, and the final Approve button had not brought about the payment single-handedly. Many arrangements designed to help the company run more smoothly had been connected over time. Most days, those connections made work faster. This time, they had carried a business plan that had already changed into a workflow that was still operating on the old one.
The investigation did not end when the whiteboard was full. The loss still had to be recorded in the accounts. The supplier's remaining arrangements had to be settled. And the responsibilities of the different employees and departments would still have to be judged on the evidence. But the company was no longer satisfied with asking only, Who approved it at the end? As the owner closed his notes, someone from sales walked past the meeting room carrying a quotation for a new customer, on the way to discuss a promising opportunity. The owner glanced up without giving it much thought. Inside a company, people could spend days, perhaps longer, untangling how a payment had happened. Outside it, other people would not necessarily put their own decisions on hold while the company worked through its explanations.
Chapter Seven — While They Debated Responsibility, the Customer Moved On
The Quotation That Looked Fine
What hurt was not anything harsh the customer had written. It was the absence of any reproach. No one demanded an immediate explanation or offered to resume negotiations if the company simply sent a corrected document. The customer had already moved on, and was letting them know politely.
The owner saw a salesperson hurrying down the corridor with a folder of quotation materials. At the time, the company was still dealing with the mistaken payment for the equipment-modification project, and several blanks remained on the meeting-room whiteboard where the investigation had been mapped out. He did not stop the salesperson. For the sales team, this was supposed to be good news: a customer they had been courting for some time was finally ready to compare formal proposals. If the price and delivery arrangements worked out, the company had a chance of winning a new order.
The salesperson had spent nearly two weeks on the account. He remembered that during their first call, the customer had not rushed to ask for the lowest price. Instead, the questions kept returning to the delivery date. The customer was revising its own production schedule. Several pieces of supporting equipment and consumable materials had to arrive by a particular point; otherwise staffing and installation plans further down the line would be affected. After-sales service could not be dismissed with a line promising “support,” either. Who would respond if something failed? When would someone come on-site? Which services would incur an additional charge? The customer wanted specifics.
In the past, answering such questions would have taken the salesperson a good deal of time. He would have had to check model numbers with the product team, ask delivery colleagues about the schedule, and work through several service documents to find the terms that applied. Now the sales assistant could turn the customer's correspondence into a requirements list, retrieve product information, compare package prices, and even draft a courteous opening and closing for the quotation letter. He gave it the requirements at noon. By the afternoon, a neatly formatted formal quotation was waiting for him. The quantities multiplied by the unit prices correctly, the discount was within the company's permitted range, and the product descriptions were clearer than anything he could have written in a hurry.
It was not that he failed to review the document. He checked the customer's name, equipment models, quantities, total price, and payment arrangements one by one. He even spotted and corrected a telephone number. The delivery period was listed as “about three weeks,” which sounded close to an estimate he remembered from an earlier internal conversation. He did not reopen the delivery department's recently updated schedule. The service section said that support would be provided under the company's standard after-sales plan, followed by a brief explanation. The customer had previously asked about on-site response. He had meant to clarify that during the next round of discussions, but did not notice that the formal document already carried the vague wording forward.
His hurry was understandable. The customer had said its team would soon be taking competing proposals into an internal review. If their quotation arrived too late, the salesperson feared they might not even make the shortlist. In earlier inquiries, the assistant's drafts had generally been accurate, and his manager had approved the time saved by using it. He checked what he could readily verify, put his name on the sending record, and used the company's usual procedure to email the quotation to the address provided. When Sent successfully appeared on the screen, he let out a long breath. An afternoon that would once have disappeared into preparing documents had not stretched into several days.
Nothing much happened over the next two days. He called once, and the customer's contact told him the proposal had been passed to purchasing and the team that would use the equipment. They were evaluating it together. He took that as encouragement: at least the quotation was being seriously considered. He returned to other inquiries, checking his inbox from time to time and imagining a follow-up call with colleagues from the delivery department when the customer had more questions. As far as he was concerned, the opportunity was still alive.
A few days later, the reply arrived. It was courteous and left little room for negotiation. The customer noted that the delivery period in the quotation did not match information provided earlier, while the scope of on-site service was not sufficiently clear. With its project schedule under pressure, the customer had decided to give priority to progressing arrangements with another supplier. The email ended by thanking the salesperson for his help and expressing a willingness to stay in touch about future projects.
He read it twice. What hurt was not anything harsh the customer had written. It was the absence of any reproach. No one demanded an immediate explanation or offered to resume negotiations if the company simply sent a corrected document. The customer had already moved on, and was letting them know politely.
The Clearer the Error Became, the Less Comfort It Offered
Every item in the quotation seemed to have a source. Yet taken together, those items had failed to tell the customer what the company could actually promise at that moment.
The salesperson immediately pulled up the original quotation and the earlier correspondence with the customer. Comparing them line by line, he discovered that “about three weeks” came from an older product-delivery document. The delivery department had since revised the estimated lead time for the relevant models to five or six weeks because of changes to the production schedule. Its new internal material also stated that special orders required separate confirmation. The old version was still in the shared folder the sales assistant could search; the newer one had been stored in a location the product department had adopted later. The machine had not miscalculated a number. The document it had used to describe the real world was no longer current.
The service wording had a similar history. An older description had grouped remote advice, basic fault guidance, and further on-site assistance into a single paragraph. The revised service plan drew clearer distinctions between response methods and the conditions under which they applied. The assistant had reused the abbreviated wording from the older version. Seeing the familiar phrase standard after-sales service, the salesperson had not recognized that the customer's repeated questions about on-site support had never really been answered. Every item in the quotation seemed to have a source. Yet taken together, those items had failed to tell the customer what the company could actually promise at that moment.
He called the customer and offered to correct the lead time and service terms immediately. The customer did not refuse the call or take the opportunity to vent. The contact listened patiently, then explained that the issue was not merely one figure in a table. The team using the equipment needed dependable delivery dates around which to organize its project. Other suppliers had given clearer schedules and service descriptions, and the purchasing process was already moving forward. Even with a corrected quotation, reconsideration would depend on the project's progress and the customer's internal decisions. The salesperson could hear that the contact was not trying to be difficult. He also understood that discovering his own mistake gave him no right to ask everyone else to suspend their work.
After the call, he went to the sales manager, then contacted the delivery and product-documentation teams. A delivery colleague found the update notice, dated several days before the quotation had gone out. The documentation specialist located the revised service description; its creation date was plain to see. The technical team checked the assistant's logs and confirmed that the generation process had followed the intended instructions. The system had assembled the quotation from the information available to it. It had not sent the document without permission. A salesperson had still performed the formal send, and the company did have review requirements. The more they learned about the sequence, the clearer it became. None of that made anyone feel immediately better.
The salesperson's greatest fear was that, because he had clicked to confirm the document, everyone would conclude he should have verified every underlying source from scratch. He did not deny that his review had been rushed, and he regretted not making one more call about delivery. But he also knew that if salespeople had to repeat every step of the assistant's work and independently prove that every background document was current, much of the time they had saved would disappear. He had not delegated the preparation in order to escape scrutiny. He had been using a tool within a division of work the company had accepted. He had thought his checks covered what mattered. Now he knew he had missed two things that mattered most to this particular customer.
The documentation specialist was no happier to hear the phrase outdated information. The new material had been published and was accessible to employees in her department. But the old file had not been withdrawn, and nothing clearly established which version superseded the other. She acknowledged that updating information meant more than placing a new file somewhere: people and systems using it needed to know which version was valid. At the same time, the sales department had never explicitly required every system preparing formal quotations to retrieve current information from a single authoritative location. Sending an update notice, she pointed out, was not the same as ensuring that every downstream workflow had received the update.
The technical staff were more cautious still. They could explain what the assistant had read and the task under which it generated the text. They could help repair gaps in its information sources. But they could not determine, simply by observing that software had run as designed, which service promise mattered most to the customer. The owner did not take this to mean that the technical team had no responsibility. He was reminded instead of the earlier payment incident: once again, several parts of the organization had reasons for doing what they did, yet together they had produced an unsuitable result. This time, the loss was not a cost entered in the accounts. It was a commercial opportunity, painstakingly developed by a salesperson, now slipping away.
The sales manager decided they would continue to contact the customer, provide a complete correction, and apologize for the inconsistent information. He also asked the delivery team whether the actual schedule could be brought forward. The last thing the company needed was to make another promise it could not keep in an attempt to recover the order. The team would not give up solely because the customer had said it was prioritizing another supplier. Nor would they claim, without evidence, that one flawed quotation was the sole reason for losing the business. Price comparisons, existing supplier relationships, technical preferences, and the customer's own timetable might all affect the decision. What they knew was that their quotation contained important defects, and that the customer was now directing its attention elsewhere.
That distinction made the salesperson oddly sad. If he could prove that the customer had left for one reason alone, the problem might have seemed simpler: find the error, correct it, win another chance. Real commercial decisions seldom offer such clean explanations. Perhaps the customer had already favored another supplier, and the faulty document merely weakened its interest in further discussion. Or perhaps it had genuinely liked their proposal but chosen the more dependable option under time pressure. From the customer's perspective, those factors arrived together. It was under no obligation to calculate how much each one had contributed to its decision.
The Customer's Clock Did Not Stop at the Meeting-Room Door
A correction cannot take everyone back to the starting point. Another email can be sent, the figures recalculated, the service terms clarified. But the customer has already spent time comparing alternatives, may have reported its findings internally, and may even have begun coordinating installation or purchasing arrangements.
The company began discussing how to respond. The owner asked for a reconstruction of the quotation's entire journey—from publication of the source material, to its retrieval and use in drafting, to the salesperson's final send. There would be other customers, and another incident like this could not be addressed merely by saying everyone should take more care next time. The sales manager needed to explain when lead times required confirmation. The product team needed to resolve the coexistence of different document versions. The technical team needed to understand why the assistant had retrieved the old file. Each task mattered. Without an investigation, the company could not distinguish stale information from inadequate review or a gap in the process, much less ask the appropriate people to improve it fairly.
Yet as the meeting continued, the conversation drifted toward who ought to be held responsible. Some argued that a person sending an official quotation had a duty to check more than the arithmetic. Others countered that expecting a salesperson under a deadline to verify every piece of background information was not a workable system. The document owners said they had published updates; the engineers said the software had operated correctly; managers emphasized that formal quotations had to be reliable. There was something worth taking seriously in every argument, along with a reluctance to see one's own work dismissed. The owner did not silence the disagreement. He knew that refusing people the chance to establish the facts would only preserve the resentment for the next incident.
The salesperson, meanwhile, kept looking toward the window. The customer's email was still on his phone. Every time he read the phrase stay in touch about future opportunities, he remembered their earlier calls. The contact had carefully asked how many people would be needed for installation and whether on-site help would be available if the equipment failed. At the time, those questions had seemed evidence of genuine interest. The conversation was still there in his message history, but the person who had asked them was now discussing another supplier's arrangements. Even if the company produced a complete account of responsibility, signed by everyone in the room, the earlier conversations would not simply pick up where they had left off.
He tried looking at it from the customer's side. If he were a purchasing manager comparing several proposals, and one supplier's lead time needed explaining again, its service terms no longer matched earlier discussions, and his own project was running against the clock, would he suspend everyone else's work just to give that supplier a chance to clarify matters? Perhaps not. The customer had its own production arrangements to protect and colleagues to whom it would have to justify its choice. Trust did not always mean believing a supplier would never make mistakes. Sometimes it meant believing the information supplied was stable enough to make plans around.
A correction cannot take everyone back to the starting point. Another email can be sent, the figures recalculated, the service terms clarified. But the customer has already spent time comparing alternatives, may have reported its findings internally, and may even have begun coordinating installation or purchasing arrangements. A contract might not yet have been signed, yet judgments and expectations might already have changed. A supplier could apologize for an error. It could not erase the experience of reading a flawed quotation, or unilaterally restore the patience with which a customer had first considered its offer.
Not every consequence was irreversible. The customer might accept the explanation and agree to another comparison; the companies might still work together on a future project. The owner did not want to describe the deal as definitively lost while the outcome remained unsettled. He asked the sales team to maintain polite, measured contact rather than repeatedly pressing for reassurance. But they had to acknowledge the change already made: for the current project, the customer had explicitly shifted its priority to another supplier. That was a real commercial development, regardless of whether the company continued trying.
The owner remembered an email he had written incorrectly to a customer when the business was young. He had spent the night explaining the mistake, but the customer never resumed discussions. There had been no AI then, no elaborate workflow, no machine preparing the material. He remembered the episode not because he had established exactly why the customer withdrew, but because he had learned, for the first time, that an error judged correctable inside a company and a customer's willingness to trust that company again were two different things. The same helplessness was returning in a workplace that now moved much faster.
In this case, the assistant had not independently decided to send a quotation. The final email had been checked and sent by a salesperson. The company had assigned responsibilities, established habits of document review, and retained an audit trail. Precisely because those arrangements existed, it was difficult to explain the incident by saying there had been no management at all. The controls were not useless. They had helped catch numerical mistakes and made it possible to reconstruct the sequence of events. They simply had not kept a document containing an obsolete delivery estimate and an incomplete service description inside the company.
That evening, the owner stopped by the sales department. The salesperson was filing the revised quotation and its accompanying explanation. Asked whether the customer had replied, he shook his head. The contact had acknowledged receiving the correction but was proceeding with the arrangements already under way. The owner did not ask again whether the salesperson had done everything possible. He asked only that the full history be preserved for future improvement. The salesperson closed the email. The words Sent successfully were still there on the screen. They proved that a document had left the company. They could not tell anyone what the customer would decide after reading it.
Back in his office, the owner found the investigation into the mistaken payment beside the sales team's account of the quotation. The first incident had taught him that identifying the last person to approve something did not reveal the full history of the agent's ability to act. The second reminded him that even a perfectly clear account of everyone's responsibilities could not make the outside world restart on the company's schedule. He did not intend to abandon accountability or use the phrase AI makes mistakes to excuse carelessness. But the next time an official quotation was ready to leave, he wanted the company to know, before it reached a customer's inbox, what must not be missing.
He did not propose a new technical architecture that evening. He asked the sales manager a simpler question. “We have review procedures. We know who's allowed to send a quotation on the company's behalf. So why did this one still go out with information that was no longer valid?” The sales manager had no immediate answer. The owner found himself wondering whether having the ability to send an email and having good reason to send this particular email now were two different things. By the time a customer had made its choice, however clearly the company could explain what had gone wrong, the moment before the document left was already gone.
Chapter Eight — Setting Rules for the Machines
Asking More Than Whose Fault It Was
The real difficulty was separating the rules that could be checked unambiguously from the language of everyday work, so the system would know when to proceed, when to stop, and when to ask a person to decide.
The faulty quotation lingered in the company longer than the salesperson had expected. Days passed, and the customer was still giving priority to discussions with another supplier. As agreed, the salesperson had sent a corrected explanation and resisted the urge to make a third impatient call. He knew the customer had received the material. He also knew an apology did not entitle him to have someone else's purchasing meetings rearranged. The rest of the business carried on. But whenever he prepared another quotation, he found himself checking the delivery date one more time.
The owner, too, stopped asking the departments to rehearse who should be blamed for the email. The review records still needed examination, and the limits of each role still needed clarification. But he no longer wanted every discussion to begin after something had already happened. First the company had welcomed the machine's efficiency, then grown tired of the human checks that seemed necessary to contain it. It had given the machine a measure of discretion, and when things went wrong, handed more decisions back to people. After several rounds, everyone seemed to be moving between two unsatisfactory choices. Now that a customer had made a decision of its own, going through the same cycle again could not end with nothing more useful than “We'll be more careful next time.”
One morning, he gathered the sales manager and several colleagues. Instead of asking them to account for the last incident, he asked, “If another quotation were going out tomorrow, what are the things we can already say must not be wrong?” The sales manager remembered the delivery date the customer had cared about so much. Formal quotations, he said, must not rely on expired delivery information. The product-documentation specialist added that delivery schedules were not the only materials with versions. Product prices and service plans changed too. A document's presence in a shared folder did not make it valid forever. The finance manager brought up discounts: an offer above the authorized limit should not become a formal quotation merely because the document looked reasonable.
As they talked, they realized that none of these requirements was new. The company already had policies on quotation validity, discount approvals, contracts, and checking customer information. Managers training new employees routinely told them not to promise an unconfirmed delivery date, grant unauthorized discounts, or offer special services without establishing that those services could be provided. Much of this depended on people remembering what mattered. Some requirements were in the staff handbook, others in approval forms, and still others survived as a question an experienced colleague might ask just before an email was sent. When the assistants were introduced, the company had kept those policies. It had not necessarily carried each requirement into the new way of working.
The owner had no wish to send out a thicker list of reminders. He remembered the review queues that had grown longer and longer. If a decision could be made by comparing dates or amounts against established conditions, why should a manager have to make the same check anew every day? On the other hand, if a requirement existed only in someone's memory, an assistant might draft ten documents from obsolete material and leave that manager searching through all ten for the same defect. The owner wanted to move requirements the company could already state clearly from things everyone should remember to checks every formal document must pass.
This was hardly a new philosophy of management. A company hiring someone did not simply hope the new employee would never make a mistake. It specified duties, spending limits, and where to seek guidance when a situation fell outside normal practice. An accountant did not skip payment controls merely because she felt confident. A salesperson could not rewrite a contract's terms at will because an order was urgent. Becoming part of the organization did not grant a machine some natural exemption from its rules. The real difficulty was separating the rules that could be checked unambiguously from the language of everyday work, so the system would know when to proceed, when to stop, and when to ask a person to decide.
Turning “Be Careful” into “Check This Before Proceeding”
The company had not abolished approvals. It had simply stopped spending human attention on every check that could reliably be performed against explicit conditions.
Product information came first. The documentation specialist put an old description of a model beside the current version and pointed out several small changes. The delivery estimate had shifted from “about three weeks” to one that had to be confirmed against the production schedule. Some service provisions now applied under different conditions. She proposed that every piece of product information used in a formal quotation carry a version and an effective status. Older versions could remain available for reference, but could not be treated as current authoritative material. A salesperson asked whether that meant stopping a quotation even if a particular statement in the old file had not changed. The specialist explained that the point was not that every sentence in an old document was necessarily wrong. The company could not ask someone preparing a quotation to guess which of those sentences still counted.
The technical team took the proposal and asked more precise questions. Who was authorized to declare a document effective? On what date did it take effect? If a document had been withdrawn, could the system distinguish it from material still approved for use? At first, the colleagues had assumed they were merely adding dates to files. Then they realized that the time a file was saved was not necessarily the time its business terms became valid. Something copied into a folder yesterday might describe arrangements from a month before. After some discussion, they agreed that the main product materials used for formal quotations would be confirmed by the people responsible for maintaining them. The assistants would use designated valid versions rather than guess which of several similar files was newest.
Sales clarified several numerical conditions as well. Authorized salespeople could offer ordinary discounts within a defined range; discounts beyond it required additional approval. Special delivery promises needed to be checked against a schedule confirmed by the delivery department. An earlier date could not simply be typed into a document because the customer wanted the goods sooner. Nor could a generic promise of “standard service” stand in for a specific agreement on customized support for a major customer. The finance manager reminded them that some conditions lent themselves to direct comparison. A price ceiling could be checked, and a discount percentage calculated. But questions such as whether a customer was especially important, or whether a promise was clear enough, often still called for a business judgment.
The technical team did not propose deploying a smarter assistant to replace every human reviewer. They began by connecting unambiguous checks to the existing quotation workflow. Was the product information drawn from an approved, effective version? Was the discount within the range authorized for the person making the offer? Were the required customer and product fields complete? If an exception called for human approval, was that approval present? When a required condition was missing, the system would prevent the quotation from advancing to the formal sending stage and explain what needed attention. Complex commercial reasoning, questions about a customer's intentions, and exceptional arrangements would remain with the people whose jobs included making those judgments.
One of the first trials encountered an outdated product document. The sales assistant drew on a historical customer file and assembled a familiar equipment package, complete with an old quotation as a starting point. The text resembled a package the company had delivered successfully before. But the checker found that one product description was no longer an approved current version, and it halted further processing of the formal quotation. The salesperson opened the warning and located the newer material. The delivery terms had indeed changed. He did not blame the assistant for the draft. Its comparison and assembly work had still saved time. The useful difference was that the system had caught a fact he could verify without having to reread an entire library of product documents.
A few days later, a colleague hoping to secure an order entered a discount larger than her usual authority allowed. The assistant calculated the amount correctly, and the quotation looked impeccable. The checker refused to let that settle the matter. A message explained that the proposed discount required further approval. The salesperson had to tell her manager why the concession might make commercial sense. After listening, the manager decided to renegotiate the terms with the customer rather than approve the discount immediately. The lower price never became a formal promise made to the customer, and the salesperson was not left thinking that an accurately calculated figure had somehow also been authorized.
These small interruptions felt different from the old review process. Previously, people often began checking the details only when the document appeared finished. Now obvious issues with dates and figures surfaced earlier, without waiting for a manager to happen upon them in an overflowing queue. Department heads gradually recovered time for questions that needed explanation: why a customer wanted unusual support, for example, or whether current capacity could support a proposed delivery arrangement. The technical team could also distinguish between drafts stopped because their source documents were invalid and those awaiting authorization for an excessive discount. The company had not abolished approvals. It had simply stopped spending human attention on every check that could reliably be performed against explicit conditions.
Procurement wanted to make similar changes. The earlier expired quotation had cost the company an expedited shipping charge, and the procurement manager still remembered how the relevant information had been available without being the information the company ought to have used. He asked his staff to document validity periods and confirmation procedures for quotations from regular suppliers, and to flag prices outside those periods. Finance proposed checking the required approvals before payment requests that needed additional authorization could move ahead. The departments had not suddenly discovered the value of rules. They were slowly bringing constraints that had once lived in separate roles into workflows where agents were taking an increasingly active part.
During a routine update, the owner asked the sales staff whether the new controls had made their work more troublesome. At first, one admitted, it had been irritating to see Cannot proceed so often. But revising a draft while it was still inside the company was far easier than explaining an error after a formal quotation reached a customer. Sometimes the warning merely identified missing evidence; it did not mean the entire proposal was wrong. Nor did every warning need to be escalated to the owner. At last, the company had the sense that it could do more than investigate mistakes after the fact. It could use some of its long-standing rules to catch known oversights before a document left the office.
What the Rules Could See—and What They Could Not
Rules can exclude some clearly unsuitable cases. They cannot make every judgment about what would be the better choice for an organization. Unless people understand what the checks actually establish, they may overlook everything the checks leave unanswered.
As the first checks won acceptance, requests multiplied. The customer service manager wanted special refunds checked for outstanding service promises. Procurement wanted automatic warnings about unusual delivery dates. Finance wanted more incomplete requests held back before reaching the approval queue. Each suggestion could be traced to something that had gone wrong in the past. The technical team heard a growing chorus: check this, restrict that, and preferably stop every unsuitable action before it happens. The owner understood the desire to turn unpleasant experiences into certainty. He also noticed that some requests became much harder to express as soon as someone asked what they really meant.
Take the rule “Quotations for important customers require extra care.” Who qualified as important? A customer with a large order? One with a long relationship? Or a small account whose current project might determine whether future business developed? Or consider “Delivery dates must be reasonable.” Three days later was not necessarily a problem, and three days earlier was not necessarily a benefit. The answer depended on production schedules, the customer's installation plans, and any special arrangement already agreed. Such judgments could, in principle, be governed by policies. But if the underlying terms had not been defined, choosing an arbitrary number and putting it into a system would only make a vague human understanding look like a fact a machine could assess precisely.
On one occasion, a salesperson was preparing a proposal for another customer. The quoted prices were within their validity period. The discount stayed below the limit. The delivery date came from the officially maintained information. Every check the company had implemented passed. The salesperson did not send it immediately. He remembered the customer mentioning on a call that it wanted help with the initial installation after the equipment arrived. The draft promised only remote technical support. That difference breached none of the existing price or document-version rules, yet it might shape how the customer understood the company's commitment. The salesperson checked with customer service and delivery, confirmed what assistance could be offered, and revised the wording.
There was no loss and no fresh story about a machine causing trouble. Precisely because someone noticed the gap before the proposal went out, the team could discuss it without the heat of another incident. All the automated checks really had passed. Why, then, had the draft still needed changing? The engineers explained that the program had checked the conditions the company had specified. It had never been tasked with establishing whether every expectation voiced in a phone call had been reflected in a contractual promise. The sales manager agreed. They should not describe no violation of the defined conditions was detected as this commercial proposal is entirely correct simply to make the system appear more dependable. Passing one kind of check did not answer every question about the business.
The documentation specialist raised a subtler issue. If the system accepted only current versions, what would happen if the person responsible for the records mistakenly marked an outdated service description as valid? Everyone quickly saw the answer. The checker could still pass it. Its job was to establish whether the document carried the status the organization had designated, not to know, independently and without evidence, every change in the real world. Some colleagues suggested adding another review. Others thought the approval process for publishing documents should be improved. Nobody found a way to eliminate all error. But the discussion made the team pay closer attention to where evidence came from, who maintained it, and how changes in business conditions traveled into subsequent work.
The owner had initially hoped that enough rules would remove the need for employees to worry about documents prepared by agents. He began to see how ill-suited that expectation was. A purchasing budget did not make every purchase within it worthwhile. A ceiling on sales discounts did not mean every compliant quotation would satisfy a customer. Complete payment documents did not establish that a project whose plans had changed should still be paid for. Rules can exclude some clearly unsuitable cases. They cannot make every judgment about what would be the better choice for an organization. Unless people understand what the checks actually establish, they may overlook everything the checks leave unanswered.
More rules also meant more situations in which different requirements had to be reconciled. A special discount might be outside the salesperson's ordinary authority but already approved in writing by an authorized manager. Product information might still be within its validity period even though delivery had just announced that the schedule for certain models needed reconfirmation. Pile every restriction into the workflow without considering how they fit together, and the system might block work that should proceed. Make exceptions too broad in the name of convenience, and the restrictions might lose their force. Managers had to establish which circumstances each rule covered, who could authorize an exception, and what evidence would show that the exception was valid. This was harder work than writing “The machine must follow the rules.” It also made the company's requirements more concrete.
The technical staff kept the new checks within the existing business applications and workflows, linking them to the company's document maintenance, authorizations, and approval procedures. There was nothing inherently wrong with that choice. The business had long relied on those processes, and many controls had proved their value over years of use. The owner had no wish to dismiss real improvements merely to pursue an abstract technical ideal. Still, when he saw a record marked Checks passed, he was learning to ask a second question: precisely what did that result establish? It meant that the defined conditions had been checked at the time. He no longer wanted to treat it as a promise that everything the company did afterward would be correct.
Near the end of the month, the sales manager put the report on the old quotation incident beside the new workflow's check records. The first explained how obsolete material had entered a quotation and how the company had corrected it after the customer made its choice. The second showed newer drafts stopped for using superseded information, and a discount outside the normal range routed for further approval. The manager felt reassured. At last, maintaining quality did not depend entirely on urging employees to pay closer attention. But when an engineer asked whether quotations were ever changed after passing the checks, he hesitated. Salespeople did sometimes tidy up formatting at the end. Occasionally, fresh conversations led them to adjust the wording. The formal send still followed the company's established procedure. The team had spent a great deal of time deciding what to check. It had not spent much time considering what happened to the document afterward.
The owner did not demand an answer on the spot. The company had made a genuine advance by turning some of its understandable management requirements into checks, and that achievement deserved to stand. Instead, he asked everyone to trace the next quotation all the way from preparation through review to delivery to the customer. They should pay particular attention to the difference between the material the checks had examined and the document that finally left the company. The rules had been written, and they had already caught some mistakes. The next question was whether, after the rules had said yes, the thing about to go out into the world was still the thing those rules had actually checked.
Chapter Nine — The Rules Were in Place. Who Was Guarding the Gate?
Which Version Had Passed?
Each action made sense when considered alone. Only when the owner put them in chronological order did he see what had happened: the approval record referred to the original document, while the customer had received a revised one.
The owner asked the sales team to trace a quotation from the moment it was prepared to the moment it was sent. He was not suggesting that the checks introduced over the previous weeks were useless. Quite the opposite. The technical team had shown him several records: one draft stopped because it cited an obsolete product description, another returned for additional approval because its discount exceeded the permitted range. Previously, such problems might not have been noticed until a manager opened an attachment. Now employees were alerted while preparing their documents. The sales manager reported that his colleagues were getting into the habit of assembling valid supporting information before thinking about pressing Send. Their queues were no longer filling up with quite so many copies of the same small mistakes.
The discovery came during an unremarkable internal review. The technical team had planned to examine a few quotations that had already been sent, to make sure the new workflow had recorded everything it was meant to. Rather than revisit the customer who had shifted toward another supplier, they chose a recent inquiry from an ordinary account. A salesperson had used the assistant to prepare a draft. The system had checked the prices against current product information, confirmed that the discount fell within the salesperson's authority, and verified that the necessary customer details were present. The sales manager had reviewed and approved it. The page showed a clear status: Checks passed. As far as the workflow record was concerned, this was exactly how the new rules were supposed to work.
But when the engineers opened the attachment in the sent email and compared it with the draft saved at the time of the checks, the delivery terms were not the same. The original read, “Estimated delivery in six weeks; exact date subject to order confirmation.” In the document that went to the customer, the line had become “Delivery within four weeks.” The total price was unchanged. So were the product models. Even the table headers and footers sat in the same places. Looking only at the approval status, anyone would have assumed the documents were identical. Side by side, however, the difference was precisely where a customer would be most likely to care.
The salesperson was surprised at first. Then he remembered a message that had arrived from the delivery team shortly before he sent the quotation. If the customer confirmed the installation date early enough, the team had said, production might be coordinated more tightly. A lead time of around four weeks could perhaps be negotiated, but capacity and the delivery plan would need to be reconfirmed. The salesperson had been dealing with another customer call. He passed the message to the assistant and asked it to update the quotation's delivery wording in line with the latest discussion. The assistant made the change and replaced the document awaiting dispatch. When the salesperson returned, the same quotation number still showed Approved. And because the new estimate had come from the delivery team, he sent the email as usual. He did not notice that four weeks might be possible had turned into the unconditional promise within four weeks.
The agent had not broken into the customer's inbox or sent anything without permission. The formal email had been sent by a salesperson with the appropriate authority. The earlier checks on pricing and discounts had genuinely taken place. The problem lay between steps that, individually, looked ordinary. The salesperson had requested a change. The assistant had updated the file in response to the information it received. The send operation had retained the earlier approval status. Each action made sense when considered alone. Only when the owner put them in chronological order did he see what had happened: the approval record referred to the original document, while the customer had received a revised one.
Fortunately, the quotation was still under discussion. The customer had not yet confirmed an order on that basis. The sales manager immediately asked the delivery team to review the schedule and then contacted the customer to clarify the terms. Four weeks was a target that might be achievable through coordination, not a firm commitment; any binding delivery date would depend on the final production plan. The customer understood and did not demand compensation or end the discussions. The owner had no intention of turning the discovery into another major incident. It was a gap found during a review, explained in time, with no evidence of a further loss. But the absence of a loss did not mean the gap had never existed. Had no one compared the two versions word for word, it might have remained hidden behind a reassuring approval record.
In the meeting room, the salesperson asked, “If the checks really happened, and the approval was real, how did we end up sending something different?” The engineer did not answer immediately. Instead, he turned his laptop around, opened both saved versions of the quotation, and asked everyone to look closely at what the system had actually examined. For the first time, the owner felt that a record saying Passed could be perfectly truthful while still inviting people to believe it guaranteed something it never had.
The Rules Had Not Been Broken, but the Document Had Moved On
The first failure raised the question of whether the information was a trustworthy account of current reality. The second raised another: was the thing finally sent out even the same thing that had been checked?
The engineers explained how the new workflow had been built. Each quotation number identified a business matter. Once a draft was generated, the system checked its document versions, amounts, and approval status at a particular point in time. The result was stored against that quotation number. When someone reached the sending stage, the business page supplied the latest available file, which a staff member confirmed before handing it to the email system. The team had pictured a quotation moving from draft to approval to dispatch. They had not seriously considered that its substantive contents might still be rewritten after approval. The system could answer, “A document under this quotation number passed a check.” It could not answer the question that now mattered: “Is the content in front of us the same content that passed?”
“Then we check it again right before sending,” someone suggested. “Wouldn't that solve it?” The engineer nodded. That would at least repair the gap they had found. Then he asked, “What if the document can still be changed afterward?” The sales manager frowned. In that case, he said, changes should be prohibited after the final check. And what if the customer sent new requirements and a change was genuinely necessary? “Run it through the process again,” the procurement manager answered at once. Several colleagues agreed. But the owner asked them not to rush the answer into another policy. First, he wanted to understand why rules that had existed and been followed had failed to constrain the document actually sent.
Working backward through the records, they found that no rule had explicitly been switched off. The price was within range. The service-document version had not been withdrawn. The manager had indeed approved the quotation. What had been overlooked was a change over time in the object to which those checks and approvals referred. The check had taken place at two in the afternoon. A little after ten past two, the assistant changed the delivery wording. The email went out later. Every step used the same quotation number, and when colleagues spoke of “that quotation,” nobody thought they were being ambiguous. But to a customer, delivery in six weeks and delivery within four weeks were not two ways of formatting a sentence. They were different commitments, each capable of affecting the customer's plans.
The owner remembered the argument in finance over the payment for the suspended equipment project. That investigation had gradually shown that contractual terms could be valid while the business decision to proceed had changed. This quotation brought a further difficulty into view. Even if the business information was correct when checked, the workflow could subsequently produce a different result that had never undergone the same checks. These were not identical failures, and telling employees to “pay more attention” would not resolve them as though they were. The first failure raised the question of whether the information was a trustworthy account of current reality. The second raised another: was the thing finally sent out even the same thing that had been checked?
The documentation specialist, listening quietly, raised a related question. The company now marked official product information with an effective version and required the designated staff to approve it before the sales team could use it. But who could alter those markers? Who had authority to make an old document valid again? She described a recent request from sales to use a provisional delivery schedule as the basis for an active quotation because the customer was in a hurry. She had declined to mark it as current without explicit confirmation from the people responsible for production planning. If anyone preparing a quotation could simply change the underlying material's status to Valid, the system could check that status perfectly and still fail to enforce the condition the company had meant to establish.
None of this meant that salespeople were untrustworthy or that the assistant was deliberately falsifying rules. Sales wanted to win business. The product team wanted accurate documents. The technical team wanted the process to run smoothly. Each aim was understandable. But the person preparing an action usually had a reason to want it completed quickly. If that same person could decide which facts to use, declare those facts confirmed, and then carry the result through to completion without another constraint, the company needed to ask whether its written rules still had any force independent of the immediate desire to finish the task.
The engineers produced a simple process diagram. External quotations ordinarily passed through the new document checks, discount verification, and managerial approval before being sent by a salesperson. But the company also retained an older, manual method of preparing quotations. When the system was unavailable or a particularly urgent inquiry arrived, employees with the relevant job permissions could still create documents in ordinary office software and email them from their work accounts. This route had not been preserved as a deliberate way around the new rules. It was simply how the company had always worked, and it still met legitimate business needs on occasion. But did the new automated checks cover it as well? Nobody could answer immediately.
The sales manager pointed out that manually prepared quotations were still governed by the company's ordinary rules. A salesperson did not gain permission to promise anything merely because the document bypassed the new interface. The engineers agreed. There was no evidence that anyone had intentionally used the older route to evade checks. Nor would the existence of another way to send a quotation prove that the old controls were worthless. Still, the owner began to see the limitation of the system's success figures. A report showing how many times automated checks had passed could describe only the matters that reached those checks. Whether work traveling through other routes was subject to equivalent constraints was a separate question. An action did not become safe simply because it was absent from the statistics.
The procurement manager thought of the warehouse. Staff could post prominent packing rules and check the contents of a shipment item by item. But if someone replaced the contents after inspection, or if goods left through another exit without the same logging procedure, the earlier checklist could not establish what had actually gone out. The analogy did not require treating everyone who handled a box as a suspect. Quite the reverse. Because most employees were trying to do their jobs properly, the company needed arrangements that could survive mistakes, last-minute changes, and differing habits. Procedures were not meant to prove that people would always be reliable. They were meant to keep the business from depending entirely on memory and luck.
The owner followed the analogy further. The company had always known that budgets should not be exceeded without authorization, contracts should not be rewritten at will, certain payments required sign-off, and goods should not leave the warehouse without the proper documents. Its boundaries had never been absent. They were dispersed among departments, rules, and systems. When people moved work through those stages relatively slowly, experienced employees could often notice inconsistencies along the way. Now an assistant could assemble, revise, and pass on material so rapidly that the next step might be ready before anyone had fully considered the previous one. The old procedures had not become useless overnight. What had become visible were the connections once maintained by habit—connections that could no longer safely rely on habit alone.
Who Was Standing at the Last Gate?
Conversely, existing workflows, access controls, and independent checks can impose meaningful constraints if they truly limit who may make changes, when checks must occur, and whether an operation can proceed after a refusal. What matters is whether the control relationship holds—not whether the components look separate on a diagram.
Over the following days, the company did not disable the assistants or require salespeople to prepare every quotation all over again. The technical team addressed the specific gap first. Any change to substantive business content—including price, delivery terms, or service commitments—would invalidate checks completed against the earlier version. Until the revised conditions had been verified, the sending step could no longer treat the file as approved. The documentation specialist also agreed with sales and delivery that provisional updates could not be declared official delivery information by the people preparing quotations themselves. These changes had a defined scope. They would not guarantee that every future commercial promise was correct. They did, however, close the gap the review had exposed.
The owner welcomed the improvements without rushing, as he had when the checks were first introduced, to turn every observation into another rule. Checks had to be defined by someone, maintained by someone, and placed where they would have effect. If the assistant preparing a quotation could still replace the content after it was checked, another verification step might simply occur at the wrong point again. If the person using an official document could change its validity status at will, the checker might diligently verify an assertion that had never been independently established. No matter how detailed the rule, it could not answer the question of who was permitted to change the facts on which the rule depended.
Finance already had some experience with this, the finance manager observed. Someone could initiate a payment request without thereby gaining the authority to approve their own payment or alter the company's financial limits. A warehouse employee could prepare an outbound shipment without being entitled to erase its dispatch record. These divisions did not always require separate software, much less the owner's personal supervision of every operation. Different responsibilities were assigned to different roles partly so that people could check one another, and partly so that the person trying to complete an action could not also set every condition for it and unilaterally declare those conditions satisfied. Employees could be trusted and their work made convenient without eliminating necessary constraints.
The engineers added that automated systems could perform many clear checks reliably. Amounts could be compared with limits, documents checked for approved status, and required authorizations verified—provided the conditions were well defined and the evidence trustworthy. Assistants could also help identify risks that were difficult to formalize, draw attention to them, and help people understand a complicated commercial situation. Not every judgment needed to become a cold numerical comparison. But if permission for an operation to proceed depended entirely on the assistant preparing it saying, “I've already checked,” then the company ought to understand exactly who it had allowed to make the final decision, however accurate that assistant might be most of the time.
“Does that mean we have to buy some special equipment to guard the gate?” the sales manager asked. An engineer shook his head. The first question was not what to buy, or whether to move an existing program onto a different computer. Two systems might sit in different rooms and still fail to constrain anything if the same business process could rewrite essential conditions, ignore a refusal, or complete the action through another route. Conversely, existing workflows, access controls, and independent checks can impose meaningful constraints if they truly limit who may make changes, when checks must occur, and whether an operation can proceed after a refusal. What matters is whether the control relationship holds—not whether the components look separate on a diagram.
The owner understood the point, though what came to mind was not a neat technical architecture. He pictured the guard at the company's entrance who authorized trucks to leave. The guard did not need to know why the warehouse had purchased the goods, or whether a customer had negotiated a favorable contract. But he did need to know which vehicle was leaving, what it carried, and whether the required documents were valid for that shipment. If the cargo list was replaced after inspection, a stamp on the old list could not prove that the new cargo had been checked. And if a truck could simply depart through an unguarded side entrance, the most meticulous inspection at the main gate would not establish control over everything leaving the premises.
Of course, an email or a software operation was not literally a truck, and not every business document needed a visible gate. Salespeople could send inquiries and discuss tentative proposals with customers without seeking solemn permission for every keystroke. What concerned the owner were actions that created formal commitments, transferred funds, changed equipment states, or otherwise had consequences in the real world. What conditions did the company require them to meet? Who could decide which conditions applied? And when those conditions were not met, who actually had the power to stop the action? Existing systems might be able to answer some of those questions. Others might call for changes to the relationships between roles and workflows. There was no reason to assume in advance that a particular kind of technology was the only possible answer.
At the final review, the salesperson opened the quotation again. He placed the version from before the revision beside the one the customer had received. The customer name, products, and price were identical; only the delivery terms differed. In the past, his questions had been whether the document was ready and whether he had received approval. Now he found himself asking exactly which conditions his manager had seen when granting that approval, and what the approval meant after those conditions changed. He did not want to push every task back onto his manager or reduce the assistant to a tool for tidying words. He simply wanted to know, the next time he saw Checks passed, what proposed action those words actually referred to.
The owner did not answer at once. He thought back over the discussions of the past few months. At first, the company had wanted work to move faster. When mistakes appeared, it had asked people to look more closely. When people could no longer keep up, it assigned review responsibilities and argued over accountability. After the customer shifted to another supplier, it began translating rules into checks. None of those efforts had been wasted. Together, they had brought the company to a more precise question than How many more checks should we add? A machine could continue proposing solutions and complete many routine tasks within its authorized scope. But before an action with real consequences was allowed to happen, on what basis should the gate say yes or no—and who should control it?
By nightfall, the office was growing quiet. Quotations for the next day were still being prepared. Someone was maintaining product information, sales colleagues were following up with customers, and the assistants continued assembling material. The owner had not ordered everyone to stop working until a perfect answer arrived. He merely added a line to the end of the meeting notes, so that next time he would not confuse two questions. If someone has approved an action, does that mean every version of it may proceed? And if a person or machine has the technical ability to carry an action out, does that mean the action should be allowed to happen here and now?
Chapter Ten — What Decisions Are Still Ours to Make?
The Company Had Never Needed Its Owner to Decide Everything
Work continued in an orderly fashion, not because every employee was incapable of making a mistake during those days, but because the company had long since agreed which decisions fell within ordinary job responsibilities and which changes required additional confirmation.
After the review, the owner did not immediately schedule another meeting with the technical team. He filed away the revised quotation and found himself leafing through a few pages of the company's old management policies. In the early days, when the business was still small, many arrangements had been settled by a handful of people talking across an office: how much a buyer could spend without further approval, whom to consult if a purchase exceeded its budget, who had to confirm that goods had arrived, and which contracts needed the owner's personal signature. The pages had yellowed, and the rules had been amended over the years, but they reminded him of something the recent arguments had almost obscured. The company had never expected him to stand behind every employee and decide every matter for them.
When the business was founded, he really had examined quotations himself, chosen suppliers, and sometimes even asked why a sample needed to be sent to a customer. It was not that he particularly enjoyed such details. There were simply too few people, and too little established procedure, to handle things any other way. As orders multiplied and employees learned their jobs, he entrusted purchasing decisions to the procurement manager, allowed sales staff to negotiate within defined limits, and left routine payments to finance under the established process. Whenever someone asked whether he was comfortable giving up that much control, he would say that as long as everyone knew which decisions belonged to whom, and when they had to stop and consult somebody else, there was no reason for the owner to become the bottleneck for the entire business.
He remembered a trip he had once taken to meet a customer in another city. His phone signal had been poor for much of the journey. When he returned, the warehouse had received its deliveries, finance had settled several payments that had come due, and sales had sent routine quotations to established customers. He had neither read every document beforehand nor called to approve each transaction. Work continued in an orderly fashion, not because every employee was incapable of making a mistake during those days, but because the company had long since agreed which decisions fell within ordinary job responsibilities and which changes required additional confirmation. The rules could be misunderstood, and some grew outdated as the business changed. Even so, they had allowed the company to function without any one person being constantly present.
Since introducing the assistants, he had briefly forgotten a lesson he already knew. When a machine made a mistake, he had wanted more human review. When people could not keep up with the reviews, he had tried to identify exactly who was signing off on what. Eventually, the company had written requirements about prices, the validity of source documents, and exceptional conditions into automated checks, reducing some predictable errors. They had kept patching the problems as they arose, until the discovery that an approved quotation was not the quotation eventually sent forced them to reconsider the underlying question. If an activity was already governed by a defined scope of authority, perhaps the important thing was not simply who pressed the last button.
He invited several department heads to talk, not about the customer who had moved toward another supplier, but about how the company had worked before there were assistants. The procurement manager explained that ordinary materials could be ordered by the purchasing team, provided the demand, supplier, and budget fell within their remit. If prices changed unexpectedly or a delivery date threatened production, they had to consult the people responsible for the business requirement. The finance manager added that knowing the payee's account details did not make a payment legitimate. There also had to be a valid business reason, the appropriate approvals, and conditions that remained satisfied at the time of payment. The sales manager put it more plainly: a salesperson might be entitled to contact customers on the company's behalf, but that did not mean they could promise any price or any delivery date.
The longer they talked, the clearer it became that the old procedures had always contained countless distinctions between may and may not. Some were written into contract forms or job descriptions; some were checked automatically when amounts exceeded a threshold; others depended on the handoffs between the warehouse, finance, and sales. They had never been perfect, nor always followed with equal care. But to dismiss them all as useless bureaucracy would make it impossible to explain how the company kept working when its owner was away, why finance sometimes returned a payment request, or why a delivery promise could not be made without confirmation from production.
“So we have always known that not every decision needs a fresh approval,” the owner said. The procurement manager nodded. “We usually knew whom to ask. What we weren't always precise about was exactly what they were approving.” The owner thought again of the quotation. It had the same customer, the same reference number, and the same salesperson. The version the manager had seen at two in the afternoon promised delivery in six weeks. The version prepared for dispatch promised delivery within four. If the question was whether a manager had approved the quotation, the record could answer yes. If the question was which promise the manager had approved, the answer became considerably less straightforward.
Agreeing to an Action Is Not the Same as Making It Happen
Being able to do something means having the means to make it happen. Whether it ought to be done, and under what conditions, must be judged separately against the applicable rules and working arrangements.
The sales manager opened the two versions of the quotation once more. This time nobody rushed to search for a new vulnerability. They simply tried to describe, in ordinary language, what each participant had done. The salesperson had requested a quotation; the assistant had helped assemble the information; the manager, acting within his responsibilities, had reviewed a document containing the products, prices, and delivery terms and agreed that it could be offered to that customer. The document had subsequently changed. The salesperson had still been able to attach it to an email and send it. The steps sat side by side in the same business record, but they answered different questions.
“When you said yes,” the owner asked the sales manager, “were you authorizing that employee to send any version of the quotation?” The manager shook his head. Of course he had approved the contents he had seen. “And if the price stays the same, but someone changes six weeks to four?” Delivery time, the manager replied, was an essential part of the offer. Before promising four weeks, someone would need confirmation from the people who actually managed delivery. The owner did not turn this into another argument about who had been sufficiently careful. Instead, he wrote down the first distinction: approval authority was the authority assigned to a role to give the necessary consent to a particular matter. The sales manager possessed that authority within its proper scope; an ordinary employee did not necessarily possess the same authority. But even the manager's approval had an object and limits.
He then asked the engineers why the salesperson had still been able to send the altered document. Because, they explained, an email system does not automatically know which delivery terms have been approved. Someone with a work account can compose a message, attach a file, and press Send. An assistant connected to the right tools may likewise be able to prepare material or even transmit it. The mail server can determine whether an account is technically permitted to send messages. It does not necessarily understand whether the attachment makes a new formal commitment that the company has not authorized. The correction the company had made after the review addressed the version-change problem they had discovered. It was not a decision to require the owner to read every email from then on.
The department heads recognized the distinction in other settings. A warehouse employee might be able to unlock a storage area without being entitled to hand over whatever goods a visitor requested. A finance employee might have access to the payment system without being allowed to transfer money whenever someone supplied an account number. A driver might be able to start a vehicle without being authorized to enter every road at every hour. Being able to do something means having the means to make it happen. Whether it ought to be done, and under what conditions, must be judged separately against the applicable rules and working arrangements. The engineers observed that this was hardly unfamiliar in their work. It was simply that, when people had stood at different points along a process, the distinctions had often been maintained through habit rather than stated explicitly.
The owner turned the sheet of paper around so everyone could see the three lines taking shape. The first was approval authority: who was entitled to grant a required approval for a particular matter. The second was legitimate execution authority: under the organization's valid rules and the conditions currently in force, who had the authority to let this specific action proceed into execution. The third was actual execution capability: who possessed the practical or technical means to cause the action to happen. A person might possess more than one of these; sometimes all three would be present in a single role. They did not have to be distributed among three people or implemented by three systems. But treating them as interchangeable could leave the organization unable to explain what mattered most when something went wrong.
Hearing the phrase execution authority, the sales manager wondered whether the company was proposing yet another person to approve dispatch at the end of the process. The owner said that was precisely what he wanted to avoid. They were not looking for another senior employee to sit at the last step. They were asking whether a particular action was entitled to occur under the company's legitimate arrangements. An ordinary sales activity that fell within the employee's assigned authority, used valid information, and met the applicable conditions might need no additional approval at all. A formal promise involving an excessive discount or an unconfirmed delivery date could not acquire the same legitimacy merely because an employee had an email account and an assistant could draft messages. By execution authority, they should mean authority under the organization's rules, not the mere possession of a tool capable of getting something done.
The procurement manager offered a counterexample. “If I approve a routine purchase under company policy, surely the employee can place the order. Why force approval and ordering apart?” The finance manager replied that there was no need to separate them artificially. A valid approval might supply the final condition needed for that purchase to proceed. But the requirement had to remain valid, the quantity and price unchanged, and the employee authorized to place that kind of order in the first place. If a manager approved one hundred units of a standard material and someone later increased the quantity to two hundred, the word Approved on the earlier record would not authorize the revised order. The distinction, she pointed out, had existed all along. Previously, working habits and system restrictions had often taken care of it. Now it needed to be stated more carefully.
The operations engineer had an example of his own. Suppose production approved maintenance on Sunday night. That did not mean an engineer could carry out the same maintenance on Monday morning without checking the state of the factory. He might have an account that let him make the change, and the technical procedure might be faultless, but the business conditions could have shifted. An approval issued for one time and set of circumstances could not simply be carried forward into another. Conversely, if the maintenance was already within an authorized time window, scope, and work plan, there was no need to wake the owner at midnight to ask him to say yes again.
“So what decision is ultimately left to us?” the owner asked. This time nobody immediately replied that he should sign everything himself. The sales manager considered the question and said that the organization could decide in advance which formal commitments needed delivery confirmation and which discounts fell within sales authority. Finance could determine the evidence and limits required for payment, as well as which roles could not unilaterally alter them. Those rules would not need to be renegotiated for every ordinary transaction. What people needed to retain was not a grip on every last operation, but the ability, through legitimate governance arrangements, to determine how rules were established, whom they governed, and when they could be changed.
The owner understood that this did not make him the sole source of every rule. Finance had specialist responsibilities for payment controls. The business teams had to judge whether delivery promises were realistic. Some obligations arose from contracts, laws, or other arrangements that could not be set aside simply because sales was in a hurry. Sales could ask for an exceptional delivery date but could not declare that production had confirmed it. Finance could demand evidence for payment but could not rewrite the underlying business facts for its own convenience. Not every condition assigned to a different role required a meeting or a vote. Precisely because those roles were responsible for different matters, one person's consent could not always substitute for another's.
Bringing the Rules to the Moment of Action
People do not need to stand over every email and every order at the final second to prove that they still govern the company. What matters is whether a legitimately governed organization can still determine the scope of action, the conditions that must be met, and who is entitled to change those conditions—and whether a particular action must respect those arrangements before it becomes real.
At this point, an engineer suggested an exercise that would take the discussion beyond general principles. Instead of asking how many more approvals they needed, they could take an actual quotation ready to go out and identify exactly what had to be true before it left the company. The sales manager brought up a draft from an ordinary piece of business. It specified a customer, a combination of products, prices, delivery arrangements, and service terms. The assistant had prepared the material, and the salesperson was ready to follow the normal process. There was no new incident and no exceptional urgency. For that very reason, it was a good example with which to test what they thought they understood.
The owner asked the group to find their answers in existing company policy rather than invent a fresh set of rules. Product information and prices had to come from currently effective, approved sources. Discounts had to fall within the authority of the sales role. Any unusual commitment required confirmation by the people responsible for it. The intended recipient and the content of the document actually sent had to match what had received the necessary approvals. When a requirement was established by policy and could be verified unambiguously, an assistant or business system could perform the check automatically. Commercial exceptions that called for judgment could go to the people authorized to decide them. As they worked through the quotation, they realized that the central question was not how often the machine had reasoned about it. It was whether this particular action, about to have an effect beyond the company's walls, satisfied the conditions that applied to it.
The owner wrote two words on the whiteboard: Execution Admission. He was not presenting the term as a product the company had purchased, nor suggesting that all they needed was another button marked Admit. It was a way of asking a more exact question. When a particular action was about to enter the stage where it would actually be carried out, did it qualify to proceed under the rules, authorizations, and necessary constraints then in force? If a required condition was not satisfied, neither the assistant's confidence nor the apparent completeness of the paperwork nor an approval obtained on an earlier occasion was sufficient to establish that this action could now take place.
The finance manager immediately asked whether an execution-admission mechanism would have prevented the advance payment for the suspended equipment project. The owner said they could not assume so. If the system checked only the contract amount, account details, and signatures, without knowing the project had been put on hold, it could still reach a positive result within the limits of the information it possessed. Essential conditions had to be defined by people who understood the business, and the facts on which those conditions depended had to reflect the current state of affairs. Execution admission was not a magical ability to read everyone's intentions. Nor did it release the company from responsibility for its information sources, policies, and exceptional cases.
The procurement manager thought of the customer who had stopped advancing the earlier sales opportunity. Even if every detail of the quotation had been current and the delivery terms entirely feasible, the customer might still have chosen someone else because of price, established relationships, or its own plans. An action permitted under company policy was not necessarily a commercial success. A properly authorized and compliant quotation did not have to appeal to the customer. The owner agreed. That was precisely the distinction he wanted to preserve. An organization could define the conditions its actions must meet without claiming to predict every outcome in the market. Rules could restrict actions that should not proceed; they could not take over every business judgment.
The engineers warned that if the company sent every check back to a person, it would soon find itself buried in approval queues again. The sales manager laughed. “Haven't we already tried that?” Since introducing automated checks, the company could verify current prices, discount limits, and necessary approvals at machine speed. The task now was not to make an employee count the figures again. It was to identify the specific action, document, and customer to which the check applied, and to establish whether the result was still valid after the conditions changed. The earlier improvements had not been wasted. They had made many clear requirements easier to enforce. If execution admission merely became a new name for making everything wait for a human signature, they would have missed the point of the whole discussion.
They went back to the work of ordinary employees. Salespeople could still contact customers, prepare drafts, and discuss proposals that did not yet constitute formal commitments. Purchasing assistants could compare suppliers and prepare routine orders. Finance assistants could check records and assemble payment requests. Activities within an employee's authorized scope, supported by valid information and subject to no additional restrictions, did not require the owner's repeated permission. But when a transaction was about to become a formal quotation, an actual payment, an external commitment, or a change to a system's state, the necessary conditions for that kind of action had to be established in a way appropriate to it. Different operations and levels of risk need not impose equally burdensome procedures. Nor could every relevant consideration be reduced to a numerical test.
The owner began to see that his earlier worries about machines had framed freedom and loss of control as the only alternatives. If assistants were allowed to work independently, it seemed they might turn any idea into an action. If they had to ask permission for everything, people would be buried under endless requests. He could now imagine a third possibility: let intelligent systems continue to analyze, prepare, and propose; let employees make judgments within their roles; and let the organization's legitimate rules determine which requirements could never be set aside when a proposed action approached the real world. Such arrangements would not fit every situation without adaptation, and they would require continuing maintenance. But they would no longer rest on the hope that machines never made mistakes or that one human reviewer could keep up forever.
He reminded the others that deciding who could set the conditions—and who was authorized to change them—would be harder than writing a checklist. Business teams sometimes needed to respond to changing markets. Finance sometimes had to defend a restriction that schedule pressure could not override. If a person trying to complete a task could simply change a necessary condition to Satisfied, the rule might remain visible on paper while losing its force in practice. The company needed legitimate arrangements for distributing those powers, allowing those responsible for different matters to take part in governance without letting a passing business preference quietly displace a binding restriction.
When the meeting ended, the owner left the terms on the whiteboard. They were not as exciting as a new software screen, and they could not recover the opportunity the company had lost, but they had changed what the question What decisions are still ours to make? meant to him. People do not need to stand over every email and every order at the final second to prove that they still govern the company. What matters is whether a legitimately governed organization can still determine the scope of action, the conditions that must be met, and who is entitled to change those conditions—and whether a particular action must respect those arrangements before it becomes real. Whether a particular employee, manager, or system has the practical means to act is a separate question that cannot be mistaken for permission.
As he packed away his laptop, an engineer asked, “We can now explain the conditions under which this quotation ought to be sent. But if someone hasn't met them, and the email system still lets them press Send, what makes us so sure it won't go out anyway?” The owner looked at the whiteboard. He did not suggest another round of checks by the sales manager. They had spent the evening defining what qualified an action for execution. How to make those requirements binding on every relevant path to a consequential operation remained unanswered. The company had finally distinguished what should be permitted. It had not yet established who—or what—could stop an action when it should not be permitted.
Chapter Eleven — Who Can Actually Make an Action Happen?
If the Answer Is No, Will the Action Stop?
The policy had not vanished, and the checks had not lied. There was simply no effective connection between them and the action itself. At that point, the rules remained on paper and in meeting minutes, while the real world carried on without them.
The question did not disappear when the meeting ended. The following morning, the owner arrived to find the sales manager and an engineer examining the quotation process on a computer. They were not preparing another elaborate report. They had laid out the steps from drafting to dispatch and were trying to establish one very practical detail: if a required condition was not satisfied, could the person sending the email still send the formal quotation?
The sales manager thought the answer ought to be simple. The company had established requirements for prices, delivery terms, and service commitments. If the system found that a requirement was not met, it should stop the quotation from going out. But the engineer opened the email application and explained how things had always worked. The business system checked a quotation first; an employee then sent the approved document. The business system could certainly refuse to approve an invalid quotation, but an employee could still open a mailbox, attach another document, and press Send. The mail server knew that the account was allowed to send email. It did not know whether the attachment's promise of delivery in four weeks was the one the manager had approved when he saw six.
The owner remembered the earlier customer opportunity that had gone badly and the later discovery of a quotation altered after it was checked. The review had taught them that examining one document did not mean the document eventually dispatched would be identical. Their subsequent discussion had clarified that the technical ability to send email did not confer legitimate authority to make any formal promise on the company's behalf. Now they had reached the question beyond both of those insights. If the company knew a particular quotation ought not to go out, what would give that ought not any force at the moment someone tried to send it?
“Then we stop sales staff from sending their own emails?” someone suggested. The sales manager shook his head immediately. His team wrote to customers every day: explaining products, arranging meetings, answering routine questions. Much of that correspondence involved no formal commitment at all. If every message needed to pass through a quotation-approval process, they would return to the days when work was finished but had to sit waiting for a signature. The engineer agreed. A mailbox served many purposes; the fact that formal quotations needed particular controls did not mean one set of commercial rules should govern every email.
They began to describe the boundary more carefully. Ordinary correspondence could continue under existing job responsibilities. A document that the company treated as a formal quotation, however, needed a defined route by which it acquired that status and was sent. If a quotation merely received a Verified label inside a business application and could then be freely edited and dispatched with another tool, the label did not govern the formal act. Conversely, if every route recognized for issuing an official quotation had to confirm the relevant conditions and the content being sent, employees could still entrust much of the preparatory work to assistants. Compliant quotations would proceed quickly. Those that failed the requirements would stop until the problem was resolved.
“But someone can always copy the text into an ordinary email,” a salesperson pointed out. The owner did not dismiss the objection. A company could not make it physically impossible for a person to communicate by every imaginable means, nor could an internal instruction control every outside channel. The first task was to define which actions counted as managed, formal business acts; which channels the company recognized as capable of creating those effects; and whether its existing systems, permissions, and procedures could constrain those channels. If another company-recognized route could produce the same formal quotation without equivalent checks, the new controls could not be called complete. If someone made a private promise outside authorized channels, that raised further questions about organizational discipline, contractual effect, and governance. A single Send button could not settle all of them.
The owner began to understand why the engineers kept asking whether there was anything that could actually stop an action at the last step. They had spent weeks discussing who should make a request, who could approve it, and what information had to be checked. All of that mattered. Yet it was possible for a reviewer to say no quite conscientiously while a real-world tool could still carry out the operation. The policy had not vanished, and the checks had not lied. There was simply no effective connection between them and the action itself. At that point, the rules remained on paper and in meeting minutes, while the real world carried on without them.
Different Gates Lead to Different Actions
The effective solution was not to place an approval form beside an order and hope the person submitting it would comply. It was to make the necessary conditions binding on the controlled operation at the moment the order was formally submitted.
When the procurement manager heard that sales was examining the point at which quotations went out, he brought along a routine purchase order prepared under the company's revised procedures. Purchasing, he said, had faced similar questions for years. Employees could request prices from suppliers, ask assistants to compare options, and prepare orders internally. But when a purchase was formally placed, the procurement system checked the employee's spending authority, the budget, and any required approvals. If the amount was outside the permitted range or necessary conditions were missing, the system could not simply display a warning and let that same controlled submission go ahead. Nor did an ordinary order that satisfied the rules require the owner to repeat his consent every time.
There was nothing magical about this arrangement. It was a familiar business practice, often summarized as “purchases must follow procedure.” If a manager approved an order for one hundred units and an employee subsequently increased it to two hundred, the system ought to recognize that the revised order was not the one that had been approved. A change of supplier or delivery terms might likewise require another check. The effective solution was not to place an approval form beside an order and hope the person submitting it would comply. It was to make the necessary conditions binding on the controlled operation at the moment the order was formally submitted.
The finance manager returned to the subject of payments. The advance payment for the suspended equipment project was still in their records, but she was not trying to reopen the search for someone to blame. She wanted to make a different point. In an ordinary company, drafting a payment request was not the same as transferring funds. A business department could submit a request; finance could examine the supporting documents; an authorized person could approve it; and the actual movement of money would still pass through the company's payment system and, ultimately, the bank or other payment provider's controls. Those responsibilities were distributed for a reason. A single document was not supposed to move from proposal to payment on one person's word alone.
She warned them, however, that the appearance of several names on a workflow was no guarantee of safety. Suppose the payment system recognized only an Approved marker, without checking which account and amount that marker referred to. The funds transferred could still differ from the payment previously reviewed. If, by contrast, the actual payment stage verified that the beneficiary, amount, and relevant authorization still matched the approved transaction, then the control would finally reach the moment when the funds were about to move. Bank controls, separation of duties, and business-system rules could each provide part of that protection. A new idea did not require the company to deny the value of arrangements that were already doing useful work.
The operations engineer gave them a more tangible picture of a final gate. A production manager could authorize maintenance on a machine that evening, and an engineer's account might allow the necessary start or stop commands. Yet some equipment also depended on its present state, physical conditions, and safety interlocks before accepting a command. Approval of the plan did not mean every action should occur regardless of the machine's condition. When a necessary safety condition was not met, a control that prevented the action was not taking over the production manager's planning decisions. It was enforcing another set of constraints that mattered just as much.
There were limits to that comparison. Not every machine had adequate interlocks, and any interlock could protect only against conditions it was designed to detect and control. A device that sensed whether a safety guard was closed could not decide whether manufacturing a particular order made commercial sense. A controller that prevented one hazardous startup could not guarantee a factory would never experience another failure. The owner thought this was precisely why the example was useful. It separated the question of who proposed and approved a plan from the question of what conditions still had to be met before a physical action could occur.
Putting the examples side by side did not reveal a single universal machine waiting to be installed. Purchase orders became formal in the procurement system. Payments passed through financial processes. Factory equipment acted through local controls. Official sales quotations depended on documents and channels of communication. The software differed, as did the people and the relevant operations. Some safeguards came from roles and procedures, others from applications, and still others from machinery. Nobody suggested moving the entire company onto one new server simply to have a place from which to issue the last word.
The engineer drew two uneven lines on the whiteboard. On one side were the activities of proposing tasks, gathering information, comparing options, and reaching decisions. On the other were the acts that caused orders to be placed, money to move, equipment to change state, and formal promises to reach customers. The two sides did not always belong to different people or machines. Sometimes one employee handled everything; sometimes an assistant could carry out much of the work automatically. But once the company designated a kind of action as controlled, the conditions governing it could not simply evaporate at the execution stage because the person—or assistant—preparing it felt confident.
“Perhaps these are all instances of the same responsibility,” the engineer said, indicating the places where conditions had to take effect on actions. The owner asked whether he meant another kind of approval. Not exactly. Approval supplied necessary consent, and business rules established applicable restrictions. What they were discussing was how that consent and those restrictions could be made effective where an action actually occurred. The control might use an approval record, but it was not necessarily another approval. It might run a software check, but it was not merely a green message on a screen. It needed a real controlling relationship to the managed action: within its defined scope, a way to prevent an operation that did not satisfy the requirements from proceeding.
Intelligence Can Keep Working, but Action Needs Boundaries
The owner felt that this distinction made room for freedom rather than diminishing it: the company did not need to restrict everything a machine could think of, but it did need to decide what the machine could ultimately cause to happen.
The owner asked the engineer to express the responsibility on the whiteboard in language all the departments could understand. After some discussion, they settled on Execution Control Layer. By layer, they meant first a distinct responsibility in the organization's design and reasoning. When a person, an agent, or another system proposed an action that would have consequences in the real world, the relevant control responsibility had to bring the currently applicable execution-admission conditions to bear on that specific action, determine whether the controlled process could proceed, and ensure that a noncompliant action was stopped wherever the control genuinely applied.
The sales manager worried that the term sounded like something the company would have to buy. The owner reminded him that they had just discussed procurement, payments, and equipment controls that already existed. There was no reason to discard useful arrangements simply because they had found a name for the responsibility they served. An Execution Control Layer did not have to be a standalone software product, a dedicated rack of hardware, or a single management console. In one company, it might largely consist of established workflows, job permissions, and business systems working together. In another, the responsibility could be distributed among payment channels, application interfaces, and equipment controllers. The important questions were not which devices carried the label but who set the boundaries, who checked the present conditions, which controlled component could permit or refuse an operation, and why the final action had to respect that result.
The procurement manager immediately saw a difficulty. “If the control points are spread around, won't we end up with nobody really responsible?” The engineer admitted that this could happen. Distributing controls did not automatically make them effective. Permission to submit a purchase order did not itself authorize a payment, and approval of a sales quotation did not authorize a production machine to start. Each control had to operate within its remit, while any dependencies between them had to be established where the business required them. Otherwise, the whiteboard might show several gates even though, in practice, everyone assumed somebody else was watching a route that nobody actually controlled.
“Wouldn't it be easier to send every operation to one place for checking?” the owner asked. Centralization could make policy management and recordkeeping easier, the engineer replied, and might suit some businesses. But there was no logical requirement that every controlled action pass through a single machine. Payments had their own operational conditions. Factory equipment could require local safeguards that did not depend on the response time of a distant network. Formal sales documents presented another sort of problem entirely. Forcing every operation through one location could introduce a new concentration of failure, or make some physical safeguards ineffective. Distributed arrangements brought challenges of coordination and consistency. The choice had to follow the business, the controls already in place, and the risks involved—not a picture of one all-powerful box.
The sales manager had another concern. Did an Execution Control Layer mean assistants could only offer advice, never do actual work? The owner remembered why they had welcomed those tireless new colleagues in the first place. He had no desire to hand every repetitive task back to employees. The answer was not to prohibit machine action. Where the company had legitimately granted authority for a defined class of work and the necessary conditions were met, an assistant could continue preparing, submitting, and even bringing about real operations through controlled routes. It could remove delays and perform certain explicit checks more consistently than people. What it could not do was treat its own confidence in a task as sufficient proof that every constraint on that action had been satisfied.
“What if the assistant prepares a document and checks it too?” the owner asked. “Aren't we back to the machine declaring itself approved?” The engineer did not answer with an absolute prohibition. In ordinary software, combining data preparation with some automated validation was common and could be adequate for low-risk work. But if the company required a critical condition to be protected from alteration by the party seeking to act, an assistant's own assertion that its checks had passed was not a substitute for controlling the operation itself. Separating intelligent decision-making from execution control as responsibilities helped them identify the relationship that needed protection. Whether a particular implementation actually achieved the necessary independence, isolation, or resistance to bypass would have to be demonstrated in that system, not assumed from its architecture diagram.
To keep the discussion practical, they returned to the sample sales quotation. The assistant could still create a draft from the customer's needs, and the salesperson could revise it as often as necessary. An exceptional discount would require approval from the proper role. A delivery date outside confirmed limits would need to be checked before becoming a commitment. At formal dispatch, however, the managed process had to establish that the content actually being sent was the content that met those conditions. It could not use a check performed on an earlier version as permission for a later one. If a necessary condition was missing, the right outcome was to pause or refuse that formal dispatch and explain what had to be supplied—not issue a warning that nobody was obliged to heed.
None of this required the assistant to behave like a nervous employee asking permission after every sentence. It could try different wording in a draft, compare product combinations, and propose ambitious commercial ideas. People could likewise debate whether an existing rule ought to change. Discussion and preparation were not themselves a payment, a binding promise, or a machine action. It was when a specific act approached the boundary the organization had defined as consequential that the applicable constraints needed to become effective. The owner felt that this distinction made room for freedom rather than diminishing it: the company did not need to restrict everything a machine could think of, but it did need to decide what the machine could ultimately cause to happen.
The finance manager cautioned against romanticizing the power to refuse. If the company encoded a mistaken rule with perfect rigor, the system might faithfully obstruct legitimate business. If an important risk had been omitted from the rules, an operation could pass every check and still produce an outcome the company had never intended. Their lost sales opportunity had not arisen because every figure in the quotation was mathematically wrong; the underlying information and the service promise had been inadequately established. Even where known disqualifying conditions could be enforced, people would still have to examine whether the conditions themselves were reasonable, whether the supporting information was current, and who could authorize changes. Making rules effective at the point of action did not mean the rules no longer needed human governance.
The operations engineer added that a gate only protected a route if the action actually passed through it. If production equipment had another path for receiving commands, or a system could substitute critical data after the check, a neatly drawn control responsibility proved nothing about those operations. The owner wanted to ask how they could demonstrate such properties. The engineer suggested that they not rush to a polished answer. What they had established was a way of approaching the problem, not proof that any existing implementation was trustworthy. Reliability would require examining where rules came from, whether state information deserved trust, how completely the controlled routes were covered, and whether a refusal would really take effect when required.
Near the end of the meeting, the sales manager reopened their example quotation. This time, he did not ask whether they needed one more signature. He asked whether the final sending step could establish that the current version met the conditions for that particular action—and, if not, what would stop it. The finance manager thought of the payment process; the procurement manager of the conditions on a submitted order; the operations engineer of the local restrictions before a machine accepted a command. They did not picture the same piece of equipment. They had begun, however, to ask the same kind of question about their different jobs.
The owner switched off the projector and looked at the two areas still marked on the whiteboard: intelligent work that kept producing ideas and proposals, and actions that could change the world outside the discussion. He had not drawn an impassable wall between them. Many legitimate actions ought to pass through without friction, and the company still needed intelligence to do useful work. Between the two areas he had left only a plain reminder: necessary conditions must not remain trapped in records of approval and checks; they must actually govern the actions within the company's controlled scope. Why those controls should be trusted, whether there were routes nobody had noticed, and how anyone could verify the answers were questions the company had barely begun to address. They would be the next difficult part of the story.
Chapter Twelve — Building Reliable Constraints Without Assuming Machines Are Infallible
The Gatekeeper Can Be Wrong, Too
Making a decision process repeatable can reduce arbitrariness. It does not relieve the people who write the rules of their responsibility to think carefully.
The whiteboard stayed in the meeting room for another two days. People passing by occasionally stopped to read it. The words Execution Control Layer made some wonder whether the company was preparing to install another system. The owner decided to leave the writing where it was. Their new understanding was worth holding on to, but finding a name for a responsibility was no reason to suppose that the difficulties surrounding it had been resolved.
The next day, the finance manager came to discuss a proposed payment procedure. She had no incident to report and no fresh complaint about another mistake. Instead, she pointed to the whiteboard and asked, “At first we trusted the assistant, and it used old information. Then we asked people to review its work, and they missed things too. Now we're talking about a protected process that decides whether an action can proceed. What happens if the checking program itself is wrong?” She was not arguing against the improvement. If anything, she was asking because she took it seriously. The advance payment for the suspended equipment project had shown her how perfectly ordinary-looking paperwork could lose touch with reality. Why should another check, merely because it was more firmly connected to execution, be immune to the same problem?
The owner asked an engineer to join them. He made no claim that the new arrangement could never fail. Instead, he drew a familiar payment request on the board and began with a modest question. Suppose the company had set a limit on how much someone in a particular role could approve in a single transaction. Could a system reliably test whether an amount exceeded that limit? Yes, said the finance manager; their existing software already did that. Could it also check whether the recipient was on the approved supplier list, or whether the authorization had expired? Usually, she replied, provided the records were current and their sources could be trusted. The conversation was not, it turned out, about finding a gatekeeper who was always wise. It was about defining conditions that could be examined one at a time and verified on their own terms.
An experienced employee reviewing a request might remember the contract, a supplier's habits, or something a colleague had said over lunch. That experience could be invaluable, but memory and fatigue could also lead the employee astray. AI could help arrange scattered material into a form that people could examine, yet that did not make it the proper authority for declaring every business judgment correct. Other questions were different: was an amount within a defined limit, was a required approval still valid, did the proposed execution time fall within a prohibited period? If such questions were specified precisely and supported by trustworthy information, the same tests could be applied repeatedly. A successful check could at least establish which defined conditions had been met, rather than rest on an assistant's assurance that everything seemed fine.
The procurement manager heard about the discussion and arrived with an ordinary purchase order. He reminded them that a deterministic check was not automatically a correct one. If someone entered a financial limit with one zero missing, the most consistent program in the world might enforce the wrong restriction with perfect consistency. If a long-discontinued supplier remained on the valid-supplier list, a successful match would prove only that the supplier's name was still on the list—not that the purchase ought to take place. Even before a rule was written into software, two people could read the same sentence and mean different things by it. Making a decision process repeatable can reduce arbitrariness. It does not relieve the people who write the rules of their responsibility to think carefully.
Rather than rush to impose dozens of additional rules, they examined the requirements they already had. Why did a particular restriction exist? Which role had proposed it? What did it cover? Under what circumstances would it cease to apply? Who was entitled to amend it? The owner recognized these as ordinary questions of management. They were questions the business had faced when setting budgets, signing contracts, and revising internal procedures. The difference was that a sentence on paper could once be interpreted by an employee facing a particular situation. If the same requirement was to affect real actions automatically and at machine speed, ambiguities could no longer be left for someone to resolve by guesswork at the last moment.
The finance manager returned to the suspended equipment project. Suppose the company had defined its payment rule as nothing more than: the amount must match the contract and finance must have approved the transaction. The problematic payment could still have passed. What had been missing was the business requirement that no further advance payment be made while the project was suspended, together with a means of getting that suspension status to the payment process in time. The owner agreed. A system was not safer simply because it contained more rules. The people legitimately responsible for governing a class of action had to identify which conditions were indispensable. A beautifully recorded approval could not make up for a necessary condition that nobody had thought to include.
Someone Must Be Able to Explain What the Rules Know
If an Execution Control Layer has value, it is not because it becomes everyone's ultimate boss. It is because it takes the necessary conditions that have been legitimately established, and that still apply, seriously at the point where a specific action is about to occur.
A week later, the engineers invited finance and sales to bring a routine piece of work each to a tabletop review. The engineer made a point of saying that this was not a performance in which the assistant would demonstrate its intelligence, nor an invitation to throw a document at it and ask it to invent a new company policy. They were going to examine the requirements the business had already established and identify what facts would be needed to evaluate them. On the table lay a normal payment request, a quotation being prepared for a customer, and a sheet on which the departments could record their proposed changes.
The first difficulty with the payment was not the amount. The beneficiary's account number was complete, and the name matched the supplier on the contract. But the engineer asked, “How do we know this is still an account we're allowed to pay?” A finance employee said the supplier information was in the company's records. The procurement manager added that changes to a supplier's bank details normally needed separate confirmation. An email claiming to come from a supplier could not, on its own, become the new authoritative account record. If it could, an assistant preparing a payment might read that email, propose a change, and at the same time treat the proposed change as already legitimate.
As they followed the issue back through the process, it became clear that comparing two account numbers was the easy part. The harder questions concerned who was allowed to confirm a change of account, what evidence that confirmation required, how it was recorded, when the old account ceased to be valid, and whether a casual note from an employee could supersede the official record. The finance manager insisted that business staff were entitled to request a change and explain its urgency, but not to dispense with account verification because a payment was due. The procurement manager did not object. Knowing which details were negotiable and which required confirmation by an appropriate role made it easier to resist unreasonable pressure, not harder.
The sales manager opened the quotation. The customer, products, prices, and delivery date were all present. Beside them, the assistant had supplied a reassuring statement: Price check passed. But the sales manager remembered that the customer had asked two days earlier to change part of the service package from remote assistance to on-site support. Did the quotation reflect that request? The person responsible for product information consulted the latest service arrangements and established that the wording needed to be revised. The price check had been correct. It had simply never been intended to judge every aspect of the service promise. Interpreting a green checkmark as proof that nothing was missing from the entire quotation would be to turn a limited result into an unlimited guarantee.
The engineer suggested that, for each necessary check, users should at least be able to answer three questions: what was being checked, which currently valid information the decision relied on, and what would actually happen if the check failed. A salesperson should not see a pleasant green tick without knowing whether it meant the prices were valid or all the requirements for a formal commitment had been satisfied. Finance should not interpret Contract exists as Project is still active. It might be impossible to explain every detail on a single screen, but the verification behind the screen had to withstand questions. When challenged, the answer could not simply be, “The system said yes.”
Someone wondered whether this would force employees to inspect every source document again, bringing back the exhaustion of endless approvals. The finance manager drew a distinction. Establishing and maintaining trustworthy records was not the same as requiring every reviewer to make the same phone calls every morning. Valid supplier accounts could be maintained through an agreed procedure and compared with current records when a payment was made. Authorized service terms could be published by the relevant department and referenced when a quotation became formal. Machines could handle a great deal of repetitive matching, while people concentrated on setting the rules, responding to genuine changes, and resolving exceptions that could not be reduced to existing conditions.
Yet the fact that a record was designated as trustworthy did not mean its contents remained true forever. The operations engineer brought up factory maintenance schedules. Permission confirmed yesterday for work tonight might no longer be appropriate if production orders changed that afternoon. Authority that was valid last week might have expired after a staffing change. If a necessary status had not been updated, even a flawless control program would be making its decision about yesterday's world. For an action that would affect reality, missing or stale information could not be treated as satisfying a required condition. Sometimes that would mean waiting until sufficiently reliable evidence was available.
The owner raised another uncomfortable possibility. Suppose a critical status check was temporarily unavailable, but the business was under intense time pressure. Could the system let the action proceed and have someone supply the paperwork afterward? The managers resisted giving one answer for every kind of activity. Ordinary information gathering could continue; drafts that made no formal commitment need not all be frozen. But for a payment, formal quotation, or specified equipment operation that company policy explicitly required to be supported by valid evidence, urgency could not by itself turn absent evidence into a pass. If an exception really was necessary, someone with the proper authority needed to establish it through a legitimate procedure. The party trying to complete the task could not simply declare itself exempt.
“But if we grant an exception today, how do we know whether it's still valid tomorrow?” the procurement manager asked. He was remembering stale prices, out-of-date lead times, and the suspended project that the payment process had forgotten. He was beginning to see that rules and permissions had a time dimension. They could have expiry dates, specified objects, or dependencies on a business state that might cease to hold. Approval granted last month could not be treated as a permanent pass for every similar action today. The whole point of making these distinctions was to prevent the company from stumbling again over the comfortable assumption that because someone had once said yes, the answer must still be yes.
At the end of the review, the engineer gathered the observations into a record that the responsible departments would still need to confirm. He had not set finance's spending limits, determined sales's service commitments, or allowed the assistant to expand its own operational authority. Those decisions remained with the legitimate business roles, sometimes involving several distinct conditions that no single department could waive. If an Execution Control Layer has value, it is not because it becomes everyone's ultimate boss. It is because it takes the necessary conditions that have been legitimately established, and that still apply, seriously at the point where a specific action is about to occur.
The Action That Happens Must Be the Action That Was Checked
What a company can realistically strive to do is acknowledge those imperfections, state its indispensable requirements clearly, make them effective wherever their satisfaction can be verified, and remain alert to omissions, exceptions, and consequences beyond its control. Such assurances are limited, but their limits do not make them worthless.
The discussion became more difficult when a salesperson raised what seemed like a tiny question. He opened a quotation that had already passed its checks and asked whether he would have to run everything again if, just before sending it, he noticed that the customer's form of address sounded awkward and changed a single word. Someone nearby thought that was excessive; changing a salutation would not alter a price. The engineer did not announce that every edit must trigger the entire process from the beginning. Instead, he asked which kinds of changes the company had decided could be made without altering the action that had been verified, and which changes touched approved conditions. Until that distinction was established, a shared filename was no basis for assuming that the old and new contents were equivalent.
They revisited the earlier faulty quotation—the one sent before the customer shifted its attention toward another supplier. In that case, the initial problem was not the absence of a recorded approval; the delivery terms and service commitments had been inadequately verified to begin with. But even where those conditions had been checked properly, imagine an employee subsequently shortening the delivery estimate from five or six weeks to three, then sending the revised document under the earlier version's approval. The check would govern a document that was already in the past. One thing had been authorized; another would reach the customer. The engineer explained that preserving an Approved record was not enough. At formal dispatch, the company also needed to establish which quotation, recipient, and approved substantive terms that record actually referred to.
The finance manager drew the parallel with payments. If the account or amount at the time funds were transferred differed from the one that had been approved, a shared project name did not make the old authorization applicable to the new payment. For the procurement manager, the corresponding details were quantities and suppliers; for the operations engineer, the machine scheduled for maintenance and the machine actually receiving a command. The scenarios were not identical, and their verification requirements could not be copied mechanically from one to another. But they raised the same question: was the object and set of conditions previously examined the very one about to produce an effect in the world?
“If we preserve the contents of the document,” the sales manager asked, “does that guarantee what we send is correct?” The engineer said no. Preservation and comparison could help show that the same contents were being used, or expose a change in something important. But the original contents might already be wrong for the customer's circumstances. A quotation that had not changed by so much as a character could still lose the order because its delivery date was too late. The company could insist that a formal commitment satisfy defined conditions; it could not thereby guarantee that the customer would accept it. A control mechanism needed to be clear about what it could prove and equally candid about what it could not.
Another possible failure lay outside the normal route. Suppose the business system now prevented an unverified formal quotation from being sent, but an employee could use another method, still recognized by the company as capable of making the same formal commitment, without equivalent checks. The rules would not cover the full set of actions they were intended to govern. The operations engineer had the same concern about machinery. If a machine accepted a command through two different interfaces and only one was subject to the restriction, it would be inaccurate to claim that the action itself was fully controlled. They began identifying the routes through which consequential actions could actually occur—not to force every email and machine into one central service, but to discover which paths could produce the effects the company meant to regulate.
The owner raised a practical problem. Systems needed maintenance. What happened when the control program was unavailable? If every managed operation stopped, the business could suffer new losses. The engineer agreed. Refusing a noncompliant operation was not cost-free, and erroneous denials, unavailable state information, or technical failures could disrupt legitimate work. Each business would need recovery procedures suited to its risks, legitimate ways for authorized people to intervene, and emergency arrangements that did not quietly become convenient everyday bypasses. But where a particular action was expressly required to meet certain conditions before proceeding, the company could not treat control unavailable, therefore allow as an innocent default requiring no explanation.
Only then did the owner feel that the gate on the whiteboard had become something real. It was not a comforting metaphor, a door that would open or close merely because the people running the company meant well. It described a set of relationships that had to withstand concrete questions. Who established the rules, and were they still applicable? Where did the facts used for checking come from, and why should anyone trust them? Which specific action had been permitted? Did the instruction finally issued still match what had been checked? And when a necessary condition was not satisfied, could the operation within the controlled scope actually go ahead? Ambiguity at any of these points might rob the arrangement of force at precisely the moment it was needed.
He did not ask the engineers to concentrate every control on one computer. Procurement's order-submission checks, the payment system's account and authorization controls, the formal quotation process, and safety interlocks at factory equipment already lived in different parts of the business. They could perform distinct duties and cooperate where necessary. What mattered was that the protective relationship actually existed: the party subject to a constraint could not simply amend an indispensable condition because it wanted the operation to succeed. Proving those relationships, examining possible faults, and testing attempts to bypass the intended route would require specific engineering work and continuing review. A diagram in a meeting room was not proof.
Before leaving, the sales manager asked one last question. “If we really keep this gate shut when it should be shut, does that mean we'll never lose another customer?” Several people laughed, but nobody treated it as a foolish question. The customer who had chosen to give another supplier priority for the earlier project was one reason all these discussions had begun. Naturally, people wanted a way to avoid feeling that disappointment again. The owner said rules could help prevent the use of obsolete information, unauthorized discounts, missing approvals, or a document that borrowed the approval of some other version. But even if all those safeguards worked, the customer might prefer a cheaper, more suitable, or more familiar supplier. No gate could remove uncertainty from a market.
On his way out that evening, the owner passed through procurement, where someone was confirming delivery arrangements with a supplier. Finance was reviewing a routine payment supported by the necessary documents. In sales, an employee had asked an assistant to prepare material for a customer conversation the next day. No one had become incapable of making mistakes merely because the company understood execution constraints more clearly, and the assistants had acquired no miraculous powers of judgment. Most ordinary work went on. Some tasks were easier than before, some had to pause until information arrived, and requirements that should not be bypassed at least had a chance to become effective where controlled actions were taken.
Back in his office, the owner looked over the flowcharts and notes accumulated during the past few weeks. He no longer felt compelled to find someone—human or machine—to whom the company could safely entrust everything forever. People became tired. Machines misunderstood. Rules overlooked things. Control software itself might need correction. What a company can realistically strive to do is acknowledge those imperfections, state its indispensable requirements clearly, make them effective wherever their satisfaction can be verified, and remain alert to omissions, exceptions, and consequences beyond its control. Such assurances are limited, but their limits do not make them worthless.
He erased the last question mark from the whiteboard where their earlier discussion had ended. He did not replace it with Safety Solved. The company still faced a question older and more difficult than a missing check. If it no longer expected any employee or machine to be right all the time, how should it arrange work, trust, and responsibility so that people and intelligent systems could each contribute in the roles they were suited to fill? That was not something the company would learn merely by adding one more rule. The organization itself would have to change.
Chapter Thirteen — From Better Agents to Better Institutions
The Company Had Never Waited for Perfect People
Even with the best finance manager in the world, the company would not put supplier registration, payment authorization, account reconciliation, and funds transfer entirely and unconditionally in that person's hands. The issue was not whether the manager meant well. A business could not rest on the hope that any one participant would always be right.
On Monday morning, the procurement manager returned a purchase order to a relatively new employee. Nothing serious had happened. Two materials with similar specifications had been entered in the same field, which would only have forced the warehouse staff to ask for clarification when the delivery arrived. The young man looked embarrassed. He explained that he had used last week's template and missed a change in the supplier's packaging units. His manager asked him to check the actual requirement against the supplier's new units of measure, then handed the order back. The exchange took perhaps ten minutes. He did not insist that the employee seek permission before filling in every field from now on, nor did he take procurement work away from him because of a single mistake.
The owner happened to pass by in time to hear the last few words. He remembered an employee from the company's early days who had known almost every supplier by name and could always find a solution when an order was urgent. That man had nevertheless once misread a delivery date. The company had not declared him untrustworthy. Instead, it changed the way unusual delivery commitments were confirmed, requiring a clear record of the agreed terms. The employee went on handling important purchases and eventually trained several younger colleagues. At the time, the owner had regarded this as ordinary management. Now he saw in that small episode an answer they had been circling for weeks.
Back in his office, he reread the company's recently drafted guidelines for using AI agents. They contained plenty of carefully considered requirements: which information had to be checked, which operations needed approval, when automated processing had to stop, and whom to notify if something unexpected occurred. There was nothing inherently wrong with them. But next to the document lay a proposal still awaiting discussion. One colleague wanted to replace the current model with a more capable one because the assistant sometimes misunderstood business instructions. Someone else proposed bringing every important result back for human review, simply to be safe. Both suggestions were well meant. They also reminded the owner of the way the company had swung between confidence and caution in the first months of adopting AI.
That afternoon, he found a time when procurement, finance, sales, and engineering could all sit down together. He did not begin by asking which model was best. Instead, he posed a question that sounded almost unrelated to artificial intelligence. “Suppose we refused to let anyone work independently until they could guarantee they'd never make a mistake. How long would this company last?” The finance manager smiled. By that standard, she said, she would not be qualified to approve payments herself. After the trouble with the equipment modification project, she knew that experience and diligence were no guarantee against overlooking a change in the business. The procurement manager pointed to the order he had returned that morning. He had not demanded that his new colleague never make another error. He had made sure the error could be found and corrected at an appropriate point.
The sales manager was slower to answer. He still thought about the customer who had turned toward other suppliers. The quotation had been prepared quickly, and the outdated information had escaped notice before the formal document went out. Nothing they said afterward had persuaded the customer to reverse that decision. Since then, he had been especially wary of anything AI generated for external use. He worried that accepting imperfection today would become an excuse tomorrow for treating lost customers as an unavoidable price of automation. The owner did not ask him to set that concern aside. He merely asked whether the company would forbid experienced salespeople from sending quotations if one of them made the same mistake. Of course it would not. It would investigate, establish responsibilities, change what needed changing, and keep doing business. Why, then, should the presence of a machine leave them choosing only between absolute trust and absolute distrust?
This did not mean that machines and employees were interchangeable. People could reflect on criticism and be held accountable through employment relationships, professional duties, and real consequences. An AI agent had neither the same standing in an organization nor the same psychological life, and it could process far more work in a short time than any single employee. One careless human action might affect one order; a badly configured automated process could repeat a misunderstanding across hundreds. The owner had no intention of treating an agent as an ordinary colleague and calling that a governance plan. What he wanted to recover was something the business already knew how to do: acknowledge the limits of everyone involved, assign responsibilities according to the work, and place appropriate safeguards where mistakes mattered.
The engineer agreed that better models, better prompts, and better data were all worthwhile. A more accurate assistant could spare people needless rework; clearer source information could prevent misunderstandings; human review could still help where the business situation resisted simple interpretation. Dismissing those improvements would be as foolish as telling the procurement manager to stop training new hires. But training people and designing sound responsibilities had never been competing alternatives. Even with the best finance manager in the world, the company would not put supplier registration, payment authorization, account reconciliation, and funds transfer entirely and unconditionally in that person's hands. The issue was not whether the manager meant well. A business could not rest on the hope that any one participant would always be right.
The owner moved the proposal for a new model into the middle of the table. They could still evaluate it, he said, but he wrote a question in the margin: Which governance requirements must remain even when the model gets better? An upgrade was no longer, in his mind, the final step toward a day when the company could dispense with caution. A capable employee might earn wider authority with experience, and a machine might reasonably be entrusted with more complicated tasks after careful evaluation. But changes in responsibility needed reasons and limits. A polished demonstration was not a sound basis for collapsing several distinct powers into a single instruction to “let it handle everything.”
Putting Trust Where It Belongs
Joint governance was not about gathering everyone around every button. It was about ensuring that a necessary right or constraint did not disappear when it mattered.
A few days later, the company began discussing a new division of responsibilities. This time, they did not start with a long list of forbidden actions, or with a calculation of how many approval clicks each department needed every day. The owner asked each manager to bring an example of work that occurred regularly: what judgment did it require, what could go wrong, and how did the existing roles share those risks? They were not trying to redesign the whole company. They wanted to make visible the distribution of authority that habit had allowed everyone to take for granted.
The procurement manager brought the record for a routine replenishment order. Inventory had fallen below its usual level; the supplier was reliable; the unit price sat within an established range. In the past, employees had handled purchases of this kind, calling the manager only when quantities exceeded a threshold or prices looked unusual. There was nothing inherently improper about letting an agent organize the requirement, compare current quotations, or even advance an ordinary low-value order within a clearly defined scope. If every roll of packing tape and stack of paper needed the manager's personal signature, he would have no time to deal with emergency deliveries or the search for replacement suppliers. The question was not whether the machine could buy whatever it liked. It was which categories, quantities, values, and business conditions it could handle on its own—and when it had to stop.
The finance manager saw an important distinction. Moving money carried different consequences, and she was unwilling to copy procurement's approach wholesale. Their existing policies already distinguished small routine expenses, properly verified accounts payable, and payments requiring special authorization. An agent could prepare documentation, flag discrepancies, and bring contracts and invoices together. Whether it should be allowed to take further automated steps in tightly defined cases was another question, involving bank-account verification, budgets, project status, and records that could be examined afterward. The ability to complete a payment request was not by itself a reason to surrender the final transfer of funds. Her caution did not invalidate the procurement manager's example. They were governing different kinds of action.
The operations engineer resisted a single permissions table for all automated tasks as well. Removing expired temporary files or producing an operating report was not equivalent to changing the configuration of equipment used in production. An action with little impact on an ordinary day might become sensitive when the factory was racing to meet deliveries. The likely consequences, the current state of operations, and the cost of recovery all mattered; labels such as “simple” and “complex” could not carry that burden alone. Some tasks could run automatically and leave an auditable record. Others required the person on duty, or needed to remain subject to production schedules and the safeguards already built into the equipment. Different departments did not have to govern their work identically, provided their responsibilities and necessary conditions were intelligible.
The sales manager's example slowed the conversation. An assistant could build a quotation from current price lists and flag incomplete delivery or service information. But a customer might ask for something outside the standard terms: an earlier delivery date and an engineer on site on a particular day. The sales manager had once tended to work out an understanding with the customer first and coordinate resources afterward. If the assistant began negotiating autonomously, who could commit the company to those terms? Sales could assess the customer relationship and the commercial opportunity, but it could not create factory capacity with an email. Production knew what could be delivered, but it might not be authorized to decide how much margin to sacrifice for an important order. A single promise could cross several legitimate lines of responsibility at once.
Someone suggested forming a small committee and letting the agent proceed if a majority agreed. The production manager was unconvinced. If production had established that no capacity existed on a particular date, three other people could not vote that capacity into existence because the customer was valuable. The finance manager added that necessary financial or budgetary restrictions could not be displaced by a lively meeting. There was plenty of room for collective discussion, and some decisions were suitable for a vote. But a responsibility entrusted to one role did not automatically become something other roles could discharge on its behalf.
The owner asked them to set voting procedures aside and unpack the proposed promise into its real-world parts. Sales could put forward the customer's request and explain why the opportunity mattered. The production manager could establish whether the schedule and resources made delivery possible. Where unusual costs or discounts were involved, someone with the proper commercial authority could decide what terms were acceptable. Customer service could confirm which kinds of on-site support it could actually provide. Each view might shape the eventual offer. But a particular confirmation counted as a necessary authorization only if the company's legitimate arrangements gave that role the relevant responsibility. An objection was not an automatic veto over everything; equally, one person's agreement could not cancel a necessary constraint belonging to another role under company policy or law.
Would that not create another endless chain of approvals? the procurement manager asked. Quite the opposite, the owner replied—that was precisely the outcome they had to avoid. An ordinary quotation based on settled terms, reliable current information, and valid authority did not suddenly require a committee merely because the company had discussed one unusual customer. The exceptional case needed coordination because someone wanted to promise more than the standard arrangements allowed. Once the relevant parties had established clear, legitimate rules, much of the repetitive checking could still be performed by existing systems and automation. Joint governance was not about gathering everyone around every button. It was about ensuring that a necessary right or constraint did not disappear when it mattered.
The engineer recorded the discussion without attempting to decide commercial terms for sales or spending limits for finance. He reminded them that even a sound arrangement could not be reduced forever to a static table. People changed jobs; supplier relationships and available production capacity shifted; service offerings evolved. Someone had to maintain the underlying facts and determine how proposed rule changes would be approved. If an agent preparing quotations could raise its own discount ceiling whenever a deadline looked tight, or treat a manager's silence as a permanent waiver of review, the new division of authority would soon mean very little.
When the meeting ended, they had produced neither a more intelligent machine nor a system that would relieve everyone of responsibility. What remained was a draft account of how the company worked: how people could raise needs and make governance decisions with legitimate authority; which routine tasks machines could handle within defined limits; how confirmations from different roles combined for particular actions; and who should explain, correct, or revise the arrangement when something went wrong. The owner did not ask everyone to sign it that afternoon and declare the transformation complete. He asked them to take it back to their departments and see whether it fitted the way work actually happened.
Not Looking for Someone Who Can Be Trusted With Everything
Accepting imperfection was not the same as excusing errors, or silently accepting more risk to achieve a higher rate of automation. What institutions could do was help errors come to light before their consequences spread, give appropriate people the knowledge and authority to act at the right moment, and reduce the chance that one participant's unchecked mistake became a loss borne by everyone else.
The notes from the meeting produced a more complicated reaction among employees than the owner had expected. One salesperson worried that, if machines took over more preparation and judgment, the person who finally clicked Send would still be blamed for everything that went wrong. A procurement employee asked whether ordinary purchases would soon be fully automated, leaving him to do nothing but clean up after the machine. People had complained about excessive approvals, but familiar responsibilities had also given them a certain security in their work. Changing who could decide what affected more than a flowchart. It unsettled ideas about professional value and where responsibility belonged.
The sales manager brought those questions back to the owner. The defective quotation had indeed been sent by someone in sales; the company could not erase that fact. But if it expected people to intervene only when something unusual arose, it had to ensure that the person intervening understood why a decision was needed, had access to dependable information, and could actually stop a commitment that should not be made. Otherwise, a name might still appear in the final approval field even as the person behind it had less and less power to understand or influence the outcome. The owner listened carefully. Their earlier argument about who reviewed and who was responsible had not become irrelevant. It could now be translated into more precise duties.
The finance manager raised another point that policy documents often skipped over. They liked to say, “Exceptions must be referred to a manager.” But what counted as an exception? What happened when that manager was away? Could an urgent task wait? Who was authorized to make an alternative arrangement? The answers affected whether employees could meaningfully say no. If the company gave a role the duty to refuse without adequate information or authority, separation of responsibilities might turn into a game of passing blame. Yet if anyone could stop every task simply by saying they were uneasy, the business could not function either. Sound rules needed space for professional judgment and a way to resolve legitimate disagreement.
The owner could see why this was harder than buying software. A vendor could describe speed, accuracy, and features. Redistributing authority meant asking who would lose a decision they had always made, who would assume a new obligation, which shortcuts used to depend on people knowing one another, and which vague understandings now needed to be made explicit. The company had been through changes of this kind before. As it grew, ordinary purchasing decisions had moved from the owner to departmental managers. When finance software was introduced, they had redefined who maintained bank accounts, checked supporting documents, and confirmed payments. Some people had found those changes inconvenient. The company had made them anyway. AI was forcing similar questions at greater speed and across more kinds of work, but the people inside the institution still deserved the same seriousness.
A few days later, the procurement manager and his young colleague completed another order for everyday materials. The assistant assembled the quantities and current quotations. The employee confirmed the real need for replenishment, and the system applied the company's existing controls. Nothing arose that required escalation. The manager glanced at the summary rather than rereading every line. With the time he saved, he spoke to a supplier about delivery reliability and showed the young employee how a low price could still be the wrong choice for the production schedule. The employee would make more mistakes and still had much to learn. Nor had the machine suddenly acquired a complete understanding of business trade-offs. But each was beginning to contribute where its abilities were useful.
Sales made a modest change too. For ordinary inquiries, the assistant continued preparing documents. When a customer wanted a shorter lead time or a special on-site service promise, the salesperson first established what the customer really needed and coordinated with the responsible departments. The assistant was no longer expected to solve the whole problem and then announce that everyone had agreed. This did not recover the earlier lost opportunity, and it could not guarantee the next quotation would win an order. The sales manager still had to read the market, understand the customer, and own the commitments made within his responsibilities. What had changed was that diligence no longer meant dragging every routine matter through the full human approval process.
Later, in an internal discussion, the owner said he had become less comfortable with the word trust when it was used without qualification. Trusting an employee's competence did not mean permitting that employee to rewrite every budget. Trusting an agent to prepare purchase orders did not entitle it to decide every purchase. Trusting a checking program did not end the obligation to govern where rules came from, what had changed in the business, or what the program actually did. Trust could attach to a category of work, a set of conditions, and a definite scope. Autonomy within that scope could support genuinely efficient cooperation. Limits beyond it need not be taken as an insult to anyone's character or ability.
Nor did he tell employees that mistakes no longer mattered. When something went wrong, the company still needed to establish what had happened, meet its responsibilities, make good whatever losses it reasonably could, and decide whether staffing, training, rules, or controls needed to change. Accepting imperfection was not the same as excusing errors, or silently accepting more risk to achieve a higher rate of automation. What institutions could do was help errors come to light before their consequences spread, give appropriate people the knowledge and authority to act at the right moment, and reduce the chance that one participant's unchecked mistake became a loss borne by everyone else.
By the end of the month, the company held no ceremony to celebrate a completed reform. The division of work still needed refinement. Some duties remained open to discussion, employees had disagreements, and engineering had a long list of unfinished tasks. The owner found that more believable than a triumphant announcement. The business had never survived by waiting until it could hire perfectly reliable people. Neither should the arrival of extraordinarily fast machines persuade it to put all its hopes in an infallible agent. It was worth improving the abilities of both people and machines. But their capacity to work together over time would depend on an institution willing to confront mistakes, respect legitimate authority, and revise its own arrangements.
In the margin of the draft responsibilities, he wrote one last practical question. If different activities warranted different levels of control, how should those requirements connect to the payment, quotation, and equipment systems the company already used? Routine work should not be burdened with needless procedures; high-risk actions needed controls strong enough for their consequences. The division of responsibilities on paper had to withstand the realities of daily work. The question was no longer whether an Execution Control Layer deserved to exist. It was how the institution's decisions could be translated into proportionate, testable controls that would continue to improve as the business changed.
Chapter Fourteen — When Governance Requires Technology to Say No
Not Every Door Needs the Same Lock
Existing permissions, definite business conditions, and occasional retrospective checks might be entirely sensible for low-risk activities whose effects were easy to reverse. Where the sums were larger, promises difficult to withdraw, or production at risk of interruption, the company needed stronger assurance that it was acting on the right object, that the relevant authority remained valid, and that an unacceptable action could actually be stopped. Improving control was not a call to impose the heaviest human approval process on every task.
After the end-of-month discussion about responsibilities, the engineering lead received his first assignment. It was not to build a new system. The owner wanted him to walk through several existing business processes and establish how they actually worked. In the meeting notes, the owner had written: reduce repetitive reviews for ordinary replenishment; verify authorization and recipients for large payments; prevent formal quotations from making unsupported delivery promises; require explicit conditions for production changes. None of that sounded especially complicated. Several managers felt they were already doing most of it. Then the engineers asked where, in each case, the final action took place. The answers were scattered across different software systems.
The procurement manager showed them the ordering screen for commonly used materials. The system could already compare quantities, supplier prices, and delivery addresses against established conditions. When stock ran low, the assistant prepared replenishment recommendations, and the manager intervened only when an order fell outside the ordinary range. The finance manager opened the payment application and demonstrated its long-standing role permissions and tiered approval records. Sales had its customer records, quotation templates, and email tools. Production operations used a separate set of equipment-management programs. The company was hardly operating without controls. Its problem was that the controls lived in different places, and the connections between them were not always clear.
Someone proposed buying a product that could bring approvals, authorizations, and records together in one place. The engineering lead did not dismiss the idea. He asked what it would replace. Payments were constrained in part by the company's software and in part by the bank's own controls. Industrial equipment had safety requirements of its own. A quotation that reached a customer involved email, files, and the terms the company was willing to recognize as binding. A central dashboard might make oversight easier, but it would not magically make every consequential action pass through that dashboard. Rebuilding everything from scratch would be expensive and could discard safeguards refined over years of use.
The owner asked for a plain, practical inventory rather than an impressive diagram. For each consequential action, where did it actually happen? Which requirements were already checked by a system? Which depended on someone remembering to look? And through what other routes could the same kind of action take place? He did not regard the old procedures as worthless, nor did he want the new arrangements to add steps merely for show. Ordering a small quantity of ordinary consumables was not equivalent to authorizing a substantial advance payment. For the first, limits on price, quantity, and supplier might cover most normal cases. For the second, a payment could be within its monetary limit and still be entirely inappropriate.
The inventory moved from department to department, and the disagreements became more concrete. Could sales send a clearly labeled preliminary proposal when a customer needed something urgently? What should happen to a payment waiting in a queue if the person who approved it lost that authority while traveling? Would a factory have to wait for the owner to wake up and sign off on every nighttime recovery action after an alarm? The cost of delay had begun to matter in the conversation alongside the consequences of error, the possibility of reversing an action, and the damage it might cause.
The engineering lead argued that the strength of control should reflect those differences. Existing permissions, definite business conditions, and occasional retrospective checks might be entirely sensible for low-risk activities whose effects were easy to reverse. Where the sums were larger, promises difficult to withdraw, or production at risk of interruption, the company needed stronger assurance that it was acting on the right object, that the relevant authority remained valid, and that an unacceptable action could actually be stopped. Improving control was not a call to impose the heaviest human approval process on every task. If three managers had to sign for a box of office supplies, the company would simply recreate the approval backlog it had worked so hard to escape.
From “Checked” to “Unable to Proceed”
People no longer asked only whether a program supported approvals. They asked which actual operation it could prevent, under what circumstances, and whether some other route lay outside the checks.
They chose formal sales quotations for the first small-scale change. Everyone remembered the lesson behind it. A prospective customer had given priority to another supplier after receiving a quotation whose figures looked plausible but whose lead time and service scope no longer reflected the business. New checks on the validity of quotation data had since caught some obsolete information. Yet the sales manager had witnessed a second problem: a document could pass the checks, change just before dispatch, and still carry a record saying it had been approved. That earlier record could not speak for the file that ultimately reached the customer.
The revised arrangement did not remove the salesperson's commercial judgment. The assistant could still organize customer requirements and prepare standard terms. Production and service staff would continue maintaining the information they were entitled to confirm. The missing piece lay between that information and the act of formally sending it. When sales settled on the final quotation, the system needed to establish precisely which document would go out. If the delivery promise, intended recipient, or special service terms had changed, approval given to the earlier document could not simply be borrowed for the new one.
Their first test used an internal address standing in for a customer. A salesperson prepared a quotation within the rules, obtained the required confirmations, and had the sending process generate the formal attachment. It worked. The page showed “Sent,” and the test mailbox received the document. The team might have called that a successful demonstration. Instead, the sales manager asked a colleague to shorten the approved lead time by a week and try again. The assistant promptly updated the attachment, while the business page still displayed the earlier approval. At the new sending step, however, the final document was checked against what had actually been confirmed. The changed version was not released under the old approval. It was sent back for renewed confirmation.
The engineer pointed out the refusal message on the screen, but the sales manager was not ready to applaud. He opened the test mailbox and checked the attachment that had actually been delivered. He wanted to know that the message had not gone out despite an error shown on the business page. Then someone asked the awkward question. What if the salesperson downloaded the file and emailed it through the ordinary mailbox instead of using the quotation system? There was a silence. No one found the question intellectually difficult. A control could only govern an action over which it had real control. The new sending step could constrain the formal quotation route. It could not, through the power of a single button, abolish every other way an employee might communicate.
They did not declare the project a failure. Instead, they examined the company's email channels, how formal commitments were recorded, and which official documents were required to leave through a managed process. Where an action could be brought under the system's control, they adjusted the entry points and permissions. Where a route could not yet be blocked technically, they continued to manage it through organizational duties, records, and review—and made sure no one claimed it enjoyed the same guarantee. Informal customer communication retained some flexibility. Business conversation could not all be reduced to fixed forms. The sales manager found these limited, explicit claims more credible than a promise that the software would “prevent every incorrect quotation.”
Finance came next, with two test payments. The first had an acceptable amount and recipient, supported by approvals that were still valid. In the second, the beneficiary account was changed after approval to another account that had not been verified. Some aspects of the old process already depended on human review, and the payment system had its own restrictions. The purpose of the new work was to connect those existing controls more clearly to the account that would actually receive the funds. In the test environment, the first transaction proceeded normally and the second stopped before submission. The engineer did not call that proof that payments were now completely safe. He asked the finance manager to confirm which payment routes the test covered, who maintained the account information, and how last-minute changes would be handled.
The finance manager proposed a harder case. Suppose the amount, recipient, and supporting documents did not change at all, but the person who had approved the payment left the relevant role while it was waiting in the queue. If company policy required approval authority still to be valid when payment was submitted, the engineering lead replied, an agreement given several days earlier could not be treated as an unconditional permission forever. That did not make the earlier approval false. It meant the approval had conditions tied to time and status. The testers revoked the simulated account's authorization and repeated the process. The result was an explicit stop and a requirement to resolve the payment again, not an invitation for the assistant to infer the company's wishes from the fact that someone had clicked Approve the week before.
A finance employee raised a reasonable concern. Would every payment be delayed whenever the permissions service briefly failed to respond? The engineers acknowledged the risk. Missing information could not be passed off as proof that a necessary condition had been met. But a blank error box was not an acceptable way to leave the business stranded either. They defined clearer messages, contacts for resolving a problem, and recovery procedures. If a payment was genuinely urgent, it needed a legitimate exception process—not an assistant quietly loosening a rule or a colleague lending someone a high-privilege account.
In operations, the questions took a different shape. Factory equipment needed routine maintenance; some checks left production untouched, while other operations could stop a line. The operations manager produced the change form they had used for years. Many hazardous steps already required checking the machine's state, and operators were not free to bypass equipment safeguards at will. The new assignment was not to pretend those protections did not exist. It was to determine whether, when an assistant proposed maintenance, the tools carrying out the work could distinguish an ordinary check from a production change, whether the approved time and scope still applied, and which interfaces actually triggered the equipment to move or change state.
They scheduled a test window that would not interfere with production. The assistant proposed an adjustment based on the usual operating information, and the maintenance staff submitted it to test equipment. One authorized action completed normally. In a second test, they deliberately changed the target equipment identifier; the instruction was refused. In a third, they moved the same operation into a prohibited production period, and the system did not carry it out. The people on site did not stop at a log entry reading “Denied.” They checked the equipment itself to confirm that its state had remained unchanged. A refusal recorded in software was useful evidence. Whether the machine had actually acted mattered more.
Then the operations manager identified a direct-control route in an older maintenance tool. It had its own protections and access requirements, but it had not been included in the new tests of agent-initiated changes. Someone suggested that the team could ignore it because the AI did not use it. The engineering lead refused to treat that assumption as permanent. Perhaps only maintenance personnel could use the route today. A software upgrade or a change in account permissions might make it accessible to automation tomorrow. They marked it as a separately managed path, reviewed who could access it, and established which operations should be brought under the shared change conditions and which remained appropriately protected by the equipment's existing safeguards. They also recorded what would need to be tested later.
After several rounds of experiments, the owner noticed that the language of the meetings had changed. People no longer asked only whether a program supported approvals. They asked which actual operation it could prevent, under what circumstances, and whether some other route lay outside the checks. Those questions helped engineers locate the real points of control. They also gave managers and employees a clearer account of the limits of what had been built.
Going Live Does Not Put an End to Risk
It was more valuable for management to understand what their assurances covered, which assumptions held them up, and what changes would make those assurances worth testing again.
One day, the procurement manager arrived with what looked like an entirely ordinary replenishment order. Why, he asked, had the system suddenly refused to let the assistant continue? The supplier had recently changed its packaging units. The quantity entered by the employee had been perfectly normal under the old units, but, converted into the new packaging, it exceeded the established range for routine orders. The refusal did not reveal an elaborate attack or wrongdoing by the supplier. It showed that a fact of business life had changed while the calculation built into the workflow still reflected the old one. The procurement manager first asked an employee to verify the actual need, then had the appropriate people update the records. Once the engineering team corrected the logic, it added similar cases to the regression tests so the same error would be less likely to recur.
The episode also reminded the owner that a program's refusal could not decide whether a purchase was commercially wise. If the packaging change had been legitimately confirmed and the old rule still blocked the order, the institution needed a proper way to revise its requirements. If, on the other hand, an employee raised the limits at will because delivery was urgent, a smooth-running system might no longer protect what it was meant to protect. Both mistaken approvals and unnecessary refusals mattered. Urgency could not justify erasing every boundary, but frequent denial was not a measure of success in itself.
The engineering lead began keeping a modest record of their experiments. For each one, it identified the business scenario, the system version, the conditions under test, the actual route by which the action would take place, and where the result was observed. When testing a refusal, the team tried to establish not merely what the software returned but whether the external action had been prevented along the controlled route. When testing an allowed action, it checked that what was finally executed matched the object that had been approved or verified. Managers did not need to understand every technical detail, but they did need to know what “tested” meant—and what remained only an intention.
The owner once asked for a single percentage. Could the engineering team tell him, preferably with one reassuring figure, how much of the company's risk was now covered? The lead thought about it and declined to invent a number. They could count known formal execution routes, tested paths, and unresolved issues. They could not turn unknown failures, commercial uncertainty, and untested ways of acting into one honest coverage rate. A bypass they had never discovered did not vanish because a report gave the project a high score. Nor would every future loss arising from a permitted action prove that the refusal mechanism had failed. It was more valuable for management to understand what their assurances covered, which assumptions held them up, and what changes would make those assurances worth testing again.
As the work continued, departments that had seldom coordinated closely began sharing a few habits. When sales changed its quotation template, it notified the people responsible for verifying the final document. When finance revised approval responsibilities, it checked how payments already in progress would be re-evaluated. Before operations upgraded management software, the team listed interfaces that might be added or changed. They fitted the necessary checks into existing change-management practices according to the likely impact. Controls changed alongside the business rather than becoming a plaque installed on launch day and never touched again.
One software upgrade soon justified the effort. A new version of the sales tool added a convenient way to share documents from a phone. Sales staff liked it. The engineers realized that if the feature could send formal commitments, the previous desktop-only checks on attachments might not apply. The company did not respond by banning mobile work. It examined what the feature could actually do, clarified whether formal quotations still had to use the managed channel, and told employees the limits of the present arrangements until the necessary changes were made. The new feature was not an enemy. The danger was assuming that a conclusion tested on an old route remained true on a newly opened one.
Employees gradually grew accustomed to a different rhythm. A refusal did not always mean that someone had made a serious mistake. Sometimes a record had expired, someone had changed roles, or a necessary confirmation had not yet arrived. Procurement staff did not have to treat every pause as evidence that engineering was obstructing business. Engineers, for their part, learned that “those are the rules” was rarely a useful explanation to a manager trying to fulfil a legitimate order. The teams agreed that refusal messages should speak in business terms where possible, identifying what was missing and who could resolve it. Where interruptions kept recurring, the relevant people would review whether the policy truly required them or the implementation had become too rigid. A constraint that could be explained, challenged, and legitimately amended had a better chance of being respected in ordinary work.
The owner asked the engineering lead whether the industry might eventually develop a single solution everyone could use. Some of the needs were indeed shared, the lead said: ways to describe a specific action across systems, convey valid approvals, record what was actually executed or refused, and test whether controls survived a change. Those common needs might encourage common interfaces and evaluation methods. But a bank payment, an industrial machine, and a customer communication involved different risks. So did large and small businesses, with their different software, practices, and rules. Even two firms in the same industry might govern themselves differently. Any widely useful approach would have to emerge through real use, practical costs, and repeated validation. They could not declare a universal standard by giving it a name today.
The answer relieved the owner. He was no longer looking for a tool that would replace all the company's institutions. A few months earlier, their favorite questions about AI had concerned how fast a model could finish a report and how many more tasks an assistant could take on. He still wanted the machines to do more. But now he asked the technical team a different kind of question. Where, at the actual point of action, would the company refuse something it had explicitly forbidden? And for ordinary work already allowed, had they introduced any unnecessary barrier that made people wait? The two concerns were not opposed. Unless both were addressed, apparent gains in efficiency might simply have been purchased by ignoring their consequences.
Near the end of the day, the sales manager received another ordinary customer inquiry. The assistant assembled current prices, available delivery schedules, and standard service information. The system applied the relevant checks, leaving the salesperson free to address the customer's actual commercial questions. A second inquiry sought a delivery date for which no production capacity had been confirmed. The assistant prepared that draft quickly too, but it did not become a document ready for formal dispatch. The sales manager saw the explanation and contacted production to discuss alternatives. No one asked him to reread every normal quotation word for word, and no one allowed the agent to make an unsupported promise simply because the customer was in a hurry.
As the engineering lead headed downstairs, he saw sales still talking with production about that special delivery date. He remembered someone complaining that all the new governance did was teach software to say no. Yet this had not been a day of paralysis. Ordinary work had continued. The matters requiring human judgment had become easier to recognize for what they were. Control could not win the customer for sales or conjure new capacity for production. It could, however, stop an unapproved commitment before it became real and leave the responsible people room to work something out.
In his work notes, the owner wrote that the company had finally begun to bring governance and technology into a relationship they could examine. Not because they had drawn a complete architecture and declared the business safe, but because, within limited and explicit scopes, they had observed how permission worked and tested how refusal worked. Tomorrow would bring new business, new exceptions, and new gaps. Deployment would not bring this work to an end. Still, now that a boundary could be more than a written instruction, the business did not have to station a person behind every button. People would keep making mistakes, and machines would too. The next question was not only how to stop errors, but how imperfect people and imperfect intelligence might learn to work together for the long term, with greater freedom and less needless anxiety.
Chapter Fifteen — Living with Imperfect Intelligence
Another Ordinary Monday
They had gradually changed what people thought responsible work looked like. Instead of trying to keep their eyes on every document, employees were becoming more willing to use their attention for the judgments that required it, provided the limits already agreed upon were genuinely respected.
It was nine o'clock on a Monday morning again, and the office coffee machine was still taking its time to heat the water. An employee in procurement had barely sat down when a supplier comparison report appeared on the work screen. Someone picked up a mug and made the familiar joke. “These new colleagues really don't know when to clock off.” The others laughed. The words were the same, but they sounded different now. Months earlier, the joke had carried both delight at how quickly the machines worked and a quiet fear that every report they produced would become another document someone had to review. Today, a finished report no longer meant another assignment that had to be checked from beginning to end.
The report laid out the week's prices and delivery terms from several regular suppliers and drew attention to a change in one supplier's minimum order quantity. Routine purchases that met the established conditions could proceed through the existing process. The portion involving additional stock and warehouse space was left for the procurement manager to decide. He did not begin the day with dozens of buttons awaiting his approval. He looked at changes in inventory, walked over to the warehouse to ask a few questions, then spoke with the colleague responsible for deliveries. Together they decided to postpone part of the order. The machine had not made every business judgment for him. It had put the decisions worth his time in front of him early enough to matter.
Finance was busy too. The reconciliation assistant had sorted several discrepancies overnight. Items caused by differences in recording dates already had corresponding verification results. Another payment had been held because the project status had recently changed. The finance employee needed to find out whether that change had formally taken effect and who was responsible for the revised payment arrangement. She did not assume a paused payment meant a crisis, any more than she assumed a history of successful checks made the pause unnecessary. Recording a transaction, establishing what had happened, and deciding that money should move were, after all, different kinds of work.
In customer service, the screens were filled with ordinary inquiries. Assistants had drafted replies using the current service terms, and cases clearly within their authorized scope were being handled through the established process. Only one message from a long-standing customer caught the team leader's attention. The customer was not complaining about product quality. He was explaining an unusual situation: the scheduled installation coincided with a shutdown for maintenance at his factory, and following the usual procedure might mean missing the next production window. The assistant had proposed several responses and assembled records of similar past cases. The team leader read them and then called the customer instead of selecting the first suggestion. What the customer needed might not be a discount at all. It might be technical support arranged to fit the narrow window when the factory would be idle.
At lunch, someone in procurement mentioned the chaotic weeks earlier in the year. They had dreaded notification sounds, certain that another pile of material would appear for approval. Sometimes people had cleared whole batches of confirmations from their phones, until the trouble began: a supplier price that was no longer valid, an advance payment processed after a project had been suspended, and a customer quotation that failed to reflect changed delivery conditions. None of those mistakes had taught the company a magical technique. They had gradually changed what people thought responsible work looked like. Instead of trying to keep their eyes on every document, employees were becoming more willing to use their attention for the judgments that required it, provided the limits already agreed upon were genuinely respected.
The customer who had received the defective quotation and then given priority to other suppliers had not returned simply because the company improved its procedures. The sales manager still thought about the loss, especially when confirming lead times and service promises. But he no longer treated every outgoing email as a gate that had to be guarded solely by his own vigilance. Ordinary quotations used reliable current information and were checked at the point of formal dispatch. Unusual promises went to the people who could answer for production capacity, service commitments, and commercial terms. He could still make a poor business decision, and the company could still lose orders. The difference was that it was trying not to leave preventable mistakes to the luck of whether someone happened to notice them in time.
We Do Not Need to Find a Perfect Colleague
Mature trust did not mean assuming that a participant could never go wrong. It meant understanding what that participant could do, what it was allowed to do, where it should turn when its abilities or authority ran out, and how errors could be recognized and put right.
Not everyone found these changes reassuring. During a departmental discussion, a young employee asked the engineering lead, “If machines keep getting better at making decisions, are we going to be left with nothing but the difficult cases?” He was asking in earnest, not trying to find fault with the new systems. At first, when AI took over the work of sorting spreadsheets, people had welcomed the relief. Later it had begun comparing contracts, handling routine decisions, and proposing ways to improve operations. Familiar forms of work were changing. Some employees gained more opportunities to exercise judgment; others feared that the experience they had worked hard to acquire would steadily lose its value.
The engineering lead did not comfort him with the claim that humans would always be better than machines. For some repetitive, demanding tasks, machines were already faster. In certain forms of specialist analysis and specific decisions, they might eventually outperform some people consistently. A company had no reason to ignore useful abilities merely to prove human superiority. The owner added that changing work did not automatically give everyone a better job. The company still had to think seriously about retraining, changing responsibilities, and the real effects of automation on employees. Execution controls could help decide the scope of what machines were allowed to do. They could not resolve questions of how work, income, and opportunities for growth were distributed among people.
The procurement manager remembered his own early days. He had not been permitted to work because he had proved he would never make a mistake. Experienced colleagues had taught him how to examine quotations, ask about lead times, and verify suppliers. A manager had let him decide within an appropriate range, and when something went wrong, they worked out why. No one had required him to demonstrate infallibility before he began. And because he could handle small orders, they had not handed him control over bank accounts, every contract, and the production plan. People had always worked within arrangements of limited trust, divided responsibilities, and continuing learning. Intelligent machines had different capacities and a different relationship to accountability. But the reason organizations needed institutions had not become obsolete overnight.
Mature trust did not mean assuming that a participant could never go wrong. It meant understanding what that participant could do, what it was allowed to do, where it should turn when its abilities or authority ran out, and how errors could be recognized and put right. Employees were subject to supervision; managers' decisions could be reviewed; business policies could be questioned. A capable agent did not have to be confined forever to a chat window that answered questions. Nor did a record of many successful tasks entitle it to powers it had never been granted. Greater capability could justify considering broader authority. It could not, without a legitimate decision, turn itself into that authority.
Several weeks later, the company experimented with giving the customer service assistant more responsibility for scheduling after-sales visits. It had previously organized customer requests. Now it could compare engineers' rosters and service areas, suggest workable appointments, and, within the standard service scope, complete bookings. The trial went well. Several tasks that had once required repeated messages between employees and customers moved much faster. Someone proposed allowing the assistant to adjust the limits of the service area itself. After all, its schedules now appeared to contain fewer errors than the old manual process. The customer service team leader shook his head. He was pleased with its performance, but changing the service area meant changing commitments to customers, engineers' working arrangements, and the company's costs. A strong scheduling algorithm was not evidence that everyone with responsibility for those matters had agreed to new terms.
No one dismissed the proposal out of hand. The assistant had identified genuine shortcomings in the existing rules. Two customers who were geographically close had been placed in different service areas, sending engineers on needless detours. Maintenance windows for some long-standing clients had not been updated to reflect new contracts. The team leader asked the assistant to organize the evidence and suggest alternatives for service, sales, and finance to consider. The machine took part in recognizing the problem and proposing changes. It did not quietly write its own proposal into the rules as though the discussion had already been settled.
Agreement did not come quickly in every case. Sales wanted to look after long-standing customers. Service had to account for the time engineers spent traveling. Finance needed to know whether extra commitments would exceed approved resources. Some requirements could be reconciled by improving the schedule; others involved real trade-offs. The outcome depended on what each party was legitimately responsible and authorized to decide under the company's governance arrangements. It was not settled by whoever spoke the loudest, nor by mechanically giving every attendee one vote. If a necessary condition involved a particular party's legitimate authority, agreement from everyone else could not substitute for that party's consent.
Only after the revised plan had been properly confirmed did the technical team put the new coverage and effective dates into the relevant workflows. They separately specified how to handle appointments already booked but not yet completed, so that new rules would not quietly erase existing promises. The young employee helped check the changes and discovered that the assistant had confused one customer's service address with its registered address. The team added a condition requiring human confirmation in such cases. They did not abandon automated scheduling because of the error, but neither did they ignore it to meet a launch date. The machine had helped the organization detect inconsistencies; human experience had caught something the machine had overlooked.
The experience gave the young employee a different understanding of his work. He had begun to think of himself as someone who merely cleaned up after AI. Now he could see that examining exceptions, verifying real conditions, and explaining why a rule should change were forms of skilled work in their own right. That did not mean everyone would feel reassured. New skills took time to learn, and the company still had responsibilities toward employees making the transition. Cooperation between people and AI should not exist only as an elegant division of labor on a presentation slide. It also had to show up in who could learn, who could challenge a decision, and whether those who would live with the consequences had a meaningful place in making it.
Rules Have to Learn to Change, Too
Mature institutions were not built around finding an all-powerful good person. They established who was entitled to decide, whose participation was required, and who could insist that the organization honor its commitments.
After the revised arrangements had been in use for a while, the owner noticed a new category of information in the reports he received. They explained why a rule had been suspended temporarily, why an action that had been blocked was later found to be permissible, and who had authorized a revision on what grounds. At first, he found this unsettling. Surely the point of establishing boundaries was to change them as rarely as possible? The engineering lead reminded him that business did not stop changing because its rules had been written into software. Suppliers altered delivery methods. Customers revised contracts. People left or moved to new roles. New services were offered. A condition that made sense last year might no longer be right today.
But the ability to amend a rule was not permission for anyone to discard a restriction whenever it became inconvenient. Employees could request an explanation or a review of a requirement that blocked legitimate work. Someone with the appropriate responsibility could establish the facts and decide whether to change it. A temporary exception, when justified, should follow defined conditions and leave an appropriate record. Conversely, a manager did not become exempt from every necessary constraint by saying, “I'm in charge.” Mature institutions were not built around finding an all-powerful good person. They established who was entitled to decide, whose participation was required, and who could insist that the organization honor its commitments.
Assistants began taking part in those improvements too. They could examine records they were permitted to use and identify a rule that repeatedly rejected ordinary applications. They could compare waiting times under old and new processes and flag a permission that no longer fitted someone's job. Sometimes an assistant spotted a problem sooner than an employee would have. Sometimes it made a poor recommendation because it lacked information from the factory floor or the customer. When people disagreed with it, the useful question was not whether a machine had the right to express an opinion. It was what evidence supported the suggestion, what still needed verifying, and which decision belonged to someone legitimately authorized to govern the matter. Listening to a machine was not the same as handing it the power to make the rules.
For a while, the owner had worried that if policies changed, models were updated, and employees remained fallible, their new way of working might never be finished. He came to accept an answer that was less exciting than a successful product launch but more useful for running a company: it was not supposed to be finished in an afternoon. Businesses revised responsibilities, budgets, and processes every year. No one imagined that a management policy could be written once and never touched again. Bringing automated action under governance also meant maintenance over time, repeated verification, and an honest account of what remained outside the controls.
Sometimes strengthening the arrangement even meant reducing automation. When an activity repeatedly encountered unfamiliar exceptions and the necessary information could not be verified promptly, the company temporarily narrowed what the assistant could do and returned critical steps to employees. Once the sources, rules, and checks were better established, it could widen the scope again. Sales staff initially worried about the effect on speed. Over time, they found it easier to explain a controlled return to human handling than to push ahead with a task whose conditions were known to be unclear. The scope of automation should move with the strength of the assurance the business could actually provide. It was not a number that could only be increased.
Nor was governance a single yardstick for judging the worth of every choice. On one occasion, two suppliers both satisfied the company's established procurement conditions. One offered a slightly lower price; the other allowed more flexibility over deliveries. The assistant judged the first to be cheaper overall. The procurement manager worried that production requirements might change in the next quarter, and after discussion they chose the second. Demand did not change after all. In hindsight, the first supplier might have saved money. That did not mean the rules had failed, or that the manager's judgment must therefore have been wiser than the machine's. Institutions could require a decision to remain within defined bounds without being able to establish in advance which of two legitimate commercial choices would prove best.
The owner grew more comfortable discussing that uncertainty openly. Employees' expertise deserved respect, and could be questioned. A model's recommendation might be valuable, but people needed to know what it relied on. Execution control could stop certain impermissible actions when the conditions had been defined and the paths were genuinely covered. It could not make every permitted decision correct. Legal disputes, ethical choices, and conflicts between people's interests could not be resolved fairly by a technical check alone. They needed organizational governance and, in some cases, industry rules, public institutions, and wider social discussion. Technology could help people shoulder more work. It could not answer every question about how people ought to live together.
When the company had first adopted AI, the owner's favorite figures were processing speeds and hours of staff time saved. He still paid attention to them. Now he also wanted to know less flattering things. Could an employee challenge a restriction that made no sense? Could a mistaken refusal be explained? Would changing a rule alter promises the company had already made? After a new feature was enabled, how much of the previous control remained effective? These questions did not lend themselves to a single attractive score or a dramatic demonstration. Gradually, though, they became part of the company's ordinary management.
Not long before, customer service had brought a disputed special appointment to the monthly business meeting. A customer believed the contract entitled them to priority service, but the company's records did not provide a sufficiently clear basis for that claim. The assistant offered two possible readings and flagged a supplemental agreement that had not yet been verified. The team leader did not ask it to choose the “most reasonable” interpretation and act on it. Nor did he make the customer wait until everyone next had time for a meeting. He asked the sales manager to locate the signed agreement promptly, had the service department provide the ordinary support that could already be confirmed, and reserved the additional promise for those authorized to decide it. The customer might not have welcomed every step. At least the company could explain what had been decided, what remained unresolved, and why uncertainty was not being treated as consent.
The Moment That Never Disappeared
One evening, the procurement manager was leaving the office when he passed the sales area and saw the sales manager still examining a contract. The customer wanted to change a delivery date at short notice. The assistant had set out the proposed amendments and differences from the original terms, clearly highlighting the parts that would become formal delivery commitments. Production had not yet confirmed the new schedule, so the document could not go out as planned. The sales manager was on the telephone discussing another date the customer might accept. He was not repeatedly clicking a button in frustration, nor had he handed the whole difficulty to the software. The part that genuinely needed to be negotiated was still his to handle.
The procurement manager stopped for a moment. He remembered the days when, under pressure to move faster, they had wanted someone to take responsibility for everything at the last possible instant. A single button had inspired both dependence and unease: click it and work could proceed; refuse, and someone would complain about the delay. Only later had they understood how many separate questions were crowded into that hesitation. Was the information trustworthy? Who had the authority to agree? Which necessary conditions were satisfied? Was the action about to take place actually the one that had been checked? No person could be expected to hold all those questions perfectly in mind, every time, while exhausted.
Now many routine operations no longer required a person to reach for that final button. Machines could prepare work, make judgments within authorized limits, and bring ordinary tasks to completion on time. Some actions stopped when the conditions were not met, with an explanation returned to the people responsible for resolving the problem. Other commercial choices still belonged to people who understood the customer, the product, and the consequences. The company would face new mistakes, new arguments, perhaps even new losses. There would be no day on which it could announce that people and machines would never get anything wrong again.
The sales manager ended his call and picked up the cup of water beside his desk. He told the procurement manager that the customer had accepted another time to talk, but the final arrangement would still need to be worked out with production tomorrow. He did not look particularly triumphant. He looked like someone who had finished one ordinary, awkward piece of work. They turned out the lights and walked toward the lift. Other computers were still on. Some assistants were preparing material for the following day. No one needed to stay behind just to watch them. Neither had anyone surrendered every decision about the company's tomorrow to the machines.
Just before the lift doors closed, the procurement manager looked back. He knew the coffee machine would still be slow the next morning, and that the reports might already be waiting when he arrived. Those colleagues who never went home would become more capable, and they would remain capable of mistakes. What mattered was not following them forever, nor waiting for the day the company could finally let go of everything. It was finding a way for human experience, machine capability, and institutions open to challenge and revision to work together—so that consequential actions, whenever they changed the real world, were subject as far as possible to legitimate constraints and meaningful verification.
Let intelligence think freely, and keep execution within bounds. That does not mean locking the future behind a door that can never move. It means ensuring that, as people learn to live alongside imperfect intelligence, they retain the ability to decide carefully which doors should open, which must stay closed, and who has the authority to reconsider where those doors belong when the world changes.
Appendix A: Key Concepts and Terminology
This appendix revisits the questions that run through the book: What can a machine do? Who has the authority to approve its actions? Which conditions must be satisfied before an action changes the real world? And how can an organization's rules genuinely constrain what happens?
It is not a technical implementation manual, nor is it intended to replace the story and discussions in the preceding chapters.
Entries marked with a C-number use the official English names and minimal definitions established in Execution Control Engineering Terminology, version 0.1. The accompanying In this book passages explain what those concepts mean within the company's story. Entries without a C-number are expressions used in this book; they are not presented as separately numbered formal concepts under the terminology specification.
The story is meant to be read from beginning to end. This appendix is here for readers who want to return to a particular idea afterward—not as a list of terms that must be memorized before the first chapter.
From an Agent's Intent to Real-World Action
AI Agent (Agent)
In this book, an agent is an AI system that can work with information, use tools, and carry out a sequence of tasks within a business workflow. It might compare supplier quotations, prepare a customer response, or advance a routine operation that falls within an authorized scope. Being able to complete a task, however, does not entitle it to decide that every real-world action should go ahead.
Further reading: Preface; Chapters 1–3; throughout the book.
Fallible Subject (C003)
Normative definition: A Fallible Subject is a subject capable of interpreting, deciding, authorizing, invoking, controlling, or executing an Action whose future judgments cannot be assumed to be invariably correct.
In this book: An inaccurate quotation need not be the result of anyone acting maliciously. Employees can overlook information, and agents can rely on outdated records. A sound organization therefore cannot build its safeguards around the hope that any one participant will always get things right.
Further reading: Chapters 3 and 15.
Intent (C002)
Normative definition: Intent is an expression of the Action, object, parameters, conditions, or outcome that a subject seeks to bring about. Even authentic and unambiguous Intent does not establish that the eventual Execution or real-world result is correct.
In this book: An assistant may propose telling a customer that delivery will take four weeks. The proposal is not, by itself, a commitment the company has approved—and it certainly does not mean the message should now be sent as a formal quotation.
Further reading: Chapters 8–11.
Execution (C001)
Normative definition: Execution is the overarching process by which an Action moves from expression, judgment, and commitment into a change in real-world state.
In this book: Sending a formal quotation, transferring funds, and changing the state of industrial equipment are actions with consequences beyond the screen. Drafting a document or completing an approval check is not the same as carrying out the real-world action. The terminology also distinguishes the broader Execution Process from the final Execution Act that actually changes real-world state.
Further reading: Chapters 10–11.
Execution Gap (C004)
Normative definition: An Execution Gap is the systemic space in which Intent, interpretation, judgment, authorization, or representation may diverge from eventual real-world Execution and its result.
In this book: A supervisor approves a quotation promising delivery in six weeks, yet the document ultimately sent to the customer promises four. A check can be correct for the material it examined while the real-world outcome diverges from what was checked.
Further reading: Chapters 7–9.
Capability, Approval, and Eligibility to Act
Approval Authority
Chapter 10 uses this expression to ask who is entitled to give a required approval for a particular matter. That entitlement has a defined role, object, and scope. Approving one version of a quotation does not automatically approve a different version created afterward.
Further reading: Chapter 10.
Authorization (C005)
Normative definition: Authorization is the eligibility boundary that determines whether a subject is allowed to possess or use a class of capability. Successful Authorization does not establish that a specific Action should execute under current State, parameters, and boundaries.
In this book: A salesperson may be authorized to communicate with customers on the company's behalf without being entitled to promise any price or delivery date they choose. Nor does an earlier approval automatically extend to a quotation whose material terms have changed.
Further reading: Chapters 8–10.
Execution Capability (C007)
Normative definition: Execution Capability is the ability of a subject, component, or path to cause a class of real-world Actions to occur.
In this book: An employee who can send an email, or a finance employee who can operate a payment system, possesses a practical means of making something happen. That capability alone does not establish that this particular quotation or payment is eligible to proceed.
Further reading: Chapters 10–11.
De Facto Ability to Execute
Chapter 10 uses this expression to emphasize a straightforward fact: someone may have the technical means to cause an action to happen. An employee can attach a file and send it from a work email account, for example, without necessarily being entitled to make the formal commitment contained in that file. The expression is closely related to Execution Capability, but stresses the existence of a workable means of acting—not permission to use it in a particular case.
Further reading: Chapter 10.
Legitimate Authority to Execute
In Chapter 10, this means the eligibility, under the organization's legitimate rules and currently applicable conditions, to let a particular action proceed into execution. The emphasis is on the standing conferred by the organization's governance arrangements for that action. It must not be treated as identical to Final Execution Authority (C010), the separately defined authority to issue a binding Allow or Deny adjudication at a particular control boundary.
Further reading: Chapter 10.
Execution Admission
Chapter 10 uses this expression to frame a specific question: At the point when an action is about to proceed into execution, does it satisfy the rules, authorizations, and necessary constraints that currently apply? An old approval is not a permanent pass. A change of recipient, terms, relevant state, or authority may mean the action must be assessed again.
Further reading: Chapter 10; continued in Chapter 11.
State (C011)
Normative definition: State is the set of facts and conditions, at a particular time, that directly bear on whether a specific Action can be adjudicated and executed.
In this book: While a payment waits in a queue, the person who approved it may move to another role. After a quotation is reviewed, its delivery terms or document version may change. What matters at execution time is the state relevant to the action that is actually about to occur.
Further reading: Chapters 10 and 14.
Making Constraints Effective at the Point of Action
Execution Control (C017)
Normative definition: Execution Control determines whether a concrete Action remains eligible to alter real-world State under current conditions—using current State, parameters, Authority, Policy, Boundary, and Evidence—and ensures that the determination can actually constrain execution.
In this book: A business application displaying Checked is not enough. The controlled path through which a quotation is formally sent or a payment submitted must actually be subject to the relevant conditions. Otherwise the check may remain a record rather than a constraint.
Further reading: Chapter 11; Chapters 14–15.
Execution Control Layer
Chapter 11 uses this expression as an organizational way of describing the responsibilities that make execution conditions effective for specific actions—and stop controlled operations from proceeding when those conditions are not met. This does not necessarily mean a separate software product, a central server, or a particular piece of hardware. The point is that the control relationship actually works, not that the arrangement looks like a distinct layer on a diagram.
Further reading: Chapter 11; Chapter 13.
Pre-Execution Control (C018)
Normative definition: Pre-Execution Control is a form of control that, before real-world side effects occur, uses current State, Policy, Authority, Boundary, and Evidence to determine whether an Action may proceed and can actually limit, delay, or deny it.
In this book: Before sending a formal quotation, the business must be able to check the actual file that is about to leave and the conditions currently applicable to it. If essential content changes after a check, the earlier result cannot simply be reused.
Further reading: Chapters 11 and 14.
Execution Boundary (C019)
Normative definition: An Execution Boundary is a boundary that, before an Action becomes real, finally constrains its State, parameters, eligibility, and permitted outcome and can actually allow or block the resulting real-world side effect.
In this book: The recurring image of the last gate points to this requirement. Where an action falls within the organization's controlled scope, a failure to satisfy necessary conditions must be capable of stopping it before its real-world effects occur.
Further reading: Chapters 9, 11, and 14.
Boundary Independence (C020)
Normative definition: Boundary Independence is the property that a control boundary does not depend for its existence on the constrained subject also having the ability to disable, rewrite, or bypass that boundary.
In this book: Putting a checking screen in a different application does not, by itself, establish an independent boundary. If the actor or path being constrained can freely change the conditions or go around the check, the separation may be only superficial.
Further reading: Chapters 9 and 11.
Fail-Secure (C023)
Normative definition: Fail-Secure is the overarching execution semantics under which, when required execution conditions, State, Evidence, or Boundary cannot be sufficiently established, uncertainty must not be converted into real-world execution. The default direction is to contract Execution Capability, not expand permission to execute.
In this book: If the system cannot confirm that a required approval is still valid, unable to verify must not quietly become approved. The organization needs a clear way to stop, explain, recover, or use an appropriately governed exception procedure.
Further reading: Chapter 14.
Trust, Evidence, and Changing the Rules
Trust (C013)
Normative definition: Trust is a bounded assumption, made for a particular execution adjudication, that permits a subject, component, or source of facts to perform a specified judgment function.
In this book: Giving employees and agents meaningful work does not require unlimited trust. The organization still needs to know the scope of their tasks, what can be checked, and what should happen when circumstances fall outside that scope.
Further reading: Chapters 9 and 15.
Evidence (C021)
Normative definition: Evidence is material capable of supporting verifiable facts about pre-execution conditions, execution adjudication, boundary State, or execution results.
In this book: A system record showing Approved may establish that a particular check or decision took place. On its own, it cannot prove that the attachment a customer actually received was the same document that had been checked.
Further reading: Chapters 9 and 14.
Governance (C022)
Normative definition: Governance is the constrained process that determines execution rules, roles, thresholds, State, and the conditions for their change and recovery. It may change future Execution Boundaries but is not the execution of each real-world Action.
In this book: Deciding who may set discount limits, change payment conditions, or approve necessary exceptions is a matter of governance. It is distinct from deciding who clicks Send or initiates a payment on a particular occasion.
Further reading: Chapter 10; Chapters 13–15.