Chapter 10: Scaling Sovereignty¶
Bob started as a single-instance monitoring agent running in a home lab. The architecture was designed to be one person's answer to a recurring problem. What this chapter describes is what happens when that architecture meets the reality of a digital agency or a managed services provider with multiple clients, multiple regulated environments, and a mandate that none of those environments bleed into each other.
The technical changes are modest; the organizational ones are harder.
The Multi-Client Problem¶
A digital agency running SQL Server for enterprise clients faces a different version of the report problem than a single DBA monitoring one environment. It is not 214 pages. It is fourteen separate 214-page reports, for fourteen clients, each with its own schema, its own compliance requirements, its own definition of "critical." The problem is not volume per client. The problem is the sum across clients, and the fact that each client's data must be treated as if the other clients do not exist.
That last point is where cloud AI fails hardest. If you send client A's query plans to an external API for analysis, and the same API processes client B's data ten minutes later, you have a concrete data residency problem that shows up in GDPR data processing records, in PCI DSS scoping reviews, in SOC 2 control attestations. A shared external API is a shared data processor. Shared data processors require agreements, documentation, and in some regulatory regimes, prior client consent.
Local inference eliminates that problem by eliminating the shared processor. Client A's query plans and client B's query plans are processed by the same Ollama instance on your hardware, but they never leave your network, and they never commingle in any external system's memory. You are the data processor. You have always been the data processor. The AI did not change that.
This is the argument for local sovereignty at agency scale. It is not philosophical. It is the concrete answer to a compliance question your legal team will eventually ask.
The Architecture at Scale¶
Scaling Bob from one environment to multiple clients requires two changes to the existing architecture.
The first change is multi-tenancy in the database layer. The BookOfBob schema on db-host at 10.0.0.14:50003 is currently single-tenant: one set of tables, one history, one remediation log. For multi-client operation, you need either separate databases per client, which is clean but expensive in SQL Server licensing terms, or a tenant-aware schema that adds a client_id column to every table and enforces row-level security so the agent serving one client cannot accidentally read another client's history.
The row-level security approach is correct here. It is also exactly the kind of database security work a DBA should be comfortable with:
-- Add client isolation to the snapshot history table
ALTER TABLE dbo.SnapshotHistory
ADD client_id INT NOT NULL DEFAULT 0;
-- Create a security policy that filters rows by client context
CREATE FUNCTION dbo.fn_SecurityFilter(@client_id AS INT)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN SELECT 1 AS result
WHERE @client_id = CAST(SESSION_CONTEXT(N'client_id') AS INT);
GO
CREATE SECURITY POLICY dbo.ClientIsolationPolicy
ADD FILTER PREDICATE dbo.fn_SecurityFilter(client_id) ON dbo.SnapshotHistory,
ADD FILTER PREDICATE dbo.fn_SecurityFilter(client_id) ON dbo.FindingHistory,
ADD FILTER PREDICATE dbo.fn_SecurityFilter(client_id) ON dbo.RemediationLog
WITH (STATE = ON);
GO
-- source: bob@c549c88
The agent sets the session context at connection time:
def get_client_connection(client_id: int) -> pyodbc.Connection:
"""
Return a database connection scoped to a specific client.
The session context enforces row-level security for the lifetime
of this connection. Do not share connections across clients.
"""
conn = pyodbc.connect(CONNECTION_STRING)
conn.execute(
"EXEC sp_set_session_context N'client_id', ?",
client_id
)
return conn
# source: bob@c549c88
With this in place, the monitoring agent for client A physically cannot read client B's history, even if a bug in the routing logic passes the wrong connection. The database enforces the isolation.
The second change is agent configuration isolation. Each client needs its own system prompt context (their specific applications, their compliance requirements, their escalation contacts) and its own operational windows (when is it acceptable to auto-apply index maintenance for this client). The cleanest approach is one configuration file per client, pulled by the agent at the start of each monitoring cycle:
Each file contains the client-specific settings: SQL Server connection string, Proxmox VM ID for snapshots, operational window schedule, alert contacts, and the path to the client-specific system prompt addendum. The base system prompt is shared. The addendum is where client-specific knowledge lives: "this client's top-10 reports run at 6 AM every Monday, expect elevated CPU and do not alert on that pattern."
The Inference Layer Does Not Need to Scale¶
One common assumption when designing multi-client architectures is that you need more hardware per client. That assumption is wrong here.
A single inference-host-1 at 10.0.0.70 with its RTX 3090 handles concurrent diagnostic requests through Ollama's OLLAMA_NUM_PARALLEL=2 setting. In practice, a monitoring cycle for one client takes between 2 and 8 seconds of inference time. With a 60-second probe cycle per client and up to a dozen clients, the inference load is easily within the capacity of a single GPU host running gemma4:26b.
The inference host does not know whose data it is processing. It receives a prompt, generates a response, returns it. The client isolation is handled entirely in the agent layer and the database layer. inference-host-1 is a shared resource; the isolation boundary is upstream of it.
This is also why the data residency argument works. The LLM at 10.0.0.70 processes client data, but that data never leaves the 10.0.0.0/24 network. The LLM is your tool. It does not have a customer identifier, a telemetry endpoint, or a terms-of-service agreement that allows the provider to use your prompts for model training. The Ollama instance on your hardware is a tool, not a vendor relationship.
The Business Case¶
The business case for local sovereignty at agency scale is a cost and compliance argument that gets stronger the more clients you add.
On the cost side: cloud API pricing for high-frequency monitoring across twelve clients would run into meaningful money. The monitoring cycle for a busy SQL Server instance, sixty seconds, ten to fifteen minutes of wait stats, top-query data, and a query plan or two, is a several-hundred-token prompt. At current GPT-4 pricing, running that continuously across twelve clients would cost more than the hardware depreciation on inference-host-1 in the first few months. After the hardware is paid off, the marginal cost per additional client is zero.
On the compliance side: the ability to tell a GDPR-regulated client in the EU that their database diagnostic data never leaves the LAN is not just a legal convenience. It is a sales differentiator. Most of the managed services providers competing in this space are using cloud tooling because it is easier to set up. Local sovereignty requires infrastructure investment and technical depth. It is not the path of least resistance, and that is exactly why it differentiates.
The combination of zero marginal inference cost and complete data residency control changes the calculation for which clients you can take on. Healthcare organizations under HIPAA, utilities under NERC CIP, financial services under PCI DSS: these are the clients with the most acute data residency requirements and the most willingness to pay for a provider who can credibly answer the compliance question. Cloud tooling cannot give them that answer. Local inference can.
The economics work out as follows. A monitoring cycle for one SQL Server instance, running every 60 seconds, generates a prompt of roughly 500 to 1,500 tokens depending on the diagnostic payload size. Call it 1,000 tokens per cycle as a conservative average. Sixty cycles per hour, 24 hours per day, 365 days per year, across twelve clients: that is roughly 6.3 billion tokens of input per year. At GPT-4 Turbo pricing as of 2025-01, input tokens cost $0.01 per 1,000. That is $63,000 per year in inference costs, before output tokens. Output tokens at $0.03 per 1,000 add roughly another $30,000 annually for typical diagnostic response lengths. Call the cloud API cost $90,000 per year to monitor twelve SQL Server instances continuously.
inference-host-1 at 10.0.0.70 with the Ryzen 9 9950X and RTX 3090 cost approximately $3,500 in hardware. Power consumption running inference continuously is roughly 400 watts, or about 3,500 kWh per year. At $0.12 per kWh, that is $420 per year in electricity. Depreciate the hardware over three years: $1,167 per year. Total cost: roughly $1,600 per year to monitor twelve clients.
The break-even against cloud API costs happens in about a week. After the hardware is paid off, every additional client is nearly free to add. A 5-DBA shop choosing between cloud AI monitoring and local inference is choosing between a service that costs more per year than many of their client contracts are worth, and a system they own outright and can extend without asking permission.
The political angle of this math is the part that gets underweighted. The agency that owns its inference layer does not have a vendor relationship that can change pricing, deprecate a model, or decide to train on your clients' data as a terms-of-service update. Cloud AI vendors have done all three of those things in the last two years. Each time they do it, the agencies dependent on them have to respond: renegotiate contracts, update privacy policies, re-evaluate compliance posture. That reactive work costs time and attention that a local inference deployment does not require.
The agency that built Bob controls what version of the model it runs, when it upgrades, what data enters the inference context, and what pricing it charges clients. None of those decisions require a third party's approval or pricing team's roadmap. That independence is the point.
Where This Goes¶
Eighteen months of building has produced an architecture that works. One DBA's home lab, two inference hosts, one SQL Server target, one monitoring loop. The core is proven. The extension to multiple clients is a configuration and schema change, not an architectural one.
The next eighteen months are about proving the extension: deploying the multi-tenant configuration in the agency context, building the track record that justifies expanding the auto-apply scope, and making the community artifacts good enough that another DBA can get from zero to a running deployment without having to read the source code to understand what is happening.
That is not a moonshot. It is the natural progression of a system that was designed to be extended. The architecture supports it. The hardware is already running. The reasoning layer is proven. Extension is configuration.
What remains is the work of making it available to more than one server room.
The book ends here. The project does not.