Terrence
Weston

AI systems, in production

I run an AI platform inside a HIPAA-regulated community health center in Kansas City, and I keep it secure, audited and useful. This page is what I build and how I work.

10+
years in IT
2.8 s
median response, down from 7.7 s
107
regression tests that block a bad update
0
changes the AI can make without a person

The problem

Switching AI on is easy. Putting it to work safely is the job.

In a HIPAA-regulated organization, confidentiality rules are not optional. Before AI touches real work, three questions need an answer.

Who can use it, and what can it see?

Sign-in, roles, and data kept apart by department.

What can it change without a person?

Nothing. Every write waits for a human, and the server enforces that, not the model, so no prompt can talk its way around it.

Can you prove what it did?

Every question, answer and tool call is logged.

The platform

One place to ask, connected to the systems IT already runs.

I architected and built the organization's AI platform, and I administer and support it. Staff ask in plain English, and specialists do the work through the tools IT already trusts. In all, I have built nine AI agents across IT, Finance and a clinical pilot.

Staff sign in with Entra ID, ask in plain English The AI platform Five specialists, routed in code: help desk, assets, logs, network, printers Zendesk help desk tickets Active Directory accounts and groups Microsoft 365 the Microsoft cloud Asset system equipment inventory IT knowledge base fed nightly by resolved tickets Staff sign in with Entra ID, ask in plain English The AI platform Five specialists, routed in code: help desk, assets, logs, network, printers Zendesk help desk tickets Active Directory accounts and groups Microsoft 365 the Microsoft cloud Asset system equipment inventory IT knowledge base fed nightly by resolved tickets

For the technical reader: each connection is a Model Context Protocol (MCP) tool server. Routing between specialists is done in code, not left to the model. The knowledge base is retrieval (RAG) over internal documents. All five specialists are in use today.

Guardrails

The AI proposes. A person approves.

Approval used to live in the model. I moved it into the server, so the AI cannot act on its own.

  1. 1
    You ask for a change
    For example, an update to a help desk ticket.
  2. 2
    The AI drafts it, and stops
    It has no way to carry the change out.
  3. 3
    The server builds a review card
    From the tool server's own preview, not the AI's summary.
  4. 4
    You tap Approve
    Only that tap executes the change.

Illustration of a review card, not a real screen.

Access, isolation, audit

Who gets in, what they can reach, and a record of all of it.

Access

Entra ID sign-in with role-based access. People use the account they already have, and their role decides what they can do.

Isolation

An isolated Finance instance with zero cross-department retrieval. The same principle as an ethical wall.

Audit

Every question, answer and tool call is logged, with response time. Reviewed continuously and used for usage reporting.

A lesson that became policy

I asked our AI how many laptops we had available to assign.

At first, it said
2,309
Wrong. It read the inventory like a document and guessed.
After I traced the problem, it said
9
Right. Now it counts the actual records.

The rule I made from it: AI can read documents, but it never estimates a number. It counts.

The same rule, for the CFO

A 90-page policy packet, answered in seconds.

The CFO asked whether the AI could use the organization’s finance policy packet. Within days he had a dedicated policy assistant on its own walled-off instance. He called it “slick” and asked for his whole team.

Every answer shows its source

The packet is split into its 43 policies. Each answer names the policy and page, and links straight to that section, so he can check it in one click.

It says when it doesn’t know

If the policies don’t cover a question, it says so instead of guessing. Tested with known answers before he ever saw it.

Fixed the real problem first

His uploads had been failing. I traced it to a text-recognition crash on the company letterhead, which was breaking letterhead PDFs across the whole platform, and fixed it everywhere.

Owned by Finance, kept private

The policy originals now live in a managed library that Finance owns, with a read-only backup for outages. Along the way I found the finance team’s site readable by the whole organization and locked it down.

Where AI fits first

I measured before I built.

Before choosing the next department for AI, I pulled every Microsoft Copilot prompt staff had sent in 90 days: 6,409 prompts from 157 people. The data picked the department, and it surfaced a compliance gap nobody had seen.

Private by design

Exported with Microsoft Purview eDiscovery, de-identified, and categorized by our own on-premises AI server. No prompt text left the network, and nobody read the prompts.

The data picked the department

Medical sent nearly half of all prompts, mostly clinical questions. That led to a clinical assistant now in a pilot with four clinicians: dosing math comes from tested calculators, not the AI, and drug facts are quoted straight from FDA labels with the source shown.

A compliance gap, found early

Some prompts held patient details, and Copilot’s web search sends parts of a question outside the Business Associate Agreement. I took leadership a decision list: policy, BAA scope, web search settings and retention.

Repeatable

The method is documented so the review can run every quarter, to measure change and choose the next department.

The clinical pilot

From a clinician’s complaint to “game changer” in one afternoon.

The pilot started with one provider. When I widened it, I chose the next clinicians from the usage data, including the director of the residency program, who trains every incoming provider and was the organization’s heaviest clinical AI user. On her first day she uploaded a clinical photo and the assistant declined. Fifteen minutes later it could describe clinical photos. Seven minutes after that she wrote back: “WOW! This is about to be a game changer.”

Describes, doesn’t diagnose

It describes a clinical photo in standard exam terms. It offers diagnostic considerations only when asked, clearly labeled, and every description states what a photo can’t show.

Every change is tested first

Nothing goes live until it passes 35 clinical test cases and replays of real pilot chats. When a resident’s chat showed the model rounding a lab index wrong in its head, that index moved to a tested calculator. Every dose number in an answer is now checked against what the calculator or FDA label actually returned before the clinician sees it.

The model was picked with data

I tested four open models that fit our GPU against about 200 cases built from real staff usage: documents, images, clinical questions and checks for made-up answers. The one we run scored best and was fastest.

Testers hear back every week

A short Friday email tells testers what they asked for, what got fixed and what’s new, so feedback keeps coming.

Built toward patient data, done right

The goal is an on-premises assistant cleared for patient information. Encrypted traffic and encrypted nightly backups, with a passed restore test, are already in place. The readiness plan ends with a formal risk assessment and compliance sign-off.

Results

Measured, not assumed.

Median response time

The chat interface I built, against the tool it replaced.
Before
7.7 s
Now
2.8 s
107

automated regression tests. A failing update is blocked before it ships.

44

live evaluation questions, run read-only against production.

Weekly

use by the Director of IT, 77 questions since July, zero errors in 30 days. The demo reached the CEO.

Judgment

Knowing what to buy, and what to turn down.

A yes from the CFO

I compared four ways to run the organization’s first on-premises AI: a cloud service, a GPU in the existing servers, a new GPU server, or a dedicated card in a spare workstation. The dedicated card serves about 10 times as many people at once as the existing-server option, at about a third of the cost per user, and avoided a $40,000+ server purchase. The CFO approved it. Live since October 2026. After launch I measured what each extra user costs in GPU memory and doubled how many people it serves at once, on the same card.

Cloud service
GPU in existing servers
New GPU server
Dedicated card
Each one weighed on cost, capacity, isolation and licensing
Approved and in production

A no to a vendor

I evaluated a commercial AI platform and ruled it out. It would not sign a BAA, the agreement HIPAA requires before a vendor may handle patient data.

Will you sign a BAA?No
So it could not touch protected health data
Ruled out

Emergency staff alerts

When the building is closed, who actually got the message?

The health center announced closures and emergencies through a manager call tree, and nobody could say who had been reached. I built a replacement that sends every alert by text and email at once and reports back person by person.

Works when the building doesn’t

Hosted in the cloud, not on-site, so it still sends when the building or network is down and the sender is at home at 5 AM.

The delivery report is the product

Each person shows delivered, failed or opted out, with a tap-to-call list of anyone not reached and a full audit trail.

Zero data entry

A nightly sync from the staff directory and the MFA system covers about 92% of active staff with mobile numbers. People who leave drop off automatically, and a safety brake stops the sync if it would remove more than 10% at once.

Only the right people can send

Single sign-on plus MFA, a short sender allow-list, and it fails closed if any security layer is misconfigured.

Carrier-approved opt-in

Text verification was rejected once for tying texts to employment. I rebuilt sign-up as text-to-join with standalone consent, terms and privacy pages, and the resubmission was approved within 30 minutes.

Hard to misfire

One screen with an exact message preview, then Practice or Real alert. The server refuses to send if the recipient count changed since the preview.

Built with Python and Flask on Azure App Service, Azure Table Storage with managed identity, Twilio text messaging, Microsoft Graph mail, Entra ID single sign-on. About $2 a month plus per-text cost.

Around the platform

The rest of the work.

A custom chat interface

FastAPI, Entra ID sign-in, and file upload with OCR and vision. It replaced the previous tool and cut the median response to 2.8 s.

Platform migrations

AnythingLLM to LibreChat to a custom interface, with version upgrades, a DEV-to-LIVE release process and a user guide.

Automation and escalation

A modular PowerShell suite in Azure DevOps, Microsoft Graph integrations with certificate-based auth, and tier 2/3 escalation for the server fleet and endpoints.

Outside work

I build this on my own time too.

An agentic kitchen assistant

Tool calling over 129 recipes and 770 meal-history entries. A vision model turns a photo of a handwritten list into structured items.

eggs x12
2 onions
rice
milk
Eggs 12
Onions 2
Rice 1
Milk 1
Illustration: photo in, structured list out

Custom MCP servers

Tool servers for infrastructure control and search, used daily with Claude Code across a four-machine self-hosted lab.

Claude
Code
MCP
servers
1
2
3
4
Four machines, one set of tool servers

Toolkit

What I work with.

AI agents and RAG
Agent design with code-based routing and specialist handoff, Model Context Protocol (MCP) servers, tool calling, structured outputs, system prompts, RAG pipelines, embeddings and vector search, OCR and vision models.
AI platform operations
Claude daily, including Claude Code. LibreChat, Open WebUI, Ollama, model serving and upgrades, usage and performance monitoring.
Security and governance
Role-based access control, audit logging, data isolation between departments, server-enforced approval guardrails, vendor risk review (BAA and data handling), HIPAA, MFA, Microsoft Purview eDiscovery (litigation email collection, content search and purge).
Testing
Automated regression suites with scripted model and tool fakes, live evaluation sets, fabricated-answer detection, model comparisons on test sets built from real usage.
Microsoft cloud
Azure, Azure Communication Services, Entra ID (app registrations, SSO and OIDC, certificate-based auth), Microsoft 365, Microsoft Graph, Exchange, Intune, Azure DevOps.
Development and automation
Python (FastAPI, Flask, SQLite, pytest), PowerShell, Bash, REST APIs and JSON, Zendesk API pipelines, JavaScript front ends, Git.
Infrastructure
GPU inference hosting (NVIDIA RTX PRO GPU), Docker, Nutanix AHV, Ubuntu, Windows Server, Active Directory, Graylog and OpenSearch, NinjaOne.

Background

From the flight line to the AI platform.

Hooked at 11 on a 100 MHz Compaq Presario. Built my first computer at 15 with $800 from a part-time job. Fixed aircraft in the Air Force, kept building computers the whole time, and came back to tech for good.

2003 to 2014

U.S. Air Force

C-5 Galaxy Crew Chief

Active duty at Dover AFB, then Electrical Systems Journeyman in the Reserve.

2015 to 2022

Delaware Hospice

Helpdesk Technician, de facto Systems Administrator

Sole IT trainer for 600+ staff hires across four offices. Led an MFA rollout.

2022 to 2024

The Purple Guys

Field Engineer 2

Lead engineer across 100+ client environments.

2024 to now

Community health center, Kansas City

Systems Administrator

Leads the organization's AI platform.

B.S., Computer and Network SecurityWilmington University, 2015. 3.87 GPA.
A.S., Information SecurityDelaware Technical and Community College, 2015.