How to Use AI on Client Files Without Sending Them to a Cloud AI Service

Why is using AI on client files a confidentiality problem?

Using AI on client files is a confidentiality problem because most AI tools run in the cloud. Paste a contract into a chatbot and the text goes to the AI company's servers, under its terms of use. For a lawyer that is not a small detail.

ABA Formal Opinion 512, issued July 29, 2024, asks you to weigh the risk that client information is disclosed to or accessed by others outside the firm before it goes into an AI tool. For self-learning tools, it says informed consent is required first, and that general, boilerplate clauses in an engagement letter are not enough.

Local AI tools such as Ollama avoid the vendor problem, but their documentation assumes you know what an embedder or a vector database is. And going local does not improve accuracy: a private model can still invent a case.

How to set up a private AI on your own computer, step by step

To set up a private AI, you check your hardware, install a local AI app, pick a model that fits your memory, point every setting at your own machine, and prove it works offline. Here is each step, done by hand.

1. Check your memory. On Windows, open Settings, System, About, and read Installed RAM. On a Mac, open About This Mac. 16 GB is the practical floor. With 8 GB, only very small models fit, and they make too many mistakes for client work.
2. Install a local AI app. AnythingLLM Desktop is free, open source under the MIT license, and runs on Windows and Mac. It reads your documents and chats with you about them. On Windows, install it for the current user only.
3. Pick the built-in AI provider. During setup, choose the app's own engine. Do not pick OpenAI, Anthropic or another cloud name, because those send your text over the internet.
4. Choose a model that fits. The model has to fit in memory while it works. Bigger memory means a bigger, more capable model. Check each model's license before business use.
5. Check every place data could leave. The LLM provider, the embedder (the small model that makes documents searchable) and the vector database (where that search index is stored) should all be local. Clear any cloud API key you find.
6. Turn off telemetry and web agents. The AnythingLLM privacy policy explains what telemetry covers. You need neither for client work.
7. Test with the internet off. Turn off Wi-Fi, unplug the cable, reopen the app, upload a public document and ask three questions only that document can answer.
8. Encrypt the disk. The client files now live on your laptop. Use BitLocker or Device encryption on Windows, or FileVault on a Mac.
9. Use one workspace per matter. The model can pull text from any document in a workspace. Two clients in one workspace means one client's facts can show up in the other's answer.
10. Check every answer. Quotes against the source, citations in Westlaw, Lexis or the official reporter, dates and dollar amounts against the document.

Steps 4 to 6 are where most people get stuck: one wrong pick sends text to a cloud service. If you want local AI for marketing work instead, AutoEarnaAI runs on Ollama.

What a local model is good at, and what it is not

This split comes from the kit's own "what it does well" page. The short version: the answer has to be in the text you gave it.

The faster way: a setup kit with the choices already made

The faster way is to follow one path where the app, the model and every setting are already decided. I built Private AI Office for solo and small law firms: a set of PDF guides you read in order, two Word templates, and a plain-text copy of every prompt.

It includes no software. You download AnythingLLM and the model from their official sources. The model picks are all Qwen models under the Apache 2.0 license, matched to three memory tiers.

What it does not do matters just as much here. It does not make you compliant with anything, and it is not legal advice. The kit's own one-line summary: your data stays on your computer, and whether your setup meets your professional duties is a decision for you and your state bar.

How to use it, step by step

1. Buy and download. Gumroad sends the zip right after payment. Open the README, then the start-here guide.
2. Check your computer. About 10 minutes. Write down your memory, free disk and, on Windows, graphics card memory, then find your tier.
3. Install AnythingLLM. Follow the Windows or Mac guide. It covers the "Windows protected your PC" box and the Mac Gatekeeper prompt, and what each one means.
4. Download your model. The which-model guide gives one file name per tier. A Starter model is about 6 GB, which the guide says takes 10 to 40 minutes on a normal office connection.
5. Lock it down before any client file. Work through the privacy switches, the offline proof test and disk encryption, in that order.
6. Ask your first real questions. Set the matter workspace to Query mode, which answers only from your uploaded documents, and start with the contract review prompts.
7. Read the ethics kit before client matters. Go through the ABA 512 checklist, check your state bar's guidance, then edit the client letter and firm policy templates.

What the prompts look like

Every prompt has fields in brackets and ends with the same rule line. This one is from the contract review set:

List every deadline, notice period, and time
limit in [contract name]. For each one give:
what must happen, who must do it, how much
time, what starts the clock, and the section.
Use only the uploaded documents. If the answer
is not there, write "Not in the documents."

Under each prompt sits a check note. For this one: confirm what starts the clock, because models often confuse "from signing" with "from notice."

The verify-everything page explains why that habit matters. In Mata v. Avianca (S.D.N.Y. 2023), lawyers filed opinions an AI chatbot had made up and were fined $5,000. In Wadsworth v. Walmart (D. Wyo. 2025), the drafting lawyer used his firm's own in-house AI tool and was fined $3,000. Private did not mean accurate.

How much time it saves

I have not timed a firm doing this setup alone against the same setup with the kit, so I will not give you a number. The kit's own plan fits the whole setup into one afternoon, most of it spent waiting for the model to download.

The bigger saving is the research you skip: which app, which model, and which of the five settings could leak data. Our free tools page has a few more small helpers.