Shadow AI at Work: A Hudson Valley Business Checklist

AI is already showing up in the businesses we encounter, sometimes with permission and sometimes without it. It often begins with one or two people experimenting, and those people are frequently company leaders.

When I talk with an owner about this, my first question isn’t whether AI is good or bad for the business. It’s whether they know where it’s already being used.

The review needs to go beyond ChatGPT or Copilot. AI can show up as a meeting recorder, browser extension, desktop application or feature in software the company already owns, which is why NIST’s AI inventory guidance includes embedded AI systems.

We’ve written about how Hudson Valley businesses can put AI to work. This article deals with the less visible part of that work: finding the tools already in use, deciding what they should be allowed to do and checking whether the controls actually work.

What is shadow AI?

We think of shadow AI much like shadow IT: technology being used for company work without IT having enough visibility or control over it. With AI, the review also needs to cover information submitted to the tool, access to business systems and any actions it can take; NIST addresses these concerns through its guidance on data privacy, connected components and human oversight (NIST Generative AI Profile).

When we look at AI from an IT and security perspective, we want answers to three questions: What company information can go into it? What systems and data can it access? And what can it do without a person approving the action? Those answers are more useful than a yes-or-no entry on an approved software list.

An owner saying “go ahead and try it” hasn’t necessarily answered any of those questions. My advice is to take control of AI in your company and lead its adoption, including the experiments happening at the leadership level.

Don’t rely only on asking employees what AI they use

Employee conversations are worth having, but they shouldn’t be the whole inventory. Compare what people tell you with the browser, endpoint and application-permission information your tools can actually collect; those are distinct discovery areas in documented AI discovery methods.

Ask IT to investigate the following:

• AI websites and accounts: Check which assistants people access, including ChatGPT, Claude, Gemini and similar services. Record whether the account is personal or organization-managed, where that can be established.
• Browser extensions: Review writing assistants, summarizers and other AI extensions. Ask what page content they can read and whether the extension is approved.
• Desktop applications: Include installed AI assistants and local AI tools. Ask whether processing stays on the device, uses a cloud service or can do both.
• Meeting and transcription tools: Check who invites bots, what they record, where transcripts go and who can access them.
• Existing business software: Ask application owners which AI features are enabled in CRM, HR, accounting and other SaaS platforms. Include trials and newly enabled features.
• OAuth applications and connectors: Review applications granted access to business accounts. Identify connections to email, SharePoint, OneDrive, calendars, CRM and other systems.
• Uploads and agents: Identify workflows involving pasted text, file uploads or agents that can send messages, change records or perform other actions.

For each tool, document who uses it, whether the account is personal or company-owned, what goes into it and who approved the use. Include the connected systems and the person responsible for reviewing access, consistent with NIST’s inventory recommendations.

Don’t accept “we block ChatGPT” as the answer to the inventory question. Website blocking doesn’t establish what’s happening in desktop applications, extensions or connected tools; those need their own review (AI discovery coverage example).

An inventory also needs someone to maintain it. Put a review on the calendar and make requests for new AI tools part of the normal IT process, rather than treating the first scan as the finished job.

Paying for an AI product doesn’t make every use of it safe

Don’t let “enterprise” or “business” on a subscription end the security review. We still want to know which identity the employee is using, what the AI can access, whether prompts or files are retained and which administrative controls are available.

Have the employee show you the account and workspace they use for company work. For a personal ChatGPT, Claude, Gemini or similar account, ask who controls it and what happens to company information when that person leaves. If the person also has an approved business account, check which one is actually being used.

Microsoft says Copilot Chat with enterprise data protection doesn’t use prompts and responses to train foundation models, but available controls depend on the subscription and web-search queries have separate handling (Microsoft Copilot Chat privacy documentation). If you’re also sorting through the licensing, our Microsoft 365 pricing and licensing article discusses the different offerings; buying a license and approving how it will be used are separate decisions.

Write down which tasks and categories of information the account is approved to handle. Before permitting confidential information, review retention, deletion, training use, contractual safeguards and logging.

For Microsoft Copilot or another tool connected to Microsoft 365, include a permissions review in the rollout plan. Test with a representative user: which files can that person access, which content can they attach, and which integrations have been enabled? Microsoft documents multiple ways organizational content can enter Copilot Chat, including uploads and selection of accessible files, so the review should cover more than typed prompts (Microsoft Copilot Chat privacy documentation).

A policy is useful, but technical controls matter too

Tell employees what information they can and cannot put into AI, but don’t make employees your only data loss prevention control. DLP tools can help detect sensitive information and, in supported configurations, warn or block someone before it is submitted (Microsoft’s documentation on controls for third-party AI apps).

At Fisch, we use tools that help us identify AI use, see account context and inspect submitted information where the deployment supports it. We still need to establish which devices and applications are covered before telling an owner what we can see.

Ask your IT team to demonstrate the controls, not just show you a dashboard:

• Discovery: Which AI sites, desktop applications and embedded features are visible? What remains outside coverage?
• Account controls: Can the system distinguish a personal account from an approved business account for this specific service? Can it enforce that distinction, or only report it?
• Sensitive-data handling: Can it detect the information your organization needs to protect? Ask separately about warnings, blocking and redaction before submission, and accept each capability only where it is supported and tested.
• File uploads: Does the control cover pasted text, attachments or both? Test representative file types and approved browsers with synthetic information.
• Activity records: Does the system retain full content, a sensitive-data classification or only an event record? Define who may access the logs and how long they are kept.
• Permissions: Who can authorize new integrations, and how will access be revoked when an experiment ends or someone leaves?

An alert isn’t the same as a block. Microsoft’s documentation describes discovery policies running in audit mode and a prompt-detection policy that doesn’t capture content, alongside separate enforcement capabilities (Microsoft AI data-security considerations).

Test a file upload and a pasted prompt using made-up sensitive information. Record what happened, whether the submission went through and which browser and account were used.

Monitoring also needs its own boundaries. Tell staff what is monitored, obtain the appropriate privacy and legal review, limit access to sensitive logs and avoid collecting more content than the review requires.

The information matters more than the tool’s name

The risk looks different across the organizations we serve in the Hudson Valley. The examples below are illustrative, not reports of Fisch client incidents.

At a medical office, the concern might be patient information pasted into an unapproved assistant. A manufacturer may need to stop customer drawings or specifications from being uploaded to a personal account. A municipality could have an AI application connected to a shared document library without anyone reviewing what it can read.

Some uses look harmless until you ask what is in the material. A meeting bot at a professional-services firm might record confidential client advice; a nonprofit testing a grant-writing tool needs to distinguish public program descriptions from private case files.

For healthcare, HHS explains that a cloud service provider creating, receiving, maintaining or transmitting electronic protected health information on an organization’s behalf generally requires an appropriate business associate agreement, together with risk analysis and the other applicable HIPAA safeguards (HHS cloud-computing guidance).

A BAA doesn’t replace the rest of those responsibilities (HHS cloud-computing guidance). Bring regulated or contract-restricted uses into your cybersecurity and compliance review before experimenting with real information.

DOWNLOAD THE SHADOW AI CHECKLIST
Get the printable 12-question Shadow AI worksheet to review your organization’s AI use and assign next steps.

The 12-question shadow AI checklist

Use this at a leadership or IT meeting. Mark each answer Complete, Needs work, Unknown or Not applicable, and give unresolved items an owner and due date.

Visibility

1. Tools and ownership: Do we know which AI tools and accounts are used for work, with a business owner and approved purpose for each?
2. Hidden entry points: Have we included browser extensions, desktop tools, meeting bots and AI features inside existing software, rather than only well-known websites?

Data

3. Permitted information: Have we defined what may be pasted or uploaded, including information prohibited from each tool?
4. Terms and configuration: Have we reviewed the actual product, account type, retention, training use and configuration?
5. Restricted uses: Have security, compliance and legal owners approved any regulated or contract-restricted use?

Identity & Access

6. Business identity: Are approved accounts protected with appropriate authentication, and have we addressed personal-account use?
7. Permissions and offboarding: Is access to files, email and other business systems limited to the task and removed when no longer needed?

Agents & Integrations

8. Connected capabilities: Do we know which integrations and agents can read data or take actions, and who authorized them?
9. Human approval: Do staff know which outputs require review and which actions require approval before sending, spending, deleting or changing records?

Incident Response

10. Reporting and containment: Does everyone know whom to contact, what to pause and which evidence to preserve after a suspected exposure?

Ongoing Governance

11. Tested controls: Have we tested discovery, logging, applicable DLP controls and workflows with nonsensitive information, recording both results and coverage gaps?
12. Scheduled review: Is someone responsible for reviewing approved tools, training, exceptions, new features and policy changes?

Completing the checklist is not certification, an audit result or proof that your organization is secure. Use it to identify decisions and evidence still missing, not to produce a reassuring score.

An AI assistant and an AI agent create different risks

There’s a meaningful difference between approving an email draft and approving a tool to send emails on your behalf. Make the same distinction when evaluating a workflow that updates customer records, creates users or changes a business system.

For a new agent, give it the least access it needs and begin read-only where practical. Require human approval for consequential actions and test with nonsensitive information before expanding permissions. Those recommendations follow NIST’s emphasis on testing and human oversight (NIST Generative AI Profile).

Treat those permissions like access granted to a user or application, not a chatbot setting. Know who can disable the workflow, and check how to revoke the application’s access as well as the employee’s account.

If sensitive information has already been shared

Report it promptly to the person responsible for security, privacy or compliance. Pause the affected workflow and avoid submitting more information while the event is assessed.

Record the tool, account, time, information involved, files uploaded and permissions granted. Preserve relevant evidence in an approved location; do not delete records reflexively or assume deleting a chat establishes that every copy has been removed.

Have the response team review containment, vendor support options and whether connected access needs to be revoked. If passwords, keys or other credentials were exposed, involve IT immediately to invalidate or rotate them and review related access.

Let the appropriate compliance and legal owners determine notification obligations from the actual circumstances. These are recommended response steps, not a finding that every mistaken prompt constitutes a reportable breach; NIST includes incident documentation and response planning in its Generative AI Profile.

Use the incident-response process you already maintain rather than inventing a separate one for every AI tool. Our cyber insurance preparation article also covers written response plans, named responsibilities and testing the escalation process.

You don’t need to finish a six-month project before acting

I’d rather see a smaller organization put a basic process to work than leave a perfect policy unfinished. Find what’s already being used, identify the uses that need immediate attention and tell people which tools and accounts they should use. Put temporary restrictions around sensitive information while the review is underway.

Then train the people involved, test the controls that are justified and assign someone to handle new requests. Our AI readiness and strategy work brings together tool inventory, risk review and governance planning; the point is to make this part of running IT, not a document everyone signs once and forgets.

Frequently asked questions

What is shadow AI?

Shadow AI is work-related AI use outside an organization’s approved tools, policies or oversight. Discovery should examine applications and sensitive-data sharing, not merely whether a popular chatbot website was visited (Microsoft shadow AI discovery guidance).

How can a business detect shadow AI?

Combine staff conversations with a technical review of browser activity, endpoints and application permissions. Supported discovery tools can inspect several of those surfaces, but the review must document its coverage limits (AI discovery coverage example).

Can employees use personal ChatGPT accounts for work?

Only where the organization explicitly permits the account and the use case. Keep confidential, regulated and contract-restricted information out of personal AI accounts unless the appropriate owners have reviewed and approved that specific arrangement.

How can companies prevent sensitive data from being entered into AI?

Use clear data rules together with tested controls for the channels employees actually use. Supported DLP implementations can warn or block sensitive sharing, but no single rule should be treated as universal protection (Microsoft AI-app control documentation).

Can IT monitor AI usage?

Yes, within the scope of the deployed tools, supported applications and configured policies. Seeing a site visit, detecting sensitive information and retaining prompt content are different capabilities, as shown in Microsoft’s AI monitoring considerations.

What is an AI governance policy?

An AI governance policy sets the organization’s rules for choosing, approving and overseeing AI. Include permitted information, account requirements, decision owners, human review, incident reporting and periodic reassessment, drawing on the governance themes in NIST’s Generative AI Profile.

Does shadow AI mean employees are acting maliciously?

No. In our experience, AI experimentation often begins with a small number of people, including company leaders; investigate the use and its risks without assuming bad intent.

Does turning off model training solve the problem?

No. Access, retention, integrations and output reliability still need review alongside training settings, consistent with the broader risks identified in NIST’s Generative AI Profile.

Should every business ban personal AI accounts?

A ban needs a workable alternative for legitimate business tasks. Decide which uses are permitted, make restrictions on business information explicit and apply them to leadership as well as employees.

Where should we begin if we have no AI policy?

Name a leadership owner, inventory current use and establish an interim rule for sensitive information. Then approve a small set of useful workflows, train the people involved and test the controls before expanding.

Find out where AI is already being used

If you don’t know which AI tools your team is using, that’s where we’d begin. Fisch can help identify AI use, review personal versus company-approved accounts and examine how business information is being handled within the environments our tools support.

From there, we can help establish AI policies and work through where identity controls, DLP and monitoring make sense. Our AI consulting and integration services also cover Microsoft Copilot and AI-powered workflows, so the conversation can move from finding unapproved use to implementing tools the business has chosen.

Call 845-896-1800 or contact Fisch Solutions to walk through what’s happening in your business and discuss an AI risk assessment. We’ll agree on what needs to be examined before beginning hands-on discovery or making changes.