top of page
Search

Reviewing Shell Scripts with AI Before They Hit 10,000 Macs

Writer: MacSmithAI
MacSmithAI
Sep 4
7 min read

Updated: 4 days ago

Somewhere in your Intune tenant there is a shell script you wrote in twenty minutes on a Thursday. It is assigned to a device group. It runs as root — that is the default, unless you flip Run script as signed-in user to Yes — and Microsoft's documentation is blunt about what happens next: a script that returns a non-zero exit code, or is simply malformed, is reported as Failed, and one that runs longer than 60 minutes is stopped and reported the same way. Nothing in that pipeline reads your logic. It just runs it, as root, on every Mac in scope.


That is the real argument for putting a review layer in front of fleet scripts. It is also why "I pasted it into Claude and it said it looked good" is not a review. Here is the order I actually run things in, what each layer reliably catches, and — more usefully — where each one stops being worth trusting.


Run the linter first, always

ShellCheck is still the cheapest quality win in Mac administration. Current release is v0.11.0 from August 2025, and on macOS it is one command away:


brew install shellcheck
shellcheck ./remediate_launchagent.sh

It is deterministic, it runs in under a second, and every finding has a code and a wiki page you can send to whoever wrote the script. Three that come up constantly in fleet code:


  • SC2086 — "Double quote to prevent globbing and word splitting." echo $1 becomes echo "$1". Unquoted expansions get split on IFS and then glob-expanded, which is exactly how a script that works on your Mac breaks on the one machine with a space in the volume name.

  • SC2164 — "Use cd ... || exit in case cd fails." If cd generated_files fails and the next line is rm -r *.c, the script cheerfully deletes in whatever directory it was already sitting in.

  • SC2115 — "Use "${var:?}" to ensure this never expands to /*." The canonical example is rm -rf "$STEAMROOT/"*. If STEAMROOT is empty, that clears the root of the volume. Writing rm -rf "${STEAMROOT:?}/"* makes the command fail instead.


Read that third one again with root privileges and 10,000 endpoints in mind. That is not a style nit.


One thing worth knowing before you get clever with shebangs: ShellCheck checks sh and bash, and will check portability against shells like dash and ksh when your shebang is #!/bin/sh. zsh is not one of the shells it supports. Intune explicitly allows #!/usr/bin/env zsh, and zsh has been the default shell for new user accounts since macOS 10.15 — so it feels like the modern choice. But writing your fleet scripts in zsh quietly opts you out of static analysis entirely. Unless you have a concrete reason to need zsh, keep #!/bin/bash or #!/bin/sh on anything that ships to devices, and keep the linter.


What AI review is genuinely good at

Once the linter is clean, the interesting failures are semantic — the script is valid shell that does the wrong thing. That is where a model earns its keep, because it is reasoning about intent rather than syntax. The findings that have actually saved me:


Idempotency. Intune scripts with a frequency configured also run again after a device restart. A model asked "what happens if this runs a second time?" will spot the appended-not-replaced config line, the duplicated LaunchDaemon, the counter that only ever increments.


Exit-code semantics. Because non-zero means Failed, and Failed feeds your retry setting, an exit code is a business decision, not a formality. Models are good at pointing out that you exit 1 on "already compliant," which turns a no-op into a fleet-wide red graph.


Root-context assumptions. Scripts running as root do not have the console user's $HOME, their defaults domain, or their keychain. Ask specifically about user context and you get a genuinely useful answer.


Explaining itself. This matters more than people admit on a small team. A junior admin who gets a paragraph on why set -e doesn't do what they think learns something; a linter code alone does not.


Where it misses, and where it invents

The honest part. A 2026 study tested five models — including GPT-4o and Claude — on whether code conforms to a stated requirement, without running it, across 1,400+ paired correct and buggy implementations. Two findings should shape how you use this:


First, detailed prompts made things worse. GPT-4o's false-negative rate on one benchmark — rejecting correct code — rose from 35.9% with a minimal prompt to 87.9% with an elaborate one. The researchers call it systematic overcorrection: asked to review carefully, models find problems that aren't there.


Second, noticing is not diagnosing. Models spotted that something was wrong 98.2–100% of the time, but correctly identified the type of bug only 59.1–70.8% of the time. So treat the flag as a real signal and the explanation as a hypothesis. When a model tells you line 34 is broken and gives you a confident reason, the line probably deserves a look — the reason is a coin flip.


The third gap is the one no benchmark measures: it does not know your environment. It does not know which binaries are on your image, which OS versions are actually in your fleet, or what your configuration profile keys are called. It will invent a defaults key or a command-line flag that reads perfectly and does not exist. Every flag, key and path in AI-suggested fleet code gets verified against vendor documentation or man on a real Mac before it ships. No exceptions.


And if you are reaching for Claude Code's built-in /security-review — it is a good tool and it also runs as a GitHub Action on pull requests, but look at what it hunts: SQL injection, XSS, authentication and authorization flaws, insecure data handling, dependency vulnerabilities. That catalogue is web-application shaped. It is not looking for an unguarded rm -rf running as root on a laptop. Anthropic says plainly that automated review should complement rather than replace existing practice, and for shell scripts that caveat is doing real work.



ShellCheck

AI review

Test Mac + pilot ring

Catches

Quoting, word splitting, unhandled cd, unguarded rm

Intent vs. implementation, idempotency, exit-code logic, root-context assumptions

Anything that actually breaks on real hardware

Misses

Anything semantic — valid shell that does the wrong thing

Your environment: profile keys, image contents, OS spread

Rare paths, OS versions and machines unlike your test unit

False alarms

Rare, and each one has an SC code and a wiki page

Common, and worse the more elaborate your prompt

None, but slow

Cost to run

Under a second, free, scriptable in CI

A minute and a few thousand tokens

Hours to days

Safe to skip?

No

Not for anything running as root

Never for a fleet-wide push


The checklist that survives contact with production

Keep the review prompt short — that is the study's finding, not a style preference — and make it answer specific questions rather than render a verdict. This is the one I use:


Review this macOS shell script. It is deployed by Intune and runs
as root on managed Macs. For each finding, quote the line and say
what input or state makes it fail. Do not rewrite the script.

1. What happens if it runs a second time, or after a reboot?
2. Which paths are destructive, and what is each one guarded by?
3. What does each exit code mean to a caller that treats
   non-zero as failure?
4. Where does it assume the console user's context while
   running as root?
5. Which commands, flags or preference keys should I verify
   against documentation before trusting?

Question five is the important one. Asking the model to flag its own unverifiable claims is far more reliable than asking it not to make them.


Then the human pass, which does not delegate: does the script need root at all; are binaries called by absolute path; are there secrets embedded in something distributed to every endpoint; does it log somewhere you can retrieve — Intune collects diagnostics from /Library/Logs/Microsoft/Intune, so writing there means a failure is recoverable without asking a user to run commands; and does it finish well inside 60 minutes on the slowest machine you support, not the newest one on your desk.


What changes at fleet scale

At one machine, a bad script is an afternoon. At 10,000, three things change. The blast radius is instant and simultaneous — Intune requires devices to be connected directly to the internet for scripts to run, so there is no slow trickle to catch it in. The variance is enormous: your test Mac represents a fraction of the OS versions, disk layouts, and half-migrated user profiles actually out there, and the LLM's confident review was based on the same narrow picture you had. And the review layers stop being optional process and start being the only thing between a typo and a Sev1 — which is the honest reason to make ShellCheck a required CI check rather than a habit, because habits don't survive a Friday afternoon hotfix.


The practical takeaway

Use all three layers, in this order, and do not let the AI step absorb the others. ShellCheck first — automated, blocking, non-negotiable, and keep your shebangs in bash or sh so it can actually run. AI review second, with a deliberately short prompt, treating flagged lines as signal and stated causes as hypotheses, and verifying every flag and key it hands you. Human review third for the questions a model cannot answer, followed by a test Mac and a small pilot ring for anything that touches system state.


If you only add one thing this quarter, add the linter to CI — it is deterministic and free, and it catches the specific class of bug that ends careers. The AI layer is the one that catches the subtler mistakes, and it is worth the minute it takes, but it is a second reader with confident opinions and no knowledge of your fleet. Treat it exactly that way.



Comments

bottom of page