Sign Your Weights: Your Model Is an Executable Your Supply Chain Ignores

Your CI pipeline is a fortress. Container images are signed and verified before deploy. Every npm and PyPI dependency resolves against a lockfile with pinned hashes. Commits require signed tags. And then, somewhere in your model-serving startup script, there's a line that downloads a multi-gigabyte blob from a model hub or an S3 bucket and loads it into memory — no signature check, no hash verification, no record of who produced it. The single artifact that most decides what your product actually does is the one artifact your supply-chain tooling has never heard of.

This isn't a hypothetical gap. Security researchers have pulled hundreds of malicious models off public hubs — models that execute attacker code the moment you load them, models crafted specifically to slip past the scanners the hubs run. The tooling to close the gap now exists: safe serialization formats, an industry signing specification, admission controllers that reject unsigned weights. Most teams just haven't noticed that "model file" belongs in the same mental category as "unaudited binary from the internet."

The Original Sin: A Format That Executes on Load

The reason model files deserve more suspicion than a JPEG comes down to serialization history. PyTorch's default save format was built on Python's pickle, and pickle is not a data format — it's a tiny instruction stream for a virtual machine that can invoke arbitrary Python callables during deserialization. Loading a pickle file is running a program someone else wrote. For years, torch.load() on a downloaded checkpoint meant exactly that.

Attackers understood this long before most ML teams did. In early 2024, JFrog researchers found more than a hundred models on Hugging Face capable of executing code on load, including one that opened a reverse shell back to an attacker-controlled host — a silent backdoor delivered as a "pretrained model." A year later, ReversingLabs documented the technique they dubbed nullifAI: malicious pickle payloads compressed in 7z instead of the expected ZIP format, which broke the hub's PickleScan detection while still being loadable by a slightly-too-helpful consumer. Protect AI, scanning millions of hub artifacts, flagged hundreds of thousands of unsafe or suspicious issues across tens of thousands of models.

The scanning arms race has gone poorly for the scanners. Through 2025, JFrog and Sonatype disclosed seven distinct bypass vulnerabilities in PickleScan alone — broken pickle streams that scanners reject but loaders accept, archive-flag manipulation, subclass tricks that evade blacklists. That pattern should sound familiar to anyone who watched antivirus lose to malware packers in the 2000s. Blacklist-based scanning of a Turing-complete format is a treadmill, not a defense.

The structural fix is to stop using executable formats entirely. Safetensors — now under the PyTorch Foundation — stores tensors as plain data with a JSON header: no opcodes, no callables, nothing to execute. ONNX and GGUF are similarly inert. A "safetensors-only" policy converts a code-execution risk into a mere data-integrity risk. That's real progress, but notice what it doesn't buy you: an attacker who can write to your model bucket no longer gets code execution at load time, but they can still swap in weights that behave differently. Which brings us to the part nobody threat-modeled.

Tampered Weights Are Worse Than Tampered Code

Here's the uncomfortable asymmetry. If an attacker modifies your application binary, you have decades of tooling that will notice: reproducible builds, binary diffing, EDR, code review on every change. If an attacker modifies your model weights, what exactly would catch it? The file is billions of opaque floating-point numbers. There is no code review for a tensor. A fine-tune that inserts a targeted backdoor — respond normally on everything except inputs containing a trigger phrase — is invisible to every eval you run, because you don't know what to eval for.

Walk through the actual attack surface:

• The registry bucket with wide write access. Most companies store fine-tuned weights in object storage. Audit who has write permission to that bucket and you'll usually find the answer is "every data scientist, several CI service accounts, and a training job that runs with admin credentials." Any one of those credentials leaking means silent weight replacement.
• The poisoned upstream checkpoint. You fine-tune from a base model pulled from a hub. Namespace squatting, lookalike organization names, and hijacked maintainer accounts have all been used to serve poisoned checkpoints. Your fine-tune inherits whatever the base contains, and your provenance trail starts from a lie.
• The compromised training pipeline. Training jobs pull dozens of Python dependencies. A malicious transitive dependency doesn't need to attack your infrastructure — it can just nudge the weights it's helping to produce while leaving the training loss curve looking perfectly normal.
• The internal re-host. Teams mirror external models into internal registries "for reliability," typically with a copy script that verifies nothing. The mirror becomes an unauditable fork: nobody can say whether the internal copy still matches upstream, because nobody recorded what upstream's bytes were.

The common thread: in every case the question you can't answer is "are the bytes we're serving the bytes we intended to serve, and who produced them?" That's not an ML question. It's the exact question software supply-chain security spent the last decade building machinery to answer — signatures, provenance attestations, transparency logs. The machinery just never got pointed at model files.

The Emerging Discipline: OMS, Sigstore, and Signed Weights

That's changing. In April 2025, the OpenSSF's model-signing project hit 1.0, and with it came the OpenSSF Model Signing (OMS) specification — a joint effort by Google, NVIDIA, and HiddenLayer to define an industry standard for cryptographically signing model artifacts. The design decisions are worth understanding because they map cleanly to how models actually differ from containers.

Loading…