Polygon
Siphon
A high-performance data loss prevention scanner, written in Rust.
Sensitive data does not sit politely in the places a policy expects. It is inside the attachment, inside the archive inside the attachment, inside the QR code on the scanned form. Siphon is built to find it there — and to be trusted when it says it found nothing.
126 categories
structural, not guessed
extracted by default
on the labelled corpus
A scanner nobody trusts is a scanner nobody runs
Most data loss prevention tooling fails in the same way. It matches a shape — sixteen digits, a dash pattern, a keyword — and reports everything that matches. The result is a queue of findings that is 95% noise, so the queue stops being read, and the one real finding in it goes out the door with the rest.
Siphon treats a detection as a claim that has to survive a test. Every always-run pattern is either checksum-validated or gated behind keyword proximity, so a number that merely looks like a card number does not become a ticket for somebody to close.
Two layers, then a test
561 patterns, 126 categories
Credit cards, national IDs across 50+ regions, secrets and API keys, banking identifiers, healthcare codes, crypto addresses, classification labels, device and biometric IDs. Each pattern carries a specificity score that sets its base confidence.
5,000+ context keywords, six languages
A single Aho-Corasick pass confirms detections by proximity in English, French, Spanish, German, Italian and Portuguese. Low-specificity patterns are suppressed entirely without a nearby keyword — they never fire on their own.
72 structural validators
Luhn, mod-97, Verhoeff, Base58Check, Bech32 polymod, ISO 3779 and the algorithm specific to each national ID. A match that fails its own check digit is not a finding.
Built for data that is trying not to be found
Exfiltration is rarely plaintext. A ten-stage normalisation pipeline undoes percent-encoding, HTML entities, zero-width injection, homoglyph substitution and NFKC variants before any pattern runs. Token-level base64, base64url, base32 and hex decoding follows nested encodings up to three layers deep.
Redaction is byte-preserving, so replacing a span cannot corrupt offsets in multi-byte text. Corrupted archives fall back to raw byte scanning rather than being skipped.
High-entropy secrets
Random tokens that match no regex, with three gating modes: near keywords, in assignments, or everywhere.
Encrypted and compressed files
File-level Shannon entropy classifies contents as normal, compressed, or likely encrypted.
Known values, exactly
Exact Data Match compares against registered HMAC-SHA256 hashes in constant time — no timing leak.
Near-duplicate documents
Locality-sensitive hashing flags a document that closely resembles one you have already classified.
What it detects
| Category | Patterns | Examples |
|---|---|---|
| National IDs | 250+ | 50+ regions — SSN, SIN, Aadhaar, NIN, DNI, CPF, CURP |
| Classification labels | 40+ | Top Secret, Confidential, HIPAA, GDPR, Attorney–Client |
| Banking & financial | 30+ | IBAN, SWIFT/BIC, ABA routing, wire transfers, securities |
| Secrets & credentials | 20+ | JWT, AWS keys, GitHub tokens, Slack tokens, Stripe keys |
| Healthcare | 10+ | DEA numbers, ICD-10, NDC codes, insurance IDs |
| Geolocation & postal | 8 | GPS coordinates, geohash, ZIP+4, UK postcode |
| Credit cards | 7 | Visa, MasterCard, Amex, Discover, JCB, Diners, UnionPay — enriched from 374,788 issuer BINs |
| Cryptocurrency | 7 | Bitcoin, Ethereum, Litecoin, Monero, Ripple, Bitcoin Cash |
| Device & biometric | 7 | IMEI, ICCID, IDFA, biometric hashes |
| Contact information | 5 | Email, phone (E.164/US/UK), IPv4/IPv6, MAC address |
Twenty-plus formats, on by default
PDF, DOCX, XLSX, ODS and ODT. ZIP, RAR, 7z and CAB, recursively. Email as EML, MBOX and Outlook MSG. Parquet and SQLite. Calendar files and web archives.
Images are decoded for embedded QR codes, Data Matrix, Aztec, PDF417 and ten one-dimensional barcode formats — the boarding pass, the scanned intake form, the screenshot of a credential. Whatever the code contains is then scanned like any other text.
Cryptographic material — .der, .p12, .pfx, .ppk, .jks, .gpg — is blocked by default, with double-extension and symlink bypasses closed.
Five actions, one API
Flag
Report findings, change nothing.
Redact
Replace in place, preserving byte length.
Obfuscate
Substitute realistic fake data — Luhn-valid cards — so downstream systems still work.
Tokenize
Reversible tokens held in a vault with role-based access and a one-hour TTL.
Reject
Refuse the input outright.
Bundled as presets — PCI-DSS, PII, credentials, healthcare, contact info — so an integration is a few lines rather than a policy project.
Fast enough to leave on
| Scenario | Throughput |
|---|---|
| Clean text | ~82 MB/s |
| Keyword-heavy text | ~29 MB/s |
| Mixed content | ~22 MB/s |
| Dense sensitive data | ~15 MB/s |
Running all 561 patterns costs almost nothing over a 100-pattern baseline. The Aho-Corasick prefilter drops every context-gated pattern whose keywords are absent before its regex runs, so a keyword-free page is scanned at baseline speed.
The dense-data figure is the honest one: it is the cost of running a checksum on every candidate. That is the trade we chose, and it is what near-zero false positives is made of.
Median of four runs at 1 MB, default release build. 510+ tests across library, integration, evasion and detection-quality suites, with an enforced labelled-corpus regression gate at 80/80 recall.
One engine, however you need to reach it
A shared scanner engine with a family of pods around it. Detection logic is identical everywhere; pods differ only in how data arrives and what is connected to the output.
siphon-core
The engine, no I/O. Embed it directly in an application to scan text before it is stored or sent.
siphon-api
HTTP text scanning with rate limiting per key, RBAC, runtime key rotation, and endpoints for patterns, policies, metrics and audit.
siphon-fs
File uploads through the extractor registry — the whole format surface behind one multipart endpoint.
siphon
Scan a file, a directory, or a pipe. JSON output for pre-commit hooks and CI.
Hardened, and auditable about it
Tamper-evident audit
Every event HMAC-SHA256 signed and verifiable, with source IP, request ID and outcome; rotating handler with symlink protection and 0600 permissions.
Compliance reporting
PCI-DSS, HIPAA, SOC 2 and GDPR reports generated from actual scan results, in JSON, text or HTML.
Policy engine
TOML rules matched by priority, with named masking profiles and an allowlist for known false positives.
Egress that does not leak
SSRF protection validating every resolved address, HTTPS enforced for SIEM adapters, retry with backoff to Splunk, Elasticsearch, Syslog and Datadog.
Tenant isolation
Namespaced cache keys prevent cross-tenant poisoning; token vaults expire with panic-safe eviction.
Beta
The engine, CLI, API and file service are in beta and running today; the admin console and detector pods are in progress. Interfaces may still change between beta releases. If your programme has a data-discovery problem that current tooling has not solved, we would rather talk about the problem than the roadmap.
LANGUAGE Rust
LICENCE MIT
STATUS Beta
SURFACE Library · CLI · HTTP services
BINDINGS Python, optional
OBSERVABILITY Prometheus metrics, on by default