How to evaluate a vendor's security promises (a buyer's checklist)
Every workspace tool says it takes security seriously. Most of them mean different things by that phrase. Here's a concrete checklist for separating architectural claims from marketing.
Every SaaS vendor's marketing site has a security page. Most of them have the same shape: a lock icon, the words "bank-grade encryption," a list of compliance certifications, and a paragraph about how seriously the company takes your data. The shape is so common that it stops carrying signal.
A buyer trying to evaluate whether a vendor is actually well-architected for confidentiality needs to look past the marketing surface. Here's a checklist that separates structural claims from PR copy. Each question has a useful answer, an acceptable answer, and a red-flag answer.
1. Who holds the encryption keys?
Useful answer: "The user. Keys are generated on the user's device, never transmitted to our servers. We hold ciphertext under keys we can't derive."
Acceptable: "The customer, via a customer-managed Key Service (BYOK / CSE / Customer Key). We never see the cleartext."
Red flag: "Industry-leading encryption." "Bank-grade." "Military-grade." These are evasions. If a vendor can't answer who holds the keys in one sentence, the answer is "we do."
2. What's the architectural ceiling on what you can produce under a legal demand?
Useful answer: "Metadata only — account existence, timestamps, connection IPs. We don't hold content we can decrypt."
Acceptable: "Content for users not using our customer-managed-key offering; metadata only for users who are."
Red flag: "We respond to legal demands per our privacy policy." This describes the policy, not the architecture. A vendor whose answer hinges on policy can be compelled to change the policy. A vendor whose answer hinges on architecture cannot — they don't have the data to produce.
3. Does the vendor publish a security architecture document?
Useful answer: Yes, with specifics — named protocols (Signal Protocol, MLS RFC 9420, nacl.box, AES-256-GCM), key flows, threat model, what's encrypted at which layer.
Acceptable: A high-level summary with enough detail to verify the claims independently.
Red flag: A glossy page with a few certificates and no technical content. If the vendor doesn't publish their architecture, you can't audit it. If you can't audit it, you're trusting the marketing.
4. Has the encryption been audited by an independent third party?
Useful answer: Yes — by a named firm (Cure53, Trail of Bits, NCC Group, Doyensec, Kudelski Security, etc.) with a public report or summary.
Acceptable: Audit is in progress; firm is named; report is committed to a date.
Red flag: "Audited by industry-leading security professionals." If the firm isn't named, the audit might be a casual review, a vendor-side checklist, or marketing.
5. Can you verify the code shipped to me matches the code published?
Useful answer: For native apps — reproducible builds + signature verification. For web — code transparency log (Sigstore / Rekor style) where bundle hashes are public.
Acceptable: Open-source code with documented build instructions; reproducibility on the roadmap.
Red flag: Closed source, no build verification, no transparency log. The vendor could ship different code to one user than to everyone else, and there's no way to tell. For a security-critical product, this is a meaningful gap.
6. Does the vendor publish a transparency report?
Useful answer: Yes, updated annually or more often. Lists legal requests received and broken down by type, the count produced on, and (for E2E vendors) the explicit framing that content cannot be produced.
Acceptable: Yes, even if sparse — the existence indicates the vendor has thought about this and is willing to be audited.
Red flag: No transparency report from a vendor large enough to receive legal requests. Either they're not receiving any (unlikely at scale) or they're choosing not to disclose them (a soft signal).
7. What features require the vendor to read your data?
Useful answer: A short list, each justified — "abuse detection on suspicious accounts uses these signals, AI features run client-side after decryption, no others."
Acceptable: A clear description of the feature surfaces that touch cleartext server-side, with reasoning.
Red flag: "Our AI features improve productivity for your team." If AI grounding requires cleartext access and the vendor doesn't acknowledge that, the vendor's architecture isn't matching its marketing — or the vendor hasn't been honest about the trade-off.
8. What happens if you lose access to the encryption keys?
Useful answer: "Your data is unrecoverable. We don't hold a copy of the key. Recovery routes through Shamir-style multi-device splits, hardware-backed credentials, or recovery codes you printed at setup."
Acceptable: A clear recovery model that doesn't involve the vendor holding the keys.
Red flag: "Click 'forgot password' and we'll restore access via email." If a password reset restores access to encrypted content, the vendor holds the keys. Full stop. Nothing else they say about encryption matters as much as this.
Putting it together
A vendor that gives useful answers to most of these is probably architecturally sound. A vendor that gives acceptable answers to most is probably on a real roadmap. A vendor that gives red-flag answers to most is using security as marketing, not as engineering — independent of how trustworthy the company's intent is, the architecture isn't there.
The right test isn't whether the vendor is trustworthy. The right test is whether the architecture is structurally limited so that trust is bounded. A vendor that holds the keys requires you to trust the vendor. A vendor that doesn't hold the keys requires you to trust the math.