The explosion of Retrieval-Augmented Generation (RAG) in enterprise B2B SaaS has introduced a new layer to the standard software architecture diagram: the Vector Database.
Whether you are evaluating AI tools for customer support, internal IT helpdesks, or automating your security questionnaires (like VeriRFP), the vendor's RAG architecture almost certainly relies on generating vector embeddings of your private data.
This introduces a novel and often misunderstood security threat. Many AI applications—in a rush to launch—have architected their databases using co-mingled multi-tenancy. For highly sensitive compliance documents (SOC 2 reports, disaster recovery plans, penetration tests), this is an unacceptable risk.
If you are a CISO evaluating an AI platform that ingests your security data, you must ask: "Is our vector index logically or physically isolated from other tenants?"
This question is not optional. It is the single most consequential architectural question you can ask a vendor that handles your compliance data with AI. The answer determines whether a subtle software defect could silently expose your most guarded technical documentation to a competitor, a regulator, or worse.
What is a Vector Embedding?
When you upload a 50-page SOC 2-aligned report, an AI application does not store it as a PDF. It chunks the document into smaller semantic pieces (paragraphs or sections) and passes them through an embedding model. The model converts the text into a significant array of numbers (a vector) that mathematically represents the "meaning" of the text.
To be more precise, a modern embedding model like OpenAI's text-embedding-3-large produces a vector of 3,072 floating-point numbers for each chunk. These numbers encode semantic relationships learned during pre-training on billions of text samples. Two chunks that discuss similar topics—say, "incident response procedures" and "breach notification workflows"—will produce vectors that are geometrically close to each other in that high-dimensional space, even if the exact wording is entirely different.
These vectors are stored in a specialized database (like Pinecone, Milvus, or pgvector). When a Sales Engineer asks a question ("How do we handle data localization in Europe?"), the query is also converted into a vector. The database mathematically searches for the closest matching vectors in the database and returns them to the LLM to draft the answer.
The critical detail that many vendors gloss over is the search mechanism itself. Vector similarity search uses algorithms like Approximate Nearest Neighbor (ANN) to find the closest vectors to the query vector. These algorithms—HNSW graphs, IVF indices, product quantization—are optimized for speed and recall. They are not inherently tenant-aware. The search traverses whatever index structure it has been given, and unless isolation is enforced at the index level, it will traverse data belonging to every tenant in the system.
The Threat of Co-Mingling
In a poorly architected, fully multi-tenant SaaS application, every customer's vectors are dumped into a single, significant index.
Company A's ISO 27001 embeddings are mixed in the exact same index as Company B's highly confidential disaster recovery plan. The application relies entirely on application-layer filtering rules (e.g., where tenant_id = 123) to ensure Company A only sees its own data.
This pattern is alarmingly common because it is the simplest to implement. A single shared Pinecone index or a single pgvector table with a tenant_id column requires the least operational overhead. Developers can ship a prototype in days. But production security is not a prototype concern, and the consequences of this shortcut compound over time.
Consider the scale of the problem. A typical enterprise compliance automation platform might ingest thousands of documents per tenant—SOC 2 Type II reports, ISO 27001 Statements of Applicability, HIPAA risk assessments, penetration test executive summaries, business continuity plans, vendor risk assessment questionnaires, and internal security policies. Multiply that by hundreds of enterprise tenants, and you have millions of highly sensitive vectors co-mingled in a single searchable index. Every one of those vectors is a potential cross-tenant exposure waiting for a single filter failure.
The Attack Vector: Semantic Bleed
If a software bug accidentally drops the tenant_id filter during a vector search query, the consequences are disastrous.
If Company B queries its vector index for "Show me the penetration test summary," and the filter drops, the vector database will simply return the most semantically similar vectors it finds across the entire database. It may return Company A's highly sensitive, unredacted penetration test results directly to Company B's Sales Engineer.
Because vectors are mathematical representations of semantic meaning, they are incredibly efficient at surfacing related concepts—even concepts they weren't supposed to find.
This is not a theoretical risk. Filter-bypass vulnerabilities are among the most common application-level defects in web software. OWASP has tracked Broken Access Control as the number-one web application security risk for years. A dropped WHERE clause in a SQL query is a well-understood vulnerability class. The difference with vector databases is that the leaked data is not a row in a table—it is the distilled semantic meaning of your most confidential security documentation, surfaced with mathematical precision.
The failure mode is also silent. Unlike a traditional SQL injection that might return obviously foreign data or trigger an error log, a vector search that bleeds across tenants simply returns plausible-looking results. Company B's user sees a coherent penetration test summary. They may not even realize it belongs to Company A. There is no schema mismatch, no foreign key violation, no stack trace. The data looks right because it is semantically relevant—it is just the wrong company's data.
Beyond Bugs: Adversarial Exploitation
The risk extends beyond accidental filter failures. A motivated attacker with a valid tenant account could deliberately craft queries designed to probe the boundaries of the vector index. By issuing carefully constructed semantic queries and analyzing the returned vectors' metadata patterns, an attacker could infer whether co-mingled data from other tenants is present in the index.
In advanced scenarios, prompt injection attacks targeting the RAG pipeline can manipulate how the system constructs its vector search queries. If an attacker can influence the query construction logic—for example, by injecting instructions into a document that gets chunked and embedded—they may be able to weaken or remove tenant-scoped filters entirely.
These attack surfaces do not exist in architectures with physically isolated indices. If Company B's search can only traverse Company B's index, no amount of query manipulation can surface Company A's data.
Architecting for True Isolation
When dealing with the most sensitive compliance data in the world, relying solely on an application-layer where clause in a shared vector index is insufficient for true enterprise security.
At VeriRFP, we architected our ingestion pipeline specifically to mitigate the risk of semantic bleed.
1. Tenant-Isolated Vector Indexing
The most secure RAG architecture mandates perfectly isolated vector indices. Company A's SOC 2 embeddings must never physically or logically reside in the same searchable namespace as Company B's data.
Whether utilizing Namespaces in modern vector databases (which physically bifurcate the search space) or entirely separate schemas/collections per tenant, the architecture must guarantee that even if an application-layer filter fails catastrophically, a cross-tenant vector search is physically impossible at the database layer.
In practice, this means each tenant's vectors are stored in a dedicated namespace or collection that the database engine treats as an entirely separate search space. When the ANN algorithm traverses its index graph to find nearest neighbors, it never encounters nodes belonging to another tenant. The isolation is structural, not conditional. There is no filter to forget, no WHERE clause to drop, no metadata tag to misconfigure. The search boundary is defined by the index itself.
This approach does introduce operational complexity. Provisioning a new namespace per tenant, managing index lifecycle, and monitoring per-tenant search performance all require more engineering effort than a single shared index. That trade-off is the price of genuine security, and for any platform handling SOC 2 reports, penetration test findings, or disaster recovery procedures, it is a trade-off worth making.
2. Ephemeral RAG Processing
Once the vectors are securely retrieved, they are passed to the drafting LLM using strict zero-retention enterprise APIs. The LLM reads the isolated vectors, drafts the response for the security questionnaire, and immediately discards the data.
Zero-retention means exactly what it says: the LLM provider does not log, store, cache, or use the input or output data for model training or any other purpose. This is a contractual and technical guarantee, not merely a policy statement. Enterprise-grade LLM APIs from providers like OpenAI and Anthropic offer explicit zero-retention agreements backed by SOC 2 Type II certifications of their own infrastructure.
The ephemeral processing model also applies to intermediate artifacts. Chunk text retrieved from the vector database should not be written to application logs, persisted in a cache layer, or stored in a message queue beyond the lifetime of the request. Every intermediate representation of the customer's data should be treated as transient and purged at the conclusion of the API call.
3. Encryption at Rest and in Transit
Isolation and ephemerality address the most critical threat vectors, but a defense-in-depth strategy requires encryption at every layer. Vectors stored in the tenant-isolated index must be encrypted at rest using AES-256 or equivalent, with keys managed through a proper key management service. Data in transit between the application, the vector database, and the LLM API must be encrypted with TLS 1.3.
For the most security-conscious organizations, customer-managed encryption keys (CMEK) provide an additional layer of control. With CMEK, the tenant holds the master key, and the vendor cannot decrypt the stored vectors without the tenant's explicit authorization. If the relationship ends, the tenant revokes the key, and the stored embeddings become cryptographically inaccessible.
What to Ask Your Vendor
The vendor risk assessment workflow is fundamentally different from a standard SaaS form. You are asking AI to analyze your company's deepest technical secrets. Before signing a contract with any AI platform that will ingest your compliance documentation, demand clear answers to these questions:
Vector isolation model. Are tenant embeddings stored in physically separate indices, logically separate namespaces, or a single shared index with metadata filtering? Only the first two are acceptable for sensitive compliance data.
Filter enforcement. If the vendor uses namespace-level isolation, how is namespace selection enforced? Is it derived from an authenticated session token, or is it passed as a user-controllable parameter? The former is secure; the latter is vulnerable to parameter tampering.
LLM data retention. Does the LLM provider retain any input or output data? Demand a copy of the zero-retention agreement and verify it covers both training exclusion and log purging.
Audit logging. Can you receive tenant-scoped audit logs showing every vector search query executed against your namespace? Without audit trails, you cannot verify isolation is working as advertised.
Penetration testing. Has the vendor conducted a third-party penetration test specifically targeting cross-tenant data access in the RAG pipeline? Ask for the executive summary.
As a security leader, you cannot afford to have your penetration test results floating in a co-mingled sea of multi-tenant embeddings. Demand vector isolation. Demand zero-retention. Evaluate the VeriRFP architecture today.
Related resources
- Security Overview - review the platform's security posture
- Product Overview - see the end-to-end architecture
- SOC 2 Aligned Automation - connect controls to questionnaire workflows