“We Support CSV” and Other Comfortable Lies
A spreadsheet is not a database. Why accepting a simple CSV export is a failure of duty that leaves the municipality with no audit trail and no legal defense.
The Flat Earth Theory of Data
The courtroom scenario is my favorite way to terrify a relaxed procurement officer.
“Imagine,” I say, “that a Citizen sues us over a denied building permit from three years ago. The judge asks for the record. We have since switched vendors.”
I slide a printed spreadsheet across the table. “This is what the old vendor gave us. It says ‘Status: Denied.’ It does not say who denied it. It does not show the timestamp. It does not include the architect’s drawings that were attached to the file.”
“The judge,” I continue, “will ask for the evidence. And you will tell him: ‘We left it in the cloud.’”
The Threat: The Decapitation of Context
A CSV (Comma Separated Values) export is the bare minimum of data transfer. It is suitable for a contact list. It is unacceptable for a government.
Data is not just text in a cell. Data is a web of relationships:
- The Audit Log: Who changed the status from “Pending” to “Approved”?
- The Hierarchy: Which case belongs to which family?
- The Attachments: Where are the scanned proofs of income?
When a vendor offers “CSV Export” as their primary exit path, they are offering to decapitate the data. They give you the body (the text) but keep the head (the context). Without the context, the data loses its legal standing. We cannot defend a decision we cannot reconstruct.
[TO EDITOR: Diagram showing a rich tree of data (Case -> History -> User -> Document) being put through a shredder that outputs a single flat line of text.]
The Treaty: The “Full Relational Dump”
We have updated our “Definition of Export” in all contracts. It is no longer enough to get the data out. We must get the Truth out.
Our clause requires:
- Linked Attachments: The export must include a folder of all uploaded files, and the database export must contain the precise file paths linking record A to document B.
- Immutable History: The export must include the
audit_logtable. We need to know that Officer Hansen viewed the file on November 12th, 2024. - The Schema Map: We require an Entity-Relationship Diagram (ERD) that explains how the tables fit together.
If the vendor claims this is “proprietary business logic,” we politely disagree. The logic is yours; the history belongs to the Public. We will not accept a partial truth.
FAQs
What is missing from a CSV?
The 'Who, When, and Why.' A CSV shows the final state of a permit, but not who approved it, when it was modified, or the original application PDF.
Why do we need the audit trail?
If a decision is challenged in court, we must prove the chain of custody. A flat text file proves nothing.
Is this technically difficult for vendors?
No. They have the database. They just don't want to build the button that gives it to you.