Removing DeWitt clauses promotes transparency in the database industry, allowing engineers to make data-driven decisions based on independent, reproducible performance comparisons rather than vendor-controlled marketing materials.
Database benchmarks are imperfect. They are also useful.
No benchmark can tell you exactly how a database will perform for your application. Workload shape, data size, region placement, storage, configuration, and cost all matter. But fair benchmarks help customers understand tradeoffs, compare options, and ask better questions before choosing infrastructure.
Many cloud vendors include language in their terms that restricts comparative benchmarking. These restrictions are called "DeWitt clauses", named after database researcher David DeWitt. That is a strange legacy for someone whose work helped move the database industry forward by measuring real systems and publishing results.
Previously, PlanetScale also included a DeWitt clause in our Acceptable Use Policy (AUP). Today, we are removing this in favor of a more open "Benchmarking" section in our AUP.
The new section reads:
You may perform benchmark tests (“Benchmark”) of the Services, provided that the Benchmark is conducted in good faith and uses a fair and transparent methodology. Please refer to PlanetScale's published benchmarking best practices. Except with respect to Beta Features, you may disclose the results of the Benchmark. If you perform or disclose, or direct or permit any third party to perform or disclose, any Benchmark of the Services, you (i) will include in such disclosure, and will disclose to PlanetScale, all information necessary to replicate such Benchmark, and (ii) agree that PlanetScale may perform and disclose the results of Benchmarks of your products or services, irrespective of any restrictions on Benchmarks in the terms governing your products and services.
Any Benchmark must be conducted in accordance with the Agreement, including this Acceptable Use Policy. The Benchmark must not interfere with the Services or misrepresent the configuration, methodology, results, or cost of the Services or any compared service.
Benchmarks have gained a bad reputation because they are frequently conducted poorly. Sometimes this is done with malicious intent, often referred to as "benchmarketing." At other times it is done out of ignorance. Many engineers are not trained in all aspects of fair benchmarking.
Anyone benchmarking PlanetScale should follow the best practices outlined in our benchmarking guide. These practices come from our deep experience benchmarking databases in the cloud where topology, server location, region, workload, and instance type differences materially impact the result.
We encourage other vendors, analysts, and practitioners to use the same standard. Benchmarks should be deep, thorough, technically sound, and transparent enough for others to understand and reproduce.
We invite other vendors to adopt this same language and standard in their own AUPs. Allow public benchmarking, remove DeWitt clauses, and hold benchmarks to clear expectations for fairness and transparency.
Customers should be able to compare the systems they rely on.
Continue reading on the original blog to support the author
Read full articleRemoving restrictive DeWitt clauses allows for honest, reproducible database performance comparisons. This transparency helps engineers make better-informed infrastructure decisions based on real-world workloads rather than marketing claims.
Proper benchmarking is critical for making informed infrastructure decisions. Without rigorous controls for network latency, hardware parity, and workload modeling, results are often biased, leading to poor architectural choices and unexpected production performance issues.
This acquisition secures the future of Drizzle ORM, ensuring long-term maintenance while keeping it open-source. It signals a deeper integration between database platforms and type-safe ORMs, directly benefiting engineers working within the TypeScript and JavaScript ecosystems.
Neki solves the scaling limits of single-instance Postgres without sacrificing compatibility or forcing application-level sharding. It provides a managed path to horizontal scaling with online operations, making it easier to handle massive workloads while keeping standard Postgres tools.
