Curated topic

data

Posts tagged with data

Cloudflare BlogSep 30, 2026

Simplifying domains for people and agents

Why it matters: This update enables programmatic domain management, allowing AI agents and developers to automate infrastructure setup. It demonstrates a high-performance serverless architecture that balances speed and data freshness across hundreds of external third-party registries.

Cloudflare Registrar now supports programmatic domain management via a new Registrar API, MCP integration, and the cf CLI.
A new sandbox environment allows developers to test registration and transfer workflows without incurring real-world costs or transactions.
The search engine was rebuilt to query 420+ extensions simultaneously using a multi-tiered approach: cached datasets, DNS lookups, and live registry checks.
The architecture leverages Cloudflare Workers for entry points, Durable Objects for search coordination, and Workers KV for high-speed data retrieval.
The system uses a progressive probing strategy to balance speed and accuracy, updating results dynamically as authoritative registry data arrives.
Support for Model Context Protocol (MCP) enables AI agents to search for, buy, and transfer domains through natural language interfaces.

#dist #data #sre

Read original

Airbnb EngineeringSep 29, 2026

Personalization without user identity

Why it matters: This approach enables effective personalization for anonymous users while navigating strict privacy laws and cookie deprecation. It offers a scalable blueprint for solving cold-start ML problems by leveraging geographic correlation instead of individual tracking.

Airbnb introduced Proximity Features to address the cold-start problem for logged-out and new users without relying on individual history.
The system groups users into buckets of approximately 1,000 based on geographic proximity derived from IP addresses, ensuring no persistent user identifier is required.
A two-phase adaptive clustering algorithm manages varying geographic densities, using fine-grained tiles in cities and coarser groupings in rural areas.
Features are computed daily across three categories: short-term engagement, long-term booking patterns, and aggregate bucket metadata.
This privacy-preserving approach complies with GDPR and mitigates the impact of third-party cookie deprecation while maintaining personalization quality.

#mlp #data #security

Read original

Cloudflare BlogSep 29, 2026

Introducing Threat Signals: agentic skills for open-source threat intelligence, free for every Cloudflare account

Why it matters: This automates the manual process of converting unstructured security blogs into actionable firewall rules. By using agentic AI to extract context and IOCs at scale, engineers can respond to emerging threats faster without manual data entry or losing critical research context.

Cloudflare launched Threat Signals, an agentic workflow tool that automates the ingestion and analysis of unstructured open-source threat intelligence.
The system uses AI skills to summarize reports, extract Indicators of Compromise (IOCs), and normalize data into a private, account-scoped dataset.
Built on Cloudflare Workflows, Browser Run, and R2, the platform converts RSS feeds into actionable security data.
Extracted indicators are stored as Threat Events, enabling users to instantly create WAF rules to block malicious traffic.
The platform addresses scalability limits of traditional RSS polling and the human bottleneck of manual report processing.
Cloudforce One’s Threat Events Platform is now free for all accounts, providing API and dashboard access to investigate private datasets.

#security #data #mlp

Read original

Cloudflare BlogSep 28, 2026

How fast is the web? Explore billions of real-user measurements with BEACON

Why it matters: BEACON provides engineers with massive-scale, real-world performance data to move beyond "works on my machine." By analyzing Core Web Vitals across diverse devices and regions, developers can make data-driven optimizations that improve the actual user experience globally.

Cloudflare released BEACON, an anonymized dataset of billions of real-user performance measurements from 10,000 major websites.
The dataset is hosted on Google BigQuery and follows RUM Archive standards, providing full histograms for Core Web Vitals like LCP, CLS, and INP.
Granular sub-part metrics reveal that resource discovery and render blocking often impact LCP more than raw download speeds.
Analysis shows significant performance disparities across browser engines, with WebKit performance lagging behind Blink in several regions.
BEACON data highlights the architectural trade-offs of Single Page Applications, which often sacrifice initial load speed for faster soft navigations.
The dataset includes industry-specific classifications, allowing researchers to compare performance benchmarks across sectors like Health, Ads, and Government.

#data #frontend #mobile

Read original

Cloudflare BlogSep 28, 2026

EmDash 1.0: the stable CMS with a secure plugin registry

Why it matters: EmDash 1.0 offers a stable, Astro-based CMS alternative with native AI agent support and high-scale infrastructure compatibility. It solves the performance and maintenance burdens of legacy systems while providing a decentralized plugin ecosystem and permissive open-source licensing.

EmDash 1.0 is a stable, open-source CMS built on Astro for production-grade websites.
Includes a decentralized plugin registry that maintains developer ownership without a central marketplace.
Supports AI agents via an MCP server, CLI, and API for automated management and migrations.
Optimized for performance using KV object caching, Hyperdrive, and Cloudflare Workers Cache.
Production-tested on the Cloudflare Blog, handling 5,000 RPS and millions of weekly pageviews.
Released under the MIT license with a global community of over 175 contributors.

#frontend #data #security

Read original

PlanetScale Tech BlogSep 28, 2026

Handling hot shards

Why it matters: Sharding strategies that work at launch often fail under scale. Understanding how to migrate from tenant-based sharding to more granular keys is crucial for maintaining performance and enabling features like cross-tenant communication without incurring application downtime.

Initial sharding by tenant ID often leads to hot shards when specific tenants grow disproportionately or require cross-tenant features.
Neki and Vitess use explicit, declarative data topologies rather than opaque automatic sharding, making data placement predictable and manageable.
Vertical scaling of individual shards provides a temporary fix for resource-heavy tenants without affecting the entire cluster's configuration.
Large whale tenants can be isolated into specific key ranges to prevent noisy neighbor issues and optimize resource allocation.
Long-term scalability often requires changing the shard key, such as moving from workspace-based to channel-based sharding for message data.
Neki supports online resharding by copying rows and streaming writes, enabling traffic cutover without application downtime.

#dist #data #sre

Read original

Netflix Tech BlogSep 25, 2026

Trading a Cloud Identity for Your Own: Workload Attestation on Managed Compute

Why it matters: This approach allows managed cloud workloads to securely acquire internal identities without manual secret injection. It ensures consistent access control and auditability across heterogeneous environments by bridging cloud-native IAM with private PKI systems.

Netflix bridges AWS IAM roles with their internal Metatron PKI using a 1:1 mapping between Data Project identities and dedicated IAM roles.
The system uses a control plane to sign workload metadata payloads, ensuring only authorized services can assert a workload's legitimacy.
Workloads prove identity by generating a pre-signed AWS STS URL, which serves as a verifiable proof of possession for the assigned IAM role.
The internal Identity service corroborates the signed metadata against the STS response to issue short-lived X.509 certificates for mutual TLS.
To scale beyond IAM role limits, Netflix shards Data Project roles across multiple dedicated AWS accounts.
A Spark plugin handles the attestation logic during driver and executor bootstrap, making the process transparent to the end-user.

#security #data #sre

Read original

Pinterest EngineeringSep 25, 2026

Partition Finalization in Pinterest’s Next-Generation DB Ingestion Framework

Why it matters: Determining when a streaming dataset is ready for batch processing is difficult. This metadata-driven approach signals data completeness based on event-time watermarks, reducing processing latency while ensuring data integrity against late-arriving records.

Pinterest developed a partition finalization mechanism for its stream-based DB ingestion framework to signal when data is complete for downstream batch processing.
The system extracts event-time statistics from CDC records and stores them as metadata within Apache Iceberg snapshot summaries during Flink checkpoints.
A centralized Watermark Generator aggregates these statistics to calculate a global watermark, accounting for late-arriving data via configurable safety buffers.
Finalization markers are emitted to a dedicated Iceberg table or as Hive _SUCCESS files once the watermark passes a partition's end time.
This approach decouples producers from consumers, allowing Airflow sensors to trigger downstream jobs based on data completeness rather than fixed schedules.
The framework uses t-digest sketches to maintain accurate event-time distributions without the overhead of storing every individual timestamp.

#data #dist

Read original

PlanetScale Tech BlogSep 25, 2026

When to choose x86-64 vs aarch64

Why it matters: Choosing the right CPU architecture impacts database performance and cost. ARM offers dedicated physical cores per vCPU, potentially providing better price-performance and more predictable scaling compared to hyperthreaded x86-64 instances.

Cloud providers define vCPUs differently; x86-64 usually employs hyperthreading while ARM (aarch64) utilizes dedicated physical cores.
In x86-64 environments, two vCPUs share a single core's resources, which can cause performance bottlenecks during high-load database operations.
ARM-based chips like AWS Graviton and Google Axion assign one full core per vCPU, offering more predictable performance.
ARM's architecture prioritizes power efficiency, leading to lower operational costs for cloud providers and better price-performance for users.
Architecture selection is a high-stakes decision for databases like Postgres, as migrating between x86 and ARM later is complex.

#data #finops #sre

Read original

PlanetScale Tech BlogSep 23, 2026

TIN Postgres search is faster, better, and cheaper

Why it matters: TIN demonstrates that optimizing Postgres full-text search can eliminate the need for external search engines. By providing sub-millisecond exact counts and supporting fuzzy matching without debouncing, it enables a superior UX while reducing architectural complexity and cost.

TIN (The Indexing Network) for Postgres enables high-performance full-text search that eliminates the need for input debouncing.
The engine supports advanced features including wildcard-based autocomplete and Levenshtein-based fuzzy matching for typo tolerance.
Unlike traditional search indexes, TIN indexes stop words, allowing for precise queries on common English phrases like 'to be or not to be'.
TIN provides exact result counts in milliseconds, even for complex multi-term queries, outperforming competitors that rely on slow estimates.
The performance efficiency allows for immediate search-as-you-type experiences with sub-15ms ranking for millions of documents.

#data #sre

Read original

Page 7 of 46

Prev 1...5 6 7 8 9...46 Next