Curated topic

mlp

Posts tagged with mlp

Netflix Tech BlogOct 14, 2026

Integrating Netflix’s Foundation Model into Personalization applications

Why it matters: This article provides a practical blueprint for integrating large-scale foundation models into production. It highlights the trade-offs between embeddings, subgraphs, and fine-tuning, helping engineers balance performance, latency, and infrastructure costs in ML pipelines.

Netflix centralizes member preference learning using a Foundation Model (FM) to power diverse personalization applications.
The Embeddings approach uses a dedicated Embedding Store and stabilization techniques to manage model drift during daily fine-tuning.
Subgraph integration embeds the FM decoder directly into application models, eliminating data staleness at the cost of higher compute complexity.
Fine-tuning adapts the FM for specific tasks like session intent prediction, utilizing Ray for distributed training and inference.
Engineers must balance the trade-offs between real-time adaptivity, infrastructure costs, and latency when choosing an integration method.
A near-real-time embedding framework was developed to address staleness while maintaining the low-cost benefits of the embedding-based approach.

#mlp #data #dist

Read original

Netflix Tech BlogOct 14, 2026

Measuring the Impact of Personalized Recommendations

Why it matters: This research solves the attribution problem in recommenders by separating organic demand from algorithmic influence. It provides a scalable way to measure personalization ROI and understand how algorithms shape discovery without degrading the user experience for measurement.

Netflix developed a hybrid measurement framework to isolate the causal impact of recommendations from underlying user preferences.
The system utilizes a low-rank choice model to learn latent representations of titles and dynamic member preference states based on viewing history.
A specific recommendation bonus parameter is used to quantify the incremental probability of a title being chosen when promoted.
The model was validated using small-scale A/B nudge tests that exogenously vary title visibility to confirm predicted substitution patterns.
Research findings indicate that personalization significantly increases total viewing time and shifts consumption toward niche content.

#mlp #data

Read original

Netflix Tech BlogOct 14, 2026

The Lifecycle of LLM-as-a-Judge: Building, Aligning, and Monitoring at scale

Why it matters: Scaling LLM applications requires automated evaluation, but judges can drift or hallucinate. This framework ensures LLM judges remain reliable by anchoring them in human judgment through a structured lifecycle, preventing silent failures in production recommendation systems.

The LLM-as-a-Judge framework uses a secondary model to evaluate primary model outputs, acting as both a quality gate and a critic for self-refinement.
A four-phase lifecycle—Birth, Training, Deployment, and Monitoring—ensures the judge remains anchored in human judgment and expert-defined rubrics.
A 'Reflector' component automatically revises evaluation rubrics by analyzing mismatches between human labels and judge rationales.
The 'Meta-Judge' validates the judge's reasoning by comparing its rationales against human-provided explanations to ensure correct decision-making logic.
Continuous human-in-the-loop (HITL) monitoring detects model drift through weekly sampling, triggering retraining when performance falls below thresholds.

#mlp #data

Read original

Netflix Tech BlogOct 14, 2026

Reliable GPU compute for large training & inference jobs

Why it matters: GPU failures are inevitable at scale, especially in distributed training where one bad node halts the entire job. This post details how to build self-healing infrastructure that detects hardware degradation early and automates remediation to maximize compute efficiency and model availability.

Netflix implements a four-layer reliability model covering hardware health, scheduler logic, training loop resilience, and observability.
Custom health checks utilize NVIDIA DCGM and node-problem-detector to identify memory errors, thermal issues, and degraded interconnects every 30 seconds.
Pre-flight checks validate GPU compute and network bandwidth before jobs start to prevent 'slow neighbor' performance degradation in gang-scheduled tasks.
Automated remediation workflows use Kubernetes taints and controllers to drain unhealthy nodes and trigger machine recycling via cloud provider APIs.
Training jobs leverage PyTorch TorchElastic and frequent checkpointing to allow workloads to survive hardware loss without manual intervention.
A dedicated observability layer tracks failure attribution and SLOs to distinguish between infrastructure faults and user-code errors.

#mlp #sre #dist

Read original

GitHub EngineeringOct 13, 2026

GitHub’s model for secret detection: Finding more secrets with less noise

Why it matters: Secret leakage is a major security risk, but high false-positive rates lead to alert fatigue. GitHub's ML approach demonstrates how to use context-aware filtering to make security tools more accurate and less intrusive, improving the overall security posture without slowing down development.

GitHub implemented a machine learning model to augment regex-based secret scanning, specifically targeting the reduction of false positive alerts.
The model evaluates the surrounding code context, such as variable assignments and file types, to distinguish between real credentials and placeholder strings.
By filtering out noise, the system helps prevent alert fatigue among developers while maintaining high recall for actual sensitive data leaks.
The solution addresses the limitation of traditional entropy-based detection, which often flags random strings in tests or documentation as secrets.
This ML-enhanced layer is integrated into GitHub's push protection workflow, blocking potential leaks before they reach the remote repository.

#security #mlp

Read original

Airbnb EngineeringOct 13, 2026

CASTLE: training natural language search before the first real query

Why it matters: CASTLE solves the cold-start problem for search systems by synthesizing realistic training data from existing behavioral signals. This allows teams to deploy high-quality natural language search models before collecting any real user queries, ensuring a strong Day 1 user experience.

CASTLE addresses the cold-start problem for natural language search by jointly generating synthetic queries and relevance labels.
It utilizes contrastive listing pairs from de-identified booking data to ensure relevance is grounded in actual user preferences.
The framework uses seed-guided prompting to align synthetic query distributions with real-world search patterns and length.
A Virtual Judge component extends coverage to long-tail intents and new listings lacking historical booking signals.
Evaluation using KL divergence shows CASTLE significantly outperforms baselines like InPars in matching real-user query statistics.

#mlp #data

Read original

Engineering at MetaOct 13, 2026

Meta’s Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI

Why it matters: Meta demonstrates how to apply LLM-scale architectures and scaling laws to recommendation systems. By unifying disparate models into a single foundation model and optimizing GPU utilization with 5D parallelism, they achieve massive performance gains while maintaining serving efficiency.

Meta transitioned from isolated per-domain ad models to a unified central foundation model (GEM) that learns across all surfaces like Facebook and Instagram.
The new Kunlun architecture applies LLM-style scaling laws to recommendation systems, enabling 2.5x model complexity and long-context learning.
LoopFM and Solaris techniques allow for efficient intelligence transfer from the foundation model to user-facing models without increasing real-time latency.
A unified compute engine enables cross-domain learning, transferring knowledge from data-rich click tasks to data-sparse conversion tasks.
Training efficiency was doubled to 20-25% model FLOPs utilization (MFU) through custom kernels and a novel 5D parallelism strategy.
The system achieved significant business impact, including an 18.9% improvement in conversions and a 12% increase in click-through rates.

#mlp #dist #data

Read original

Engineering at MetaOct 12, 2026

OCP Summit 2026: Powering AI with Open Innovations in Scale-Up, Scale-Out, and Scale-Across Networks

Why it matters: As AI clusters reach gigawatt scale, traditional networking hits thermal and bandwidth limits. Meta’s open-source liquid cooling, 102.4T switches, and standardized protocols provide a blueprint for building the sustainable, high-performance infrastructure required for next-gen LLM training.

Meta introduced IcePack, its first liquid-cooled network rack, and IceCube, a 102.4T switch platform supporting Broadcom and NVIDIA ASICs.
The Flattened Planar Fabric (FPF) was unveiled as a multiplane scale-out fabric designed to optimize GPU connectivity and reduce latency.
MetaRoCE, an open-source RDMA transport protocol, was released to improve efficiency and reliability in AI-specific network traffic.
The Open Switch Platform Architecture (OSPA) initiative was launched to standardize switch platform layers across the industry.
Innovations in 'scale-across' networking include petabit-scale backend aggregation (BAG) and SRv6 regional backbones for multi-site clusters.
Meta's MTIA 300 system now utilizes Tahan, a 51.2T Ethernet-based scale-up switch, with plans for future 102.4T ESUN-based designs.

#dist #mlp #sre

Read original

GitHub EngineeringOct 12, 2026

GitHub Copilot app for Beginners: How to teach Copilot about your codebase

Why it matters: Providing AI with codebase context reduces repetitive prompting and ensures generated code follows team standards. By using custom instructions and MCP servers, engineers can automate complex workflows and integrate external tools directly into their AI-assisted development cycle.

Use .github/copilot-instructions.md to define repository-wide rules for architecture, naming conventions, and testing standards.
Implement path-specific instructions in .github/instructions using an applyTo field for granular guidance on specific file types.
Define agent skills in .github/skills to package repeatable procedures like quality checks, release preparation, or debugging workflows.
Create custom agents in .github/agents for specialized, multi-file tasks such as accessibility reviews or complex refactoring.
Leverage Model Context Protocol (MCP) servers to connect Copilot to external tools and services like Playwright for browser testing.
Combine these tools to reduce manual prompting and ensure AI-generated code adheres to existing project standards and workflows.

#mlp #culture

Read original

Engineering at MetaOct 12, 2026

Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations

Why it matters: Engineers can optimize resource utilization by turning idle, preemptible capacity into value-generating work. Meta's approach shows how to build elastic ML systems that scale dynamically with available hardware while protecting baseline latency and reliability through isolation and load shedding.

Meta developed a system to aggregate idle, fragmented inference capacity into a shared opportunistic pool for ads recommendations.
The system utilizes two elasticity levers: Rank More Ads (RMA) to increase candidate evaluation and Serve Better Models (SBM) for higher-quality inference.
Workloads are designed to be preemptible and elastic, allowing them to expand or contract instantly as capacity is reclaimed by baseline services.
Physical and logical isolation, combined with feedback loops, ensures that opportunistic work never compromises the latency or reliability of dedicated capacity.
Value-aware load shedding at both client and server levels allows for graceful degradation during sudden capacity shifts.

#mlp #dist #finops

Read original

Page 3 of 54

Prev 1 2 3 4 5...54 Next