This approach shows how to collapse complex multi-stage recommendation pipelines into a single generative task. It proves that distilled 4B-parameter LLMs can provide production-grade performance for high-scale personalization, significantly improving user intent understanding and engagement.
Lin Zhu | Sr. Staff Machine Learning Engineer; Manan Kalra | Machine Learning Engineer II; Byung Eun Jeon | Sr. Machine Learning Engineer; Ye Liu | Staff Machine Learning Engineer; Xiangyi Chen | Sr. Machine Learning Engineer; Jaewon Yang | Principal Machine Learning Engineer; Jinwen Xu | Manager II, Machine Learning Engineering; Tingting Zhu | Sr. Manager, Engineering; Sudarshan Lamkhede | Director, Machine Learning Engineering
Pinterest is built to get inspired and then turn the inspiration into realization — a dinner, a renovation, a wedding, a new skill. That only works if we understand more than the Pin a user just clicked. The useful unit of personalization isn’t the thing someone just clicked, it’s what they’re trying to accomplish. Such goals unfold over weeks and months, across dozens of sessions, through hundreds of saves.
We call those goals journeys: the intersection of a user’s interests, intent, and context at a point in time. A journey is a sequence of interactions, often spanning sessions, centered on one interest and a clear intent. For example, “Summer dresses” are an interest. “Figuring out what’s in style for a summer wedding I’m attending in August” is a journey. Someone might be interested in five at once: a kitchen remodel, a vegetable garden, learning sourdough and the mix changes as life does.
If we can name those journeys, recommendations and notifications can help someone finish a project instead of bouncing between unrelated interests. This post is about how we infer them in production: a fine-tuned 4-billion-parameter large language model (LLM) that reads a chronological activity log and emits a ranked list of journey names, in the user’s locale.
We didn’t start there. The initial user journey system clustered keywords and attached attributes with separate machine learning models. This post is the follow-up which covers how we formulate journey inference as generation, how we distill frontier-model quality into a 4B student we can actually serve, how the serving stack holds up at notification scale, and what we’re building next. The recipe is domain-agnostic: any platform with sequential user activity and structured intent targets (e-commerce, streaming, search) can adopt the same activity-to-intent formulation, collapsing classical multi-stage recommendation pipelines into a single generative model, making generative journey inference a general recipe for personalization beyond Pinterest.
Most recommenders optimize the next item. However, journeys on Pinterest are longer-horizon. Planning a kitchen is not the same as a one-off click on a cabinet Pin, and it isn’t well described by a durable interest like “Home Decor”. The cabinet Pin, the backsplash search, and the garden-bed save might be one project, two projects, or noise. Getting that distinction right changes what we retrieve, what we write in a notification, and when we should stay quiet.
So the output we want is not a taxonomy label. It’s a short, specific, ranked list of a user’s active journeys, with specific names they’d recognize.
We had almost no labeled journeys when we started, so we stayed lean. We reused pretrained text embeddings, started from a few hundred human examples, and built a multi-stage pipeline:
That pipeline powered the first journey-aware notifications. Against interest-based notifications, it lifted email click rate by 88%, push open rate by 32%, and positive survey feedback by 23%.
It also compounded its own errors. Every stage was locally reasonable; together they fragmented coherent goals.
To solve these problems, we stopped treating journey inference as a chain of transformations and started treating it as one act of generation: read the activity, write the intent. The model gets a single input prompt containing a semi-static profile plus a chronological activity log, and returns strict JSON — a ranked list of journey names:
User Profile:
gender: female; language: en-US; country: US; age: 30-35
User Owned Boards: Kitchen Reno 2026; Weeknight Dinners; Garden Plans
User Activities:
2026-06-14 search shaker cabinets.
2026-06-14 save subway tile backsplash, kitchen backsplash; quartz countertop, white kitchen.
2026-06-12 click raised garden bed, vegetable garden; tomato trellis.
2026-06-09 search sourdough starter discard.
2026-05-30 save cast iron skillet cornbread, dutch oven recipes.
...
{"journeys": ["Modern Farmhouse Kitchen", "Raised Bed Vegetable Garden",
"Sourdough Bread Making", "Cast Iron Recipes"]}Notice what’s gone: no keyword extraction step, no embeddings, no clusters, no separate naming model. One model reads the log and writes the answer.
Some deliberate choices in the prompt:
The output contract is deliberately simple: drop non-strings, dedupe case-insensitively, and the first occurrence wins. The output is parsed defensively because a generative system that occasionally emits prose where you expected JSON is a system that occasionally drops a user’s entire personalization payload.
The other thing worth saying out loud is that once you collapse the pipeline, the prompt becomes the ranking policy and the safety policy. Everything the staged system encoded in model weights and thresholds now has to be clearly stated. In production it asks for up to about twelve distinct journeys, each 1–5 words, in the user’s locale, ranked by:
It also encodes the failure modes we observed with the legacy journey clustering system: don’t merge unrelated topics; drop one-off outliers; treat auto-generated tags as noisy; lightly diversify the top of the list only among journeys that are already well supported. Empty output is allowed when the activity doesn’t support anything but inventing unsupported journeys is not.
These instructions aren’t just decorative. The diversity pass replaced a similarity-penalty diversifier. The dislike rule replaced a hand-tuned negative-signal filter. Few tag-noise instructions exist because auto-generated annotations are genuinely noisy, and the model will happily build a journey out of a classification artifact if you let it. Prompts like this deserve the same review discipline as model code, so we treat changes to them as launches.
Clustering was the right cold-start system, and it still exists as a fallback. Production inference is now LLM-first: one model, one decode, structured output.
Frontier models can infer coherent journeys without task-specific training, but they are too expensive to run for hundreds of millions of people on Pinterest regularly. Instead, we use them as teachers to generate synthetic training examples: each activity history is paired with a high-quality target journey. We also found that teacher outputs improved when the model was allowed to use additional reasoning tokens before producing its final structured response.
We fine-tuned open-weight Qwen3 students with 0.6B, 1.7B, 4B and 8B parameters. Supervised fine-tuning (SFT) teaches the model from the teacher-generated examples, while low-rank adaptation (LoRA) lets us update small adapters rather than every model parameter.
Quality improved rapidly during the first few thousand examples, then largely stopped increasing. Adding more similar examples provided little benefit; selecting diverse, high-quality examples — especially difficult consolidation and multilingual cases — was more valuable.
Model size presented a separate trade-off. The 0.6B and 1.7B students were faster, but produced less relevant and consistent journeys on the harder cases. Among the sizes we tested, the 4B model offered the best balance between output quality and serving throughput, so it became our production student.
Table 1: Same-user comparison of the staged clustering pipeline against the fine-tuned LLM. Fragments consolidated (Cabinets + Countertops → Modern Farmhouse Kitchen). Generic buckets got specific (Art → Penguin Art Drawing). Dropped intents came back (Cast Iron Recipes). Noise disappeared (Celebrity News, Memes). And user B got journeys in Spanish — something the keyword pipeline structurally couldn’t do, because it named journeys out of an English-annotation vocabulary.
Scoring hundreds of millions of people on Pinterest is a batch problem with online-shaped latency. We run a full refresh when the algorithm changes, and a daily incremental pass over recently active users, so journeys move when behavior does.
We benchmarked the offline workflow against serving the same 4B model behind an online inference stack. Online serving won on throughput. The reason is a set of now-mature decode optimizations that a naive offline loop typically doesn’t get for free: PagedAttention, so the key-value cache isn’t wasted on padding; continuous batching, so a finished sequence immediately yields its GPU slot to the next user instead of waiting for the slowest prompt in a fixed batch. The serving stack is NVIDIA Dynamo in front of vLLM decode workers on L40S GPUs. Frontend and decode workers scale independently, and the frontend routes with key-value (KV) cache awareness so requests sharing prompt structure land on workers that already hold the relevant prefix, which matters because every request shares a long, stable system prompt. We enable prefix caching and chunked prefill and cap concurrent sequences per worker.
In production, a cluster on the order of 100 GPUs sustains about 775 requests per second end-to-end, with a median around 1.2 seconds and p95 around 3.0 seconds. The batch side is where the cheaper wins live. Inference runs as a daily fan-out over journey-eligible users, sharded into small per-job batches with bounded concurrency inside each batch, so a slow model response back-pressures one shard instead of the pipeline. The single most effective throughput lever we added wasn’t a GPU at all: we track when each user was last inferred and skip anyone processed within the last couple of days. Journeys don’t change hour to hour, and paying to re-derive an unchanged journey is the most expensive kind of waste — the kind that looks like progress on a dashboard.
If a decode fails, we retry that user rather than storing a partial list. An empty but valid JSON list is a successful inference; some users don’t have a supportable journey that day, and we’d rather send nothing journey-specific than a generic “Art”. We suppress inference of sensitive traits at multiple levels before anything is stored:
Inference writes a ranked list of journeys per user into a store that notification serving reads at send time, and each journey name then becomes a retrieval query. We built embedding-based retrieval models to pull Pins for each user journey which are further ranked by our Pin ranking models. Journeys the user was recently notified about are fatigued, and journeys they’ve given explicit negative feedback on are suppressed outright. Only then does the notification framework decide whether to send at all, and on which channel.
There is no ground-truth label for a user’s latent goal. We evaluate at three levels, from inspection to a causal test.
The fine-tuned Qwen-4B model is the production default. Everything in this section is under experiment — trained or rolling out, not what every user gets today.
Tags mean every Pin costs a handful of tokens and drags along a dependency on Pin-to-text mappings. Semantic IDs (SIDs) are the alternative: each Pin is compressed by a residual-quantized variational autoencoder (RQ-VAE) into five levels of 11 bits each, which we pack into a single 64-bit integer and unpack at prompt-build time into special tokens:
<sid_begin><sid1_1263><sid2_973><sid3_725><sid4_793><sid5_1727><sid_end>
A general-purpose model has no idea what <sid3_725> means, so we bridge the gap with continued pre-training (CPT): extend the tokenizer with the SID vocabulary, then run next-token prediction over interleaved SID sequences, Pin titles, and tags until the model has mapped its world knowledge onto our catalog. SFT goes on top, so that given a SID sequence the model writes journeys.
SIDs only work better for users with dense engagement, so production is a fallback ladder: fewer than ten SID-carrying events and we fall back to tags for the whole prompt; a single event missing a SID falls back to tags for that event; and search queries always keep their literal text, because a typed query is the highest-fidelity signal we have. The bet is fewer input tokens, no dependency on annotations, and a shared item language between recommenders and LLMs. The cost is interpretability — a tag-based prompt can be read by a human mid-debugging session. A SID prompt cannot.
Onsite search and saves remain the strongest intent signals, but they can be sparse for dormant or low-activity users. We therefore use recent offsite actions — such as page visits, add-to-carts, sign-ups, and checkouts — as a complementary signal, adding a compact, safety-filtered summary to capture timely shopping or planning intent beyond Pinterest.
Offsite evidence remains secondary: onsite outweighs offsite, and checkout is stronger than add-to-cart or a page visit. We only add it when onsite activity is sparse, cap its prompt footprint, and avoid ranking offsite-only journeys at the top. We are testing whether better prompting is enough or whether a fine-tuned offsite-aware model is needed.
Once inference is generative, adding an output is a prompt-and-fine-tune change rather than a new service.
Distillation got us strong journeys, but it has a ceiling. The student is optimized to match the teacher, so its quality is bounded by the teacher’s sense of what users want. To go further, we need to optimize directly toward user preferences, using real feedback from users themselves.
Much of that signal already exists. Journey emails include a module asking “Are you interested in <journey>?” with Yes and No buttons. Each answer is logged, and a No immediately suppresses that journey for that user. Beyond these explicit answers, engagement inside journey-aware notification sessions gives us a much larger implicit signal: saves, clicks, and hides on the Pins we retrieved for a journey.
This feedback comes with a caveat: it is pointwise. Each response is a verdict on a single journey, while the model generates a ranked list. To bridge that gap, we split training into two stages. First, a mid-training stage applies preference optimization to single journeys across a large volume of feedback. Given a user’s activity, the model learns to make a journey that users confirm or engage with more likely, and a journey they reject less likely. Then, we formulated feedback-as-edit: the chosen response is the model’s list with the Yes journeys kept or the No journeys removed, and the rejected response applies the reverse edit. Starting from the mid-training checkpoint, fine-tuning restores list-level behavior: ranking, diversity, and the output format described earlier.
Improving the LLM with preference optimization and real user feedback is still an active area of experimentation. We are exploring how to weight explicit versus implicit feedback, and how to formulate a loss that is amenable to scaling, one whose gains follow a scaling law as feedback volume grows. Offline results are promising, and online A/B tests are now running. We hope to share more details in a follow-up blog post or publication.
We’d like to thank Justin Tran, Darien Boyd, Sarah Schwager, Shipeng Yu, Randy Tumalle, James Hernandez, Mengtian Hu, Dimitra Tsiaousi, Yaxin Li, Ying Wei, Sam Owens, Gloria Lee, Justin Pangilinan, Ye Tian, Rui Liu, Anya Trivedi, Binghui Gong, Tianqi Wang, Fangzheng Tian, Eric Tam, Mengying Yang, Lauren Fratamico, Andrew von Rosenbach, Ignacy Debicki and other contributors from Notifications, ML Platform, GenAI Serving, and Content Safety teams for their insights and support!
From Activity to Intent: Generating User Journeys with LLMs was originally published in Pinterest Engineering Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.
Continue reading on the original blog to support the author
Read full articleStandard retrieval models often collapse diverse user interests into a single vector. This architecture demonstrates how to scale multi-intent retrieval using transformers and GPU-optimized batching, significantly improving relevance and engagement in massive-scale recommendation systems.
Scaling recommendation models is often limited by network bandwidth rather than compute. This demonstrates how to overcome communication bottlenecks in embedding-heavy architectures, enabling massive model training with near-linear efficiency and optimized infrastructure costs.
Managing user-sequence data is notoriously expensive and prone to training-serving skew. This unified architecture reduces operational costs and ensures data consistency across the ML lifecycle, enabling faster iteration on sequence-aware models like Transformers for recommendation systems.
It demonstrates how to scale multimodal LLMs for production by combining expensive VLM extraction with efficient dual-encoder retrieval. This architecture allows platforms to organize billions of items into searchable collections while maintaining high precision and low operational costs.