Posts tagged with data

Instagram EngineeringNov 26, 2019

Why it matters: This article provides a blueprint for building massive-scale recommendation engines. It demonstrates how custom DSLs and multi-stage filtering balance high-velocity experimentation with the extreme computational efficiency required to serve millions of users in real-time.

Instagram uses a three-stage ranking funnel to filter billions of media items into a personalized feed for each user in real-time.
Engineers developed IGQL, a C++ optimized domain-specific language, to allow for high-level algorithm design with low-latency execution.
The system utilizes 'ig2vec' account embeddings to identify topical similarities based on user interaction sequences, similar to word2vec.
Facebook’s FAISS library is used for efficient nearest-neighbor retrieval across millions of account embeddings.
The infrastructure supports massive scale, processing 65 billion features and making 90 million model predictions every second.

#mlp #data #dist

Read original

Instagram EngineeringOct 11, 2019

Making instagram.com faster: Part 3 — cache first

Why it matters: Cache-first rendering provides immediate UI feedback but creates complex state sync challenges. This approach shows how to use Git-like rebase patterns in Redux to ensure user interactions aren't lost when merging stale cached data with fresh server responses.

Implemented cache-first rendering by storing a subset of the Redux store in IndexedDB to allow immediate page hydration.
Addressed race conditions where user interactions on cached data, such as likes or comments, could be overwritten by incoming server responses.
Developed a staging mechanism that treats cached state as a local branch and server data as master, performing a rebase-like operation for state updates.
Created a staging API using stagingAction and stagingCommit to queue dispatched actions while network requests are pending.
Used a Redux reducer enhancer to apply queued actions to the fresh server state before committing it to the main store.
Achieved significant performance gains, including a 2.5% improvement in feed display time and an 11% improvement in stories tray display time.

#frontend #data

Read original

Instagram EngineeringSep 24, 2019

Interview with Tamar Shapiro, Instagram’s Head of Analytics

Why it matters: This interview offers a look into how Instagram uses data science and experimentation to drive product strategy. It highlights the intersection of technical leadership, user-centric culture, and the professional development skills necessary to succeed in high-scale engineering organizations.

Tamar Shapiro leads Instagram's analytics team, overseeing data scientists and engineers focused on experimentation and data-driven product decisions.
A key project discussed is the 'private like counts' test, which aimed to shift user focus from quantity to quality of interactions.
Instagram's engineering culture is characterized as highly collaborative with a 'people first' value system centered on the user community.
Effective communication and context sharing are emphasized as vital for maintaining alignment in fast-paced development environments.
Career growth advice for engineers includes prioritizing networking outside immediate teams and building confidence to advocate for accomplishments.

#data #culture

Read original

Page 38 of 38

Prev 1...36 37 38Next