BlogCategoryTag

Header

Footer

PreFooter

ProseBlock

Blog

SocialLinks

Prose

AnnouncementContextProvider

UnnamedGlobalGroupOfVariants2

Screen

Segment

Experiment

Experiment2

scale.com (Production)

plasmic-nav

plasmic-embed-css

plasmic-basic-components

plasmic-tabs

plasmic-cms

react-chartjs-2

react-slick

react-quill

Weidan Guo

Will LeVine

Engineering

<h3>Background</h3> The rise of social media platforms has driven the popularity of user-generated content (UGC) and it is changing from the bottom up at a pace faster than ever. Platforms are competing for user attention and engagement - some are winning, but some are falling behind. How can a social media platform stay ahead of the game? One of the most effective ways to incentivize users to create and engage with content on the platform is by curating and promoting trends. A trend is a type of content repeated and shared amongst creators -&nbsp;a specific subject matter, action, or format that rises in popularity and eventually becomes viral. Common types of trends include dance challenges, transitions, and comedy skits. While videos in a trend follow a similar format, creators usually express their creativity and individuality by adding a personal touch to the videos. For that reason, the variety of the videos within the same trend can be wide and unpredictable. The key elements of these video trends include the video’s image, motion, audio, and text overlay. For example, a transition trend is a trend where the participants show a sudden change in their appearance, often signified by a dramatic sound effect in the background music and sometimes accompanied by dance moves, lip syncing, and text overlay. The lifecycle of a trend can vary from days to months depending on various factors. To remain competitive, social media platforms need to detect the trends while they are still emerging and put them to use immediately by matching them to the right creators and promoting them to the suitable audience.&nbsp; <a href="https://scale.com/content-understanding" rel="noopener noreferrer" target="_blank">Scale Content Understanding</a> improves platform experiences by enriching content metadata, discovering trend insights, and flagging sensitive content. One feature of Content Understanding is content data enrichment for better trend detection. To facilitate this process, we developed proprietary algorithms to detect trends early, quickly, and thoroughly.&nbsp; <h3>ML Clustering Overview</h3> <img src="https://drive.tiny.cloud/1/0lbhu31lrngl0y5eo8e6p0jckqgeikc94mnkaig43yuahupv/449d8647-5509-470c-a6cf-ea38c14a62f3"> We took a human-in-the-loop (HitL) machine learning approach to detect trends. We represent trends as clusters of videos estimated from video-extracted embeddings. Once formed, these clusters are sent to a human pipeline (which we call “Cluster Labeling”) where taskers answer a series of questions about the videos in a given cluster - namely, if a sample of the videos corresponds to a trend and the potential name of the trend. The following are the phases of the ML Clustering process: <h3>Phase 1: Embedding Extraction</h3> To extract embeddings, we use a variety of multi-modal information extraction machine learning models, each of which is trained on large-scale, common datasets with high distribution overlap to the social media domain. <h3>Phase 2: Clustering</h3> Once the embeddings are extracted, we perform a clustering pipeline to take in these embeddings and output clusters. Overview Given the embeddings, we first perform <a href="https://arxiv.org/pdf/1802.03426.pdf" rel="noopener noreferrer" target="_blank">Uniform Manifold Approximation and Projection (UMAP)</a> dimensionality reduction (inspired by <a href="https://arxiv.org/abs/2203.05794" rel="noopener noreferrer" target="_blank">BERTopic</a>). UMAP takes in high-dimensional data and reduces it down to a lower dimensional space scalably (e.g. UMAP performs computation on tens of thousands of points on the order of tens of seconds). It reduces dimensionality in a way that preserves global Riemannian geometric structure and pairwise distances. On these low-dimensional embeddings, we perform an iterative hierarchical clustering process. Each individual point in embedding space is initially represented as a cluster.&nbsp;We then calculate the proximity between all pairs of clusters. We calculate the proximity of two clusters in a given pair as the variance in embedding space (i.e. the average sum of squares from the joined cluster mean embedding vector) of the points corresponding to if these clusters were to be joined. The two clusters that are closest to each other are joined given the above definition of proximity. This cluster-joining process is performed iteratively until all clusters are above a specified variance that is provided as a hyperparameter. <img src="https://drive.tiny.cloud/1/0lbhu31lrngl0y5eo8e6p0jckqgeikc94mnkaig43yuahupv/24053e18-8fbc-46a8-91ba-4ea274599636"> Stop Condition Variance Hyperparameter We carefully tune the variance hyperparameter in the stop condition to balance between making the clusters too coarse or too granular. For large variance hyperparameters, the final clusters are allowed to span a wide region in embedding space, meaning they are not semantically similar enough to represent trends (which tend to be extremely specific). Diametrically opposed, for small variance hyperparameters, the clusters end up with too few videos for users in the cluster labeling pipeline to recognize the trend. We therefore carefully tune the stop condition variance hyperparameter. This provides a balance between the clusters being fine enough that they represent groups of videos that are semantically similar enough to correspond to trends, and coarse enough that users from the cluster labeling pipeline can recognize the trend. This hyperparameter is normalized in order to reduce the amount of per-dataset manual tuning. <h3>Phase 3: Cluster Post-Processing</h3> To improve the quality and throughput of the taskers, we prune out clusters that are composed of a number of videos that is smaller than our definition of a trend. We also add a post-processing density score to help taskers focus on the clusters which are tightest in embedding space - i.e. the ones that are most likely to be semantically similar. This density score is calculated as the intra-cluster variance of each cluster in the dimensionality-reduced embedding space. <h3>Conclusion</h3> Our trend detection product consistently performs at a higher than 90% recall and 90% precision in production. Our HitL ML approach enables us to stay flexible with customization towards specific content niches while maintaining a high throughput. We are able to process tens of millions of videos with a few hours of turnaround. Our priority is to not only detect trends promptly, but also equip our customers with the data and insights on these trends to take actions immediately.&nbsp; At Scale, we believe investing in data is the key to building better platform experiences and discovering trend insights. If you’re interested in learning more about <a href="https://scale.com/content-understanding" rel="noopener noreferrer" target="_blank">Scale Content Understanding</a>, speak to our team today.&nbsp;

image.jpg

Discovering short-form video trends is challenging. In this blog post, we explain how we use an ML + HiTL approach for faster and higher-quality trend detection.

Discovering Emerging Short Form Video Trends with Clustering and HitL

open-dataset-nucleus

slugs must be all lowercase. ie: ‘my-slug’

events-banner

This entry represents both interview quote highlight and interview chat highlight because of Builder limitations. Choose the fields that matches either of them.

interview-highlight

If it's selected, the author and quote fields will be taken, otherwise the chat fields. The default is for chat.

resources-download-model

transform-session

CTA to open the session. Leave blank if you don't want a cta for that session

Link for the CTA. Usually the link to the exchange session

blog-filter-type

event-transform-x-ticket

solution

homepage-quotes

logos

homepage-content

builder-debug

transfrom-x-in-person

event-model

Toggle if the event is featured. Featured events will be moved to the top of the list

guides-cta

summit-2023-partners

sales

z[deprecated] page

quotes

z[deprecated] compare

product

blog-category-model

announcement-banner

accelerate

Vercel Powered and Edge Computed OG Image. Preferred over og img file upload. If this field is non empty, it will take precedent over the uploaded file

summit-2023-speakers

homepage-resources

customers

Display order on the page /customers. Higher values show first

page-cross-product-webinar

interview

event-transform

Event model for new marketing events page

event-2-0

homepage-event

custom-fonts

transform-x-speaker

Be careful to not add any trailing white spaces to the slug. Slugs must be all lowercase. ie: ‘my-slug’

Order of the featured speakers displayed on the home page for transformX

z[deprecated] document-ai

summit-2023-agenda

event-converge

hosts

author

transform-x

Page: /In-Person. Section: Hero. Text: Title Text

Page: /In-Person. Section: Slider Card. Text: Title Text

Page: /In-Person. Section: Slider Card. Text: Card Body

Page: /In-Person. Section: Hero. Text: Tagline

open-dataset

z[deprecated] customers

URL redirections for Website. Disclaimer: takes 5~ min for each added or edited.

url-redirects

Use relative path. use `/blog/foo` not `https://scale.com/blog/foo`

legal

event

Optional second image intended to show on the grid view, should be a more squared image closer to 1:1 aspect ratio than the one used in the Cover field.

guides

z[deprecated] guides-old

z[deprecated] guides-lp

partners

blog-model

Option to list or not list this post in the Blog index. Enabled = not list in the index, Disabled = list in the index

Products

Government

Solutions

Resources

Customers

Pricing →

Discovering Emerging Short Form Video Trends with Clustering and HitL

The future of your industry starts here.