Concept: Microsoft has explained a vast neural network model that it has been deploying in production to increase the relevance of Bing searches. The model, dubbed a “sparse” neural network, complements current large Transformer-based networks like OpenAI’s GPT-3, according to the company.
Nature of Disruption: Microsoft incorporates ‘Make Every feature Binary’ (MEB) model to improve nuanced relationships between search and webpage terms. The sparse, large-scale model includes 135 billion parameters (ML model parts learnt from historical training data) and over 200 billion binary features (reflecting nuanced connections between queries and documents). MEB, according to Microsoft, can connect single facts to features, allowing the model to grasp specific facts in more depth. MEB, which was trained on more than 500 billion query and document pairings from three years of Bing searches, is now in production for 100% of Bing searches in all regions and languages, according to the company. According to Microsoft, MEB can continue to learn with additional data provided, showing that model capacity grows as more data gets added. It is updated daily by constantly training with the most recent daily click data and using an auto-expiration approach that examines each feature’s timestamp and filters out features that haven’t been shown in the past 500 days. MEB can also highlight what users don’t want to view for a query by identifying negative connections between words or phrases. Understanding these negative associations might help eliminate irrelevant search results.
Outlook: In the area of ML, transformer-based models have gotten a lot of attention. As Microsoft previously stated, these models excel at recognizing semantic connections and have been used to improve Bing search. However, they may miss more subtle connections between search and webpage keywords that go beyond pure semantics. According to Microsoft, putting MEB into production resulted in a nearly 2% boost in clickthrough rates on the top search results, as well as a more than 1% reduction in manual search query reformulation. Furthermore, MEB reduced pagination clicks by approximately 1.5%. Users who need to click the “next page” button haven’t found what they’re looking for on the first page. To summarise, Microsoft believes that very large sparse neural networks, such as MEB, can learn complex correlations in addition to the capabilities of Transformer-based neural networks, and that this enhanced comprehension of search language helps the whole search ecosystem.
This article was originally published in Verdict.co.uk