In October this year, Google launched its biggest update since RankBrain in 2015. BERT, an acronym for Bidirectional Encoder Representations from Transformers, is an update to Google’s core search algorithm to improve its language comprehension capabilities. Thanks to BERT, Google will now understand the context of our search queries better.
This blog will try to understand what BERT is about and how it will affect your search.
So, let’s dive in.
Let’s begin by understanding that BERT may be a new concept; it isn’t brand-new. Google introduced the concept of BERT in a paper entitled ‘BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding’, released in 2018. An excerpt from the paper reads:
“Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from the unlabeled text by joint conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.”
According to the paper, a language model that’s bidirectional trained has a deeper understanding of the context and flow of language than any unidirectional model.
BERT uses a Transformer, an attention mechanism that learns the contextual relation between words and sub-words in any text.
The Transformer encoder reads the entire sequence of words simultaneously instead of the directional models that read the text sequentially (left to right or right to left). Such a property allows the model to understand the context of a word and what words are in its surroundings (left or right).
The training model for the BERT model involves two steps: pre-training and fine-tuning. In pre-training, the model gets trained on unlabeled data using Masked LM and Next next-sentence prediction (discussed later). For fine-tuning, the BERT model is initialized with pre-training parameters and all the parameters are fine-tuned using labelled data.
BERT is pre-trained using two unsupervised tasks: Masked LM (MLM) and Next Sentence Prediction (NSP). Let’s discuss this one by one.
Masked LM (MLM): In MLM, 15% of the words in any text sequence are masked and fed into the model. The model tries to predict the value of the masked words based on the context provided by the surrounding, non-masked words.
Next Sentence Prediction (NSP): In this training task, the model receives pairs of sentences as input to learn to predict if the second sentence in the pair follows the first sentence in the original document. The sentences are chosen so that if there are two sentences, A and B, in the pair, 50% of the time, B is the actual sentence that follows A in the document and the remaining 50% of the time, B is a random sentence.
After pre-training, a layer gets added to the core model to fine-tune BERT for specific tasks. Fine-tuning makes BERT suitable for a wide range of applications.
Google has estimated that the update will affect around 10% of the search queries. While these may be significant, considering the vast number of searches conducted on Google daily, it doesn’t seem to affect SEO as much. This is because the update will primarily target long-tailed and conversational queries that SEOs do not target as heavily.
Before the update was launched, Google had difficulty understanding the context of certain queries, which would have been pretty clear to any human being. But post-BERT, the search engine has begun to show highly relevant results for the same queries. Here are some examples taken from Google blog:
Image Courtesy: Google
In the example cited above, the user queries, ‘2019 Brazil traveller to USA need a visa.’ In this example, the word ‘to’ and its association with other words in the text are crucial to understanding the query.
Before the BERT update, Google algorithms didn’t understand this relationship and generated results about US citizens travelling to Brazil. But now, Google comprehends this nuance and, as a result, returns relevant answers.
Here’s yet another example:
Image Courtesy: Google
In the above example, the user types a query, ‘do estheticians stand a lot at work?’. Before BERT came into the picture, Google used to match terms and considered ‘stand’ as ‘standalone’. Now it understands the nuances of this text rightfully comprehends ‘stand’ and returns relevant answers.
SEO or search engine optimization is the process of optimizing your site for search engines. So, any update that affects the search algorithm will impact your optimization process.
Read: The Next Big Thing in SEO Updates in 2020
Now the question is: What can you do to optimize your content for search engines post-BERT? Danny Sullivan from Google answers:
“There is nothing to optimize for with BERT, nor anything for anyone to rethink. The fundamentals of us seeking to reward great content remain unchanged.”
So, the fundamentals of your SEO strategy are going to remain the same. However, you will need to focus on two things:
BERT will help Google understand the context of user queries better. As people make more and more enquiries on Google, they try to get fast answers to their queries. So, going forward, the search engine results page will show results that best fit the user intent and not necessarily the exact searched term. There may be situations when the search term isn’t even included in the answers.
Featured Snippets are a format supposed to provide users with concise, direct answers to their queries right on the search engine results page, without requiring the user to click on a specific result. These snippets may be in the form of text, video, list or table.
For example, if you search “how many countries are there in Asia”, you will get a direct reply as the very first result, as shown below.
As per Google, BERT tries to understand natural user language and long queries. With featured snippets, it’s possible to provide exact answers to such queries quickly, hence the emphasis on featured snippets.
Read: How to Optimize the Website for Google Featured Snippets
Content marketing involves the creation of valuable, relevant and compelling content aimed at a clearly defined audience. Because BERT will make Google understand natural language better, long-tailed queries and featured snippets are the areas where content marketing and social media marketing professionals need to focus their attention on.
Content marketers will need to create content that sounds more human-like and provides quick answers to user queries while offering value.
Here are some steps you can take to optimize your content marketing efforts.
To begin with, you can look for specific keywords using a keyword research tool such as Google Keyword Planner, Keyword Surfer and Keywordtool.io. A quick analysis will reveal how popular that keyword is, what kind of content ranks, and how difficult it is for you to rank on that keyword.
Now that you have an idea of what kind of keywords to use, you can write new content or optimize an existing one in order to accommodate these keywords and make your content relevant for users. Keyword analysis offers insights on what kind of content is popular among users so you can create content that interests users and answers their queries.
With technology growing by leaps and bounds, search engines are becoming more intelligent and intuitive daily. Content marketing, social media marketing and email marketing professionals need to step up their game to keep pace with these developments. They have to weave their strategy around the user and churn out content that caters to his specific needs.