What is BERT Update and How Does It Affect Your Search?

In October this year, Google launched its biggest update since RankBrain in 2015. BERT, an acronym for Bidirectional Encoder Representations from Transformers is an update to the core search algorithm of Google aimed to improve its language comprehension capabilities. Thanks to BERT, Google will now understand the context of our search queries better.

In this blog, we will try to understand what BERT is all about and how it will affect your search.

So, let’s dive in.

Let’s begin by understanding that BERT may be a new concept, it isn’t brand-new. The concept of BERT was introduced by Google in a paper entitled ‘BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding’ released in 2018. An excerpt from the paper reads:

“Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.”

According to the paper, a language model that’s bidirectional trained has a deeper understanding of the context and flow of language than any unidirectional model.

BERT uses Transformer, an attention mechanism that learns the contextual relation between words and sub-words in any given text.

The Transformer encoder reads the entire sequence of words at once as opposed to the directional models that read the text in a sequential manner (left to right or right to left). Such a property allows the model to understand the context of a word in relation to what words are in its surroundings (left or right).

The training model for BERT model involves two steps: pre-training and fine-tuning. In pre-training, the model gets trained on unlabeled data using Masked LM and Next Sentence Prediction (discussed later). For fine-tuning, the BERT model is initialized with pre-training parameters and all the parameters are fine-tuned using labelled data.

Pre-training BERT

BERT is pre-trained using two unsupervised tasks namely, Masked LM (MLM) and Next Sentence Prediction (NSP). Let’s discuss these one by one.

Masked LM (MLM): In MLM, 15% of the words in any text sequence are masked and then fed into the model. The model tries to predict the value of the masked words based on the context provided by the surrounding, non-masked words.

Next Sentence Prediction (NSP): In this training task, the model receives pairs of sentences as input in order to learn to predict if the second sentence in the pair follows the first sentence in the original document. The sentences are chosen in such a manner that if there are two sentences A and B in the pair, 50% of the time B is the actual sentence that follows A in the document and the remaining 50% of the time B is a random sentence.

Fine-Tuning BERT

After pre-training, a layer gets added to the core model to fine-tune BERT for specific tasks. Fine-tuning makes BERT suitable for a wide range of applications.

What will BERT affect the most?

Google has estimated that the update will affect around 10% of the search queries. While these may be significant in number considering the vast number of searches conducted on Google every single day, it doesn’t seem to affect SEO as much. This is because the update will primarily target long-tailed queries and conversational queries that SEOs do not target as heavily.

Before the update was launched, Google had difficulty understanding the context of certain queries which would have been pretty clear to any human being. But post-BERT, the search engine has begun to show highly relevant results for the same queries. Here are some examples taken from Google blog:

Image Courtesy: Google

In the example cited above, the user makes a query ‘2019 brazil traveler to USA need a visa.’ In this example, the word ‘to’ and its association with other words in the text are crucial to understanding the query.

Prior to the BERT update, Google algorithms didn’t understand this relationship and popped up results about US citizens travelling to Brazil. But now, Google comprehends this nuance and, as a result, returns relevant answers.

Here’s yet another example:

Image Courtesy: Google

In the above example, the user types a query ‘do estheticians stand a lot at work’. Before BERT came into picture, Google used to match terms and considered ‘stand’ as ‘standalone’.  Now it understands the nuances of this text and rightfully comprehends ‘stand’ and returns relevant answers.

How will BERT affect SEO?

SEO or search engine optimization is the process of optimizing your site for search engines. So, any update that affects the search algorithm will impact your optimization process.

Read: The Next Big Thing in SEO Updates in 2020

Now the question is: What can you do to optimize your content for search engines post BERT? Danny Sullivan from Google answers:

“There is nothing to optimize for with BERT, nor anything for anyone to be rethinking. The fundamentals of us seeking to reward great content remain unchanged.”

So, the fundamentals of your SEO strategy are going to remain the same. However, you will need to focus on two things:

1. Optimizing Content for User Intent
2. Featured Snippets

1) Optimizing Content for User Intent

BERT will make Google better at understanding the context of user queries. As people make more and more enquiries on Google, they try to get fast answers to their queries. So, going forward, the search engine results page will show results that best fit the user intent and not necessarily the exact searched term. There may be situations when the search term isn’t even included in the answers.

2) Featured Snippets

Featured Snippets are a format supposed to provide users with concise, direct answers to their queries right on the search engine results page, without requiring the user to click on a specific result. These snippets may be in the form of text, video, list or table.

For example, if you search “how many countries are there in Asia”, you will get a direct reply as the very first result as shown below.

Now as per Google, BERT tries to understand natural user language and long queries. With featured snippets, it’s possible to provide exact answers to such queries quickly, hence the emphasis on featured snippets.

Read: What are Featured Snippets?

How will BERT affect Content Marketing?

Content marketing involves the creation of valuable, relevant and compelling content aimed at a clearly-defined audience. Because BERT will make Google understand natural language better, long-tailed queries and featured snippets are the areas where content marketing and social media marketing professionals need to focus their attention on.

Content marketers will need to create content that sounds more human-like and provides quick answers to user queries while offering value.

Here are some steps you can take to optimize your content marketing efforts.

1) Perform Keyword Research

To begin with, you can look up for specific keywords using a keyword research tool such as Google Keyword Planner, Keyword Surfer and Keywordtool.io. A quick analysis will reveal how popular that keyword is, what kind of content ranks on that keyword and how difficult it is for you rank on that keyword.

Read: 10 Free Keyword Research Tools (That Aren’t Google Keyword Planner)

2) Create Optimized Content

Now that you have an idea of what kind of keywords to use, you can write new content or optimize an existing one in order to accommodate these keywords and make your content relevant for users. Keyword analysis offers insights on what kind of content is popular among users, so you can create content that users are interested in and also answers their queries.

Wrapping it up

With technology growing by leaps and bounds, search engines are becoming more intelligent and intuitive with each passing day. Content marketing, social media marketing and email marketing professionals need to step up their game in order to keep pace with these developments. They have to weave their strategy around the user and churn out content that caters to his specific needs.