Search Engine
Every search engine has three main phases
1. Crawling
Search Engine : Browsing is the process where it all begins. It means visiting or searching for data on the web. As part of this process, the website will start taking data.
In this, its main task is to scan the web and gather details about each page.
To do this, by going to each page of the website, it searches the title, images, keywords and other linked pages, etc.
The work of different crawlers is also different from each other, like when someone looks at the page layout, someone sees an advertisement on the pages, then someone checks for spam links.
But how does the web end up being browsed?
An automatic bot, also called a spider, that visits each page one by one in the shortest possible time.
To do this, it uses a link to the page and finds out where to go next.
In the early days, Google Spider could read hundreds of pages per second, which has now become thousands of pages per second.
When a web crawler visits a web page, it collects all the links on the page and adds them to its must-visit list as the next page.
He then visits the next page in his list and in the same way this cycle continues and repeats itself over and over.
Web crawlers will also revisit the previous page whenever you make any changes to them.
This means that if any site is linked to an indexed site, Google’s bots crawl that site very quickly.
Some pages are browsed very quickly.
Although many sites are crawled very deeply, sometimes if the hierarchy of the site’s pages is very complex, the crawler will leave them.
2. Indexing
The indexing process occurs when the data is processed and stored in the database after being crawled.
Let me explain this with a very simple example.
When we study, we study the entire syllabus. Read each question and answer, but in the exam write only what is asked on the answer sheet, as well as crawling the entire website.
But it indexes the post by the query itself.
This is why the search engine sometimes shows it on the results page by modifying the title and meta description written in the blog.
It is not necessary for a blogger to display the same title and description when writing a blog post.
It shows the same thing written for the targeted keyword, but the post ranks for many keywords and the title and description of the other keywords may be different.
I have seen this many times in my post that when a post is ranking for many other keywords besides the target keyword, then the title changes for different keywords.
3. Search and Evaluation
Retrieval is when a search engine processes your query and displays the content most similar to your query.
Most search engines differentiate themselves from other search engines by the search process.
They maintain their own criteria for which query the page content is suitable for, selecting pages based on the query to be searched accordingly.
This is why the results of a search query on Google and Bing are different.
The ranking algorithm searches billions of pages to show you the pages most similar to the query you’re looking for.
This algorithm created by the companies is very complex, so it is kept secret for its security.
They also don’t want a webmaster to run their search algorithm after their own and go wrong at the beginning.
If their method works, then the work of many hardworking people who work hard to create content and rank it based on that will be wasted.
Yes, it’s possible that the actual engine can be abused, but now it’s not that easy.
A search engine basically ranks a website based on how many times a keyword has been used on a page, and this also creates the problem of keyword stuffing.
This means that keywords are repeated too often for no reason throughout the page.
Then came the concept of the importance of the link. They started giving more importance to those sites whose inbound links were very high, as it used to show the popularity of the site based on relevance.
But because of this, spamming has increased a lot all over the internet. Nowadays, only those links that have good site authority are important.
It gives more importance to those links that come from a government agency instead of which web directory the link comes from.
Today, the ranking algorithm has become more of a mystery than ever before. And not only is search engine optimization important, good rankings also depend on high-quality content and a great user experience.
