What is the custom robot.txt in blogger?


#1

Hello Bloggers :stuck_out_tongue: :stuck_out_tongue: :stuck_out_tongue:

How Are You all Today…?? May GOD Bless you
I want to know about custom Robot.txt in blogger…? What is this and how to edit or use it…? I’ll wait for suggestion… So Please stay with me, :slight_smile: :slight_smile: :slight_smile: :slight_smile:

Thanks in Advance


#2

I’m Fine :smiley: And May God Bless You More :blush:

Robots.txt is a file which tells search engines to index your blog or not and other crawlers like adsense bot , image bot etc.

You Should Leave It As Default As It Is Best For Indexing And One Simple Change Can Make Your Blog To No Get Indexed !

Therefore don’t edit it :smiley:

Happy Blogging :blush:


#3

this code is in my Custom Robot.txt… Is it fine or not?

Blogger Sitemap generated on 2015.11.05

User-agent: * Disallow: /search Allow: / Sitemap: http://urdupoetry-s.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500 Sitemap: http://urdupoetry-s.blogspot.com/sitemap.xml

and thanks for suggestion :slight_smile: :slight_smile: :*


#4

Please Give ME A Pic Of Your Search Preferences Settings Page :wink:


#5

yeah Here is this


#6

Click On No In Enable Custom Robots.txt Content ! And Then It Will Be Best For You !

I’ve Also Done The Same For My Blog :smiley:


#7

ok as u say…:slight_smile: :slight_smile: :smiley: but in Search console i also have this…


#8

NOW go to Search Console And Look For Robots.txt tester then you’ll find all things in green ! :smiley:


#9

See the screenshot of Robot.txt tester


#10

Yes That’s What I’m Saying that you’ll find 0 errors and 0 warnings which is best for a blog !

Keep Asking Such Questions :smiley: We’ll Always Help You :heart:


#11

this is the same before ,


#12

But Keeping It Default Is The Best Option :smiley:


#13

ok…

anyway thanks you so much :slight_smile: :blush::wink:


#14

@paaris_sohail, the thing is- the default robot.txt file contains the code that tells the search bots not to include search pages, archive pages and label pages in the database of Google.

This file can be useful for your if you wish to tell search bots not to index a particular or some pages with any term of your blogger blog (same for other blogs and websites).

As @Nitishk said, the default is best, and it is right because the default robot.txt file is created by Blogger team and is for Google Blogger blogs only and so, it is designed by keeping in mind, the pages that need to be indexed.