robots.txt is a text file webmasters create to instruct robots how to crawl and index pages on your website.
Robots.txt is a file that is used to tell search robots which folders or directory would you allow to be indexed. You also mention here the directories that you don’t want to be indexed.
Please read this posts:
Compress CSS and Squeeze it up!