Robots.txt Generator

Quick Presets

User-Agent Rules

seconds

Generated robots.txt


                


                
            


            

                


                

                    
Common User Agents

                    
    
                        
  • * - All bots
    • 
                        
  • Googlebot - Google
    • 
                        
  • Bingbot - Bing
    • 
                        
  • Slurp - Yahoo
    • 
                        
  • DuckDuckBot - DuckDuckGo
    • 
                        
  • facebot - Facebook
    • 
                        
  • Twitterbot - Twitter
    • 
                    

                


                

                    
Related Tools