.Google restricts robots.txt help to 4 industries, clarifying its own viewpoint on unsupported regulations.Google only sustains four details robots.txt industries.In need of support ordinances in robots.txt will be ignored.Think about auditing your robots.txt documents due to this improve.