User-agent: * Disallow: *.json$ Disallow: *.xml$ Disallow: /test/ Disallow: /ajax/ Disallow: /api/ Disallow: /css/ Disallow: /js/ Disallow: /login/ Disallow: /you/ Disallow: /logout/