Skip to content

crawlersNoticias
crawlersNoticias
Commit abad6379 authored by umorales's avatar umorales
Browse files
Options

quadratin

parent 7897ed13
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
spiders/daily/quadratinEdomex/quadratinEdomex/settings.py
View file @ abad6379
...@@ -18,7 +18,7 @@ FEED_EXPORT_ENCODING="utf-8" ...@@ -18,7 +18,7 @@ FEED_EXPORT_ENCODING="utf-8"
#USER_AGENT = "planaMayor (+http://www.yourdomain.com)" #USER_AGENT = "planaMayor (+http://www.yourdomain.com)"
# Obey robots.txt rules # Obey robots.txt rules
ROBOTSTXT_OBEY = True ROBOTSTXT_OBEY = False
# Configure maximum concurrent requests performed by Scrapy (default: 16) # Configure maximum concurrent requests performed by Scrapy (default: 16)
#CONCURRENT_REQUESTS = 32 #CONCURRENT_REQUESTS = 32
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment