Skip to content

crawlersNoticias
crawlersNoticias
Commit 54388987 authored by Renán Sosa Guillen's avatar Renán Sosa Guillen
Browse files
Options

crawlers update

parent 4149fd45
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 2 deletions
otros_sitios/diarioYaqui/diarioYaqui/spiders/noticias.py
View file @ 54388987
......@@ -35,7 +35,6 @@ class QuotesSpider(scrapy.Spider):
def parse(self, response):
print response.url
pagination = response.xpath('//ul[@class="page-numbers"]/li/a/@href').extract()
if ( len(pagination) > 0 ):
pagination = pagination[-2].strip('/')
......@@ -64,6 +63,6 @@ class QuotesSpider(scrapy.Spider):
text += remove_tags(paragraph) + '\n'
item['text'] = text
item['url'] = response.url
print item['title']
# print item['title']
yield item
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment