Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
crawlersNoticias
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
4
Issues
4
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
m3
crawlersNoticias
Commits
d1ae93ff
Commit
d1ae93ff
authored
4 months ago
by
Ulises Morales Ramírez
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
animal politico
parent
5e8eddfd
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
3 deletions
+1
-3
noticias.py
...s/daily/animalPolitico/animalPolitico/spiders/noticias.py
+1
-3
No files found.
spiders/daily/animalPolitico/animalPolitico/spiders/noticias.py
View file @
d1ae93ff
...
...
@@ -83,8 +83,6 @@ class NoticiasSpider(scrapy.Spider):
headers
=
{
'Content-Type'
:
'application/json'
}
print
(
payload
)
yield
scrapy
.
Request
(
url
=
url
,
...
...
@@ -112,6 +110,6 @@ class NoticiasSpider(scrapy.Spider):
item
[
'title'
]
=
remove_tags
(
node
.
get
(
"title"
))
item
[
'topic'
]
=
remove_tags
(
node
.
get
(
"contentTypeName"
))
item
[
'text'
]
=
remove_tags
(
node
.
get
(
"contentRendered"
))
item
[
'url'
]
=
self
.
start_urls
[
0
]
+
node
.
get
(
"uri"
)
item
[
'url'
]
=
"https://"
+
self
.
allowed_domains
[
0
]
+
"/"
+
node
.
get
(
"uri"
)
item
[
'author'
]
=
node
.
get
(
"author"
,
{})
.
get
(
"node"
,
{})
.
get
(
"name"
)
yield
item
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment