Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
crawlersNoticias
Project
Project
Details
Activity
Releases
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
4
Issues
4
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
m3
crawlersNoticias
Commits
6beff693
Commit
6beff693
authored
Oct 26, 2018
by
Renán Sosa Guillen
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
downloaders
parent
7fe26ef5
Changes
5
Show whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
22 additions
and
10 deletions
+22
-10
download_backwards.py
crawler_script/download_backwards.py
+5
-0
download_by_day.py
crawler_script/download_by_day.py
+5
-0
download_often.py
crawler_script/download_often.py
+1
-1
tracker_backwards.py
crawler_script/tracker_backwards.py
+5
-4
tracker_by_day.py
crawler_script/tracker_by_day.py
+6
-5
No files found.
crawler_script/
crawl
_backwards.py
→
crawler_script/
download
_backwards.py
View file @
6beff693
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script para la descarga constante de histórico de medios del tipo "descarga_hacia_atras".
"""
import
sys
import
json
import
os
...
...
crawler_script/
crawl_all
.py
→
crawler_script/
download_by_day
.py
View file @
6beff693
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script para la descarga constante de histórico de medios del tipo "descarga_por_dia".
"""
import
sys
import
json
import
os
...
...
crawler_script/
crawl
_often.py
→
crawler_script/
download
_often.py
View file @
6beff693
...
...
@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-
"""
Script para la descarga automatica de noticias por rss
Script para la descarga automatica de noticias por rss
"""
import
sys
...
...
crawler_script/
backwards_tracker
.py
→
crawler_script/
tracker_backwards
.py
View file @
6beff693
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script para el trackeo de histórico de medios del tipo "descarga_hacia_atras".
"""
import
sys
import
json
import
os
import
datetime
"""
Script para la descarga de histórico de medios del tipo "descarga_hacia_atras".
"""
today
=
datetime
.
datetime
.
now
()
baseDir
=
"/home/geoint/virtualHDD/m3/noticias/"
scrapyDir
=
"/home/geoint/crawlersNoticias/"
...
...
crawler_script/tracker.py
→
crawler_script/tracker
_by_day
.py
View file @
6beff693
#!/usr/bin/python
# -*- coding: utf-8 -*-
import
sys
import
json
import
os
import
datetime
"""
Descarga las noticias de un sitio desde entre dos fechas especificas.
Descarga las noticias de un sitio desde entre dos fechas especificas
para medios del tipo 'descarga_por_dia'
.
USO:
tracker.py data.json
"""
import
sys
import
json
import
os
import
datetime
# today = datetime.datetime.now()
baseDir
=
"/home/geoint/virtualHDD/m3/noticias/"
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment