Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
Corryvreckan
Corryvreckan
Commits
fe0df33b
Commit
fe0df33b
authored
Sep 19, 2019
by
Simon Spannagel
Browse files
Improve test data download script to always remove untarred folder before
parent
a2b4061d
Pipeline
#1102677
passed with stages
in 24 minutes and 13 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
testing/download_data.py
View file @
fe0df33b
#!/usr/bin/env python
#
#
d
ownload raw test data files
or not (depending on the moon position)
#
D
ownload raw test data files
, check integrity and untar
from
__future__
import
print_function
,
unicode_literals
import
hashlib
...
...
@@ -8,6 +8,7 @@ import io
import
os
import
os.path
import
urllib
import
shutil
import
sys
import
tarfile
try
:
...
...
@@ -37,6 +38,8 @@ def check(path, checksum):
if
not
os
.
path
.
isfile
(
path
):
return
False
if
not
sha256
(
path
)
==
checksum
:
print
(
'
\'
%s
\'
exists, wrong checksum, deleting'
%
path
)
os
.
remove
(
path
)
return
False
print
(
'
\'
%s
\'
checksum ok'
%
path
)
return
True
...
...
@@ -64,5 +67,14 @@ if __name__ == '__main__':
else
:
datasets
=
{
_
:
DATASETS
[
_
]
for
_
in
sys
.
argv
[
1
:]}
for
name
,
checksum
in
datasets
.
items
():
# Download tarball if necessary
download
(
name
=
name
,
checksum
=
checksum
)
# Delete existing untarred files
target
=
os
.
path
.
join
(
BASE_TARGET
,
name
)
if
os
.
path
.
exists
(
target
)
and
os
.
path
.
isdir
(
target
):
print
(
'
\'
%s
\'
deleting existing folder'
%
target
)
shutil
.
rmtree
(
target
)
# Untar anew from tarball
untar
(
name
)
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment