Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
eget
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Registry
Registry
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Ivan Ivlev
eget
Commits
860d06ac
Commit
860d06ac
authored
Nov 11, 2017
by
Vitaly Lipatov
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
rewrite eget
parent
38ab6652
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
22 additions
and
17 deletions
+22
-17
eget
eget
+22
-17
No files found.
eget
View file @
860d06ac
#!/bin/sh
# eget - simply shell on wget for loading directories over http
# eget - simply shell on wget for loading directories over http
(wget does not support wildcard for http)
# Example use:
# eget
f
tp://ftp.altlinux.ru/pub/security/ssl/*
# eget
ht
tp://ftp.altlinux.ru/pub/security/ssl/*
#
# Copyright (C) 2014-2014, 2016 Etersoft
# Copyright (C) 2014 Daniil Mikhailov <danil@etersoft.ru>
...
...
@@ -42,33 +42,37 @@ fi
if
[
-z
"
$1
"
]
;
then
echo
"eget - wget wrapper"
>
&2
echo
"Run with URL, like
f
tp://somesite.ru/dir/*.log"
>
&2
echo
"Run with URL, like
ht
tp://somesite.ru/dir/*.log"
>
&2
exit
1
fi
# If ftp protocol or have no asterisk, just download
# TODO: use has()
if
echo
"
$1
"
|
grep
-q
"^ftp://"
||
echo
"
$1
"
|
grep
-qv
"[*?]"
;
then
# If ftp protocol, just download
if
echo
"
$1
"
|
grep
-q
"^ftp://"
;
then
$WGET
$WGET_OPTION_TARGET
"
$1
"
exit
fi
# drop mask part (if has /$, not changed)
URL
=
$(
echo
"
$1
"
|
grep
"/$"
||
dirname
"
$1
"
)
# If have no wildcard symbol like asterisk and no / at the end, just download
if
[
"
$URL
"
!=
"
$1
"
]
&&
echo
"
$1
"
|
grep
-qv
"[*?]"
;
then
$WGET
$WGET_OPTION_TARGET
"
$1
"
exit
fi
echo
"Fall to http workaround"
URL
=
$(
echo
"
$1
"
|
grep
"/$"
||
dirname
"
$1
"
)
# mask allowed only in last part of path
MASK
=
$(
basename
"
$1
"
)
get_index
()
{
MYTMPDIR
=
"
$(
mktemp
-d
)
"
INDEX
=
$MYTMPDIR
/index
$WGET
$URL
-O
$INDEX
}
# TODO: skip create_fake_files for full dir
# add * if full dir
#[ "$URL" != "$1" ] && MASK="*"
print_files
()
{
cat
$INDEX
|
grep
-o
-E
'href="([^\*/"#]+)"'
|
cut
-d
'"'
-f2
$WGET
-O-
$URL
|
\
grep
-o
-E
'href="([^\*/"#]+)"'
|
cut
-d
'"'
-f2
}
create_fake_files
()
...
...
@@ -79,19 +83,20 @@ create_fake_files()
print_files |
while
read
-r
line
;
do
touch
$DIRALLFILES
/
$(
basename
"
$line
"
)
done
rm
-f
$INDEX
}
download_files
()
{
ERROR
=
0
# TODO: test fix / at the end
for
line
in
$DIRALLFILES
/
$MASK
;
do
[
-r
"
$line
"
]
||
{
ERROR
=
1
;
break
;
}
$WGET
$URL
/
$(
basename
"
$line
"
)
||
ERROR
=
1
done
return
$ERROR
}
get_index
||
{
rm
-rf
"
$MYTMPDIR
"
;
exit
;
}
MYTMPDIR
=
"
$(
mktemp
-d
)
"
create_fake_files
download_files
||
echo
"There was some download errors"
>
&2
rm
-rf
"
$MYTMPDIR
"
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment