Automatically rename a downloaded file by curl to avoidfilename conflict.
in [Shell]
|
Prev: Automatically rename a downloaded file by curl to avoid filename conflict.
Next: stratifying values in a file ...
From: Chris F.A. Johnson on 4 Feb 2010 01:38 On 2010-02-04, Hongyi Zhao wrote: > Hi all, > > When I download a file by curl, I use -o switch to write the file as a > local file, say, myfile. The issue is: if a file with the same > filename already exists in the target folder, the original file will > be overwrited. So, I want to rename this downloaded file to something > like this: myfile.1 in order to avoid filename conflict in the above > case; in gerneral, if myfile.1 also has already exists in the > destination folder, use myfile.2 as the filename for the curl's > output; and so on ... dlfile=myfile dfnum=1 while [ -e "$myfile.$dfnum" ] do dfnum=$(( $dfnum + 1 )) done dlfile=$dlfile.$dfnum curl -o "$dlfile" ... -- Chris F.A. Johnson, author <http://shell.cfajohnson.com/> =================================================================== Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress) Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress) ===== My code in this post, if any, assumes the POSIX locale ===== ===== and is released under the GNU General Public Licence =====
From: Stephane CHAZELAS on 4 Feb 2010 05:57 2010-02-04, 16:30(+08), Hongyi Zhao: > On Thu, 04 Feb 2010 16:12:30 +0800, Hongyi Zhao > <hongyi.zhao(a)gmail.com> wrote: > >>while [ -e "$dlfile.$dfnum".pdf ] > > Should be > > while [ -e "$dlfile.$dfnum" ] [...] No, those are subject to race conditions, use set -C, redirections (no -o in curl or -O- in wget). ( set -C ext= n=1 until command exec 3> "$file$ext" ext=.$((++n)) done exec curl ... >&3 ) -- St�phane
From: Stephane CHAZELAS on 4 Feb 2010 07:18 2010-02-04, 20:10(+08), Hongyi Zhao: > On Thu, 4 Feb 2010 10:57:07 +0000 (UTC), Stephane CHAZELAS > <stephane_chazelas(a)yahoo.fr> wrote: > >>No, those are subject to race conditions, use set -C, > > I'm a newbie of bash/shell programming. What do you mean by saying > *race conditions*? Could you please give me some more hints? I mean when you do if (can I do-that) then do-that There's always a chance that in between the test (can I do-that) and the action (do-that), the condition has changed. For instance, if two instances of your script where run at the same time, it could happen. It's probably not critical here, but this kind of coding is known to have introduced countless security issues, so it's a good habit to avoid it. >>redirections (no -o in curl or -O- in wget). >> >>( >> set -C >> ext= n=1 >> until command exec 3> "$file$ext" >> ext=.$((++n)) >> done >> exec curl ... >&3 >>) > > Could you please give the complete code segment for my issue? [...] url=http://www.google.com/ file=myfile ( set -C ext= n=0 until command exec 3> "$file$ext"; do ext=.$((++n)) done exec curl "$url" >&3 ) -- St�phane
From: Kaz Kylheku on 4 Feb 2010 21:32 On 2010-02-04, Hongyi Zhao <hongyi.zhao(a)gmail.com> wrote: > Hi all, > > When I download a file by curl, I use -o switch to write the file as a > local file, say, myfile. The issue is: if a file with the same > filename already exists in the target folder, the original file will > be overwrited. So, I want to rename this downloaded file to something > like this: myfile.1 in order to avoid filename conflict in the above > case; in gerneral, if myfile.1 also has already exists in the > destination folder, use myfile.2 as the filename for the curl's > output; and so on ... > > Any hints for the above purpose? With all these .1 .2 .3 ... files littering a single directory, how will you know which is which? File foo.2 might be last week's file, whereas bar.3 is yesterday's. Moreover, foo.1 and foo.2 might not even be related; what if they come from different sites, and just coincidentally have the same name? Maybe you should start a new directory for each separate download session, to keep things cleanly separated and organized. If the files are related together (.1 .2 ... are in fact versions of the same file), then download into a version control sandbox, allowing the download to overwrite, and check in the revisions. GIT is good for this purpose because it's fast, and the working copy /is/ the repository. No separate repo to set up.
From: Chris F.A. Johnson on 7 Feb 2010 00:06
On 2010-02-07, Hongyi Zhao wrote: > On Thu, 4 Feb 2010 12:18:29 +0000 (UTC), Stephane CHAZELAS ><stephane_chazelas(a)yahoo.fr> wrote: > >> ext= n=0 > > Why this line must has a white space after *ext=*? If it didn't, it would assign 'n=0' to ext. -- Chris F.A. Johnson, author <http://shell.cfajohnson.com/> =================================================================== Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress) Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress) ===== My code in this post, if any, assumes the POSIX locale ===== ===== and is released under the GNU General Public Licence ===== |