Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#354571 - 31/08/2012 06:19 Need some help with scripting
StigOE
addict

Registered: 27/10/2002
Posts: 568
Anyone fancy giving me some scripting help?

I have a Linux machine where I have a lot of folders within a folder and I need to make a tar.gz of each of the subfolders and then delete the original subfolder Something like
Code:
Folder
    Sub1
       File1
       File2
       .
       .
       FileN
    Sub2
       File1
       File2
       .
       .
       FileN


After the script I end up with
Code:
Folder
    Sub1.tar.gz
    Sub2.tar.gz


Eventually, I'll probably tar.gz the resulting files as well, ending up with only one file.
I tried yesterday to select all the folders, right-clicking and selecting Create archive. After it had run for about 16 hours, it had done about 2/3 of the selected folders. Seems it will tar.gz one folder, add it to the finished result, tar.gz the next folder, untar the result before adding the second folder. As the tar-file grows, it will take longer and longer time to add one folder. When I stopped it, the resulting file was 12Gb and 178k files...

I think this script will be quite simple, butI'm not very adept at scripting... smile

Top
#354572 - 31/08/2012 07:59 Re: Need some help with scripting [Re: StigOE]
julf
veteran

Registered: 01/10/2001
Posts: 1307
Loc: Amsterdam, The Netherlands
How about this?

Code:

#!/bin/sh

for subdir in *
do
        if [ -d "$subdir" ]
        then
                tar cvfz  "$subdir".tar.gz "$subdir"/*
                #rm -r "$subdir"
        fi
done


(commented out the rm -r - only uncomment when you are sure everything works the way you want!)

Top
#354573 - 31/08/2012 09:17 Re: Need some help with scripting [Re: julf]
StigOE
addict

Registered: 27/10/2002
Posts: 568
Thanks, Julf.

That worked. I removed the verbose output from tar and added a line above it instead (echo "Archiving $subdir"). It got a bit old watching 170k+ lines scroll across the screen... smile

Top
#354574 - 31/08/2012 09:29 Re: Need some help with scripting [Re: StigOE]
julf
veteran

Registered: 01/10/2001
Posts: 1307
Loc: Amsterdam, The Netherlands
Ah, yes, should have removed that after testing...

Note that the script is not very robust - it is a typical 2-minute off-the-cuff thing...

Top
#354576 - 31/08/2012 10:03 Re: Need some help with scripting [Re: julf]
StigOE
addict

Registered: 27/10/2002
Posts: 568
Still, it worked for what I needed.

Top
#354577 - 31/08/2012 14:37 Re: Need some help with scripting [Re: julf]
canuckInOR
carpal tunnel

Registered: 13/02/2002
Posts: 3212
Loc: Portland, OR
Originally Posted By: julf
Ah, yes, should have removed that after testing...

Note that the script is not very robust - it is a typical 2-minute off-the-cuff thing...

It looks more robust than what I'd have suggested:
Quote:

find . -maxdepth 1 -type d | xargs -I% tar --remove-files cf %.tgz %/*

Top
#354580 - 31/08/2012 15:12 Re: Need some help with scripting [Re: StigOE]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
Originally Posted By: StigOE
Eventually, I'll probably tar.gz the resulting files as well, ending up with only one file.

gzipping a tar file that contains only other gzipped files is going to gain you very little beyond using up some CPU time.
_________________________
Bitt Faulk

Top
#354581 - 31/08/2012 20:11 Re: Need some help with scripting [Re: wfaulk]
tanstaafl.
carpal tunnel

Registered: 08/07/1999
Posts: 5546
Loc: Ajijic, Mexico
Originally Posted By: wfaulk
gzipping a tar file that contains only other gzipped files is going to gain you very little beyond using up some CPU time.
You'll probably actually lose a miniscule amount of hard drive space due to the overhead of the zipping program, but you will have a cleaner-looking directory tree.

To some of us OCD types, that is worthwhile. smile

tanstaafl.
_________________________
"There Ain't No Such Thing As A Free Lunch"

Top
#354582 - 31/08/2012 20:33 Re: Need some help with scripting [Re: tanstaafl.]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
You won't have a cleaner tree by doing the extra gzip. You can just have the top level file be a tar file, rather than a gziped tar file.

(you don't really have the equivalent of a tar file in the Windows world, a tar file is kind of like a zip file but with no compression)
_________________________
Remind me to change my signature to something more interesting someday

Top
#354589 - 01/09/2012 04:18 Re: Need some help with scripting [Re: andy]
StigOE
addict

Registered: 27/10/2002
Posts: 568
I may not gain anything by gzipping the resulting files, but it's easier to just select all the files and select Create archive than to do the commands in the shell. I don't really have much problems using the shell, but my collegues on the opposite crew aren't very Linux-savvy.

Top
#354591 - 01/09/2012 06:05 Re: Need some help with scripting [Re: StigOE]
andy
carpal tunnel

Registered: 10/06/1999
Posts: 5916
Loc: Wivenhoe, Essex, UK
Can you not just do a tarred but not zipped archive via the same route ?
_________________________
Remind me to change my signature to something more interesting someday

Top
#354593 - 01/09/2012 13:18 Re: Need some help with scripting [Re: andy]
StigOE
addict

Registered: 27/10/2002
Posts: 568
If you mean using the file browser? Don't know. Maybe by changing the extension when inputting the file name, but it works this way as well. That's good enough for me... smile

Top