DZone Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world

Ahmed Talaat

Ahmed has posted 1 posts at DZone. View Full User Profile

Avoiding Tar's (or Any Other Commandline's) Argument List Too Long Error

10.10.2006

| 17299 views |

        When you try to tar a large number of files (cp, mv, or anything else) you may hit the limit of ARG_MAX. You can overcome this by generating a list of files then feeding it to tar as follows:

# generate the file list
find -iname '*.ext' > fileList.txt

#run tar passing it the file list
tar czvf archive.tar.gz --files-from fileList.txt

# a shorthand of this would be:
find . -name '*.ext' -print | tar -cvzf archive.tar.gz --files-from -

Tags:

shell
TAR
Unix

Comments

Snippets Manager replied on Tue, 2007/02/20 - 2:39pm

xargs might invoke the command multiple times (read the man page for xargs). This isn't what you want when you are trying to build an archive. Although it is possible to update a pre-existing archive, this requires different arguments to tar (which xargs cannot provide), not to mention the fact that the archive is compressed and tar simply cannot work in this fashion. So the original poster's method is correct. But anyway, I hate tar files that unarchive into the current directory. I think it's always better to place all the files into their own directory and archive that (which eliminates the need for this piece of shell code in the first place).