Reducing dump file size

Submitted by mattkesseler on
j-kerbl

Hi Matt,

have a look in http://www.cfdem.com/media/DEM/docu/dump.html

here it states:

If the filename ends with ”.gz”, the dump file (or files, if “*” or “%” is also used) is written in gzipped format. A gzipped dump file will be about 3x smaller than the text version, but will also take longer to write.

Give it a try!

Cheers,
Josef

Thu, 30 Mar 2017 - 13:17 Permalink
mattkesseler

In reply to by j-kerbl

Hi Josef,

Thanks, I thought there would be some common sense thing like this that I'd overlook. :)

I'll let you know if I have any problems with this.

Matt.

Thu, 30 Mar 2017 - 14:21 Permalink
mattkesseler

In reply to by j-kerbl

Just wanted to double check whether this approach would work with dump custom/vtk? Or would I need to save as a gz, then unzip and convert to vtk once I have downloaded the files?

Matt.

Thu, 30 Mar 2017 - 14:24 Permalink
j-kerbl

Hi Matt,

no, the direct gzip works only with regular dumps. It isn't implemented for the vtks yet. If you post-process on the same machine, you can alternatively try the binary dumps of vtk, which need less space too. However, transferring them to anther machine might be dangerous.

Cheers,
Josef

Wed, 5 Apr 2017 - 15:18 Permalink

Hi all. I am currently doing a simulation that takes dump files of a 1.5 million particle granular slide every 2000 timesteps. Each ascii .vtk file takes up about 150 MB and the granular slide concludes in about 5000000 timesteps; i.e. the final file size of the dataset is roughly 375 GB. I am looking for ways to reduce this filesize while preserving the same data as each save interval corresponds to a frame in my laboratory recording video. Is there any way for LIGGGHTS to dump into a compressed folder for instance, and how much space would that save?