Ubuntu-12.04 [SOLVED]: How to delete duplicate files with the same size and date?

Ubuntu-12.04 [SOLVED]: How to delete duplicate files with the same size and date?

Home Forums Ubuntu 12.04 Ubuntu-12.04 [SOLVED]: How to delete duplicate files with the same size and date?

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #24535

    Anonymous

    QuestionQuestion

    For a reason I do not know, Postfix decided to duplicate all the emails I received in a specific folder after my return of vacation. Instead of having ~1000 emails in that folder, I now have 401120! That’s way too much for Thunderbird, so the mail client refuse to download any of them.

    When I ssh into the mail server, I can see the following:

    -rw-------   68 user user     1415 Aug  3 17:45 502716210.M218413P5883.mail.domain.com,S=1415,W=1445:2,
    -rw-------   68 user user     1415 Aug  3 17:45 502716327.M661584P5946.mail.domain.com,S=1415,W=1445:2,
    -rw-------   68 user user     1415 Aug  3 17:45 502716331.M210364P5948.mail.domain.com,S=1415,W=1445:2,
    -rw-------   68 user user     1415 Aug  3 17:45 502716412.M161709P6009.mail.domain.com,S=1415,W=1445:2,
    -rw-------   42 user user     2393 Aug  3 17:44 502714577.M151447P4492.mail.domain.com,S=2393,W=2447:2,
    -rw-------   42 user user     2393 Aug  3 17:44 502715910.M338737P5564.mail.domain.com,S=2393,W=2447:2,
    -rw-------   42 user user     2393 Aug  3 17:44 502715914.M370949P5566.mail.domain.com,S=2393,W=2447:2,
    -rw-------   42 user user     2393 Aug  3 17:44 502715964.M213570P5615.mail.domain.com,S=2393,W=2447:2,
    

    I can see that Postfix gave all files their own name, but when I look with nano, I can see that all files with the same size and date are identical.

    How can I delete all the duplicate files, to get only one file for each size-date?

    #24536

    Anonymous

    Accepted AnswerAnswer

    copy this to a file (say: kill_dupes) and add run rights (chmod a+x kill_dupes) and run it

    #!/bin/bash
    
    ls -gGlt | sort -k4,6 -k3,3n > filelist.txt
    
    awk '
    BEGIN   {
            printf("#!/bin/bashnn");
            filename0 = $7;
            filesize0 = $3;
            filedate0 = $4$5$6;
        }
    
    NF>2    {
            filename1 = $7;
            filesize1 = $3;
            filedate1 = $4$5$6;
            if(filesize1==filesize0 && filedate1==filedate0) {
                printf("rm %sn", filename1);
            } else {
                filename0 = filename1;
                filesize0 = filesize1;
                filedate0 = filedate1;
            }       
        }
    ' filelist.txt > rmdupes.sh
    

    then verify the content of rmdupes.sh file

    head rmdupes.sh
    

    please check it before actually running it.

    (is 502715964.M213570P5615.mail.domain.com,S=2393,W=2447:2 an actual filename?)

    Source: https://askubuntu.com/questions/946214/how-to-delete-duplicate-files-with-the-same-size-and-date
    Author: Macsek
    Creative Commons License
    This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.