Securely deleting files

[white scorpion]

Maybe I misread somewhere, but you don't appear to be resetting the seed before you call GenerateRandomByte again, which means you will get a different number each time.
This is bad for two reasons. The whole point of overwriting with an inverse pattern is to flatten out the magnetic echos, if you overwrite with the inverse of another random number that's just the same as overwriting with a random number. Secondly, while it may seem like a good idea to write different values, this creates interference between boundaries, but you're supposed to be avoiding interference and flattening out the magnetic signals. So it's still a better idea to fill the whole file with the same number, and then its inverse..

You're totally right.. Somewhere in all changes i made to this code i must have forgotten to put the

Code Select Expand


mov eax,backup
mov myseed,eax

part back in the loops...
Will fix right away!

Generate a random value, save it, use that. Flush the file. NOT the previous value, save that, and use it. This is also obviously more effiecent than calling GenerateRandomByte each time.

This won't be more efficient since would have to call WriteFile for every byte (or DWORD) that is generated. Calling GenerateRandomByte for each byte is more efficient then WriteFile.

EDIT THis should be it then.

Code Select Expand


RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
    LOCAL hFile:DWORD
    LOCAL inputfilesize:DWORD
    LOCAL backup:DWORD
    LOCAL BytesWr:DWORD
    LOCAL shredbuf[1024*10]:BYTE
    LOCAL buffer_times:DWORD
    LOCAL my_remainder:DWORD

    cmp number_of_times,0
    jnz @F 
    xor eax,eax
    dec eax
    ret
@@:
    invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
    FILE_ATTRIBUTE_NORMAL or FILE_FLAG_NO_BUFFERING, NULL
    .if eax==NULL
@@:
        invoke CloseHandle,hFile
        xor eax,eax
        dec eax
    ret
    .endif   
    mov hFile,eax
    invoke GetFileSize,eax,NULL
    mov ebx,512
    xor edx,edx
    div ebx
    .if edx==0
        mul ebx
    .else
        mul ebx
        add eax,512
    .endif       
    mov inputfilesize,eax
    invoke GetTickCount
    mov backup,eax
    mov myseed,eax
   .if inputfilesize<=sizeof shredbuf
        mov edx,number_of_times
;----Rounds Small Start -----       
times_loop_small:
        push edx       
        mov ecx,inputfilesize 
        lea esi,shredbuf       
@@:
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        mov ecx,inputfilesize
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        pop edx
        dec edx
        jnz times_loop_small   
;----Rounds Small End ------- 
    .else
        mov eax,inputfilesize
        mov ebx,sizeof shredbuf
        xor edx,edx
        div ebx
        mov buffer_times,eax
        mov my_remainder,edx
        mov edx,number_of_times
;----Rounds Big Start -----       
times_loop_big:
        push edx
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop1: 
        push eax     
        mov ecx,sizeof shredbuf
        lea esi,shredbuf       
@@:       
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop1
        mov ecx,my_remainder
        .if ecx==0
            jmp second_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL   
second_round: 
mov eax,backup
        mov myseed,eax           
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop2:   
        push eax   
        mov ecx,sizeof shredbuf
        lea esi,shredbuf
@@:       
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop2
        mov ecx,my_remainder
        .if ecx==0
            jmp finish_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL       
finish_round:
        pop edx
        dec edx
        jnz times_loop_big
;----Rounds Small End -------         
    .endif       
    invoke CloseHandle,hFile
    invoke DeleteFile,file_to_delete
    xor eax,eax
    ret
RemoveFile endp
I'm really starting to get confused with this code since i 've rewritten it so many times now  :bg

[Tedd]

Generate a random value, save it, use that. Flush the file. NOT the previous value, save that, and use it. This is also obviously more effiecent than calling GenerateRandomByte each time.

This won't be more efficient since would have to call WriteFile for every byte (or DWORD) that is generated. Calling GenerateRandomByte for each byte is more efficient then WriteFile.

What?! :dazzled:

Code Select Expand

repeat
    value = GenerateRandomByte
    repeat
        fill the whole buffer with 'value'
        write buffer chunks up to size of file
        flush file
        value = NOT(value)
        fill the whole buffer with 'value'
        write buffer chunks up to size of file
        flush file
    until bored
until bored

Why would you need to call WriteFile for every byte? You do need to call it for every chunk, and then flush the file, so that it's actually written to disk - which is exactly what you want otherwise the driver will cache it and only the last write will have any effect on the physical disk.

[Tedd]

Code Select Expand

    invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
    FILE_ATTRIBUTE_NORMAL or FILE_FLAG_NO_BUFFERING, NULL
    .if eax==NULL
@@:
        invoke CloseHandle,hFile
        xor eax,eax
        dec eax
    ret
    .endif   
    mov hFile,eax
No!
What is the value of hFile when CreateFile returns NULL? And if the file wasn't opened successfully, why are you attempting to close it again??
Not that this is correct anyway - CreateFile returns INVALID_HANDLE_VALUE (-1) upon failure.

Code Select Expand

    invoke GetFileSize,eax,NULL
    mov ebx,512
    xor edx,edx
    div ebx
    .if edx==0
        mul ebx
    .else
        mul ebx
        add eax,512
    .endif       
    mov inputfilesize,eax
..simplified..

Code Select Expand

    invoke GetFileSize,eax,NULL
    add eax,1FFh          ;512 - 1 (so rounds up to full sector)
    and eax,0FFFFFE00h    ;ffffffffh - 512 (round to sector length)
    mov inputfilesize,eax

Code Select Expand

@@:
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
  .
  .
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
Nooooooo.. ::)
Use ONE value for the entire file. Then use its inverse for the ENTIRE file.
What you're currently doing is no better than filling the file with random data numerous times - which may be fairly effective - but then your program should be considerably simpler.


What's the deal with the 'rounds'?? You're writing buffer-fulls, so a simple 'how-much-left-until-end-of-file' counter that you subtract from should work better than doubling the code to handle full-buffer/not-full-buffer.


My advice: delete THIS file, start over again - taking everything you've learned - and write a much better, more organised, and less messy, correct program.

[Darrel]

Corrected DigitalShredder. It now handles files of size 0.



[attachment deleted by admin]

[white scorpion]

Tedd, flushing the file to disk doesn't necessarily write to disk (it could be cached), hence the writefile.
I know there are some stupid mistakes in above code since the check you see (if eax==NULL) was from another APi call. I must have left it there when changing the code.
The code in the program itself already has been updated almost immediately after my previous post. I just forgot to change it here.

here's the current function. I must say that i still have to change the LOCAL variable of 10kb to either a global, or using HeapAlloc.

Code Select Expand


;****************************************************************
; the function which securely removes the inputfile
;****************************************************************
RemoveFile proc file_to_delete:DWORD,number_of_times:DWORD
    LOCAL hFile:DWORD
    LOCAL inputfilesize:DWORD
    LOCAL backup:DWORD
    LOCAL BytesWr:DWORD
    LOCAL shredbuf[1024*10]:BYTE
    LOCAL buffer_times:DWORD
    LOCAL my_remainder:DWORD

    cmp number_of_times,0
    jnz @F 
    xor eax,eax
    dec eax
    ret
@@:
    invoke CreateFile,file_to_delete,GENERIC_WRITE, 0, NULL, OPEN_EXISTING,\
    FILE_ATTRIBUTE_NORMAL or FILE_FLAG_NO_BUFFERING, NULL
    .if eax==INVALID_HANDLE_VALUE
@@:
        xor eax,eax
        dec eax
    ret
    .endif   
    mov hFile,eax
    invoke GetFileSize,eax,NULL
    mov ebx,512
    xor edx,edx
    div ebx
    .if edx==0
        mul ebx
    .else
        mul ebx
        add eax,512
    .endif       
    mov inputfilesize,eax
    invoke GetTickCount
    mov backup,eax
    mov myseed,eax
   .if inputfilesize<=sizeof shredbuf
        mov edx,number_of_times
;----Rounds Small Start -----       
times_loop_small:
        push edx       
        mov ecx,inputfilesize 
        lea esi,shredbuf       
@@:
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        mov ecx,inputfilesize
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        invoke WriteFile,hFile,addr shredbuf,inputfilesize,addr BytesWr,NULL
        pop edx
        dec edx
        jnz times_loop_small   
;----Rounds Small End ------- 
    .else
        mov eax,inputfilesize
        mov ebx,sizeof shredbuf
        xor edx,edx
        div ebx
        mov buffer_times,eax
        mov my_remainder,edx
        mov edx,number_of_times
;----Rounds Big Start -----       
times_loop_big:
        push edx
        mov eax,backup
        mov myseed,eax
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop1: 
        push eax     
        mov ecx,sizeof shredbuf
        lea esi,shredbuf       
@@:       
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop1
        mov ecx,my_remainder
        .if ecx==0
            jmp second_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL   
second_round: 
mov eax,backup
        mov myseed,eax           
        invoke SetFilePointer,hFile,0,0,FILE_BEGIN
        mov eax,buffer_times
bufloop2:   
        push eax   
        mov ecx,sizeof shredbuf
        lea esi,shredbuf
@@:       
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,sizeof shredbuf,addr BytesWr,NULL
        pop eax
        dec eax
        jnz bufloop2
        mov ecx,my_remainder
        .if ecx==0
            jmp finish_round
        .endif
        lea esi,shredbuf
@@:
        invoke GenerateRandomByte
        not eax
        mov byte ptr [esi],al
        inc esi
        dec ecx
        jnz @B
        invoke WriteFile,hFile,addr shredbuf,my_remainder,addr BytesWr,NULL       
finish_round:
        pop edx
        dec edx
        jnz times_loop_big
;----Rounds Small End -------         
    .endif       
    invoke CloseHandle,hFile
    invoke DeleteFile,file_to_delete
    xor eax,eax
    ret
RemoveFile endp



;****************************************************************
;The function which will generate random bytes for the deletion of inputfile
;****************************************************************
GenerateRandomByte proc
        mov eax,myseed
        mov ebx,127923
        mul ebx
        xor eax,479573
        xor eax,edx
        mov myseed,eax
        xor edx,edx
        div mybase
        mov eax,edx
        ret
GenerateRandomByte endp

Let me know what you think of this one.

[Tedd]

Tedd, flushing the file to disk doesn't necessarily write to disk (it could be cached), hence the writefile.

No, that is exactly what it does! Flushing a file writes the data to disk, that is the whole purpose of flushing! It empties all buffers for the file and causes them to be written to the disk. WriteFile writes the data to file-buffers almost always (yes, I see you've used FILE_FLAG_NO_BUFFERING).

So you've corrected one mistake, very good, but what about everything else I mentioned? Most importantly that your entire approach is just wrong.

::) I give up.

[white scorpion]

Please read this thread on asmcommunity. As you can see i've followed up almost every advise i've been given there.

Now you are telling me to start using FlushViewOfFile again?

How do i know whom to listen to? If one respected coder says one thing (with explanation), and the other one says another thing ( with explanation), then this makes it pretty difficult to make the right choice.
(on this particular subject i've messured the I/O writes to disk with both options and WriteFile gives as result as much bytes as 10 times the filesize. FlushViewOfFile on the other hands doesn't even show 1 time the filesize as writes to disk.
This makes it even more difficult for me to see which one is best. When you look at the harddisk led on your computer with both functions, you can see that with WriteFile the led is constantly burning and you can hear the harddisk working, with FlushViewOfFile this isn't the case.

I totally agree with you on one thing: start from scratch and rewrite the function.
But i first want a fully working and correct function before i start writing it all over again with the risk of having to edit that one dozens of times again, this doesn't have anything to do with discarding your advise  :P

[Tedd]

The intention of "flushing" is to empty buffers to disk/storage to ensure a correct and up-to-date version is stored. What windows actually does in the implementations of certain functions is a different matter (FlushViewOfFile 'should' flush dirty buffers, but maybe it does it lazily), and not something you have enough control over :bdg However, what you do want to do is flush the file -- how you achieve that is a different matter :P But since you have FILE_FLAG_NO_BUFFERING then there shouldn't a problem with buffering when using WriteFile, so it's all good (but without that hint, it would most likely buffer in all cases.)
See also: FlushFileBuffers :wink (then again, it could still be lazy. Stick with writefile+hint)

The problem (as I see it) with your current program is that it fills the file with DIFFERENT bytes. In my (obviously deranged) opinion this is the wrong thing to do. Choose one byte and fill the whole file with just that value. Then overwrite that with the inverse. Repeat ad nauseum, for different values. The idea is to flatten the magnetic fields on the surface of the disk in order to make recovery difficult/impossible. If you write different values to each byte, this will cause interference, which is precisely what is used for advanced recovery and why it's possible to recover a file even after it has been overwritten (hence the purpose of this program!) I'm sure overwriting the file with anything a large number of times will eventually make recovery impossible, but somehow trying to 'flatten' the data seems more effective (to me and my deranged brain.)

[white scorpion]

I understand what you mean, but in theory this shouldn't matter.

whether i write: 11101101 and then 00010010 at every byte, or 10101101 vs 01010010 on one byte position and then write another byte and its reverse on another location shouldn't really matter.
In both cases you have all 1's and all 0's on every possible location.

It would be easier to write a program which just writes 0x00 and 0xFF at every byte location, but these are often the first rounds in these programs, the rest is filled with random bytes.
I still am not sure why there are several different techniques, since in the end it shouldn't matter since you are writing to every bit location both 1's and 0's.

The way my program does this is what is recommended to me to do this.
I haven't got the technical equipment to verify which technique would be best, so i decided to pick the one which is used most, random bytes.
This of course makes the program a bit more complicated then simply writing all 11111111's and 00000000's, but if i think about it:

-if a 10 times overwritten file can be retrieved within 10 hours (just a number, no real value i think) of magnetical workings before it is completely gone, then it would be better to disturb this by writing random bytes rather then one and the same byte since the difference at locations can be easier retrieved with all same bytes.
This is just my thought.  i could very well be wrong.

[Tedd]

whether i write: 11101101 and then 00010010 at every byte, or 10101101 vs 01010010 on one byte position and then write another byte and its reverse on another location shouldn't really matter.
In both cases you have all 1's and all 0's on every possible location.

But it does matter. It doesn't matter much to the data-tracks - which is where your bytes are actually written to, but it does matter to the gaps between these tracks - which is where the interference is caused and what is used to retrieve the lost data. If you only consider the data-tracks, then simply overwriting the file once will erase the data, but then why go to the trouble of filling with random data and inverses multiple times? The point is that it's the gaps that are used for recovery, but you don't have write access to the gaps, so you have to write multiple times in order to influence the 'echos' in these gaps to make data recovery more difficult.

[white scorpion]

I know how hardware recovery works in theory, but this doesn't explain why writing a same byte on every location in the file is better then writing random bytes.
Both also reversed of course.

Perhaps i will use both techniques just to be sure.
Write 0x00 & 0xFF both 3 times, and then writing random byte and its reverse both 2 times.
Total times overwritten will still be 10.
So i could do it like this:
1st: 0x00
2nd: 0xFF
3rd: random
4th reverse of 3
5th: 0x00
6th: 0xFF
7th: random
8th: reverse of 7
9th 0x00
10th: 0xFF

Would this be better in your opinion?

[Tedd]

No, not really.
Pick one method and stick with it.

[Mark Jones]

Perhaps research "Magnetic Hysteresis." This is after all, what you're trying to defeat. :thumbu

[white scorpion]

Absolutely worth researching!
I've downloaded some pdf's on this topic which i will start reading asap.

Well, the pdf's i downloaded are lab manuals which doesn't contain any info which i can use :(

I've rewritten the complete function. It now uses 0x00 & 0xFF to overwrite the file.
depending on the number of times given as argument it will overwrite the file like this:

1st:  0x00
2nd: 0xFF
3rd: 0x00
4th: 0xFF
5th: 0x00
6th: 0xFF
....

Attached you will find the function. please let me know what to improve. I've used HeapAlloc with 64k allocation.



[attachment deleted by admin]

[asmfan]

Allow me to post my version of procedure (the idea is the main thing)
I wrote it in parallel with White Scorp. Scorp, you can freely use it in your program, modify and add...
it's basically for compare our code style...

[attachment deleted by admin]