Ensuring files are copied correctly
in [Visual Basic]
|
Prev: Re basic to vb
Next: String decoding
From: Dee Earley on 25 May 2010 07:47 On 23/05/2010 14:12, Jim Mack wrote: > CRC32 has the benefit of being very fast to calculate, and if you're > already reasonably sure that the file is correct -- you've done the > obvious, like check byte length -- then CRC32 is probably enough. Even > for very long files, the odds of a mis-hit are minuscule. The reason > is that it's millions of times more likely to generate a false > negative than a false positive. Err, how? An algorithm should always give the same output for the same input, especially when used as a hash, but the different input data may well lead to to the same output, giving the opposite to your statement. -- Dee Earley (dee.earley(a)icode.co.uk) i-Catcher Development Team iCode Systems (Replies direct to my email address will be ignored. Please reply to the group.)
From: Jim Mack on 25 May 2010 10:52 Dee Earley wrote: > On 23/05/2010 14:12, Jim Mack wrote: >> CRC32 has the benefit of being very fast to calculate, and if >> you're already reasonably sure that the file is correct -- you've >> done the obvious, like check byte length -- then CRC32 is probably >> enough. Even for very long files, the odds of a mis-hit are >> minuscule. The reason is that it's millions of times more likely >> to generate a false negative than a false positive. > > Err, how? That was unclear. Of course I don't mean a bad result from good data. What I mean is that if there is corruption, the odds of getting the "right" result anyway are millions to one against. So a match in size, then in CRC, can be taken as confirmation with a very high degree of certainty. -- Jim Mack Twisted tees at http://www.cafepress.com/2050inc "We sew confusion"
From: Tony Toews [MVP] on 25 May 2010 15:21 Dee Earley <dee.earley(a)icode.co.uk> wrote: >> I've had a request to double check that files which my utility as >> copied from the network server to the client PC has been copied >> correctly. Presumably by using a CRC although it looks like I could >> also use SHA or MD5.. >> >> Any comments on Calculating CRC32 With VB >> http://www.vbaccelerator.com/home/vb/code/libraries/CRC32/article.asp >> >> Any better algorithms out there? > >If it works... :) Hey if I can save 13 bytes of a 64Kb program and 10 milliseconds off a 13 second run then it's worth spending three days, right? <smile> >Bear in mind that doing the calculations on the copying end means it >needs to transfer the data twice, possibly showing the same or different >corruption each time. Hmm, good point. I hadn't thought of that. I gotta think about that one a bit more. Glad I asked! >You would really need something else (ideally the server) to generate >the checksums that you then check against on the client. Good idea but that's not going to happen very often as my utility is generally not used by or known to the IT Nazi's. Tony -- Tony Toews, Microsoft Access MVP Tony's Main MS Access pages - http://www.granite.ab.ca/accsmstr.htm Tony's Microsoft Access Blog - http://msmvps.com/blogs/access/ For a convenient utility to keep your users FEs and other files updated see http://www.autofeupdater.com/ Granite Fleet Manager http://www.granitefleet.com/
From: Tony Toews [MVP] on 25 May 2010 16:43 Dee Earley <dee.earley(a)icode.co.uk> wrote: >Bear in mind that doing the calculations on the copying end means it >needs to transfer the data twice, possibly showing the same or different >corruption each time. > >You would really need something else (ideally the server) to generate >the checksums that you then check against on the client. The way I was and am going to continue to do this is the first user to encounter a new/changed file is to compute the CRC/hash of the server copy of the file and save the CRC/hash on the server. However I will give the utility administrator the ability to recompute the CRC/hashes on demand, show any differences and save those on the server. This should handle the situation where a user workstation has a problem and calculates them incorrectly. I will also only display a message to the user and allow them to continue if desired. And if strange things happen on the workstation oh well, they were warned. Tony -- Tony Toews, Microsoft Access MVP Tony's Main MS Access pages - http://www.granite.ab.ca/accsmstr.htm Tony's Microsoft Access Blog - http://msmvps.com/blogs/access/ For a convenient utility to keep your users FEs and other files updated see http://www.autofeupdater.com/ Granite Fleet Manager http://www.granitefleet.com/
From: Jason Keats on 9 Jun 2010 07:43
Tony Toews [MVP] wrote: > Jason Keats<jkeats(a)melbpcDeleteThis.org.au> wrote: > >> I've used the following for SHA1: >> http://vb.wikia.com/wiki/SHA-1.bas >> >> It seems to be faster than the CRC32 implementation you're using, is >> less likely to produce duplicate values (if that's important) and it's >> certainly more secure (only important if you're going to use it with >> passwords, etc.). > > Ah, interesting. I'll do some timing tests. I've got a 300 Mb Access > file which would be suitable for such.<smile> > > The concern is about unspecified and unknown oddities happening > between the file server and the client PC when copying down files. > Which is the primary purpose of my utility. These oddities should > never happen. Of course.<smile> If the file server, network, and > client PC are working just fine. And they probably will 99.99999% of > the time. > > Tony For the record, the SHA1 function I mentioned is not faster than CRC32. But you knew that already, right? CRC32 is about 10x faster. I made a mistake in my testing - hashing the file name, rather than the file! |