Conversion from UTF32 to UTF8 for review
in [MFC]
|
From: Peter Olcott on 2 Jun 2010 17:58 On 6/2/2010 4:40 PM, Joseph M. Newcomer wrote: > See below... > On Wed, 02 Jun 2010 10:52:28 -0500, Peter Olcott<NoSpam(a)OCR4Screen.com> wrote: > >> On 6/1/2010 1:04 PM, Joseph M. Newcomer wrote: >>> See below... >>> On Tue, 01 Jun 2010 10:34:40 -0500, Peter Olcott<NoSpam(a)OCR4Screen.com> wrote: >>> >>>> If you measure my code against the incorrect standard that it is >>>> specifically encoded to be the fastest possible encoding, even then it >>>> is not abysmal. All of the performance improvements that you suggested >>>> don't result in as much as a doubling in speed. >>>> http://www.ocr4screen.com/UTF8.cpp >>>> >>>> From benchmarking my code against the code that Hector posted a link to >>>> http://bjoern.hoehrmann.de:80/utf-8/decoder/dfa/ >>>> This other code was only 37% faster. >>> ***** >>> "Only" 37% faster? Actually 37% is a pretty big number in terms of performance! Most >>> attempts to "improve" performance are lucky if they get single-digit percentage >>> improvement. As someone who spent a nontrivial amount of his life worrying about these >>> issues, I can say that 37% is a SUBSTANTIAL performance improvement! >>> >>> And if it were 1% faster, it would still prove your code was not the fastest possible. But >>> 37%? You aren't even in the running in this contest! >>> joe >> >> I did a better job of benchmarking and I made changes that slowed my >> code down, removing the gotos and using a std::vector<uint8_t> instead >> of NUL the terminated uint8_t*. His code is now 267% faster than mine. > **** > Code that is faster by a factor of 3 means your code is not even worth looking at. Nobody > will care whether it is correct or not, if it is nearly 3 times slower than an > alternative. > > We would sweat blood to get 10% improvement. So if you are not competitive with some > other code by a factor of 3, don't even waste your time. Use the other code. > joe Your experience it apparently not at all typical. Much of the code that I have reviewed from places that I have worked don't even aim for anything better than about 500% slower than the fastest code. Most often this code is about 20-fold or more slower than the fastest code. I once re-wrote a method that my supervisor wrote for a 120-fold improvement in speed. From 2.5 minutes to 1.25 seconds wall clock time. The problem is that many programmers don't even think about making their code fast, they only think about getting it working. They figure that once its working, then they can make it faster, but, this never happens because as soon as it works, they get their next assignment. > **** >> >> The benchmark timed how long it took to decode 100 instances of the >> entire Unicode set encoded as UTF-8. > Joseph M. Newcomer [MVP] > email: newcomer(a)flounder.com > Web: http://www.flounder.com > MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 2 Jun 2010 18:14 On 6/2/2010 4:58 PM, Peter Olcott wrote: > On 6/2/2010 4:40 PM, Joseph M. Newcomer wrote: >> See below... >> On Wed, 02 Jun 2010 10:52:28 -0500, Peter >> Olcott<NoSpam(a)OCR4Screen.com> wrote: >> >>> On 6/1/2010 1:04 PM, Joseph M. Newcomer wrote: >>>> See below... >>>> On Tue, 01 Jun 2010 10:34:40 -0500, Peter >>>> Olcott<NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> If you measure my code against the incorrect standard that it is >>>>> specifically encoded to be the fastest possible encoding, even then it >>>>> is not abysmal. All of the performance improvements that you suggested >>>>> don't result in as much as a doubling in speed. >>>>> http://www.ocr4screen.com/UTF8.cpp >>>>> >>>>> From benchmarking my code against the code that Hector posted a >>>>> link to >>>>> http://bjoern.hoehrmann.de:80/utf-8/decoder/dfa/ >>>>> This other code was only 37% faster. >>>> ***** >>>> "Only" 37% faster? Actually 37% is a pretty big number in terms of >>>> performance! Most >>>> attempts to "improve" performance are lucky if they get single-digit >>>> percentage >>>> improvement. As someone who spent a nontrivial amount of his life >>>> worrying about these >>>> issues, I can say that 37% is a SUBSTANTIAL performance improvement! >>>> >>>> And if it were 1% faster, it would still prove your code was not the >>>> fastest possible. But >>>> 37%? You aren't even in the running in this contest! >>>> joe >>> >>> I did a better job of benchmarking and I made changes that slowed my >>> code down, removing the gotos and using a std::vector<uint8_t> instead >>> of NUL the terminated uint8_t*. His code is now 267% faster than mine. >> **** >> Code that is faster by a factor of 3 means your code is not even worth >> looking at. Nobody >> will care whether it is correct or not, if it is nearly 3 times slower >> than an >> alternative. >> >> We would sweat blood to get 10% improvement. So if you are not >> competitive with some >> other code by a factor of 3, don't even waste your time. Use the other >> code. >> joe > > Your experience it apparently not at all typical. Much of the code that > I have reviewed from places that I have worked don't even aim for > anything better than about 500% slower than the fastest code. Most often > this code is about 20-fold or more slower than the fastest code. I once > re-wrote a method that my supervisor wrote for a 120-fold improvement in > speed. From 2.5 minutes to 1.25 seconds wall clock time. > > The problem is that many programmers don't even think about making their > code fast, they only think about getting it working. They figure that > once its working, then they can make it faster, but, this never happens > because as soon as it works, they get their next assignment. > >> **** >>> >>> The benchmark timed how long it took to decode 100 instances of the >>> entire Unicode set encoded as UTF-8. >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm > Certainly the early compiler technology from Microsoft was not the way that you suggest. The early MS compilers took 60-fold longer than Borland, and the code they produced took 300% more time to run.
From: Liviu on 2 Jun 2010 19:26 "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote... > On 6/2/2010 4:58 PM, Peter Olcott wrote: >> On 6/2/2010 4:40 PM, Joseph M. Newcomer wrote: >>> **** >>> >>> We would sweat blood to get 10% improvement. So if you are not >>> competitive with some other code by a factor of 3, don't even waste >>> your time. Use the other code. >> >> Your experience it apparently not at all typical. Much of the code >> that I have reviewed from places that I have worked don't even aim >> for anything better than about 500% slower than the fastest code. Just because that's been your personal experience, or whatever you made out of it, that doesn't make it any more "typical" than Joe's. >> The problem is that many programmers don't even think about making >> their code fast, they only think about getting it working. They >> figure that once its working, then they can make it faster [...] That is indeed the generally accepted approach for the implementation phase, once the design is sound and solid. Correctly implementing the design should be the first concern. By the way, did you fix the validation bug still present in the latest code you submitted in the other thread? > Certainly the early compiler technology from Microsoft was not the way > that you suggest. The early MS compilers took 60-fold longer than > Borland, and the code they produced took 300% more time to run. As a blanket statement the above is not only false and unproved, but also provably wrong and at the opposite end of the truth. Once again, you are pulling nonsense out of thin air, and then wonder why people can't take you seriously. Back in the summer of '91 I have run extensive speed tests between the C compilers du jour, for what Joe would call that 10% edge in a calculation-intensive application. MS v6 consistently beat Borland v2, and the now forgotten Zortech v3 beat both in floating point emulation mode (no, Virginia, CPUs didn't always have an FPU). Liviu
From: Joseph M. Newcomer on 2 Jun 2010 19:33 See below.... On Wed, 02 Jun 2010 16:58:49 -0500, Peter Olcott <NoSpam(a)OCR4Screen.com> wrote: >On 6/2/2010 4:40 PM, Joseph M. Newcomer wrote: >> See below... >> On Wed, 02 Jun 2010 10:52:28 -0500, Peter Olcott<NoSpam(a)OCR4Screen.com> wrote: >> >>> On 6/1/2010 1:04 PM, Joseph M. Newcomer wrote: >>>> See below... >>>> On Tue, 01 Jun 2010 10:34:40 -0500, Peter Olcott<NoSpam(a)OCR4Screen.com> wrote: >>>> >>>>> If you measure my code against the incorrect standard that it is >>>>> specifically encoded to be the fastest possible encoding, even then it >>>>> is not abysmal. All of the performance improvements that you suggested >>>>> don't result in as much as a doubling in speed. >>>>> http://www.ocr4screen.com/UTF8.cpp >>>>> >>>>> From benchmarking my code against the code that Hector posted a link to >>>>> http://bjoern.hoehrmann.de:80/utf-8/decoder/dfa/ >>>>> This other code was only 37% faster. >>>> ***** >>>> "Only" 37% faster? Actually 37% is a pretty big number in terms of performance! Most >>>> attempts to "improve" performance are lucky if they get single-digit percentage >>>> improvement. As someone who spent a nontrivial amount of his life worrying about these >>>> issues, I can say that 37% is a SUBSTANTIAL performance improvement! >>>> >>>> And if it were 1% faster, it would still prove your code was not the fastest possible. But >>>> 37%? You aren't even in the running in this contest! >>>> joe >>> >>> I did a better job of benchmarking and I made changes that slowed my >>> code down, removing the gotos and using a std::vector<uint8_t> instead >>> of NUL the terminated uint8_t*. His code is now 267% faster than mine. >> **** >> Code that is faster by a factor of 3 means your code is not even worth looking at. Nobody >> will care whether it is correct or not, if it is nearly 3 times slower than an >> alternative. >> >> We would sweat blood to get 10% improvement. So if you are not competitive with some >> other code by a factor of 3, don't even waste your time. Use the other code. >> joe > >Your experience it apparently not at all typical. Much of the code that >I have reviewed from places that I have worked don't even aim for >anything better than about 500% slower than the fastest code. Most often >this code is about 20-fold or more slower than the fastest code. I once >re-wrote a method that my supervisor wrote for a 120-fold improvement in >speed. From 2.5 minutes to 1.25 seconds wall clock time. > >The problem is that many programmers don't even think about making their >code fast, they only think about getting it working. They figure that >once its working, then they can make it faster, but, this never happens >because as soon as it works, they get their next assignment. **** Maybe it was because we wrote code we thought was as fast as possible, then had to make it even faster. We used to refer to it as going into "blood from turnips" mode. **** > >> **** >>> >>> The benchmark timed how long it took to decode 100 instances of the >>> entire Unicode set encoded as UTF-8. >> Joseph M. Newcomer [MVP] >> email: newcomer(a)flounder.com >> Web: http://www.flounder.com >> MVP Tips: http://www.flounder.com/mvp_tips.htm Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Peter Olcott on 2 Jun 2010 22:40
On 6/2/2010 6:26 PM, Liviu wrote: > "Peter Olcott"<NoSpam(a)OCR4Screen.com> wrote... >> On 6/2/2010 4:58 PM, Peter Olcott wrote: >>> On 6/2/2010 4:40 PM, Joseph M. Newcomer wrote: >>>> **** >>>> >>>> We would sweat blood to get 10% improvement. So if you are not >>>> competitive with some other code by a factor of 3, don't even waste >>>> your time. Use the other code. >>> >>> Your experience it apparently not at all typical. Much of the code >>> that I have reviewed from places that I have worked don't even aim >>> for anything better than about 500% slower than the fastest code. > > Just because that's been your personal experience, or whatever you > made out of it, that doesn't make it any more "typical" than Joe's. No what probably makes Joe's case atypical is the degree of knowledge and expertise that he has. Very few developers have Phds in computer science. > >>> The problem is that many programmers don't even think about making >>> their code fast, they only think about getting it working. They >>> figure that once its working, then they can make it faster [...] > > That is indeed the generally accepted approach for the implementation > phase, once the design is sound and solid. Correctly implementing the > design should be the first concern. > > By the way, did you fix the validation bug still present in the latest > code you submitted in the other thread? There was no validation bug that I am aware of, and in fact the code produces identical results to the excellent link that Hector posted. I did however add the enhancement of rejecting 0xD800 to 0xDFFF. > >> Certainly the early compiler technology from Microsoft was not the way >> that you suggest. The early MS compilers took 60-fold longer than >> Borland, and the code they produced took 300% more time to run. > > As a blanket statement the above is not only false and unproved, but > also provably wrong and at the opposite end of the truth. Turbo Pascal 3.0 against the MS Pascal compiler at the time. I specifically timed these results myself. > > Once again, you are pulling nonsense out of thin air, and then wonder > why people can't take you seriously. > > Back in the summer of '91 I have run extensive speed tests between > the C compilers du jour, for what Joe would call that 10% edge in > a calculation-intensive application. MS v6 consistently beat Borland > v2, and the now forgotten Zortech v3 beat both in floating point > emulation mode (no, Virginia, CPUs didn't always have an FPU). Maybe by '91 this was the case. This was not the case when Turbo Pascal 3.0 was being sold. I still have a copy of the original manual that came with the compiler that I bought brand new for $31.00. > > Liviu > > |