std::search versus strstr for std::vector<char>
in [C++]
|
From: Andrew on 10 Mar 2010 04:51 I am testing and performance tuning some code written by a colleague. The code intent is to read the next XML node from an open file stream. The node is returned as a std::string with the begin and end tags suitable for parsing by xercesc. It works but for some strange reason it runs like a dog with no legs when built using Visual Studio in debug. It runs quite slowly in release mode as well. When using the GNU compiler under cygwin it runs like greased lightning whether compiled with -g or not. When using the GNU compiler I did some profiling using -p and gprof and just as I expected, all the time is spent searching for the tags. This is done using std::search. The buffer to be searched is a std::vector<char>. This is ok, it is the obvious bulk of the work and I exected gprof to say this is where it was spending the time. My test programs takes about 2 seconds to run. But when I use studio in debug the picture changes. I used Very Sleepy to do sample-based profiling. It spends all its time entering and leaving critical regions in locks that are taken in the implementation of std::search and the iterators passed to it. The time taken jumps up to nearly 3 minutes! In release it takes about 5 seconds, which is much better than 3 minutes but still more than twice as slow as GNU in debug. I have spent some time with this colleague, trying to persaude him to move from C-like coding and manual memory mgmt to using more C++-like code and self-managing objects. Now that I've finally achieved this I am wondering if we might be better off using strstr on the data buffer associated with the std::vector<char>. He was using strstr before but with various problems (like not ensuring it is NUL-terminated). I advised on using the STL and std C++ facilities rather than C functions. Has anyone else looked at the performance of std::search versus strstr on the underlying buffer? Why would one implementation takes so many locks that it cripples performance? What would people here do? Revert to using the C-like strstr......? Regards, Andrew Marlow -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Martin B. on 11 Mar 2010 02:55 Andrew wrote: > I am testing and performance tuning some code written by a colleague. > The code intent is to read the next XML node from an open file stream. > The node is returned as a std::string with the begin and end tags > suitable for parsing by xercesc. It works but for some strange reason > it runs like a dog with no legs when built using Visual Studio in > debug. It runs quite slowly in release mode as well. When using the > GNU compiler under cygwin it runs like greased lightning whether > compiled with -g or not. > > When using the GNU compiler .... > > My test programs takes about 2 seconds to run. But when I use studio > in debug the picture changes. I used Very Sleepy to do sample-based > profiling. It spends all its time entering and leaving critical > regions in locks that are taken in the implementation of std::search > and the iterators passed to it. The time taken jumps up to nearly 3 > minutes! In release it takes about 5 seconds, which is much better > than 3 minutes but still more than twice as slow as GNU in debug. > > .... Check out MSDN / checked iterators: http://msdn.microsoft.com/en-us/library/aa985965%28VS.80%29.aspx http://msdn.microsoft.com/en-us/library/aa985982%28VS.80%29.aspx _SECURE_SCL _HAS_ITERATOR_DEBUGGING .... try setting both to 0 br, Martin -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Goran on 11 Mar 2010 02:54 On Mar 10, 10:51 pm, Andrew <marlow.and...(a)googlemail.com> wrote: > Has anyone else looked at the performance of std::search versus strstr > on the underlying buffer? Why would one implementation takes so many > locks that it cripples performance? What would people here do? Revert > to using the C-like strstr......? Looks like you have _SECURE_SCL to 1 (that is the default and even in release). That causes STL shipped with MSVC to use "checked iterators", and uses mutexes (critical sections) and otherwise imposes a non-negligeable performance penalty (it's a speed-safety trade-off). Try defining _SECURE_SCL to 0 to get "raw" iterators. Goran. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Nick Hounsome on 11 Mar 2010 04:31 On 10 Mar, 21:51, Andrew <marlow.and...(a)googlemail.com> wrote: > My test programs takes about 2 seconds to run. But when I use studio > in debug the picture changes. I used Very Sleepy to do sample-based > profiling. It spends all its time entering and leaving critical > regions in locks that are taken in the implementation of std::search > and the iterators passed to it. The time taken jumps up to nearly 3 > minutes! In release it takes about 5 seconds, which is much better > than 3 minutes but still more than twice as slow as GNU in debug. I don't know about the main cause of your problem but STL will usually compare realy badly to C functions when compiled with DEBUG because non of the template stuff will be inlined. Without DEBUG and with optimisation STL SHOULD be more efficient. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Andrew on 16 Mar 2010 10:40 On 11 Mar, 19:55, "Martin B." <0xCDCDC...(a)gmx.at> wrote: > Andrew wrote: > > I am testing and performance tuning some code written by a colleague. > > The code intent is to read the next XML node from an open file stream. > > The node is returned as a std::string with the begin and end tags > > suitable for parsing by xercesc. It works but for some strange reason > > it runs like a dog with no legs when built using Visual Studio in > > debug. It runs quite slowly in release mode as well. When using the > > GNU compiler under cygwin it runs like greased lightning whether > > compiled with -g or not. > > > When using the GNU compiler .... > > > My test programs takes about 2 seconds to run. But when I use studio > > in debug the picture changes. I used Very Sleepy to do sample-based > > profiling. It spends all its time entering and leaving critical > > regions in locks that are taken in the implementation of std::search > > and the iterators passed to it. The time taken jumps up to nearly 3 > > minutes! In release it takes about 5 seconds, which is much better > > than 3 minutes but still more than twice as slow as GNU in debug. > > > .... > > Check out MSDN / checked iterators:http://msdn.microsoft.com/en-us/library/aa985965%28VS.80%29.aspxhttp://msdn.microsoft.com/en-us/library/aa985982%28VS.80%29.aspx > > _SECURE_SCL > _HAS_ITERATOR_DEBUGGING > > ... try setting both to 0 Unfortunately, this work has to fit into a large project with a large number of libraries, all of which have to be built with secure iterators turned on when building in debug. For various reasons that I won't go into here it does make sense to have these checks for most containers. But IMO it does not make sense when the container is being used as a string. I am coming to the conclusion that when using std::vector<char> to deal with an array of char instead of a std::string, one has to ensure that the array is NUL-terminated and use the C string functions like strstr in preference to the std functions that take iterators. Otherwise it will run like a dog with no legs with Studio in debug. What a pity. So now instead of std::vector<char>::iterator it = std::search(buff.begin(), buff.end(), xmlitem.begin(), xmlitem.end()); I have: // Avoid std::search due to cost of checked iterators in debug with Visual Studio. const char* foundPosition = strstr(&buff[0], xmlitem.c_str()); Regards, Andrew Marlow -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
|
Pages: 1 Prev: filtering (a view on) an STL container Next: List initialization: a weird example |