Indexing Error
in [Matlab]
|
Prev: help
Next: Force-Feedback Joystick
From: Walter Roberson on 8 Jul 2010 14:57 Matt Smerud wrote: > sliderValue = round(sliderValue) * 5 * 10^-5; round(sliderValue) will create an integral value, and multiplying that by 5 will still be an integral value, but 10^(-5) is not exactly representable in binary floating point so sometimes after you multiply by that the result you get back will not be integral. You perhaps wish to instead use sliderValue = round(sliderValue * 5 * 10^(-5));
From: Matt Smerud on 8 Jul 2010 16:00
Walter Roberson <roberson(a)hushmail.com> wrote in message <i15781$fia$1(a)canopus.cc.umanitoba.ca>... > Matt Smerud wrote: > > > sliderValue = round(sliderValue) * 5 * 10^-5; > > round(sliderValue) will create an integral value, and multiplying that by 5 > will still be an integral value, but 10^(-5) is not exactly representable in > binary floating point so sometimes after you multiply by that the result you > get back will not be integral. You perhaps wish to instead use > > sliderValue = round(sliderValue * 5 * 10^(-5)); Hmm. I didn't know that. I'll change my code around so I don't have to worry about it. Thanks for the tip. |