From: dpb on 1 Dec 2009 20:04 Webbiz wrote: .... > question is WHY is it changing to Sci-Not? .... Unless you're formatting it, the output format is at the mercy of the default i/o VB rtl...see other response for more specifically... --
From: Webbiz on 1 Dec 2009 20:11 On Tue, 1 Dec 2009 19:41:28 -0500, "Rick Rothstein" <rick.newsNO.SPAM(a)NO.SPAMverizon.net> wrote: >That's a problem then... once the numbers are placed in a numeric variable, >they lose all their formatting (numbers are just numbers, they have no >format). The problem with using a value from the list of numbers is that >none of your values might be "full". For example, what if all your numbers >are **supposed** to be shown to 3 decimal points, but all your values just >happen to, by rare chance, have only one or two decimal places... if you try >using one of these numbers in my expression, you will not get 3-decimal >places shown. You also can't just select a number from the list because you >might not get a "full" number either. Where are these numbers coming from? >Perhaps you can go back to the source to get a representative formatted text >representation of the number and use that. Here's an idea Rick. The application requires a minimum of 3 years data to work properly (this is not my charting program, btw). That's at least 600 data values, all Singles. In such a scenario, it's not only rare, it's impossible to not find the full format of the dataset. So what do you think of this: Within the routine that actually loads the data into the app, note each value for the number of digits following the decimal. If I'm not mistaken, the left is not important because "#. " will expand accordingly. It's right of the decimal that needs to be determined. So as each value passes by, if any value ends up with more values right of the decimal, the format is adjusted. This format is then saved in a Global String variable. Therefore, what I'm looking for would be the maximum needed values right of the decimal, right? So if this were discovered, the string variable could be set by... Select Case DecCnt Case 0 sDataFormat = "#" Case 1 sDataFormat = "#.#" Case 2 sDataFormat = "#.##" ....etc. Or is this too clumsy? Webbiz
From: dpb on 1 Dec 2009 21:44 Webbiz wrote: .... > Or is this too clumsy? All depends on what you really need...if you can't a priori determine the number of decimal digits required I suppose that's one possibility if the data are always loaded from a text file. If you were to load other floating point data it's possible you would always end up w/ the full precision of the variable owing to the rounding issues previously discussed. I'd think knowing simply the actual magnitude of the numbers if these are prices would be sufficient to determine a reasonable estimate of the precision required but I suppose that may be too naive a solution. --
From: Ralph on 1 Dec 2009 21:49 "Webbiz" <nospam(a)noway.com> wrote in message news:blo8h5l82ecd6c0efqg64j7s4smq29cgq9(a)4ax.com... > I'm trying to display the results of an averaging function in the same > decimal format of the values that were averaged. Yet, the results end > up in Scientific Notation, which is not desired. > We've had this conversation before IIRC. Frankly for your whole application you need to either use the VB Currency datatype which will provide some consistency, or write your own 'Decimal' datatype routines (probably better as you tend to flop about on precision) that are based on integer math. [OR - buy a math/financial/statistical package. Unfortunately they tend to be pricey, unless you can use C/C++. There are many open source C/C++ libraries available.which you can use to create ActiveX or standard DLLs.] No financial/statistical application that needs to be reliable, consistent, and accurate uses floating datatypes for its historical data, journals, accounts, or projections. And all math is integer unless floating math is absolutely required - and even then the specific precision is rigorously enforced going in and coming out. -ralph
From: Jim Mack on 1 Dec 2009 22:53
Webbiz wrote: > Jim Mack wrote: > >> Webbiz wrote: >> >>> Then why are my Single values, showing in Watch as .5938 for >>> example, showing as a Scientific Notation value after some math >>> and assigned to another Single variable? >>> >>> If I send the original single value to the text box, it will show >>> up as .5938. But do some subtraction, for example, of two of these >>> Singles and assign to another Single, or directly to the textbox, >>> and it displays as Scentific Notation? >> >> A textbox can't display a single, it can only display strings. And >> anything you show in Watch or Debug etc is a also string. >> >> If you want to control how something appears on the screen (i.e. in >> the string domain), use Format. > > > Jim, I know this. Apparently not completely. Singles do not have a 'format'. They cannot change from 'regular' to 'scientific'. That is strictly a function of being displayed as a string. If you want them to appear a certain way then you have to format them that way. You are NOT seeing singles. You're seeing strings, which are the result of formatting. Accept the default VB formatting, and you get what VB decides. Use Format and get what you want. It's really pretty simple. -- Jim Mack Twisted tees at http://www.cafepress.com/2050inc "We sew confusion" > > What I'm saying is that the value is changing from a regular decimal > value to Scientific Notation. So when I go to display it in a text > box, it shows as Scientific Notation. Yes, format is what will need > to be used, and that's still in discussion in this thread. However, > my question is WHY is it changing to Sci-Not? > > Thanks. > > Webbiz |