Scientific to Decimal
in [Visual Basic]
|
From: Webbiz on 30 Nov 2009 19:29 I'm trying to display the results of an averaging function in the same decimal format of the values that were averaged. Yet, the results end up in Scientific Notation, which is not desired. So I added the Format() function to the averaging function, but it is not returning any values at all. I've been searching the net for the reason why, but have not found anything other than the fomat I'm using. What am I doing wrong? Private Function DisplayAvgATR(ByRef Ranges() As Single) As String Dim n As Long Dim sngTotal As Single Dim sngAvg As Single For n = 0 To UBound(Ranges) sngTotal = sngTotal + Ranges(n) Next n 'Return the Average sngAvg = sngTotal / (UBound(Ranges) + 1) DisplayAvgATR = Format(sngAvg, "#.#") End Function While sngAvg actually has a value, say 6.0028E-02, it is not being returned as .060028. Thanks. Webbiz
From: Nobody on 30 Nov 2009 19:54 "Webbiz" <nospam(a)noway.com> wrote in message news:blo8h5l82ecd6c0efqg64j7s4smq29cgq9(a)4ax.com... > While sngAvg actually has a value, say 6.0028E-02, it is not being > returned as .060028. It should return ".1" in that case, which is what I am getting. You asked only for one number after the decimal point, and Format rounds to that. If you need multiple digits, you need to add another parameter, NumDecimalPlaces. Example: DisplayAvgATR = Format(sngAvg, "#." & String(NumDecimalPlaces, "#")) Also look at Round() function, but due to precision, you may get 0.060000000001 because in some cases a number cannot be precisely representing by binary values, so always use Format() function to display your data.
From: MikeD on 30 Nov 2009 21:21 "Webbiz" <nospam(a)noway.com> wrote in message news:blo8h5l82ecd6c0efqg64j7s4smq29cgq9(a)4ax.com... > I'm trying to display the results of an averaging function in the same > decimal format of the values that were averaged. Yet, the results end > up in Scientific Notation, which is not desired. > > So I added the Format() function to the averaging function, but it is > not returning any values at all. > > I've been searching the net for the reason why, but have not found > anything other than the fomat I'm using. > > What am I doing wrong? > > Private Function DisplayAvgATR(ByRef Ranges() As Single) As String > Dim n As Long > Dim sngTotal As Single > Dim sngAvg As Single > > For n = 0 To UBound(Ranges) > sngTotal = sngTotal + Ranges(n) > Next n > > > > 'Return the Average > sngAvg = sngTotal / (UBound(Ranges) + 1) > DisplayAvgATR = Format(sngAvg, "#.#") > > End Function > > While sngAvg actually has a value, say 6.0028E-02, it is not being > returned as .060028. Why would it when your format string is "#.#'? That's going to format it to just one decimal place. Use as many # signs for the decimal part as you want decimal places. And just to nitpick a bit here, since your function is not really displaying anything, you might want to consider renaming it...for example CalcAvgATR or even just simply CalcAvg since it will calculate the average for anything that meets the parameter criteria of an array of type Single (kinda making a guess that ATR indicates a set of specific values). And one more nitpick. Your function is making an assumption that the passed in array will be 0-based. That's probably OK since that's the default for an array. But you can make it generic so that it will work regardless of what the lower bound is by doing this: Private Function CalcAvg(ByRef Ranges() As Single) As String Dim n As Long Dim sngTotal As Single Dim sngAvg As Single For n = LBound(Ranges) To UBound(Ranges) sngTotal = sngTotal + Ranges(n) Next n 'Return the Average sngAvg = sngTotal / (UBound(Ranges) - LBound(Ranges) + 1) CalcAvg = Format$(sngAvg, "#.######") End Function And one last nitpick, since the function returns a String, use the String version of the Format function, as shown above. Some people will even say there's no need for the sngAvg variable. Just do this: CalcAvg = Format$(sngTotal / (UBound(Ranges) - LBound(Ranges) + 1), "#.######") And strictly speaking they'd be right. There is no absolute reason to use a local variable to hold the result of the calculation, but when I'm debugging and stepping through code, I like to have the results of calculations assigned to variables to make it easier to get a tooltip of the result of that calculation in the IDE (and yes I know you could position the mousepointer on an opening parenthesis or select the expression to get a tooltip showing the result, but it's much easier to just position the mousepointer within a variable name). And the millisecond (or less) that the local variable is using memory...not an issue as far as I'm concerned. -- Mike
From: DanS on 30 Nov 2009 22:04 "MikeD" <nobody(a)nowhere.edu> wrote in news:u9#uQ0icKHA.1592(a)TK2MSFTNGP06.phx.gbl: > > > "Webbiz" <nospam(a)noway.com> wrote in message > news:blo8h5l82ecd6c0efqg64j7s4smq29cgq9(a)4ax.com... >> I'm trying to display the results of an averaging function in the >> same decimal format of the values that were averaged. Yet, the >> results end up in Scientific Notation, which is not desired. >> >> So I added the Format() function to the averaging function, but it is >> not returning any values at all. >> >> I've been searching the net for the reason why, but have not found >> anything other than the fomat I'm using. >> >> What am I doing wrong? >> >> Private Function DisplayAvgATR(ByRef Ranges() As Single) As String >> Dim n As Long >> Dim sngTotal As Single >> Dim sngAvg As Single >> >> For n = 0 To UBound(Ranges) >> sngTotal = sngTotal + Ranges(n) >> Next n >> >> >> >> 'Return the Average >> sngAvg = sngTotal / (UBound(Ranges) + 1) >> DisplayAvgATR = Format(sngAvg, "#.#") >> >> End Function >> >> While sngAvg actually has a value, say 6.0028E-02, it is not being >> returned as .060028. > > > Why would it when your format string is "#.#'? That's going to format > it to just one decimal place. Use as many # signs for the decimal > part as you want decimal places. Isn't that use # for deciaml places if needed, or use 0 for required.... x = 7.12345 format (x,"##.############") will return 7.12345 while format (x,"00.000000000000") will return 07.12345000000
From: MikeD on 30 Nov 2009 22:54
"DanS" <t.h.i.s.n.t.h.a.t(a)r.o.a.d.r.u.n.n.e.r.c.o.m> wrote in message news:Xns9CD3E0AA14322thisnthatroadrunnern(a)216.196.97.131... > "MikeD" <nobody(a)nowhere.edu> wrote in > news:u9#uQ0icKHA.1592(a)TK2MSFTNGP06.phx.gbl: > >> >> >> "Webbiz" <nospam(a)noway.com> wrote in message >> news:blo8h5l82ecd6c0efqg64j7s4smq29cgq9(a)4ax.com... >>> I'm trying to display the results of an averaging function in the >>> same decimal format of the values that were averaged. Yet, the >>> results end up in Scientific Notation, which is not desired. >>> >>> So I added the Format() function to the averaging function, but it is >>> not returning any values at all. >>> >>> I've been searching the net for the reason why, but have not found >>> anything other than the fomat I'm using. >>> >>> What am I doing wrong? >>> >>> Private Function DisplayAvgATR(ByRef Ranges() As Single) As String >>> Dim n As Long >>> Dim sngTotal As Single >>> Dim sngAvg As Single >>> >>> For n = 0 To UBound(Ranges) >>> sngTotal = sngTotal + Ranges(n) >>> Next n >>> >>> >>> >>> 'Return the Average >>> sngAvg = sngTotal / (UBound(Ranges) + 1) >>> DisplayAvgATR = Format(sngAvg, "#.#") >>> >>> End Function >>> >>> While sngAvg actually has a value, say 6.0028E-02, it is not being >>> returned as .060028. >> >> >> Why would it when your format string is "#.#'? That's going to format >> it to just one decimal place. Use as many # signs for the decimal >> part as you want decimal places. > > Isn't that use # for deciaml places if needed, or use 0 for required.... > > x = 7.12345 > > format (x,"##.############") will return 7.12345 > > while format (x,"00.000000000000") will return > > 07.12345000000 You have to use as many placeholders in your format string as you want for minimum decimal places. He only had one decimal placeholder, so the function would only return one decimal. If the format string is "#.##" and the value to format is .12345, the formatted string is going to be ".12" whereas if the format string is "#.#####", the formatted string will be ".12345". If the format string is "#.####" the formatted string will be ".1235" because that 5 in the 5th place is going to cause the 4 to get rounded up as the format string dictates only 4 decimal places (if I did all that correctly in my head). I think your confusion might be because # works a little differently when left of the decimal place. If you have a format string of "#.##" and the value to format is 500.5, the formatted string will be "500.5" even though there was only a single # character left of the decimal point in the format string. -- Mike |