SASware ballot #18 Proc Transpose preserving variable
in [SAS]
|
From: Ted Clay on 26 Jan 2010 00:24 Kevin, OK, a solution exists -- hex format. (Thank you, Data) How would you implement it? One possibility is that the character representation would look like a hex constant, with quotes followed by "X". Would this happen only when necessary to prevent loss of precision, or all the time? It seems a shame to make integers unreadable just because you request "NOFORMAT". Clearly there are some design trade-offs to wrestle with. Ted -----Original Message----- From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Data _null_; Sent: Monday, January 25, 2010 6:39 PM To: SAS-L(a)LISTSERV.UGA.EDU Subject: Re: SASware ballot #18 Proc Transpose preserving variable attributes HEX16. 105 data _null_; 5106 x=1/3; 5107 xc=put(x,hex16.); 5108 newx=input(xc,hex16.); 5109 if x=newx then put 'Same'; 5110 else put 'Different'; 5111 put xc=; 5112 run; Same xc=3FD5555555555555 On 1/25/10, Ted Clay <tclay(a)ashlandhome.net> wrote: > Here is a test case trying to cause data loss when converting numeric to > character to numeric: > > data _null_; > x=1/3; > xc=put(x,30.28); > newx=input(xc,30.28); > if x=newx then put 'Same'; > else put 'Different'; > put xc=; > run; > > Different > xc=0.3333333333333300000000000000 > > Is there a way to fiddle with this code to make "Same" come out? Is this > character representation truly at the limit of resolution, comparable to how > numbers are stored in SAS? Or is there more information "in there" that can > be somehow coaxed out? I found that hard-coding an extra "3" in XC did > nothing, but adding an extra "33" did the trick. > > To put this in perspective, a separate Proc Compare showed the difference > was on the order of E-15. > > Ted > > -----Original Message----- > From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Data > _null_; > Sent: Monday, January 25, 2010 2:13 PM > To: SAS-L(a)LISTSERV.UGA.EDU > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > attributes > > Just having some fun. > > When the data are converted to character by the first transpose it > will be impossible in some situations to get back to the original > data. I believe you and Ted have discussed this already. > > That, is one reason I suggest that a PROC to read a data set CONTROL > data set could be useful but PROC TRANSPOSE is not the proper tool. > > I use PROC TRANPOSE often but I rarely need to get back to the > original data or some even modified version with the same structure. > > I guess I don't see why the data flip-flop is so useful to you or why > you need it. Maybe if you showed some of the code you are using that > requires macro or whatever I would understand better. > > > On 1/25/10, Kevin Myers <KevinMyers(a)austin.rr.com> wrote: > > Oh come on Null... > > > > As mentioned in the message, the example program is merely an > > overly-simplistic test case strictly intended to illustrate and exercise > all > > of the necessary features. Any real application would include additional > > logic between the two transpose steps, or read the desired attributes > values > > from a "self-defining" external file. > > > > The real trick is for you to simulate the *second* transpose step with > > comparably simple code of your own. For you to say that the proposed > > TRANSPOSE enhancements are not needed, that is what you must be able to > > accomplish. So, are you up to the challenge? > > > > ----- Original Message ----- From: "Data _null_;" <iebupdte(a)GMAIL.COM> > > To: <SAS-L(a)LISTSERV.UGA.EDU> > > Sent: Monday, January 25, 2010 15:12 > > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > > attributes > > > > > > > > > On 1/25/10, Kevin Myers <KevinMyers(a)austin.rr.com> wrote: > > > > > > > proc transpose data=original out=skinny type length format informat > > force > > > > noformat; by obsno; > > > > var _all_; run; > > > > > > > > proc transpose data=skinny out=final noinformat; by obsno; > > > > id _NAME_; idlabel _LABEL_; idtype _TYPE_; idlength _LENGTH_; > idformat > > > > _FORMAT_; idinformat _INFORMAT_; > > > > var col1; run; > > > > > > > > If you can manage to accomplish something similar with code that is > > anywhere > > > > remotely close to something this simple, then you win a (virtual) > > cookie. > > > > > > > > > > Seems obvious: data=original > > > > > > > > > > > > > > >
From: Kevin Myers on 26 Jan 2010 01:54 Yes I agree Ted. I expected (perhaps unjustifiably) that formatting values using BEST30. or something similar would output character values that could be re-converted to numeric with no loss in precision. Although forcing use of the hex format could technically solve the double conversion precision issue, to me the unreadability of the resulting character representations for numeric values would be undesirable in many cases. Perhaps there is a bug in the format or corresponding informat used in your test case which prevents full precision conversion from/to numeric. However, I suspect it is more likely that the true source of the problem is unavoidable round off error when converting exact decimal fractions to binary represenatation. If so, then I don't believe there is any easy way around this problem using a decimal character representation for numeric values. After considering various possibilities, I suggest the following modifications to the previous NOFORMAT/NOINFORMAT idea: The user can specify zero or one of the following two options: NOINFORMAT - for transposed values where source is char, if dest is char value will be read unformatted, else value read using 30. HEXINFORMAT - for transposed values where source is char, if dest is char value will be read unformatted, else value read using HEX16. Neither of these options have any effect if the source variable is numeric. The user can specify zero or one of the following two options: NOFORMAT - for transposed values where dest is char, if source is char value will be written unformatted, else value written using BEST30. HEXFORMAT - for transposed values where dest is char, if source is char value will be written unformatted, else value written using HEX16. Neither of these options have any effect if the destination variable is numeric. What do you think? ----- Original Message ----- From: "Ted Clay" <tclay(a)ASHLANDHOME.NET> To: <SAS-L(a)LISTSERV.UGA.EDU> Sent: Monday, January 25, 2010 23:24 Subject: Re: SASware ballot #18 Proc Transpose preserving variable attributes > Kevin, > OK, a solution exists -- hex format. (Thank you, Data) How would you > implement it? One possibility is that the character representation would > look like a hex constant, with quotes followed by "X". Would this happen > only when necessary to prevent loss of precision, or all the time? It > seems > a shame to make integers unreadable just because you request "NOFORMAT". > Clearly there are some design trade-offs to wrestle with. > Ted > > > > -----Original Message----- > From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Data > _null_; > Sent: Monday, January 25, 2010 6:39 PM > To: SAS-L(a)LISTSERV.UGA.EDU > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > attributes > > HEX16. > > 105 data _null_; > 5106 x=1/3; > 5107 xc=put(x,hex16.); > 5108 newx=input(xc,hex16.); > 5109 if x=newx then put 'Same'; > 5110 else put 'Different'; > 5111 put xc=; > 5112 run; > > Same > xc=3FD5555555555555 > > > On 1/25/10, Ted Clay <tclay(a)ashlandhome.net> wrote: >> Here is a test case trying to cause data loss when converting numeric to >> character to numeric: >> >> data _null_; >> x=1/3; >> xc=put(x,30.28); >> newx=input(xc,30.28); >> if x=newx then put 'Same'; >> else put 'Different'; >> put xc=; >> run; >> >> Different >> xc=0.3333333333333300000000000000 >> >> Is there a way to fiddle with this code to make "Same" come out? Is this >> character representation truly at the limit of resolution, comparable to > how >> numbers are stored in SAS? Or is there more information "in there" that > can >> be somehow coaxed out? I found that hard-coding an extra "3" in XC did >> nothing, but adding an extra "33" did the trick. >> >> To put this in perspective, a separate Proc Compare showed the difference >> was on the order of E-15. >> >> Ted >>
From: Ted Clay on 26 Jan 2010 16:53 Kevin, You could bundle lots of features into "ATTRIBSIN" and "ATTRIBSOUT". Then you are free to use "NOFORMAT" without confusion. The compromise is in accepting defaults names. ATTRIBSIN would look for and use any standard-named variable, like _FORMAT_, _TYPE_, and _INFORMAT_. In fact you could have it look for _NAME_ and _LABEL_ too if the user does not give an ID or IDLABEL statement. (That might be the least-surprising behavior, in fact.) ATTRIBSOUT would produce standard-named variables, only when needed, or with "FORCE", always. Your modification would require these options, and no additional statements: For output: ATTRIBSOUT NOFORMAT HEXFORMAT FORCE For Input: ATTRIBSIN NOFORMATIN HEXFORMATIN Ted -----Original Message----- From: Kevin Myers [mailto:KevinMyers(a)austin.rr.com] Sent: Tuesday, January 26, 2010 10:46 AM To: Ted Clay Subject: Re: SASware ballot #18 Proc Transpose preserving variable attributes I agree regarding potential usability concerns surrounding FORMAT vs. NOFORMAT and INFORMAT vs. NOINFORMAT. I don't really like the NOFORMAT and NOINFORMAT option names, but haven't been able to come up with anything better that doesn't result in very long and cumbersome names. Any suggestions? Your suggested warning seems like a good idea. ----- Original Message ----- From: "Ted Clay" <tclay(a)ashlandhome.net> To: "'Kevin Myers'" <KevinMyers(a)AUSTIN.RR.COM> Sent: Tuesday, January 26, 2010 11:57 Subject: RE: SASware ballot #18 Proc Transpose preserving variable attributes > This works, technically speaking, though the solution is not particularly > aesthetic. > Speaking of usability, your "FORMAT" and "NOFORMAT" options are not > opposites, but that is what a user might expect, by the "principle of > least > surprise". Also, maybe a WARNING should be issued if NOINFORMAT writes > out > a numeric value whose unformatted character representation fills more than > 14 (or some number) spaces, which would indicate that some precision has > been lost. So most of the time "NOFORMAT" would work fine, and the user > would find out when "HEXFORMAT" is needed. > Ted > > > > -----Original Message----- > From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Kevin > Myers > Sent: Monday, January 25, 2010 10:54 PM > To: SAS-L(a)LISTSERV.UGA.EDU > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > attributes > > Yes I agree Ted. I expected (perhaps unjustifiably) that formatting > values > using BEST30. or something similar would output character values that > could > be re-converted to numeric with no loss in precision. Although forcing > use > of the hex format could technically solve the double conversion precision > issue, to me the unreadability of the resulting character representations > for numeric values would be undesirable in many cases. > > Perhaps there is a bug in the format or corresponding informat used in > your > test case which prevents full precision conversion from/to numeric. > However, I suspect it is more likely that the true source of the problem > is > unavoidable round off error when converting exact decimal fractions to > binary represenatation. If so, then I don't believe there is any easy way > around this problem using a decimal character representation for numeric > values. > > After considering various possibilities, I suggest the following > modifications to the previous NOFORMAT/NOINFORMAT idea: > > The user can specify zero or one of the following two options: > NOINFORMAT - for transposed values where source is char, if dest is char > value will be read unformatted, else value read using 30. > HEXINFORMAT - for transposed values where source is char, if dest is char > value will be read unformatted, else value read using HEX16. > Neither of these options have any effect if the source variable is > numeric. > > The user can specify zero or one of the following two options: > NOFORMAT - for transposed values where dest is char, if source is char > value > will be written unformatted, else value written using BEST30. > HEXFORMAT - for transposed values where dest is char, if source is char > value will be written unformatted, else value written using HEX16. > Neither of these options have any effect if the destination variable is > numeric. > > What do you think? > > > ----- Original Message ----- > From: "Ted Clay" <tclay(a)ASHLANDHOME.NET> > To: <SAS-L(a)LISTSERV.UGA.EDU> > Sent: Monday, January 25, 2010 23:24 > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > attributes > > >> Kevin, >> OK, a solution exists -- hex format. (Thank you, Data) How would you >> implement it? One possibility is that the character representation would >> look like a hex constant, with quotes followed by "X". Would this happen >> only when necessary to prevent loss of precision, or all the time? It >> seems >> a shame to make integers unreadable just because you request "NOFORMAT". >> Clearly there are some design trade-offs to wrestle with. >> Ted >> >> >> >> -----Original Message----- >> From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Data >> _null_; >> Sent: Monday, January 25, 2010 6:39 PM >> To: SAS-L(a)LISTSERV.UGA.EDU >> Subject: Re: SASware ballot #18 Proc Transpose preserving variable >> attributes >> >> HEX16. >> >> 105 data _null_; >> 5106 x=1/3; >> 5107 xc=put(x,hex16.); >> 5108 newx=input(xc,hex16.); >> 5109 if x=newx then put 'Same'; >> 5110 else put 'Different'; >> 5111 put xc=; >> 5112 run; >> >> Same >> xc=3FD5555555555555 >> >> >> On 1/25/10, Ted Clay <tclay(a)ashlandhome.net> wrote: >>> Here is a test case trying to cause data loss when converting numeric to >>> character to numeric: >>> >>> data _null_; >>> x=1/3; >>> xc=put(x,30.28); >>> newx=input(xc,30.28); >>> if x=newx then put 'Same'; >>> else put 'Different'; >>> put xc=; >>> run; >>> >>> Different >>> xc=0.3333333333333300000000000000 >>> >>> Is there a way to fiddle with this code to make "Same" come out? Is >>> this >>> character representation truly at the limit of resolution, comparable to >> how >>> numbers are stored in SAS? Or is there more information "in there" that >> can >>> be somehow coaxed out? I found that hard-coding an extra "3" in XC did >>> nothing, but adding an extra "33" did the trick. >>> >>> To put this in perspective, a separate Proc Compare showed the >>> difference >>> was on the order of E-15. >>> >>> Ted >>> > > >
From: xlr82sas on 27 Jan 2010 12:28 HI, For some transposes you may be able to use this sleezy cheesy trick. I store code in the label. You can automate this with datastep function vtype, vlabel or macros below. For a complete set of attribute macros see utl_tipweb.txt at http://homepage.mac.com/magdelina/.Public/utl.html Macro below data bignum; label big ="input(big,hex16.)"; label name ="name"; set sashelp.class(keep=name); big=put(constant('BIG'),hex16.); output; stop; run; proc transpose data=bignum out=bigxpo; var name big; run; proc transpose data=bigxpo out=xpoxpo; id _name_; var col1; idl _label_; run; Data typget; set xpoxpo; bignum=%utl_varlabel(xpoxpo,big); run; %macro utl_varlabel(dsn,var)/des="Variable label"; %local dsid posv rc; %let dsid = %sysfunc(open(&dsn,i)); %let posv = %sysfunc(varnum(&dsid,&var)); %sysfunc(varlabel(&dsid,&posv)) %let rc = %sysfunc(close(&dsid)); %mend utl_varlabel; or use datastep vtype, call label %macro utl_vartype(dsn,var)/des="Variable type returns N or C"; %local dsid posv rc; %let dsid = %sysfunc(open(&dsn,i)); %let posv = %sysfunc(varnum(&dsid,&var)); %sysfunc(vartype(&dsid,&posv)) %let rc = %sysfunc(close(&dsid)); %mend utl_vartype; On Jan 25, 6:19 pm, KevinMy...(a)AUSTIN.RR.COM (Kevin Myers) wrote: > Yes, FORCE was a late addition not included in my original proposal, but > necessary to make the proposed options and statements more useful. Using > the FORCE option would force TRANSPOSE to create appropriate attribute > variables even if no variable being transposed has a given attribute. For > example, with the current implementation, a LABEL variable (e.g. _LABEL_) is > only created if at least one of the variables being transposed actually has > a label. This can adversely complicate subsequent steps that must add extra > logic to alter processing depending upon whether or not the _LABEL_ variable > exists. Using FORCE would eliminate the need for such logic by causing the > LABEL variable to be created whether or not any of the variables being > transposed actually have a label. In this case, the LABEL variable would > have blank values for all observations. Make sense? > > s/KAM > > > > ----- Original Message ----- > From: "Ted Clay" <tc...(a)ashlandhome.net> > To: "'Kevin Myers'" <KevinMy...(a)AUSTIN.RR.COM>; <SA...(a)LISTSERV.UGA.EDU> > Sent: Monday, January 25, 2010 16:36 > Subject: RE: SASware ballot #18 Proc Transpose preserving variable > > attributes > > > Kevin, > > Did you mean to type "FORCE" on Proc Transpose? Did I miss a feature in > > your proposal? > > Ted > > > -----Original Message----- > > From: SAS(r) Discussion [mailto:SA...(a)LISTSERV.UGA.EDU] On Behalf Of Kevin > > Myers > > Sent: Monday, January 25, 2010 1:23 PM > > To: SA...(a)LISTSERV.UGA.EDU > > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > > attributes > > > Oh come on Null... > > > As mentioned in the message, the example program is merely an > > overly-simplistic test case strictly intended to illustrate and exercise > > all > > of the necessary features. Any real application would include additional > > logic between the two transpose steps, or read the desired attributes > > values > > from a "self-defining" external file. > > > The real trick is for you to simulate the *second* transpose step with > > comparably simple code of your own. For you to say that the proposed > > TRANSPOSE enhancements are not needed, that is what you must be able to > > accomplish. So, are you up to the challenge? > > > ----- Original Message ----- > > From: "Data _null_;" <iebup...(a)GMAIL.COM> > > To: <SA...(a)LISTSERV.UGA.EDU> > > Sent: Monday, January 25, 2010 15:12 > > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > > attributes > > >> On 1/25/10, Kevin Myers <KevinMy...(a)austin.rr.com> wrote: > >>> proc transpose data=original out=skinny type length format informat > >>> force > >>> noformat; by obsno; > >>> var _all_; run; > > >>> proc transpose data=skinny out=final noinformat; by obsno; > >>> id _NAME_; idlabel _LABEL_; idtype _TYPE_; idlength _LENGTH_; idformat > >>> _FORMAT_; idinformat _INFORMAT_; > >>> var col1; run; > > >>> If you can manage to accomplish something similar with code that is > >>> anywhere > >>> remotely close to something this simple, then you win a (virtual) > >>> cookie. > > >> Seems obvious: data=original- Hide quoted text - > > - Show quoted text -
From: xlr82sas on 27 Jan 2010 20:20 Here s a cheesy, sleezy trick to get the original floating point number back after on a double transpose. I execute the variable label. This could be automated with functions vtype and vlabel. Macro is given below for other tools see utl_tipweb.txt at http://homepage.mac.com/magdelina/.Public/utl.html data bignum; label big ="input(big,hex16.)"; label name ="name"; set sashelp.class(keep=name); big=put(constant('BIG'),hex16.); output; stop; run; proc transpose data=bignum out=bigxpo; var name big; run; proc transpose data=bigxpo out=xpoxpo; id _name_; var col1; idl _label_; run; Data typget; set xpoxpo; bignum=%utl_varlabel(xpoxpo,big); run; %macro utl_varlabel(dsn,var)/des="Variable label"; %local dsid posv rc; %let dsid = %sysfunc(open(&dsn,i)); %let posv = %sysfunc(varnum(&dsid,&var)); %sysfunc(varlabel(&dsid,&posv)) %let rc = %sysfunc(close(&dsid)); %mend utl_varlabel; On Jan 25, 6:19 pm, KevinMy...(a)AUSTIN.RR.COM (Kevin Myers) wrote: > Yes, FORCE was a late addition not included in my original proposal, but > necessary to make the proposed options and statements more useful. Using > the FORCE option would force TRANSPOSE to create appropriate attribute > variables even if no variable being transposed has a given attribute. For > example, with the current implementation, a LABEL variable (e.g. _LABEL_) is > only created if at least one of the variables being transposed actually has > a label. This can adversely complicate subsequent steps that must add extra > logic to alter processing depending upon whether or not the _LABEL_ variable > exists. Using FORCE would eliminate the need for such logic by causing the > LABEL variable to be created whether or not any of the variables being > transposed actually have a label. In this case, the LABEL variable would > have blank values for all observations. Make sense? > > s/KAM > > > > ----- Original Message ----- > From: "Ted Clay" <tc...(a)ashlandhome.net> > To: "'Kevin Myers'" <KevinMy...(a)AUSTIN.RR.COM>; <SA...(a)LISTSERV.UGA.EDU> > Sent: Monday, January 25, 2010 16:36 > Subject: RE: SASware ballot #18 Proc Transpose preserving variable > > attributes > > > Kevin, > > Did you mean to type "FORCE" on Proc Transpose? Did I miss a feature in > > your proposal? > > Ted > > > -----Original Message----- > > From: SAS(r) Discussion [mailto:SA...(a)LISTSERV.UGA.EDU] On Behalf Of Kevin > > Myers > > Sent: Monday, January 25, 2010 1:23 PM > > To: SA...(a)LISTSERV.UGA.EDU > > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > > attributes > > > Oh come on Null... > > > As mentioned in the message, the example program is merely an > > overly-simplistic test case strictly intended to illustrate and exercise > > all > > of the necessary features. Any real application would include additional > > logic between the two transpose steps, or read the desired attributes > > values > > from a "self-defining" external file. > > > The real trick is for you to simulate the *second* transpose step with > > comparably simple code of your own. For you to say that the proposed > > TRANSPOSE enhancements are not needed, that is what you must be able to > > accomplish. So, are you up to the challenge? > > > ----- Original Message ----- > > From: "Data _null_;" <iebup...(a)GMAIL.COM> > > To: <SA...(a)LISTSERV.UGA.EDU> > > Sent: Monday, January 25, 2010 15:12 > > Subject: Re: SASware ballot #18 Proc Transpose preserving variable > > attributes > > >> On 1/25/10, Kevin Myers <KevinMy...(a)austin.rr.com> wrote: > >>> proc transpose data=original out=skinny type length format informat > >>> force > >>> noformat; by obsno; > >>> var _all_; run; > > >>> proc transpose data=skinny out=final noinformat; by obsno; > >>> id _NAME_; idlabel _LABEL_; idtype _TYPE_; idlength _LENGTH_; idformat > >>> _FORMAT_; idinformat _INFORMAT_; > >>> var col1; run; > > >>> If you can manage to accomplish something similar with code that is > >>> anywhere > >>> remotely close to something this simple, then you win a (virtual) > >>> cookie. > > >> Seems obvious: data=original- Hide quoted text - > > - Show quoted text -
First
|
Prev
|
Pages: 1 2 3 4 5 6 Prev: Can Proc Import define data column numbers? Next: Proc ARIMA - by groups |