dsread - Windows command-line utility for SAS7BDAT files
in [SAS]
|
From: Alan Churchill on 26 Feb 2010 12:06 Again, piping is easy. I'll try and toss something together this weekend and pipe delimited, Excel, and XML. I am a bit unclear on what it is you need. If I dump the data to a delimited file, for example, I treat the values as doubles (do not print hex representation) so you should have better representation anyway. You can always convert them as needed. Alan Alan Churchill Savian www.savian.net Office: (719) 687-5954 Cell: (719) 310-4870 -----Original Message----- From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of xlr82sas Sent: Thursday, February 25, 2010 7:43 PM To: SAS-L(a)LISTSERV.UGA.EDU Subject: Re: dsread - Windows command-line utility for SAS7BDAT files Hi Alan and Chis What interests me is communications with open source products, like R, perl and MySQL. Export and import needs to be lossless. My understanding is that SAS -ODBC/OleDB... requires you to bring up SAS(ODBC server) and the server has to be closed manually. I feel this is unsatisfactory. The crux of the issue is IEEE floating point. I think HEX16. would be lossless, all I have to in R is convert the character hex16. to floating point. The nice thing about dsread is it is light, fast and you do not need SAS and it looks like it supports pipes. On Feb 25, 12:15 pm, xlr82sas <xlr82...(a)aol.com> wrote: > On Feb 25, 10:43 am, xlr82sas <xlr82...(a)aol.com> wrote: > > > > > > > On Feb 25, 6:45 am, ChrisBLong <ch...(a)oview.co.uk> wrote: > > > > Hi, > > > > Readers of this group may be interested in dsread, available athttp://www.oview.co.uk/dsread > > > > It's a command-line utility that understands the SAS7BDAT file > > > format. It lets you examine the structure of datasets conveniently > > > from the command-line, and converts SAS7BDAT data into valid CSV > > > format for import into other software. > > > > All comments and suggestions gratefully received, > > > > Chris. > > > Hi Chris, > > > Congratulations for reading SAS datasets. WPS has powerfull > > capabilities. Thanks!!! Competition is great. > > > Even though you only create csv's, I see this as a great product > > because you do not need SAS and with pipes users can programtically > > get at SAS data from other languages. Also CSVs are also very amenable > > to EXCEL. > > > ===========================================================================� �=================================================== > > > Just some thoughts: > > > Any chance you could create a lossless output format, like SAS > > export datasets, but allow for longer names and > > character values greater than 200 bytes. This would open up SAS > > datasets to other languages. The format would have to be open. > > > This could be a really big deal, if instead of a csv, you created R > > dataframes, if called from R. An even bigger deal would be if you > > created a SAS dataset from an R dataframe. > > > XML would be another nice output. > > > A silient ODBC would also be great. > > > I bet you can use pipes whith yow command line interface. > > My apologies. > > I assumed your site was somehow affiliated with WPS. > It looks like it may not be. > > Which makes what you have done all the more remarkable. > > I was hoping you honored formats because I wanted to associate hex16 > with the numeric columns so I could create a lossless csv, but it did > not work.- Hide quoted text - > > - Show quoted text -
From: xlr82sas on 26 Feb 2010 21:55 On Feb 26, 9:06 am, alan.church...(a)SAVIAN.NET (Alan Churchill) wrote: > Again, piping is easy. I'll try and toss something together this weekend and > pipe delimited, Excel, and XML. > > I am a bit unclear on what it is you need. If I dump the data to a delimited > file, for example, I treat the values as doubles (do not print hex > representation) so you should have better representation anyway. You can > always convert them as needed. > > Alan > > Alan Churchill > Savianwww.savian.net > Office: (719) 687-5954 > Cell: (719) 310-4870 > > > > -----Original Message----- > From: SAS(r) Discussion [mailto:SA...(a)LISTSERV.UGA.EDU] On Behalf Of > > xlr82sas > Sent: Thursday, February 25, 2010 7:43 PM > To: SA...(a)LISTSERV.UGA.EDU > Subject: Re: dsread - Windows command-line utility for SAS7BDAT files > > Hi Alan and Chis > > What interests me is communications with open source products, > like R, perl and MySQL. Export and import needs to be lossless. My > understanding is that SAS -ODBC/OleDB... requires you to bring up > SAS(ODBC server) and the server has to be closed manually. I feel this > is unsatisfactory. The crux of the issue is IEEE floating point. I > think HEX16. would be lossless, all I have to in R is convert the > character hex16. to floating point. The nice thing about dsread is > it is light, fast and you do not need SAS and it looks like it > supports pipes. > > On Feb 25, 12:15 pm, xlr82sas <xlr82...(a)aol.com> wrote: > > On Feb 25, 10:43 am, xlr82sas <xlr82...(a)aol.com> wrote: > > > > On Feb 25, 6:45 am, ChrisBLong <ch...(a)oview.co.uk> wrote: > > > > > Hi, > > > > > Readers of this group may be interested in dsread, available > athttp://www.oview.co.uk/dsread > > > > > It's a command-line utility that understands the SAS7BDAT file > > > > format. It lets you examine the structure of datasets conveniently > > > > from the command-line, and converts SAS7BDAT data into valid CSV > > > > format for import into other software. > > > > > All comments and suggestions gratefully received, > > > > > Chris. > > > > Hi Chris, > > > > Congratulations for reading SAS datasets. WPS has powerfull > > > capabilities. Thanks!!! Competition is great. > > > > Even though you only create csv's, I see this as a great product > > > because you do not need SAS and with pipes users can programtically > > > get at SAS data from other languages. Also CSVs are also very amenable > > > to EXCEL. > > =========================================================================== > =================================================== > > > > Just some thoughts: > > > > Any chance you could create a lossless output format, like SAS > > > export datasets, but allow for longer names and > > > character values greater than 200 bytes. This would open up SAS > > > datasets to other languages. The format would have to be open. > > > > This could be a really big deal, if instead of a csv, you created R > > > dataframes, if called from R. An even bigger deal would be if you > > > created a SAS dataset from an R dataframe. > > > > XML would be another nice output. > > > > A silient ODBC would also be great. > > > > I bet you can use pipes whith yow command line interface. > > > My apologies. > > > I assumed your site was somehow affiliated with WPS. > > It looks like it may not be. > > > Which makes what you have done all the more remarkable. > > > I was hoping you honored formats because I wanted to associate hex16 > > with the numeric columns so I could create a lossless csv, but it did > > not work.- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - Thanks for the hex16 consideration. With this I should be able to try 'lossless' communications with R. I won't have to shorten variable names to 8 bytes or restrict char length to 200. I like the fact that I can get character lengths from your 'contents' feature, users do not have to worry about traucation. Thanks again Regards
From: xlr82sas on 26 Feb 2010 22:23 On Feb 26, 9:06 am, alan.church...(a)SAVIAN.NET (Alan Churchill) wrote: > Again, piping is easy. I'll try and toss something together this weekend and > pipe delimited, Excel, and XML. > > I am a bit unclear on what it is you need. If I dump the data to a delimited > file, for example, I treat the values as doubles (do not print hex > representation) so you should have better representation anyway. You can > always convert them as needed. > > Alan > > Alan Churchill > Savianwww.savian.net > Office: (719) 687-5954 > Cell: (719) 310-4870 > > > > -----Original Message----- > From: SAS(r) Discussion [mailto:SA...(a)LISTSERV.UGA.EDU] On Behalf Of > > xlr82sas > Sent: Thursday, February 25, 2010 7:43 PM > To: SA...(a)LISTSERV.UGA.EDU > Subject: Re: dsread - Windows command-line utility for SAS7BDAT files > > Hi Alan and Chis > > What interests me is communications with open source products, > like R, perl and MySQL. Export and import needs to be lossless. My > understanding is that SAS -ODBC/OleDB... requires you to bring up > SAS(ODBC server) and the server has to be closed manually. I feel this > is unsatisfactory. The crux of the issue is IEEE floating point. I > think HEX16. would be lossless, all I have to in R is convert the > character hex16. to floating point. The nice thing about dsread is > it is light, fast and you do not need SAS and it looks like it > supports pipes. > > On Feb 25, 12:15 pm, xlr82sas <xlr82...(a)aol.com> wrote: > > On Feb 25, 10:43 am, xlr82sas <xlr82...(a)aol.com> wrote: > > > > On Feb 25, 6:45 am, ChrisBLong <ch...(a)oview.co.uk> wrote: > > > > > Hi, > > > > > Readers of this group may be interested in dsread, available > athttp://www.oview.co.uk/dsread > > > > > It's a command-line utility that understands the SAS7BDAT file > > > > format. It lets you examine the structure of datasets conveniently > > > > from the command-line, and converts SAS7BDAT data into valid CSV > > > > format for import into other software. > > > > > All comments and suggestions gratefully received, > > > > > Chris. > > > > Hi Chris, > > > > Congratulations for reading SAS datasets. WPS has powerfull > > > capabilities. Thanks!!! Competition is great. > > > > Even though you only create csv's, I see this as a great product > > > because you do not need SAS and with pipes users can programtically > > > get at SAS data from other languages. Also CSVs are also very amenable > > > to EXCEL. > > =========================================================================== > =================================================== > > > > Just some thoughts: > > > > Any chance you could create a lossless output format, like SAS > > > export datasets, but allow for longer names and > > > character values greater than 200 bytes. This would open up SAS > > > datasets to other languages. The format would have to be open. > > > > This could be a really big deal, if instead of a csv, you created R > > > dataframes, if called from R. An even bigger deal would be if you > > > created a SAS dataset from an R dataframe. > > > > XML would be another nice output. > > > > A silient ODBC would also be great. > > > > I bet you can use pipes whith yow command line interface. > > > My apologies. > > > I assumed your site was somehow affiliated with WPS. > > It looks like it may not be. > > > Which makes what you have done all the more remarkable. > > > I was hoping you honored formats because I wanted to associate hex16 > > with the numeric columns so I could create a lossless csv, but it did > > not work.- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - Hi Alan, Good question Alan. Just saw your question. Many binary values cannot be represented exactly in decimal. My understanding is that the csv format is decimal. Consider this: data lostsig; PI=constant('PI'); hex=put(PI,hex16.); put "This is an exact character representation of PI " hex=; dec=put(PI, 19.17); put "Here is an inexact decimal representation for PI " dec= //; /* LETS SEE IF WE CAN RECOVER THE TRUE VALUES FROM THE CHARACTER VALUES */ recover_from_decimal_string=input(dec, 19.17); recover_from_hex_string =input(hex,hex16.); if recover_from_hex_string = PI then put "I recovered PI exactly from the hex string"; else put "I was unable to recover PI exactly from the hex string"; if recover_from_decimal_string = PI then put "I recovered PI exactly from the decimal string"; else put "I was unable to recover PI exactly from the decimal string"; run; This is an exact character representation of PI HEX=400921FB54442D18 Here is an inexact decimal representation for PI DEC=3.14159265358979000 I recovered PI exactly from the hex string I was unable to recover PI exactly from the decimal string It is critical for serious statistical analysis that we do not lose significance when sending floats to R. I will test this with R this weekend to make sure I can go to form SAS to R and Back without loss.
From: xlr82sas on 26 Feb 2010 22:29 On Feb 26, 9:06 am, alan.church...(a)SAVIAN.NET (Alan Churchill) wrote: > Again, piping is easy. I'll try and toss something together this weekend and > pipe delimited, Excel, and XML. > > I am a bit unclear on what it is you need. If I dump the data to a delimited > file, for example, I treat the values as doubles (do not print hex > representation) so you should have better representation anyway. You can > always convert them as needed. > > Alan > > Alan Churchill > Savianwww.savian.net > Office: (719) 687-5954 > Cell: (719) 310-4870 > > > > -----Original Message----- > From: SAS(r) Discussion [mailto:SA...(a)LISTSERV.UGA.EDU] On Behalf Of > > xlr82sas > Sent: Thursday, February 25, 2010 7:43 PM > To: SA...(a)LISTSERV.UGA.EDU > Subject: Re: dsread - Windows command-line utility for SAS7BDAT files > > Hi Alan and Chis > > What interests me is communications with open source products, > like R, perl and MySQL. Export and import needs to be lossless. My > understanding is that SAS -ODBC/OleDB... requires you to bring up > SAS(ODBC server) and the server has to be closed manually. I feel this > is unsatisfactory. The crux of the issue is IEEE floating point. I > think HEX16. would be lossless, all I have to in R is convert the > character hex16. to floating point. The nice thing about dsread is > it is light, fast and you do not need SAS and it looks like it > supports pipes. > > On Feb 25, 12:15 pm, xlr82sas <xlr82...(a)aol.com> wrote: > > On Feb 25, 10:43 am, xlr82sas <xlr82...(a)aol.com> wrote: > > > > On Feb 25, 6:45 am, ChrisBLong <ch...(a)oview.co.uk> wrote: > > > > > Hi, > > > > > Readers of this group may be interested in dsread, available > athttp://www.oview.co.uk/dsread > > > > > It's a command-line utility that understands the SAS7BDAT file > > > > format. It lets you examine the structure of datasets conveniently > > > > from the command-line, and converts SAS7BDAT data into valid CSV > > > > format for import into other software. > > > > > All comments and suggestions gratefully received, > > > > > Chris. > > > > Hi Chris, > > > > Congratulations for reading SAS datasets. WPS has powerfull > > > capabilities. Thanks!!! Competition is great. > > > > Even though you only create csv's, I see this as a great product > > > because you do not need SAS and with pipes users can programtically > > > get at SAS data from other languages. Also CSVs are also very amenable > > > to EXCEL. > > =========================================================================== > =================================================== > > > > Just some thoughts: > > > > Any chance you could create a lossless output format, like SAS > > > export datasets, but allow for longer names and > > > character values greater than 200 bytes. This would open up SAS > > > datasets to other languages. The format would have to be open. > > > > This could be a really big deal, if instead of a csv, you created R > > > dataframes, if called from R. An even bigger deal would be if you > > > created a SAS dataset from an R dataframe. > > > > XML would be another nice output. > > > > A silient ODBC would also be great. > > > > I bet you can use pipes whith yow command line interface. > > > My apologies. > > > I assumed your site was somehow affiliated with WPS. > > It looks like it may not be. > > > Which makes what you have done all the more remarkable. > > > I was hoping you honored formats because I wanted to associate hex16 > > with the numeric columns so I could create a lossless csv, but it did > > not work.- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - Hi, I think you may have misunderstood me, when I said pipe, I did not mean pipe delimited. I meant filename pyp pipe "dsread.exe < data.sas7bdat"; I do plan on using dsread this weekend. I will share my experieces with you.
From: xlr82sas on 27 Feb 2010 01:09 On Feb 26, 7:29 pm, xlr82sas <xlr82...(a)aol.com> wrote: > On Feb 26, 9:06 am, alan.church...(a)SAVIAN.NET (Alan Churchill) wrote: > > > > > > > Again, piping is easy. I'll try and toss something together this weekend and > > pipe delimited, Excel, and XML. > > > I am a bit unclear on what it is you need. If I dump the data to a delimited > > file, for example, I treat the values as doubles (do not print hex > > representation) so you should have better representation anyway. You can > > always convert them as needed. > > > Alan > > > Alan Churchill > > Savianwww.savian.net > > Office: (719) 687-5954 > > Cell: (719) 310-4870 > > > -----Original Message----- > > From: SAS(r) Discussion [mailto:SA...(a)LISTSERV.UGA.EDU] On Behalf Of > > > xlr82sas > > Sent: Thursday, February 25, 2010 7:43 PM > > To: SA...(a)LISTSERV.UGA.EDU > > Subject: Re: dsread - Windows command-line utility for SAS7BDAT files > > > Hi Alan and Chis > > > What interests me is communications with open source products, > > like R, perl and MySQL. Export and import needs to be lossless. My > > understanding is that SAS -ODBC/OleDB... requires you to bring up > > SAS(ODBC server) and the server has to be closed manually. I feel this > > is unsatisfactory. The crux of the issue is IEEE floating point. I > > think HEX16. would be lossless, all I have to in R is convert the > > character hex16. to floating point. The nice thing about dsread is > > it is light, fast and you do not need SAS and it looks like it > > supports pipes. > > > On Feb 25, 12:15 pm, xlr82sas <xlr82...(a)aol.com> wrote: > > > On Feb 25, 10:43 am, xlr82sas <xlr82...(a)aol.com> wrote: > > > > > On Feb 25, 6:45 am, ChrisBLong <ch...(a)oview.co.uk> wrote: > > > > > > Hi, > > > > > > Readers of this group may be interested in dsread, available > > athttp://www.oview.co.uk/dsread > > > > > > It's a command-line utility that understands the SAS7BDAT file > > > > > format. It lets you examine the structure of datasets conveniently > > > > > from the command-line, and converts SAS7BDAT data into valid CSV > > > > > format for import into other software. > > > > > > All comments and suggestions gratefully received, > > > > > > Chris. > > > > > Hi Chris, > > > > > Congratulations for reading SAS datasets. WPS has powerfull > > > > capabilities. Thanks!!! Competition is great. > > > > > Even though you only create csv's, I see this as a great product > > > > because you do not need SAS and with pipes users can programtically > > > > get at SAS data from other languages. Also CSVs are also very amenable > > > > to EXCEL. > > > =========================================================================== > > =================================================== > > > > > Just some thoughts: > > > > > Any chance you could create a lossless output format, like SAS > > > > export datasets, but allow for longer names and > > > > character values greater than 200 bytes. This would open up SAS > > > > datasets to other languages. The format would have to be open. > > > > > This could be a really big deal, if instead of a csv, you created R > > > > dataframes, if called from R. An even bigger deal would be if you > > > > created a SAS dataset from an R dataframe. > > > > > XML would be another nice output. > > > > > A silient ODBC would also be great. > > > > > I bet you can use pipes whith yow command line interface. > > > > My apologies. > > > > I assumed your site was somehow affiliated with WPS. > > > It looks like it may not be. > > > > Which makes what you have done all the more remarkable. > > > > I was hoping you honored formats because I wanted to associate hex16 > > > with the numeric columns so I could create a lossless csv, but it did > > > not work.- Hide quoted text - > > > > - Show quoted text -- Hide quoted text - > > > - Show quoted text - > > Hi, > > I think you may have misunderstood me, when I said pipe, I did not > mean pipe delimited. > > I meant > > filename pyp pipe "dsread.exe < data.sas7bdat"; > > I do plan on using dsread this weekend. > > I will share my experieces with you.- Hide quoted text - > > - Show quoted text - FYI I just posted this to the R group If I do the following sprintf("%A",pi) "0X1.921FB54442D18" I have this 16 byte character string hx<-"400921FB54442D18" This is the exact hex16 representation of PI in IEEE float that R uses in Intel 32bit(little endian) Windows SAS uses the same representation. 11 bit exponent and 53 bit mantissa. I want to do is recreate the float exactly from the 16 char hex something like MyPI<-readChar(hx,numeric(),16) or in SAS MyPI=input("400921FB54442D18",hex16.); put MyPI=; MYPI=3.1415926536 What I am trying to do is set up a lossless transfer method from SAS to R
First
|
Prev
|
Next
|
Last
Pages: 1 2 3 4 Prev: update a column based on a column in another table Next: Unstacking Data |