Why does %include correct memory problem?
in [SAS]
|
From: Joe Matise on 9 Nov 2009 10:09 Actually, it involves a long, NOT automatically generated rename statement (hard coded). Your explanation makes a lot of sense - it's possible that after some point (either the length limit, or a shorter length) the EE has trouble and ends up inserting funny characters trying to deal with that 'trouble' (or just as a result of the trouble). -Joe On Mon, Nov 9, 2009 at 9:03 AM, Jonathan Goldberg <jgoldberg(a)biomedsys.com>wrote: > I gather that the original problem involved a long, automatically > generated, rename statement. > > I began to wonder about the "long" part. What are the length limitations > of SAS statements? I can't find anything in the manual but the SAS > knowledge base included this: > > Usage Note 15883: Length limitations when submitting SAS code > > There are many ways to submit SAS code and the length limitation is > different depending on how you submit your code. The following are > the length limitations as imposed by the various submission methods > prior to SAS 9.2 and for SAS 9.2 and beyond: > > * Submitting a SAS program via a batch file imposes a maximum > length limit of 256 characters prior to SAS 9.2. For SAS 9.2 > and beyond you can use the new SAS system option of LRECL= to > specify a range of 1-32767 to use for reading and writing > external files. The default for the LRECL= system option is > 256. This option is valid in the configuration file, at SAS > invocation, the OPTIONS statement, or the SAS Systems Option > window. Under z/OS, the LRECL= system option is recognized only > for reading and writing HFS files. > > * Submitting a SAS program from the Program Editor imposes a > maximum length limit of 384 characters prior to SAS 9.2. For > SAS 9.2 and beyond you can use the new SAS system option of > DMSPGMLINESIZE= to specify a range of 136-960 to specify the > maximum number of characters in a Program Editor line. The > default for the DMSPGMLINESIZE= system option is 136. This > option is valid in the configuration file or at SAS invocation. > > * Submitting a SAS program from the Enhanced Editor imposes a > maximum length limit of 960 characters prior to SAS 9.2. For > SAS 9.2 and beyond the maximum length is extended to 6,000 > characters. > > * Submitting SAS code using the IOM submit via the metadata server > or a stored process server imposes a maximum length limit of > 2,000 characters prior to SAS 9.1. For SAS 9.1 the maximum > length limit extends to 6,000 characters. For SAS 9.2 and > beyond the maximum length limit extends to 32,767 characters. > > * Submitting SAS code using %include <file> imposes a maximum > length limit of 2,000 characters prior to SAS 9.1. For SAS 9.1 > and beyond the maximum length limit is 6,000 characters. > > For specific details under your operating system, see the SAS > Companion documentation for your operating environment. > > So, prior to Version 9.2 the enhanced editor had a length limitation of > 960 and %include had a limit of 2000. > > Is this consistent with the error message received? I wish I knew. >
From: Sigurd Hermansen on 9 Nov 2009 12:48 Doug: Tom mentions possibilities that I hadn't considered. Does the first instance of the strange characters error coincide with a statement that breaks across two lines? Does the original INCLUDE'd file have some form of automatic line number or other prefix, or padding, that SAS might misinterpret as text? Does placing the variable rsavry1c in a COMPRESS or LEFT function and leaving the original %INCLUDE file the same eliminate that error? Sounds as if a particular set of circumstances might trigger an error during the read of the %INCLUDE file but not appear in the file itself. S -----Original Message----- From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Doug Cacialli Sent: Sunday, November 08, 2009 11:08 PM To: SAS-L(a)LISTSERV.UGA.EDU Subject: Re: Why does %include correct memory problem? I'll take a look in a hex editor tonight or tomorrow and report back. I think if that's the case, it has to be SAS inserting those characters ... I retyped that line and the lines surrounding it so many times. In a few instances, the sequence occurred elsewhere but always nearby (when I omitted the line altogether, for instance). I'm unsure why this would occur in a system-specific manner, however; Art was able to run the script without issue. Maybe it's version specific .... I believe he's running 9.1.3. I do find that possibility rather disturbing (but not entirely subversive :-). -Doug. === Douglas Cacialli, M.A. - Doctoral candidate Clinical Psychology Training Program University of Nebraska-Lincoln Lincoln, Nebraska 68588-0308 === On Sun, Nov 8, 2009 at 9:49 PM, Sigurd Hermansen <HERMANS1(a)westat.com> wrote: > Art: > I don't see why breaking a %INCLUDE'd file into multiple files should correct an error due to an invisible sequence of non-printing (and, perhaps, non-displaying characters). In my range of experience, %INCLUDE merely shifts the program pointer from the file that contains the %INCLUDE to another file. > > Breaking a program file into subfiles may save files in a way that eliminates aberrant characters. Has Doug looked at the savry1crit = ??? �??? rsavry1c; statement in the original file in a hex editor? The fact that the error repeats in successive program executions suggests that the original file includes unknown to us a sequence of characters that doesn't display in a text editor. Were SAS responsible for inserting nuisance characters in a program, it would more likely happen at the beginning or end of the %INCLUDE'd script. > > As a general rule, one does not always get what you see (WYSISNOTWYG) in ASCII or EBCDIC files. Some find that disturbing, or even subversive, but computers explore the full range of alternatives, even if we don't realize they exist. > S > > -----Original Message----- > From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of Arthur Tabachneck > Sent: Sunday, November 08, 2009 6:11 PM > To: SAS-L(a)LISTSERV.UGA.EDU > Subject: Why does %include correct memory problem? > > Doug ran into a problem, yesterday, because his recode statement was too > long for his system, editor or some combination thereof. > > Joe and I suggested breaking the recodes into a number of data steps, and > Joe also suggested breaking the code into multiple recode statements, but > using %include to incorporate them into one data step. > > As Doug mentioned, Joe's suggestion worked. � My question is why? > > I presume it is because the code is executed as soon as it is %include'd, > but I'd definitely appreciate confirmation of that and, better yet, some > direction regarding where to look in the manual(s) that would suggest using > %include to correct for such problems. > > Art > --------- > On Sun, 8 Nov 2009 10:35:47 -0600, Doug Cacialli <doug.cacialli(a)GMAIL.COM> > wrote: > >>This problem has been solved, although I'm unable to explain why. >>Cutting-and-pasting the offending data step into a separate script and >>using %INCLUDE in it's place produced the expected behavior (e.g., >>everything is appropriately renamed). � Thanks to Joe Matise and Art >>Tabachneck for their excellent suggestions. >> >>-Doug. >> >>=== >>Douglas Cacialli, M.A. - Doctoral candidate >>Clinical Psychology Training Program >>University of Nebraska-Lincoln >>Lincoln, Nebraska 68588-0308 >>=== >> >> >> >>On Sat, Nov 7, 2009 at 3:45 PM, Doug Cacialli <doug.cacialli(a)gmail.com> > wrote: >>> I'm running SAS 9.2 in both virtualized XP Pro (Virtual PC on Win 7) >>> and Vista Ultimate environments. � The error occurs in both OS's. >>> >>> I'm renaming a few hundred variables, so if length could be an issue >>> I've most likely hit the point where it will be an issue. � Some of the >>> reliability variables are tweaked slightly, so renaming groups of >>> variables (e.g., RENAME var1-var100 = rvar1-rvar100) won't work. � I'll >>> try your suggestion, see what happens, and let you know the result. >>> >>> I searched the archives, but nothing jumped out at me. � I'll take >>> another look there too. � Thanks SO much. >>> >>> -Doug. >>> >>> === >>> Douglas Cacialli, M.A. - Doctoral candidate >>> Clinical Psychology Training Program >>> University of Nebraska-Lincoln >>> Lincoln, Nebraska 68588-0308 >>> === >>> >>> >>> >>> On Sat, Nov 7, 2009 at 3:36 PM, Joe Matise <snoopy369(a)gmail.com> wrote: >>>> That's been a problem posted on the list before... � I don't recall > exactly >>>> what you do, but I think it's related to the length of the program or >>>> something else. >>>> >>>> Are you using 9.1.3? � The version would help, it's possible it's been > fixed >>>> in a newer version. >>>> >>>> Anyhow, if it's length related, try taking half of the program or so and >>>> putting it in a second file, and either just running both programs >>>> separately one after the other, or using %include to run the second half >>>> from the first half. � I'm not sure, but my vague memory tells me that > might >>>> be the solution that was found before. >>>> >>>> -Joe >>>> >>>> >>>> >>>> On Sat, Nov 7, 2009 at 1:05 PM, Doug Cacialli <doug.cacialli(a)gmail.com> >>>> wrote: >>>>> >>>>> Greetings SAS-L, >>>>> >>>>> I've encountered a troublesome little niggle that's led me to re-join >>>>> the SAS-L listserv after being gone for a couple years. >>>>> >>>>> I'm working with a data set that's been passed on from >>>>> person-to-person for the past few years. � Half of the data set >>>>> contains reliability data (data coded by a second rater) structured >>>>> such that there's one observation per ID, and the second coder's data >>>>> is contained in variables with an "r" prefix. � The other half of the >>>>> data set contains reliability data structured such that the second >>>>> coder's data comprises a second observation with the same ID. � I'm >>>>> trying to write a script that will: (1) separate out any observations >>>>> with duplicate ID's, and; (2) rename the variables with an "r" prefix, >>>>> so that they can then be merged back in, resulting in a homogeneously >>>>> structured data set. >>>>> >>>>> Everything works as I except it to, except the RENAME statement. � I'm >>>>> encountering the following error in the log: >>>>> >>>>> ... >>>>> 2215 savryevaluator = revaluat >>>>> 2216 savry1 = rsavry1 >>>>> 2217 savry1crit = ??? �??? rsavry1c; >>>>> � � � � � � � � � � � � � - � � - >>>>> � � � � � � � � � � � � � 22 � 200 >>>>> � � � � � � � � � � � � � 200 >>>>> ERROR 22-322: Expecting a name. >>>>> >>>>> ERROR 200-322: The symbol is not recognized and will be ignored. >>>>> � � � � � � � � � � � � � � � � � - >>>>> � � � � � � � � � � � � � � � � � 24 >>>>> ERROR 24-322: Variable name is not valid. >>>>> >>>>> 2217! savry1crit = ??? �??? rsavry1c; >>>>> � � � � � � � � � � � � � � � � � � � � � - >>>>> � � � � � � � � � � � � � � � � � � � � � 22 >>>>> ERROR 22-322: Syntax error, expecting one of the following: a name, -, > :, >>>>> ;. >>>>> >>>>> 2218 � * � � � � � � � savry2crit = � � � � � � � � � � rsavry2c >>>>> ERROR: Missing numeric suffix on a numbered variable list (�- > rsavry1c). >>>>> ERROR: Old and new variable name specifications for RENAME must be of >>>>> the same type. Statement >>>>> � � � is ignored. >>>>> 2219 savry3 = � � � � � � � � � � � � rsavry3 >>>>> 2220 savry3crit = � � � � � � � � � � rsavry3c >>>>> 2221 savry4 = � � � � � � � � � � � � rsavry4 >>>>> ... >>>>> >>>>> That jumbled sequence of characters in line 2217 doesn't appear in my >>>>> script. � I tried re-writing that line and the lines surrounding it and >>>>> it reappears in the log when I run the script. � I've tried looking at >>>>> the file in notepad, but it looks the same in the enhanced editor. >>>>> >>>>> Does anyone have any suggestions here? � I'd be happy to learn either >>>>> what's going on to generate the error, or another way to accomplish >>>>> the task. � Any help would be appreciated. >>>>> >>>>> Thanks everyone. >>>>> >>>>> -Doug. >>>>> >>>>> === >>>>> Douglas Cacialli, M.A. - Doctoral candidate >>>>> Clinical Psychology Training Program >>>>> University of Nebraska-Lincoln >>>>> Lincoln, Nebraska 68588-0308 >>>>> === >>>> >>>> >>> >
From: rjf2 on 9 Nov 2009 22:20 strange characters in log which abend sas indicate that the text file of the program may contain tab characters. These tabs are easily seen in a professional text editor such as UltraEdit and easily changed: <alt> Format Tabs to spaces there is a SAS note on this. Ron the editor maven -----Original Message----- From: owner-sas-l(a)listserv.uga.edu [mailto:owner-sas-l(a)listserv.uga.edu] On Behalf Of Arthur Tabachneck Sent: Sunday, November 08, 2009 6:11 PM To: SAS-L(a)LISTSERV.UGA.EDU; Doug Cacialli Subject: Why does %include correct memory problem? Doug ran into a problem, yesterday, because his recode statement was too long for his system, editor or some combination thereof. Joe and I suggested breaking the recodes into a number of data steps, and Joe also suggested breaking the code into multiple recode statements, but using %include to incorporate them into one data step. As Doug mentioned, Joe's suggestion worked. My question is why? I presume it is because the code is executed as soon as it is %include'd, but I'd definitely appreciate confirmation of that and, better yet, some direction regarding where to look in the manual(s) that would suggest using %include to correct for such problems. Art --------- On Sun, 8 Nov 2009 10:35:47 -0600, Doug Cacialli <doug.cacialli(a)GMAIL.COM> wrote: >This problem has been solved, although I'm unable to explain why. >Cutting-and-pasting the offending data step into a separate script and >using %INCLUDE in it's place produced the expected behavior (e.g., >everything is appropriately renamed). Thanks to Joe Matise and Art >Tabachneck for their excellent suggestions. > >-Doug. > >=== >Douglas Cacialli, M.A. - Doctoral candidate Clinical Psychology >Training Program University of Nebraska-Lincoln Lincoln, Nebraska >68588-0308 === > > > >On Sat, Nov 7, 2009 at 3:45 PM, Doug Cacialli <doug.cacialli(a)gmail.com> wrote: >> I'm running SAS 9.2 in both virtualized XP Pro (Virtual PC on Win 7) >> and Vista Ultimate environments. The error occurs in both OS's. >> >> I'm renaming a few hundred variables, so if length could be an issue >> I've most likely hit the point where it will be an issue. Some of >> the reliability variables are tweaked slightly, so renaming groups of >> variables (e.g., RENAME var1-var100 = rvar1-rvar100) won't work. >> I'll try your suggestion, see what happens, and let you know the result. >> >> I searched the archives, but nothing jumped out at me. I'll take >> another look there too. Thanks SO much. >> >> -Doug. >> >> === >> Douglas Cacialli, M.A. - Doctoral candidate Clinical Psychology >> Training Program University of Nebraska-Lincoln Lincoln, Nebraska >> 68588-0308 === >> >> >> >> On Sat, Nov 7, 2009 at 3:36 PM, Joe Matise <snoopy369(a)gmail.com> wrote: >>> That's been a problem posted on the list before... I don't recall exactly >>> what you do, but I think it's related to the length of the program >>> or something else. >>> >>> Are you using 9.1.3? The version would help, it's possible it's >>> been fixed >>> in a newer version. >>> >>> Anyhow, if it's length related, try taking half of the program or so >>> and putting it in a second file, and either just running both >>> programs separately one after the other, or using %include to run >>> the second half from the first half. I'm not sure, but my vague >>> memory tells me that might >>> be the solution that was found before. >>> >>> -Joe >>> >>> >>> >>> On Sat, Nov 7, 2009 at 1:05 PM, Doug Cacialli >>> <doug.cacialli(a)gmail.com> >>> wrote: >>>> >>>> Greetings SAS-L, >>>> >>>> I've encountered a troublesome little niggle that's led me to >>>> re-join the SAS-L listserv after being gone for a couple years. >>>> >>>> I'm working with a data set that's been passed on from >>>> person-to-person for the past few years. Half of the data set >>>> contains reliability data (data coded by a second rater) structured >>>> such that there's one observation per ID, and the second coder's >>>> data is contained in variables with an "r" prefix. The other half >>>> of the data set contains reliability data structured such that the >>>> second coder's data comprises a second observation with the same >>>> ID. I'm trying to write a script that will: (1) separate out any >>>> observations with duplicate ID's, and; (2) rename the variables >>>> with an "r" prefix, so that they can then be merged back in, >>>> resulting in a homogeneously structured data set. >>>> >>>> Everything works as I except it to, except the RENAME statement. >>>> I'm encountering the following error in the log: >>>> >>>> ... >>>> 2215 savryevaluator = revaluat >>>> 2216 savry1 = rsavry1 >>>> 2217 savry1crit = ??? �??? rsavry1c; >>>> - - >>>> 22 200 >>>> 200 >>>> ERROR 22-322: Expecting a name. >>>> >>>> ERROR 200-322: The symbol is not recognized and will be ignored. >>>> - >>>> 24 ERROR 24-322: Variable name is >>>> not valid. >>>> >>>> 2217! savry1crit = ??? �??? rsavry1c; >>>> - >>>> 22 ERROR 22-322: Syntax >>>> error, expecting one of the following: a name, -, :, >>>> ;. >>>> >>>> 2218 * savry2crit = rsavry2c >>>> ERROR: Missing numeric suffix on a numbered variable list (�- rsavry1c). >>>> ERROR: Old and new variable name specifications for RENAME must be >>>> of the same type. Statement >>>> is ignored. >>>> 2219 savry3 = rsavry3 >>>> 2220 savry3crit = rsavry3c >>>> 2221 savry4 = rsavry4 >>>> ... >>>> >>>> That jumbled sequence of characters in line 2217 doesn't appear in >>>> my script. I tried re-writing that line and the lines surrounding >>>> it and it reappears in the log when I run the script. I've tried >>>> looking at the file in notepad, but it looks the same in the enhanced editor. >>>> >>>> Does anyone have any suggestions here? I'd be happy to learn >>>> either what's going on to generate the error, or another way to >>>> accomplish the task. Any help would be appreciated. >>>> >>>> Thanks everyone. >>>> >>>> -Doug. >>>> >>>> === >>>> Douglas Cacialli, M.A. - Doctoral candidate Clinical Psychology >>>> Training Program University of Nebraska-Lincoln Lincoln, Nebraska >>>> 68588-0308 === >>> >>> >>
First
|
Prev
|
Pages: 1 2 Prev: what is this and how do I fix it? Proc GMAP Next: Delete adjacent obs with repeated info |