Scoping with Module
in [Mathematica]
|
From: Patrick Scheibe on 6 May 2010 04:52 Hi, I cannot explain the following behaviour of Module. Maybe someone else can: in the global scope f[t_] = t; DownValues[f] results in {HoldPattern[f[t_]] :> t} as expected. Simply wrapping it with Module, without making f lokal does not change this behaviour. Module[{}, f[t_] = t; DownValues[f] ] results still in {HoldPattern[f[t_]] :> t}. Say I want to replace the right side of the definition with a rule r = {t :> 1}; f[t_] = t /. r; DownValues[f] and we get {HoldPattern[f[t_]] :> 1} Module[{}, r = {t :> 1}; f[t_] = t /. r; DownValues[f] ] still the expected result {HoldPattern[f[t_]] :> 1} Now (sorry for the lengthy introduction) making r lokal Module[{r}, r = {t :> 1}; f[t_] = t /. r; DownValues[f] ] and we get {HoldPattern[f[t$_]] :> t$}. The Pattern t_ was renamed before the replacement could act. Using the local variable r to set a global r2 and it's working again. Module[{r}, r = {t :> 1}; r2 = r; f[t_] = t /. r2; DownValues[f] ] So there must be at least one thing about Module I did'nt understand. I thought the main difference between Block and Module is the lexical scoping!? But the only thing which was renamed is the lokal usage of r. Any ideas? Cheers Patrick 7.0 for Mac OS X x86 (64-bit) (February 19, 2009)
From: Leonid Shifrin on 7 May 2010 06:23 Hi Patrick, My understanding is that Mathematica uses aggressive variable renaming (more than is strictly necessary for many cases) in case it sees nested scoping constructs with conflicting variables (variables with the same name). This manifests itself in the renaming happening if these variables occur not just in the declaration lists, but also in the body, like here: scoping1[{a},scoping2[{b},a+b] In such cases, <b> will be often renamed to something like b$ even before the actual binding happens, for example: In[1]:= Function[a, Function[b, Print[a + b]]][c] Out[1]= Function[b$, Print[c + b$]] Note that in many cases this is not strictly necessary, but it is probably easier and / or more efficient to do this excessive renaming than to work on the case-by-case basis. Now, both Set with patterns and Module are scoping constructs. In your starting examples there are no local variables in the Module declaration , so no conflicts of the above type are possible. However, in this one: Module[{r}, r = {t :> 1}; f[t_] = t /. r; DownValues[f] ] you do have a possibility of a conflict of the above type, and so Mathematica renames a dummy variable in the inner scoping construct - <t> in f[t_] = t /. r; in this case. At the same time, <t> in the r = {t :> 1}; remains global since no patterns / scoping is involved here - thus the result you observed. In your final example: Module[{r}, r = {t :> 1}; r2 = r; f[t_] = t /. r2; DownValues[f] ] while the situation looks pretty similar from the evaluation viewpoint, <r> does not appear inside the definition f[t_] = t /. r2; lexically (this happens only dynamically), and for lexical scoping constructs (at least as they exist in Mathematica) this makes all the difference. At the time of variable name conflict resolution (this is before any code inside Module starts to execute), Mathematica can no longer sense any danger, and thus no renaming happens. I don't know whether this behavior should be sonsidered satisfactory or not, since it is certainly not very intuitive. I'd say that it's still ok: when we take a line like f[t_] = t /. r;, we should treat <t> as a local lexically scoped variable (the fact that it evaluates to global <t> is related to evaluation, not scoping). Therefore, there is generally no guarantee that injecting there r = {t :> 1}; (which is defined outside the scope of Set in f[t_] = t /. r;) will work. I think that our puzzlement originates from mixing scoping and evaluation. From the fact that Set evaluates the r.h.s and uses global values of <t> for entries of <t> on the r.h.s. (this is what we are used to from our experiences of working with global variables), we subconsciously expect that this is true for all entries of <t> on the r.h.s, while this is only guaranteed for non-lexical entries of <t> - those that were not explicitly (lexically), but rather dynamically, present in the code before evaluation started. The problem is made worse by the fact that in many common cases it does work (like the case of global variables and no surrounding scoping constructs). The bottom line is that IMO the construct you looked at is not reliable from the start in that it attempts to do scope engineering and the results are then necessarily dependent on the details of Mathematica internal implementation of lexical scoping constructs. Then it is probably a matter of taste whether or not to use constructs like this. Personally, I do it from time to time for my own purposes, but I would hesitate to put them into "production" code. Besides, I believe that in most cases there exist more reliable alternatives. Regards, Leonid On Thu, May 6, 2010 at 12:52 PM, Patrick Scheibe < pscheibe(a)trm.uni-leipzig.de> wrote: > Hi, > > I cannot explain the following behaviour of Module. Maybe someone else > can: > > in the global scope > > f[t_] = t; > DownValues[f] > > results in {HoldPattern[f[t_]] :> t} as expected. Simply wrapping it > with Module, without > making f lokal does not change this behaviour. > > Module[{}, > f[t_] = t; > DownValues[f] > ] > > results still in {HoldPattern[f[t_]] :> t}. > Say I want to replace the right side of the definition with a rule > > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > > and we get {HoldPattern[f[t_]] :> 1} > > Module[{}, > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > ] > > still the expected result {HoldPattern[f[t_]] :> 1} > > Now (sorry for the lengthy introduction) making r lokal > > Module[{r}, > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > ] > > and we get {HoldPattern[f[t$_]] :> t$}. The Pattern t_ was renamed > before the replacement could act. > Using the local variable r to set a global r2 and it's working again. > > Module[{r}, > r = {t :> 1}; > r2 = r; > f[t_] = t /. r2; > DownValues[f] > ] > > So there must be at least one thing about Module I did'nt understand. > I thought the main difference between Block > and Module is the lexical scoping!? But the only thing which was > renamed is the lokal usage of r. > Any ideas? > > Cheers > Patrick > > 7.0 for Mac OS X x86 (64-bit) (February 19, 2009) > >
From: Patrick Scheibe on 11 May 2010 06:26 Hi, Thanks Leonid for your answer. From the point of a user I find it very hard to explain to a newbie (who started to see the usefulness of lokalized variables) why this works sol = First@ NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y[x], {x, 0, 30}]; func[x_] = y[x] /. sol; Plot[func[x], {x, 0, 30}] while this doen't Module[{sol, y, x, func}, sol = First@ NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y[x], {x, 0, 30}]; func[x_] = y[x] /. sol; Plot[func[x], {x, 0, 30}] ] Additionally, nothing prevents you from doing something like this t$ = 1; Module[{f, g, t}, g[t_] := Sin[t]; f[t_] = g[t]; DownValues[f] ] Out[2]= {HoldPattern[f$560[t$_]] :> Sin[1]} Since most system variables are named like $... there is maybe someone who accidently uses t$ as additional variable and gets some really strange behavior. The whole point is really not that important but I just thought that there is more behind the idea of the scoping inside Module. Why doesn't has Block this behavoir? Cheers Patrick Am May 6, 2010 um 12:37 PM schrieb Leonid Shifrin: > Hi Patrick, > > My understanding is that Mathematica uses aggressive variable > renaming (more than is strictly necessary for many cases) in case it > sees nested scoping constructs with conflicting variables (variables > with the same name). This manifests itself in the renaming > happening if these variables > occur not just in the declaration lists, but also in the body, like > here: > > scoping1[{a},scoping2[{b},a+b] > > In such cases, <b> will be often renamed to something like b$ even > before the actual binding happens, > for example: > > In[1]:= Function[a, Function[b, Print[a + b]]][c] > > Out[1]= Function[b$, Print[c + b$]] > > Note that in many cases this is not strictly necessary, but it is > probably easier and / or more efficient to do this excessive > renaming than to work on the case-by-case basis. Now, both Set with > patterns and Module are scoping constructs. In your starting > examples there are no local variables in the Module declaration , so > no conflicts of the above type are possible. However, in this one: > > Module[{r}, > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > ] > > you do have a possibility of a conflict of the above type, and so > Mathematica renames a dummy variable in the inner scoping construct > - <t> in f[t_] = t /. r; in this case. At the same time, <t> in the > r = {t :> 1}; remains global since no patterns / scoping is involved > here - thus the result you observed. In your final example: > > Module[{r}, > r = {t :> 1}; > r2 = r; > f[t_] = t /. r2; > DownValues[f] > ] > > while the situation looks pretty similar from the evaluation > viewpoint, <r> does not appear inside the definition f[t_] = t /. > r2; lexically (this happens only dynamically), and for lexical > scoping constructs (at least as they exist in Mathematica) this > makes all the difference. At the time of variable name conflict > resolution (this is before any code inside Module starts to > execute), Mathematica can no longer sense any danger, and thus no > renaming happens. > > I don't know whether this behavior should be sonsidered satisfactory > or not, since it is certainly not very intuitive. I'd say that it's > still ok: when we take a line like f[t_] = t /. r;, we should treat > <t> as a local lexically scoped variable (the fact that it evaluates > to global <t> is related to evaluation, not scoping). Therefore, > there is generally no guarantee that injecting there r = {t :> 1}; > (which is defined outside the scope of Set in f[t_] = t /. r;) will > work. I think that our puzzlement originates from mixing scoping and > evaluation. From the fact that Set evaluates the r.h.s and uses > global values of <t> for entries of <t> on the r.h.s. (this is what > we are used to from our experiences of working with global > variables), we subconsciously expect that this is true for all > entries of <t> on the r.h.s, while this is only guaranteed for non- > lexical entries of <t> - those that were not explicitly (lexically), > but rather dynamically, present in the code before evaluation > started. The problem is made worse by the fact that in many common > cases it does work (like the case of global variables and no > surrounding scoping constructs). > > The bottom line is that IMO the construct you looked at is not > reliable from the start in that it attempts to do scope engineering > and the results are then necessarily dependent on the details of > Mathematica internal implementation of lexical scoping constructs. > Then it is probably a matter of taste whether or not to use > constructs like this. Personally, I do it from time to time for my > own purposes, but I would hesitate to put them into "production" > code. Besides, I believe that in most cases there exist more > reliable alternatives. > > Regards, > Leonid > > > > On Thu, May 6, 2010 at 12:52 PM, Patrick Scheibe <pscheibe(a)trm.uni-leipzig.de > > wrote: > Hi, > > I cannot explain the following behaviour of Module. Maybe someone else > can: > > in the global scope > > f[t_] = t; > DownValues[f] > > results in {HoldPattern[f[t_]] :> t} as expected. Simply wrapping it > with Module, without > making f lokal does not change this behaviour. > > Module[{}, > f[t_] = t; > DownValues[f] > ] > > results still in {HoldPattern[f[t_]] :> t}. > Say I want to replace the right side of the definition with a rule > > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > > and we get {HoldPattern[f[t_]] :> 1} > > Module[{}, > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > ] > > still the expected result {HoldPattern[f[t_]] :> 1} > > Now (sorry for the lengthy introduction) making r lokal > > Module[{r}, > r = {t :> 1}; > f[t_] = t /. r; > DownValues[f] > ] > > and we get {HoldPattern[f[t$_]] :> t$}. The Pattern t_ was renamed > before the replacement could act. > Using the local variable r to set a global r2 and it's working again. > > Module[{r}, > r = {t :> 1}; > r2 = r; > f[t_] = t /. r2; > DownValues[f] > ] > > So there must be at least one thing about Module I did'nt understand. > I thought the main difference between Block > and Module is the lexical scoping!? But the only thing which was > renamed is the lokal usage of r. > Any ideas? > > Cheers > Patrick > > 7.0 for Mac OS X x86 (64-bit) (February 19, 2009) > >
From: Leonid Shifrin on 12 May 2010 07:33 Hi Patrick, >From the point of a user I find it very hard to explain to a newbie (who > started to see the usefulness of lokalized variables) why this works > > sol = First@ > NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y[x], {x, 0, 30}]; > func[x_] = y[x] /. sol; > Plot[func[x], {x, 0, 30}] > > while this doen't > > Module[{sol, y, x, func}, > sol = First@ > NDSolve[{y'[x] == y[x] Cos[x + y[x]], y[0] == 1}, y[x], {x, 0, 30}]; > func[x_] = y[x] /. sol; > Plot[func[x], {x, 0, 30}] > ] > Yes, I agree that this behavior is not intuitive. But I'd say that what the novice user should be told up front is that she should *not* expect the first version of your example or similar constructs to work, in the first place. In principle, in func[x_] = y[x] /. sol, the system is free to rename x in all lexical occurrences, it just does not do it. So, I think that in a sense we should blame our own expectations in such cases - we are just too used to "good". I think that the documentation should make much more clear the distinction between scoping stage and evaluation stage. The fact that your first example works is a consequence of both the scoping stage behavior (which happens first), and then the evaluation. In a way, it works only due to the "good will" of the scoping mechanism which in this case did not rename things. We should not count on it. So, IMO what must be explained is not why the second example did not work (this I think is normal), but why the first did. > > Additionally, nothing prevents you from doing something like this > > t$ = 1; > Module[{f, g, t}, > g[t_] := Sin[t]; > f[t_] = g[t]; > DownValues[f] > ] > > Out[2]= {HoldPattern[f$560[t$_]] :> Sin[1]} > > Since most system variables are named like $... there is maybe someone who > accidently uses t$ as additional variable and gets some really strange > behavior. > Yes, I think this is another important point. Lexical scoping in Mathematica is emulated by symbol renaming, and since it is an emulation, it can be broken. I think there were many discussions of this issue here on the group, and the bottom line of all these discussions seems to have been to avoid using symbols with a dollar sign in their names. > > The whole point is really not that important but I just thought that there > is more behind the idea of the scoping inside Module. Why doesn't has Block > this behavoir? > I think you mean cases like this: In[1]:= Module[{a}, Hold[Function[b, Print[a + b]][c]]] Out[1]= Hold[Function[b$, Print[a$592 + b$]][c]] In[2]:= Block[{a}, Hold[Function[b, Print[a + b]][c]]] Out[2]= Hold[Function[b, Print[a + b]][c]] My guess is that the implementation of lexical scoping in Mathematica is well-aware of the difference between dynamic and lexical scoping. Block is not dangerous to inner lexical scoping constructs really: its effect is dynamical, that is - it shows up only at evaluation stage which is already after all lexical bindings (together with name collision resolution etc) take place. To put it differently, the action of Block is equivalent to a (temporal) change in the global state and does not affect lexical scoping per se, only evaluation. Therefore, for constructs that do not ordinarily produce new symbols (With, Function,Set,SetDelayed), it is guaranteed to be happening already after they bind their variables. Rule and RuleDelayed seem to protect their variables against any (dynamic or lexical) outer scoping construct except themselves, without renaming. For Module, OTOH, Block is harmless simply because Module produces unique symbols to which external Block-s can not normally have access (since Block binds dynamically based on the exact name occurrences), unless again we take some deliberate steps to break the scoping by for example feeding special symbols to Block. Outer lexical scoping constructs are however potentially dangerous to inner ones, since they bind their variables to the body from outside to inside. So, my guess would be that, when we start to execute code with nested lexical scoping constructs, the most external scoping construct first "looks" at the (held) code(its body) and does the necessary renamings in inner scoping constructs, then binds its own local variables (after renamings in inner scoping construsts have happened), and finally executes its body. At least, this is what I think should be happening (I may be wrong of course). OTOH, we also know that Rule and RuleDelayed are different from other lexical scoping constructs in that they do not respect the inner ones (unlike say With, Module, Function), which IMO has both a good side (ability to do scope surgery with them), and a bad side (lots of opportunities for nasty and subtle scoping bugs). Anyway, to conclude, one thing I am sure of is this: given that the details of internal implementation of scoping constructs (renaming scheme etc) seem to be manifesting themselves pretty often in examples like the one you gave, the documentation could at least be more clear and perhaps give some more details on what is going on behind the scenes, so that at least some of these things could be understood logically rather than guessed or memorized like magic spells (for example, I have no idea what percent of my guesses above about the scoping mechanism is correct). Regards, Leonid > Cheers > Patrick > > Am May 6, 2010 um 12:37 PM schrieb Leonid Shifrin: > > > Hi Patrick, >> >> My understanding is that Mathematica uses aggressive variable renaming >> (more than is strictly necessary for many cases) in case it sees nested >> scoping constructs with conflicting variables (variables with the same >> name). This manifests itself in the renaming happening if these variables >> occur not just in the declaration lists, but also in the body, like here: >> >> scoping1[{a},scoping2[{b},a+b] >> >> In such cases, <b> will be often renamed to something like b$ even before >> the actual binding happens, >> for example: >> >> In[1]:= Function[a, Function[b, Print[a + b]]][c] >> >> Out[1]= Function[b$, Print[c + b$]] >> >> Note that in many cases this is not strictly necessary, but it is probably >> easier and / or more efficient to do this excessive renaming than to work on >> the case-by-case basis. Now, both Set with patterns and Module are scoping >> constructs. In your starting examples there are no local variables in the >> Module declaration , so no conflicts of the above type are possible. >> However, in this one: >> >> Module[{r}, >> r = {t :> 1}; >> f[t_] = t /. r; >> DownValues[f] >> ] >> >> you do have a possibility of a conflict of the above type, and so >> Mathematica renames a dummy variable in the inner scoping construct - <t> in >> f[t_] = t /. r; in this case. At the same time, <t> in the >> r = {t :> 1}; remains global since no patterns / scoping is involved here >> - thus the result you observed. In your final example: >> >> Module[{r}, >> r = {t :> 1}; >> r2 = r; >> f[t_] = t /. r2; >> DownValues[f] >> ] >> >> while the situation looks pretty similar from the evaluation viewpoint, >> <r> does not appear inside the definition f[t_] = t /. r2; lexically (this >> happens only dynamically), and for lexical scoping constructs (at least as >> they exist in Mathematica) this makes all the difference. At the time of >> variable name conflict resolution (this is before any code inside Module >> starts to execute), Mathematica can no longer sense any danger, and thus no >> renaming happens. >> >> I don't know whether this behavior should be sonsidered satisfactory or >> not, since it is certainly not very intuitive. I'd say that it's still ok: >> when we take a line like f[t_] = t /. r;, we should treat <t> as a local >> lexically scoped variable (the fact that it evaluates to global <t> is >> related to evaluation, not scoping). Therefore, there is generally no >> guarantee that injecting there r = {t :> 1}; (which is defined outside the >> scope of Set in f[t_] = t /. r;) will work. I think that our puzzlement >> originates from mixing scoping and evaluation. From the fact that Set >> evaluates the r.h.s and uses global values of <t> for entries of <t> on the >> r.h.s. (this is what we are used to from our experiences of working with >> global variables), we subconsciously expect that this is true for all >> entries of <t> on the r.h.s, while this is only guaranteed for non-lexical >> entries of <t> - those that were not explicitly (lexically), but rather >> dynamically, present in the code before evaluation started. The problem is >> made worse by the fact that in many common cases it does work (like the >> case of global variables and no surrounding scoping constructs). >> >> The bottom line is that IMO the construct you looked at is not reliable >> from the start in that it attempts to do scope engineering and the results >> are then necessarily dependent on the details of Mathematica internal >> implementation of lexical scoping constructs. Then it is probably a matter >> of taste whether or not to use constructs like this. Personally, I do it >> from time to time for my own purposes, but I would hesitate to put them into >> "production" code. Besides, I believe that in most cases there exist more >> reliable alternatives. >> >> Regards, >> Leonid >> >> >> >> On Thu, May 6, 2010 at 12:52 PM, Patrick Scheibe < >> pscheibe(a)trm.uni-leipzig.de> wrote: >> Hi, >> >> I cannot explain the following behaviour of Module. Maybe someone else >> can: >> >> in the global scope >> >> f[t_] = t; >> DownValues[f] >> >> results in {HoldPattern[f[t_]] :> t} as expected. Simply wrapping it >> with Module, without >> making f lokal does not change this behaviour. >> >> Module[{}, >> f[t_] = t; >> DownValues[f] >> ] >> >> results still in {HoldPattern[f[t_]] :> t}. >> Say I want to replace the right side of the definition with a rule >> >> r = {t :> 1}; >> f[t_] = t /. r; >> DownValues[f] >> >> and we get {HoldPattern[f[t_]] :> 1} >> >> Module[{}, >> r = {t :> 1}; >> f[t_] = t /. r; >> DownValues[f] >> ] >> >> still the expected result {HoldPattern[f[t_]] :> 1} >> >> Now (sorry for the lengthy introduction) making r lokal >> >> Module[{r}, >> r = {t :> 1}; >> f[t_] = t /. r; >> DownValues[f] >> ] >> >> and we get {HoldPattern[f[t$_]] :> t$}. The Pattern t_ was renamed >> before the replacement could act. >> Using the local variable r to set a global r2 and it's working again. >> >> Module[{r}, >> r = {t :> 1}; >> r2 = r; >> f[t_] = t /. r2; >> DownValues[f] >> ] >> >> So there must be at least one thing about Module I did'nt understand. >> I thought the main difference between Block >> and Module is the lexical scoping!? But the only thing which was >> renamed is the lokal usage of r. >> Any ideas? >> >> Cheers >> Patrick >> >> 7.0 for Mac OS X x86 (64-bit) (February 19, 2009) >> >> >> >
|
Pages: 1 Prev: DistributeDefinitions statement appears to create memory leak Next: NDSolve backward in time |