More /.{I->-1} craziness
in [Mathematica]
|
Prev: Persistent assumption
Next: Financial Data - Currencies
From: Vince Virgilio on 3 Jan 2010 03:37 On Jan 2, 5:06 am, Leonid Shifrin <lsh...(a)gmail.com> wrote: > Regarding this issue, I think I entirely agree with what David Bailey and > other people said: I don't consider replacement rules as a mathematical tool > for end users, but rather as an inner layer of Mathematica, which is also > exposed for flexibility / convenience and intended primarily to be used by > the more advanced users. [ . . . ] SNIP Leonid, Replacement rules are as mathematical as 'Set' rules; to classify them otherwise would mislead. The simple difference between the two types of rules is that Replace is manual while Set is automatic. Both provide the math concept of 'function', and both can apply to structures that are non-mathematical. User discretion chooses between them. 'More advanced users' likely will find more uses for manual rule sets. On the other hand, they appear in Roman Maeder's introductory books. (Was it "Introduction to Programming in Mathematica"?) Vince Virgilio
From: DrMajorBob on 3 Jan 2010 03:37 I think the way we enter and understand InputForm is syntax; FullForm is the way they're stored internally. So Mathematica's pattern matching isn't what I'd call syntactical... unless you mean "syntactical on a hidden (though discoverable) level". Bobby On Sat, 02 Jan 2010 04:06:32 -0600, Leonid Shifrin <lshifr(a)gmail.com> wrote: > Regarding this issue, I think I entirely agree with what David Bailey and > other people said: I don't consider replacement rules as a mathematical > tool > for end users, but rather as an inner layer of Mathematica, which is also > exposed for flexibility / convenience and intended primarily to be used > by > the more advanced users. In this way, they can implement some missing > functionality themselves at their own risk without the need to wait for a > new Mathematica release. It is stated in the documentation that rule > substitution is purely syntax-based, and therefore not guaranteed to > always > make sense. > > I don't see how this by itself makes the design inconsistent: either you > are > the end-user without advanced Mathematica skills and then you have to > stick > to the built-in commands like Conjugate designed specifically to deal > with > the problem (complex conjugation here), or you use the lower-level tools > like replacement rules but then you are on your own - the system will > blindly do the replacements according to the syntax of your rules, and > it is > then your responsibility to use them correctly. What I would agree with > is > that the documentation could have made this borderline more clear-cut. > But I > don't think that this is a problem on the level of design. > > Regards, > Leonid > > > > On Fri, Jan 1, 2010 at 2:39 AM, Richard Fateman > <fateman(a)cs.berkeley.edu>wrote: > >> Leonid Shifrin wrote: >> ... >> >> > >> > I think that there are not many more objects in Mathematica which are >> as >> > tricky as <I> or Infinity in terms of pattern-matching. >> >> >> I agree. >> That's why it can be fixed. >> >> Here's a beginning of a short list for the "we're not just talking >> syntactic replacement-- version of substitution": >> >> If the user says -i --> i, then do Complex[a_,-b_] -> Complex[a,b]. >> If the user says x^2 --> y, then do x^(-2)-> 1/y also. >> >> I assume this list can be enlarged somewhat, and could even be left >> open-ended by user option of some sort. [e.g. should x^2--> y also >> change x^3 to x*y? or to y^(3/2) or ....] >> >> .... >> >> > It would perhaps be nice if such >> > cases were more systematically documented, but they have nothing to do >> with >> > bugs, >> >> You are right if you mean "bug in Mathematica implementation of >> intended design" (this is not such a bug). >> >> But there is another concept: >> "bug in Mathematica design, contrary to reasonable mathematical >> expectation" >> (this IS such a bug). >> >> > -- DrMajorBob(a)yahoo.com
From: Leonid Shifrin on 3 Jan 2010 03:38 Hi Richard, Below I describe rather extensively my view on the issues you raised. Just to make myself clear, it is not my intention to get involved in an endless debate on these topics. I try to adhere to DRY (don't repeat yourself) principle whenever I feel appropriate, so I detail my view on these subjects below with the intention to do it only once. But I will certainly appreciate your feedback. On Sat, Jan 2, 2010 at 9:06 AM, R Fateman <fateman(a)cs.berkeley.edu> wrote: > Leonid Shifrin wrote: > >> Regarding this issue, I think I entirely agree with what David Bailey and >> other people said: I don't consider replacement rules as a mathematical tool >> for end users, but rather as an inner layer of Mathematica, which is also >> exposed for flexibility / convenience and intended primarily to be used by >> the more advanced users. >> > > Unfortunately many users or potential users are not as sophisticated in > their understanding of the distinction between the underlying mechanisms of > a syntax-driven > transformation system. They simply take the marketing blurbs about "A > system for doing mathematics" as a description suggesting that --hey, I do > mathematics too. They don't really know what "syntax" means and they don't > think they need to know, because syntax is not part of their mathematics > education. > Well, if these people don't understand the importance of syntax for doing any formal sicence, regardless of whether it is done by a human or a computer, and somehow believe that some software is able to completely automate this problem away without any further efforts on their side - too bad for them and their current and future employers. Every tool used blindly will eventually produce nonsense. Mathematica is a research tool. I view it as a tool for explorations, tests, verifications and sometimes discoveries, but not a substitute for domain knowledge, intuition, right questions to ask and anticipation for possible correct answers. I have a fair amount of research experience in Physics (mathematical and theoretical), and while I was frustrated with Mathematica at times, it has been overall incredibly helpful for the problems I have been working on. And the reason that I was sure about the correctness of my results was that I was doing many non-trivial checks such as alternative derivations, numerics vs. analytical results, limiting cases, asymptotics, etc - this in the first place, and my proficiency in Mathematica in the second. Also, I probably have more diverse programming background than most professional mathematicians and physicists - this is what I did (along with math) as a kid before I started doing Physics, and this is what I do now for a living after having quit Physics (I have some asm, Pascal, Fortran, C++ and a substantial C experience and work currently as an enterprise Java / web developer). So, hopefully I have both perspectives on Mathematica. What I think is that your dissatisfaction with Mathematica is a result of the clash of cultures. From a programmer / computer scientist viewpoint, Mathematica probably has lots of what can look as "undefined behavior" or at least as a violation of the principle of minimal surprise. But research in (pure) science is done differently. Most physicists and mathematicians I know are basic Mathematica users but are generally quite satisfied with Mathematica. They don't care as much about what Mathematica does incorrectly (no decent journal will accept Mathematica or any other CAS-based derivation as a central part of any proof anyway - but this is not to say that they are not annoyed by real bugs), as they care about what they *can* do with Mathematica in principle. They may have some wrong beliefs, like believing that Mathematica can not do lots of things it actually can do, or that it is always dog slow with numerics, but they seldom run into problems of the kind you often mention, simply because they by far don't have your level of sophistication with Mathematica - so they have no way to come up with such problems. And I would argue that this kind of advanced Mathematica skills is more characteristic of people working either in Computer Science or in the more applied fields where some math is necessary, such as engineering or finance, for instance (this is IMO because the research problems in pure science are usually more unique and to a much lesser degree amenable to automation, thus programming skills are not as relevant). I can also speak for myself: most of the time, when I am using some advanced Mathematica (programming) constructs, it is a programmer in me, not a scientist, who is the driving force for it. As far as I can tell, most software systems and programs exist to automate (part of the) work which must otherwise be done by a human. The degree of automation can be very different, but I have a feeling that for software used in industry it is generally much higher (or at least that's the goal) than for that used in research (I don't mean the software that say controlls a particle accelerator etc - this I consider "industrial", even if used for scientific purposes). In particular, many industrial software systems are authorized to make lots of high - level decisions by themselves, with humans often becoming operators who monitor the system's work and intervene only in special circumstances. But for research software, I have a feeling that, while automation is of course important, still most of high-level decisions are left for a human - simply because it is much harder to automate research, due to its very nature and requirement to be original. So I think that it is inappropriate to subject Mathematica to requirements typical for non-research software - hopefully it will never be intended to replace the person who is using it, and will always remain a tool. Arguably, a skilled mathematician or physicist has her own ways of checking the correctness of the result. Mathematica is always a tool, not a magic box that is guaranteed to always produce the right answer (given that lots of times in research there are ambiguities in the questions asked). If one has no means to ensure that the answer is correct, this means that he has at most a single perspective of the problem he is solving. But if so, this means that he does not really understand it, and this has nothing to do with Mathematica. > > Now a person educated as a computer scientist would generally know a fair > amount about syntax, and might be willing to use > "A system that uses syntax-directed transformation rules for computation". > In fact there are several such systems that have been > designed, starting in the early 1960s. In deference to Steve C's > reluctance to allow the names of other computer systems to appear > in mathgroup, I won't name them. But at least 6 come to my mind. > I still don't understand the reluctance of people to say "OK, > mathematician-who-doesn't-know syntax" ... HERE's the substitution facility > for YOU. > and write the program. Or at least a first cut of one, so that it can be > refined. Unfortunately, I don't have comparable experiences with other CAS, so I can't say anything useful here. One thing that I find remarkable about Mathematica is the level of integration of different parts and the fact that it still remains relatively simple system at its core, given the amount of built-in functionality included in the kernel. > > > In this way, they can implement some missing functionality themselves at > their own risk without the need to wait for a new Mathematica release. It is > stated in the documentation that rule substitution is purely syntax-based, > and therefore not guaranteed to always make sense. > > It says that it won't always make sense? Hm. (I am traveling and don't > have Mathematica with me, and can't check...) Doesn't make sense?! > How could that be.. The last part is my interpretation. But the problems of using the correct syntax and its correct interpretation exist in any formal science. In my view, the syntax of Mathematica language is not really isomorphic to the language of any specific domain of Mathematics, and should not be. It is currently a number of very high-level commands targeted to occasional users and performing well-defined standard mathematical operations, such as solving equations or inequalities of some kind, etc. But IMO more importantly in the long term, it is also a language - a building material, optimized for creation of sub-languages for (mathematical) knowledge representation and manipulations with mathematical objects. It is then targeted at advanced users / progammers with both domain knowledge and Mathematica skills who can correctly implement these sub-languages, adding to the functionality available to the first target audience. I would agree that there is currently a gap in between these two target audiences, which would include for example some mathematicians without advanced Mathematica skills who want to use Mathematica for their research, and be able to push it to the limits. But I still don't think of this as a design flaw. At the moment, this may be inconvenient, but evolutionary I think this is a win - the system's generality makes it flexible enough to evolve and smoothly integrate new functionality. The same generality is also responsible for the occasional nonsense produced by a blind application of replacement rules. But I think that in time, the corresponding "intermediate layer" of Mathematica will emerge with enough functionality to allow these people do their work without immersing themselves in complexities of Mathematica's inner workings, and that will close the gap (already now, lots of extra functionality is available through add-on packages). As David Park said recently, we are still the early users. > It must make sense to SOME people. Maybe even me or you. So now there are > more levels. > The high-priest, keeper of the mysterium(us?). The second level priest who > understands syntax but for whom some transformations "don't make sense",a > person who not a true syntax-geek. Perhaps this is the typical programmer > who learns some Mathematica.... > The third level, maybe a skilled mathematician? The fourth level, some > novice, unsophisticated student learning math; and maybe down the ladder > further ? > I stick to my view of replacement rules as being aimed primarily at advanced users, or at least as a tool that should be used with much care. I wish the documentation was more clear about it. The fact that the functional and procedural layers are built on top of the rule-based engine speaks for itself - these layers are more managable by non-experts and less error-prone to use (apart from efficiency gains). When beginners get excited by replacement rules and start using them left and right, this very often leads to trouble (I recall myself some while ago :)). Imagine for example a web framework written say in Lisp, with some high-level API (or DSL) exposed to basic users. Say, the inner workings of the framework are documented for the benefits of the more advanced users. Imagine then that a basic user of the framework learns a few basic things about Lisp and starts to use Lisp to do what the API or DSL is supposed to be used for, trying to combine his own Lisp functions with the calls to the API. I suppose you wouldn't be surprised if our hypothetical user would frequently end up with something different from what he intended. Also, I agree with David Park here. When we learn math at school, we spend several years to learn the subtleties of essentially a very similar activity: when we do math, a lot of "pattern-matching" is happening in our head when we decide which identities or equations to use. I have no reason to think that the syntax of Mathematica *core* language must be designed to be extremely easy to learn, given the level of generality required for it. Besides, the core language is actually not so difficult to learn either. I think that part of the problem is that most Mathematica introductions are too pragmatic in a sense - they want to get you up to speed in solving field - specific problems as quickly as possible and as a result omit a proper discussion of the fundamental language structures (I tried to do it differently in my book). Even worse, field-specific elements are often mixed with parts of Mathematica language, which I find very confusing. Pragmatically and in the short run, this seems to be a right thing to do given that most (potential) Mathematica users at present are not high-school students but professionals with no spare time to properly learn Mathematica (perhaps, this will change). But for would-be long-term Mathematica users, learning Mathematica this way is learning it the hard way. Best regards and happy 2010, Leonid
From: Leonid Shifrin on 3 Jan 2010 03:38 On Sat, Jan 2, 2010 at 2:51 PM, DrMajorBob <btreat1(a)austin.rr.com> wrote: > I think the way we enter and understand InputForm is syntax; FullForm is > the way they're stored internally. > > So Mathematica's pattern matching isn't what I'd call syntactical... unless > you mean "syntactical on a hidden (though discoverable) level". > Yes Bobby, That's exactly what I mean. When I talk about syntax, I mean FullForm, not InputForm. Perhaps a more precise way of saying it would be to say that pattern-matching is based on exact symbolic form (or structure) of the expression (tree), without any "meaning" attached to any of its parts - this is what I mean by syntax-based. Regards, Leonid > Bobby > > > On Sat, 02 Jan 2010 04:06:32 -0600, Leonid Shifrin <lshifr(a)gmail.com> > wrote: > > Regarding this issue, I think I entirely agree with what David Bailey and >> other people said: I don't consider replacement rules as a mathematical >> tool >> for end users, but rather as an inner layer of Mathematica, which is also >> exposed for flexibility / convenience and intended primarily to be used by >> the more advanced users. In this way, they can implement some missing >> functionality themselves at their own risk without the need to wait for a >> new Mathematica release. It is stated in the documentation that rule >> substitution is purely syntax-based, and therefore not guaranteed to >> always >> make sense. >> >> I don't see how this by itself makes the design inconsistent: either you >> are >> the end-user without advanced Mathematica skills and then you have to >> stick >> to the built-in commands like Conjugate designed specifically to deal with >> the problem (complex conjugation here), or you use the lower-level tools >> like replacement rules but then you are on your own - the system will >> blindly do the replacements according to the syntax of your rules, and it >> is >> then your responsibility to use them correctly. What I would agree with is >> that the documentation could have made this borderline more clear-cut. But >> I >> don't think that this is a problem on the level of design. >> >> Regards, >> Leonid >> >> >> >> On Fri, Jan 1, 2010 at 2:39 AM, Richard Fateman <fateman(a)cs.berkeley.edu >> >wrote: >> >> Leonid Shifrin wrote: >>> ... >>> >>> > >>> > I think that there are not many more objects in Mathematica which are >>> as >>> > tricky as <I> or Infinity in terms of pattern-matching. >>> >>> >>> I agree. >>> That's why it can be fixed. >>> >>> Here's a beginning of a short list for the "we're not just talking >>> syntactic replacement-- version of substitution": >>> >>> If the user says -i --> i, then do Complex[a_,-b_] -> Complex[a,b]. >>> If the user says x^2 --> y, then do x^(-2)-> 1/y also. >>> >>> I assume this list can be enlarged somewhat, and could even be left >>> open-ended by user option of some sort. [e.g. should x^2--> y also >>> change x^3 to x*y? or to y^(3/2) or ....] >>> >>> .... >>> >>> > It would perhaps be nice if such >>> > cases were more systematically documented, but they have nothing to do >>> with >>> > bugs, >>> >>> You are right if you mean "bug in Mathematica implementation of >>> intended design" (this is not such a bug). >>> >>> But there is another concept: >>> "bug in Mathematica design, contrary to reasonable mathematical >>> expectation" >>> (this IS such a bug). >>> >>> >>> >> > > -- > DrMajorBob(a)yahoo.com >
From: David Bailey on 3 Jan 2010 03:39
Richard Fateman wrote: > Leonid Shifrin wrote: > ... > >> I think that there are not many more objects in Mathematica which are as >> tricky as <I> or Infinity in terms of pattern-matching. > > > I agree. > That's why it can be fixed. > > Here's a beginning of a short list for the "we're not just talking > syntactic replacement-- version of substitution": > > If the user says -i --> i, then do Complex[a_,-b_] -> Complex[a,b]. > If the user says x^2 --> y, then do x^(-2)-> 1/y also. > > I assume this list can be enlarged somewhat, and could even be left > open-ended by user option of some sort. [e.g. should x^2--> y also > change x^3 to x*y? or to y^(3/2) or ....] > > .... > >> It would perhaps be nice if such >> cases were more systematically documented, but they have nothing to do with >> bugs, > > You are right if you mean "bug in Mathematica implementation of > intended design" (this is not such a bug). > > But there is another concept: > "bug in Mathematica design, contrary to reasonable mathematical expectation" > (this IS such a bug). > Would you really want to use a system which had been arbitrarily hacked by well meaning individuals to create exceptions that suited their taste? Perhaps the documentation should make it clearer which operations are supposed to always perform mathematically valid operations, and which are meant to operate on the structure of expressions, oblivious to their meaning. I think this is an important distinction. If you pass an expression to Integrate (say), you expect to get the integral of that expression back as a result (possibly in symbolic form still involving an integral sign) - anything else is a bug. However, other operations - such as ReplaceAll - are not *defined* mathematically, they are defined structurally, and as I pointed out earlier, can be used to produce explicitly invalid expression transformations. Are you really saying that Mathematica doesn't need structural operations? If it does, why pick on ReplaceAll? David Bailey http://www.dbaileyconsultancy.co.uk |