Why C for operating systems
in [General Programming]
|
From: Patricia Shanahan on 7 Dec 2009 14:28 BGB / cr88192 wrote: .... > for example, pointer arithmetic, ... is just another part of the job... This, to my mind, is the very worst thing about C. I have seen far too many hard-to-debug operating system bugs due to widespread pointer arithmetic. It would be far better if C pointer arithmetic were constrained to conform to the standard. That way, when a data structure gets clobbered the code doing the clobbering would have to be something related to the data structure. Patricia
From: Chris M. Thomasson on 7 Dec 2009 14:38 "Pascal J. Bourguignon" <pjb(a)informatimago.com> wrote in message news:87iqciepsv.fsf(a)galatea.local... > "Chris M. Thomasson" <no(a)spam.invalid> writes: > >> "Pascal J. Bourguignon" <pjb(a)informatimago.com> wrote in message >> news:87r5r6etl5.fsf(a)galatea.local... >>> "Chris M. Thomasson" <no(a)spam.invalid> writes: >>>> I don't quite understand what you mean. I was writing about actually >>>> implementing the Judy Array from scratch. Can that be done in Lisp? >>> >>> Of course, you can implement any algorithm in Lisp. >> >> >> Can I dynamically allocate an object in Lisp on a specific alignment >> boundary, and get a pointer to it that I can pack with a reference >> count and use single-word atomic operations to mutate it? Say I want >> object on 4096 byte boundary, and I want to steal bits in pointer to >> pack in meta-data. For instance, how would I implement the following >> proxy garbage collection algorithm in Lisp: [...] > > > Definitely. See for example: > http://darcs.informatimago.com/public/lisp/common-lisp/heap.lisp > with: > http://darcs.informatimago.com/public/lisp/clisp/raw-memory.lisp > and: > http://darcs.informatimago.com/public/lisp/clisp/raw-memory-lib.c > > All the pointer and bit fiddling is done in lisp in heap.lisp. Okay. Well, I need to study up on Lisp because the code looks a bit foreign to me. When you get some free time, and don't mind wasting some of it... Could you perhaps show me how to create an analog of the following 32-bit C hack in Lisp: _________________________________________________________________ #include <stddef.h> #include <assert.h> #include <string.h> #include <limits.h> typedef char static_assert [ sizeof(void*) * CHAR_BIT == 32U && sizeof(unsigned short) * CHAR_BIT == 16U && sizeof(size_t) == sizeof(void*) ? 1 : -1 ]; #define ALIGN_UP(mp_ptr, mp_align) \ ((void*)((((size_t)(mp_ptr)) + \ ((mp_align) - 1U)) & ~(((mp_align) - 1U)))) #define CHUNK_SIZE 0xFFFFU #define CHUNK_MASK 0xFFFF0000U #define META_MASK 0x0000FFFFU int main(void) { unsigned short meta; unsigned char* chunk_ptr; unsigned char raw_memory[(CHUNK_SIZE * 2U) - 1U] = { '\0' }; unsigned char* align_ptr = ALIGN_UP(raw_memory, CHUNK_SIZE); unsigned char* mutate_ptr = align_ptr + USHRT_MAX; strcpy((char*)align_ptr, "Hello World!"); chunk_ptr = (unsigned char*)(((size_t)mutate_ptr) & CHUNK_MASK); assert(chunk_ptr == align_ptr); meta = (unsigned short)(((size_t)mutate_ptr) & META_MASK); assert(meta == USHRT_MAX); assert(! strcmp((char*)chunk_ptr, "Hello World!")); return 0; } _________________________________________________________________ > This implementation of a heap with garbage collection allows me to > share memory between different lisp processes (possibly running > different implementations). Sounds pretty good. > In the case of threads, almost all the Common Lisp implementations > provide threads and manage memory for them with a garbage collector. > I don't think the current common Common Lisp implementation have a > multi-threaded garbage collector (working in parallel with the user > threads), but if it was required, this is something that would be done > at the level of the implementation (you could check with the > commercial implementation Scienner Common Lisp which is targetting > high performance and multi-core processors, it might have a parallel > garbage collector). > > > So, to conclude here, bit fiddling is in general left to the > implementation, so that the lisp programmers may concentrate on his > problem solving, but if his problem concerns bit fiddling, there's no > difficulty in doing that in lisp. Most lisp implementations are > written in lisp anyways. Thanks for the information Pascal.
From: Chris M. Thomasson on 7 Dec 2009 14:46 "Patricia Shanahan" <pats(a)acm.org> wrote in message news:gPqdncAsILnoyoDWnZ2dnUVZ_tudnZ2d(a)earthlink.com... > BGB / cr88192 wrote: > ... >> for example, pointer arithmetic, ... is just another part of the job... > > This, to my mind, is the very worst thing about C. FWIW, I need the ability to manipulate pointer values directly, so pointer arithmetic is very important to me. > I have seen far too many hard-to-debug operating system bugs due to > widespread pointer arithmetic. It would be far better if C pointer > arithmetic were constrained to conform to the standard. That way, when a > data structure gets clobbered the code doing the clobbering would have > to be something related to the data structure.
From: BGB / cr88192 on 7 Dec 2009 14:48 "Pascal J. Bourguignon" <pjb(a)informatimago.com> wrote in message news:87fx7nf6hu.fsf(a)galatea.local... > "BGB / cr88192" <cr88192(a)hotmail.com> writes: > >>> That C is not better than high level programming languages for system >>> programming since it needs the same extensions to be effective. >>> >> >> not really, since most of these "extensions" are part of the core >> language >> by default, and so not seen as extensions by those using the language. >> >> for example, pointer arithmetic, ... is just another part of the job... > > This is not a discriminating example. I've been doing pointer > arithmetic with languages such as Pascal or Common Lisp with no > problem. > Pascal has pointers, yes... but, I am not debating about Pascal (FWIW, most Pascals fall into a similar "implementation bracket" as C and C++, in most regards). the main problem with Pascal is that it is unlikely one would be able to gain much of any real community or industry support if using such a language in a project. CL has plenty of its own sets of issues... > >>> There's no reason why the system should be programmed the C way. All >>> the niceties of high level languages are good for system programming >>> too, and shouldn't be disabled. >>> >> >> they would need to be disabled in order to be able to get the language to >> be >> able to bootstrap itself... > > Bootstrapping a language has little in common with bootstrapping a > system or a run-time environment. > in the case of something like the JVM, one would have to essentially do both at the same time... there is a way to reduce the costs, but that would be by creating a restricted subset and a specialized compiler (granted, from JBC to ASM is allowed here). most likely, the main ommision needed for such a bootstrapped mode would be GC, interfaces, and exceptions. at this point, it would likely be fairly easy to compile it cleanly to native code. then the main extension needed would be to add a few built-in (or implemented in ASM) classes for such things as pointer arithmetic, ... which could in turn be used for implementing the garbage collector and other things. FWIW, it would be much less effort to bootstrap such an effort in C (or C++), which is how the existing JVM's I am aware of tend to work. > >> for example, you can't use the GC before the GC is operational, and you >> can't use the GC for the code implementing the GC. this much is >> essentially >> a hard limit. > > Actually, you can use a garbage collected heap before the garbage > collector is active. Allocation on garbage collected heaps is usually > much more simplier than with a non garbage collected heap, so you can > start allocating memory for a good amount of work (think that you've > got 4GB or RAM in low end CPUs nowadays) before starting to need the > garbage collector. > some restrictions are needed. you code for allocating a class can't be a dynamically allocated class... at the least, core allocation machinery would need to be done via static methods, and as above, the code for things like pointer management, ... would also need to be static methods. something like this is "not proper" for usual Java development, which demands creating an instance of damn near everything... nevermind that "java.lang.String" contains strings, which by the JVM spec are implementations of themselves (in my effort, this turned into an ugly mess of hacks, where a built-in "pre-String" class was used, and the classloader getting 'String' loaded essentially replaced the pre-String class). > Two other points: > > - the Common Lisp specification doesn't mandate garbage collection. > Garbage collection or memory management is just not mentionned by > the standard. If you need to have a sys:free function to free > memory managed without a garbage collector, you can have that in a > fully conformant Common Lisp implementation. > > - most implementations who provide a garbage collector, also provide a > way to disable it (temporarily). > granted, the issue is not strictly with CL, only that CL has its own issues which I doubt will ever gain it widespread acceptance... > > >> such a restricted subset is then needed, and the system can switch to a >> more >> full-featured form only after bootstrapping... > > It is trivial to write the code to generate the lower level code > needed to load a run-time environment. > but, code is needed to run in order to be able to bootstrap itself in the first place... unless you are proposing to use an HLL just to write a compiler to trans-compile itself to C or ASM in order to get the thing up-and-running, but it is hard to see the "achievement" of this (if the whole point is to use the HLL for all its HLL glory in something like kernel development...). > > >> this would be analogous to how C usually requires a runtime library to be >> written before one can use the runtime library. granted, it can be noted >> that one can write pretty much the entire runtime library (apart from a >> few >> small pieces) in C. >> >> any language capable of doing similarly also needs to be able to do >> similarly. >> >> >> while this allows an implementation of a language designed for the task, >> it >> does not allow one not designed for this... >> >> >> for example, system programming in Java would be possible, but apart from >> a >> specialized compiler with a lot of extensions, one would need a fairly >> large >> amount of ASM (maybe 10-15 kloc?...) to be able to essentially bootstrap >> the >> language (such as the garbage collector, core language facilities, OO and >> Interface machinery, ...). > > But this amount of "assembler" can be generated easily with a compiler > written in Java for the subset of Java needed to bootstrap the system, > like it is done in Squeak for Smalltalk. > this would seem to be a very different argument from the prior... anyways, I had thought you were claiming such a "subset" would have been unecessary?... here, the subset is clearly necessary: in order to be compiled into the bootstrap code. either way, it is not much of an achievement, since something like C does not need such a "bootstrap" mode, as it can be written to mostly bootstrap itself unmodified. then it once again has a point: it can be used to bootstrap an HLL?... so, the point: wasn't it supposed to be that all this was not necessary?... so, what do we have then: an OS written in a mix of C, ASM, and Java?... AFAIK, this is fairly close to what is used on a lot of cell phones... but, this is no real achievement here, since in this case the C has not had its usefulness "disproved"... > >> why is it then that, in such efforts, it is far more common to bootstrap >> Java via C?... > > The only reason, is that we bootstrap always on POSIX (unix) systems > written in C, and that C is the designated portable assembler for > these systems. But this technical point is of no importance: the Java > system is till written in Java. > the JVM's are typically themselves written in large part in C, only that much of the non-core machinery is Java... so, the JVM is typically some glob of support machinery: the GC, object system, classloader, ... then the whole rest of the island is Java... as for the general "Unix-like architecture", this is not likely to go away any time soon, as it is currently the most practical architecture on modern CPU's (they fit almost hand-in-glove). > >> granted, Java is a poor example of a non-C HLL, but FWIW, it is about the >> only one at present which poses that much of a threat... > > "Threat"? > > >> nearly everything else has become marginal... > > "Unpopular"? > > Are we discussing popularity? I though we were on a technical newsgroup. > programming language life or death is their popularity... ofterwise, they fall off into the land of essential non-existance... popularity, community acceptance, standard practices, existing legacy codebases / heritage, ... these are the essential basis of the existence of a tachnology. everything that is now is based in the foundation built by those who have come before, and the possible futures are based on the solid foundations of past actions and precedents...
From: BGB / cr88192 on 7 Dec 2009 14:57
"Patricia Shanahan" <pats(a)acm.org> wrote in message news:gPqdncAsILnoyoDWnZ2dnUVZ_tudnZ2d(a)earthlink.com... > BGB / cr88192 wrote: > ... >> for example, pointer arithmetic, ... is just another part of the job... > > This, to my mind, is the very worst thing about C. > > I have seen far too many hard-to-debug operating system bugs due to > widespread pointer arithmetic. It would be far better if C pointer > arithmetic were constrained to conform to the standard. That way, when a > data structure gets clobbered the code doing the clobbering would have > to be something related to the data structure. > bit, it is also essential to pull off many of the things C is able to do. otherwise, people would have to resort to assembler to do these things... so, in a way, it is a double-edged sword: with power comes responsibility, ... granted, I am not claiming that people should use it for whatever comes to mind, as I have also seen my share of code making some fairly nasty use of pointer arithmetic, but granted, it is just another part of the job. FWIW, even C# added pointers and the ability to do some C like tasks (granted, one has to go through some amount of "red tape" to do so), and I figure it is unlikely MS would have added anything of the sort unless there were some good reason to do so (it is in addition to the more abstract Java-like object model). > Patricia |