Cobol's File System Vs. RDBMs...
in [Cobol]
|
Prev: COBOL: Don`t Call It a Comeback
Next: All X'0D' lost during reading line sequential file using microfocus se
From: Robert on 24 Jul 2008 21:18 On Thu, 24 Jul 2008 12:05:37 -0700 (PDT), Richard <riplin(a)azonic.co.nz> wrote: >Adding a new column in a database does not get it populated with >appropriate data. Existing programs that don't reference the column >will insert it as Null, or will fail if they haven't constructed the >INSERT to cater for unreferenced columns. Anyone who writes an INSERT without naming the columns DESERVES whatever bad happens. The same goes for SELECT *. We have programs that read the database dictionary, write select lists and copybooks to be included/copied at compile time. They also write an initialization structure containing default values.
From: Pete Dashwood on 24 Jul 2008 21:26 "HeyBub" <heybub(a)NOSPAMgmail.com> wrote in message news:8LednfeyoZGWLBXVnZ2dnUVZ_vninZ2d(a)earthlink.com... > Pete Dashwood wrote: >> >> (Don't take HeyBub's post seriously... he is winding you up... :-)) >> > > You think? Yep :-) I understand there is an element of seriousness in what you're saying, but if you go down that route it ends in tears. IT does NOT own the data and IT should not have exclusive access to it. People who OWN the data should be able to manipulate it if they want to. If they screw it up it is on their own heads. There is an argument for ensuring they are properly trained in the use of things like Easytrieve, spreadsheets etc. before turning them loose, IF they are going to be manipulating data. In all the examples you give below, people screwed up because they didn't understand what they were doing, were not properly controlled and there were no proper procedures in place. I wouldn't allow trained people who DO know what they're doing to run an Easytrieve that could make updates against live data... and why wasn't it possible to recover from backup? I can't see a problem with users manipulating data as long as it is a copy and it is checked before being applied to live. It is THEIR information...:-) Permissions and controls ensure that authorised people get access to data. Audit trails enable us to see what they did. How much more accountability do you need? The problem with the "exclusively for IT" approach is that it creates an "Us and Them" divide and it perpetuates the aura of mysticism and elitism around the IT department. In this day and age when kids at school are using spread sheets and databases there is no reason why people can't be responsible for their own data and simply provide IT with what is necessary to mantain the corporate data backbone. Open access to data via spreadsheets and the network is a major competitive edge for enlightened companies. Keeping stuff closed up so that only a designated elite can use it, is just plain wrong. > > > What's the sense of having data validation in an edit program if some > weasel can come in downstream and diddle with the data? What's the difference between an IT weasel and a non IT one? :-) If you alllow data diddling you deserve whatever you get. > > I had to write a report writer program. By actual count, 2700 of the 3000 > statements were to check (and correct) the data on a master file! I should > have known; when I asked in what order the data were in, I was told: "It > depends on what we did with the data before you get it." > > "You, uh, SORT the master file?" I asked. > > "Sure," said the manager. Is that so terrible? :-) It could make sense. We store master data on RDBs in any old sequence and order it at the time it is retrieved... Pete. -- "I used to write COBOL...now I can do anything." > >
From: Pete Dashwood on 24 Jul 2008 21:59 "softWare design" <sabraham(a)baxglobal.com> wrote in message news:c06daa0b-42dc-4c04-a88a-9ddd2a900e50(a)p31g2000prf.googlegroups.com... On Jul 23, 6:28 pm, "Pete Dashwood" <dashw...(a)removethis.enternet.co.nz> wrote: > > I agree with you that the COBOL file system is "closed", inasmuch as you > need to write a COBOL (or other) program which knows the structure of it, > in > order to access it. Certainly, some systems have ODBC drivers that permit > access but the basic problem is that, unlike a DBMS, the structure of the > data is not held in the file... (there is no metadata). > > This means that unless you can write a program or get one written, you > can't > access it. (Compare this with an RDBMS, for example, where anyone who > knows > SQL and has the right permissions can access the data.) > I understand that RDBMs offers greater visibility to the file contents, but how about performance considerations? [Pete] My experience has been that an RDB will (almost) never perform as fast as a VSAM or ISAM file (except for the most very trivial kinds of access). However, you get many other benefits from using it, and the difference is not so great that it becomes problematic. For example, I ran something against an ISAM file yesterday and noted that it returned around 130 records per second for sequential access. When I ran the same logic against an RDB it returned 80 records per second. (FETCHing on a cursor through ODBC...) BUT, I can tweak the RDB without having any effect on the program logic and when I did this, it returned 126 records a second. This is all using ESQL (which I consider to be pretty obsolete...) I believe the same code written in LINQ (as a Lambda function with deferred execution) on a dual core processor, will return over 200 records a second, when accessing the RDBMS, but I can't prove it and I haven't tried it. Probably using ADO would give better performance also. (It does not hold a connection while the cursor is running, so overall throughput is better...) The point is that there will be times when the RDBMS performs way below what you would expect from ISAM or VSAM. But these times are rare, and you can fiddle with, test, tune, and tweak, the RDBMS in ways you never could with flat files. Most of the time the differences in performance are negligible. Only in cases where there is very large sequential processing involving SQL cursors would you need to intervene. For random access, a human can't tell the difference. [/Pete] Can a relational database file system slows down the Cobol application? [Pete] Yes. But so can linked ISAM files... [/Pete] Are there any performance bottlenecks when Cobol application handles large volume of data? Will Cobol application take much longer to run if I/O operations access the RDBMs file system as if they were ISAM files? [Pete] These are really the same question rephrased. The answer is: It depends... There COULD be a problem but there MAY not be. The important thing is that with the RDBMS you can tune and tweak in a separate data access layer without having any impact on your program logic. I separate all SQL out from my applications into a separate data access layer which is comprised of Object based components. The applications just instantiate a database access layer (DAL) object and invoke its methods. Everything flows through an interface. I can change the RDBMS without any impact at all on the applications, I could move some of the DAL objects to LINQ and they would still be accessible from COBOL, without change to existing code. I can write new objects into the Data Access Layer in either COBOL or C# and it has no impact on the existing COBOL logic. It is a separation layer. In this day and age the multiple benefits of using RDBMS far outweigh any minor performance differences. I am spending a lot of my time lately helping people move from ISAM to RDBMS. Pete. -- "I used to write COBOL...now I can do anything."
From: Pete Dashwood on 24 Jul 2008 22:03 "Robert" <no(a)e.mail> wrote in message news:nq6i845e22uaisefi4por9iaa84v1gv46l(a)4ax.com... > On Thu, 24 Jul 2008 15:17:26 -0700 (PDT), softWare design > <sabraham(a)baxglobal.com> wrote: > >>On Jul 23, 6:28 pm, "Pete Dashwood" >><dashw...(a)removethis.enternet.co.nz> wrote: >>> >>> I agree with you that the COBOL file system is "closed", inasmuch as you >>> need to write a COBOL (or other) program which knows the structure of >>> it, in >>> order to access it. Certainly, some systems have ODBC drivers that >>> permit >>> access but the basic problem is that, unlike a DBMS, the structure of >>> the >>> data is not held in the file... (there is no metadata). >>> >>> This means that unless you can write a program or get one written, you >>> can't >>> access it. (Compare this with an RDBMS, for example, where anyone who >>> knows >>> SQL and has the right permissions can access the data.) >>> >> >> >> >>I understand that RDBMs offers greater visibility to the file >>contents, but how about performance considerations? >> >>Can a relational database file system slows down the Cobol >>application? > > They can speed it up, because they can use parallel processing on the > server side (even if > the server is the same box). > >>Are there any performance bottlenecks when Cobol application >>handles large volume of data? > > The bottleneck is that Cobol applications are single-threaded. To load > large volumes with > Cobol we partition the database and run a Cobol process for each > partition. I've run as > many as 1,000 parallel processes loading a single table. > >>Will Cobol application take much longer to run if I/O operations >>access the RDBMs file system as if they were ISAM files? > > An RDBMS running set operations can be DRAMATICALLY faster than a Cobol or > other > procedural language running individual row operations. I recently rewrote > a Cobol-like > PL/SQL program into non-procedural straight SQL. Run time went from eight > days running 80 > parallel processes to a half hour running one (client) process. > > Non-trivial database operations don't run on a single table, they join > tables. Joining > with SQL is MUCH faster than reading with cursors and joining with > procedural logic. > > If you MUST handle invidual rows in Cobol, for instance loading a table, > you want a SQL > command to operate on a large array of rows, say 1,000. Issuing an INSERT > for each row > kills performance. Doing so complicates error handling. If one row fails > to insert, say > due to a foreign key constraint, the whole array fails and the server > doesn't tell you > which row(s) are in error. In that case, you must insert them > individually. You can, and > should, also fetch arrays of rows. If you think you need a cursor, you're > still thinking > in Cobol. Good post, Robert. I've been looking at set operations as replacement for cursor based ones but I simply haven't had time to really get my head round it yet. How would you insert multiple records, then? Pete. -- "I used to write COBOL...now I can do anything."
From: Pete Dashwood on 24 Jul 2008 22:08
"Howard Brazee" <howard(a)brazee.net> wrote in message news:taoh84dumtgpk7btco9a5o2d6p41rff6rn(a)4ax.com... > On Thu, 24 Jul 2008 12:37:33 -0700 (PDT), softWare design > <sabraham(a)baxglobal.com> wrote: > >>On Jul 24, 11:17 am, Howard Brazee <how...(a)brazee.net> wrote: >>> >>> This leads me to ask: >>> 1. What do you man by columns in an index file? >>> 2. What is the relationship between being open or closed and being >>> able to add columns? >>> >> >> >>Suppose I want to change my data model by adding a column >>to a table (field to a file record). With a relational database >>I add the column and the job is done. Any program that needs >>to work with that new column can on-the-fly, and any existing >>programs that do not care about the change stays untouched. >> >>The Cobol index file system is considered to be Closed simply >>because it requires a knowledge of the file definition/structure, >>in order to access the stored data. An open file system should >>allow privileged users to access the data and generate queries >>on-the-fly in the desired format they need, and facilitates any >>column insertions without the need to write special programs. >> > > You have a unique view of what "open" and "closed" mean in our > industry. No he doesn't... I share it. :-) Pete. -- "I used to write COBOL...now I can do anything." |