Cobol's File System Vs. RDBMs...
in [Cobol]
|
Prev: COBOL: Don`t Call It a Comeback
Next: All X'0D' lost during reading line sequential file using microfocus se
From: Howard Brazee on 25 Jul 2008 10:25 On Fri, 25 Jul 2008 07:56:43 -0500, Robert <no(a)e.mail> wrote: >Imagine test driving a new car the way we test new software. > >'Drive around the block? That's a Unit Test. We need to take it on a long trip, say 500 >miles, with the whole family aboard. We need to test it in summer weather and in snow. We >need to puncture a tire to make sure the spare works.' Some of the car testing was done by the car manufacturer with crash test dummies, tire blowouts, and extreme conditions. The users didn't do all of those tests, but that doesn't mean they weren't done. And when I bought a new spread sheet program, I assumed that the testing had already been done - I only was interested in how it fit my needs, and my testing was limited to what I was interested in. So I guess I'm missing your point here. Do you mean we should spend as much testing a new program design as GM spends testing a new car design?
From: Clark F Morris on 25 Jul 2008 13:30 On Fri, 25 Jul 2008 13:59:34 +1200, "Pete Dashwood" <dashwood(a)removethis.enternet.co.nz> wrote: > > >"softWare design" <sabraham(a)baxglobal.com> wrote in message >news:c06daa0b-42dc-4c04-a88a-9ddd2a900e50(a)p31g2000prf.googlegroups.com... >On Jul 23, 6:28 pm, "Pete Dashwood" ><dashw...(a)removethis.enternet.co.nz> wrote: >> >> I agree with you that the COBOL file system is "closed", inasmuch as you >> need to write a COBOL (or other) program which knows the structure of it, >> in >> order to access it. Certainly, some systems have ODBC drivers that permit >> access but the basic problem is that, unlike a DBMS, the structure of the >> data is not held in the file... (there is no metadata). >> >> This means that unless you can write a program or get one written, you >> can't >> access it. (Compare this with an RDBMS, for example, where anyone who >> knows >> SQL and has the right permissions can access the data.) >> > > > >I understand that RDBMs offers greater visibility to the file >contents, but how about performance considerations? > >[Pete] > >My experience has been that an RDB will (almost) never perform as fast as a >VSAM or ISAM file (except for the most very trivial kinds of access). >However, you get many other benefits from using it, and the difference is >not so great that it becomes problematic. > >For example, I ran something against an ISAM file yesterday and noted that >it returned around 130 records per second for sequential access. When I ran >the same logic against an RDB it returned 80 records per second. (FETCHing >on a cursor through ODBC...) > >BUT, I can tweak the RDB without having any effect on the program logic and >when I did this, it returned 126 records a second. This is all using ESQL >(which I consider to be pretty obsolete...) I believe the same code written >in LINQ (as a Lambda function with deferred execution) on a dual core >processor, will return over 200 records a second, when accessing the RDBMS, >but I can't prove it and I haven't tried it. Probably using ADO would give >better performance also. (It does not hold a connection while the cursor is >running, so overall throughput is better...) > >The point is that there will be times when the RDBMS performs way below what >you would expect from ISAM or VSAM. But these times are rare, and you can >fiddle with, test, tune, and tweak, the RDBMS in ways you never could with >flat files. Most of the time the differences in performance are negligible. >Only in cases where there is very large sequential processing involving SQL >cursors would you need to intervene. For random access, a human can't tell >the difference. As someone who did a lot of tuning of VSAM for both sequential and random access, I guarantee you that a lot of tuning can be done external to the application. Batch LSR, Control interval (block) and control area (grouping of control intervals) sizes, and buffer allocations all were tools used. File allocations in CICS also were the subject of much tuning effort. The buffering of sequential reads and writes also can affect performance as can block sizes. I be live that there are similar tuning opportunities in the other environments. > >[/Pete] > >Can a relational database file system slows down the Cobol >application? > >[Pete] > >Yes. But so can linked ISAM files... > >[/Pete] > > > >Are there any performance bottlenecks when Cobol application >handles large volume of data? > >Will Cobol application take much longer to run if I/O operations >access the RDBMs file system as if they were ISAM files? > >[Pete] > >These are really the same question rephrased. The answer is: It depends... > >There COULD be a problem but there MAY not be. > >The important thing is that with the RDBMS you can tune and tweak in a >separate data access layer without having any impact on your program logic. >I separate all SQL out from my applications into a separate data access >layer which is comprised of Object based components. The applications just >instantiate a database access layer (DAL) object and invoke its methods. >Everything flows through an interface. I can change the RDBMS without any >impact at all on the applications, I could move some of the DAL objects to >LINQ and they would still be accessible from COBOL, without change to >existing code. I can write new objects into the Data Access Layer in either >COBOL or C# and it has no impact on the existing COBOL logic. It is a >separation layer. > >In this day and age the multiple benefits of using RDBMS far outweigh any >minor performance differences. > >I am spending a lot of my time lately helping people move from ISAM to >RDBMS. > >Pete.
From: Clark F Morris on 25 Jul 2008 14:00 On Fri, 25 Jul 2008 14:08:13 +1200, "Pete Dashwood" <dashwood(a)removethis.enternet.co.nz> wrote: > > >"Howard Brazee" <howard(a)brazee.net> wrote in message >news:taoh84dumtgpk7btco9a5o2d6p41rff6rn(a)4ax.com... >> On Thu, 24 Jul 2008 12:37:33 -0700 (PDT), softWare design >> <sabraham(a)baxglobal.com> wrote: >> >>>On Jul 24, 11:17 am, Howard Brazee <how...(a)brazee.net> wrote: >>>> >>>> This leads me to ask: >>>> 1. What do you man by columns in an index file? >>>> 2. What is the relationship between being open or closed and being >>>> able to add columns? >>>> >>> >>> >>>Suppose I want to change my data model by adding a column >>>to a table (field to a file record). With a relational database >>>I add the column and the job is done. Any program that needs >>>to work with that new column can on-the-fly, and any existing >>>programs that do not care about the change stays untouched. >>> >>>The Cobol index file system is considered to be Closed simply >>>because it requires a knowledge of the file definition/structure, >>>in order to access the stored data. An open file system should >>>allow privileged users to access the data and generate queries >>>on-the-fly in the desired format they need, and facilitates any >>>column insertions without the need to write special programs. >>> >> >> You have a unique view of what "open" and "closed" mean in our >> industry. > >No he doesn't... I share it. :-) I don't. If there is a good repository function where the data is stored that describes all files, data base tables, views, fields and data base columns, then the method of storage is irrelevant. Data base tables can be set up with wonderfully obscure or ambiguous table and column names either due to lack of discipline or historic limitations that no longer apply. While it is easier to control most kinds of access using a DBMS (network, hierarchical or relational) a good query mechanism probably can handle files as well. While you may be comfortable with all privileged users being able to do things on the fly, I have memories of the parsing of the word sales and whether or not a given transaction was a sale or reduction to sales. Part of the problem is that we do not set up systems that allow users to test their assumptions nor are many properly set up for good audit train on corrective action. People are not trained on the nuances of the systems that exist (I had one case where I was asked to write a special run to delete a data base row and by investigating how the system worked, I was able to tell the requester how she could do it herself). Newer systems have tools for help creation so it isn't as hit or miss as it was with the old CICS systems, especially with packages. I totally support giving people the tools to do their jobs. I firmly believe that packages should provide the capabilities for ad hoc queries but there must be adequate support within the organization for people to truly understand what they are asking. Is the customer the individual store within the chain, the division within the chain, the chain, or the conglomerate that owns the chain? > >Pete.
From: Pete Dashwood on 25 Jul 2008 20:24 "Howard Brazee" <howard(a)brazee.net> wrote in message news:a6lj84dmbqtfgi4653ie4vuaqc603mcbpa(a)4ax.com... > On Sat, 26 Jul 2008 01:02:19 +1200, "Pete Dashwood" > <dashwood(a)removethis.enternet.co.nz> wrote: > >>While it may seem that way to you, that isn't what I intended. My >>intention >>was that the owners of the data would have the right to manipulate it. >>Rather than "them as touched it last, owns it" it was more along the lines >>of "them as owns it can touch it any time they like." > > Ownership of data isn't a two-valued thing. Some data are owned to > various degrees by accounting, sales, finance, management, the > customers, and the state. We can't give all of them full rights to > manipulate them nor even to read them. > > Instead, we need to have established rules that are more complex than > assigning a particular owner for a specific datum. Why? In today's world data can be easily shared by people who are authorised to access it. My post has never suggested that only ONE "owner" should be authorised. I am trying to make the point here that data is NOT owned exclusively by IT, that's all. Pete. -- "I used to write COBOL...now I can do anything."
From: Pete Dashwood on 25 Jul 2008 20:41
"Howard Brazee" <howard(a)brazee.net> wrote in message news:0lkj84pn1grm75p6ufq1otp5cq66d84hr5(a)4ax.com... > On Fri, 25 Jul 2008 13:26:13 +1200, "Pete Dashwood" > <dashwood(a)removethis.enternet.co.nz> wrote: > >>I understand there is an element of seriousness in what you're saying, but >>if you go down that route it ends in tears. >> >>IT does NOT own the data and IT should not have exclusive access to it. >> >>People who OWN the data should be able to manipulate it if they want to. > > Sometimes there are directives from on high limiting ownership > privileges. These may be legal issues, or policies within a > company. Maybe sales "owns" the sales data - but the salesmen > can't manipulate them however they want. > > Obviously, neither should IT. But IT needs to enforce the guidelines > that the enterprise and society demand. OK, so you see IT as being the "Data Police"? I don't. It would appear that both you and Doc have pushed my argument beyond where I wanted it to go. What about companies that don't have an IT department? (They outsource their Network support and run packages and spreadsheets...) What about places where the "IT department" is about Network support and Helpdesk? (There are more and more of these arising...) In the last century when all data processing was the province of the IT department, who administered the central mainframe repository, the idea arose that IT were the professionals and the ONLY ones who should be allowed to deal with data. The only point I'm trying to make here is that in today's world data can be easily shared and access to it controlled by configurations. It doesn't need the data police or IT as "Big Brother" telling people what they can do with their data. I never suggested (in fact quite the contrary) that there would be no audit trails or checks and balances, or implementation of corporate policy regarding information. All I'm saying is you don't need "IT" to do this, and, in fact, it is far better when the business are responsible for their own data. That's it, no more from me on this :-) Pete. -- "I used to write COBOL...now I can do anything." |