Prev: JCL Builder or script HELP
Next: Need sanity check on return from EZASOKET call
From: Kellie Fitton on 16 May 2005 11:47
Hello EveryOne,

I am developing a softWare application for a multiUser operation
in a netWork environment, the software should facilitate dataEntry
functions, and provide capabilities for an information system
as well. Also, the program has to handle concurrent Read-Write
intensive requests, withOut any bottlenecks or performance blips.

However, I realize that shared dataFiles are indispensable in such
an environment and with such an application, and consideration
must be put forward to correctly create and design sharedFiles,
vis-a-vis fileHandling mechanism.

So, I got the source code cut out with some moot questions that
I would like to share with you, hopefully you can shed some lights
on the layOut, limits and pitfalls, when developing multiTasking
and multiUser softWare programs.

1). Should shared dataFiles use records locking schemes?

2). Should shared dataFiles use data compression mechanism?

3). Should shared dataFiles (largeSize) be split into several
smaller files?

4). Should shared dataFiles be handled by an I/O dynamically
called module per file, that service all processes running
concurrently?

5). Should shared dataFiles be handled by an I/O independent
module per file, that runs separately and communicates with
the application thru shared memory, and service all processes
running concurrently?


Regards, Kellie.

From: Richard on 16 May 2005 15:36
> 1). Should shared dataFiles use records locking schemes?

Absolutely. Though different techniques can be used for different
applications and different business needs.

For example in an oin-line order entry system it needs to be decided
how the 'race' for stock should be handled. If there is only a small
number of products then you cannot afford to lock the stock records
over a data-entry interaction because there will be too many clashes.
If there may be stock shortages over many different products, however,
then the stock record can be locked while the available stock is held
awaiting quantity to be entered.

You should also consider using a locking hierarchy. For example it may
be that to work on debtor's transactions every program that does so
must first lock the debtors' master record. If all correctly follow
the rule then a requirement for multiple locks (kept locks) can be
avoided.

Also ensure that you do specific UNLOCKs when finished. Typically you
may read through a section of records, such as all order lines for an
order. It is necessary to read until the order number changes, there
is a risk that the program will leave a lock on the first record of the
next order.

> 2). Should shared dataFiles use data compression mechanism?

Compression probably will use less disk space (it need not) and this
means fewer disk transfers to access a set of data which may make
processing faster.

> 3). Should shared dataFiles (largeSize) be split into several
> smaller files?

Depends on the value of 'large'. If I thought that in a few years the
data file may get to be a Gigabyte then I may consider using some
sectioning of data.

> 4). Should shared dataFiles be handled by an I/O dynamically
> called module per file, that service all processes running
> concurrently?

That depends on the architecture of the system. Are you, for example,
going to get just one 'dynamically called module' or will you actually
get an iteration of the module for each run-unit (with or without
shared code) ?

Generally you won't get a _CALLed_ routine to 'service all processes'.


Perhaps you are asking if you should have a separate 'file server'
run-unit that interacts with the programs using IPC in some way, as in
Q5.

That will depend on how you need to balance your network traffic. For
example in a networked system with the programs running on client
machines if a search is to be done by examining each record then the
_whole_ file must be sent across the network one record at a time
(plus, probably, thousands of index blocks).

If a server program runs on the file server and this does searches then
only the 'found' records are sent to the client program.

However, if the architecture of the system is centralised where all the
programs run on an 'application server' accessed via terminal systems
(eg X or putty to a Linux application or RCA clients to a TSE server)
then all disk traffic is local within the box and only screen images go
accross the network.

> 5). Should shared dataFiles be handled by an I/O independent
> module per file, that runs separately and communicates with
> the application thru shared memory, and service all processes
> running concurrently?

'Shared memory' implies that all the run-units are in one machine. In a
client-server situation you need to use IPC that is _not_ shared
memory.

From: Robert Jones on 16 May 2005 16:23
Hello Kelly

While I agree with Richard's good advice, I would suggest that you
consider using a relational database with SQL instead of using files
for such an application. The sharing and locking processes are
arguably more comprehensive and less prone to programmer error such as
forgetting to lock, share, unlock when required. They also tend to
come with a fairly good set of management, backup and recovery
features. They also tend to scale up well, especially when the data is
eventually accessed from a variety of programs written in a variety of
languages on a variety of platforms.

I am not saying that a file system is not appropriate in your
circumstances, but that you should at least consider the database
alternative and weigh the pros and cons. For example, you may not know
how to use a database, or you have to work with pre-existing files, or
a file system might be faster, etc..

regards, Robert

From: Kellie Fitton on 16 May 2005 16:48
Hi Richard,

> Absolutely. Though different techniques can be used for different
> applications and different business needs.

I would like to use a manual lock with kept locks for muliple record
locking,
however, what would happen if some records are locked, and the
workStation
experience a powerOutage of some sort? I know the operating system will
release
the records automatically, but I donn't know after how long, though?

> Compression probably will use less disk space (it need not) and this
> means fewer disk transfers to access a set of data which may make
> processing faster.

doesn't compression de-compression of the file records causes some
performance
problems or read/write operation delay?

> That depends on the architecture of the system. Are you, for
example,
> going to get just one 'dynamically called module' or will you
actually
> get an iteration of the module for each run-unit (with or without
> shared code) ?

I am trying to choose between a dynamically called or an independent
I/O module.
Dynamically speaking though, it will be an iteration of the same module
for each
runUnit within the same application. I want to run the interFace
program on a client
machine, while the application runs on the file server, so the shared
dataFiles can
be used efficiently on the netWork. Also, am looking for a way to
reduce the
network traffic to exactly one roundTrip per I/O operation, if I can do
that with my
cobol code. Do you consider that approach wise?

> 'Shared memory' implies that all the run-units are in one machine. In
a
> client-server situation you need to use IPC that is _not_ shared
> memory.

All the runUnits are in one machine for each end-user --- provided
though, the main
application will run on the file server, where the main file searching
will take place
accordingly. Doesn't that approach reduces the roundTrip delay in the
network
connections and in the dataRate transfer as well??

Regards, Kellie.

From: Kellie Fitton on 16 May 2005 17:39
Hello Robert,

You can lead a horse to the water, however, if you can make him float,
you got
something pretty good. :---))

you are raising an exellent point though --- however, I think that a
skillful
programmer with clairvoyant thinking, and a well thought program design
approach,
can produce or at least emulate a large database management system
functionality. Not to mention also, COBOL does an exellent job of
BackUp,
Restore and archive any type of files.

Personally, I think the only weakLink in my indexed file design, is the
actual
indexed files per se, indexed files tend to bog down when they get very
large in
size, however, COBOL systems can have a workAround that issue pretty
easy
as Richard pointedOut above.

Regards, Kellie.

 |  Next  |  Last
Pages: 1 2 3 4 5 6
Prev: JCL Builder or script HELP
Next: Need sanity check on return from EZASOKET call