Prev: About running Linux on Windows (See: old thread: LBW)
Next: Difference between service and process??
From: yirgster on 11 May 2010 17:51
I need to deal with an EFBIG at work.

(1) how do I determine the maximum allowed size of a file

(a) rlimit is not particularly helpful:

RLIMIT_FSIZE: unlimited unlimited

(b) I didn't see a relevant #define under sysconf.

(c) Will this be done as a check in the kernel prior to the write()
or, is a partial write() possible. (posix, see below seems to imply
that it can.)

(d) we have both 32 and 64 bit environments to support.

posix says (write, ftruncate):

------------- posix ------------
EFBIG
An attempt was made to write a file that exceeds the implementation-
defined maximum file size [XSI] [Option Start] or the process' file
size limit, [Option End] and there was no room for any bytes to be
written.
----------- end posix ------------

Well, what is there is room for some bytes to be written?

I don't want to specify the rlimit via setrlimit().

My original idea was to determine the limit and see if any write() or
ftruncate() would exceed it, and cut it off at that point. (In our
particular app we would then create a 2nd file and so on. There are
only a couple of places in the code where EFBIG is a real possibility.
ENOSPC is eventually possible but it's exceedingly unlikely and in any
case considered an "acceptable" error in that we're not responsible
for it.

(I also looked at http://www.opengroup.org/platform/lfs.html (large
file support), but I didn't see anything here.

Also, I will just mention, this same problem needs to be dealt with
under winX.
From: Chris Friesen on 11 May 2010 18:49
On 05/11/2010 03:51 PM, yirgster wrote:
> I need to deal with an EFBIG at work.
>
> (1) how do I determine the maximum allowed size of a file

In general, it depends on the filesystem being used. ext2 for instance
can write a 2TB file when using 4K page size.

> (d) we have both 32 and 64 bit environments to support.

Doesn't matter, both support sizes greater than 4GB using 64-bit offsets.

>
> posix says (write, ftruncate):
>
> ------------- posix ------------
> EFBIG
> An attempt was made to write a file that exceeds the implementation-
> defined maximum file size [XSI] [Option Start] or the process' file
> size limit, [Option End] and there was no room for any bytes to be
> written.
> ----------- end posix ------------
>
> Well, what is there is room for some bytes to be written?

ftruncate() should fail with EFBIG or EINVAL if you specify a file size
that is larger than supported. (By the filesystem or by rlimit.)

You may also find it useful to check fstatvfs() to see how much room is
left in the filesystem.

Chris
From: yirgster on 11 May 2010 22:11
On May 11, 3:49 pm, Chris Friesen <cbf...(a)mail.usask.ca> wrote:
> On 05/11/2010 03:51 PM, yirgster wrote:
>
> > I need to deal with an EFBIG at work.
>
> > (1) how do I determine the maximum allowed size of a file
>
> In general, it depends on the filesystem being used.  ext2 for instance
> can write a 2TB file when using 4K page size.
>
> > (d) we have both 32 and 64 bit environments to support.
>
> Doesn't matter, both support sizes greater than 4GB using 64-bit offsets.
>
>
>
> > posix says (write, ftruncate):
>
> > -------------  posix ------------
> > EFBIG
> > An attempt was made to write a file that exceeds the implementation-
> > defined maximum file size [XSI] [Option   Start]  or the process' file
> > size limit, [Option End]  and there was no room for any bytes to be
> > written.
> > ----------- end posix ------------
>
> > Well, what is there is room for some bytes to be written?
>
> ftruncate() should fail with EFBIG or EINVAL if you specify a file size
> that is larger than supported.  (By the filesystem or by rlimit.)
>
> You may also find it useful to check fstatvfs() to see how much room is
> left in the filesystem.
>
> Chris

Thanks! I had completed glazed past EINVAL!

> check fstatvfs

Yes, I'll keep this in mind for other circumstances. Thanks again. In
this case an ENOSPC is a customer configuration/environment issue
(lack of proper resource). If the program doesn't end they'll never
take action. They never look at warning msgs in the log. (We have
nothing intelligent to do upon ENOSPC although the error is passed
back up to provide a function trace in the log.)
From: Rainer Weikusat on 12 May 2010 07:07
yirgster <yirg.kenya(a)gmail.com> writes:

[...]

> Yes, I'll keep this in mind for other circumstances. Thanks again. In
> this case an ENOSPC is a customer configuration/environment issue
> (lack of proper resource).

While this is IMO not compliant, some people believe that write
et. al. should return zero instead of signalling an error condition
when running out of space.
From: Casper H.S. Dik on 12 May 2010 07:29
Rainer Weikusat <rweikusat(a)mssgmbh.com> writes:

>yirgster <yirg.kenya(a)gmail.com> writes:

>[...]

>> Yes, I'll keep this in mind for other circumstances. Thanks again. In
>> this case an ENOSPC is a customer configuration/environment issue
>> (lack of proper resource).

>While this is IMO not compliant, some people believe that write
>et. al. should return zero instead of signalling an error condition
>when running out of space.

They would be wrong:

If a write() requests that more bytes be written than there is room for
(for example, [XSI] [Option Start] the process' file size limit or
[Option End] the physical end of a medium), only as many bytes as there
is room for shall be written. For example, suppose there is space for
20 bytes more in a file before reaching a limit. A write of 512 bytes
will return 20. The next write of a non-zero number of bytes would give
a failure return.


My reading of the standard says that write() can only return 0 when
it is called with nbytes == 0.

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
 |  Next  |  Last
Pages: 1 2
Prev: About running Linux on Windows (See: old thread: LBW)
Next: Difference between service and process??