Prev: Solaris 10 installation and RAID config
Next: Anyone using ZFS on a Dell 1850 w/ LSI Megaraid?
From: hume.spamfilter on 9 Sep 2009 11:30
neilsolent <n(a)solenttechnology.co.uk> wrote:
>
>> Can you do an 'ls -lL /dev/md/dsk/d6' ?for me, please?
>
> dsys01a # ls -lL /dev/md/dsk/d6
> brw-r----- 1 root sys 85, 6 Jun 5 2004 /dev/md/dsk/d6

And what does:

lsof | grep '85,6' | grep -v VDIR

.... list?

--
Brandon Hume - hume -> BOFH.Ca, http://WWW.BOFH.Ca/
From: Richard B. Gilbert on 9 Sep 2009 16:48
neilsolent wrote:
> We have a Solaris 8 SPARC Sun-Fire-V440 system (I know, very old). But
> it is very critical to us.
>
> Filesystem /var seems to have become corrupt - "df -k" reports it 100%
> full in terms of diskspace (inodes at 3%) but the size of the files is
> not enough to account for this (5GB) of space usage. Hence deleting
> files won't help.
>
> We had some trouble a while back - deleting an old core dump in /var/
> cores to free space caused the system to crash. After some fsck and
> lower-level work the system was eventually brought back up.
>
> We could reboot it now - fsck would run on /var and *might* help - but
> we are not sure the system will come up!
>
> Questions are: is it possible/worthwhile to recreate /var afresh, and
> how could this be done?
>
> Many thanks in advance

Well, you COULD boot from CD or DVD, mount the /var partition and do
some cleanup.

You could also remove /var and recreate it in a larger partition if you
have one available or could make one available.

5GB would be enough for many systems but it appears that you may need
more. You may also want to look at which processes and/or users are
allocating space in /var and why! This might provide clues as to how
much space you actually need. The solution could be as simple as
putting some idiot out of his misery! It could also be some complex and
difficult problem with seemingly legitimate usage.

Please let us know if you learn anything interesting!
From: neilsolent on 11 Sep 2009 04:26
Thanks for updates everyone.

dsys01a # find /proc/ -type f -links 0
/proc/724/fd/0
/proc/724/fd/1
/proc/724/fd/2
/proc/724/fd/3
/proc/17304/fd/0
/proc/17304/fd/1
/proc/17304/fd/2
/proc/24190/fd/0
/proc/24190/fd/1
/proc/24190/fd/2


dsys01a # lsof | grep '85,6' | grep -v VDIR
picld 60 root txt VREG 85,6 360
366114 /var/ld/ld.config
rpcbind 386 root txt VREG 85,6 360
366114 /var/ld/ld.config
inetd 409 root txt VREG 85,6 360
366114 /var/ld/ld.config
syslogd 423 root txt VREG 85,6 360
366114 /var/ld/ld.config
syslogd 423 root 3w VREG 85,6 25326
5661 /var (/dev/md/dsk/d6)
cron 424 root txt VREG 85,6 360
366114 /var/ld/ld.config
cron 424 root 1w VREG 85,6 8338349
32 /var/cron/log
cron 424 root 2w VREG 85,6 8338349
32 /var/cron/log
utmpd 449 root txt VREG 85,6 360
366114 /var/ld/ld.config
sendmail 475 root txt VREG 85,6 360
366114 /var/ld/ld.config
efdaemon 478 root txt VREG 85,6 528
383007 /var/ld/sparcv9/ld.config
sshd 485 root txt VREG 85,6 360
366114 /var/ld/ld.config
BESClient 489 root txt VREG 85,6 360
366114 /var/ld/ld.config
BESClient 489 root 3uW VREG 85,6 0
399915 /var/opt/BESClient/.BESClientRunning.write
BESClient 489 root 4u VREG 85,6 1021
230983 /var/opt/BESClient/__BESData/__Global/Logs/20090911.log
BESClient 489 root 6r VREG 85,6 4166
293972 /var/opt/BESClient/__BESData/actionsite/Action 56550.fxf
BESClient 489 root 7r VREG 85,6 4166
293861 /var/opt/BESClient/__BESData/actionsite/Action 56547.fxf
BESClient 489 root 8r VREG 85,6 4166
293974 /var/opt/BESClient/__BESData/actionsite/Action 56553.fxf
BESClient 489 root 9r VREG 85,6 780279
625216 /var/opt/BESClient/__BESData/Enterprise Security/2008 Security
Bulletins (x64).fxf
BESClient 489 root 10r VREG 85,6 677360
625332 /var/opt/BESClient/__BESData/Enterprise Security/2008 Security
Bulletins (Apps).fxf
BESClient 489 root 11r VREG 85,6 353830
625330 /var/opt/BESClient/__BESData/Enterprise Security/2007 Security
Bulletins (x64).fxf
BESClient 489 root 12r VREG 85,6 1025775
625194 /var/opt/BESClient/__BESData/Enterprise Security/2007 Security
Bulletins.fxf
BESClient 489 root 13r VREG 85,6 1304004
625217 /var/opt/BESClient/__BESData/Enterprise Security/2008 Security
Bulletins.fxf
BESClient 489 root 14u VREG 85,6 240
305028 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56547.dat
BESClient 489 root 15u VREG 85,6 240
305029 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56550.dat
BESClient 489 root 16u VREG 85,6 240
304994 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56542.dat
BESClient 489 root 17u VREG 85,6 240
305047 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56553.dat
BESClient 489 root 18u VREG 85,6 240
305024 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56544.dat
BESClient 489 root 19u VREG 85,6 240
305061 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56561.dat
BESClient 489 root 20u VREG 85,6 240
305000 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56543.dat
BESClient 489 root 21u VREG 85,6 240
305048 /var/opt/BESClient/__BESData/actionsite/__Local/Tmp/Action
56556.dat
oracle 522 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 524 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 526 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 528 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 530 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 532 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 534 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 536 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 538 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
oracle 540 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
esmd 671 root txt VREG 85,6 360
366114 /var/ld/ld.config
esmcifd 674 root txt VREG 85,6 360
366114 /var/ld/ld.config
xntpd 695 root txt VREG 85,6 360
366114 /var/ld/ld.config
tnslsnr 724 oracle txt VREG 85,6 360
366114 /var/ld/ld.config
From: neilsolent on 11 Sep 2009 04:33

> Are there any messages in //var/adm/messages or in the kernel; what

Just these (of course):
Sep 6 03:10:36 dsys01a ufs: [ID 213553 kern.notice] NOTICE:
realloccg /var: file system full
Sep 6 03:10:40 dsys01a last message repeated 1 time
Sep 6 03:10:44 dsys01a ufs: [ID 845546 kern.notice] NOTICE: alloc: /
var: file system full

that log is on /var itself !

> does "fsck -n" report on /var.  (free blocks, fragments).

I am a bit loathe to run fsck until I feel confident it won't cause
the system to hang due to corruption in /var (bit paranoid at the
moment). Will probably be running it soon anyway !
From: hume.spamfilter on 11 Sep 2009 10:10
neilsolent <n(a)solenttechnology.co.uk> wrote:
> dsys01a # find /proc/ -type f -links 0

Some of that output is interesting. If you add '-ls' to the end, what do
you see?

And something that just occurred to me... what does 'metastat -c d6' tell
you?

Finally, fsck -n tells fsck not to open the filesystem for writing. It
shouldn't harm the system to run.

--
Brandon Hume - hume -> BOFH.Ca, http://WWW.BOFH.Ca/
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: Solaris 10 installation and RAID config
Next: Anyone using ZFS on a Dell 1850 w/ LSI Megaraid?