Prev: Keychain with Opera browser
Next: Odd USB Extension Cable
From: David Empson on 7 Jan 2010 06:47
Sn!pe <snipe(a)spambin.fsnet.co.uk> wrote:

> Tim Lance <nope(a)nada.com> wrote:
>
> > >> My local hard drive is getting full so I'd like to clear some cruft.
> > >> I find that /private/var/log/kernel.log.1has grown to over 6 GB;
> > >> how can I reclaim some of this disk space, please? Can I simply
> > >> delete it? TIA.
> > >
> > > You should look at the log to figure out what is filling it up. My
> > > kernel log is only 745k, with mostly messages from VMware and Cisco VPN
> > > client in it.
> >
> > No joke. Mine is 684 KB. Whole log folder is 144.8 NB.
>
> Right, I'll investigate that when I have a moment. It looks as though
> I'm about to find Time Machine useful because the file has already
> gone with no apparent ill-effects. Meanwhile, I presume that
> /private/var/log/kernel.log is the active file of the moment?
> I'll keep a eye on that too.

Yes, that is the active kernel log file.

The Console application (in /Applications/Utilties) is the easiest way
to browse through the various logs on the computer. For Leopard and
later, click the "Show Log List" button at the left end of the toolbar.

The groupings vary between system versions, but the "FILES" heading
shows the actual log files, organised into directories: ~/Library/Logs
is log files for your user account, /Library/Logs is log files for the
whole computer (typically from higher level elements of Mac OS X, and
some third party software), and /private/var/log is the lower level logs
maintained by the BSD subsystem and various other components.


What is supposed to happen is that many of the active log files
(including kernel.log) are automatically archived by a tool called
'newsyslog' that runs every 30 minutes.

The default configuration for most of the logs is to only archive them
after they exceed 1 MB, keep between 3 and 7 archived logs, and to
compress the archives (using bzip2).

The archiving procedure results in a series of numbered files named like
"kernel.log.0.bz2", "kernel.log.1.bz2", etc. up to the maximum number (5
for kernel.log). As the number increases, the archive gets older, so
each time the log is archived, newsyslog has to delete the oldest one,
and rename all the others by incrementing their number, then it creates
a new ".0.bz2" file with the most recent log, and empties the active
log.


In your case, you had a "kernel.log.1" file which was 6 GB. That is not
a normal situation that the system should have got into without some
help.

The most likely explanation is that at some point, something or someone
extracted a "kernel.log.1.bz2" file (producing "kernel.log.1"). The
resulting expanded file isn't managed automatically by newsyslog, so it
just sat there wasting disk space.

--
David Empson
dempson(a)actrix.gen.nz
From: David Empson on 7 Jan 2010 18:01
Sn!pe <snipe(a)spambin.fsnet.co.uk> wrote:

> David Empson <dempson(a)actrix.gen.nz> wrote:
>
> [snip interesting explanation]
>
> > The most likely explanation is that at some point, something or someone
> > extracted a "kernel.log.1.bz2" file (producing "kernel.log.1"). The
> > resulting expanded file isn't managed automatically by newsyslog, so it
> > just sat there wasting disk space.
>
> Thanks very much, David, that all makes sense.
> When I looked in /private/var/log/ just now I found:-
>
> kernel.log 590 KB
> kernel.log.0.bz2 520 KB
> [kernel.log.1] 6 GB (deleted previously)
> kernel.log.2 323 MB
> kernel.log.3.bz2 1.8 MB
> kernel.log.4.bz2 61 KB
>
> - so it looks as though something similar has happened to
> kernel.log.2 also.

Yep.

It may have been interesting to know how it got that big in the first
place - at some point you managed to get 6 GB of diagnostic messages
from the kernel in about half an hour (assuming no shut downs or
restarts in that period). kernel.log.2 is also unusually large, and
kernel.log.3 would probably be in the order of 40 MB given the
compression ratio of the others.

The other interesting question is why your sequence numbers are
sequential - there is no overlapping kernel.log.1.bz2 and
kernel.log.2.bz2.

This leads to a much more likely explanation of what caused the problem.

I expect that newsyslog is able to manage archived logs whether or not
they have been compressed.

At some point in the past, the following sequence occurred:

1. The kernel was operating normally. It eventually accumulated 1 MB of
date in kernel.log, which newsyslog compressed to form the file which is
now your kernel.log.4.bz2. At that point you would have had:

kernel.log 0 B
kernel.log.0.bz2 61 KB
(and four earlier archived logs)

2. Something started generating a lot of kernel log data, resulting in
the next file accumulating what is probably 40 MB or more of data before
newsyslog ran again. That log was compressed and archived to produce the
following:

kernel.log 0 B
kernel.log.0.bz2 1.8 MB
kernel.log.1.bz2 61 KB
(and three earlier archived logs)

3. The pace of kernel log messages increased enough that 323 MB was
logged before newsyslog ran again.

At this point, the startup disk didn't have enough space to store the
compressed copy of the log, so newsyslog archived the uncompressed log.
This resulted in:

kernel.log 0 B
kernel.log.0 323 MB
kernel.log.1.bz2 1.8 MB
kernel.log.2.bz2 61 KB
(and two earlier archived logs)

4. You probably freed up some disk space shortly after this, but the
kernel log messages were accelerating out of control, resulting in 6 GB
being logged before newsyslog ran again.

Due to lack of disk space, the uncompressed log was archived again. This
resulted in:

kernel.log 0 B
kernel.log.0 6 GB
kernel.log.1 323 MB
kernel.log.2.bz2 1.8 MB
kernel.log.3.bz2 61 KB
(and one earlier archived log)

5. Given the peculiar way your computer was behaving, I expect you
restarted or had a kernel panic about this point. After the restart, the
kernel was behaving normally.

6. The kernel eventually accumulated another 1 MB of log messages, at
which point newsyslog rotated the archives again to get the pattern you
reported above.


As you've manually deleted two archived log files, leaving a gap, it may
take a while for the normal pattern of sequence numbers to be restored,
but I expect newsyslog will handle it correctly.

--
David Empson
dempson(a)actrix.gen.nz
From: John Varela on 7 Jan 2010 21:38
On Thu, 7 Jan 2010 09:25:41 UTC, snipe(a)spambin.fsnet.co.uk (Sn!pe)
wrote:

> John Varela <OLDlamps(a)verizon.net> wrote:
>
> > > My local hard drive is getting full so I'd like to clear some cruft.
> > > I find that /private/var/log/kernel.log.1has grown to over 6 GB;
> > > how can I reclaim some of this disk space, please? Can I simply
> > > delete it? TIA.
> >
> > How did you get the size in bytes? I'm unfamiliar with the Un*x
> > command line, and all I can get is the size in blocks, but I don't
> > know how big is a block.

Evidently it's 512 bytes. My System Log is 88 MB, which seems
awfully large after only a few months of running Snow Leopard.
Still, I suppose Snow Leopard will be superseded and a new log file
started long before the file reaches a GB.

> I looked at 'Get Info' in Finder for the number of bytes.

Ah so. You are running in an administrator account. /private can't
be accessed from a user account, which is why I went to Terminal and
sudo'ed. I shoulda thought to go to the admin account.

> I was alerted to the size of the file by using Disk Inventory X,
> which gives a graphical map view by file type and size of the
> entire contents of your disk. It's very pretty and quite useful.

--
John Varela
Trade NEWlamps for OLDlamps for email
From: David Empson on 7 Jan 2010 23:20
John Varela <OLDlamps(a)verizon.net> wrote:

> On Thu, 7 Jan 2010 09:25:41 UTC, snipe(a)spambin.fsnet.co.uk (Sn!pe)
> wrote:
>
> > John Varela <OLDlamps(a)verizon.net> wrote:
> >
> > > > My local hard drive is getting full so I'd like to clear some cruft.
> > > > I find that /private/var/log/kernel.log.1has grown to over 6 GB;
> > > > how can I reclaim some of this disk space, please? Can I simply
> > > > delete it? TIA.
> > >
> > > How did you get the size in bytes? I'm unfamiliar with the Un*x
> > > command line, and all I can get is the size in blocks, but I don't
> > > know how big is a block.
>
> Evidently it's 512 bytes. My System Log is 88 MB, which seems
> awfully large after only a few months of running Snow Leopard.
> Still, I suppose Snow Leopard will be superseded and a new log file
> started long before the file reaches a GB.

What dates are the archived copies of system.log? This command in
Terminal will list them all, with file sizes in bytes:

ls -l /var/log/system*

This is what mine shows:

-rw-r----- 1 root admin 10596 8 Jan 00:30 /var/log/system.log.0.bz2
-rw-r----- 1 root admin 15424 7 Jan 00:31 /var/log/system.log.1.bz2
-rw-r----- 1 root admin 10355 6 Jan 00:30 /var/log/system.log.2.bz2
-rw-r----- 1 root admin 1576 5 Jan 00:30 /var/log/system.log.3.bz2
-rw-r----- 1 root admin 18926 5 Jan 00:12 /var/log/system.log.4.bz2
-rw-r----- 1 root admin 23765 26 Dec 00:30 /var/log/system.log.5.bz2
-rw-r----- 1 root admin 32591 21 Dec 00:52 /var/log/system.log.6.bz2
-rw-r----- 1 root admin 12088 18 Dec 00:30 /var/log/system.log.7.bz2

My file sizes vary betwen 1.5K and 32.5K. Note that my most recent logs
have been archived roughly every 24 hours, near midnight, but there is a
bit of a gap to the previous few.

The archiving of system.log is based on a rule that requires the
computer to be active between midnight and 1 a.m. If your computer is
never active at that time of day, system.log will never be archived.

The newsyslog tool that handles this should be run automatically every
half hour, so there are at least two points between midnight and 1 a.m.
at which the archiving of system.log may occur.

This seems like a rather peculiar way of handling it - I imagine a
significant proportion of Mac users are not night owls.

I'd almost classify this as a bug. I think it should have a secondary
trigger condition based on the size of system.log, which could trigger
at any time of day when newsyslog is run.

On my system, the broken pattern of dates means I was using my computer
between midnight and 1 a.m. on Dec 18, 21 and 26, but not on any other
day in late December. I then didn't use it again after midnight until
Jan 5. Not sure why it ended up with two archived logs on that day.

I'll need to check some computers running older system versions to see
if the rule/mechanism has changed in Snow Leopard.

> > I looked at 'Get Info' in Finder for the number of bytes.
>
> Ah so. You are running in an administrator account. /private can't
> be accessed from a user account, which is why I went to Terminal and
> sudo'ed. I shoulda thought to go to the admin account.

/private/var/log certainly can be accessed from a non-admin account.
(Doing it from Finder would require "Go to Folder".)

Some of the files in that folder cannot be opened by a non-admin, but
you can get information about them.

--
David Empson
dempson(a)actrix.gen.nz
From: John Varela on 8 Jan 2010 19:58
On Fri, 8 Jan 2010 04:20:13 UTC, dempson(a)actrix.gen.nz (David
Empson) wrote:

> The archiving of system.log is based on a rule that requires the
> computer to be active between midnight and 1 a.m. If your computer is
> never active at that time of day, system.log will never be archived.

Thanks for that info. I'll let the computer run overnight tonight.

--
John Varela
Trade NEWlamps for OLDlamps for email
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: Keychain with Opera browser
Next: Odd USB Extension Cable