Prev: dm: use __GFP_HIGH instead PF_MEMALLOC
Next: macvlan: implement VEPA and private mode
From: KAMEZAWA Hiroyuki on 14 Dec 2009 23:50
On Mon, 14 Dec 2009 20:30:37 -0800 (PST)
David Rientjes <rientjes(a)google.com> wrote:

> On Tue, 15 Dec 2009, KAMEZAWA Hiroyuki wrote:
>
> > I'm now preparing more counters for mm's statistics. It's better to
> > wait and to see what we can do more. And other patches for total
> > oom-killer improvement is under development.
> >
> > And, there is a compatibility problem.
> > As David says, this may break some crazy software which uses
> > fake_numa+cpuset+oom_killer+oom_adj for resource controlling.
> > (even if I recommend them to use memcg rather than crazy tricks...)
> >
>
> That's not at all what I said. I said using total_vm as a baseline allows
> users to define when a process is to be considered "rogue," that is, using
> more memory than expected. Using rss would be inappropriate since it is
> highly dynamic and depends on the state of the VM at the time of oom,
> which userspace cannot possibly keep updated.
>
> You consistently ignore that point: the power of /proc/pid/oom_adj to
> influence when a process, such as a memory leaker, is to be considered as
> a high priority for an oom kill. It has absolutely nothing to do with
> fake NUMA, cpusets, or memcg.
>
You also ignore that it's not sane to use oom kill for resource control ;)

Anyway, rss patch was dropped as you want.
I'll prepare other ones.

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: KAMEZAWA Hiroyuki on 15 Dec 2009 00:10
On Tue, 15 Dec 2009 13:35:46 +0900
KAMEZAWA Hiroyuki <kamezawa.hiroyu(a)jp.fujitsu.com> wrote:

> On Mon, 14 Dec 2009 20:30:37 -0800 (PST)
> David Rientjes <rientjes(a)google.com> wrote:
> > > 2 ideas which I can think of now are..
> > > 1) add sysctl_oom_calc_on_committed_memory
> > > If this is set, use vm-size instead of rss.
> > >
> >
> > I would agree only if the oom killer used total_vm as a the default, it is
> > long-standing and allows for the aforementioned capability that you lose
> > with rss. I have no problem with the added sysctl to use rss as the
> > baseline when enabled.
> >
> I'll prepare a patch for adds
>
> sysctl_oom_kill_based_on_rss (default=0)
>
Hmm..

But for usual desktop users, using rss as default,as memory-eater-should-die.
sounds reasoable to me.
Hmm...maybe some automatic detection logic is required.

As my 1st version shows,

CONSTRAINT_CPUSET -> use vm_size
CONSTRAINT_LOWMEM -> use lowmem_rss
CONSTRAINT_NONE -> use rss
seems like a landing point for all stake holders. No ?

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: KAMEZAWA Hiroyuki on 15 Dec 2009 00:20
On Mon, 14 Dec 2009 20:57:53 -0800 (PST)
David Rientjes <rientjes(a)google.com> wrote:

> On Tue, 15 Dec 2009, KAMEZAWA Hiroyuki wrote:
>
> > > That's not at all what I said. I said using total_vm as a baseline allows
> > > users to define when a process is to be considered "rogue," that is, using
> > > more memory than expected. Using rss would be inappropriate since it is
> > > highly dynamic and depends on the state of the VM at the time of oom,
> > > which userspace cannot possibly keep updated.
> > >
> > > You consistently ignore that point: the power of /proc/pid/oom_adj to
> > > influence when a process, such as a memory leaker, is to be considered as
> > > a high priority for an oom kill. It has absolutely nothing to do with
> > > fake NUMA, cpusets, or memcg.
> > >
> > You also ignore that it's not sane to use oom kill for resource control ;)
> >
>
> Please read my email. Did I say anything about resource control AT ALL?
> I said /proc/pid/oom_adj currently allows userspace to define when a task
> is "rogue," meaning its consuming much more memory than expected. Those
> memory leakers should always be the optimal result for the oom killer to
> kill. Using rss as the baseline would not allow userspace to effectively
> do the same thing since it's dynamic and depends on the state of the VM at
> the time of oom which is probably not reflected in the /proc/pid/oom_adj
> values for all tasks. It has absolutely nothing to do with resource
> control, so please address this very trivial issue without going off on
> tangents. Thanks.

What I can't undestand is the technique to know whether a (unknown) process is
leaking memory or not by checking vm_size.
And, why don't you use overcommit_memory when you can depends on vm_size ?

Thanks,
-Kame


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: KAMEZAWA Hiroyuki on 17 Dec 2009 18:40
On Thu, 17 Dec 2009 14:23:39 -0800 (PST)
David Rientjes <rientjes(a)google.com> wrote:

> On Tue, 15 Dec 2009, KAMEZAWA Hiroyuki wrote:
>
> > What I can't undestand is the technique to know whether a (unknown) process is
> > leaking memory or not by checking vm_size.
>
> Memory leaks are better identified via total_vm since leaked memory has a
> lower probability of staying resident in physical memory.
>
Because malloc() writes header on newly allcoated memory, (vm_size - rss) cannot
be far from a some important program which wakes up once in a
day or sleep in the day works in the night.

I hope user knows expected memory size of applications, but I know it can't.
Sigh...

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
First  |  Prev  | 
Pages: 1 2
Prev: dm: use __GFP_HIGH instead PF_MEMALLOC
Next: macvlan: implement VEPA and private mode