rwsem: use single atomic update for sem count when waking up readers.
in [Kernel]
|
Prev: [RESEND] Handle instruction cache maintenance fault properly
Next: mqueue: fix kernel BUG caused by double free() on mq_open()
From: David Howells on 12 May 2010 07:10 Michel Lespinasse <walken(a)google.com> wrote: > - * - there must be someone on the queue > + * - there must be someone on the queue Why did you change this comment? This is still a guarantee up_xxxx() must make about the state of the rwsem. > + waiter = list_entry(sem->wait_list.next, struct rwsem_waiter, list); > + if (!(waiter->flags & RWSEM_WAITING_FOR_WRITE)) > + goto readers_only; > + > if (downgrading) > - goto dont_wake_writers; > + /* Caller's lock is still active, so we can't possibly > + * succeed waking writers. > + */ > + goto out; It's a nice idea to do it this way round - it puts the wake-up-reader path first and puts the downgrader on the slower path. > - /* if we came through an up_xxxx() call, we only only wake someone up > + /* There's a writer at the front of the queue - try to grant it the > + * write lock. However, we only only wake someone up > * if we can transition the active part of the count from 0 -> 1 > */ Two spaces after a full stop, please, and can you please adjust the comment so that it fills out to 80 chars. E.g: /* There's a writer at the front of the queue - try to grant it the * write lock. However, we only only wake someone up if we can * transition the active part of the count from 0 -> 1 */ instead of: /* There's a writer at the front of the queue - try to grant it the * write lock. However, we only only wake someone up * if we can transition the active part of the count from 0 -> 1 */ > + retry_readers: > + oldcount = rwsem_atomic_update(woken, sem) - woken; > + if (!downgrading && (oldcount & RWSEM_ACTIVE_MASK)) The problem with doing this here is that you may just have wasted all the work you did working out what woken is going to be. That may have been quite slow as the CPU may have had to get hold of a bunch of cachelines that weren't in its cache. Furthermore, you are doing this under a spinlock, so you may have lost your right to wake anyone up, and you'll be blocking the CPU that will be allowed to perform the wakeup. Incrementing the count first nets you a guarantee that you have the right to wake things up. You may point out that if there's no contention, then what your revised code does doesn't slow anything down. That's true, but on modern CPU's, neither does the old code as the exclusively held cache line will lurk in the CPU's cache until there's contention on it. David -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: David Howells on 12 May 2010 07:40
Michel Lespinasse <walken(a)google.com> wrote: > + * write lock. However, we only only wake someone up Can you get rid of the excess 'only' whilst you're at it? David -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |