Prev: Connecting via Win98 driver.
Next: Problems in sending mails to netdev@vger.kernel.org
From: Andrew Gideon on 2 Aug 2010 15:22
I'm using Linux-based routers, and I want to limit individual hosts'
bandwidth, or sometimes limit the bandwidth of a collection of hosts
together (ie. the hosts share an allocation of bandwidth).

A factor that appears important is that there's only one place where all
the traffic can be seen: the interface on the router that is providing
gateway service for the host's LAN. Once an outbound packet is on the
router, it may leave the router through one of several different
interfaces. So, from what I can tell, my only option is ingress policing.

This is my first question: Am I right about this? Or is there a way to
do egress shaping such that packets will be counted against one specific
allocation even through the packets will be egressing via multiple
interfaces?

The document http://lartc.org/howto/lartc.qdisc.advice.html suggests that
I'm doing the right thing:

If you *are* forwarding it, use a TBF on the interface you
are forwarding the data to. Unless you want to shape traffic
that may go out over several interfaces, in which case the
only common factor is the incoming interface. In that case
use the Ingress Policer.

but I'm hoping that this recommendation is out of date.

[I'm hoping because I'd prefer to be more flexible: to permit hosts to go
over their allocated bandwidth if there's bandwidth available. But this
requires fancier shaping; I don't see how I can do it via policing.]

I've set up ingress policing as follows:

/sbin/tc qdisc add dev eth0 handle ffff: ingress
/sbin/tc filter add dev eth0 parent ffff: \
protocol ip prio 20 \
u32 match ip src 10.10.10.10/32 \
police index 2 rate 200kbit buffer 40k drop \
flowid :1

This seems almost right when I test it with a single TCP circuit (I've
been using scp as my test). A single scp gets about 320Kb/second steady
state (or a copy rate of about 25KB/s).

But when I run multiple circuits (multiple scp operations concurrently),
bandwidth consumption increases. 10 copies running at one time gets an
aggregate of about 600Kb/second.

This is my second question: Why would that be occurring (and "how can I
prevent it" is my third)? If tc operates on a per-packet basis, why does
having multiple TCP circuits (or scp copies running) make a difference?

Thanks...

Andrew
From: Arimus on 3 Aug 2010 04:10
On Aug 2, 8:22 pm, Andrew Gideon <c182driv...(a)gideon.org> wrote:
[snip]
> This is my first question: Am I right about this?  Or is there a way to
> do egress shaping such that packets will be counted against one specific
> allocation even through the packets will be egressing via multiple
> interfaces?

You have to use ingress filtering in your case - each network
interface has its own queuing policy (even if its just the pfifo_fast
default) - so if my traffic leaves on nics eth0, eth3, eth4 each one
will only see a third of my traffic. Using ingress filtering on the
router though will see all my traffic and afaiu will dequeue the
traffic according to the policy in place.


[snip]

> /sbin/tc qdisc add dev eth0 handle ffff: ingress
> /sbin/tc filter add dev eth0 parent ffff: \
>     protocol ip prio 20 \
>     u32 match ip src 10.10.10.10/32 \
>     police index 2 rate 200kbit buffer 40k drop \
>     flowid :1
>
> This seems almost right when I test it with a single TCP circuit (I've
> been using scp as my test).  A single scp gets about 320Kb/second steady
> state (or a copy rate of about 25KB/s).
>
> But when I run multiple circuits (multiple scp operations concurrently),
> bandwidth consumption increases.  10 copies running at one time gets an
> aggregate of about 600Kb/second.  
>
> This is my second question: Why would that be occurring (and "how can I
> prevent it" is my third)?  If tc operates on a per-packet basis, why does
> having multiple TCP circuits (or scp copies running) make a difference?
>

Pass :(

R.

From: Andrew Gideon on 3 Aug 2010 10:56
On Tue, 03 Aug 2010 01:10:42 -0700, Arimus wrote:

> On Aug 2, 8:22 pm, Andrew Gideon <c182driv...(a)gideon.org> wrote: [snip]
>> This is my first question: Am I right about this?  Or is there a way to
>> do egress shaping such that packets will be counted against one
>> specific allocation even through the packets will be egressing via
>> multiple interfaces?
>
> You have to use ingress filtering in your case - each network interface
> has its own queuing policy (even if its just the pfifo_fast default) -
> so if my traffic leaves on nics eth0, eth3, eth4 each one will only see
> a third of my traffic. Using ingress filtering on the router though will
> see all my traffic and afaiu will dequeue the traffic according to the
> policy in place.

I'm looking into using something called ifb for this. I'd actually
started looking at this over a year ago, but it got put aside and then I
forgot about it.

I'm not sure if this is fully available in CentOS yet, though, which is
what I'm using on my routers. I found this:

http://bugs.centos.org/view.php?id=3673

which suggests CentOS problems, but it is fairly out of date.

According to:

http://www.linuxfoundation.org/collaborate/workgroups/networking/ifb

(and some helpful USENET posts by Andy Furniss from 2009/02 on this
group) ifb seems to do what I want: It seems to provide a common resource
for multiple [egress] interfaces where shaping can be implemented. At
least, this is what I'm hoping.

[...]

> Pass

I figured this one out. I was measuring traffic on the switch port where
the scp traffic was originating and on the router interface which was
receiving the traffic. This included dropped ("policed") packets!

When I finally thought to look on the receiving side, I saw the proper
bandwidth regardless of how many circuits were running. So this was an
error in how I was measuring results; not a problem in the actual
policing.

Thanks...

Andrew
From: buck on 4 Aug 2010 13:37
Andrew Gideon <c182driver1(a)gideon.org> wrote in
news:x6W5o.6809$6F6.2974(a)news.usenetserver.com:
> I'm looking into using something called ifb for this. I'd actually
> started looking at this over a year ago, but it got put aside and then
> I forgot about it.
> Thanks...
> Andrew

I dislike IFB. If it does not work for you, look into IMQ at
http://www.linuximq.net/ .
IWFM.
--
gypsy
From: Andrew Gideon on 4 Aug 2010 13:44
On Wed, 04 Aug 2010 17:37:35 +0000, buck wrote:

> I dislike IFB.

Why? How is IMQ better/different [for shaping over multiple interfaces]?

- Andrew
 |  Next  |  Last
Pages: 1 2
Prev: Connecting via Win98 driver.
Next: Problems in sending mails to netdev@vger.kernel.org