Prev: [ENCODING QUEUE] - anyone think of a good way to setup a queuingsystem so everyone gets their turn
Next: It's Friday (a MySQL Question)
From: Ashley Sheridan on 13 Aug 2010 17:08
On Fri, 2010-08-13 at 14:48 -0600, Tristan wrote:

> i have a list of people that uploaded their videos waiting to be encoded.
>
> these are the line items for the db
>
> id | member_id | created | status
>
> I just want to make sure that everyone gets a turn processing.
>
> I was thinking WHERE DISTINCT(member_id) and created (oldest)
>
> but i see some flaw in that.
>
>
> if someone uploaded 20 vids and then someone else uploads 30 vids they'll
> still have to wait until the 20 finish.
>
> Almost have to keep a list of the users being processed in another table and
> if someone else comes along append them in.
>
> say we have 5 people that uploaded multiple videos. I want it to go like
>
> sandy, tom, jim, harry, star
>
> and then back to
>
> sandy, tom, jim, harry, star
>
> then if someone else comes into the mix. add them to that queue
>
> Thanks, T


You can probably do it with the one video list in the DB. Each time your
video encoder has a free spot it should obtain a list of videos from
this table from the last time period, both processed and waiting to be
queued. Look up the first one from this list and check to see if a video
for the same user has been processed within a specific time period. If
yes, then move to the next video in the list and check that in the same
manner, and repeat until you find one. If you don't find any, then you
can just pick any video from the list.

You should be able to process several videos at the same time on a
decent server, and a 64-bit system should make mincemeat of an encode as
it should do that type of number crunching more quickly.

Thanks,
Ash
http://www.ashleysheridan.co.uk


From: Tristan on 13 Aug 2010 17:24
Here's a question for you about encoding on the server. If two videos are
encoding at the same time. Do they share the processing power.

lets say we have 2 vids same size

1 encodes at 1 minute

and then if 2 are going at the same time does it encode them both in 2
minutes?

Thanks, T

On Fri, Aug 13, 2010 at 3:08 PM, Ashley Sheridan
<ash(a)ashleysheridan.co.uk>wrote:

> On Fri, 2010-08-13 at 14:48 -0600, Tristan wrote:
>
> i have a list of people that uploaded their videos waiting to be encoded.
>
> these are the line items for the db
>
> id | member_id | created | status
>
> I just want to make sure that everyone gets a turn processing.
>
> I was thinking WHERE DISTINCT(member_id) and created (oldest)
>
> but i see some flaw in that.
>
>
> if someone uploaded 20 vids and then someone else uploads 30 vids they'll
> still have to wait until the 20 finish.
>
> Almost have to keep a list of the users being processed in another table and
> if someone else comes along append them in.
>
> say we have 5 people that uploaded multiple videos. I want it to go like
>
> sandy, tom, jim, harry, star
>
> and then back to
>
> sandy, tom, jim, harry, star
>
> then if someone else comes into the mix. add them to that queue
>
> Thanks, T
>
>
> You can probably do it with the one video list in the DB. Each time your
> video encoder has a free spot it should obtain a list of videos from this
> table from the last time period, both processed and waiting to be queued.
> Look up the first one from this list and check to see if a video for the
> same user has been processed within a specific time period. If yes, then
> move to the next video in the list and check that in the same manner, and
> repeat until you find one. If you don't find any, then you can just pick any
> video from the list.
>
> You should be able to process several videos at the same time on a decent
> server, and a 64-bit system should make mincemeat of an encode as it should
> do that type of number crunching more quickly.
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>
>
From: Ashley Sheridan on 13 Aug 2010 17:41
On Fri, 2010-08-13 at 15:24 -0600, Tristan wrote:

> Here's a question for you about encoding on the server. If two videos are
> encoding at the same time. Do they share the processing power.
>
> lets say we have 2 vids same size
>
> 1 encodes at 1 minute
>
> and then if 2 are going at the same time does it encode them both in 2
> minutes?
>
> Thanks, T
>
> On Fri, Aug 13, 2010 at 3:08 PM, Ashley Sheridan
> <ash(a)ashleysheridan.co.uk>wrote:
>
> > On Fri, 2010-08-13 at 14:48 -0600, Tristan wrote:
> >
> > i have a list of people that uploaded their videos waiting to be encoded.
> >
> > these are the line items for the db
> >
> > id | member_id | created | status
> >
> > I just want to make sure that everyone gets a turn processing.
> >
> > I was thinking WHERE DISTINCT(member_id) and created (oldest)
> >
> > but i see some flaw in that.
> >
> >
> > if someone uploaded 20 vids and then someone else uploads 30 vids they'll
> > still have to wait until the 20 finish.
> >
> > Almost have to keep a list of the users being processed in another table and
> > if someone else comes along append them in.
> >
> > say we have 5 people that uploaded multiple videos. I want it to go like
> >
> > sandy, tom, jim, harry, star
> >
> > and then back to
> >
> > sandy, tom, jim, harry, star
> >
> > then if someone else comes into the mix. add them to that queue
> >
> > Thanks, T
> >
> >
> > You can probably do it with the one video list in the DB. Each time your
> > video encoder has a free spot it should obtain a list of videos from this
> > table from the last time period, both processed and waiting to be queued.
> > Look up the first one from this list and check to see if a video for the
> > same user has been processed within a specific time period. If yes, then
> > move to the next video in the list and check that in the same manner, and
> > repeat until you find one. If you don't find any, then you can just pick any
> > video from the list.
> >
> > You should be able to process several videos at the same time on a decent
> > server, and a 64-bit system should make mincemeat of an encode as it should
> > do that type of number crunching more quickly.
> >
> > Thanks,
> > Ash
> > http://www.ashleysheridan.co.uk
> >
> >
> >


Depends on several factors:

Do you have a multi-core processor? (unlikely that you don't)

Does your encoding software know about the different cores, i.e. can it
take advantage of them?

Do you have enough RAM on the machine to be shared efficiently by each
encode?

Do your hard drives have decent caching systems? If each encode were on
a separate physical disc, then that would speed things up, but only
marginally.

Sometimes a graphics card can help with encoding issues. Does your
system have such a card, and is the system able to make use of the
capability?

Incidentally, what software are you using to produce the encodes? Is it
something like ffmpeg or mencoder?

Thanks,
Ash
http://www.ashleysheridan.co.uk


 | 
Pages: 1
Prev: [ENCODING QUEUE] - anyone think of a good way to setup a queuingsystem so everyone gets their turn
Next: It's Friday (a MySQL Question)