Prev: problem running kernel compiled on different machine
Next: How do I fsck an XFS file system in "Squeeze"
From: Cameron Hutchison on 30 May 2010 21:50
I have a few debian unstable boxes that I like to keep up-to-date.

Currently, I run apt-cacher-ng (a proxy for apt-get which stores
packages so they don't need to be downloaded again) on my gateway box
(lenny 32-bit), so I don't double download from my couple of other boxes
(sid 64-bit). This works well enough, but I want to make it better.

My internet connection has an off-peak period (2am-8am) where downloads
are not counted in the monthly quota. I want my downloads to occur
during this window, automatically.

The problem is that the sid boxes are not powered-up during this window
so I cannot just simply schedule a cronjob to do an apt-get -d
dist-upgrade.

I would like my lenny box to do this on behalf of my sid boxes. I dont
really want to mirror the whole archive. Is there some way to do an
intelligent partial mirror of the archive of what is installed on other
boxes, without too much overhead of managing a package list? (i.e. I dont
want to have to manually update some package list on my lenny box when I
install a new package on a sid box).

I've looked at apt-proxy, apt-cacher, apt-cacher-ng and approx, but they
all appear to download on demand, not according to a schedule.


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/85e.4c0313ab.76bef(a)getafix.xdna.net
From: Celejar on 30 May 2010 23:30
On Mon, 31 May 2010 01:40:59 -0000
Cameron Hutchison <lists(a)xdna.net> wrote:

> I have a few debian unstable boxes that I like to keep up-to-date.
>
> Currently, I run apt-cacher-ng (a proxy for apt-get which stores
> packages so they don't need to be downloaded again) on my gateway box
> (lenny 32-bit), so I don't double download from my couple of other boxes
> (sid 64-bit). This works well enough, but I want to make it better.
>
> My internet connection has an off-peak period (2am-8am) where downloads
> are not counted in the monthly quota. I want my downloads to occur
> during this window, automatically.
>
> The problem is that the sid boxes are not powered-up during this window
> so I cannot just simply schedule a cronjob to do an apt-get -d
> dist-upgrade.
>
> I would like my lenny box to do this on behalf of my sid boxes. I dont
> really want to mirror the whole archive. Is there some way to do an
> intelligent partial mirror of the archive of what is installed on other
> boxes, without too much overhead of managing a package list? (i.e. I dont
> want to have to manually update some package list on my lenny box when I
> install a new package on a sid box).
>
> I've looked at apt-proxy, apt-cacher, apt-cacher-ng and approx, but they
> all appear to download on demand, not according to a schedule.

You could use use the output of 'dpkg --get-selections' on the Sid
boxes, pruning it using grep or similar to keep only the lines ending
with 'install'. Combine the results from the various Sid boxes, and
then have the gateway box get those packages. This could be scripted
easily enough, and there would be no need for manually updating
anything.

Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/20100530232849.13be6e74.celejar(a)gmail.com
From: Cameron Hutchison on 31 May 2010 00:10
Celejar <celejar(a)gmail.com> writes:
>Cameron Hutchison <lists(a)xdna.net> wrote:

>> Is there some way to do an
>> intelligent partial mirror of the archive of what is installed on other
>> boxes, without too much overhead of managing a package list? (i.e. I dont
>> want to have to manually update some package list on my lenny box when I
>> install a new package on a sid box).

>You could use use the output of 'dpkg --get-selections' on the Sid
>boxes, pruning it using grep or similar to keep only the lines ending
>with 'install'.

ok. that's easy.

>Combine the results from the various Sid boxes,

so's that.

>and then have the gateway box get those packages.

hmmm. How? Is there an existing tool that will do this? Doing it
manually (in a script) would require too much work (essentially
implementing apt-get -d dist-upgrade against a specified package index
and package list).


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/c07.4c03366f.8a7ce(a)getafix.xdna.net
From: Celejar on 31 May 2010 00:40
On Mon, 31 May 2010 04:09:19 -0000
Cameron Hutchison <lists(a)xdna.net> wrote:

> Celejar <celejar(a)gmail.com> writes:
> >Cameron Hutchison <lists(a)xdna.net> wrote:
>
> >> Is there some way to do an
> >> intelligent partial mirror of the archive of what is installed on other
> >> boxes, without too much overhead of managing a package list? (i.e. I dont
> >> want to have to manually update some package list on my lenny box when I
> >> install a new package on a sid box).
>
> >You could use use the output of 'dpkg --get-selections' on the Sid
> >boxes, pruning it using grep or similar to keep only the lines ending
> >with 'install'.
>
> ok. that's easy.
>
> >Combine the results from the various Sid boxes,
>
> so's that.
>
> >and then have the gateway box get those packages.
>
> hmmm. How? Is there an existing tool that will do this? Doing it
> manually (in a script) would require too much work (essentially
> implementing apt-get -d dist-upgrade against a specified package index
> and package list).

How about using 'aptitude download', possibly feeding the package list
to aptitude with xargs? The 'download' action downloads to the current
directory, so I suppose that you'd use a temp directory, and that would
have the side effect of getting a copy stored by the cacher.

Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator


--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/20100531003508.36f3ddde.celejar(a)gmail.com
From: Cameron Hutchison on 31 May 2010 01:20
Celejar <celejar(a)gmail.com> writes:

>Cameron Hutchison <lists(a)xdna.net> wrote:

>> >and then have the gateway box get those packages.
>>
>> hmmm. How? Is there an existing tool that will do this? Doing it
>> manually (in a script) would require too much work (essentially
>> implementing apt-get -d dist-upgrade against a specified package index
>> and package list).

>How about using 'aptitude download', possibly feeding the package list
>to aptitude with xargs? The 'download' action downloads to the current
>directory, so I suppose that you'd use a temp directory, and that would
>have the side effect of getting a copy stored by the cacher.

The problem is the the dpkg architecture (and distribution) of the host
is different to the targets. 'aptitude download' is going to use the
native dpkg architecture, and is going to look in /etc/apt/sources.list
for the sources to use.

It looks like it may be possible to use something like 'aptitude
extract-cache-subset' on the targets and set Apt::Architecture to the
target on the command line. apt.conf(5) has a number of details of how I
can override most of the host environment.

Hmmm, I can see how this might work now.

* periodically run aptitude extract-cache-subset on the targets and
send that to the host
* every night on the host, run 'aptitude -d dist-upgrade', but
overriding all the directories for configs, indexes, pkgstates, etc and
architecture to point to what has been captured by extract-cache-subset.
Also point aptitude to the local apt-cacher-ng proxy so all downloads
go through it.
* delete the downloaded files, since apt-cacher-ng has already cached
them.

Thanks for the ideas. Perhaps I can knock this up fairly easily. I'll
certainly learn more about apt than I probably want to :-)



--
To UNSUBSCRIBE, email to debian-user-REQUEST(a)lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster(a)lists.debian.org
Archive: http://lists.debian.org/eb9.4c0344bd.22067(a)getafix.xdna.net
 |  Next  |  Last
Pages: 1 2
Prev: problem running kernel compiled on different machine
Next: How do I fsck an XFS file system in "Squeeze"