MASTER_SITE defined by google search result
in [Ports]
|
From: Emanuel Haupt on 22 Jul 2010 06:17 Usually when I find that a port can't fetch the according distfile I simply google for the distfile with the following search string (e.g. for rsync): intitle:"index of" rsync-3.0.7.tar.gz This approach almost always delivers plenty of results. I then manually download one of the results into my DISTDIR and restart the normal build process which then automatically takes care of the integrity (checksum, size) of the distfile(s). I thought, why not automate this and I came up with a simple perl script (requires www/p5-Google-Search): http://people.freebsd.org/~ehaupt/distfile-search/distfile-search Adding the following make magic to /etc/make.conf # distfile search ..if defined(DISTFILE_SEARCH_BACKUP) pre-fetch-script: MASTER_SITE_BACKUP+= `/root/bin/distfile-search ${ALLFILES}` ..elif defined(DISTFILE_SEARCH_OVERRIDE) pre-fetch-script: MASTER_SITE_OVERRIDE= `/root/bin/distfile-search ${ALLFILES}` ..endif will let you choose whether you want to overwrite the pre defined MASTER_SITES: root(a)foo: /usr/ports/x11/xorg/ $ make -DDISTFILE_SEARCH_OVERRIDE fetch or add the search result as backup MASTER_SITES: root(a)foo: /usr/ports/x11/xorg/ $ make -DDISTFILE_SEARCH_BACKUP fetch In the later example you even might want to define DISTFILE_SEARCH_BACKUP globally. As a proof of concept I was able to download all distfiles for x11/xorg with this method (make -DDISTFILE_SEARCH_OVERRIDE fetch-recursive). Please let me know what you think. Emanuel _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: Chip Camden on 22 Jul 2010 11:32 Quoth Emanuel Haupt on Thursday, 22 July 2010: > Usually when I find that a port can't fetch the according distfile I > simply google for the distfile with the following search string (e.g. > for rsync): > > intitle:"index of" rsync-3.0.7.tar.gz > > This approach almost always delivers plenty of results. I then manually > download one of the results into my DISTDIR and restart the normal > build process which then automatically takes care of the integrity > (checksum, size) of the distfile(s). > > I thought, why not automate this and I came up with a simple perl > script (requires www/p5-Google-Search): > > http://people.freebsd.org/~ehaupt/distfile-search/distfile-search > > Adding the following make magic to /etc/make.conf > > # distfile search > .if defined(DISTFILE_SEARCH_BACKUP) > pre-fetch-script: > MASTER_SITE_BACKUP+= `/root/bin/distfile-search ${ALLFILES}` > .elif defined(DISTFILE_SEARCH_OVERRIDE) > pre-fetch-script: > MASTER_SITE_OVERRIDE= `/root/bin/distfile-search ${ALLFILES}` > .endif > > will let you choose whether you want to overwrite the pre defined > MASTER_SITES: > > root(a)foo: /usr/ports/x11/xorg/ $ make -DDISTFILE_SEARCH_OVERRIDE fetch > > or add the search result as backup MASTER_SITES: > > root(a)foo: /usr/ports/x11/xorg/ $ make -DDISTFILE_SEARCH_BACKUP fetch > > In the later example you even might want to define > DISTFILE_SEARCH_BACKUP globally. > > As a proof of concept I was able to download all distfiles for x11/xorg > with this method (make -DDISTFILE_SEARCH_OVERRIDE fetch-recursive). > > Please let me know what you think. > > Emanuel Pretty cool! Of course, if DISTFILE_SEARCH_BACKUP is defined, make will always execute a search even if it isn't needed -- but it appears to be fast so perhaps that isn't a problem. -- Sterling (Chip) Camden | sterling(a)camdensoftware.com | 2048D/3A978E4F http://camdensoftware.com | http://chipstips.com | http://chipsquips..com
From: Ade Lovett on 22 Jul 2010 12:55 On Jul 22, 2010, at 10:32 , Chip Camden wrote: > Quoth Emanuel Haupt on Thursday, 22 July 2010: >> >> http://people.freebsd.org/~ehaupt/distfile-search/distfile-search >> >> Adding the following make magic to /etc/make.conf >> >> # distfile search >> .if defined(DISTFILE_SEARCH_BACKUP) >> pre-fetch-script: >> MASTER_SITE_BACKUP+= `/root/bin/distfile-search ${ALLFILES}` >> .elif defined(DISTFILE_SEARCH_OVERRIDE) >> pre-fetch-script: >> MASTER_SITE_OVERRIDE= `/root/bin/distfile-search ${ALLFILES}` >> .endif This is likely worthy of a bit of cleanup, adding the code somewhere under ports/Tools/scripts, finding a suitable usage variable, and hooking it into bsd.port.mk (or bsd.fetch.mk, whenever someone[tm]) gets around to splitting out all the fetching stuff from bsd.port.mk ;) -aDe _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: RW on 22 Jul 2010 13:35 On Thu, 22 Jul 2010 12:17:31 +0200 Emanuel Haupt <ehaupt(a)FreeBSD.org> wrote: > Usually when I find that a port can't fetch the according distfile I > simply google for the distfile with the following search string (e.g. > for rsync): > > intitle:"index of" rsync-3.0.7.tar.gz > > This approach almost always delivers plenty of results. I then > manually download one of the results into my DISTDIR and restart the > normal build process How often do you actually do this? Whenever I have a problem with a distfile it's normally a rerolled file that's been propagated to all mirrors. _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org"
From: Emanuel Haupt on 22 Jul 2010 14:06
RW <rwmaillists(a)googlemail.com> wrote: > On Thu, 22 Jul 2010 12:17:31 +0200 > Emanuel Haupt <ehaupt(a)FreeBSD.org> wrote: > > > Usually when I find that a port can't fetch the according distfile I > > simply google for the distfile with the following search string > > (e.g. for rsync): > > > > intitle:"index of" rsync-3.0.7.tar.gz > > > > This approach almost always delivers plenty of results. I then > > manually download one of the results into my DISTDIR and restart the > > normal build process > > How often do you actually do this? Whenever I have a problem with a > distfile it's normally a rerolled file that's been propagated to all > mirrors. Unfortunately, recently much more often. Especially with non default OPTIONS that pull in additional patches/distfiles. Emanuel _______________________________________________ freebsd-ports(a)freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscribe(a)freebsd.org" |