Debian Bug report logs - #642159
debian-installer preseed broken with apt-cacher-ng mirror

Package: choose-mirror; Maintainer for choose-mirror is Debian Install System Team <debian-boot@lists.debian.org>;

Affects: debian-installer

Reported by: Matt Weatherford <mbw@u.washington.edu>

Date: Mon, 19 Sep 2011 21:24:05 UTC

Severity: normal

Tags: d-i

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#642159; Package debian-installer. (Mon, 19 Sep 2011 21:24:08 GMT) (full text, mbox, link).


Acknowledgement sent to Matt Weatherford <mbw@u.washington.edu>:
New Bug report received and forwarded. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Mon, 19 Sep 2011 21:24:09 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Matt Weatherford <mbw@u.washington.edu>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: debian-installer preseed broken with apt-cacher-ng mirror
Date: Mon, 19 Sep 2011 13:44:47 -0700
Package: debian-installer
Version: 20110106+squeeze3
Severity: normal
Tags: d-i




Copied from an Ubuntu bug report, since it resembles my problem as well:


In my preseed file I set "d-i mirror/http/proxy string http://apt-cacher:3142/", this is an apt-cacher-ng proxy. This works correctly for installing the system.

I expect "d-i mirror/http/proxy" to only apply to the debian installer http proxy (apt proxy), and all other http applications, e.g., wget, curl, lynx to not use that setting.

Instead when I run applications from the Kickstart %post section, all HTTP connections are proxied; curl, wget, lynx, hg (mercurial), everything. Once the system installs and reboots all is good, the only remanence of the mirror/http/proxy setting is in /etc/apt/apt.conf, which is fine (though I think that should be configurable too). mirror/http/proxy is setting the environment variable http_proxy.

Repo:
Set "d-i mirror/http/proxy string http://apt-cacher:3142/" in a preseed (use your local mirror).

Use a Kickstart post section like this:
%post --interpreter=/bin/bash
exec < /dev/tty3 > /dev/ttyS0
chvt 3
(
echo "## http_proxy = $http_proxy"
echo "## lynx dump head"
lynx -dump -head http://example.com
echo "## wget"
wget -O /root/test http://example.com
) 2>&1 | /usr/bin/tee /var/log/post_install.log
chvt 1

After install check /var/log/post_install.log and it will look like:
## http_proxy = http://apt-cacher:3142/
## lynx dump head
HTTP/1.1 403 Forbidden file type or location
Date: Thu Apr 22 16:29:10 2010
Server: Debian Apt-Cacher NG/0.4.6
## wget
--2010-04-22 09:29:10-- http://example.com
Resolving apt-cacher... 10.0.0.10
Connecting to apt-cacher|10.0.0.10|:3142... connected.
Proxy request sent, awaiting response... 403 Forbidden file type or location
2010-04-22 09:29:10 ERROR 403: Forbidden file type or location.

The workaround for me it to reset $http_proxy at the top of %post:
export http_proxy=""
Then all the applications that use http behave normally.

Suggestion:
Don't set $http_proxy with "d-i mirror/http/proxy", maybe set $apt_http_proxy if d-i needs an environment variable. For users that need a general proxy a setting like "d-i http/proxy" could be created.




-- System Information:
Debian Release: 6.0.2
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-5-686 (SMP w/16 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#642159; Package debian-installer. (Wed, 20 Feb 2013 17:48:03 GMT) (full text, mbox, link).


Acknowledgement sent to Alexander Perlis <aperlis@math.lsu.edu>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Wed, 20 Feb 2013 17:48:04 GMT) (full text, mbox, link).


Message #10 received at 642159@bugs.debian.org (full text, mbox, reply):

From: Alexander Perlis <aperlis@math.lsu.edu>
To: 642159@bugs.debian.org
Subject: Re: debian-installer preseed broken with apt-cacher-ng mirror
Date: Wed, 20 Feb 2013 11:34:50 -0600
This debian-installer bug is perhaps more serious than might seem from 
the original bug report. It affects preseed/run and preseed/include. Not 
clear how to adapt the work-around from the original bug report.

The bug also affects fetching in preseed/late_command, as well as 
subsequent attempts to re-fetch the original preseed file (which comes 
up if you have priority=low or some other installer error to bring up 
the installer menu, and as you proceed through the steps it will direct 
you to the preseed fetch menu option). Particularly confusing is that 
Ctrl-Alt-F4 shows a wget 403 error yet tcpdump on port 80 on the preseed 
server shows no activity, and furthermore if you go into BusyBox in the 
installer and do a manual wget there is no error.

I can't determine a work-around, which might for example be to tell 
apt-cacher-ng to serve up (and not cache) local files, but no luck so 
far. I considered replacing apt-cacher-ng with two copies of approx (one 
for debian, one for ubuntu), but approx seems to be a more apt specific 
cache that won't serve up arbitrary local files.

Consequently I avoid fetching additional preseed files entirely, with a 
growing "one-liner" script in preseed/late_command. It seems that much 
of the additional power of preseeding is unavailable right now, in the 
context of an apt proxy cache, until this debian-installer bug can be 
addressed.



Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#642159; Package debian-installer. (Wed, 20 Feb 2013 18:15:03 GMT) (full text, mbox, link).


Acknowledgement sent to afm 404 <afm404@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Wed, 20 Feb 2013 18:15:03 GMT) (full text, mbox, link).


Message #15 received at 642159@bugs.debian.org (full text, mbox, reply):

From: afm 404 <afm404@gmail.com>
To: 642159@bugs.debian.org
Subject: i do this (workaround not a fix)
Date: Wed, 20 Feb 2013 18:12:12 +0000
[Message part 1 (text/plain, inline)]
(had problems with proxy setting before, tho i can't remember what)

### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string 192.168.1.74:3142
d-i mirror/http/directory string /ftp.uk.debian.org/debian
#d-i mirror/http/proxy

ofc this leaves the proxy in /etc/apt/sources.list after install
but i didn't have a problem with that
you could just sed it out or something in the late_command
[Message part 2 (text/html, inline)]

Bug reassigned from package 'debian-installer' to 'apt-cacher-ng'. Request was from Holger Levsen <holger@layer-acht.org> to control@bugs.debian.org. (Sat, 20 Jul 2013 13:21:12 GMT) (full text, mbox, link).


No longer marked as found in versions debian-installer/20110106+squeeze3. Request was from Holger Levsen <holger@layer-acht.org> to control@bugs.debian.org. (Sat, 20 Jul 2013 13:21:12 GMT) (full text, mbox, link).


Added indication that 642159 affects debian-installer Request was from Holger Levsen <holger@layer-acht.org> to control@bugs.debian.org. (Sat, 20 Jul 2013 13:21:13 GMT) (full text, mbox, link).


Bug reassigned from package 'apt-cacher-ng' to 'debian-installer'. Request was from Eduard Bloch <edi@gmx.de> to control@bugs.debian.org. (Sun, 21 Jul 2013 09:12:08 GMT) (full text, mbox, link).


Bug reassigned from package 'debian-installer' to 'choose-mirror'. Request was from Holger Levsen <holger@layer-acht.org> to control@bugs.debian.org. (Sun, 21 Jul 2013 10:15:19 GMT) (full text, mbox, link).


Information forwarded to debian-bugs-dist@lists.debian.org, Debian Install System Team <debian-boot@lists.debian.org>:
Bug#642159; Package choose-mirror. (Tue, 10 Dec 2013 18:24:04 GMT) (full text, mbox, link).


Acknowledgement sent to Javier Lopez <chilicuil@ubuntu.com>:
Extra info received and forwarded to list. Copy sent to Debian Install System Team <debian-boot@lists.debian.org>. (Tue, 10 Dec 2013 18:24:04 GMT) (full text, mbox, link).


Message #30 received at 642159@bugs.debian.org (full text, mbox, reply):

From: Javier Lopez <chilicuil@ubuntu.com>
To: 642159@bugs.debian.org
Subject: Do we need a separate mirror/apt/proxy option?
Date: Tue, 10 Dec 2013 12:20:07 -0600
I've dug a little bit in the code and agree with Matt that probably we require
and extra option (or even more if we want to control when to leave the mirror in
/etc/apt/apt.conf), it seems for me that http_proxy (mirror/http/proxy) is being
abused. It's set|retrieve several times during installation, and used for
several purposes.

/usr/lib/fetch-url/http (debian-installer-utils)
  23    # use the proxy for wgets (should speed things up)
  24    if db_get mirror/$proto/proxy; then
  25            export ${proto}_proxy="$RET"
  26    fi

/bin/choose-mirror (choose-mirror.c, choose-mirror)
  656 static int set_proxy(void) {   
  657   char *px = add_protocol("proxy");
  658   char *proxy_var;       
  659 
  660   proxy_var = xasprintf("%s_proxy", protocol);
  661 
  662   debconf_get(debconf, px);
  663   if (debconf->value != NULL && strlen(debconf->value)) {
  664           if (strstr(debconf->value, "://")) {
  665                   setenv(proxy_var, debconf->value, 1);

/usr/lib/apt-setup/generators/50mirror (apt-setup)
  225  if [ "$protocol" = http ]; then
  226     db_get mirror/$protocol/proxy
  227     proxy="$RET"
  228     if [ -n "$proxy" ]; then
  229         if ! grep -iq "Acquire::$protocol::Proxy" $ROOT/etc/apt/apt.conf.new; then
  230            echo "Acquire::$protocol::Proxy \"$proxy\";" >> $ROOT/etc/apt/apt.conf.new

This make the installer to fail on several situations, let's suppose
mirror/http/proxy is set on a PXE APPEND environment this way:

APPEND DEBCONF_DEBUG=5 priority=critical locale=en_US interface=auto mirror/http/proxy=http://10.99.88.1:3142 url=http://domain.com/file.preseed initrd=initrd.gz

The installer will fail while fetching http://domain.com/file.preseed. This is
because /usr/lib/fetch-url/http set 'export http_proxy=http://10.99.88.1:3142
before retrieving the preseed file.

Now, if export ${proto}_proxy="$RET" is commented on /usr/lib/fetch-url/http,
the installer will go one step further, it'll retrieve the preseed file, but
still will fail when downloading other files on preseed/run, preseed/include and
preseed/late_command. This is due to /bin/choose-mirror setting again http_proxy
on setenv(proxy_var, debconf->value, 1);

Finally, if that line is commented as well, the installer will finish
successfully while still using the mirror because apt-setup use the variable to
configure apt-get before base-installer get into the action. The proxy will also
be available on next boot after installation.

I think this can be fixed, if we add a mirror/apt/proxy debconf variable and use
it on /usr/lib/apt-setup/generators/50mirror while leaving choose-mirror and
/usr/lib/fetch-url/http untouched. Then people could use mirror/http/proxy to
control connections for general applications and mirror/apt/proxy for
controlling debian repositories.



Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Tue Mar 19 02:47:42 2024; Machine Name: buxtehude

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.