Project

General

Profile

Bug #9029

Proxy authentication is not working for HTTPS

Added by Jim Pingle 4 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Category:
Operating System
Target version:
Start date:
10/09/2018
Due date:
% Done:

100%

Estimated time:
Affected Version:
2.4.x
Affected Architecture:
All

Description

When a user enters proxy details on system_advanced_misc.php it should trigger the firewall to use the proxy for things like pkg. This seems to work for HTTP URLs, but not HTTPS.

We are setting up the environment properly as far as I can see. Packet captures show the request going to the proxy, the proxy responds that authentication is required, but the client never sends the request with authentication details.

: set | grep http
http_proxy    198.51.100.23:3128
http_proxy_auth    basic:*:jimp:jimp
http_proxy_auth_pass    jimp
http_proxy_auth_user    jimp
http_proxy_port    3128
: env | grep HTTP
HTTP_PROXY=198.51.100.23:3128
HTTP_PROXY_AUTH=basic:*:jimp:jimp

Even on 2.4.5 snapshots this fails with an error such as:

pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-core/packagesite.txz: Proxy Authentication Required

As a quick test, a fetch of an HTTPS URL also fails, but HTTP succeeds:

: fetch -o - https://files00.netgate.com/
fetch: https://files00.netgate.com/: Proxy Authentication Required
: fetch -o - http://files00.netgate.com/
[...]
-                                                     4872  B   14 kBps 00m00s

Last time this came up in #6949 there was a FreeBSD bug referenced at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194483 that may or may not be related.

It was working at one time before 2.4-RELEASE but broke again at some point.

This also appears to affect stock FreeBSD 11.2, easy to replicate with fetch

env HTTP_PROXY="198.51.100.23:3128" HTTP_PROXY_AUTH="basic:*:jimp:jimp" fetch -o - https://files00.netgate.com/

Just substitute in a proxy IP address/user/pass that you can reach which is set to require auth.

Annoying but not critical, can be moved ahead if fixing is prohibitively difficult, but we did have it fixed before, so maybe a patch or two were not carried forward.

History

#1 Updated by Steve Beaver 3 months ago

  • Assignee set to Renato Botelho

#2 Updated by Renato Botelho 2 months ago

  • Status changed from New to In Progress

#3 Updated by Renato Botelho 2 months ago

  • Status changed from In Progress to Feedback
  • % Done changed from 0 to 100

#4 Updated by Vladimir Lind 2 months ago

On 2.4.5-DEVELOPMENT (amd64) built on Tue Nov 20 18:52:24 EST 2018:

: set | grep http
http_proxy 10.1.1.1:3128
http_proxy_auth basic:*:Lind:lll
http_proxy_auth_pass lll
http_proxy_auth_user Lind
http_proxy_port 3128

I have no live proxy configured but these commands don't give me "proxy auth required" message what is probably supposed to be:

: pkg install acme
Updating pfSense-core repository catalogue...
^C
: fetch -o - https://files00.netgate.com/
^Cfetch: transfer interrupted

: fetch -o - http://files00.netgate.com/

#5 Updated by Jim Pingle about 2 months ago

  • Status changed from Feedback to This Sprint

Fetch works at the command line but pkg still does not.

: fetch -o - https://files00.netgate.com/
<html>
<head><title>Index of /</title></head>
<body bgcolor="white">
[...]
: pkg update -f
Updating pfSense-core repository catalogue...
pkg: Repository pfSense-core load error: access repo file(/var/db/pkg/repo-pfSense-core.sqlite) failed: No such file or directory
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-core/meta.txz: Proxy Authentication Required
repository pfSense-core has no meta file, using default settings
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-core/packagesite.txz: Proxy Authentication Required
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg: Repository pfSense load error: access repo file(/var/db/pkg/repo-pfSense.sqlite) failed: No such file or directory
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-pfSense_devel/meta.txz: Proxy Authentication Required
repository pfSense has no meta file, using default settings
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-pfSense_devel/packagesite.txz: Proxy Authentication Required
Unable to update repository pfSense
Error updating repositories!

The HTTP proxy variables are not present in the pkg environment

: pkg -vv | grep -i http
VULNXML_SITE = "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2";
HTTP_USER_AGENT = "pkg/1.10.5";
    "pkg+http",
    "pkg+https",
    "https",
    "http",
    url             : "pkg+https://beta.pfsense.org/packages/pfSense_master_amd64-core",
    url             : "pkg+https://beta.pfsense.org/packages/pfSense_master_amd64-pfSense_devel",

It also fails to show a package list in the GUI.

The output of `pkg_env()` in PHP appears to include the proxy info as expected, however:

array(7) {
  ["LANG"]=>
  string(1) "C" 
  ["HTTP_USER_AGENT"]=>
  string(46) "pfSense/2.4.5-DEVELOPMENT:8ca57b876e3cb6b3a8e5" 
  ["ASSUME_ALWAYS_YES"]=>
  string(4) "true" 
  ["FETCH_TIMEOUT"]=>
  int(5)
  ["FETCH_RETRY"]=>
  int(2)
  ["HTTP_PROXY"]=>
  string(18) "198.51.100.23:3128" 
  ["HTTP_PROXY_AUTH"]=>
  string(17) "basic:*:jimp:jimp" 
}

#6 Updated by Renato Botelho about 2 months ago

  • Status changed from This Sprint to Feedback

I've improted the same patch to pkg's internal libfetch. It'll be available on pkg-1.10.5_6

#7 Updated by Jim Pingle about 2 months ago

  • Status changed from Feedback to Resolved

Works great now!

Also available in: Atom PDF