Project

General

Profile

Actions

Bug #9029

closed

Proxy authentication is not working for HTTPS

Added by Jim Pingle about 6 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Category:
Operating System
Target version:
Start date:
10/09/2018
Due date:
% Done:

100%

Estimated time:
Plus Target Version:
Release Notes:
Affected Version:
2.4.x
Affected Architecture:
All

Description

When a user enters proxy details on system_advanced_misc.php it should trigger the firewall to use the proxy for things like pkg. This seems to work for HTTP URLs, but not HTTPS.

We are setting up the environment properly as far as I can see. Packet captures show the request going to the proxy, the proxy responds that authentication is required, but the client never sends the request with authentication details.

: set | grep http
http_proxy    198.51.100.23:3128
http_proxy_auth    basic:*:jimp:jimp
http_proxy_auth_pass    jimp
http_proxy_auth_user    jimp
http_proxy_port    3128
: env | grep HTTP
HTTP_PROXY=198.51.100.23:3128
HTTP_PROXY_AUTH=basic:*:jimp:jimp

Even on 2.4.5 snapshots this fails with an error such as:

pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-core/packagesite.txz: Proxy Authentication Required

As a quick test, a fetch of an HTTPS URL also fails, but HTTP succeeds:

: fetch -o - https://files00.netgate.com/
fetch: https://files00.netgate.com/: Proxy Authentication Required
: fetch -o - http://files00.netgate.com/
[...]
-                                                     4872  B   14 kBps 00m00s

Last time this came up in #6949 there was a FreeBSD bug referenced at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194483 that may or may not be related.

It was working at one time before 2.4-RELEASE but broke again at some point.

This also appears to affect stock FreeBSD 11.2, easy to replicate with fetch

env HTTP_PROXY="198.51.100.23:3128" HTTP_PROXY_AUTH="basic:*:jimp:jimp" fetch -o - https://files00.netgate.com/

Just substitute in a proxy IP address/user/pass that you can reach which is set to require auth.

Annoying but not critical, can be moved ahead if fixing is prohibitively difficult, but we did have it fixed before, so maybe a patch or two were not carried forward.

Actions #1

Updated by Anonymous about 6 years ago

  • Assignee set to Renato Botelho
Actions #2

Updated by Renato Botelho about 6 years ago

  • Status changed from New to In Progress
Actions #3

Updated by Renato Botelho about 6 years ago

  • Status changed from In Progress to Feedback
  • % Done changed from 0 to 100
Actions #4

Updated by Vladimir Lind about 6 years ago

On 2.4.5-DEVELOPMENT (amd64) built on Tue Nov 20 18:52:24 EST 2018:

: set | grep http
http_proxy 10.1.1.1:3128
http_proxy_auth basic:*:Lind:lll
http_proxy_auth_pass lll
http_proxy_auth_user Lind
http_proxy_port 3128

I have no live proxy configured but these commands don't give me "proxy auth required" message what is probably supposed to be:

: pkg install acme
Updating pfSense-core repository catalogue...
^C
: fetch -o - https://files00.netgate.com/
^Cfetch: transfer interrupted

: fetch -o - http://files00.netgate.com/

Actions #5

Updated by Jim Pingle about 6 years ago

  • Status changed from Feedback to 13

Fetch works at the command line but pkg still does not.

: fetch -o - https://files00.netgate.com/
<html>
<head><title>Index of /</title></head>
<body bgcolor="white">
[...]
: pkg update -f
Updating pfSense-core repository catalogue...
pkg: Repository pfSense-core load error: access repo file(/var/db/pkg/repo-pfSense-core.sqlite) failed: No such file or directory
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-core/meta.txz: Proxy Authentication Required
repository pfSense-core has no meta file, using default settings
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-core/packagesite.txz: Proxy Authentication Required
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg: Repository pfSense load error: access repo file(/var/db/pkg/repo-pfSense.sqlite) failed: No such file or directory
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-pfSense_devel/meta.txz: Proxy Authentication Required
repository pfSense has no meta file, using default settings
pkg: https://beta.pfsense.org/packages/pfSense_master_amd64-pfSense_devel/packagesite.txz: Proxy Authentication Required
Unable to update repository pfSense
Error updating repositories!

The HTTP proxy variables are not present in the pkg environment

: pkg -vv | grep -i http
VULNXML_SITE = "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2";
HTTP_USER_AGENT = "pkg/1.10.5";
    "pkg+http",
    "pkg+https",
    "https",
    "http",
    url             : "pkg+https://beta.pfsense.org/packages/pfSense_master_amd64-core",
    url             : "pkg+https://beta.pfsense.org/packages/pfSense_master_amd64-pfSense_devel",

It also fails to show a package list in the GUI.

The output of `pkg_env()` in PHP appears to include the proxy info as expected, however:

array(7) {
  ["LANG"]=>
  string(1) "C" 
  ["HTTP_USER_AGENT"]=>
  string(46) "pfSense/2.4.5-DEVELOPMENT:8ca57b876e3cb6b3a8e5" 
  ["ASSUME_ALWAYS_YES"]=>
  string(4) "true" 
  ["FETCH_TIMEOUT"]=>
  int(5)
  ["FETCH_RETRY"]=>
  int(2)
  ["HTTP_PROXY"]=>
  string(18) "198.51.100.23:3128" 
  ["HTTP_PROXY_AUTH"]=>
  string(17) "basic:*:jimp:jimp" 
}
Actions #6

Updated by Renato Botelho about 6 years ago

  • Status changed from 13 to Feedback

I've improted the same patch to pkg's internal libfetch. It'll be available on pkg-1.10.5_6

Actions #7

Updated by Jim Pingle about 6 years ago

  • Status changed from Feedback to Resolved

Works great now!

Actions #8

Updated by Alex D almost 4 years ago

I have a fresh pfSense 2.4.5 installation here, same problem with pkg.
Fetch command works but pkg doesn't.

pkg -vv shows no proxy variable but it seems that the requests are sent to the proxy. However they are not authenticated.

Do I need to open an additional issue ?

Actions #9

Updated by Jim Pingle almost 4 years ago

Post on the forum first to diagnose your issue.

Actions

Also available in: Atom PDF