[PATCH] emerge --getbinpkg: https support for If-Modified-Since

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] emerge --getbinpkg: https support for If-Modified-Since

Zac Medico-2
When https certificate and hostname verification is enabled for
stdlib http clients (PEP 476), use python for If-Modified-Since
header support. When python lacks PEP 476 support, continue to
use FETCHCOMMAND for https certificate and hostname verification
(see security bug 469888).

X-Gentoo-bug: 625246
X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=625246
---
 pym/portage/dbapi/bintree.py | 10 ++++++----
 pym/portage/util/_urlopen.py | 12 ++++++++++++
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/pym/portage/dbapi/bintree.py b/pym/portage/dbapi/bintree.py
index c833968c2..95bd5dbf8 100644
--- a/pym/portage/dbapi/bintree.py
+++ b/pym/portage/dbapi/bintree.py
@@ -18,7 +18,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
  'portage.util:atomic_ofstream,ensure_dirs,normalize_path,' + \
  'writemsg,writemsg_stdout',
  'portage.util.path:first_existing',
- 'portage.util._urlopen:urlopen@_urlopen',
+ 'portage.util._urlopen:urlopen@_urlopen,have_pep_476@_have_pep_476',
  'portage.versions:best,catpkgsplit,catsplit,_pkg_str',
 )
 
@@ -851,9 +851,9 @@ class binarytree(object):
  download_timestamp + ttl > time.time():
  raise UseCachedCopyOfRemoteIndex()
 
- # Don't use urlopen for https, since it doesn't support
- # certificate/hostname verification (bug #469888).
- if parsed_url.scheme not in ('https',):
+ # Don't use urlopen for https, unless
+ # PEP 476 is supported (bug #469888).
+ if parsed_url.scheme not in ('https',) or _have_pep_476():
  try:
  f = _urlopen(url, if_modified_since=local_timestamp)
  if hasattr(f, 'headers') and f.headers.get('timestamp', ''):
@@ -965,6 +965,8 @@ class binarytree(object):
  "\n")
  rmt_idx = pkgindex
  except EnvironmentError as e:
+ # This includes URLError which is raised for SSL
+ # certificate errors when PEP 476 is supported.
  writemsg(_("\n\n!!! Error fetching binhost package" \
  " info from '%s'\n") % _hide_url_passwd(base_url))
  # With Python 2, the EnvironmentError message may
diff --git a/pym/portage/util/_urlopen.py b/pym/portage/util/_urlopen.py
index 4cfe183b1..fc9db74a0 100644
--- a/pym/portage/util/_urlopen.py
+++ b/pym/portage/util/_urlopen.py
@@ -26,6 +26,18 @@ if sys.hexversion >= 0x3000000:
 #  and the file-'mtime'
 TIMESTAMP_TOLERANCE = 5
 
+
+def have_pep_476():
+ """
+ Test whether ssl certificate verification is enabled by default for
+ stdlib http clients (PEP 476).
+
+ @returns: bool, True if ssl certificate verification is enabled by
+ default
+ """
+ return hasattr(__import__('ssl'), '_create_unverified_context')
+
+
 def urlopen(url, if_modified_since=None):
  parse_result = urllib_parse.urlparse(url)
  if parse_result.scheme not in ("http", "https"):
--
2.13.0


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] emerge --getbinpkg: https support for If-Modified-Since

Brian Dolbec-3
On Mon, 31 Jul 2017 00:11:09 -0700
Zac Medico <[hidden email]> wrote:

> When https certificate and hostname verification is enabled for
> stdlib http clients (PEP 476), use python for If-Modified-Since
> header support. When python lacks PEP 476 support, continue to
> use FETCHCOMMAND for https certificate and hostname verification
> (see security bug 469888).
>
> X-Gentoo-bug: 625246
> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=625246
> ---
>  pym/portage/dbapi/bintree.py | 10 ++++++----
>  pym/portage/util/_urlopen.py | 12 ++++++++++++
>  2 files changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/pym/portage/dbapi/bintree.py
> b/pym/portage/dbapi/bintree.py index c833968c2..95bd5dbf8 100644
> --- a/pym/portage/dbapi/bintree.py
> +++ b/pym/portage/dbapi/bintree.py
> @@ -18,7 +18,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
>   'portage.util:atomic_ofstream,ensure_dirs,normalize_path,' +
> \ 'writemsg,writemsg_stdout',
>   'portage.util.path:first_existing',
> - 'portage.util._urlopen:urlopen@_urlopen',
> + 'portage.util._urlopen:urlopen@_urlopen,have_pep_476@_have_pep_476',
>   'portage.versions:best,catpkgsplit,catsplit,_pkg_str',
>  )
>  
> @@ -851,9 +851,9 @@ class binarytree(object):
>   download_timestamp +
> ttl > time.time(): raise UseCachedCopyOfRemoteIndex()
>  
> - # Don't use urlopen for https, since
> it doesn't support
> - # certificate/hostname verification
> (bug #469888).
> - if parsed_url.scheme not in
> ('https',):
> + # Don't use urlopen for https, unless
> + # PEP 476 is supported (bug #469888).
> + if parsed_url.scheme not in
> ('https',) or _have_pep_476(): try:
>   f = _urlopen(url,
> if_modified_since=local_timestamp) if hasattr(f, 'headers') and
> f.headers.get('timestamp', ''): @@ -965,6 +965,8 @@ class
> binarytree(object): "\n")
>   rmt_idx = pkgindex
>   except EnvironmentError as e:
> + # This includes URLError which is
> raised for SSL
> + # certificate errors when PEP 476 is
> supported. writemsg(_("\n\n!!! Error fetching binhost package" \
>   " info from '%s'\n") %
> _hide_url_passwd(base_url)) # With Python 2, the EnvironmentError
> message may diff --git a/pym/portage/util/_urlopen.py
> b/pym/portage/util/_urlopen.py index 4cfe183b1..fc9db74a0 100644
> --- a/pym/portage/util/_urlopen.py
> +++ b/pym/portage/util/_urlopen.py
> @@ -26,6 +26,18 @@ if sys.hexversion >= 0x3000000:
>  #  and the file-'mtime'
>  TIMESTAMP_TOLERANCE = 5
>  
> +
> +def have_pep_476():
> + """
> + Test whether ssl certificate verification is enabled by
> default for
> + stdlib http clients (PEP 476).
> +
> + @returns: bool, True if ssl certificate verification is
> enabled by
> + default
> + """
> + return hasattr(__import__('ssl'),
> '_create_unverified_context') +
> +
>  def urlopen(url, if_modified_since=None):
>   parse_result = urllib_parse.urlparse(url)
>   if parse_result.scheme not in ("http", "https"):

looks fine

--
Brian Dolbec <dolsen>


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] emerge --getbinpkg: https support for If-Modified-Since

Zac Medico-2
On Mon, Jul 31, 2017 at 8:10 AM, Brian Dolbec <[hidden email]> wrote:

> On Mon, 31 Jul 2017 00:11:09 -0700
> Zac Medico <[hidden email]> wrote:
>
>> When https certificate and hostname verification is enabled for
>> stdlib http clients (PEP 476), use python for If-Modified-Since
>> header support. When python lacks PEP 476 support, continue to
>> use FETCHCOMMAND for https certificate and hostname verification
>> (see security bug 469888).
>>
>> X-Gentoo-bug: 625246
>> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=625246
>> ---
>>  pym/portage/dbapi/bintree.py | 10 ++++++----
>>  pym/portage/util/_urlopen.py | 12 ++++++++++++
>>  2 files changed, 18 insertions(+), 4 deletions(-)
>>
>
> looks fine

Thanks, pushed:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=1d821469d6b72ce051b02908f17302c500945788

--
Thanks,
Zac