libressl status

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

libressl status

Paul B. Henson
What is the current status/thoughts regarding libressl? Reviewing the
bug and some past threads, it sounds like the initial plan was to make
openssl a virtual and let either classic openssl or libressl fulfull it?
I'm not sure if things have changed from that viewpoint, but it really
doesn't seem they're going to be plug and play compatible 8-/. libressl
offers functionality openssl doesn't and vice versa, and playing nicely
with each other doesn't seem to be on the agenda of either. It seems it
might make more sense to treat them more like openssl and gnutls, where
they both provide similar ssl functionality but a given package might
use one, the other, or either?

The specific reason for my current inquiry is that the latest openntpd
release includes the new support from openbsd for "constraints", where
basically you can verify ntp time sources by checking their time
relative to a trusted TLS server (which provides the time in HTTP
headers). This functionality requires libtls, part of libressl. openssl
provides no compatible functionality, so this is a case where they're
not plug-and-play, openntpd requires libressl specifically.

Thanks...

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Hanno Böck-3
On Thu, 2 Apr 2015 16:49:20 -0700
"Paul B. Henson" <[hidden email]> wrote:

> What is the current status/thoughts regarding libressl? Reviewing the
> bug and some past threads, it sounds like the initial plan was to make
> openssl a virtual and let either classic openssl or libressl fulfull
> it? I'm not sure if things have changed from that viewpoint, but it
> really doesn't seem they're going to be plug and play compatible 8-/.
> libressl offers functionality openssl doesn't and vice versa, and
> playing nicely with each other doesn't seem to be on the agenda of
> either.

The latest state is that there is an overlay, but making the portage
tree compatible with libressl is not that trivial.

A large number of core packages are upstream-incompatible with
libressl. Most of them are actually programming languages (python, php,
ruby) that contain bindings to functions libressl has removed.
This could be fixed by the upstreams with some ifdefs, but right now
you can't just switch out libressl.


> It seems it might make more sense to treat them more like
> openssl and gnutls, where they both provide similar ssl functionality
> but a given package might use one, the other, or either?

Tricky thing here, because then you'd need to rename the libs. E.g.
libssl to liblibressl or something.
But then every program with a build environment to link to libssl would
first have to be patched to link to our specialized libressl variant.


> The specific reason for my current inquiry is that the latest openntpd
> release includes the new support from openbsd for "constraints", where
> basically you can verify ntp time sources by checking their time
> relative to a trusted TLS server (which provides the time in HTTP
> headers). This functionality requires libtls, part of libressl.
> openssl provides no compatible functionality, so this is a case where
> they're not plug-and-play, openntpd requires libressl specifically.

I'm eager to use that, too, and was disappointed to read it requires
libressl :-)
Is there a way to split libtls off libressl? Because that might be at
least for this case an option: Continue to use openssl, but have libtls
laying around. Not sure if it is possible to have libtls using
libcrypt/libssl functions from openssl.


--
Hanno Böck
http://hboeck.de/

mail/jabber: [hidden email]
GPG: BBB51E42

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

hasufell
In reply to this post by Paul B. Henson
On 04/03/2015 01:49 AM, Paul B. Henson wrote:
> What is the current status/thoughts regarding libressl? Reviewing the
> bug and some past threads, it sounds like the initial plan was to make
> openssl a virtual and let either classic openssl or libressl fulfull it?

Not anymore. We will go for "libressl" USE flag for the same reason
there is a "libav" USE flag now (working subslots etc).

> I'm not sure if things have changed from that viewpoint, but it really
> doesn't seem they're going to be plug and play compatible 8-/. libressl
> offers functionality openssl doesn't and vice versa, and playing nicely
> with each other doesn't seem to be on the agenda of either. It seems it
> might make more sense to treat them more like openssl and gnutls, where
> they both provide similar ssl functionality but a given package might
> use one, the other, or either?
>

Renaming library file names is a no-go, imo. Same story with symlink
hacks via eselect.

> The specific reason for my current inquiry is that the latest openntpd
> release includes the new support from openbsd for "constraints", where
> basically you can verify ntp time sources by checking their time
> relative to a trusted TLS server (which provides the time in HTTP
> headers). This functionality requires libtls, part of libressl. openssl
> provides no compatible functionality, so this is a case where they're
> not plug-and-play, openntpd requires libressl specifically.
>

Well, since openntpd is developed by BSD guys, no wonder about that
decision... I guess you could still try to provide a compatibility patch
for openssl.

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

hasufell
In reply to this post by Paul B. Henson
On 04/03/2015 01:49 AM, Paul B. Henson wrote:
>
> The specific reason for my current inquiry is that the latest openntpd
> release includes the new support from openbsd for "constraints", where
> basically you can verify ntp time sources by checking their time
> relative to a trusted TLS server (which provides the time in HTTP
> headers). This functionality requires libtls, part of libressl. openssl
> provides no compatible functionality, so this is a case where they're
> not plug-and-play, openntpd requires libressl specifically.
>

Also, feel free to provide a pull request for the current
openssl-incompatible openntpd at
https://github.com/gentoo/libressl

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Paul B. Henson
In reply to this post by Hanno Böck-3
On Fri, Apr 03, 2015 at 01:59:25AM +0200, Hanno Böck wrote:

> Tricky thing here, because then you'd need to rename the libs. E.g.
> libssl to liblibressl or something.
> But then every program with a build environment to link to libssl would
> first have to be patched to link to our specialized libressl variant.

Yah, I dunno. But I don't have a warm fuzzy about them being
interchangable any time soon or even longterm. It's not a goal AFAIK.

> Is there a way to split libtls off libressl? Because that might be at
> least for this case an option: Continue to use openssl, but have libtls
> laying around. Not sure if it is possible to have libtls using
> libcrypt/libssl functions from openssl.

I'm pretty sure libtls won't currently compile against openssl, although
I haven't taken a detailed look as to why. It is true that openntpd has
no direct dependency on libressl, only the libtls API, so theoretically
if libressl's libtls could be patched to work with openssl or if openssl
released their own API compatible libtls it would be happy.

I asked a similar question on the pkgsrc mailing list:

http://mail-index.netbsd.org/tech-pkg/2015/03/30/msg014532.html

They're pretty much decided on allowing both openssl and libressl to be
installed concurrently and for a given application to use one or the
other. The specific method for that packaging system is what they call a
prefix; basically instead of /usr/pkg/lib/libssl it would be
/usr/pkg/libressl/lib/libssl, and packages that needed it would get the
right magic flags for the headers and libraries to be found.

All openntpd does is use libtls to make an HTTPS HEAD request. It might
be simpler to just have it use libcurl or some other existing https
library instead of trying to get libressl/libtls working, although that
would decrease the "security" aspect of it only using openbsd audited code.


Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Paul B. Henson
In reply to this post by hasufell
On Fri, Apr 03, 2015 at 01:31:53PM +0200, hasufell wrote:

> Not anymore. We will go for "libressl" USE flag for the same reason
> there is a "libav" USE flag now (working subslots etc).

Um, ok. That still only allows one or the other to be installed though,
right? So if you want a package that only works with openssl and one
that only works with libressl, you are left wanting :)?

> decision... I guess you could still try to provide a compatibility patch
> for openssl.

I mentioned that to Brent Cook, who is currently maintaining the
portable version, and I don't think that's something he'd pull in
upstream. I suppose we could have a local gentoo patch.

I guess I'll just let this simmer for now and see how things develop. My
preference (I think, at least at the moment) would be for both
implementations to be able to coexist, like openssl and gnutls. It looks
like that's the way it's heading in pkgsrc (the other place I'm
maintaining openntpd), which should make things relatively simple there.
If that's not going to be an option with Gentoo hopefully the best
alternative will become clearer at some point ;).

Thanks...


Reply | Threaded
Open this post in threaded view
|

Re: libressl status

hasufell
On 04/05/2015 06:44 AM, Paul B. Henson wrote:
> On Fri, Apr 03, 2015 at 01:31:53PM +0200, hasufell wrote:
>
>> Not anymore. We will go for "libressl" USE flag for the same reason
>> there is a "libav" USE flag now (working subslots etc).
>
> Um, ok. That still only allows one or the other to be installed though,
> right? So if you want a package that only works with openssl and one
> that only works with libressl, you are left wanting :)?
>

Yep. Your only solution is to try something really hackish like NixOS.
They would allow any such combination. But it comes with a price.

Or you do it manually outside of the PM.

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Rich Freeman
In reply to this post by Paul B. Henson
On Sun, Apr 5, 2015 at 12:34 AM, Paul B. Henson <[hidden email]> wrote:
>
> They're pretty much decided on allowing both openssl and libressl to be
> installed concurrently and for a given application to use one or the
> other. The specific method for that packaging system is what they call a
> prefix; basically instead of /usr/pkg/lib/libssl it would be
> /usr/pkg/libressl/lib/libssl, and packages that needed it would get the
> right magic flags for the headers and libraries to be found.
>

WTF.

If you're going to fork a library, and don't intend to keep the
packages API-compatible, then change the filenames.  What is so hard
about this?  LIbressl was even an outside fork, so it didn't come with
any of the baggage of "we're the real libssl team" or whatever.

Sure, we can do the USE=libressl route like we did with libav, but
since this is still new would it make more sense to just rename the
libressl files so that they can still go in /usr/lib but without being
called libssl?  Then any package that wants to use them will need to
have their build logic changed accordingly.  They aren't drop-in
replacements for each other anyway, as much as people would wish they
were, so we should resist the urge to pretend that they are.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

hasufell
On 04/05/2015 01:17 PM, Rich Freeman wrote:

> If you're going to fork a library, and don't intend to keep the
> packages API-compatible, then change the filenames.  What is so hard
> about this?  LIbressl was even an outside fork, so it didn't come with
> any of the baggage of "we're the real libssl team" or whatever.

Libressl is (widely) API compatible with openssl. As said numerous times
before: they broke the API when they thought it is security relevant.

This is about the fact that they _added_ features which are not present
in openssl. However, openntpd still compiles with openssl.

>
> Sure, we can do the USE=libressl route like we did with libav, but
> since this is still new would it make more sense to just rename the
> libressl files so that they can still go in /usr/lib but without being
> called libssl?  Then any package that wants to use them will need to
> have their build logic changed accordingly.  They aren't drop-in
> replacements for each other anyway, as much as people would wish they
> were, so we should resist the urge to pretend that they are.
>

By that you are effectively forking libressl and causing a huge mess
downstream for both developers and users. It may even cause cross-distro
breakage. I can name several examples of this happening due to debian
going it's own way. Last of which affected openclonk (upstream merged a
patch from a debian dev who expected any distro does the same hacks they
do).

So please, have a little sense for QA. Those ideas are just making it
worse. This is something that has to be resolved upstream. If they don't
cooperate long-term, then their fork will just die out for sure (and for
good). However, I currently don't see strong signs for that.

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Diego Elio Pettenò-2
In reply to this post by Paul B. Henson

On 5 April 2015 at 05:44, Paul B. Henson <[hidden email]> wrote:
> I guess I'll just let this simmer for now and see how things develop. My
> preference (I think, at least at the moment) would be for both
> implementations to be able to coexist, like openssl and gnutls. It looks
> like that's the way it's heading in pkgsrc (the other place I'm
> maintaining openntpd), which should make things relatively simple there.
> If that's not going to be an option with Gentoo hopefully the best
> alternative will become clearer at some point ;).


The problem with that is that now you have to make sure that transitive dependencies are still functional.

Since as you point out the two packages are vastly API compatible, it makes them ABI incompatible and conflicting. The functions can have the same name, and vastly the same parameters, but they may be using different size for data, for instance. I pointed this out last year[1][2] already.

Symbol collision is a nasty problem because it's almost invisible as long as the API/ABI is close enough, but for libraries like OpenSSL/LibreSSL, this is a huge security risk, too.
Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Rich Freeman
On Sun, Apr 5, 2015 at 8:23 AM, Diego Elio Pettenò
<[hidden email]> wrote:
>
> Since as you point out the two packages are vastly API compatible, it makes
> them ABI incompatible and conflicting.

++

If they really want to improve the security of function calls that
they consider inherently secure, they should just introduce new
functions and deprecate the old ones, or make old ones wrappers around
new ones if that is appropriate.  If they go with the deprecation
route then they need to avoid using the same SONAMEs, and if they go
the wrapper route they need to make sure that they're compatible
across SONAMEs.  Of course, any re-implementation could have bugs, but
the key is that upstream has to recognize these incompatibilities as
being valid bugs that they intend to fix.  Right now if something
designed to link against openssl breaks against libressl there is a
good chance that libressl will just say it is intentional, which then
leaves us in the position of having to deal with the mess.

I agree that renaming things isn't a great solution either, and that
this really needs to be fixed upstream.  However, I don't really like
it going into the tree the way it is, because sooner or later we're
going to end up with blockers or bugs.  Either you can't install foo
with barssl, or you can, and it just might break but nobody really
keeps track of that.

It really doesn't matter which is the better implementation - this is
just a really messy way to do a transition.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

hasufell
On 04/05/2015 03:25 PM, Rich Freeman wrote:

> On Sun, Apr 5, 2015 at 8:23 AM, Diego Elio Pettenò
> <[hidden email]> wrote:
>>
>> Since as you point out the two packages are vastly API compatible, it makes
>> them ABI incompatible and conflicting.
>
> ++
>
> If they really want to improve the security of function calls that
> they consider inherently secure, they should just introduce new
> functions and deprecate the old ones, or make old ones wrappers around
> new ones if that is appropriate.  If they go with the deprecation
> route then they need to avoid using the same SONAMEs, and if they go
> the wrapper route they need to make sure that they're compatible
> across SONAMEs.  Of course, any re-implementation could have bugs, but
> the key is that upstream has to recognize these incompatibilities as
> being valid bugs that they intend to fix.  Right now if something
> designed to link against openssl breaks against libressl there is a
> good chance that libressl will just say it is intentional, which then
> leaves us in the position of having to deal with the mess.
>
> I agree that renaming things isn't a great solution either, and that
> this really needs to be fixed upstream.  However, I don't really like
> it going into the tree the way it is, because sooner or later we're
> going to end up with blockers or bugs.  Either you can't install foo
> with barssl, or you can, and it just might break but nobody really
> keeps track of that.
>
> It really doesn't matter which is the better implementation - this is
> just a really messy way to do a transition.
>

You are ranting at the wrong place. If you want to make a difference,
take this to the openbsd mailing lists.

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Rich Freeman
On Sun, Apr 5, 2015 at 2:35 PM, hasufell <[hidden email]> wrote:
>
> You are ranting at the wrong place. If you want to make a difference,
> take this to the openbsd mailing lists.
>

It seems unlikely that this would make much of a difference.  I think
that allowing this package to create another ffmpeg vs libav mess is a
mistake.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Diego Elio Pettenò-2
On 5 April 2015 at 19:59, Rich Freeman <[hidden email]> wrote:
> It seems unlikely that this would make much of a difference.  I think
> that allowing this package to create another ffmpeg vs libav mess is a
> mistake.

To be honest, the upstream developers should be fairly in the known
regarding my opinion expressed in the blog posts, as they replied and
read them before.

We could also find other ways to deal with this by installing the
packages as different library names and subverting pkg-config to
choose between the two, but that would only work for packages using
pkg-config to begin with and does not solve the transitive
dependencies problem.

Diego Elio Pettenò — Flameeyes
https://blog.flameeyes.eu/

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

hasufell
In reply to this post by Rich Freeman
On 04/05/2015 08:59 PM, Rich Freeman wrote:
> On Sun, Apr 5, 2015 at 2:35 PM, hasufell <[hidden email]> wrote:
>>
>> You are ranting at the wrong place. If you want to make a difference,
>> take this to the openbsd mailing lists.
>>
>
> It seems unlikely that this would make much of a difference.

It doesn't make one here.

> I think
> that allowing this package to create another ffmpeg vs libav mess is a
> mistake.
>

If you want to do some crazy downstream hackery, you'll have to do that
yourself and count me out. It's not even clear what you want to fix.

It was known from the start that we do not have ABI-compatibility.

The only "problem" right now are libressl-exclusive features. These are
currently optional codepaths afais, so packages still compile with openssl.

Everything else is future prediction. That's why libressl is still in an
overlay.

Reply | Threaded
Open this post in threaded view
|

RE: libressl status

Paul B. Henson
In reply to this post by hasufell
> From: hasufell
> Sent: Sunday, April 05, 2015 4:34 AM
>
> However, openntpd still compiles with openssl.

Well, the current stable openntpd in portage compiles with openssl but that's not surprising as it is ancient and predates libressl :). The current unstable openntpd actually has no ssl dependencies and needs neither openssl nor libressl to compile and function. It is the most recent upstream portable release that added an optional dependency on libressl for tls constraint functionality, that version is not yet in portage. It will work without libressl just as well as the current unstable openntpd does, you just won't have access to the new feature. So it's not really critical, but at some point it would be nice to get it working one way or the other.

> By that you are effectively forking libressl and causing a huge mess
> downstream for both developers and users.

What are the downsides of the approach pkgsrc is tentatively taking, where there are no modifications to libressl but the libraries are installed in an alternative location? Packages that require libressl can just use the appropriate linker options to find those libraries rather than the openssl ones?

> worse. This is something that has to be resolved upstream. If they don't
> cooperate long-term, then their fork will just die out for sure (and for
> good). However, I currently don't see strong signs for that.

I don't think their fork will ever die; even if no one outside of openbsd uses the portable version, it is now the official ssl provider for openbsd and I am sure will continue to be used by them as well as the portable versions of any of their other applications such as openntpd...



Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Jason A. Donenfeld-3
In reply to this post by hasufell

Solution:

Packages that will compile against either one get "|| (openssl libressl)". Packages that require a specific one will just have that one listed. Openssl will block Libressl and vice versa.

The result of this arrangement is that systems that require few packages will probably be able to have libressl installed, so long as all their ssl-using apps are okay with libressl. Systems that require openssl-only packages won't get libressl. Then, overtime, upstreams and patch writers will gradually fill in the gaps, and eventually most packages will support both.

This is the reasonable evolutionary approach. And it'll provide a good ground for gradually rolling out libressl support.

On Apr 5, 2015 10:35 PM, "hasufell" <[hidden email]> wrote:
On 04/05/2015 08:59 PM, Rich Freeman wrote:
> On Sun, Apr 5, 2015 at 2:35 PM, hasufell <[hidden email]> wrote:
>>
>> You are ranting at the wrong place. If you want to make a difference,
>> take this to the openbsd mailing lists.
>>
>
> It seems unlikely that this would make much of a difference.

It doesn't make one here.

> I think
> that allowing this package to create another ffmpeg vs libav mess is a
> mistake.
>

If you want to do some crazy downstream hackery, you'll have to do that
yourself and count me out. It's not even clear what you want to fix.

It was known from the start that we do not have ABI-compatibility.

The only "problem" right now are libressl-exclusive features. These are
currently optional codepaths afais, so packages still compile with openssl.

Everything else is future prediction. That's why libressl is still in an
overlay.

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Diego Elio Pettenò-2
On 6 April 2015 at 23:07, Jason A. Donenfeld <[hidden email]> wrote:
> Packages that will compile against either one get "|| (openssl libressl)".
> Packages that require a specific one will just have that one listed. Openssl
> will block Libressl and vice versa.

Doesn't work, you'd have to rebuild *all* the packages built with one
or the other when switching, or (if the SONAME were the same) they
would be built against a different ABI that they are built against,
which can be possibly causing security issues as data structure size
(and thus meaning of the content) changes.

Diego Elio Pettenò — Flameeyes
https://blog.flameeyes.eu/

Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Diego Elio Pettenò-2
In reply to this post by Paul B. Henson

On 6 April 2015 at 23:06, Paul B. Henson <[hidden email]> wrote:
> What are the downsides of the approach pkgsrc is tentatively taking, where there are no modifications to libressl but the libraries are installed in an alternative location? Packages that require libressl can just use the appropriate linker options to find those libraries rather than the openssl ones?

https://blog.flameeyes.eu/2014/07/libressl-drop-in-and-abi-leakage

Let's say you have program foo that uses libtls and libssl functions *and* CURL.

Now you built CURL against openssl, but foo requires libtls, so you link it against libressl. What happens when you run foo?

Symbols from openssl's libssl and libressl's libssl have the same name for API compatibility, right? So whichever of the two is loaded first (I *think* that's libressl but it is of course loader-dependent and in the case of glibc it depends on the environment variables too) will provide the symbols. But their ABIs are different so the calls coming from CURL (built against OpenSSL) will provide different data structures and parameter size than expected by libressl, or vice-versa.

Given these are security functions, the collisions are even riskier than just crashing, especially as *most* of the functions will likely work either way, as long as the same set of allocators/users/deallocators are used…

Diego Elio Pettenò — Flameeyes
https://blog.flameeyes.eu/
Reply | Threaded
Open this post in threaded view
|

Re: libressl status

Ben de Groot-2
In reply to this post by Jason A. Donenfeld-3
On 7 April 2015 at 06:07, Jason A. Donenfeld <[hidden email]> wrote:
> Solution:
>
> Packages that will compile against either one get "|| (openssl libressl)".
> Packages that require a specific one will just have that one listed. Openssl
> will block Libressl and vice versa.

This would result in a similar mess as we have been dealing with in
the ffmpeg / libav situation. Please don't do that. Make the choice
explicit, and don't rely on semi-automagic dependency resolution.
Also, explain clearly to users what the default implementation is and
why.

--
Cheers,

Ben | yngwin
Gentoo developer

12