Changing policy about -Werror

classic Classic list List threaded Threaded
119 messages Options
1 ... 3456
Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Sergei Trofimovich-6
On Fri, 14 Sep 2018 19:40:13 +0300
Alon Bar-Lev <[hidden email]> wrote:

> On Fri, Sep 14, 2018 at 12:34 AM Sergei Trofimovich <[hidden email]> wrote:
> >
> > On Tue, 11 Sep 2018 12:44:38 +0300
> > Alon Bar-Lev <[hidden email]> wrote:
> >
> > I'm personally in favour of not allowing -Werror
> > to be in build system unconditionally.
> >
> > Maintainer is free to implement --enable-werror any way
> > it's convenient to run on their own extended sanity checks
> > and optionally expose it to users. Be it USE flag or
> > EXTRA_ECONF option.  
>
> This discussion is not for downstream to have a more strict policy
> than upstream.

Correct.

To clarify: I was talking only about packages with following properties:
1. existing -Werror enabled upstream by default (or unconditionally)
2. disabled by default downstream by default (WRT current policy)

Nothing more.

> > Do you intend to keep -Werror for case when ebuild goes
> > stable as well?  
>
> Correct.
>
> > If you do it then what is your workflow to fix breakages
> > appearing in stable packages caused by minor environment
> > changes like toolchain tweaks?  
>
> Correct.
>
> > Add another round of stabilization on each arch team? It
> > sounds like your default assumption that code needs to be
> > changed in a semantically significant way: add annotations
> > that might not like some toolchains, remove unused code.  
>
> No dependency of toolchain nor annotations.
> A strict policy of no warnings will require changes as dependencies
> including toolchain are progressing.
> This creates an overhead for selected packages.
> A maintainer and the non-stable team should be able to know the package status.
> Preferably this may be done by automation, I appreciate the work of
> Toralf Förster with tinderbox to automate check various cases.
> When a new slot of toolchain is available, maintainers should check
> their packages in any case, there are other issues and side affects
> that we experienced, especially in C++ packages.
> For these packages usually there are 3 for each slot: the current
> stable, the next stable and the non-stable, so the overhead is
> constrained.

Sorry. I'm afraid I missed your answer. I'll restate the question again:

  If you do it then what is your workflow to fix breakages
  appearing in stable packages caused by minor environment
  changes like toolchain tweaks?  

I mean mechanically as a package maintainer.

Since you did not mention it I see a few alternatives:
- revbump a package with a backport of a local fix and fast-track
  stabilization without usual soak time in ~arch
- pull new upstream version and fast-track stabilization without
  usual soak time in ~arch
- no revbump, apply the patch as-is and hope it does not break
  existing users.
- anything else?

> > Unused variable is a good example of typical benign warning:
> > it was there all the time, it's not a new bug and does not
> > warrant need for an immediate fix.  
>
> Unused variable is a good example of CRITICAL potential issue

I understand you point. I disagree with it.

My litmus test: if behaviour of the package did not change after
the fix then the issue was not real.

> > Toolchain just happend to get better at control flow graph
> > analysis. Fix can easily wait for next release and save
> > everyone's time.  
>
> Once again, the number of permutation of build and architecture may
> reveal issues that are cannot be detected on maintainer machine.
> If a fix is a real issue that is found in stable package, do you
> suggest to wait for next release to save whose time?

It's up to maintainer to decide on how to resolve the issue once
maintainer understands the scope of the problem.

> Once again I outlined the cases in which -Werror can be preserved, I
> will repeat... (a) upstream has strict policy of no-warnings, (b)
> upstream added -Werror, (c) downstream opinion is that upstream is
> following the policy, (d) upstream is friendly, (e) downstream accepts
> the potential maintenance overhead.

Your proposal is clear. Thanks for restating it.

I still think keeping -Werror enabled by default makes more harm
than good.

> > Gentoo does not normally run tests on user's systems either.
> > Tests are arguably a lot more precise in pointing out real
> > problems in software.  
>
> Correct. I believe that this may be revisit as well, for selected
> packages in which tests are stable run them on user machines. There is
> no reason why we cannot add a directive to ebuild to enable tests by
> default and let user to disable this to save CPU/time of build.

Additional thanks for considering an option to disable tests back.

--

  Sergei

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Richard Yao-2
In reply to this post by Michael Orlitzky


> On Sep 14, 2018, at 3:29 PM, Michael Orlitzky <[hidden email]> wrote:
>
>> On 09/14/2018 01:52 PM, Rich Freeman wrote:
>>
>> Wouldn't the flip side of this be demonstrating that this has actually
>> caused issues?  If following upstream discovers no bugs and also
>> causes no issues, why not leave it to maintainer discretion?
>>
>
> We know it causes issues, there are hundreds of bugs about it (bugzilla
> stops counting at 500 on a search for "Werror").
>
> No one has answered the question: what do you do when a stable package
> breaks because of a new warning?
>
> If there's no answer to that question that doesn't involve making an
> unofficial in-place downstream-only edit to a piece of code that is (by
> the opposing argument) intensely security-critical in a stable package,
> then we're all wasting our time talking about this.
Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner.
>


Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Alon Bar-Lev-2
In reply to this post by Sergei Trofimovich-6
On Fri, Sep 14, 2018 at 10:53 PM Sergei Trofimovich <[hidden email]> wrote:

>
> On Fri, 14 Sep 2018 19:40:13 +0300
> Alon Bar-Lev <[hidden email]> wrote:
>
> >
> > No dependency of toolchain nor annotations.
> > A strict policy of no warnings will require changes as dependencies
> > including toolchain are progressing.
> > This creates an overhead for selected packages.
> > A maintainer and the non-stable team should be able to know the package status.
> > Preferably this may be done by automation, I appreciate the work of
> > Toralf Förster with tinderbox to automate check various cases.
> > When a new slot of toolchain is available, maintainers should check
> > their packages in any case, there are other issues and side affects
> > that we experienced, especially in C++ packages.
> > For these packages usually there are 3 for each slot: the current
> > stable, the next stable and the non-stable, so the overhead is
> > constrained.
>
> Sorry. I'm afraid I missed your answer. I'll restate the question again:
>
>   If you do it then what is your workflow to fix breakages
>   appearing in stable packages caused by minor environment
>   changes like toolchain tweaks?
>
> I mean mechanically as a package maintainer.
>
> Since you did not mention it I see a few alternatives:
> - revbump a package with a backport of a local fix and fast-track
>   stabilization without usual soak time in ~arch

Fix in place if false positive.
Revision bump if positive.

> - pull new upstream version and fast-track stabilization without
>   usual soak time in ~arch

No reason to wait for upstream if obvious positive just like any bug fix.

> - no revbump, apply the patch as-is and hope it does not break
>   existing users.

Correct.

> - anything else?

Patching the package (stable and unstable) to remove -Werror if too
many of them and downstream maintainer reaches to a conclusion that
the overhead is too high and benefit is little and provide this
feedback to upstream to work together on a new release of upstream
which resolves all warnings with newer toolchain.

>
> > > Unused variable is a good example of typical benign warning:
> > > it was there all the time, it's not a new bug and does not
> > > warrant need for an immediate fix.
> >
> > Unused variable is a good example of CRITICAL potential issue
>
> I understand you point. I disagree with it.
>
> My litmus test: if behaviour of the package did not change after
> the fix then the issue was not real.

But how can you get the report to evaluate if it is real or not? I
fail to understand this argument that is repeated over and over.
Everyone is smart after the did... while we are looking for the
trigger to evaluate.

> > > Toolchain just happend to get better at control flow graph
> > > analysis. Fix can easily wait for next release and save
> > > everyone's time.
> >
> > Once again, the number of permutation of build and architecture may
> > reveal issues that are cannot be detected on maintainer machine.
> > If a fix is a real issue that is found in stable package, do you
> > suggest to wait for next release to save whose time?
>
> It's up to maintainer to decide on how to resolve the issue once
> maintainer understands the scope of the problem.

Correct. My believe is that it is up to maintainer to decide.

> > Once again I outlined the cases in which -Werror can be preserved, I
> > will repeat... (a) upstream has strict policy of no-warnings, (b)
> > upstream added -Werror, (c) downstream opinion is that upstream is
> > following the policy, (d) upstream is friendly, (e) downstream accepts
> > the potential maintenance overhead.
>
> Your proposal is clear. Thanks for restating it.
>
> I still think keeping -Werror enabled by default makes more harm
> than good.

Yes, but we are not talking about by default, right?

> > > Gentoo does not normally run tests on user's systems either.
> > > Tests are arguably a lot more precise in pointing out real
> > > problems in software.
> >
> > Correct. I believe that this may be revisit as well, for selected
> > packages in which tests are stable run them on user machines. There is
> > no reason why we cannot add a directive to ebuild to enable tests by
> > default and let user to disable this to save CPU/time of build.
>
> Additional thanks for considering an option to disable tests back.
>

Any mechanism that is fully supported by upstream and downstream
maintainer find it stable should be enabled by default. I do see the
benefit of disabling tests not because they may break as per the same
argument I would like to have these reported and investigated, but to
save resources (time and CPU) which may be significant.

Thanks,
Alon

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Michael Orlitzky
In reply to this post by Richard Yao-2
On 09/14/2018 03:58 PM, Richard Yao wrote:
>>
>> No one has answered the question: what do you do when a stable package
>> breaks because of a new warning?
>>
>> ...>
> Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner.
>>

They would be uncovered during GCC stabilization, but then you're right
back in the original situation: how do you fix the stable package? The
only answer that doesn't violate some other policy is to patch it in a
new revision and wait for it to stabilize again.

Other questions arise: Do we block stabilization of clang et al.?

If we can simply remove -Werror because it's been a month, were the
warnings ever really important to begin with?

How many packages do we want to make the toolchain team stop and fix
before they can do their jobs?

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Rich Freeman
On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky <[hidden email]> wrote:

>
> On 09/14/2018 03:58 PM, Richard Yao wrote:
> >>
> >> No one has answered the question: what do you do when a stable package
> >> breaks because of a new warning?
> >>
> >> ...>
> > Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner.
> >>
>
> They would be uncovered during GCC stabilization, but then you're right
> back in the original situation: how do you fix the stable package? The
> only answer that doesn't violate some other policy is to patch it in a
> new revision and wait for it to stabilize again.
>
> Other questions arise: Do we block stabilization of clang et al.?
>

Presumably we could make it a blocker, so then portage won't install
the new stable toolchain.  That buys time and only affects users of
that particular package.  But, as I pointed out before you can do that
without using -Werror - just block installation with an unqualified
toolchain.

You would only use an approach like this for packages where QA was
fairly important, so the inconvenience would be worth it.

--
Rich

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Fabian Groffen-2
On 14-09-2018 16:29:43 -0400, Rich Freeman wrote:

> On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky <[hidden email]> wrote:
> >
> > On 09/14/2018 03:58 PM, Richard Yao wrote:
> > >>
> > >> No one has answered the question: what do you do when a stable package
> > >> breaks because of a new warning?
> > >>
> > >> ...>
> > > Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner.
> > >>
> >
> > They would be uncovered during GCC stabilization, but then you're right
> > back in the original situation: how do you fix the stable package? The
> > only answer that doesn't violate some other policy is to patch it in a
> > new revision and wait for it to stabilize again.
> >
> > Other questions arise: Do we block stabilization of clang et al.?
> >
>
> Presumably we could make it a blocker, so then portage won't install
> the new stable toolchain.  That buys time and only affects users of
> that particular package.  But, as I pointed out before you can do that
> without using -Werror - just block installation with an unqualified
> toolchain.
>
> You would only use an approach like this for packages where QA was
> fairly important, so the inconvenience would be worth it.
Perhaps, if one persists on going this route, only do this for platforms
that upstream supports, such that arches which will suffer from this
(typically ppc, sparc, ...) don't have to be blocked by this.

Fabian

--
Fabian Groffen
Gentoo on a different level

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Alon Bar-Lev-2
On Sat, Sep 15, 2018 at 12:02 AM Fabian Groffen <[hidden email]> wrote:

>
> On 14-09-2018 16:29:43 -0400, Rich Freeman wrote:
> > On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky <[hidden email]> wrote:
> > >
> > > On 09/14/2018 03:58 PM, Richard Yao wrote:
> > > >>
> > > >> No one has answered the question: what do you do when a stable package
> > > >> breaks because of a new warning?
> > > >>
> > > >> ...>
> > > > Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner.
> > > >>
> > >
> > > They would be uncovered during GCC stabilization, but then you're right
> > > back in the original situation: how do you fix the stable package? The
> > > only answer that doesn't violate some other policy is to patch it in a
> > > new revision and wait for it to stabilize again.
> > >
> > > Other questions arise: Do we block stabilization of clang et al.?
> > >
> >
> > Presumably we could make it a blocker, so then portage won't install
> > the new stable toolchain.  That buys time and only affects users of
> > that particular package.  But, as I pointed out before you can do that
> > without using -Werror - just block installation with an unqualified
> > toolchain.
> >
> > You would only use an approach like this for packages where QA was
> > fairly important, so the inconvenience would be worth it.
>
> Perhaps, if one persists on going this route, only do this for platforms
> that upstream supports, such that arches which will suffer from this
> (typically ppc, sparc, ...) don't have to be blocked by this.

Exactly in these cases the -Werror is useful as if upstream expects no
warnings then any warning should block installation and trigger bug
report. In Gentoo in many cases we use packages on platform has no
access to, our feedback to upstream is valuable. A great example is
gnutls in which we collectively (maintainer, unstable users,
architecture teams, stable users) found issues on architectures that
almost nobody other than Gentoo has access to.

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Fabian Groffen-2
On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:

> >
> > Perhaps, if one persists on going this route, only do this for platforms
> > that upstream supports, such that arches which will suffer from this
> > (typically ppc, sparc, ...) don't have to be blocked by this.
>
> Exactly in these cases the -Werror is useful as if upstream expects no
> warnings then any warning should block installation and trigger bug
> report. In Gentoo in many cases we use packages on platform has no
> access to, our feedback to upstream is valuable. A great example is
> gnutls in which we collectively (maintainer, unstable users,
> architecture teams, stable users) found issues on architectures that
> almost nobody other than Gentoo has access to.
>
I don't believe Gentoo users are (supposed to be) an extension of
upstreams.  If upstreams insist on that, they should make their software
non-free, adding a non-modification clause or something.  In any case,
it is not Gentoo's job IMHO.  In the end it is Gentoo who needs to care
for its users.  I prefer we do that by giving them an option to become
that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
disables by default.

As maintainer and/or enthusiastic user, like you wrote for gnutls, I
would be more than happy to provide build logs/errors for all the arches
I have access to.  So like I wrote before, I think we should consider
case-by-case basis to make it easy to do so.

Fabian

--
Fabian Groffen
Gentoo on a different level

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Alon Bar-Lev-2
On Sat, Sep 15, 2018 at 12:29 AM Fabian Groffen <[hidden email]> wrote:

>
> On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:
> > >
> > > Perhaps, if one persists on going this route, only do this for platforms
> > > that upstream supports, such that arches which will suffer from this
> > > (typically ppc, sparc, ...) don't have to be blocked by this.
> >
> > Exactly in these cases the -Werror is useful as if upstream expects no
> > warnings then any warning should block installation and trigger bug
> > report. In Gentoo in many cases we use packages on platform has no
> > access to, our feedback to upstream is valuable. A great example is
> > gnutls in which we collectively (maintainer, unstable users,
> > architecture teams, stable users) found issues on architectures that
> > almost nobody other than Gentoo has access to.
> >
>
> I don't believe Gentoo users are (supposed to be) an extension of
> upstreams.

This is exactly what I think that is special about Gentoo, and the
reason I use Gentoo. Unlike other distribution Gentoo is the closest
thing of using upstream. A maintainer in Gentoo who is not see himself
part of the upstream packages he maintains has far less impact than a
maintainer who does see himself as part of upstream or is upstream.

Per your statement, we should not allow any architecture or setup that
upstream, such as exact versioning, architecture or toolchain.

> If upstreams insist on that, they should make their software
> non-free, adding a non-modification clause or something.  In any case,
> it is not Gentoo's job IMHO.

Then we cannot re-distribute or patch, how is it related to the
discussion? We are talking about open source projects and I know it is
cliche... the "greater good" and helping the "free open source
movement" a a viable alternative. I thought this is what unite us
here.

> In the end it is Gentoo who needs to care
> for its users.  I prefer we do that by giving them an option to become
> that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
> disables by default.

Do you think someone do not care about the users? Do you actually
think upstream does not care about users? I do not understand this
statement. If downstream maintainer believes that upstream is friendly
for the Gentoo overhead (which is higher than binary distributions) or
create the relationship in which Gentoo is 1st citizen at upstream,
why do you think users cannot use vanilla upstream?

> As maintainer and/or enthusiastic user, like you wrote for gnutls, I
> would be more than happy to provide build logs/errors for all the arches
> I have access to.  So like I wrote before, I think we should consider
> case-by-case basis to make it easy to do so.

This entire discussion is to allow case-by-case and not black and
white approach recently enforced.

Regards,
Alon

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Richard Yao-2
In reply to this post by Michael Orlitzky

> On Sep 14, 2018, at 4:20 PM, Michael Orlitzky <[hidden email]> wrote:
>
> On 09/14/2018 03:58 PM, Richard Yao wrote:
>>>
>>> No one has answered the question: what do you do when a stable package
>>> breaks because of a new warning?
>>>
>>> ...>
>> Wouldn’t this be largely covered as part of GCC stabilization? We could reserve the right to kill -Werror in a package where it blocks GCC stabilization if the maintainer does not handle it in a timely manner.
>>>
>
> They would be uncovered during GCC stabilization, but then you're right
> back in the original situation: how do you fix the stable package? The
> only answer that doesn't violate some other policy is to patch it in a
> new revision and wait for it to stabilize again.
This depends on the issue.
>
> Other questions arise: Do we block stabilization of clang et al.?
We probably should start doing that once Clang is able to build everything, but someone would need to volunteer to handle it. It is a big job.
>
> If we can simply remove -Werror because it's been a month, were the
> warnings ever really important to begin with?
That was a suggestion to handle maintainer non-response. You can already do whatever you want if the maintainer is non-responsive after telling him in a bug that you will do something if a response is not made in a reasonable period (e.g. 2 weeks). I am just pointing it out.
>
> How many packages do we want to make the toolchain team stop and fix
> before they can do their jobs?
Presumably, the maintainers would handle this. If they cannot, they should not be honoring upstream’s -Werror policy.

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Richard Yao-2
In reply to this post by Fabian Groffen-2


> On Sep 14, 2018, at 5:28 PM, Fabian Groffen <[hidden email]> wrote:
>
> On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:
>>>
>>> Perhaps, if one persists on going this route, only do this for platforms
>>> that upstream supports, such that arches which will suffer from this
>>> (typically ppc, sparc, ...) don't have to be blocked by this.
>>
>> Exactly in these cases the -Werror is useful as if upstream expects no
>> warnings then any warning should block installation and trigger bug
>> report. In Gentoo in many cases we use packages on platform has no
>> access to, our feedback to upstream is valuable. A great example is
>> gnutls in which we collectively (maintainer, unstable users,
>> architecture teams, stable users) found issues on architectures that
>> almost nobody other than Gentoo has access to.
>>
>
> I don't believe Gentoo users are (supposed to be) an extension of
> upstreams.  If upstreams insist on that, they should make their software
> non-free, adding a non-modification clause or something.  In any case,
> it is not Gentoo's job IMHO.  In the end it is Gentoo who needs to care
> for its users.  I prefer we do that by giving them an option to become
> that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
> disables by default.
I am in complete agreement on this. Users should not be guinea pigs to help upstream unless they opt into it.

>
> As maintainer and/or enthusiastic user, like you wrote for gnutls, I
> would be more than happy to provide build logs/errors for all the arches
> I have access to.  So like I wrote before, I think we should consider
> case-by-case basis to make it easy to do so.
>
> Fabian
>
> --
> Fabian Groffen
> Gentoo on a different level


Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Fabian Groffen-2
In reply to this post by Alon Bar-Lev-2
I think you misunderstood what I wrote, or I wasn't clear enough.
Richard summed up my intention nicely in his response.

Fabian

On 15-09-2018 00:46:24 +0300, Alon Bar-Lev wrote:

> On Sat, Sep 15, 2018 at 12:29 AM Fabian Groffen <[hidden email]> wrote:
> >
> > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:
> > > >
> > > > Perhaps, if one persists on going this route, only do this for platforms
> > > > that upstream supports, such that arches which will suffer from this
> > > > (typically ppc, sparc, ...) don't have to be blocked by this.
> > >
> > > Exactly in these cases the -Werror is useful as if upstream expects no
> > > warnings then any warning should block installation and trigger bug
> > > report. In Gentoo in many cases we use packages on platform has no
> > > access to, our feedback to upstream is valuable. A great example is
> > > gnutls in which we collectively (maintainer, unstable users,
> > > architecture teams, stable users) found issues on architectures that
> > > almost nobody other than Gentoo has access to.
> > >
> >
> > I don't believe Gentoo users are (supposed to be) an extension of
> > upstreams.
>
> This is exactly what I think that is special about Gentoo, and the
> reason I use Gentoo. Unlike other distribution Gentoo is the closest
> thing of using upstream. A maintainer in Gentoo who is not see himself
> part of the upstream packages he maintains has far less impact than a
> maintainer who does see himself as part of upstream or is upstream.
>
> Per your statement, we should not allow any architecture or setup that
> upstream, such as exact versioning, architecture or toolchain.
>
> > If upstreams insist on that, they should make their software
> > non-free, adding a non-modification clause or something.  In any case,
> > it is not Gentoo's job IMHO.
>
> Then we cannot re-distribute or patch, how is it related to the
> discussion? We are talking about open source projects and I know it is
> cliche... the "greater good" and helping the "free open source
> movement" a a viable alternative. I thought this is what unite us
> here.
>
> > In the end it is Gentoo who needs to care
> > for its users.  I prefer we do that by giving them an option to become
> > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
> > disables by default.
>
> Do you think someone do not care about the users? Do you actually
> think upstream does not care about users? I do not understand this
> statement. If downstream maintainer believes that upstream is friendly
> for the Gentoo overhead (which is higher than binary distributions) or
> create the relationship in which Gentoo is 1st citizen at upstream,
> why do you think users cannot use vanilla upstream?
>
> > As maintainer and/or enthusiastic user, like you wrote for gnutls, I
> > would be more than happy to provide build logs/errors for all the arches
> > I have access to.  So like I wrote before, I think we should consider
> > case-by-case basis to make it easy to do so.
>
> This entire discussion is to allow case-by-case and not black and
> white approach recently enforced.
>
> Regards,
> Alon
>
--
Fabian Groffen
Gentoo on a different level

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Alon Bar-Lev-2
In reply to this post by Richard Yao-2
On Sat, Sep 15, 2018 at 1:14 AM Richard Yao <[hidden email]> wrote:

> > On Sep 14, 2018, at 5:28 PM, Fabian Groffen <[hidden email]> wrote:
> >
> > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:
> >>>
> >>> Perhaps, if one persists on going this route, only do this for platforms
> >>> that upstream supports, such that arches which will suffer from this
> >>> (typically ppc, sparc, ...) don't have to be blocked by this.
> >>
> >> Exactly in these cases the -Werror is useful as if upstream expects no
> >> warnings then any warning should block installation and trigger bug
> >> report. In Gentoo in many cases we use packages on platform has no
> >> access to, our feedback to upstream is valuable. A great example is
> >> gnutls in which we collectively (maintainer, unstable users,
> >> architecture teams, stable users) found issues on architectures that
> >> almost nobody other than Gentoo has access to.
> >>
> >
> > I don't believe Gentoo users are (supposed to be) an extension of
> > upstreams.  If upstreams insist on that, they should make their software
> > non-free, adding a non-modification clause or something.  In any case,
> > it is not Gentoo's job IMHO.  In the end it is Gentoo who needs to care
> > for its users.  I prefer we do that by giving them an option to become
> > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
> > disables by default.
> I am in complete agreement on this. Users should not be guinea pigs to help upstream unless they opt into it.

A new release of upstream is out, early adopters (what we call
unstable users) are guinea pings.
A new release is stabilized, users are guinea pings.
A new toolchain that upstream did not test, users are guinea pings.
A new dependency version or a Gentoo virtual with "compatible
library", users are guinea pings.
Let's say upstream does not have access to architecture X we at Gentoo
decide to support this architecture, maintainer do not have access to
this architecture as well, architecture team is guinea pings, but it
does not actually use the package, then back to early adopters and
users.

This process has nothing to do with -Werror, our process relays on
users as guinea pings, by definition developers and arch teams cannot
test entire software and all permutation of the software.

The -Werror (if supported by upstream and downstream, I outlined the
conditions many times) is a tool (among other) to help stop the
process at early stage when suspicious finding is there to allow deal
with the situation to make sure that the software is compatible with
an environment or permutation that upstream and maintainer do not have
direct access to. It is a tool to help users to have better system
integrity (once again, provided some conditions apply).

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Sergei Trofimovich-6
In reply to this post by Richard Yao-2
On Fri, 14 Sep 2018 11:54:57 -0400
Richard Yao <[hidden email]> wrote:

> >> My read of this is that the warning occurs regardless of optimization level, but it could somehow be improved by optimization.
> >>
> >> As for the last, it is for uninitialized variable reads. However, I think you are misinterpreting the claim. The way that optimization level could affect warning generation would be if the warning generation came after optimization passes that could hide reads. That means that -O3 would prevent the warning.
...
> Either provide code examples that generate warnings in a way that demonstrates that I am incorrect

To make code behave differently it needs substantial amount of code
to provide you an example. You need to him O2<->O3 behaviour delta
after all. But I will try (for a different warning, it should not matter
much).

Below is a reduced example of a larger C++ program.
Many thanks to Ulya for providing recent example!

In this example -O3 manages to inline/const-propagate deep enough
and find out potential null-deref. -O2 was not able to do it.

$ bash -x ./mk_.sh
+ LANG=C
+ g++-8.2.0 -O2 -c 1.cc -Wnull-dereference
+ g++-8.2.0 -O3 -c 1.cc -Wnull-dereference
1.cc: In function 'bool foo(std::vector<int>)':
1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
   typename h = e - d >> *g;
                ~~~~~~^~~~~
1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
   typename h = e - d >> *g;
                ~~~~~~^~~~~

$ cat 1.cc
#include <vector>
template <typename a, typename b> a c(a d, a e, b f, int *g) {
  typename h = e - d >> *g;
  for (; h;) if (f(*d)) if (f(*d)) return d;
  return d;
}
template <typename i, typename b> i o(i d, i e, b f, int *g) {
  return c(d, e, f, g);
}
template <typename i, typename b> i oo(i d, i e, b f, int *g) {
  return c(d, e, f, g);
}
template <typename j, typename b> j k(j d, j e, b f, int *g) {
  return o(d, e, f, g);
}
template <typename j, typename b> j kk(j d, j e, b f, int *g) {
  return oo(d, e, f, g);
}
bool cmpp(int);
bool foo(std::vector<int> l) {
  std::vector<int>::const_iterator ib,
      ie = l.end(), m, n = k(ib, ie, cmpp, 0) = kk(m, ie, cmpp, 0);
  return m == n;
}

--

  Sergei

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Richard Yao-2


> On Sep 14, 2018, at 7:07 PM, Sergei Trofimovich <[hidden email]> wrote:
>
> On Fri, 14 Sep 2018 11:54:57 -0400
> Richard Yao <[hidden email]> wrote:
>
>>>> My read of this is that the warning occurs regardless of optimization level, but it could somehow be improved by optimization.
>>>>
>>>> As for the last, it is for uninitialized variable reads. However, I think you are misinterpreting the claim. The way that optimization level could affect warning generation would be if the warning generation came after optimization passes that could hide reads. That means that -O3 would prevent the warning.
> ...
>> Either provide code examples that generate warnings in a way that demonstrates that I am incorrect
>
> To make code behave differently it needs substantial amount of code
> to provide you an example. You need to him O2<->O3 behaviour delta
> after all. But I will try (for a different warning, it should not matter
> much).
Thanks. I had been incorrect about -O3 giving not us some additional information for warnings. My apologies for the confusion.

>
> Below is a reduced example of a larger C++ program.
> Many thanks to Ulya for providing recent example!
>
> In this example -O3 manages to inline/const-propagate deep enough
> and find out potential null-deref. -O2 was not able to do it.
>
> $ bash -x ./mk_.sh
> + LANG=C
> + g++-8.2.0 -O2 -c 1.cc -Wnull-dereference
> + g++-8.2.0 -O3 -c 1.cc -Wnull-dereference
> 1.cc: In function 'bool foo(std::vector<int>)':
> 1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
>   typename h = e - d >> *g;
>                ~~~~~~^~~~~
> 1.cc:3:22: warning: null pointer dereference [-Wnull-dereference]
>   typename h = e - d >> *g;
>                ~~~~~~^~~~~
>
> $ cat 1.cc
> #include <vector>
> template <typename a, typename b> a c(a d, a e, b f, int *g) {
>  typename h = e - d >> *g;
>  for (; h;) if (f(*d)) if (f(*d)) return d;
>  return d;
> }
> template <typename i, typename b> i o(i d, i e, b f, int *g) {
>  return c(d, e, f, g);
> }
> template <typename i, typename b> i oo(i d, i e, b f, int *g) {
>  return c(d, e, f, g);
> }
> template <typename j, typename b> j k(j d, j e, b f, int *g) {
>  return o(d, e, f, g);
> }
> template <typename j, typename b> j kk(j d, j e, b f, int *g) {
>  return oo(d, e, f, g);
> }
> bool cmpp(int);
> bool foo(std::vector<int> l) {
>  std::vector<int>::const_iterator ib,
>      ie = l.end(), m, n = k(ib, ie, cmpp, 0) = kk(m, ie, cmpp, 0);
>  return m == n;
> }
>
> --
>
>  Sergei
>


Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Sergei Trofimovich-6
In reply to this post by Alon Bar-Lev-2
On Fri, 14 Sep 2018 23:15:28 +0300
Alon Bar-Lev <[hidden email]> wrote:

> > > > Unused variable is a good example of typical benign warning:
> > > > it was there all the time, it's not a new bug and does not
> > > > warrant need for an immediate fix.  
> > >
> > > Unused variable is a good example of CRITICAL potential issue  
> >
> > I understand you point. I disagree with it.
> >
> > My litmus test: if behaviour of the package did not change after
> > the fix then the issue was not real.  
>
> But how can you get the report to evaluate if it is real or not?

You don't. You don't get a report until user explicitly asks for it or
encounters real bug.

I would follow the same model Gentoo does for FEATURES=test
and FEATURES=strict/stricter.

If people like reporting build or test issues and have enough time
for it they will enable the flag.

> while we are looking for the trigger to evaluate.

My take on it: we are not looking for them on every users' machine
and it's ok to miss bugs and false positives as a result.

--

  Sergei

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Chí-Thanh Christopher Nguyễn
In reply to this post by Richard Yao-2
Richard Yao schrieb:

>> To make code behave differently it needs substantial amount of code
>> to provide you an example. You need to him O2<->O3 behaviour delta
>> after all. But I will try (for a different warning, it should not matter
>> much).
> Thanks. I had been incorrect about -O3 giving not us some additional information for warnings. My apologies for the confusion.
>>
>> Below is a reduced example of a larger C++ program.
>> Many thanks to Ulya for providing recent example!

Not that it matters now, but there are examples of packages building at -O2
but failing to build at -O3 optimization levels, due to -Werror.

One is dev-libs/libcss: https://bugs.gentoo.org/626752


Best regards,
Chí-Thanh Christopher Nguyễn

Reply | Threaded
Open this post in threaded view
|

Re: acceptable alternatives to -Werror, was: Changing policy about -Werror

Chí-Thanh Christopher Nguyễn
In reply to this post by Mike Auty-3
Mike Auty schrieb:
>> Installation will proceed, but the user will get a big fat warning that
>> the sys-fs/zfs package is potentially broken.
>
> This seems like a sure-fire way to make users paranoid and/or
> desensitized?  People will learn to ignore warnings if we make them big
> red and flashing but then say they're only potential breakages (and they
> subsequently discover that most everything runs fine).  If they learn to
> ignore big red flashing warnings it'll be more difficult when they're
> not potential breakages but actual ones we've accurately identified...

Maybe "big fat warning" was a bit of an overstatement. I meant the same kind
of notice that @preserved-rebuild set produces today. Or the "configuration
files need updating" message.

Much like the aforementioned, the affected package might continue to work
totally fine, or be broken in subtle ways. It might bother the user enough to
report a bug, but hopefully not enough to turn them away from Gentoo.

Best regards,
Chí-Thanh Christopher Nguyễn

Reply | Threaded
Open this post in threaded view
|

Re: Changing policy about -Werror

Alon Bar-Lev-2
In reply to this post by Chí-Thanh Christopher Nguyễn
On Sat, Sep 22, 2018 at 1:33 AM Chí-Thanh Christopher Nguyễn
<[hidden email]> wrote:

>
> Richard Yao schrieb:
>
> >> To make code behave differently it needs substantial amount of code
> >> to provide you an example. You need to him O2<->O3 behaviour delta
> >> after all. But I will try (for a different warning, it should not matter
> >> much).
> > Thanks. I had been incorrect about -O3 giving not us some additional information for warnings. My apologies for the confusion.
> >>
> >> Below is a reduced example of a larger C++ program.
> >> Many thanks to Ulya for providing recent example!
>
> Not that it matters now, but there are examples of packages building at -O2
> but failing to build at -O3 optimization levels, due to -Werror.
>
> One is dev-libs/libcss: https://bugs.gentoo.org/626752
>

It is matter, and shows that for selected packages in which upstream
has strict policy of no warning, each warning should be investigated
as it may be a true issue. The tool compiler provide to find these
edge condition should not be ignored nor overridden. The fact that a
package "builds" does not mean it is free of bugs.

Regards,
Alon

1 ... 3456