Devmanual text on ChangeLogs

classic Classic list List threaded Threaded
50 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: git?

Andreas K. Huettel
Am Sonntag 01 Mai 2011, 12:54:52 schrieb Panagiotis Christopoulos:
> > What is it really that is holding us up?  A dev to spearhead the move?
>
> I had the same question yesterday but after checking [1], I can tell
> that it's not so simple as it seems when you first think of it.
>
> [1] https://bugs.gentoo.org/show_bug.cgi?id=333531

Anyway it would be very nice to get some sort of status update. Apart from one
comment by you, all these bugs have not been touched since september...


--

Andreas K. Huettel
Gentoo Linux developer
[hidden email]
http://www.akhuettel.de/


signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Andreas K. Huettel
In reply to this post by Samuli Suominen-4
Am Sonntag 01 Mai 2011, 11:06:47 schrieb Samuli Suominen:
> ... the time alone if you have to stop on each package to wait for
> echangelog to get done just doubles the amount of time you have to put
> into committing them. That's just not worth the effort.

Ever heard of opening a second terminal? :D


--

Andreas K. Huettel
Gentoo Linux developer
[hidden email]
http://www.akhuettel.de/


signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

pva0xd
In reply to this post by Panagiotis Christopoulos
В Вск, 01/05/2011 в 13:44 +0300, Panagiotis Christopoulos пишет:

> On 12:06 Sun 01 May     , Samuli Suominen wrote:
> > So not only they are rather useless, and information you can easily
> get
> > from sources.gentoo.org, they take your time as well.
>
> Then, let's change it to:
> <snip>
> "Though not mandatory, it is highly recommended that file removals are
> also recorded the same way."
> </snip>

Panagiotis, there is no use in ChangeLog if information there is not
reliable. With policy you suggest one will have to check ChangeLog first
and then in half cases go to sources.gentoo.org to find out when ebuild
was removed. Two actions instead of one: either look ChangeLog or go to
web for cvs history.

Also, repoman commit is even slower then echangelog and thus nobody
waits for it to finish: just call it in console and switch to other
deals then just return back to check. If there are very many commits it
possible to script them. The only thing Samuli needs to do is to use
ecommit() hook:

ecommit() {
        echangelog "$@"
        repoman commit -m "$@"
}

But Samuli knows this very well and Fabian's answer describes situation
best.

--
Peter.


Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Duncan-42
In reply to this post by Fabian Groffen-2
Fabian Groffen posted on Sun, 01 May 2011 12:00:17 +0200 as excerpted:

> Attachment not shown: MIME type chemical/x-genbank; filename
> ChangeLog.gen

Had to laugh at that one. =:^)

--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman


Reply | Threaded
Open this post in threaded view
|

Re: Re: Devmanual text on ChangeLogs

Fabian Groffen-2
On 01-05-2011 14:55:24 +0000, Duncan wrote:
> Fabian Groffen posted on Sun, 01 May 2011 12:00:17 +0200 as excerpted:
>
> > Attachment not shown: MIME type chemical/x-genbank; filename
> > ChangeLog.gen
>
> Had to laugh at that one. =:^)

Apologies, the .gen extension apparently made the MIME match to this
thing I'd never heard before.  You can just open it as normal text file,
though.


--
Fabian Groffen
Gentoo on a different level

Reply | Threaded
Open this post in threaded view
|

Re: git? [was: Re: Devmanual text on ChangeLogs]

Maciej Mrozowski-2
In reply to this post by Eray Aslan-2
On Sunday 01 of May 2011 12:09:15 Eray Aslan wrote:
> On Sun, May 01, 2011 at 12:06:47PM +0300, Samuli Suominen wrote:
> > ... the time alone if you have to stop on each package to wait for
> > echangelog to get done just doubles the amount of time you have to put
> > into committing them. That's just not worth the effort.
>
> Won't moving the tree to git will make this a moot discussion?  These and
> similar solutions look more and more lika a band-aid to the defecencies
> of cvs.

No, because ChangeLogs could be dropped even now (and generated for rsync
using cvs2cl tool) since ebuild history is already available in CVS
(sources.gentoo.org). This discussion is about what tree changes are and which
aren't relevant enough for users to be redundantly documented in ChangeLog
files assuming they're to be kept for now.

--
regards
MM

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Brian Harring-2
In reply to this post by Samuli Suominen-4
On Sun, May 01, 2011 at 12:06:47PM +0300, Samuli Suominen wrote:
> ... the time alone if you have to stop on each package to wait for
> echangelog to get done just doubles the amount of time you have to put
> into committing them. That's just not worth the effort.

This argument sucks; if the tool is problematic... fix the tool.  
Simple example, why is is it interactive?

Add a -m <message> option to it; no longer have to watch it, just fire
the command in a term (or in screen) w/ the message given to it
already.

Beyond that... I suspect *everyone* would appreciate optimization done
to echangelog.  From a quick look... seems like it's cvs status, than
a cvs diff.  Trying to collapse that into a single op, falling back to
status might not be a bad thing (or parallelizing the requests so the
slowness of cvs doesn't cause sequentially stack up).

Either way.. fix the tool, rather than just doing the wrong thing.


> So not only they are rather useless, and information you can easily get
> from sources.gentoo.org, they take your time as well.

I think the dial up users would have a real issue with your "easily
get from sources.g.o" statement- same for users like myself when I'm
in public transit/flying/working somewhere than work and at home (I
actually do use those logs when I'm checking depgraph/pcheck issues).

Either way, fix the tool, or prove that the tool can't go any faster,
and *then* it's a potential discussion.  Right now it really isn't,
imo.
~brian

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Markos Chandras-2
In reply to this post by Fabian Groffen-2
On Sun, May 01, 2011 at 12:00:17PM +0200, Fabian Groffen wrote:

> On 30-04-2011 11:46:37 +0300, Petteri Räty wrote:
> > http://devmanual.gentoo.org/ebuild-writing/misc-files/changelog/index.html
> >
> > There doesn't seem to be a common opinion on what the policy for
> > ChangeLog entries is. See:
> >
> > http://archives.gentoo.org/gentoo-dev/msg_f829da2375f1ceab766a800913cc4998.xml
> >
> > I propose a simple new text: "Every commit should have an entry in
> > ChangeLog." If we eventually autogenerate them from git logs this would
> > happen any way (unless some kind of filtering system is in the middle)
> > so we could already start now. I think it's better to have more than
> > less information available to users.
>
> Because everytime we need something more sophisticated people come up
> with the holy git grail, here is the script to generate a
> echangelog-style ChangeLog from CVS, right here, right now.
>
> It's a naive implementation, but the output shows the differences
> between the committed log, and what would be generated from CVS.  (The
> usernames could be looked up easily, but I was too lazy to do that.)
> People can use this to judge if "autogeneration from VCS" is a good
> thing or not.
>
> My conclusion is that you probably want to maintain the ChangeLog
> manually.
>
> I also attached a sample of the script output for net-p2p/transmission
> for convenience.
>
>
> --
> Fabian Groffen
> Gentoo on a different level
Since most ( if not all ) of us use the same message on the Changelog
and on the commit log, it probably worth the effort of having the rsync
servers create the Changelogs before populate the portage tree. Having
the servers do that, will also allow us to provide cut down Changelogs
( lets say keep that last 10 entries ) so we can provide a more minimal
portage tree, size wise. A huge portage tree might not be a problem for
most of us but it sure is for embedded and all kind of similar systems.

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Brian Harring-2
On Sun, May 01, 2011 at 10:08:31PM +0100, Markos Chandras wrote:
> Since most ( if not all ) of us use the same message on the Changelog
> and on the commit log, it probably worth the effort of having the rsync
> servers create the Changelogs before populate the portage tree. Having
> the servers do that, will also allow us to provide cut down Changelogs
> ( lets say keep that last 10 entries ) so we can provide a more minimal
> portage tree, size wise. A huge portage tree might not be a problem for
> most of us but it sure is for embedded and all kind of similar systems.

This opens up a bit of nastyness; either the service would have to
resign all manifests (which defeats a fair bit of the signing intent),
or ChangeLog's would have to pulled in full from cvs, generated
strictly server side (else manifest will have stale chksums for it),
and ChangeLog will have to exist outside of all validation.

So... either resigning everywhere for regen, or having no validation
asserted on the ChangeLog- meaning certain men in the middle have a
nice area to inject some unfriendly things for anyone who happens to
read it.

~harring

Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Markos Chandras-2
On Sun, May 01, 2011 at 03:33:25PM -0700, Brian Harring wrote:

> On Sun, May 01, 2011 at 10:08:31PM +0100, Markos Chandras wrote:
> > Since most ( if not all ) of us use the same message on the Changelog
> > and on the commit log, it probably worth the effort of having the rsync
> > servers create the Changelogs before populate the portage tree. Having
> > the servers do that, will also allow us to provide cut down Changelogs
> > ( lets say keep that last 10 entries ) so we can provide a more minimal
> > portage tree, size wise. A huge portage tree might not be a problem for
> > most of us but it sure is for embedded and all kind of similar systems.
>
> This opens up a bit of nastyness; either the service would have to
> resign all manifests (which defeats a fair bit of the signing intent),
> or ChangeLog's would have to pulled in full from cvs, generated
> strictly server side (else manifest will have stale chksums for it),
> and ChangeLog will have to exist outside of all validation.
>
> So... either resigning everywhere for regen, or having no validation
> asserted on the ChangeLog- meaning certain men in the middle have a
> nice area to inject some unfriendly things for anyone who happens to
> read it.
>
> ~harring
>
Thats a fair point but the way I see it we need to make a balanced
choice. Obviously is not feasible to have the rsync servers
resign everything. This would require having all the gpg keys on the rsync
servers, fetch the developer's name from the last cvs commit and use his
key to resign it. It doesn't look that smart to me.
        Leaving Changelogs unprotected might be a bit of a trouble but it
certainly is not that big a deal. Nothing serious can happen if someone
hijacks a plain text file.
        In case people want to ensure
end-to-end point integrity, we can use a separate GPG key for the rsync
server. However, this will make our GPG keys useless, and having a
single key to sing 10.000 Manifest files does not look good either.

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Duncan-42
In reply to this post by Markos Chandras-2
Markos Chandras posted on Sun, 01 May 2011 22:08:31 +0100 as excerpted:

> Having the servers do that, will also allow us to provide cut down
> Changelogs ( lets say keep that last 10 entries ) so we can provide a
> more minimal portage tree, size wise.

What about cutting it to the largest whole number of entries that can fit
in 4 KB, since many filesystems use 4 KB blocks anyway, with files always
taking a whole number of blocks?

If the file's going to use 4 KB anyway, might as well take advantage of it.

Taking a look at the top of my last synced portage changelog as what's
likely an example from the verbose end, that's thirteen entries, here,
plus the header at the top.  For most packages I imagine it'd be something
like 20 entries.

(Yes, I know some filesystems don't have that restriction and in fact use
one myself, but if we're going for some arbitrary file size, 4 KB is about
as reasonable a choice as it gets, precisely /because/ that's the default
block size for so many widely used fss.)

--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman


Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Duncan-42
In reply to this post by Markos Chandras-2
Markos Chandras posted on Sun, 01 May 2011 23:49:06 +0100 as excerpted:

> On Sun, May 01, 2011 at 03:33:25PM -0700, Brian Harring wrote:
>> On Sun, May 01, 2011 at 10:08:31PM +0100, Markos Chandras wrote:
>>> Since most ( if not all ) of us use the same message on the Changelog
>>> and on the commit log, it probably worth the effort of having the
>>> rsync servers create the Changelogs before populate the portage tree.

>> This opens up a bit of nastyness; either the service would have to
>> resign all manifests (which defeats a fair bit of the signing intent),
>> or ChangeLog's would have to pulled in full from cvs, generated
>> strictly server side (else manifest will have stale chksums for it),
>> and ChangeLog will have to exist outside of all validation.

> Thats a fair point but the way I see it we need to make a balanced
> choice. Obviously is not feasible to have the rsync servers resign
> everything. [But] having all the gpg keys on the rsync servers [...]
> doesn't look that smart to me.

> Leaving Changelogs unprotected might be a bit of a trouble but it
> certainly is not that big a deal. Nothing serious can happen if someone
> hijacks a plain text file.

> In case people want to ensure end-to-end point integrity, we can use
> a separate GPG key for the rsync server. However, this will make our GPG
> keys useless, and having a single key to sing 10.000 Manifest files does
> not look good either.

What about having a dedicated server-based changlog-signing key?  That's
still a lot of signing with a single key, but as you observed, the hazards
of a loss of integrity there aren't as high as with most of the tree
content.  It'd require changes, but I don't believe they're out of line
with that required for the rest of the proposal.

--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman


Reply | Threaded
Open this post in threaded view
|

Re: Re: Devmanual text on ChangeLogs

Brian Harring-2
On Sun, May 01, 2011 at 11:23:40PM +0000, Duncan wrote:
> What about having a dedicated server-based changlog-signing key?  That's
> still a lot of signing with a single key, but as you observed, the hazards
> of a loss of integrity there aren't as high as with most of the tree
> content.  It'd require changes, but I don't believe they're out of line
> with that required for the rest of the proposal.

It means the only real trust that clients can level is on that key-
since it will be the last signer (thus /the/ signer) across all pkgs.

Get at that key, and you've got the tree, versus the current form,
crack all signing keys and you've got the tree.

Mind you this is ignoring eclasses, but getting eclasses sorted will
be mildly pointless if the rest of the solution has been
weakened/gutted since.

Point is, it's not *just* about having a signature on it- it's about
mapping the trust of that signature back, and sectioning/containing
compromises.  What y'all are suggesting guts that layered defense.
~brian

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Re: Devmanual text on ChangeLogs

Rich Freeman
On Sun, May 1, 2011 at 7:31 PM, Brian Harring <[hidden email]> wrote:
> Get at that key, and you've got the tree, versus the current form,
> crack all signing keys and you've got the tree.

Well, more like get any one of the keys and you get the tree, since
portage only validates that a trusted key signed a package, and not
that the key belonged to the package maintainer.

In any case, the whole way that manifest signing works does not really
preserve a signature from end-to-end.  If I sign three files and
somebody else signs two files, they end up overwriting my signature.

So, if a mirror checks all the sigs, makes a change, and re-signs with
its own key that isn't much less secure than what we have now.  I
wouldn't actually distribute the work all the way to the mirrors
though - I'd have a central server generate the changelogs, sign them,
and then propagate that to the mirror network.  You just need to
protect that one server really well then.

If you really want to have dev->user trust with no broken links then
the signatures would need to be associated with each file - not just
the whole manifest.  Plus, the local portage would need to check the
metadata cache for consistency.

In any case, I see manifest signing as a relatively minor issue here.
It seems like the more fundamental debate is how much metadata we
really should be distributing all the way to end-user systems, vs
keeping it in a repository like a cvs log.  Sure, offline access is
useful, but the question is whether it is useful enough.

My personal feeling is that we should keep the changelogs as-is, and
include removals, until we're on git.  Then we should re-evaluate.

Rich

Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Ulrich Müller
In reply to this post by Markos Chandras-2
>>>>> On Sun, 1 May 2011, Markos Chandras wrote:

> Since most ( if not all ) of us use the same message on the
> Changelog and on the commit log, it probably worth the effort of
> having the rsync servers create the Changelogs before populate the
> portage tree.

A separate ChangeLog has the advantage that entries for trivial
changes can be omitted. Most people wouldn't want to read entries
about comment changes, for example. Also entries can be edited, which
is not possible if the ChangeLog is generated from commit messages.

> Having the servers do that, will also allow us to provide cut down
> Changelogs ( lets say keep that last 10 entries ) so we can provide
> a more minimal portage tree, size wise.

Ten is way too small. Chances are that after one round of
stabilisations the ChangeLog entry for the last real change to the
package will be gone. We should keep at least one year (better two)
of history, because our aim is that users' systems should still be
upgradeable after this time. And IMHO emerge -l should give the user
the full list of changes since his last update.

Ulrich

Reply | Threaded
Open this post in threaded view
|

Re: Re: Devmanual text on ChangeLogs

Markos Chandras-2
In reply to this post by Rich Freeman
On Sun, May 01, 2011 at 07:43:48PM -0400, Rich Freeman wrote:
> On Sun, May 1, 2011 at 7:31 PM, Brian Harring <[hidden email]> wrote:
> > Get at that key, and you've got the tree, versus the current form,
> > crack all signing keys and you've got the tree.
>
> My personal feeling is that we should keep the changelogs as-is, and
> include removals, until we're on git.  Then we should re-evaluate.
>
> Rich
>
Git migration won't happen anytime soon. Why postpone the problem
instead of mitigating it? The solution can easily be migrated to git if
needed.

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Re: Devmanual text on ChangeLogs

Markos Chandras-2
In reply to this post by Brian Harring-2
On Sun, May 01, 2011 at 04:31:08PM -0700, Brian Harring wrote:

> On Sun, May 01, 2011 at 11:23:40PM +0000, Duncan wrote:
> > What about having a dedicated server-based changlog-signing key?  That's
> > still a lot of signing with a single key, but as you observed, the hazards
> > of a loss of integrity there aren't as high as with most of the tree
> > content.  It'd require changes, but I don't believe they're out of line
> > with that required for the rest of the proposal.
>
> It means the only real trust that clients can level is on that key-
> since it will be the last signer (thus /the/ signer) across all pkgs.
>
> Get at that key, and you've got the tree, versus the current form,
> crack all signing keys and you've got the tree.
>
> Mind you this is ignoring eclasses, but getting eclasses sorted will
> be mildly pointless if the rest of the solution has been
> weakened/gutted since.
>
> Point is, it's not *just* about having a signature on it- it's about
> mapping the trust of that signature back, and sectioning/containing
> compromises.  What y'all are suggesting guts that layered defense.
> ~brian
Then the only choice here is to ignore Changelogs from Manifests and
live with that. You have your changelogs unprotected but you keep your
ebuilds safe(?). As I said, it is a balanced choice that has to be made.

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Markos Chandras-2
In reply to this post by Ulrich Müller
On Mon, May 02, 2011 at 02:04:57AM +0200, Ulrich Mueller wrote:

> >>>>> On Sun, 1 May 2011, Markos Chandras wrote:
>
> Ten is way too small. Chances are that after one round of
> stabilisations the ChangeLog entry for the last real change to the
> package will be gone. We should keep at least one year (better two)
> of history, because our aim is that users' systems should still be
> upgradeable after this time. And IMHO emerge -l should give the user
> the full list of changes since his last update.
>
> Ulrich
>
And how are you going to accomplish that assuming that you deliver a
trimmed down ChangeLog version? Would emerge -l fetch the full
changelog over the inet? And why do you even want to see the full history of
the package? Nobody cares about the ebuild changes that occurred in 2005.
And if he does, this is just a corner case, and sources.g.o is your friend.

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Duncan-42
In reply to this post by Rich Freeman
Rich Freeman posted on Sun, 01 May 2011 19:43:48 -0400 as excerpted:

> On Sun, May 1, 2011 at 7:31 PM, Brian Harring <[hidden email]>
> wrote:
>> Get at that key, and you've got the tree, versus the current form,
>> crack all signing keys and you've got the tree.
>
> Well, more like get any one of the keys and you get the tree, since
> portage only validates that a trusted key signed a package, and not that
> the key belonged to the package maintainer.

OK, so everything in a manifest signs together, and if the changelog as-is
gets server-signed, so does the rest of the manifest.

I see the problem there, but there are ways around it.  As I said, changes
may be necessary, but they aren't huge compared to the scope of the whole
idea.

What about having the server-generated changelogs separate from the rest
of the package, say in a changelogs dir, one such dir per category with
for example portage's changelog then located at
sys-apps/changelogs/portage, thus preventing between-category naming
collisions (we've been there!)?

Then the server could generate and sign the changelogs without interfering
with the package manifests and their signatures.  The changelogs would all
be signed by the same key, but it wouldn't be used for signing anything
else, thus not interfering with actual package security at all.

--
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman


Reply | Threaded
Open this post in threaded view
|

Re: Devmanual text on ChangeLogs

Nirbheek Chauhan-2
In reply to this post by Ulrich Müller
On Mon, May 2, 2011 at 5:34 AM, Ulrich Mueller <[hidden email]> wrote:

>>>>>> On Sun, 1 May 2011, Markos Chandras wrote:
>
>> Since most ( if not all ) of us use the same message on the
>> Changelog and on the commit log, it probably worth the effort of
>> having the rsync servers create the Changelogs before populate the
>> portage tree.
>
> A separate ChangeLog has the advantage that entries for trivial
> changes can be omitted. Most people wouldn't want to read entries
> about comment changes, for example. Also entries can be edited, which
> is not possible if the ChangeLog is generated from commit messages.
>

Trivial commit messages can be omitted from the final ChangeLog very
easily. We just need to decide on a token to add to the commit message
— either [trivial] in the subject, or #trivial in the body, or similar

I don't get why someone would want to edit ChangeLogs. Could you list
some use-cases besides editing of typos?

--
~Nirbheek Chauhan

Gentoo GNOME+Mozilla

123