Mailing List Archive

Git migration progress for MW core
Hi all,

As some of you are probably aware, we've got a test repository converting
phase3 to git up and running on gerrit. You should be able to clone by

`git clone https://gerrit.wikimedia.org/r/p/test/mediawiki/core.git`

Couple of caveats (things I'm gonna try and fix):
* Permissions aren't sorted yet, so it's only supporting anonymous clones,
no pushing yet.
* The revision graph is crazy. svn:mergeinfo is unreliable and we're pretty
much unable to build a cohesive history without a *lot* of manual labor. Right
now I'm thinking of just dropping the mergeinfo so the branches look like linear
graphs cherry picking from master. Not perfect, but less annoying than now.

But yay progress! Clone it. Try it out, see what works (and what doesn't).
I'll try to get the permissions sorted later today so we can go ahead and
try some test pushes (and merges).

Things we still need to do:
* Make a git-setup like we've done for the other git repos. This will setup your
environment/hooks/etc for you.
* Figure out our WMF branching/deployment strategy, since we're very SVN-
centric right now with this.

Thanks for any input, guys.

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On Tue, Dec 13, 2011 at 11:44 AM, Chad <innocentkiller@gmail.com> wrote:
> Couple of caveats (things I'm gonna try and fix):
> * Permissions aren't sorted yet, so it's only supporting anonymous clones,
> no pushing yet.
> * The revision graph is crazy. svn:mergeinfo is unreliable and we're pretty
> much unable to build a cohesive history without a *lot* of manual labor. Right
> now I'm thinking of just dropping the mergeinfo so the branches look like linear
> graphs cherry picking from master. Not perfect, but less annoying than now.
>

Also there's two stupid commits at the head of master due to my
mistake when initially pushing the repo. That won't happen again
on subsequent tests or the real conversion.

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
Hi Chad,

Reposurgeon (http://catb.org/~esr/reposurgeon/ ) might be a useful tool to
help fix the svn history.

Best,

Diederik

On Tue, Dec 13, 2011 at 11:47 AM, Chad <innocentkiller@gmail.com> wrote:

> On Tue, Dec 13, 2011 at 11:44 AM, Chad <innocentkiller@gmail.com> wrote:
> > Couple of caveats (things I'm gonna try and fix):
> > * Permissions aren't sorted yet, so it's only supporting anonymous
> clones,
> > no pushing yet.
> > * The revision graph is crazy. svn:mergeinfo is unreliable and we're
> pretty
> > much unable to build a cohesive history without a *lot* of manual labor.
> Right
> > now I'm thinking of just dropping the mergeinfo so the branches look
> like linear
> > graphs cherry picking from master. Not perfect, but less annoying than
> now.
> >
>
> Also there's two stupid commits at the head of master due to my
> mistake when initially pushing the repo. That won't happen again
> on subsequent tests or the real conversion.
>
> -Chad
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On Tue, Dec 13, 2011 at 8:44 AM, Chad <innocentkiller@gmail.com> wrote:

> Hi all,
>
> As some of you are probably aware, we've got a test repository converting
> phase3 to git up and running on gerrit. You should be able to clone by
>
> `git clone https://gerrit.wikimedia.org/r/p/test/mediawiki/core.git`
>

\o/


> * The revision graph is crazy. svn:mergeinfo is unreliable and we're pretty
> much unable to build a cohesive history without a *lot* of manual labor.
> Right
> now I'm thinking of just dropping the mergeinfo so the branches look like
> linear
> graphs cherry picking from master. Not perfect, but less annoying than now.
>

I'd tend to agree. Subideal but at least manageable, especially since we've
usually been pretty good about saying which revs we merged in the commit
summaries.


> But yay progress! Clone it. Try it out, see what works (and what doesn't).
> I'll try to get the permissions sorted later today so we can go ahead and
> try some test pushes (and merges).
>

It's taking a few more minutes than I expect for the initial clone (though
that's usually not performance-critical); of course since there's a lot
more data in there than a SVN checkout I don't mind so much.

Final checkout sizes:
* SVN trunk/phase3 only: 145M
* git core, all branches with full history since a 2003 code reorg: 198M

Niiiiiiice!

Release branches from 1.1 to 1.18 are present... no individual release tags
visible though. Did we figure out how to get those done?

History (including 'git blame' view) seems to have transferred most
contributors' identities over nicely, using @users.mediawiki.org fake
addresses and adding realnames.


As viewed in gitk, branch points off of master are visible though as noted
the various merges don't preserve much data. Oh wells!

Things we still need to do:
> * Make a git-setup like we've done for the other git repos. This will
> setup your
> environment/hooks/etc for you.
> * Figure out our WMF branching/deployment strategy, since we're very SVN-
> centric right now with this.
>

... and extensions. Whee! ;)

WMF branching/deployment should actually be a lot more git-friendly, since
we can pull in the entire REL1_18 or REL1_19 or whatever into the
deployment branch in a single command instead of cherry-picking every one.
The key is making sure we've signed off of them so we know what's going
into release, naturally. :)

Merging individual fixes into the release branch would first be done by
cherry-picking, most likely.

-- brion
_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On 13 December 2011 17:02, Diederik van Liere <dvanliere@gmail.com> wrote:

> Reposurgeon (http://catb.org/~esr/reposurgeon/ ) might be a useful tool to
> help fix the svn history.


In particular, Eric Raymond (who develops it) is interested in large
and troublesome repos that teach him new and interesting things. If
ours is particularly gnarly, we might turn up new bugs :-D


- d.

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On 13/12/11 20:54, David Gerard wrote:
> On 13 December 2011 17:02, Diederik van Liere <dvanliere@gmail.com> wrote:
>
>> Reposurgeon (http://catb.org/~esr/reposurgeon/ ) might be a useful tool to
>> help fix the svn history.
>
>
> In particular, Eric Raymond (who develops it) is interested in large
> and troublesome repos that teach him new and interesting things. If
> ours is particularly gnarly, we might turn up new bugs :-D
>
> - d.

Thanks for the name, I had remembered seeing a "crazy" guy that wanted
to port hard histories but didn't remember who he was.
(to my defense, those blog entries don't name the author :) )

The blog post:
http://esr.ibiblio.org/?p=3839



_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On Tue, Dec 13, 2011 at 11:44 AM, Chad <innocentkiller@gmail.com> wrote:
> * Permissions aren't sorted yet, so it's only supporting anonymous clones,
> no pushing yet.
>

This has been done. If you pull the lastest from master you'll find
a git-setup file and you should be able to push your changes with
`git push-for-review`

Right now the group who can approve merges to master is small,
but I can hand this out liberally while we're figuring things out, just
ping me on IRC if you want it.

-Chad

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On Thu, Dec 15, 2011 at 9:43 AM, Chad <innocentkiller@gmail.com> wrote:
> On Tue, Dec 13, 2011 at 11:44 AM, Chad <innocentkiller@gmail.com> wrote:
>> * Permissions aren't sorted yet, so it's only supporting anonymous clones,
>> no pushing yet.
>>
>
> This has been done. If you pull the lastest from master you'll find
> a git-setup file and you should be able to push your changes with
> `git push-for-review`
>

Whoops, need to clarify: that's if you have an LDAP/SVN account.
If you do, you'll need to re-clone from the other[0].

-Chad

[0] ssh://<user>@gerrit.wikimedia.org:29418/test/mediawiki/core

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
> On Thu, Dec 15, 2011 at 9:43 AM, Chad <innocentkiller@gmail.com> wrote:
> that's if you have an LDAP/SVN account.
> If you do, you'll need to re-clone from the other[0].
>
> [0] ssh://<user>@gerrit.wikimedia.org:29418/test/mediawiki/core

Couldn't get this to work on Windows 7. I did the following steps:
1. copied my svn.wikimedia.org:22 session to gerrit.wikimedia.org:29418 in
PuTTY
2. In commandline: git clone
ssh://siebrand@gerrit.wikimedia.org:29418/test/mediawiki/core mwcore

Result:
<pop-up with: PuTTY Fatal Error: Disconnected. No supported authentication
methods available (server sent: public key)
Cloning into mwcore...
fatal: The remote end hung up unexpectedly

Any suggestions on how to get this working?

Siebrand


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
On Thu, Dec 15, 2011 at 10:18 AM, Siebrand Mazeland
<s.mazeland@xs4all.nl> wrote:
>> On Thu, Dec 15, 2011 at 9:43 AM, Chad <innocentkiller@gmail.com> wrote:
>> that's if you have an LDAP/SVN account.
>> If you do, you'll need to re-clone from the other[0].
>>
>> [0] ssh://<user>@gerrit.wikimedia.org:29418/test/mediawiki/core
>
> Couldn't get this to work on Windows 7. I did the following steps:
> 1. copied my svn.wikimedia.org:22 session to gerrit.wikimedia.org:29418 in
> PuTTY
> 2. In commandline: git clone
> ssh://siebrand@gerrit.wikimedia.org:29418/test/mediawiki/core mwcore

Same result on *nix, I'm assuming this means we need somebody to add
us to gerrit from svn?

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: Git migration progress for MW core [ In reply to ]
You need a labs account for this to work. If you'd like one, email me
with the following information:

1. Your preferred wiki user name. This will also be your gerrit/git
user name. If you want your git username to be your real name, then
make this your real name.
2. Your svn account name
3. Your preferred email address

We're in closed beta, but all svn account holders are welcome to join
at this time.

- Ryan

On Thu, Dec 15, 2011 at 9:01 AM, OQ <overlordq@gmail.com> wrote:
> On Thu, Dec 15, 2011 at 10:18 AM, Siebrand Mazeland
> <s.mazeland@xs4all.nl> wrote:
>>> On Thu, Dec 15, 2011 at 9:43 AM, Chad <innocentkiller@gmail.com> wrote:
>>> that's if you have an LDAP/SVN account.
>>> If you do, you'll need to re-clone from the other[0].
>>>
>>> [0] ssh://<user>@gerrit.wikimedia.org:29418/test/mediawiki/core
>>
>> Couldn't get this to work on Windows 7. I did the following steps:
>> 1. copied my svn.wikimedia.org:22 session to gerrit.wikimedia.org:29418 in
>> PuTTY
>> 2. In commandline: git clone
>> ssh://siebrand@gerrit.wikimedia.org:29418/test/mediawiki/core mwcore
>
> Same result on *nix, I'm assuming this means we need somebody to add
> us to gerrit from svn?
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l