Mailing List Archive

WikiScripts
Hello!

As many of you probably know, during last weeks (and also a lot of
time in 2009 and 2010) I have been working on the WikiScripts
extension (formerly InlineScripts), which allows to embed scripts into
pages.

The language it uses, as well as the instruction of how to use it, are
here: <http://www.mediawiki.org/wiki/Extension:WikiScripts>. Any clean
up of the docs would be appreciated.

(the rest of the mail probably should be read after reading the
documentation; right now it's short, because I'm lazy!)

The reason why we want our own language at all:
* Code maintainability. Right now we have horrible templates like the
legendary {{Citation/core}} nobody can really understand.
* I personally hope that it will be more performance-wise and easier
to optimize (due to fact that it has pretty straightforward
context-free grammar).
* It allows us to increase potential complexity of the code,
introducing things like string functions

The reason why we use our own language is:
* We can restrict is as we want (for example, the for( ; ; )
instruction is not present);
* We can make it platform-independent since it is written in PHP
* Based on some experience with writing those scripts and my generic
experience as well, I have added some nice features absent from
different languages, mainly the append keyword (see docs).

The reason why from InlineScripts it was changed to non-inline model:
* This is the only way we can adequately implement functions.
Previously I wanted to implement them as calling another template
directly from script, but parser overhaul seems useless, and this way
we have a better control over it.
* We can add syntax highlighting
* We can add the code editor
* It is easier to track inclusions that way
* Greater potential for global scripts

== The plan ==
Right now the extension is almost ready. There are certain features I
would like to see there (some are listed below), but no
backwards-incompatible dramatic changes. However, you may want to make
them, so here is my plan
1. We settle down all issues we will not be able to change later due
to b/c. I made a list of some below, you may want to add more. Before
we settle them, we cannot move further.
2. At this point I want someone at WMF review the language description
and make sure it is OK with them.
3. Once the details are settled, we may start doing
internationalization, documentation, cleanup and testing.
4. Probably some live test deployment here.
5. ???
6. PROFIT. WMF-wide deployment here.

== The questions ==
Here are some questions I found worth discussing. We need to discuss
them before we move on:
1. Case sensitivity. Right now the language is case-sensitive. We may
want to change that, since case-insensitivity is more user-friendly.
2. Transclusion from wikitext. I have added two hooks ({{#invoke:}}
and {{s:}}) and will probably add two more for global scripts
({{#globalinvoke:}} and {{g:}}). Any better ideas for hook names are
welcome.
3. Library. Right now I developed a set of conventions in order to
make the function naming more consistent and modular (with ability to
add new modules by extension).
4. Your question here.

== Stability and performance ==
Right now my benchmarking shows that a simple template which uses
{{#switch}} is as effective as a simple module using associated
arrays. You may benchmark yourself and put the results on the wiki. If
we find the problems with performance on the real use cases during
live trial (or before, or after), we have a lot possible performance
tweaks, including:
* Function output caching;
* Built-in optimizer;
* Separate cache for scripts (there aren't as many as templates), in
order to increase hit rate;
* Rewrite of parser in C;
* Rewrite of whole interpreter in C (probably not necessary, 50% to
75% of time is parsing);
* Rewrite of whole MediaWiki in <s>Python</s><s>C</s> HipHop-compatible PHP!

As of stability, we have built-in platform-independent limits (like
evaluation count) and most of insecure features (recursion, infinite
loops) are disabled (you can still make a deep nested loop, but you
better write the scripts as if Domas knew where you live).

== Future ==
Some possible future tweaks:
* switch construction
* Built-in code editor (with highlighting and auto-ident)
* AJAX-based debugging stuff right in the edit form (syntax check,
calling the function from the edit form instead of preview, etc).
* Global scripts! That would be (probably) easier than global
templates; the only tricky part would be tracking changes and
invalidating local caches.
* Your suggestion submitted here:
https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=WikiScripts

--vvv

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: WikiScripts [ In reply to ]
Victor Vasiliev wrote:
> As many of you probably know, during last weeks (and also a lot of
> time in 2009 and 2010) I have been working on the WikiScripts
> extension (formerly InlineScripts), which allows to embed scripts into
> pages.
>
> The language it uses, as well as the instruction of how to use it, are
> here: <http://www.mediawiki.org/wiki/Extension:WikiScripts>. Any clean
> up of the docs would be appreciated.

This all looks really great!

I have two questions, which are somewhat related:

* Has there a been a discussion/agreement about using a new scripting
language on Wikimedia wikis? I know Lua, JavaScript, StringFunctions and a
bunch of other options have been endlessly discussed in the past. Does your
e-mail mean that a decision is made and this can move forward?

* What level of Wikimedia sysadmin support is there for this? I know you've
worked on large projects in the past and have had difficulty getting code
reviewed/deployed. I see Tim's name in some of the page histories (e.g.,
/Brainstorming), but some direct clarification about this would be great.

MZMcBride



_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: WikiScripts [ In reply to ]
2011/8/18 Victor Vasiliev <vasilvv@gmail.com>
>
> Hello!
>
> As many of you probably know, during last weeks (and also a lot of
> time in 2009 and 2010) I have been working on the WikiScripts
> extension (formerly InlineScripts), which allows to embed scripts into
> pages.

Excellent!

Just little bits of constructive criticism:

> 1. Case sensitivity. Right now the language is case-sensitive. We may
> want to change that, since case-insensitivity is more user-friendly.

No, case-insensitivity is user-UNfriendly. In MediaWiki links and
templates are case-sensitive (except the first letter) and it is a
Good Thing. The script language must also be case-sensitive. Hey, it's
less work for you.

> 2. Transclusion from wikitext. I have added two hooks ({{#invoke:}}
> and {{s:}}) and will probably add two more for global scripts
> ({{#globalinvoke:}} and {{g:}}). Any better ideas for hook names are
> welcome.

One-letter names will be very confusing. 's' and 'g' can mean just
about anything. I would offer better names, but i don't even
understand what are these supposed to mean.

--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
‪“We're living in pieces,
I want to live in peace.” – T. Moore‬

_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: WikiScripts [ In reply to ]
On 18/08/11 13:52, MZMcBride wrote:
> * Has there a been a discussion/agreement about using a new scripting
> language on Wikimedia wikis? I know Lua, JavaScript, StringFunctions and a
> bunch of other options have been endlessly discussed in the past. Does your
> e-mail mean that a decision is made and this can move forward?

In the past week, I've been working on a prototype Lua solution. I
wrote an extension for PHP that embeds a Lua interpreter, with some
extra features (such as CPU time and memory limits) that weren't
present in the previous solutions. I will add support for it to the
existing Lua extension for MediaWiki and add some extra features to
that extension.

When that's done, we'll be able to compare it with WikiScripts in
terms of performance and flexibility and to choose the most suitable
solution.

I discussed WikiScripts extensively with Victor at Wikimania. At the
start of the conference week, I was very keen on the idea, but as the
week went on and I did more research, I became less keen on it and
more keen on Lua. I told Victor that this was the situation, and that
deployment of WikiScripts would depend on its benchmark performance
and other criteria.

I'm very anxious about getting this right. I think the current
situation with template programming on Wikimedia wikis is terrible,
and I think it's largely my fault. The language is ugly and
inaccessible, causing metatemplate editing to be confined to a small
group of elite editors, and it's slow, causing save times of 40
seconds or more on popular articles. Perhaps by supporting the right
solution here, I can gain some absolution.

https://bugzilla.wikimedia.org/show_bug.cgi?id=26786#c24

http://www.mediawiki.org/wiki/Extension:Lua

-- Tim Starling


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: WikiScripts [ In reply to ]
On 19/08/11 04:50, Tim Starling wrote:
<snip>
> I'm very anxious about getting this right. I think the current
> situation with template programming on Wikimedia wikis is terrible,
> and I think it's largely my fault. The language is ugly and
> inaccessible, causing metatemplate editing to be confined to a small
> group of elite editors, and it's slow, causing save times of 40
> seconds or more on popular articles. Perhaps by supporting the right
> solution here, I can gain some absolution.

At the time you implemented the templates and parserfunctions, the
community gave you a huge "thank you!". It was probably better to have
barely usable feature than no feature at all :-)

When you wrote your code, I am sure you never thought the community will
push your system to its limit. So there is nobody to blame you on that.

For the good part, the functions lowered the pressure imposed to the few
core devs by the community since YOU enabled them to create new features!

We probably should have created PHP extensions for the most used
templates. The enwiki {{citation}} template comes to mind.


Just to help you on the way to absolution, please remember your day is
in roughly 2 months!!!
http://en.wikipedia.org/wiki/Wikipedia:Tim_Starling_Day

--
Ashar Voultoiz


_______________________________________________
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l