Lua scripting/status

Last update on: <section begin="latest"/>2013-04-monthly<section end="latest"/>

2011-08-31

<section begin=2011-08-31/>Volunteer Victor Vasiliev worked on a MediaWiki extension to embed scripts into pages; this was a result of discussions over the years about replacing ad-hoc template- and ParserFunctions-based logic by a more efficient and powerful solution. Tim Starling discussed the extension with Victor to become more familiar with his work, and researched other alternatives. He (Tim) wrote a PHP extension embedding a Lua interpreter, and added support for it to the existing Lua MediaWiki extension for backward compatibility.<section end=2011-08-31/>

2011-09-30

<section begin=2011-09-30/>Tim Starling did some prototyping and testing on this project. He created a test harness that can be used to trial the various approaches, and made a Wikitext-to-Lua template converter to test the relative performance. <section end=2011-09-30/>

2011-10-31

<section begin=2011-10-31 />This project was mostly on hold in October.<section end=2011-10-31 />

2012-01-26

<section begin=2012-01-26/>We made a decision to move forward with Lua. Details coming to wikitech-l<section end=2012-01-26/>

2012-01-31

<section begin=2012-01-31/>A team of Wikimedia engineers agreed on Lua as the language to implement as a production-ready replacement for MediaWiki markup-based templates. Tim Starling will lead this effort after the 1.19 deployment and Git migration.<section end=2012-01-31/>

2012-04-05

<section begin=2012-04-05/>Tim Starling is currently working on the replacement of ordinary wikitext markup with Lua and says that he is halfway done, and hopes to finish in the next few weeks.<section end=2012-04-05/>

2012-04-19

<section begin=2012-04-19/>Tim's work continues. The standalone engine is working and Tim is working on a standard library (it's pretty small so far). Lua can't yet call templates. There will be two ways of passing text (Tim will post to wikitech-l later about it). Parser integration features are remaining, and Tim may ask WMF's Features team for help. We'd like to give people a sandbox/console feature so people can play with it (this would be a CodeEditor feature request).

The Lua timeline is part of the site performance timeline. We aim for Scribunto and its Lua engine to start running on a small, simple test wiki in Labs in May, so other people can start to try it out. Then the aim is to get feature complete, deploy to mediawiki.org, leave it there a few months to look for technical problems and to discuss it with the community, and only later deploy to all other wikis. Per the Goals timeline, it will be early 2013 by the time of rollout to WMF sites.

To help prepare for this, Tim will lead tutorial sessions at Berlin Hackathon 2012. There will be an intro to programming in Lua (for people who have programmed in other languages before) and an intro to programming for people who have only ever made wikitext templates. In both cases, he'll show simple examples of how to convert a wikitext template into Lua. At the end, every participant will have actually converted a real MediaWiki template into Lua.<section end=2012-04-19/>

2012-04-monthly

<section begin=2012-04-monthly/>Tim Starling started the implementation of a replacement for MediaWiki markup-based templates, using the Lua scripting language, embedded via the Scribunto extension. The current roadmap aims for a deployment to Labs in May 2012, then to mediawiki.org; full deployment to Wikimedia sites is scheduled for 2013. Tim will lead tutorial sessions at the Berlin Hackathon 2012.<section end=2012-04-monthly/>

2012-05-10

<section begin=2012-05-10/>You can try it now at the Labs prototyping site. Patrick Reilly spoke about the Lua scripting work in a presentation on 3 May 2012 (scroll to about 22:00 or 27:00 to see it).<section end=2012-05-10/>

2012-05-monthly

<section begin="2012-05-monthly"/>Tim Starling's Scribunto extension is now available for testing in Wikimedia Labs. Tim also led tutorial sessions at the Berlin Hackathon 2012.<section end="2012-05-monthly"/>

2012-06-18

<section begin="2012-06-18"/>Tim Starling led tutorial sessions in June and videos (first session, second session) are now available on Vimeo. They will be on Wikimedia Commons by mid-July. Ross Andrews is now working on documentation in the form of help/tutorial pages, especially describing the MediaWiki interface. Once that's done, Tim will promote the prototyping site on Labs more heavily, and at some point after that, we will install the Scribunto extension on mediawiki.org. Full deployment to Wikimedia sites is scheduled for 2013.<section end="2012-06-18"/>

2012-06-monthly

<section begin="2012-06-monthly"/>Tim Starling led tutorial sessions in June and videos (first session, second session) are now available on Vimeo. They will be on Wikimedia Commons by mid-July. Ross Andrews is now working on documentation in the form of help/tutorial pages, especially describing the MediaWiki interface. Once that's done, Tim will promote the prototyping site on Labs more heavily, and at some point after that, we will install the Scribunto extension on mediawiki.org. Work on Lua was paused in late June to catch up on other activities. Full deployment to Wikimedia sites is scheduled for 2013.<section end="2012-06-monthly"/>

2012-07-monthly

<section begin="2012-07-monthly"/>Tim Starling has added a debug console to test code snippets. We believe we're ready to deploy Lua to the WIkimedia cluster, starting with test2 in August, followed by mediawiki.org. We plan to let Lua incubate on mediawiki.org while we test the performance characteristics with key templates, and work out a deployment plan for larger wikis that includes community involvement. <section end="2012-07-monthly"/>

2012-08-monthly

<section begin="2012-08-monthly"/>The Scribunto extension has been deployed to test2.wikipedia.org and www.mediawiki.org, and several editors are porting existing templates such as Cite over to Lua (see recent changes in the "Module:" namespace)<section end="2012-08-monthly"/>

2012-09-06

<section begin="2012-09-06"/>Tim Starling: "I finished work on a basic profiler yesterday, and now I'm adding some time and date functions to the default environment."<section end="2012-09-06"/>

2012-09-monthly

<section begin="2012-09-monthly"/>Tim Starling created a basic profiler to work with Lua code, added some time and date functions to the default environment, and fixed bugs. Experimentation continues with Lua on mediawiki.org and Lua on test2wiki.<section end="2012-09-monthly"/>

2012-10-18

<section begin="2012-10-18"/>This has been on hold with many other small projects and site issues taking priority. Next up for Tim is to gather more profiling data, comparing Lua template performance to Wikitext template performance.<section end="2012-10-18"/>

2012-11-20

<section begin="2012-11-20"/>Brad Jorsch and Chad Horohoe have joined Tim Starling on this project. Brad has built a TemplateSandbox which will help in debugging Lua scripts (and old fashioned templates, too!). Chad is working on a shared repository for scripts, and Tim has been extending the API. His latest work has been around adding multi-lingual APIs for handling things like plurals and such within Lua. We're currently seeking a volunteer product manager to help out with the rollout of this.<section end="2012-11-20"/>

2012-11-monthly

<section begin="2012-11-monthly"/>Brad Jorsch and Chad Horohoe have joined Tim Starling on this project. Brad has built a template sandbox which will help in debugging both Lua scripts and regular templates. Chad is working on a shared repository for scripts, and Tim has been extending the API. His latest work has been around adding multilingual APIs for handling things like plurals within Lua. We're currently seeking a volunteer product manager to help out with the roll-out of this.<section end="2012-11-monthly"/>

2013-01-monthly

<section begin="2013-01-monthly"/>Lua development was put on hold through the Ashburn data center migration. We've now resumed work on Lua, with Brad Jorsch and Tim Starling making more functions available in Lua that are currently already available in template parser functions.<section end="2013-01-monthly"/>

2013-02-monthly

<section begin="2013-02-monthly"/>We deployed Lua/Scribunto to several wikis, including English Wikipedia, on February 18th. The current plan is to deploy to the remaining wikis on March 13th.<section end="2013-02-monthly"/>

2013-03-monthly

<section begin="2013-03-monthly"/>We launched Lua scripting on all wikis, wrote about the launch's wider significance, and held IRC office hours. In March we also added frame:callParserFunction() and frame:extensionTag(), improved CPU time accounting, and allowed argument expansion to be excluded. We have patches outstanding for "text" module including unstrip functionality, as well as improved debug output. We've also made significant improvements to templates since the launch. <section end="2013-03-monthly"/>

2013-04-monthly

<section begin="2013-04-monthly"/>Some bugs were fixed and internationalization changes merged this month; no major changes were made. The community continues to develop Lua-based templates, such as the citation templates on the English Wikipedia.<section end="2013-04-monthly"/>