Mediawiki collection extension installation: Difference between revisions

The educational technology and digital learning wiki
Jump to navigation Jump to search
 
(25 intermediate revisions by the same user not shown)
Line 28: Line 28:
* persisted, loaded and shared
* persisted, loaded and shared
* rendered as PDF (see [http://www.mediawiki.org/wiki/Extension:PDF_Writer Extension:PDF_Writer])
* rendered as PDF (see [http://www.mediawiki.org/wiki/Extension:PDF_Writer Extension:PDF_Writer])
* exported as ODF Text Document (see  [http://www.mediawiki.org/wiki/Extension:OpenDocument_Export Extension:OpenDocument_Export])
* exported as ODF Text Document (see  [http://www.mediawiki.org/wiki/Extension:OpenDocument_Export Extension:OpenDocument_Export]) - not sure
* exported as DocBook XML (see [http://www.mediawiki.org/wiki/Extension:XML_Bridge Extension:XML_Bridge])
* exported as DocBook XML (see [http://www.mediawiki.org/wiki/Extension:XML_Bridge Extension:XML_Bridge]) - not sure
* ordered as a printed book at http://pediapress.com/
* ordered as a printed book at http://pediapress.com/


There are two major levels of installation/configuration
There are two major levels of installation/configuration
* Just install the Mediawiki extension and use PediaPress free rendering service (ok for low volume wikis)
* Just install the Mediawiki extension and use PediaPress free rendering service (ok for low volume wikis). As of Sept. 13 2018 this service is disabled, but should be back at some point after some major upgrading work.
* Install your own rendering services (that requires much more work)
* Install your own rendering services (that requires much more work and you will have to survive the documentation jungle)


This page includes some centralized help links and installation tips made for our own use - [[User:Daniel K. Schneider|Daniel K. Schneider]] 16:31, 4 May 2009 (UTC).
This page includes some centralized help links and installation tips made for our own use in May 2009, updated several times since then. Latest attempt was to get this working again for MW 1.31 (sept. 2018). After upgrading the service stopped working, but the issue is probably related to messy installation of python libraries. We sort of fixed this.


See also:
See also:
* [[Wiki book]]
* [[Wiki book]]
* [[Mediawiki2latex]]. As of 2019-2020, this is the easiest solution. In particular if your wiki doesn't use complex templates that are different from Wikipedia and if you also like producing ODF files.
* [[Help:Books]] (for collection authors and users)
* [[Help:Books]] (for collection authors and users)
* [[Mediawiki]] for a short list of useful extensions.
* [[Mediawiki]] for a short list of useful extensions.
* [http://edutechwiki.unige.ch/mediawiki/index.php?title=Mediawiki_collection_extension_installation&oldid=41347 old version including some Solaris tips] that I removed since outdated.
* [http://edutechwiki.unige.ch/mediawiki/index.php?title=Mediawiki_collection_extension_installation&oldid=41347 old version including some Solaris tips] that I removed since outdated.
== Related News ==
Our current PDF creator installation (feb 2019) has trouble rendering images that sit in tables and galleries and that are part of books. Curiously, generating individual files does seem to work like before.
As of Feb 2019, there seems to be some progress with respect to creating a new architecture.
* Read [https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality Reading/Web/PDF Functionality]
There is an alternative, which as of Feb 7 does not work with wiki books in our wiki since we use different than wikipedia URL's
* https://de.wikibooks.org/wiki/Benutzer:Dirk_H%C3%BCnniger/wb2pdf
* We wrote a [https://de.wikibooks.org/wiki/Benutzer_Diskussion:Dirk_H%C3%BCnniger/wb2pdf#Wikis_mit_%22nice%22_ULRs notice in the discussion page], and we hope that the author is willing to fix this.
There are several interesting features: It can create epub and ODT, there is a way to handle templates (not tested), Latex equations do work, not surprising :). On could export the Latex and then hand tune.


== Help pages ==
== Help pages ==


Documentation entry page at PediaPress
Documentation for the collection extension and the servers doing the work in the background is very chaotic and depending on the solution you will adopt...
 
As far as Wikipedia is concerned, there are some  [https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality news] (sept. 2018). In the past few years there have been attempts to replace the old (and reliable) solution with [https://www.mediawiki.org/wiki/Offline_content_generator Offline content generator] (also [https://wikitech.wikimedia.org/wiki/OCG here]), but that project seems to have become a failure.
 
'''Documentation for mw-lib''' (the old but still working solution)
 
Documentation entry page at PediaPress:
* [http://pediapress.com/code/ Pediapress.com/code]
* [http://pediapress.com/code/ Pediapress.com/code]


Line 168: Line 187:
== mwlib and servers installation on Ubuntu ==
== mwlib and servers installation on Ubuntu ==


Tested in this server, using Ubuntu 10.04.4 LTS, Ubuntu 12.04.2 LTS and Ubuntu 14.04 LTS(server editions). '''Instructions below are only valid for Ubuntu 14 LTS. I just tested this again on a fresh server - [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]])
Tested in this server, using Ubuntu 10.04.4 LTS, 12.04.2 LTS, 14.04 LTS, 16.01 LTS (server editions). '''Instructions below are only valid for Ubuntu 14 LTS and have been more or less updated for 16 LTS and 18 LTS. [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]])


'''Notice''': mwlib now has been replaced by [https://www.mediawiki.org/wiki/Offline_content_generator OCG] on Wikipedias (at least that's what they say). However, OCG seems to be less stable so far and maybe requires even more installation efforts. That being said, I should at some point migrate and find time for testing - [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]]) 12:30, 13 April 2015 (CEST)
'''Notice''': In the documentation, mwlib has been replaced in 2015 by [https://www.mediawiki.org/wiki/Offline_content_generator OCG] on Wikipedias (at least that's what they say). However, OCG seems to be less stable so far and maybe requires even more installation efforts. However, as of sept. 2018, this "new" product that we actually never used, seems to be dead. So, until Pediapress comes up with a new solution, we will stick to mw-lib - [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]]) 11:22, 13 September 2018 (CEST)


Since my first install unil the time of writing (dec 2012 / august 2013 / April 2015), the [http://mwlib.readthedocs.org/en/0.13.2/index.html manual] at PediaPress is fine and should include everything a system person needs to know in order to understand and configure the service. However, the installation instructions are not good enough. The pip installer will find Python packages but cannot add missing Ubuntu libraries.
Since my first install unil the time of writing (dec 2012 / august 2013 / April 2015), the [http://mwlib.readthedocs.org/en/0.13.2/index.html manual] at PediaPress is fine and should include everything a system person needs to know in order to understand and configure the service. However, the installation instructions are not good enough. The pip installer will find Python packages but cannot add missing Ubuntu libraries.
Line 184: Line 203:
* if you don't have a fresh Ubuntu machine, you may have lot's of older junk in your python library. Be prepared to uninstall and upgrade older libraries. In my Ubuntu they sit here:
* if you don't have a fresh Ubuntu machine, you may have lot's of older junk in your python library. Be prepared to uninstall and upgrade older libraries. In my Ubuntu they sit here:
: /usr/local/lib/python2.7/dist-packages (formerly, for 10.04, python 2.6)
: /usr/local/lib/python2.7/dist-packages (formerly, for 10.04, python 2.6)
* After upgrading to Ubuntu 18.01 LTS, there are now duplicates and I will have to fix this - [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]]) 11:22, 13 September 2018 (CEST)
* As mentioned above, you will have to install a whole bunch of packages, but on a fresh Ubuntu machine this should be really easy. The 14LTS install on April 10 2015 went quickly and rather smoothly.
* As mentioned above, you will have to install a whole bunch of packages, but on a fresh Ubuntu machine this should be really easy. The 14LTS install on April 10 2015 went quickly and rather smoothly.


Line 200: Line 220:
  pip install --upgrade pip
  pip install --upgrade pip


In case the Ubuntu pip version doesn't work, you could install a fresh one (this happened to me sometimes in the past)
In case the Ubuntu pip version doesn't work, you could install a fresh one (this happened to me sometimes in the '''past''')
  git clone https://github.com/pypa/pip.git
  git clone https://github.com/pypa/pip.git
  cd pip
  cd pip
Line 234: Line 254:
'''Install Ubuntu packages'''
'''Install Ubuntu packages'''


Below is a complete list of the Ubuntu packages I installed. Some is already on the system, but that's neither a problem for apt-get nor for your server. If you get error messages when you install mwlib (see below), then it is likely that some packages are missing....
Below is a complete list of the Ubuntu packages I installed sometimes in the past. Some is already on the system, but that's neither a problem for apt-get nor for your server. If you get error messages when you install mwlib (see below), then it is likely that some packages are missing or that you got doubles installed (from non-standard setup procedures)


  sudo apt-get install -y gcc \
  sudo apt-get install -y gcc \
Line 272: Line 292:


=== Install ===
=== Install ===
(for Ubuntu 14)
(written for Ubuntu 14, but should also be ok for Ubunt 18)


  pip install -i http://pypi.pediapress.com/simple/ mwlib
  pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ mwlib
  : This will take some time, since files have to be compiled
  : This will take some time, since files have to be compiled
  pip install -i http://pypi.pediapress.com/simple/ mwlib.rl
  pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ mwlib.rl


For each missing piece, try to upgrade, e.g. for apipkg:
For each missing piece, try to upgrade, e.g. for apipkg:
Line 284: Line 304:


After fixing missing dependencies, e.g. a missing image library, '''you may have to force upgrade'''. Successfully installed python packages that will break during runtime, will not upgrade without --force-reinstall.
After fixing missing dependencies, e.g. a missing image library, '''you may have to force upgrade'''. Successfully installed python packages that will break during runtime, will not upgrade without --force-reinstall.
  pip install --upgrade --force-reinstall -i http://pypi.pediapress.com/simple/ pil
  pip install --upgrade --trusted-host pypi.pediapress.com --force-reinstall -i http://pypi.pediapress.com/simple/ pil
  pip install --upgrade --force-reinstall -i http://pypi.pediapress.com/simple/ mwlib
  pip install --upgrade --trusted-host pypi.pediapress.com --force-reinstall -i http://pypi.pediapress.com/simple/ mwlib
  pip install --upgrade --force-reinstall -i http://pypi.pediapress.com/simple/ mwlib.rl
  pip install --upgrade --trusted-host pypi.pediapress.com --force-reinstall -i http://pypi.pediapress.com/simple/ mwlib.rl
 
== Upgrades ==


=== Upgrades ===
'''Gevent problem'''
( tested to Ubuntu 16 / Debian 8 and MW 1.27 )


Make sure that Python and its modules are upgraded, e.g. I had to
gevent.coros was missing, mw-lib used a dead module of gevent :( So I had to backtrack
pip install --upgrade gevent --force-reinstall
Actually this still did not fix the problem, since gevent.coros was missing, mw-lib used a dead module of gevent :( So I had to backtrack
  pip install gevent==1.0.2
  pip install gevent==1.0.2
Some Ubuntu 16 LTS problems (you can most likely skip these)


Then I encountered another ugly problem: requirements.py fails. Edit the file and make a change
Then I encountered another ugly problem: requirements.py fails. Edit the file and make a change
Line 303: Line 324:
  new: MARKER_EXPR = originalTextFor(MARKER_EXPR(""))("marker")
  new: MARKER_EXPR = originalTextFor(MARKER_EXPR(""))("marker")
Warning: This is not a good idea, but it's what I found.
Warning: This is not a good idea, but it's what I found.
=== Upgrade to Ubuntu 18 ===
Some python libraries did change place or could be outdated. I suggest uninstalling and reinstalling.
sudo -H pip uninstall lxml
sudo pip uninstall greenlet
Then install them again. As you will see, some of these are already there...
sudo -H pip install lxml
sudo -H pip install greenlet


(thanx to [[User:Kghbln|Kghbln]] !)
(thanx to [[User:Kghbln|Kghbln]] !)
Line 309: Line 342:
     pip install --trusted-host pypi.pediapress.com --upgrade --force-reinstall -i http://pypi.pediapress.com/simple/ pil
     pip install --trusted-host pypi.pediapress.com --upgrade --force-reinstall -i http://pypi.pediapress.com/simple/ pil
Step 2 - upgrade "mwlib"
Step 2 - upgrade "mwlib"
    sudo su
     pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ --upgrade mwlib
     pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ --upgrade mwlib
Step 3 - upgrade "mwlib.rl"
Step 3 - upgrade "mwlib.rl"
Line 342: Line 376:
  xpdf test.pdf
  xpdf test.pdf


Try it on yours:
Try it on yours (however, '''this does not work anymore''' when I tried last time (Ubuntu 18, sept. 2018)
  mw-zip -c http://edutechwiki.unige.ch/mediawiki/ -o test2.zip Mediawiki_collection_extension_installation
  mw-zip -c http://edutechwiki.unige.ch/mediawiki/ -o test2.zip Mediawiki_collection_extension_installation
  mw-render -c test2.zip -o test2.pdf -w rl
  mw-render -c test2.zip -o test2.pdf -w rl
Line 426: Line 460:


'''Killing jobs launched from a terminal'''
'''Killing jobs launched from a terminal'''
jobs
 
jobs
  [1]  Running                nserve.py &  (wd: /data/portails/mediawiki)
  [1]  Running                nserve.py &  (wd: /data/portails/mediawiki)
  [2]  Running                mw-qserve &  (wd: /data/portails/mediawiki)
  [2]  Running                mw-qserve &  (wd: /data/portails/mediawiki)
Line 432: Line 467:
  [4]+  Running                postman.py --cachedir /data/mwcache &
  [4]+  Running                postman.py --cachedir /data/mwcache &


kill %1
kill %1
kill %2
kill %2
etc.
etc.


=== Installing a script ===
=== Installing a script ===
Line 569: Line 604:


(1) Readjust some image sizes
(1) Readjust some image sizes
I have the suspicion that this extension cannot handle pictures that are typical for cell phones, e.g. 1 840 × 3 552 pixels (4K). Try reducing the size of the uploaded image.


(I'll have to be more precise about this, but I'll first need to analyze both a wiki book I got and the generated PDF...)
(I'll have to be more precise about this, but I'll first need to analyze both a wiki book I got and the generated PDF...)
Line 579: Line 616:
* To select colors that show on gray printers, you may use a color scheme designer such as [http://colorschemedesigner.com/ colorschemedesigner.com] and simulate color vision deficiency with "full colorblindness". Else print your drawing, before you import to wiki.
* To select colors that show on gray printers, you may use a color scheme designer such as [http://colorschemedesigner.com/ colorschemedesigner.com] and simulate color vision deficiency with "full colorblindness". Else print your drawing, before you import to wiki.
* A radical solution is to use gray images already upfront in the wiki. This way you are sure to get it somewhat right. Note: "grey" is spelled "gray" in CSS and X11.
* A radical solution is to use gray images already upfront in the wiki. This way you are sure to get it somewhat right. Note: "grey" is spelled "gray" in CSS and X11.
(3) Galleries may not work.
Workaround: Use tables instead. However, the latter also may encounter problems
But before that try using widths=.... and heights=.... parameters. It seems (as of MW 1.27 / April 2018) that simple galleries with these parameters do work, unless they are too large or include a ''perrow'' attribute. Try creating several galleries instead of using perrow ?
Always use the '''file:''' (or '''fichier:''', in french) prefix. Without it, it will ignore the line with the picture inside.


=== Stuff that the parser doesn't handle well ===
=== Stuff that the parser doesn't handle well ===
Line 584: Line 629:
As of mid May 2009 (so this may change)
As of mid May 2009 (so this may change)


* Some extensions (like graphwiz) are not supported, see above. There is no solution for this, except not using these.
* Some extensions (like graphwiz) are not supported, see above. We use the following workaround: Put the graphwiz code in the discussion page and edit there. Then just link to the image (click on the smal "i" on bottom right of the picture in the discussion page). Also make sure to use large fonts, e.g. Size 24. The graph sitting in the page can be made smaller.
Exemple (discussion page [[:fr:Stitch Era - logiciel de broderie machine et de hotfix]])
<source lang="XML">
<graphviz border="frame" caption="Flux pour créer un design de broderie avec Stitch Era" alignment="left" location="none">
graph SE_workflow_francais_2 {
labelfontname = "Arial";
node [fontsize="24", fontname="Arial"];
edge [fontsize="22", fontname="Arial", labelfloat=false];
 
...
</source>
Example link:
<nowiki>
[[File:Discussion Stitch Era logiciel de broderie machine et de hotfix graph SE workflow francais 2 dot.png|900px|thumb|none|Modèle workflow pour créer une broderie avec Stitch Era]]
</nowiki>


* Some extensions that are not supported wont' matter, e.g. pageby. But you will have to modify the source code for filtering if you own your own server (see above) or file a request.
* Some extensions that are not supported wont' matter, e.g. pageby. But you will have to modify the source code for filtering if you own your own server (see above) or file a request.
Line 594: Line 653:
=== Conditional inclusion/exclusion ===
=== Conditional inclusion/exclusion ===


'''This section is outdated, needs to be adjusted to using CSS''' - [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]]) 12:27, 26 August 2013 (CEST)
Use the following CSS classes in a wiki page. You also can use this within templates. E.g. an example is the in [https://edutechwiki.unige.ch/fr/Mod%C3%A8le:Tutoriel tutoriel template] of the french sister wiki.


'''Warning''': This wasn't tested with Pediapress. If you plan to buy a book, use these on the very first pages, then have a look at the preview.
'''Only in print'''
<source lang="HTML5">
<div class="onlyinprint">
...
</div>
</source>


'''1) Exclude templates'''
'''Hide in print'''


You may exclude any template from the PDF generation you wish by adding them to the [[:Category:Exclude in print]]. Use with care, since this will filter for all users !
<source lang="HTML5">
<div class="noprint">
.....
</div>
</source>


'''2) Exclude certain specific content'''
You should modify the CSS of your wiki in [[MediaWiki:Common.css]] and the following line:


By using [[Template:Hide in print]], certain specific content, such as a few words or an image, can be excluded from printing.
<source lang="CSS">
 
/* collection extension */
<code>This content will be printed.<nowiki>{{Hide in print|This content will not be printed.}}</nowiki>This content will be printed.</code>
.onlyinprint {display: none}
 
</source>
Alternative solution: use the class="noprint" within a div or span tag.
 
'''3) Include certain specific content only in print versions'''
The [[Template:Only in print]] can be used to insert content that shall only be visible in offline versions.
 
Example: <code>print this and display it in the browser <nowiki>{{Only in print|this is only in PDFs or printed books visible, not with the browser}}</nowiki> this is visible in the browser and in print as well.</code>
 
Alternative solution: use the class="hidden" within a div or span tag.


'''4) Substitute templates'''
'''Excluding page sections'''


You can create a print version of a template under the name "TEMPLATENAME'''/Print'''" with TEMPLATENAME being the name of the original template.
Placing a div before a == title == does not seem to work.


(more to come ....) - [[User:Daniel K. Schneider|Daniel K. Schneider]] 12:54, 20 May 2009 (UTC)
Workaround: Use <nowiki><h2></nowiki> instead


[[Category: Installation tips]]
[[Category: Installation tips]]
[[Category:Mediawiki documentation]]
[[Category:Mediawiki documentation]]

Latest revision as of 13:32, 14 February 2020

Category:MediaWiki extension
Mediawiki collection extension installation
Extension name Collection
About this article / disclaimer
Logo Placeholder.png
Screenshot Placeholder.png
Location of the main author Germany
Coordinates of the main authors 51.163818, 10.447831
Developers PediaPress.com, Siebrand Mazeland, Marcin Cieślak
Licences GPL
Description This extension makes it possible to collect a number of pages. Collections can be edited, persisted and optionally retrieved as PDF, ODF or DocBook (XML)
Mediawiki requirements Version 1.17 and later
Dependencies
Related extensions (documented here)
Related extensions [[Is related to information::PDF Writer, Open Document Export]]
Discussion There are two ways of using this extension.
  • Easy: Install the extension and use the PediaPress collection and rendering services
  • Install your own collection and rendering services
Language support Many
Status stable
First release date
Last release date (as of !)
Last version number
Programming language PHP, Python
Alternatives
Website home page
Publications
Support websites web site, web site
Example websites
Last edited

[UPDATE: Please also read the discussion page ! Both Debian/Ubuntu upgrades and MW 1.27 upgrades do require changes that are not documented here] - Daniel K. Schneider (talk) 21:54, 13 September 2016 (CEST)

Introduction

The Mediawiki collection extension allows a user to organize personal selections of pages in a collection. For an educational or academic wiki, this is a must-have extension. E.g. you could use to create print manuals and textbooks or also to valorize student productions at the end of year (hey "daddy" we made a book...)

Collections can be:

There are two major levels of installation/configuration

  • Just install the Mediawiki extension and use PediaPress free rendering service (ok for low volume wikis). As of Sept. 13 2018 this service is disabled, but should be back at some point after some major upgrading work.
  • Install your own rendering services (that requires much more work and you will have to survive the documentation jungle)

This page includes some centralized help links and installation tips made for our own use in May 2009, updated several times since then. Latest attempt was to get this working again for MW 1.31 (sept. 2018). After upgrading the service stopped working, but the issue is probably related to messy installation of python libraries. We sort of fixed this.

See also:

Related News

Our current PDF creator installation (feb 2019) has trouble rendering images that sit in tables and galleries and that are part of books. Curiously, generating individual files does seem to work like before.

As of Feb 2019, there seems to be some progress with respect to creating a new architecture.

There is an alternative, which as of Feb 7 does not work with wiki books in our wiki since we use different than wikipedia URL's

There are several interesting features: It can create epub and ODT, there is a way to handle templates (not tested), Latex equations do work, not surprising :). On could export the Latex and then hand tune.

Help pages

Documentation for the collection extension and the servers doing the work in the background is very chaotic and depending on the solution you will adopt...

As far as Wikipedia is concerned, there are some news (sept. 2018). In the past few years there have been attempts to replace the old (and reliable) solution with Offline content generator (also here), but that project seems to have become a failure.

Documentation for mw-lib (the old but still working solution)

Documentation entry page at PediaPress:

Help (can also be used for informal bug reports)

Information about the collection extension and related server-side software

Installing the whole suite requires some installation skills, but should go fairly smoothly on any Unix system and should be easy on a Debian-based Linux.

Collection Mediawiki extension installation and tuning

If you own a small MediaWiki and do not plan to customize PDF rendering, all you need is this extension. If you use this service often and/or you plan to tweak, then you also have to install the rendering servers. See the sections below.

Base installation of the extension

The collection extension installs like any other Mediawiki extensions. Really easy with Mediawiki=> 1.14 (Spring 2009).

(1) Installing

Get it from Extension:Collection on Mediawiki.org.

You may try the latest version, however sometimes it doesn't work with your MW installation. E.g. it breaks for MW 1.16.4 on April 20 2011. Get it from GIT

cd extensions
git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Collection.git
git branch -r
git checkout REL1_25

Read:

README.txt

You then can just leave all the defaults and the PDF will be generated by PediaPress. However, if you have a slow server or a high traffic server, you also should install a local render server (read the whole rest of this page). When we first installed this extensions in 2008, the Pediapress server did loose pages due to server overload. By now, this problem should be fixed (not tested ....)

(2) Tuning messages

If you have and old and slow server, I suggest changing the file Collection.i18n.php and change the string (adjust to the power of your server)

'coll-rendering_text' => "<p><strong>Please wait while the document is being generated.
Depending on the size of book you may have to wait 5, 10, 15 minutes or longer.
</strong></p> 
 ......"

'coll-save_collection_text'       => 'Choose a storage location for your book and enter a name:',
(3) Permissions (important !)

If you want users to be able to save and to share collections, add these permission to file Localsettings.php:

$wgGroupPermissions['user']['collectionsaveasuserpage'] = true;
$wgGroupPermissions['user']['collectionsaveascommunitypage'] = true;

or maybe a more restrictive alternative:

$wgGroupPermissions['user']['collectionsaveasuserpage'] = true;
$wgGroupPermissions['sysop']['collectionsaveascommunitypage'] = true;
(4) Templates and categories for "books" pages

You should define the following (language-dependant) templates and categories:

  • Template:Saved_book (Grab a copy from a wikipedia and modify).
  • Category:Books
  • Category:Book tool (not really needed)
(5) Conditional inclusion (after June 2013 version)

For content to be excluded from the PDF, use the CSS class "noprint":

 <div class="noprint">blub noprint</div>

For content to be included in PDFs and excluded in the wiki page, use the following CSS

<div class="onlyinprint">blob only in print</div>

This css class is ignored by the PDF renderer and thus displayed normally in the PDF. Modify your MediaWiki:Common.css file to hide the content in the wiki page.

.onlyinprint {display: none}

Of course, you then could create simple templates, e.g. a user could write

There is a {{hide in print|unprintable}} {{only in print|printable}} word.

Template:Hide in print:

 <includeonly><span class="noprint">{{{1}}}</span></includeonly>

Template:Only in print:

 <includeonly><span class="onlyinprint">{{{1}}}</span></includeonly>


Tweaking the collection extension

Read again

Add in Localsettings.php (if not already done) the rendering engines you will support. In case you also installed a local rendering server, typing mw-render --list-writers will list the ones you installed.

Example setting for Localsettings.php:

$wgCollectionFormats = array(
          'rl' => 'PDF',
          'odf' => 'ODT',
      );

In Localsettings.php, if not already done so, add the name and port of the mwlib server.

$wgCollectionMWServeURL = "http://xxx.yyy:8899";

Extra stuff:

For the license, make sure to give the correct RAW wiki URL. Or if this doesn't work, remove the line and the user will see a URL.

$wgLicenseURL = "http://edutechwiki.unige.ch/mediawiki/index.php?title=EduTech_Wiki:Copyrights&action=raw";

Limit number of articles that a book can contain, e.g.

$wgCollectionMaxArticles = 150;

mwlib and servers installation on Ubuntu

Tested in this server, using Ubuntu 10.04.4 LTS, 12.04.2 LTS, 14.04 LTS, 16.01 LTS (server editions). Instructions below are only valid for Ubuntu 14 LTS and have been more or less updated for 16 LTS and 18 LTS. Daniel K. Schneider (talk)

Notice: In the documentation, mwlib has been replaced in 2015 by OCG on Wikipedias (at least that's what they say). However, OCG seems to be less stable so far and maybe requires even more installation efforts. However, as of sept. 2018, this "new" product that we actually never used, seems to be dead. So, until Pediapress comes up with a new solution, we will stick to mw-lib - Daniel K. Schneider (talk) 11:22, 13 September 2018 (CEST)

Since my first install unil the time of writing (dec 2012 / august 2013 / April 2015), the manual at PediaPress is fine and should include everything a system person needs to know in order to understand and configure the service. However, the installation instructions are not good enough. The pip installer will find Python packages but cannot add missing Ubuntu libraries.

  • In aug 2013, someone made a Setup a render server on Ubuntu 12.04 LTS article on mediawiki.org]. Read that article first, it looks better than mine :) However, since I am not a system administrator, I have to write down what I do, since it helps me understanding and make things go faster for upgrades. In addition, the manual suggest running some kind of virtual Python environment. The solution below is simpler, but probably not as safe.

mwlib refers to a whole series of software that will allow:

  • to extract wikipages to zip files
  • to do your own PDF/etc rendering from those zip's
  • to send the zips to PediaPress for printing

This section describes the easy way, however easy does not mean real easy. Depending on your Ubuntu version and the state of your server, you could suffer:

  • if you don't have a fresh Ubuntu machine, you may have lot's of older junk in your python library. Be prepared to uninstall and upgrade older libraries. In my Ubuntu they sit here:
/usr/local/lib/python2.7/dist-packages (formerly, for 10.04, python 2.6)
  • After upgrading to Ubuntu 18.01 LTS, there are now duplicates and I will have to fix this - Daniel K. Schneider (talk) 11:22, 13 September 2018 (CEST)
  • As mentioned above, you will have to install a whole bunch of packages, but on a fresh Ubuntu machine this should be really easy. The 14LTS install on April 10 2015 went quickly and rather smoothly.

Prerequisites

Have root access and become root (or use 'sudo ...')

Install/check python

Firstly, you must have python 2.7.* installed (usually already done) plus the pip packaging manager (not done on a fresh machine), plus a lot of graphics and text libraries.

apt-get install python
apt-get install python-pip

pip is a tool for installing and managing Python packages, such as those found in the Python Package Index. It's a replacement for easy_install that was used in the past.

In case pip is installed, make sure that you got the latest version

pip install --upgrade pip

In case the Ubuntu pip version doesn't work, you could install a fresh one (this happened to me sometimes in the past)

git clone https://github.com/pypa/pip.git
cd pip
python setup.py install

Dependencies

(this is not complete, see the Ubuntu package list below)

  • Perl => 5
  • g++
  • Latex
    • Also compile texvc in mediawiki/math directory
  • Blahtexml


Install Ubuntu packages

Below is a complete list of the Ubuntu packages I installed sometimes in the past. Some is already on the system, but that's neither a problem for apt-get nor for your server. If you get error messages when you install mwlib (see below), then it is likely that some packages are missing or that you got doubles installed (from non-standard setup procedures)

sudo apt-get install -y gcc \
dvipng\
g++\
git-core\
imagemagick\
libevent-dev\
libfreetype6-dev\
libjpeg-dev\
liblcms-dev\
libxml2-dev\
libxslt-dev\
libz-dev\
make\
ocaml-nox\
pdftk\
ploticus\
python-all-dev\
python-dev\
python-imaging\
python-lxml\
python-pip\
python-virtualenv\
python\
tcl\
texlive-latex-recommended\
tk
webp\

Below some extra python stuff that seems to be needed.

pip install greenlet
pip install gevent

Pillow, (the PIL Python imaging library) seems to be installed from PediaPress.

Install

(written for Ubuntu 14, but should also be ok for Ubunt 18)

pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ mwlib
: This will take some time, since files have to be compiled
pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ mwlib.rl

For each missing piece, try to upgrade, e.g. for apipkg:

 pip install --upgrade apipkg

In addition, you may have to remove packages, see below. You also may have to remove/reinstall mwlib and mwlib.rl if you did something wrong ...

After fixing missing dependencies, e.g. a missing image library, you may have to force upgrade. Successfully installed python packages that will break during runtime, will not upgrade without --force-reinstall.

pip install --upgrade --trusted-host pypi.pediapress.com --force-reinstall -i http://pypi.pediapress.com/simple/ pil
pip install --upgrade --trusted-host pypi.pediapress.com --force-reinstall -i http://pypi.pediapress.com/simple/ mwlib
pip install --upgrade --trusted-host pypi.pediapress.com --force-reinstall -i http://pypi.pediapress.com/simple/ mwlib.rl

Upgrades

Gevent problem

gevent.coros was missing, mw-lib used a dead module of gevent :( So I had to backtrack

pip install gevent==1.0.2

Some Ubuntu 16 LTS problems (you can most likely skip these)

Then I encountered another ugly problem: requirements.py fails. Edit the file and make a change

cd /usr/local/lib/python2.7/dist-packages/packaging/
cp requirements.py requirements.old.py

Edit the requirements.py file and change the following

old: MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker")
new: MARKER_EXPR = originalTextFor(MARKER_EXPR(""))("marker")

Warning: This is not a good idea, but it's what I found.

Upgrade to Ubuntu 18

Some python libraries did change place or could be outdated. I suggest uninstalling and reinstalling.

sudo -H pip uninstall lxml
sudo pip uninstall greenlet

Then install them again. As you will see, some of these are already there...

sudo -H pip install lxml
sudo -H pip install greenlet

(thanx to Kghbln !)

Step 1 - reinstall and upgrade "pil"

   pip install --trusted-host pypi.pediapress.com --upgrade --force-reinstall -i http://pypi.pediapress.com/simple/ pil

Step 2 - upgrade "mwlib"

   sudo su
   pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ --upgrade mwlib

Step 3 - upgrade "mwlib.rl"

   pip install --trusted-host pypi.pediapress.com -i http://pypi.pediapress.com/simple/ --upgrade mwlib.rl

Step 4 - update "Mwlib server daemon"

   per suggestion of the example below

Step 5 - create user "mw-serve" since "www-data" will no longer work

   adduser mw-serve

Step 6 - change permissions of the log directory

   chown mw-serve:mw-serve -R /var/log/mediawiki/

Step 7 - start render servers

   /etc/init.d/mw-serve start

Step 8 - enjoy

Uninstalling old/useless and interfering python packages

Only works with a recent version of pip, e.g. pip-2.6 or better !!

Syntax:

pip uninstall package_name

Sometimes, you may just have to remove packages manually (probably older not finished installs ?)

cd /usr/local/lib/python2.6/dist-packages
rm -r package_name

.... but this is not recommended IMHO

Test

Get something from Wikipedia (this is the default)

mw-zip -c :en -o test.zip Acdc Number
mw-render -c test.zip -o test.pdf -w rl

Look at test.pdf and enjoy ....

xpdf test.pdf

Try it on yours (however, this does not work anymore when I tried last time (Ubuntu 18, sept. 2018)

mw-zip -c http://edutechwiki.unige.ch/mediawiki/ -o test2.zip Mediawiki_collection_extension_installation
mw-render -c test2.zip -o test2.pdf -w rl
xpdf test2.pdf

Running the render servers

On the machine

Firstly, create a directory where files can be collected. Then, make them writable for the process that will run mwlib. For example:

mkdir /data/mwcache
chown www-data:www-data mwcache/

The following will launch the combo. Each program has various options. At least for testing, I suggest to accept the defaults. However, define a cache directory for the nslave.

nserve &
mw-qserve &
# directory mwcache must be writable to the user that runs the servers
nslave --cachedir /data/mwcache &
postman --cachedir /data/mwcache &

Note about older versions

  • In earlier versions (e.g. the ones that installed in 2012), the scripts were called nserve.py, nslave.py, etc. as opposed to nserve, nslave, etc.
  • After upgrading, make damn sure to fix the script !! (I lost like 2 hours with this ....)

In LocalSettings.php of the wiki

You must add the following line, since by default your wiki will use the PediaPress rendering service:

 $wgCollectionMWServeURL = "http://machine.you.domain:port";

For example (using the default port):

$wgCollectionMWServeURL = "http://edutechwiki.unige.ch:8899";

Log information output by the render services

All these servers will write back log information to the console. E.g. if you test creating a simple PDF file you might see something like this (slightly formatted and edited, version with the old file names for scripts.):

=== report Wed Dec 12 14:15:13 2012 ===
have 0 jobs
count: 0
all channels idle

new-collection 1 'http://edutechwiki.unige.ch/mediawiki'	'rl'
2012-12-12T14:15:14 mwlib.serve.info >> render 9da9af29fbb8b623 rl
129.194.xxx- - [2012-12-12 14:15:14] "POST / HTTP/1.0" 200 200 0.012274
129.194.xxx- - [2012-12-12 14:15:14] "POST / HTTP/1.0" 200 215 0.002743
0 1.42332196236 ['mw-zip', '-o', '/data/mwcache/9d/9da9af29fbb8b623/collection.zip', 
 '-m', '/data/mwcache/9d/9da9af29fbb8b623/metabook.json', 
 '--status', 'qserve://localhost:14311/9da9af29fbb8b623:makezip', 
 '--config', 'http://edutechwiki.unige.ch/mediawiki', 
 '--template-blacklist', 'MediaWiki:PDF Template Blacklist', 
 '--template-exclusion-category', 'Exclude in print', 
 '--print-template-prefix', 'Print',
 '--print-template-pattern', '$1/Print']
finish: 9da9af29fbb8b623:makezip: None
0 2.253221035 ['mw-render', '-w', 'rl', '-c', '/data/mwcache/9d/9da9af29fbb8b623/collection.zip', 
 '-o', '/data/mwcache/9d/9da9af29fbb8b623/output.rl', 
 '--status', 'qserve://localhost:14311/9da9af29fbb8b623:render-rl', 
 '--template-blacklist', 'MediaWiki:PDF Template Blacklist', 
 '--template-exclusion-category', 'Exclude in print', 
 '--print-template-prefix', 'Print', '--print-template-pattern', '$1/Print', 
 '--language', 'en']
finish: 9da9af29fbb8b623:render-rl: 
 {'url': 'http://129.194.7.75:8898/cache/9d/9da9af29fbb8b623/output.rl', 
  'suggested_filename': 'Mediawiki collection extension installation',
  'size': 125658}

watchdog: dropped 0 jobs, marked 2 jobs with a deadline
[....]
=== report Wed Dec 12 14:15:33 2012 ===
have 2 jobs
count: 2
all channels idle
[.....]
=== report Wed Dec 12 14:16:53 2012 ===
have 2 jobs
count: 2
all channels idle
129.194.xxx - - [2012-12-12 14:16:59] "POST / HTTP/1.0" 200 565 0.039982
129.194.xxx - - [2012-12-12 14:17:01] "POST / HTTP/1.0" 200 565 0.002620
129.194.xxx - - [2012-12-12 14:17:01] "GET /cache/9d/9da9af29fbb8b623/output.rl HTTP/1.0" 200 125919 0.001031

Killing jobs launched from a terminal

jobs
[1]   Running                 nserve.py &  (wd: /data/portails/mediawiki)
[2]   Running                 mw-qserve &  (wd: /data/portails/mediawiki)
[3]-  Running                 nslave.py --cachedir /data/mwcache &
[4]+  Running                 postman.py --cachedir /data/mwcache &
kill %1
kill %2
etc.

Installing a script

(not yet done, weekend is coming up - Daniel K. Schneider (talk) 19:21, 10 April 2015 (CEST)

You could imagine redirecting logfiles to a log file, running the processes under a better user than root and also writing a start/stop script in case you boot.

In fact, Pediapress suggests using runit for process supervision. daemontools is similar solution. Another alternative is to use supervisor. The advantage of such a solution is that processes are automatically restarted after a process dies (that can happen for various reasons). Installing supervisor is what I personally would attempt since it doesn't seem to interfere much with standard practice from what I could understand and it doesn't look too complicated. Anyhow, in the meantime I made this script (suggestions welcome, I really am not a sysadmin)

Prerequisites:

  • Have a user you can run the processes with. E.g. yourself. I just used www-data  but you probably will like /var/log/mediawiki See this note for Debian 8
  • Make sure that the log file directory is writable by this user
chown mw-serve:mw-serve -R /var/log/mediawiki/

File etc/init.d/mw-serve used on a typical installation

#!/bin/sh

PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin

case "$1" in
        start)
                su - mw-serve -c 'nserve >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
                su - mw-serve -c 'mw-qserve >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
                su - mw-serve -c 'nslave --cachedir /var/log/mediawiki/ >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
                su - mw-serve -c 'postman --cachedir /var/log/mediawiki/ >> /var/log/mediawiki/mw-serve_log.txt 2>&1 &'
        ;;
        stop)
                mv /var/log/mediawiki/mw-serve_log.txt /var/log/mediawiki/mw-serve_log.txt.old
                killall nserve
                killall mw-qserve
                killall nslave
                killall postman
        ;;
        force-reload|restart)
                $0 stop
                $0 start
        ;;
        *)
                echo "Usage: /etc/init.d/mw-serve {start|stop}"
                exit 1
        ;;
esac

exit 0


File /etc/init.d/mw-serve used locally for EduTechWiki

#! /bin/bash

PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin

case "$1" in
    start)
	su - www-data -c 'nserve >> /data/mwcache/log.txt 2>&1  &'
	su - www-data -c 'mw-qserve >> /data/mwcache/log.txt 2>&1 &'
	su - www-data -c 'nslave --cachedir /data/mwcache/ >> /data/mwcache/log.txt 2>&1 &'
	su - www-data -c 'postman --cachedir /data/mwcache/ >> /data/mwcache/log.txt 2>&1 &'
    ;;
  stop)
	mv /data/mwcache/log.txt /data/mwcache/log.old
	killall nserve
	killall mw-qserve
        killall nslave
        killall postman
    ;;
  force-reload|restart)
    $0 stop
    $0 start
    ;;
  *)
    echo "Usage: /etc/init.d/mw-serve {start|stop}"
    exit 1
    ;;
esac

exit 0

Make it executable

chmod 755 /etc/init.d/mw-serve

Then link it in the appropriate RC directories, most importantly:

cd /etc/rc1.d
ln -s ../init.d/mw-serve K20mw-serve
cd /etc/rc2.d
ln -s ../init.d/mw-serve S20mw-serve
cd /etc/rc3.d
ln -s ../init.d/mw-serve S20mw-serve

Tweaking

  • For larger servers you need to multiply .... read the official manual
  • For better quality of service, you must run the scripts with a process supervisor software (See the manual).

Tips for creating wiki books

Creating a serious book

It is best just to "hand edit" a stored book. E.g. you may start by adding a category or isolated articles to a book with the collection interface, but once you got most of the articles:

  • Save the book
  • Then select it from the category:books and then just edit the page in the "normal way". The structure and syntax to respect is demonstrated by the following example:
{{saved_book}}
== My Book ==
=== Example ===
;Foo
:[[First article]]
:[[Second article]]
:[[Third article|This article renamed in the book]]
;Bar
:[[Fourth article]]
:[[Fifth article]]
:[{{fullurl:Sixth article|oldid=20}}Sixth article version:20]
[[Category:Books]]

Install the Mediawiki source extension

If you have articles that include computer source code (XML, ActionScipt or whatever), you also should install the source extension (CodeSyntaxHighlight MediaWiki for formatting computer code.

If you use XML within "pre" tags, the parser may become confused.

Also, the printed Pediapress book will look much prettier. Editing all your wiki pages may represent work, but user reading the on-line pages also will be grateful, colored and indented code is really much more readable !

Pictures and drawings

(1) Readjust some image sizes

I have the suspicion that this extension cannot handle pictures that are typical for cell phones, e.g. 1 840 × 3 552 pixels (4K). Try reducing the size of the uploaded image.

(I'll have to be more precise about this, but I'll first need to analyze both a wiki book I got and the generated PDF...)

  • Don't use large pictures when smaller ones are readable
  • Create screendumps from smallest possible areas if you plan to show hairy details (see also the screen capture tutorial).

(2) Color Printed PDF is mostly grey (affordable Pediapress books certainly are). E.g. when you create drawings or annotate images with text, you should make sure that shades of grey still allow to identify critical elements. A related very difficult issue is writing the text. Avoid writing about "blue" and "green" arrows and "green" or "yellow" dots ...

  • To select colors that show on gray printers, you may use a color scheme designer such as colorschemedesigner.com and simulate color vision deficiency with "full colorblindness". Else print your drawing, before you import to wiki.
  • A radical solution is to use gray images already upfront in the wiki. This way you are sure to get it somewhat right. Note: "grey" is spelled "gray" in CSS and X11.

(3) Galleries may not work.

Workaround: Use tables instead. However, the latter also may encounter problems

But before that try using widths=.... and heights=.... parameters. It seems (as of MW 1.27 / April 2018) that simple galleries with these parameters do work, unless they are too large or include a perrow attribute. Try creating several galleries instead of using perrow ?

Always use the file: (or fichier:, in french) prefix. Without it, it will ignore the line with the picture inside.

Stuff that the parser doesn't handle well

As of mid May 2009 (so this may change)

  • Some extensions (like graphwiz) are not supported, see above. We use the following workaround: Put the graphwiz code in the discussion page and edit there. Then just link to the image (click on the smal "i" on bottom right of the picture in the discussion page). Also make sure to use large fonts, e.g. Size 24. The graph sitting in the page can be made smaller.

Exemple (discussion page fr:Stitch Era - logiciel de broderie machine et de hotfix)

 <graphviz border="frame" caption="Flux pour créer un design de broderie avec Stitch Era" alignment="left" location="none">
graph SE_workflow_francais_2 {
labelfontname = "Arial";
node [fontsize="24", fontname="Arial"];
edge [fontsize="22", fontname="Arial", labelfloat=false];

...

Example link: [[File:Discussion Stitch Era logiciel de broderie machine et de hotfix graph SE workflow francais 2 dot.png|900px|thumb|none|Modèle workflow pour créer une broderie avec Stitch Era]]

  • Some extensions that are not supported wont' matter, e.g. pageby. But you will have to modify the source code for filtering if you own your own server (see above) or file a request.
  • &nbsp; prints "as is", therefore avoid! Use ":" and "::" etc. to indent lines for example
  • Source code (either within "pre" or "source" tags that follow a picture will be printed over the picture. Probable reason. The renderer will reduce a picture and try to wrap text around it. Source code cannot be wrapped or not as well. Workaround: Move the image either 20 lines above or after the code.

Conditional inclusion/exclusion

Use the following CSS classes in a wiki page. You also can use this within templates. E.g. an example is the in tutoriel template of the french sister wiki.

Only in print

<div class="onlyinprint">
...
</div>

Hide in print

<div class="noprint">
.....
</div>

You should modify the CSS of your wiki in MediaWiki:Common.css and the following line:

/* collection extension */
.onlyinprint {display: none}

Excluding page sections

Placing a div before a == title == does not seem to work.

Workaround: Use <h2> instead