Pybossa: Difference between revisions

The educational technology and digital learning wiki
Jump to navigation Jump to search
No edit summary
mNo edit summary
 
(18 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Citizen science software
{{Citizen science software
|field_project_name=Pybossa
|field_project_name=Pybossa
|field_logo=Pybossa.jpg
|field_project_access_URL=http://pybossa.com/getting-started/
|field_logo=Pybossa.png
|field_programming_languages=Python
|field_programming_languages=Python
|field_software_license=Free open source
|field_software_license=Free open source
|field_project_description=According to the official documentation (April 2013), “PyBossa is an open source platform for crowd-sourcing online (volunteer) assistance to perform tasks that require human cognition, knowledge or intelligence (e.g. image classification, transcription, information location etc). PyBossa was inspired by the BOSSA crowdsourcing engine but is written in python (hence the name!). It can be used for any distributed tasks application but was initially developed to help scientists and other researchers crowd-source human problem-solving skills!”  
|field_project_description=According to the official documentation (April 2013), “PyBossa is an open source platform for crowd-sourcing online (volunteer) assistance to perform tasks that require human cognition, knowledge or intelligence (e.g. image classification, transcription, information location etc). PyBossa was inspired by the BOSSA crowdsourcing engine but is written in python (hence the name!). It can be used for any distributed tasks application but was initially developed to help scientists and other researchers crowd-source human problem-solving skills!”  
Currently (April 2013), pyBossa is functional Beta. New features, new templates, code snippets, and more documentation will be provided.
Currently (April 2015), pyBossa is fully functional.  
 
[http://crowdcrafting.org/ Crowdcrafting] provides a hosting service.
|field_purpose_of_project=PyBossa is there to help researchers, civic hackers and developers to create projects where anyone around the world with some time, interest and an internet connection can contribute. It’s a 100% open-source Unlike, say, “mechanical turk” style projects, PyBossa is not designed to handle payment or money — it is designed to support volunteer-driven projects.
|field_purpose_of_project=PyBossa is there to help researchers, civic hackers and developers to create projects where anyone around the world with some time, interest and an internet connection can contribute. It’s a 100% open-source Unlike, say, “mechanical turk” style projects, PyBossa is not designed to handle payment or money — it is designed to support volunteer-driven projects.
|field_volonteer_computing=no
|field_volonteer_thinking=yes
|field_volonteer_sensing=yes
|field_volonteer_gaming=somewhat
|field_system_overview=pyBossa system (like Crowdcrafting.org) has three main components:  
|field_system_overview=pyBossa system (like Crowdcrafting.org) has three main components:  
The BOSSA core  
* The BOSSA core  
A task presenter, i.e. a HTML5 frontend that will load the task data into the DOM of a web page. Task Presenters can be either HTML5/JavaScript code that sits in the pyBossa server or they can be made with any language communicating through the API.  
* A task presenter, i.e. a HTML5 frontend that will load the task data into the DOM of a web page. Task Presenters can be either HTML5/JavaScript code that sits in the pyBossa server or they can be made with any language communicating through the API.  
A task creator, i.e. a script that will upload tasks for the application to the BOSSA core.  
* A task creator, i.e. a script that will upload tasks for the application to the BOSSA core.  
There are two approaches for creating pyBossa applications:  
There are two approaches for creating pyBossa applications:  
Using the web interface of a pyBossa server  
* Using the web interface of a pyBossa server  
Using the REST API  
* Using the REST API  
You can create applications that use existing templates with different types of data. Else, you must learn how to create applications with Python. You will have to install a virtual Python environment in your machine, code, create the application that then can be "pushed" into a pyBossa server.
You can create applications that use existing templates with different types of data. Else, you must learn how to create applications with Python. You will have to install a virtual Python environment in your machine, code, create the application that then can be "pushed" into a pyBossa server.
|field_screenshot=Pybossa-app1.png
|field_screenshot=Pybossa-app1.png
|field_screenshot_description=Build application with Pybossa. Result of a test.
|field_screenshot_description=Build application with Pybossa. Result of a test.
|field_completion_level=Low
|field_has_project_size=100
|field_last_edition=2013/09/09
|field_team_leadermm=Daniel Lombraña
|field_team_link=http://scifabric.com/team/
|field_contact_person=Clara Sánchez-Puga
|field_contact_person_URLs=clara@scifabric.com
|field_team_location=Spain
|field_completion_level=Medium
|field_last_edition=2015/11/09
}}
}}
{{Documentation
|field_documentation_name=pybossa test
|field_documentation_link=pybossa test link
|field_official=No
|field_documentation_type=Book
}}
== Creating a simple application ==
=== Create an account and start a project ===
* Create an account on http://crowdcrafting.org/
* Click on the '''Create''' button on top menu to create an application
: On a first and second screen, fill in the fields that will describe the application. You are allowed to change these later.
You can find your applications in your "personal" pull down menu.
Name -> My Profile
=== Select items you will work with ===
As of Nov 2015, there are 4 options, i.e. objects that you can you can present to the users for some pattern recognition task
* images
* sounds
* videos
* PDFs
E.g. select images
=== Define the type of task ===
There are three options:
* Classify
* Describe
* Count
E.g. select classify
=== Define a question ===
* Make sure to type an answer for each field (prefilled content does not work)
* You can add response items
Next, add another question or two.
=== Add a source ===
You can choose between Dropbox and Flickr. Both provide you with simliar features and a test project can be built with either one. Using Flickr is probably a bit easier.
Make sure to add all pictures you need. Click on all pictures you want to import.
To add / change pictures, you can
* MyProfile -> Select your project
* click on <code>Tasks</code>
* Click on <code>Import</code>
The documentation as nov Nov 2015, is not totally in synch. E.g. there is no need to create a task presenter as it is [http://docs.pybossa.com/en/latest/user/overview.html described in the quickstart manual]
== Customizing simple projects ==
There are several methods for customizing a project. Firstly you could add more tasks either using the Flickr/DropBox interface or using the CSV or JSON taks importer.
=== Modify tasks ===
If you are just testing it is best to delete existing tasks. Since tasks are identified by I don't know what, PyBossa will refuse to reimport tasks that already have been defined, even in a different way)
To do so
Settings -> Delete tasks
'''Export to CSV and edit in spreadsheet'''
This allows for example to manually add pictures and also to change URLs. However, you carefully must respect custom fields that are made for each different type of presenter.
Tasks -> Export
* Click on Download CSV. This will generate a zip file with a csv file, that you then can open in Google Docs (or any texteditor or spreadsheet application)
After making changes, you can import the CSV from various sources (Google Drive, URLs, DropBox)
'''Edit a JSON File'''
For people with a little computer science background, editing a JSON file is an alternative.
Export your tasks
Task -> Export
* Download JSON
'''Import of a modified task file'''
For a CSV:
Tasks -> Import
Use either Google Drive or DropBox.
For the JSON:
Use the API (more later ...)
=== Change presentation of tasks ===
== Old stuff ==
(needs to be checked)
=== pyBossa templates ===
pyBossa templates are reusable applications templates that implement popular crowd crafting problems. Templates include: a ''task creator'', a ''task presenter'', a ''tutorial'', an ''application description, etc.
As of April 2013, there are five templates available on the crowdsourcing.org server. You can adapt these to your own needs in various ways.
{| class="wikitable"
|-
! Template !! Task template !! Description
|-
| EpiCollect Plus Project || EpiCollect Plus || Interface to an [http://plus.epicollect.net/ EpiCollect] project
|-
| Geo-coding || Urban Park Task || Allows users to tag a park in a list of cities of the whole world. The result is then shown in a 3D Globe (rendered with WebGL)
|-
| Image Pattern Recognition || Flickr Person Finder || Uses the Flickr web service as the source of the data, pulls a picture from a Flick Feed and then asks the user a question with several possible answers.
|-
| Transcribing documents || PDF transcription || An application that allows transcribing PDF documents (e.g. old texts)
|-
| Sound Pattern Recognition || SoundCloud || ...
|}
In addition, there are two special templates allowing to import data from CSV files or Google Drive Spreadsheets. In order to use that data, you will have to write the code for a presenter.
{| class="wikitable"
|-
! Template !! Task template !! Description
|-
| CSV || Public link to a CSV file || ?
|-
| Google Drive Spreadsheet || Import a Google Drive Spreadsheet || ?
|}
All PyBossa templates are available in [https://github.com/PyBossa GitHub repository], e.g. you can add these to your own pyBossa server or modify the code and push to a pyBossa server.
==== Data exports ====
Both tasks and task runs can be exported either in CSV or JSON format for analysis.
=== Adapting the built-in Image pattern recognition template ===
This example will show how to adapt the built-in ''Image pattern recognition'' using the public pyBossa sever.
Example that you only can view once, unless you connect from a different IP. It is really just a test, e.g. only includes three pictures of the 3D printers I own ....
* http://crowdcrafting.org/app/3D_printers_demo/
Below, we summarize the steps.
==== Create an account and an application ====
* Create an account on http://crowdcrafting.org/
* Click on the '''Create''' button on top menu to create an application
: Fill in 8 fields that will describe the application. You are allowed to change these later.
You can find your applications in your "personal" pull down menu.
Name -> My Profile
==== Select an existing application template ====
In the '''Settings''' for your application (personal pull-down menu -> My applications):
* Click on ''Import tasks'' and  select the '''Image Pattern Recognition''' template
You now will see a link to a Google spreadsheet and that you can substitute by your own. Therefore, do not use this link, create your own as explained below.
* Go to [https://drive.google.com/ Google Drive] and log in. If you don't have a Coogle account you will have to create one.
* Open the [https://docs.google.com/spreadsheet/ccc?key=0AsNlt0WgPAHwdHFEN29mZUF0czJWMUhIejF6dWZXdkE#gid=0 this spreadsheet]
* Create a copy and ''share it'' on your own google drive. With the open file, use menu File->Share.
* Replace all the pictures and maybe also the questions. Do not change the headings, since the presenter will use these.
: col1 = question
: col2 = thumbnail picture
: col3 = credits page
: col4 = full picture
... of course you can link to any picture from any site. E.g. we took ours from this wiki:
==== Create/Modify the task presenter ====
Pick the ''Image recognition task'' (the others will not work with the template)
You now can change the number and the phrasing of the answers. As you can see below, we added an answer:
<source lang="java">
<!-- If the user clicks this button, the saved answer will be value="maybe"-->
<button class="btn btn-answer"
        value='Maybe'><i class="icon icon-white icon-comment-alt"></i>Maybe</button>
</source>
'''Adapting the buttons'''
PyBossa uses the [http://twitter.github.io/bootstrap/index.html Twitter bootstrap] CSS framework.
* ''icon-comment-alt'' refers to a [http://fortawesome.github.io/Font-Awesome/ special font] that was made for this framework. In principle, you can use any of these font names from [http://fortawesome.github.io/Font-Awesome/design.html this list].
* Same principle for the button color styles, e.g. ''btn_answer'' would be light grey. See the [http://twitter.github.io/bootstrap/base-css.html#buttons Twitter bootstrap buttons list]
Original fragment:
<source lang="java">
<div id="answer"> <!-- Start DIV for the submission buttons -->
<!-- If the user clicks this button, the saved answer will be value="yes"-->
<button class="btn btn-success btn-answer" value='Yes'><i class="icon icon-white icon-thumbs-up"></i> Yes</button>
<!-- If the user clicks this button, the saved answer will be value="no"-->
<button class="btn btn-danger btn-answer" value='No'><i class="icon icon-white icon-thumbs-down"></i> No</button>
<!-- If the user clicks this button, the saved answer will be value="NotKnown"-->
<button class="btn btn-answer" value='NotKnown'><i class="icon icon-white icon-question-sign"></i> I don't know</button>
</div><!-- End of DIV for the submission buttons -->
</source>
New fragment:
<source lang="java">
<div id="answer"> <!-- Start DIV for the submission buttons -->
<!-- If the user clicks this button, the saved answer will be value="yes"-->
<button class="btn btn-success btn-answer" value='Yes'><i class="icon icon-white icon-thumbs-up"></i> Yes</button>
<!-- If the user clicks this button, the saved answer will be value="no"-->
<button class="btn btn-danger btn-answer" value='No'><i class="icon icon-white icon-thumbs-down"></i> No</button>
<!-- If the user clicks this button, the saved answer will be value="maybe"-->
<button class="btn btn-answer" value='Maybe'><i class="icon icon-white icon-comment-alt"></i>Maybe</button>
<!-- If the user clicks this button, the saved answer will be value="NotKnown"-->
<button class="btn btn-answer" value='NotKnown'><i class="icon icon-white icon-question-sign"></i> I don't know</button>
</div><!-- End of DIV for the submission buttons -->
</source>
The result looks like this:
[[File:Pybossa-app1.png|400px|thumbnail|none|Adapted PyBossa application]]
=== A simple example reusing existing code ===
{{stub}}
1. Create an account and/or an application as above
==== Create an application ====
* Install a Python virtual environment. This way you can be sure that dependencies will remain ok (not overwritten by system upgrades, etc.)
* Start from an existing template, e.g. from the [http://docs.pybossa.com/en/latest/user/tutorial.html Application tutorial] or from an application you could download from the [https://github.com/PyBossa GitHub repository]
.....
==== Create and push the application to the pyBossa server ====
python createTasks.py -u http://crowdcrafting.org -k API-KEY
* Select your application (personal pull-down menu on top -> My Applications
* Open your application.
=== Links ===
==== Official documentation ====
Documentation is available on [http://docs.pybossa.com/ docs.pybossa.com/]. For example:
* [http://docs.pybossa.com/en/latest/overview.html PyBossa Overview]
* [http://docs.pybossa.com/en/latest/build_with_pybossa.html Build with PyBossa]
* [http://docs.pybossa.com/en/latest/user/overview.html Quick overview: Creating your own Application]
* [http://docs.pybossa.com/en/latest/user/tutorial.html Step by step tutorial on creating an Application] (more advanced use)
* [http://crowdcrafting.org/ Crowdcrafting.org]. Official PyBossa server that hosts many projects. Anyone can create his/her own.
==== Example applications ====
* [http://irevolution.net/2013/01/20/digital-humanitarian-micro-tasking/ Digital Humanitarian Response: Moving from Crowdsourcing to Microtasking]
==== Other ====
* [http://okfnlabs.org/about Open Knowledge Labs] has posts concerning pyBossa, e.g. about new features
{{Free text}}

Latest revision as of 21:07, 26 November 2015

Low
Medium
High
Complete

Cs Portal > List of citizen science software > Pybossa -(2015/11/09)

Pybossa.png
Pybossa.png
CCLlogo.png
CCLlogo.png


IDENTIFICATION

Pybossa homepage
  • Programming languages: Python
  • Software license: Free open source
  • Number of projects: 100
  • Number of users:
Start date :
  • Beta start date : N/A
  • End date :
Projects developed with

Description [[Has project description::According to the official documentation (April 2013), “PyBossa is an open source platform for crowd-sourcing online (volunteer) assistance to perform tasks that require human cognition, knowledge or intelligence (e.g. image classification, transcription, information location etc). PyBossa was inspired by the BOSSA crowdsourcing engine but is written in python (hence the name!). It can be used for any distributed tasks application but was initially developed to help scientists and other researchers crowd-source human problem-solving skills!” Currently (April 2015), pyBossa is fully functional.

Crowdcrafting provides a hosting service.]] Purpose PyBossa is there to help researchers, civic hackers and developers to create projects where anyone around the world with some time, interest and an internet connection can contribute. It’s a 100% open-source Unlike, say, “mechanical turk” style projects, PyBossa is not designed to handle payment or money — it is designed to support volunteer-driven projects.

COMPUTING
THINKING
SENSING
GAMING
SOME
WHAT

TEAM

MAIN TEAM LOCATION
Loading map...

Spain

DEVELOPERS TEAM

+ Information about the team

OVERVIEW

Pybossa-app1.png

SYSTEM OVERVIEW pyBossa system (like Crowdcrafting.org) has three main components:

  • The BOSSA core
  • A task presenter, i.e. a HTML5 frontend that will load the task data into the DOM of a web page. Task Presenters can be either HTML5/JavaScript code that sits in the pyBossa server or they can be made with any language communicating through the API.
  • A task creator, i.e. a script that will upload tasks for the application to the BOSSA core.

There are two approaches for creating pyBossa applications:

  • Using the web interface of a pyBossa server
  • Using the REST API

You can create applications that use existing templates with different types of data. Else, you must learn how to create applications with Python. You will have to install a virtual Python environment in your machine, code, create the application that then can be "pushed" into a pyBossa server.

Other information

field_project_name Pybossa
field_project_purpose Support citizen science activities:
Computing: NO Thinking: YES
Sensing: YES Gaming: Somewhat
field_home_page_URL
Total number of projects (estimate) 100
Total individual participants (estimate)
field_project_start_date
field_comments
pyBossa system (like Crowdcrafting.org) has three main components:
  • The BOSSA core
  • A task presenter, i.e. a HTML5 frontend that will load the task data into the DOM of a web page. Task Presenters can be either HTML5/JavaScript code that sits in the pyBossa server or they can be made with any language communicating through the API.
  • A task creator, i.e. a script that will upload tasks for the application to the BOSSA core.

There are two approaches for creating pyBossa applications:

  • Using the web interface of a pyBossa server
  • Using the REST API

You can create applications that use existing templates with different types of data. Else, you must learn how to create applications with Python. You will have to install a virtual Python environment in your machine, code, create the application that then can be "pushed" into a pyBossa server.

Completion level Medium

Documented projects in this wiki:


Sorry, the PDF print version needs yet to be done ....

Thinking: yes Computing: no Sensing: yes Gaming: somewhat


pybossa test Go

No

Book


Creating a simple application

Create an account and start a project

On a first and second screen, fill in the fields that will describe the application. You are allowed to change these later.

You can find your applications in your "personal" pull down menu.

Name -> My Profile

Select items you will work with

As of Nov 2015, there are 4 options, i.e. objects that you can you can present to the users for some pattern recognition task

  • images
  • sounds
  • videos
  • PDFs

E.g. select images

Define the type of task

There are three options:

  • Classify
  • Describe
  • Count

E.g. select classify

Define a question

  • Make sure to type an answer for each field (prefilled content does not work)
  • You can add response items

Next, add another question or two.

Add a source

You can choose between Dropbox and Flickr. Both provide you with simliar features and a test project can be built with either one. Using Flickr is probably a bit easier.

Make sure to add all pictures you need. Click on all pictures you want to import.

To add / change pictures, you can

  • MyProfile -> Select your project
  • click on Tasks
  • Click on Import

The documentation as nov Nov 2015, is not totally in synch. E.g. there is no need to create a task presenter as it is described in the quickstart manual

Customizing simple projects

There are several methods for customizing a project. Firstly you could add more tasks either using the Flickr/DropBox interface or using the CSV or JSON taks importer.

Modify tasks

If you are just testing it is best to delete existing tasks. Since tasks are identified by I don't know what, PyBossa will refuse to reimport tasks that already have been defined, even in a different way) To do so

Settings -> Delete tasks

Export to CSV and edit in spreadsheet

This allows for example to manually add pictures and also to change URLs. However, you carefully must respect custom fields that are made for each different type of presenter.

Tasks -> Export
  • Click on Download CSV. This will generate a zip file with a csv file, that you then can open in Google Docs (or any texteditor or spreadsheet application)

After making changes, you can import the CSV from various sources (Google Drive, URLs, DropBox)

Edit a JSON File

For people with a little computer science background, editing a JSON file is an alternative.

Export your tasks

Task -> Export
  • Download JSON

Import of a modified task file

For a CSV:

Tasks -> Import
Use either Google Drive or DropBox.

For the JSON:

Use the API (more later ...)

Change presentation of tasks

Old stuff

(needs to be checked)

pyBossa templates

pyBossa templates are reusable applications templates that implement popular crowd crafting problems. Templates include: a task creator, a task presenter, a tutorial, an application description, etc.

As of April 2013, there are five templates available on the crowdsourcing.org server. You can adapt these to your own needs in various ways.

Template Task template Description
EpiCollect Plus Project EpiCollect Plus Interface to an EpiCollect project
Geo-coding Urban Park Task Allows users to tag a park in a list of cities of the whole world. The result is then shown in a 3D Globe (rendered with WebGL)
Image Pattern Recognition Flickr Person Finder Uses the Flickr web service as the source of the data, pulls a picture from a Flick Feed and then asks the user a question with several possible answers.
Transcribing documents PDF transcription An application that allows transcribing PDF documents (e.g. old texts)
Sound Pattern Recognition SoundCloud ...

In addition, there are two special templates allowing to import data from CSV files or Google Drive Spreadsheets. In order to use that data, you will have to write the code for a presenter.

Template Task template Description
CSV Public link to a CSV file ?
Google Drive Spreadsheet Import a Google Drive Spreadsheet ?

All PyBossa templates are available in GitHub repository, e.g. you can add these to your own pyBossa server or modify the code and push to a pyBossa server.

Data exports

Both tasks and task runs can be exported either in CSV or JSON format for analysis.

Adapting the built-in Image pattern recognition template

This example will show how to adapt the built-in Image pattern recognition using the public pyBossa sever.

Example that you only can view once, unless you connect from a different IP. It is really just a test, e.g. only includes three pictures of the 3D printers I own ....

Below, we summarize the steps.

Create an account and an application

Fill in 8 fields that will describe the application. You are allowed to change these later.

You can find your applications in your "personal" pull down menu.

Name -> My Profile

Select an existing application template

In the Settings for your application (personal pull-down menu -> My applications):

  • Click on Import tasks and select the Image Pattern Recognition template

You now will see a link to a Google spreadsheet and that you can substitute by your own. Therefore, do not use this link, create your own as explained below.

  • Create a copy and share it on your own google drive. With the open file, use menu File->Share.
  • Replace all the pictures and maybe also the questions. Do not change the headings, since the presenter will use these.
col1 = question
col2 = thumbnail picture
col3 = credits page
col4 = full picture

... of course you can link to any picture from any site. E.g. we took ours from this wiki:

Create/Modify the task presenter

Pick the Image recognition task (the others will not work with the template)

You now can change the number and the phrasing of the answers. As you can see below, we added an answer:

<!-- If the user clicks this button, the saved answer will be value="maybe"-->
<button class="btn btn-answer" 
        value='Maybe'><i class="icon icon-white icon-comment-alt"></i>Maybe</button>

Adapting the buttons PyBossa uses the Twitter bootstrap CSS framework.

  • icon-comment-alt refers to a special font that was made for this framework. In principle, you can use any of these font names from this list.
  • Same principle for the button color styles, e.g. btn_answer would be light grey. See the Twitter bootstrap buttons list

Original fragment:

<div id="answer"> <!-- Start DIV for the submission buttons -->
<!-- If the user clicks this button, the saved answer will be value="yes"-->
<button class="btn btn-success btn-answer" value='Yes'><i class="icon icon-white icon-thumbs-up"></i> Yes</button>
<!-- If the user clicks this button, the saved answer will be value="no"-->
<button class="btn btn-danger btn-answer" value='No'><i class="icon icon-white icon-thumbs-down"></i> No</button>
<!-- If the user clicks this button, the saved answer will be value="NotKnown"-->
<button class="btn btn-answer" value='NotKnown'><i class="icon icon-white icon-question-sign"></i> I don't know</button>
</div><!-- End of DIV for the submission buttons -->

New fragment:

<div id="answer"> <!-- Start DIV for the submission buttons -->
<!-- If the user clicks this button, the saved answer will be value="yes"-->
<button class="btn btn-success btn-answer" value='Yes'><i class="icon icon-white icon-thumbs-up"></i> Yes</button>
<!-- If the user clicks this button, the saved answer will be value="no"-->
<button class="btn btn-danger btn-answer" value='No'><i class="icon icon-white icon-thumbs-down"></i> No</button>
<!-- If the user clicks this button, the saved answer will be value="maybe"-->
<button class="btn btn-answer" value='Maybe'><i class="icon icon-white icon-comment-alt"></i>Maybe</button>
<!-- If the user clicks this button, the saved answer will be value="NotKnown"-->
<button class="btn btn-answer" value='NotKnown'><i class="icon icon-white icon-question-sign"></i> I don't know</button>
</div><!-- End of DIV for the submission buttons -->

The result looks like this:

Adapted PyBossa application

A simple example reusing existing code

Draft

1. Create an account and/or an application as above

Create an application

  • Install a Python virtual environment. This way you can be sure that dependencies will remain ok (not overwritten by system upgrades, etc.)

.....

Create and push the application to the pyBossa server

python createTasks.py -u http://crowdcrafting.org -k API-KEY
  • Select your application (personal pull-down menu on top -> My Applications
  • Open your application.

Links

Official documentation

Documentation is available on docs.pybossa.com/. For example:

  • Crowdcrafting.org. Official PyBossa server that hosts many projects. Anyone can create his/her own.

Example applications

Other


Bibliography

BIBLIOGRAPHY