Deep learning

Draft

Introduction

According to Wikipedia (Oct 27 2016), “Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and non-linear transformations.”

Most deep learning algorithms are a kind of artificial neural network, which are defined by Wikipedia as “[[..]] computational approach which is based on a large collection of neural units loosely modeling the way the brain solves problems with large clusters of biological neurons connected by axons. Each neural unit is connected with many others, and links can be enforcing or inhibitory in their effect on the activation state of connected neural units. Each individual neural unit may have a summation function which combines the values of all its inputs together. There may be a threshold function or limiting function on each connection and on the unit itself such that it must surpass it before it can propagate to other neurons. These systems are self-learning and trained rather than explicitly programmed and excel in areas where the solution or feature detection is difficult to express in a traditional computer program.”

Another popular method are [Support vector machine support vector] (SVM) machines. Wikipedia defines SVMs, also called support vector networks as “supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier.”

Deep learning in educational data mining / learning analytics

Deep learning is an interesting family of algorithms for data mining and text mining in particular. E.g. with a supervised learning algorithm it is possible to identify "good" from "not as good" text in a given domain.

Software

See:

• Comparison of deep learning software

For R

• MxNet

Links

Bibliographies

• Deep Learning Papers Reading Roadmap]

Courses

• https://www.coursera.org/learn/neural-networks Neural networks] (coursera)
• UFLDL Tutorial

For various text mining see: Deep Learning for Text Mining from Scratch (free technical online courses)

Discussion

• Neural networks vs support vector machines: are the second definitely superior?

Bibliography

General context and overviews

• Anaya AR, Boticario JG (2011) Application of machine learning techniques to analyse student interactions and improve the collaboration process. Expert Syst Appl 38: 1171–1181

• Ferguson, Rebecca (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6) pp. 304–317. http://oro.open.ac.uk/36374/

• Kotsiantis, S.B. Use of machine learning techniques for educational proposes: a decision support system for forecasting students’ grades, Artif Intell Rev (2012) 37: 331. doi:10.1007/s10462-011-9234-x http://link.springer.com/article/10.1007/s10462-011-9234-x

Deep learning

• Steve Engels, Vivek Lakshmanan, and Michelle Craig. 2007. Plagiarism detection using feature-based neural networks. SIGCSE Bull. 39, 1 (March 2007), 34-38. DOI=http://dx.doi.org/10.1145/1227504.1227324 (software plagiarism)