To Örebro University

Robot grasping depends on the specific manipulation scenario: the object, its properties, task and grasp constraints. Object-task affordances facilitate semantic reasoning about pre-grasp configurations with respect to the intended tasks, favoring good grasps. We employ probabilistic rule learning to recover such object-task affordances for task-dependent grasping from realistic video data.

Real-world scenes involve many objects that interact with each other in complex semantic patterns. For example, a bar scene can be naturally described as having a variable number of chairs of similar size, close to each other and aligned horizontally. This high-level interpretation of a scene relies on semantically meaningful entities and is most generally described using relational representations or (hyper-) graphs. Popular in early work on syntactic and structural pattern recognition, relational representations are rarely used in computer vision due to their pure symbolic nature. Yet, today recent successes in combining them with statistical learning principles motivates us to reinvestigate their use. In this paper we show that relational techniques can also improve scene classification. More specifically, we employ a new relational language for learning with kernels, called kLog. With this language we define higher-order spatial relations among semantic objects. When applied to a particular image, they characterize a particular object arrangement and provide discriminative cues for the scene category. The kernel allows us to tractably learn from such complex features. Thus, our contribution is a principled and interpretable approach to learn from symbolic relations how to classify scenes in a statistical framework. We obtain results comparable to state-of-the-art methods on 15 Scenes and a subset of the MIT indoor dataset.

Object grasping is a key task in robot manipulation. Performing a grasp largely depends on the object properties and grasp constraints. This paper proposes a new statistical relational learning approach to recognize graspable points in object point clouds. We characterize each point with numerical shape features and represent each cloud as a (hyper-) graph by considering qualitative spatial relations between neighboring points. Further, we use kernels on graphs to exploit extended contextual shape information and compute discriminative features which show improvement upon local shape features. Our work for robot grasping highlights the importance of moving towards integrating relational representations with low-level descriptors for robot vision. We evaluate our relational kernel-based approach on a realistic dataset with 8 objects.

While any grasp must satisfy the grasping stability criteria, good grasps depend on the specific manipulation scenario: the object, its properties and functionalities, as well as the task and grasp constraints. We propose a probabilistic logic approach for robot grasping, which improves grasping capabilities by leveraging semantic object parts. It provides the robot with semantic reasoning skills about the most likely object part to be grasped, given the task constraints and object properties, while also dealing with the uncertainty of visual perception and grasp planning. The probabilistic logic framework is task-dependent. It semantically reasons about pre-grasp configurations with respect to the intended task and employs object-task affordances and object/task ontologies to encode rules that generalize over similar object parts and object/task categories. The use of probabilistic logic for task-dependent grasping contrasts with current approaches that usually learn direct mappings from visual perceptions to task-dependent grasping points. The logic-based module receives data from a low-level module that extracts semantic objects parts, and sends information to the low-level grasp planner. These three modules define our probabilistic logic framework, which is able to perform robotic grasping in realistic kitchen-related scenarios.

Understanding images in terms of logical and hierarchical structures is crucial for many semantic tasks, including image retrieval, scene understanding and robotic vision. This paper combines robust feature extraction, qualitative spatial relations, relational instance-based learning and compositional hierarchies in one framework. For each layer in the hierarchy, qualitative spatial structures in images are detected, classified and then employed one layer up the hierarchy to obtain higher-level semantic structures. We apply a four-layer hierarchy to street view images and subsequently detect corners, windows, doors, and individual houses.

Complex multi-stage decision making problems often involve uncertainty, for example, regarding demand or processing times. Stochastic constraint programming was proposed as a way to formulate and solve such decision problems, involving arbitrary constraints over both decision and random variables. What stochastic constraint programming currently lacks is support for the use of factorized probabilistic models that are popular in the graphical model community. We show how a state-of-the-art probabilistic inference engine can be integrated into standard constraint solvers. The resulting approach searches over the And-Or search tree directly, and we investigate tight bounds on the expected utility objective. This significantly improves search efficiency and outperforms scenario-based methods that ground out the possible worlds.

Understanding the knowledge that resides in a Bayesian network can be hard, certainly when a large network is to be used for the first time, or when the network is complex or has just been updated. Tools to assist users in the analysis of Bayesian networks can help. In this paper, we introduce a novel general framework and tool for answering exploratory queries over Bayesian networks. The framework is inspired by queries from the constraint-based mining literature designed for the exploratory analysis of data. Adapted to Bayesian networks, these queries specify a set of constraints on explanations of interest, where an explanation is an assignment to a subset of variables in a network. Characteristic for the methodology is that it searches over different subsets of the explanations, corresponding to different marginalizations. A general purpose framework, based on principles of constraint programming, data mining and knowledge compilation, is used to answer all possible queries. This CP4BN framework employs a rich set of constraints and is able to emulate a range of existing queries from both the Bayesian network and the constraint-based data mining literature.

Constraint programming is used for a variety of real-world optimisation problems, such as planning, scheduling and resource allocation problems. At the same time, one continuously gathers vast amounts of data about these problems. Current constraint programming software does not exploit such data to update schedules, resources and plans. We propose a new framework, that we call the Inductive Constraint Programming loop. In this approach data is gathered and analyzed systematically, in order to dynamically revise and adapt constraints and optimization criteria. Inductive Constraint Programming aims at bridging the gap between the areas of data mining and machine learning on the one hand, and constraint programming on the other hand.

Constraint programming is used for a variety of real-world optimisa-tion problems, such as planning, scheduling and resource allocation prob-lems. At the same time, one continuously gathers vast amounts of dataabout these problems. Current constraint programming software does notexploit such data to update schedules, resources and plans. We propose anew framework, that we call theInductive Constraint Programming loop.In this approach data is gathered and analyzed systematically, in order todynamically revise and adapt constraints and optimization criteria. In-ductive Constraint Programming aims at bridging the gap between theareas of data mining and machine learning on the one hand, and constraintprogramming on the other hand.

A successful integration of constraint programming and data mining has the potential to lead to a new ICT paradigm with far reaching implications. It could change the face of data mining and machine learning, as well as constraint programming technology. It would not only allow one to use data mining techniques in constraint programming to identify and update constraints and optimization criteria, but also to employ constraints and criteria in data mining and machine learning in order to discover models compatible with prior knowledge.

This book reports on some key results obtained on this integrated and cross- disciplinary approach within the European FP7 FET Open project no. 284715 on “Inductive Constraint Programming” and a number of associated workshops and Dagstuhl seminars. The book is structured in five parts: background; learning to model; learning to solve; constraint programming for data mining; and showcases.

Human-robot interaction often occurs in the form of instructions given from a human to a robot. For a robot to successfully follow instructions, a common representation of the world and objects in it should be shared between humans and the robot so that the instructions can be grounded. Achieving this representation can be done via learning, where both the world representation and the language grounding are learned simultaneously. However, in robotics this can be a difficult task due to the cost and scarcity of data. In this paper, we tackle the problem by separately learning the world representation of the robot and the language grounding. While this approach can address the challenges in getting sufficient data, it may give rise to inconsistencies between both learned components. Therefore, we further propose Bayesian learning to resolve such inconsistencies between the natural language grounding and a robot’s world representation by exploiting spatio-relational information that is implicitly present in instructions given by a human. Moreover, we demonstrate the feasibility of our approach on a scenario involving a robotic arm in the physical world.

Regularization is one of the key concepts in machine learning, but so far it has received only little attention in the logical and relational learning setting. Here we propose a regularization and feature selection technique for such setting, in which one commonly represents the structure of the domain using an entity-relationship model. To this end, we introduce a notion of locality that ties together features according to their proximity in a transformed representation of the relational learning problem obtained via a procedure that we call “graphicalization”. We present two techniques, a wrapper and an efficient embedded approach, to identify the most relevant sets of predicates which yields more readily interpretable results than selecting low-level propositionalized features. The proposed techniques are implemented in the kernel-based relational learner kLog, although the ideas presented here can also be adapted to other relational learning frameworks. We evaluate our approach on classification tasks in the natural language processing and bioinformatics domain.

Recently, the combination of probability, logic and learning has received considerable attention in the artificial intelligence and machine learning communities; see e.g. Getoor and Taskar (2007); De Raedt et al. (2008). Computational logic often plays a major role in these developments since it forms the theoretical backbone for much of the work in probabilistic programming and logical and relational learning. Contemporary work in this area is often application- and experiment-driven, but is also concerned with the theoretical foundations of formalisms and inference procedures and with advanced implementation technology that scales well.

Neural-symbolic computation aims at integrating robust connectionist learning algorithms with sound symbolic rea-soning. The recent impact of neural learning, in particular of deep networks, has led to the creation of new representa-tions that have, so far, not really been used for reasoning. Results on neural-symbolic computation have shown to of-fer powerful alternatives for knowledge representation, learning and inference in neural computation. This paper presents key challenges and contributions of neural-symbolic computation to this area.

At the present time, omics experiments are commonly used in wet lab practice to identify leads involved in interesting phenotypes. These omics experiments often result in unstructured gene lists, the interpretation of which in terms of pathways or the mode of action is challenging. To aid in the interpretation of such gene lists, we developed PheNetic, a decision theoretic method that exploits publicly available information, captured in a comprehensive interaction network to obtain a mechanistic view of the listed genes. PheNetic selects from an interaction network the sub-networks highlighted by these gene lists. We applied PheNetic to an Escherichia coli interaction network to reanalyse a previously published KO compendium, assessing gene expression of 27 E. coli knock-out mutants under mild acidic conditions. Being able to unveil previously described mechanisms involved in acid resistance demonstrated both the performance of our method and the added value of our integrated E. coli network.

In clonal systems, interpreting driver genes in terms of molecular networks helps understanding how these drivers elicit an adaptive phenotype. Obtaining such a network-based understanding depends on the correct identification of driver genes. In clonal systems, independent evolved lines can acquire a similar adaptive phenotype by affecting the same molecular pathways, a phenomenon referred to as parallelism at the molecular pathway level. This implies that successful driver identification depends on interpreting mutated genes in terms of molecular networks. Driver identification and obtaining a network-based understanding of the adaptive phenotype are thus confounded problems that ideally should be solved simultaneously. In this study, a network-based eQTL method is presented that solves both the driver identification and the network-based interpretation problem. As input the method uses coupled genotype-expression phenotype data (eQTL data) of independently evolved lines with similar adaptive phenotypes and an organism-specific genome-wide interaction network. The search for mutational consistency at pathway level is defined as a subnetwork inference problem, which consists of inferring a subnetwork from the genome-wide interaction network that best connects the genes containing mutations to differentially expressed genes. Based on their connectivity with the differentially expressed genes, mutated genes are prioritized as driver genes. Based on semisynthetic data and two publicly available data sets, we illustrate the potential of the network-based eQTL method to prioritize driver genes and to gain insights in the molecular mechanisms underlying an adaptive phenotype. The method is available at http://bioinformatics.intec.ugent.be/phenetic_eqtl/index.html.

Molecular profiling experiments have become standard in current wet-lab practices. Classically, enrichment analysis has been used to identify biological functions related to these experimental results. Combining molecular profiling results with the wealth of currently available interactomics data, however, offers the opportunity to identify the molecular mechanism behind an observed molecular phenotype. In this paper, we therefore introduce ‘PheNetic’, a user-friendly web server for inferring a sub-network based on probabilistic logical querying. PheNetic extracts from an interactome, the sub-network that best explains genes prioritized through a molecular profiling experiment. Depending on its run mode, PheNetic searches either for a regulatory mechanism that gave explains to the observed molecular phenotype or for the pathways (in)activated in the molecular phenotype. The web server provides access to a large number of interactomes, making sub-network inference readily applicable to a wide variety of organisms. The inferred sub-networks can be interactively visualized in the browser. PheNetic's method and use are illustrated using an example analysis of differential expression results of ampicillin treated Escherichia coli cells. The PheNetic web service is available at http://bioinformatics.intec.ugent.be/phenetic/.

Recently, there has been a lot of attention for statistical relational learning and probabilistic programming, which provide rich representations for coping with uncertainty, with structure and for learning. In this talk I shall focus on probabilistic *logic* programming languages, which naturally belong to both of these paradigms as they combine the power of a programming language with a possible world semantics. They are typically based on Sato’s distribution semantics and they have been studied for over twenty years now. In this talk, I shall introduce the concepts underlying probabilistic logic programming, their semantics, different inference and learning mechanisms and I shall then present some recent extensions towards dealing with continuous distributions and dynamics. This is the framework of distributional clauses that is being applied to robotics, for tracking relational worlds in which objects or their properties are occluded in real time, and to planning. Finally, I shall discuss some open challenges and opportunities for research.

AI has been successful in automating scientific reasoning processes in e.g. the life science (with the Robot Scientists). The question that I want to ask is whether it is possible to automate the processes involved in data science? I also want to answer that question in the course of our ERC AdG project SYNTH on “Synthesising Inductive Data Models”.

To start the discussion on this topic, it is useful to look at the famous knowledge discovery cycle, where one typically starts from raw data, select and pre-process the data, identify the data mining task, use the right data mining algorithms, and then interpret the results and possibly iterate. It turns out that most of the existing approaches to automating this process, such as the automated statistician and meta-learning, algorithm portfolio and configuration approaches assume the learning task is known and we only need to identify the right algorithm and parameters to find the optimal task. It is well-known in the data mining community that this step takes typically only about 20% of the time, while the preprocessing and task identification take 80% of the time.

The question that I am interested in is what we can do to automate the pre-processing and task identification aspects, particularly for non-experts in data science.

The talk focussed on two issues. The first was the synthesis of a set of constraints that hold in tabular data (say a set of excel tables). The second was concerned with the use of hybrid probabilistic programs in dynamic domains and its applications in simple robotics and planning settings.

More details on our probabilistic programming language Problog can be found at https://dtai.cs.kuleuven.be/problog/.

Inductive logic programming is the subfield of machine learning that uses  first-order logic to represent hypotheses and data. Because first-order logic is expressive and declarative, inductive logic programming specifically targets problems involving structured data and background knowledge. Inductive logic programming tackles a wide variety of problems in machine learning, including classification, regression, clustering, and reinforcement learning, often using “upgrades” of existing propositional machine learning systems. It relies on logic for knowledge representation and reasoning purposes. Notions of coverage, generality, and operators for traversing the space of hypotheses are grounded in logic, see also  logic of generality. Inductive logic programming systems have been applied to important applications in bio- and chemo-informatics, natural language processing,...

Applying machine learning and data mining to novel applications is cumbersome. This observation is the prime motivation for the interest in languages for learning and mining. This note provides a gentle introduction to three types of languages that support machine learning and data mining: inductive query languages, which extend database query languages with primitives for mining and learning, modelling languages, which allow to declaratively specify and solve mining and learning problems, and programming languages, that support the learning of functions and subroutines. It uses an example of each type of language to introduce the underlying ideas and puts them into a common perspective. This then forms the basis for a short analysis of the state-of-the-art

The Tacle system that learns formulae and constraints in spreadsheet and tabular data was presented. It combines ideas from program induction with principles of constraints learning and logic programming. Tacle starts from one or more tables in .csv format and attempts to find the constraints and formulae that hold in the dataset. This is work that was published in the Machine Learning Journal.

One hypothesis is more general than another one if it covers all instances that are also covered by the latter one. The former hypothesis is called a  generalization of the latter one, and the latter a  specialization of the former. When using logical formulae as hypotheses, the generality relation coincides with the notion of logical entailment, which implies that the generality relation can be analyzed from a logical perspective. The logical analysis of generality, which is pursued in this chapter, leads to the perspective of induction as the inverse of deduction. This forms the basis for an analysis of various logical frameworks for reasoning about generality and for traversing the space of possible hypotheses. Many of these frameworks (such as for instance, θ-subsumption) are employed in the field of inductive...

On the occasion of the 25th Benelearn, I will reflect on some historical and sociological developments related to the field of machine learning. In doing so, I shall take an AI perspective and contrast it with a statistical perspective. I shall also briefly introduce some newly emerging trends that I am particularly excited about, in particular, languages for machine learning and the prospect of automating machine learning.

Probabilistic programs combine the power of programming languages with that of probabilistic graphical models. There has been a lot of progress in this paradigm over the past twenty years. This talk will introduce probabilistic logic programming languages, which are based on Sato's distribution semantics and which extend probabilistic databases. The key idea is that facts or tuples can be annotated with probabilities that indicate their degree of belief. Together with the rules that encode domain knowledge they induce a set of possible worlds. After an introduction to probabilistic programs, which will cover semantics,inference, and learning, the talk will sketch some emerging applications in knowledge based systems, in cognitive robotics and in answering probability questions. The first is concerned with learning rules in knowledge based systems such as CMU's Never Ending Language Learning, the second with learning probabilistic action denitions and using these for planning to grasp certain objects, the nal one with the answering of challenging mathematical exercises about probability that are formulated in natural language 

Probabilistic logic programs [4] combine the power of a pro- gramming language with a possible world semantics; they are typically based on Sato’s distribution semantics [9, 8], and it is possible to learn their parameters and to some extent also their structure. They have been studied for over twenty years now. In this talk, I shall introduce the state of the art in probabilistic logic programs and report on some recent progress in applying this paradigm to challenging applications. The first application domain will be that of robotics, where we have developed extensions of the basic distribution semantics to cope with dynamics as well continuous distributions [5]. The resulting representations are now being used to learn multi-relational object affordances, which specify the conditions under which actions can be applied on particular objects [6, 7]. The second application is in a biological domain, where a decision theoretic extension of the distribution semantics [10] is the underlying inference engine of the PheNetic system [2], which extracts from an in- teractome, the sub-network that best explains genes prioritized through a molecular profiling experiment. Finally, I shall report on our results in applying ProbFOIL [3] to the problem of machine reading in CMU’s Never Ending Language Learning system [1]. ProbFOIL is an extension of the traditional rule-learning system FOIL for use with the distribution semantics.

Probabilistic and logical reasoning are the cornerstones of many developments in artificial intelligence. Over the past 20 years, there has been a lot of attention to combining these two forms of reasoning. This has resulted in a rich variety of representations, languages and systems for dealing with probabilistic logical reasoning. These approaches have also been applied in machine learning context. In the first part of this talk, I shall provide a gentle introduction to such logics using ProbLog and ProPPR.

In the second part, I shall illustrate their use and promise on two challenging applications : in bioinformatics and in recommender systems. The first is based on ongoing work in Leuven with Dries Van Daele, the second on ongoing work with Sirawit Sopchoke and Prof. Masayuki Numao from ISIR, Osaka.

Probabilistic logic programs combine the power of a programming language with a possible world semantics; they are typically based on Sato's distribution semantics [5] and they have been studied for over twenty years now. They have recently been extended towards defining continuous distributions and dynamics, which enables their use in robotics and perception [1]. The talk shall briefly introduce these formalisms and then present some progress on synthesising such probabilistic programs from examples, both in the discrete and the continuous case. For the discrete case, I shall report on our results in applying ProbFOIL [5] to the problem of machine reading in CMU's Never Ending Language Learning system. ProbFOIL is an extension of the traditional rule-learning system FOIL for use with the distribution semantics. For the continuous case, I shall present our ongoing work in learning affordances in robotics, where the goal is to learn the conditions under which actions can be applied on particular objects [2,3 ]. [1] B. Gutmann, I. Thon, A. Kimmig, M. Bruynooghe, and L. De Raedt. The magic of logical inference in probabilistic programming. Theory and Practice of Logic Programming, 2011(11), pages 663-680. [2] Nitti, D., De Laet, T., De Raedt, L. (2013). A particle filter for hybrid relational domains. in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS 2013 (pp. 2764-2771). [3] Moldovan, B., Moreno, P., van Otterlo, M., Santos-Victor, J., De Raedt, L. (2012). Learning relational affordance models for robots in multi-object manipulation tasks. In Proc. IEEE International Conference on Robotics and Automation, ICRA 2012. 2012 (pp. 4373 -4378). [4] De Raedt, L. Dries, A., Thon, I., Van den Broeck, G., Verbeke, M. Inducing Probabilistic Relational Rules from Probabilistic Examples, In Proc. International Joint Conference on AI, IJCAI 2015, in press. [5] Sato, T., A Statistical Learning Method for Logic Programs with Distribution Semantics, In Proc. 12th International Conference on Logic Programming, ICLP 1995, pp. 715--729

Inspired by recent successes towards automating highly complex jobs like programming and scientific experimentation, the ultimate goal of this project is to automate the task of the data scientist when developing intelligent systems, which is to extract knowledge from data in the form of models. More specifically, this project wants to develop the foundations of a theory and methodology for automatically synthesising inductive data models. An inductive data model (IDM) consists of 1) a data model (DM) that specifies an adequate data structure for the dataset (just like a database), and 2) a set of inductive models (IMs), that is, a set of patterns and models that have been discovered in the data. While the DM can be used to retrieve information about the dataset and to answer questions about specific data points, the IMs can be used to make predictions, propose values for missing data, find inconsistencies and redundancies, etc. The task addressed in this project is to automatically synthesise such IMs from past data and to use these to support the user when making decisions. It will be assumed that the data set consists of a set of tables, that the end-user interacts with the IDM via a visual interface, and the data scientist via a unifying IDM language offering a number of core IMs and learning algorithms. The key challenges to be tackled in SYNTH are: 1) the synthesis system must ”learn the learning task”, that is, it should identify the right learning tasks and learn appropriate IMs for each of these; 2) the system may need to restructure the data set before IM synthesis can start; and 3) a unifying IDM language for a set of core patterns and models must be developed. The approach will be implemented in open source software and evaluated on two challenging application areas: rostering and sports analytics.

Following a general trend in artificial intelligence, the fields machine learning and data mining have recently witnessed a growing interest in the use of declarative techniques. What is essential in this paradigm is that the user be provided with a way to declaratively specify what the problem is rather than having to outline how that solution needs to be computed. This corresponds to a model + solver-based approach in which the user specifies the problem in a high level modelling language and the system automatically transforms such models into a format that can be used by a solver to efficiently generate a solution. This should be much easier for the user than having to implement or adapt an algorithm that computes a particular solution to a specific problem. Therefore, declarative methods could have a radical impact on the fields of machine learning and data mining. In this talk, I shall report on this new trend in machine learning and data mining from two different perspectives. The first is that of a user of existing declarative methods such as constraint programming and answer set programming, where I shall report on experiences, successes and challenges with using this type of technology. The second is that of a developer of declarative languages and solvers for machine learning and data mining, where I shall provide a gentle introduction to different types of languages such as inductive query languages, which extend database query languages with primitives for mining and learning, modelling languages for constraint-based mining, and probabilistic and other programming languages that support machine learning.

A simple but non-trivial setting for automating data science is introduced. Given are a set of worksheets in a spreadsheet and the goal is to automatically complete some values. We also outline elements of the Synth framework that tackles this task: Synth-a-Sizer, an automated data wrangling system for automatically transforming the problem into attribute-value format; TacLe, an inductive constraint learning system for inducing formulas in spreadsheets; Mercs, a versatile predictive learning system; as well as the autocompletion component that integrates these systems.

Editors' Introduction to the Special Issue on the Sixth International Symposium on Information and Communication Technology (SoICT 2015, Hue City, Vietnam).

This Special consists of a selection of the best papers from the 6th International Symposium on Information and Communication Technology -SoICT 2015. Since 2010, SoICT has been organised annually. The symposium provided an academic forum for researchers to share their latest research findings and to identify future challenges in computer science. In 2015, SoICT was held in Hue Royal city, Vietnam, during December 3-4th, 2015. SoICT 2015 was an international symposium that covered four major areas of research including Artificial Intelligence and Big Data, Network and Security, Human-Computer Interaction, Software Engineering and Applied Computing.

We investigate the problem of learning constraint satisfac-tion problems from an inductive logic programming perspective. Con-straint satisfaction problems are the underlying basis for constraint pro-gramming and there is a long standing interest in techniques for learningthese. Constraint satisfaction problems are often described using a rela-tional logic, so inductive logic programming is a natural candidate forlearning such problems. So far, there is however only little work on theintersection between learning constraint satisfaction problems and induc-tive logic programming. In this article, we point out several similaritiesand di↵erences between the two classes of techniques that may inspirefurther cross-fertilization between these two fields.

We investigate the problem of learning constraint satisfaction problems from an inductive logic programming perspective. Constraint satisfaction problems are the underlying basis for constraint programming and there is a long standing interest in techniques for learning these. Constraint satisfaction problems are often described using a relational logic, so inductive logic programming is a natural candidate for learning such problems. So far, there is however only little work on the intersection between learning constraint satisfaction problems and inductive logic programming. In this note, we point out several similarities and differences between the two classes of techniques and use these to propose several interesting research challenges.

We study the problem of inducing logic programs in a probabilistic setting, in which both the example descriptions and their classification can be probabilistic. The setting is incorporated in the probabilistic rule learner ProbFOIL(+), which combines principles of the rule learner FOIL with ProbLog, a probabilistic Prolog. We illustrate the approach by applying it to the knowledge base of NELL, the Never-Ending Language Learner.

Neural-symbolic and statistical relational artificial intelligence both integrate frameworks for learning with logical reasoning. This survey identifies several parallels across seven different dimensions between these two fields. These cannot only be used to characterize and position neural-symbolic artificial intelligence approaches but also to identify a number of directions for further research.

Statistical relational learning a.k.a. probabilistic inductive logic programming deals with machine learning and data mining in relational domains where observations may be missing, partially observed, or noisy. In doing so, it addresses one of the central questions of artificial intelligence – the integration of probabilistic reasoning with machine learning and first-order and relational representations – and deals with all related aspects such as reasoning, parameter estimation, and structure learning.

An intelligent agent interacting with the real world will encounter individual people, courses, test results, drugs prescriptions, chairs, boxes, etc., and needs to reason about properties of these individuals and relations among them as well as cope with uncertainty.

Uncertainty has been studied in probability theory and graphical models, and relations have been studied in logic, in particular in the predicate calculus and its extensions. This book examines the foundations of combining logic and probability into what are called relational probabilistic models. It introduces representations, inference, and learning techniques for probability, logic, and their combinations.

The book focuses on two representations in detail: Markov logic networks, a relational extension of undirected graphical models and weighted first-order predicate calculus formula, and Problog, a probabilistic extension of logic programs that can also be viewed as a Turing-complete relational extension of Bayesian networks.

A multitude of different probabilistic programming languages exists today, allextending a traditional programming language with primitives to support modeling ofcomplex, structured probability distributions. Each of these languages employs its own prob-abilistic primitives, and comes with a particular syntax, semantics and inference procedure.This makes it hard to understand the underlying programming concepts and appreciate thedifferences between the different languages. To obtain a better understanding of probabilisticprogramming, we identify a number of core programming concepts underlying the primi-tives used by various probabilistic languages, discuss the execution mechanisms that theyrequire and use these to position and survey state-of-the-art probabilistic languages and theirimplementation. While doing so, we focus on probabilistic extensions oflogicprogramminglanguages such as Prolog, which have been considered for over 20 years.

Motivated by an analogy with matrix decomposition, we introduce the novel problem of relational decomposition. In matrix decomposition, one is given a matrix and has to decompose it as a product of other matrices. In relational decomposition, one is given a relationrand one has to decompose it as a conjunctive query of a particular formq:–q1 ∧ ... ∧ qn. Furthermore, the de-composition has to satisfy certain constraints (e.g. that r≈q holds). Relational decomposition is thus the inverse problem of querying as one is given the result of the query and has to compute the relations constituting the query itself.

We show that relational decomposition generalizes several well-studied problems in data mining such as tiling, boolean matrix factorization, and discriminative pat-tern set mining. Furthermore, we provide an initial strategy for solving relational decomposition problems that is based on answer set programming. The resulting problem formalizations and corresponding solvers fit within the declarative modelling paradigm for data mining.

Motivated by an analogy with matrix decomposition, we introduce the novel problem of relational decomposition. In matrix decomposition one is given a matrix and has to decompose it as a product of other matrices. In relational decomposition, one is given a relation r and one has to decompose it as a conjunctive query of a particular form q:–q1 ∧ ... ∧ qn. Furthermore, the de-composition has to satisfy certain constraints (e.g. that r ≈ q holds). Relational decomposition is thus the inverse problem of querying as one is given the result of the query and has to compute the relations constituting the query itself.

We show that relational decomposition generalizes several well-studied problems in data mining such as tiling, boolean matrix factorization, and discriminative pat-tern set mining. Furthermore, we provide an initial strategy for solving relational decomposition problems that is based on answer set programming. The resulting problem formalizations and corresponding solvers fit within the declarative modelling paradigm for data mining.

While constraints are ubiquitous in artificial intelligence and constraints are also commonly used in machine learning and data mining, the problem of learning constraints from examples has received less attention. In this paper, we discuss the problem of constraint learning in detail, indicate some subtle differences with standard machine learning problems, sketch some applications and summarize the state-of-the-art.