Artcode detection in images

Abstract

We are in a world of thinly disguised aesthetic visual markers. How do people know of their presence is the initial step into this augmented world. This thesis is concerned with such special aesthetic markers — Artcodes, which are topological markers that implement and extend the d-touch system. While current Artcode applications focus on the activities after decoding, this thesis examines the activities before decoding, investigating the question of how to discover such “invisible” visual markers as Artcodes. Interacting with an invisible sensing system (vision-based AR system here) is a longstanding question in the HCI community. This question is proposed as a formal computer vision problem — Artcode detection, including two dependent subproblems: Artcode classification (classifying an input image or image patch as either containing an Artcode or not) and Artcode localisation (predicting the locations of Artcodes in an image), and addressed using machine learning methods. ARTCODEPRESENCE and ARTCODEGUIDE approaches are proposed in this thesis to deal with Artcode detection proposal. ARTCODEPRESENCE is a machine learning-based Artcode proposal generator for dealing with Artcode classification, incorporating the classification methods (Random forests or SVM) and the Shape of the Orientation Histogram (SOH) feature vector. SOH is an Artcode prediction feature vector proposed by this thesis to describe such topological object classes as Artcodes. Based on ARTCODEPRESENCE, ARTCODEGUIDE has been developed to deal with the localisation of Artcodes, guiding people to Artcodes by cues such as a heat map showing where Artcodes can appear. The two systems are evaluated using new collected datasets: True and Extended Artcode datasets and a ‘Simulated-in-the-Wild’ Artcode dataset. Experimental results have shown their effectiveness in addressing Artcode detection problem. This thesis also proposes a framework for evaluating Artcode binarisation and decoding. This framework will evaluate how the preprocessing binarisation affects Artcode decoding, by quantitatively measuring the performance of a binarisation method, and guiding people to select an appropriate binarisation method in the context of an Artcode application. Metamorphic relations (MRs), from the metamorphic software testing field, are adopted to enhance the performance of Artcode classification. An MR augmented classification framework is proposed in this thesis. Experimental evaluation shows MR-augmented classifiers achieve much better performance than non-augmented classifiers, showing the potential for MRs to be used in enhancing machine learning tasks. To the best of my knowledge, this is the first time that MRs have been used beyond the context of software testing.

Date of Award	12 Jul 2020
Original language	English
Awarding Institution	Univerisity of Nottingham
Supervisor	Dave Towey (Supervisor), Andrew French (Supervisor) & Steve Benford (Supervisor)