Means for Finding Meaningful Levels of a Hierarchical Sequence Prior to Performing a Cluster Analysis

David Allen Olsen

2014

Abstract

When the assumptions underlying the standard complete linkage method are unwound, the size of a hierarchical sequence reverts back from n levels to n(n-1)/2 +1 levels, and the time complexity to construct a hierarchical sequence of cluster sets becomes O(n^4). Moreover, the post hoc heuristics for cutting dendrograms are not suitable for finding meaningful cluster sets of an n(n-1)/2 +1-level hierarchical sequence. To overcome these problems for small-n, large-m data sets, the project described in this paper went back more than 60 years to solve a problem that could not be solved then. This paper presents a means for finding meaningful levels of an n(n-1)/2 +1-level hierarchical sequence prior to performing a cluster analysis. By finding meaningful levels of such a hierarchical sequence prior to performing a cluster analysis, it is possible to know which cluster sets to construct and construct only these cluster sets. This paper also shows how increasing the dimensionality of the data points helps reveal inherent structure in noisy data. The means is theoretically validated. Empirical results from four experiments show that finding meaningful levels of a hierarchical sequence is easy and that meaningful cluster sets can have real world meaning.

Paper Citation

in Harvard Style

Olsen D. (2014). Means for Finding Meaningful Levels of a Hierarchical Sequence Prior to Performing a Cluster Analysis . In Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-989-758-039-0, pages 21-33. DOI: 10.5220/0005040600210033

in Bibtex Style

@conference{icinco14,
author={David Allen Olsen},
title={Means for Finding Meaningful Levels of a Hierarchical Sequence Prior to Performing a Cluster Analysis},
booktitle={Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2014},
pages={21-33},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005040600210033},
isbn={978-989-758-039-0},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 11th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - Means for Finding Meaningful Levels of a Hierarchical Sequence Prior to Performing a Cluster Analysis
SN - 978-989-758-039-0
AU - Olsen D.
PY - 2014
SP - 21
EP - 33
DO - 10.5220/0005040600210033