Determining the Sexual Identities of Prehistoric Cave Artists using Digitized Handprints

A Machine Learning Approach

BY
James Z. Wang1∗, Weina Ge2, Dean R. Snow3, Prasenjit Mitra1, and C. Lee Giles1
1 College of Information Sciences and Technology,
2 Department of Computer Science and Engineering,
3 Department of Anthropology,
The Pennsylvania State University, University Park, Pennsylvania

ABSTRACT

The sexual identities of human handprints inform hypotheses regarding the roles of males and females in prehistoric contexts. Sexual identity has previously been manually determined by measuring the ratios of the lengths of the individual’s fingers as well as by using other physical features. Most conventional studies measure the lengths manually and thus are often constrained by the lack of scaling information on published images. We have created a method that determines sex by applying modern machine-learning techniques to relative measures obtained from images of human hands. This is the first known attempt at substituting automated methods for time-consuming manual measurement in the study of sexual identities of prehistoric cave artists. Our study provides quantitative evidence relevant to sexual dimorphism and the sexual division of labor in Upper Paleolithic societies. In addition to analyzing historical handprint records, this method has potential applications in criminal forensics and human-computer interaction.

Categories and Subject Descriptors

H.4.m [Information Systems Applications]: Miscellaneous;
I.5.4 [Pattern Recognition]: Applications

General Terms
Algorithm, Experimentation, Human Factors

Keywords
Archaeology, Upper Paleolithic, Handprint, Image Analysis,
Prehistoric Cave Art

∗ To whom correspondence should be addressed; E-mail:
jwang@psu.edu

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MM’10, October 25–29, 2010, Firenze, Italy. Copyright 2010 ACM 978-1-60558-933-6/10/10 ...$10.00.

1. INTRODUCTION

[illustration]

From the ornately stenciled hands on the breathtaking wall panel of the Gua Tewet cave of eastern Borneo to the hands scattered around the Spotted Horses of the Pech-Merle cave in southern France, prehistoric handprints and hand stencils have fascinated archaeologists and art historians for many years (Fig. 1). Our work is motivated by a particular interest in the sexual dimorphism of hands in parietal art [4, 14]. There is considerable anthropological interest in sexual dimorphism and sexual division of labor in Upper Paleolithic societies. For instance, if biologically modern humans pair-bonded more strongly than did the Neanderthals with whom they were competing, such pair-bonding behavior might help explain some of the competitive advantage of modern humans as well as the legacy of pair-bonding among living humans today. Much recent research into the sexual identities of the makers of prehistoric handprints and hand stencils [5] simply assumed that males made them. Recent research by Snow, however, provided evidence that females also actively participated in the cave art [14]. His initial research showed that among six hand stencils found in caves in France, four could be identified as female hands. Research in additional caves, supported by the National Geographic Society, confirmed with a larger sample that females probably made a majority (75%) of the hand stencils.

1.1 RelatedWork

Many existing methods for handprint sexual identification are limited because they are based on measurements of three fingers (index, ring, and little) and the overall hand length. First, the tedious manual measuring process inhibits studies on large-scale image data sets across age groups and ethnic populations. Unlike many other biometric data, handprint measurements across such populations are not currently available from large standard data sets and thus must be acquired according to privacy protocols. Second, the measuring process is prone to subjective errors when determining the starting and ending points of the length of
the digit. Third, the absolute measurements require scale information that is consistently missing in the published photographs. To overcome these limitations, we leveraged advanced image processing and machine learning techniques in order to infer sexual identities from handprints. Our method uses normalized relative measures to achieve scale invariance, which allows the investigation of a larger portion of the publicly available data than would otherwise be available. Automatic classification of sex based on handprint images is a new technique. Sex classification based on other types of images has focused on features such as facial images [6, 12], gait analysis from walking video sequences [3], and integrated visual and audio cues [19]. Biometric security research has investigated hand-based human identification systems using various features [1, 8, 20, 21]. Sex classification based only on hand images is challenging. Most people can instantly make a fairly accurate judgment of sex classification based on a photograph of a face. However, untrained observers often find it difficult to determine the sex based only on a handprint. It seems that for sexual identification hands contain less distinctive information than faces. Hand recognition is also different from individual human identification because any proposed approach that will successfully distinguish between two sex categories must be robust with regard to the significant variance among the hands of different subjects. The remainder of the paper is organized as follows: the classification method is described in Section 2. The experimental results are provided in Section 3. We conclude and suggest future research in Section 4.

2. AUTOMATIC SEX CLASSIFICATION

Our sex classification procedure consists of three major steps. First, the handprint image is segmented, and the hand contour is extracted. Second, points of interest (POIs), e.g., the finger tips and the valleys between the fingers, are located on the contour to compute the hand geometric features, including the lengths and widths of the fingers. These features are normalized to be insensitive to scale differences. Finally, a Support Vector Machine (SVM) [16] classifier that has been trained on manually classified hand images is used to predict the sexual identity of the hand.

2.1 Hand Segmentation

Our automated sex classification system needs to handle both color and gray images because handprints in artworks do not usually have skin colors. The hand segmentation algorithm must be robust enough so that fuzzy handprints from artworks can be processed. For each color hand image, we first convert it from the RGB color space to the Hue, Saturation, Value (HSV) color space due to the improved perception characteristics of the HSV space. Previous work has also shown that skin colors tend to form tighter clusters in the HSV space than in RGB space [9]. This property makes the separation between clusters easier. For gray images, the method remains the same except that we segment the hand based on image intensities instead of the color components. In the rest of the paper, to simplify the explanation, we assume the picture is colored. We perform K-means clustering on the pixel values to segment an input image into two components: the hand blob and the background. The K-means algorithm is a fast and effective clustering method that partitions feature vectors into K groups by minimizing a sum-of-squares cost function [7]. Wang et al. [18] used K-means in the SIMPLIcity image retrieval system to achieve real-time
segmentation of images. We set K = 2 as we assume the hand is against a relatively clear and monochromatic background. There is a vast volume of literature on detecting hands from images with a rich background. Because our primary goal is sex classification, we confine the scope of our problem with this simplified assumption. The segmentation results usually contain small debris and require post-processing to extract a clean and complete hand blob. Here we apply connected component analysis [10] to extract the largest connected component in the foreground as the desired hand blob.

2.2 Hand Feature Detection

The extracted hand blob is analyzed to locate the POIs, i.e., finger tips and valleys, that correspond to the central points of the high curvature sections on the hand contour. A contour of length n is denoted C = {p1, p2, ..., pn}, where pi = (x, y) is the pixel coordinate of a point on the contour. Because the contour is often not smooth, direct computation of the curvature on the contour may not be reliable. Instead, we compute an angle signature from the contour. For each pi, we define its neighborhood of length l along the contour as

[illustration]

For this work, we set l = 50. We also define the backward vector and the forward vector as

[illustration]

The angle signature is generated by proceeding along the contour and recording the angle between [illustration] and [illustration]

[illustration]

In the ideal case, the angle signature should have nine valleys, which correspond to the POI. However, as shown in Fig. 2(E), although less jaggy than the point curvature, the angle signature can still be noisy and has many false hits. Therefore, a non-minimum suppression process [17] is applied to the signature to locate the true valleys. The prior knowledge of the hand shape is utilized to map the POI to the corresponding finger tips and valleys. ordered set of POI, starting from the thumb to the little finger.
Geometric features of the hand and the relative location of other crucial marks on the hand like knuckles [13] can be further computed from the POI. For each digit, we sample λ segments for the finger width, which is set to five in our study. We denote the set of features D = {di : i = 1, ..., 5λ + 7}, where di (i = 1, ..., 5) are the finger lengths; di (i = 6, 7) are the palm measurements, and di (i = 8, ..., 5λ + 7) are the finger widths samples (Fig. 2(H), for details refer to [13]).
To compensate for the image scale problem, the middle finger is selected as the reference digit, and the rest of the features are normalized by dividing its length. We further extend D with three ratios r1 = d2/d4, r2 = d2/d5, and r3 = d4/d5. In [11], r1 has been shown to be sexually
dimorphic. A ratio of 1.00 for women and 0.98 for men was reported in a Liverpool sample. The other two ratios are added according to the observation that both should be high for females [14]. We discard the length of the thumb because this feature has been empirically observed to be unstable and consequently unreliable. The normalized length of the middle finger is also omitted because it is always one. Recall that the middle finger is the reference digit for normalization. To summarize, for λ = 5, we have a final feature set ˆD of 33 features with three finger lengths, 25 finger width samples, two palm measurements, and three
finger length ratios.
A summary of the hand segmentation, POI detection, and geometric feature extraction algorithms is illustrated in Fig. 2.

2.3 Sex Classification using SVM

[illustration]

[illustration]

To implement the sex classification, we use a binary SVM classifier [16], which once trained can predict a given hand image as either male or female. In our experiments, we use Radial Basis Function (RBF) as the kernel. The normalized feature set ˆD from each hand image forms a 33-dimensional feature vector that is supplied to the SVM classifier. There are a total of 175 training images from two sex classes, including 85 male samples and 90 female samples. Figs. 3 and 4 show some example images in these data sets. The training samples come from two collections.

• We refer to the first data set as the scan data, which consists of 68 scans of handprints (half female and half male) [14]. There is one scan for each hand of an individual subject.

• The second data set, which we refer as the star data, is a collection of digital photos of 107 movie star handprints (51 male and 56 female) that we photographed outside of the Chinese Theater in Hollywood.

The SVM classifier, trained over the union of these sets, is used to predict the sexual identity of the handprints from the cave data, which contains handprints found in prehistoric caves in France [14]. There is good reason to assume that measures based on the handprints of modern humans are applicable to hand stencils found in Upper Paleolithic caves [14].
Ideally, the same automatic process should be applied to all training and testing handprint images. However, the hand contours of most handprints in the scan and cave cases are often difficult to detect accurately due to the way they were created. Even experts may not be able to quickly discern with certainty the boundary of the handprint. To overcome this, we manually extracted the hand contours for images in these two data sets. Automatic image segmentation method is used to detect the hand contours for the star data. All the images were rescaled to the same resolution, with 480 pixels on the longest edge, prior to further processing. We tested several classifiers [15], including a SVM with different kernels, a linear discriminant analysis classifier, and a nearest-neighbor classifier. The SVM with the Gaussian Radian Basis Function kernel demonstrated the highest cross-validation accuracy.

3. EXPERIMENTAL FINDINGS

The performance assessment of our classifier was based on ten rounds of cross validation over the training data. During each round, fifty samples were randomly chosen as the holdout validation set. The remaining 125 samples were used for training. An average accuracy of 72% was achieved for our large sample of modern European hands. The continuum of modern hands reveals considerable overlap between male and female specimens in the middle range.

[illustration]

[illustration]

When scaled against modern hands, stencils from 32 caves in France and Spain tended to fall near the ends of that continuum, suggesting that sexual dimorphism was more pronounced during the Upper Paleolithic. Because of this distinction, the sexual identities of Upper Paleolithic hand stencils can be determined with greater certainty than would be the case for any modern specimen. As shown in Figs. 5 and 6, two of our predictions contradict findings in a previous study [14].
Because neither our machine-based method nor a handprint expert can determine the sexual identities reliably without making mistakes, when there is a contradiction like this it can be difficult to tell which method has the correct prediction. For our study of the cave data in particular, because the extraction of the hand contours is done manually for the cave data, consistency of the machinebased extraction algorithm cannot be maintained. As a result, further analysis of these results is necessary.

[illustration]

To demonstrate the robustness and potential future applications of our algorithm, we tested it using photos of hands that were taken by high-resolution digital cameras. Our algorithm exhibited high tolerance to the variance in the poses of the hands, illumination conditions, and existence of accessories, such as watches, wristbands, and rings (Fig. 7). Our automatic sex classification system does not require any specialized input devices or any user supervision for parameter tuning and processes handprints in real time on a single-CPU Linux computer.

4. CONCLUSIONS AND FUTUREWORK

An automated real-time method for classifying the sex of images of human handprints has been developed in order to facilitate archaeological research. The method is based on machine learning and image processing techniques. Realworld images from various sources, including handprints of prehistoric cave artists, have been tested.
This work represents one part of our continuing work to develop cybertools for archeology [15] and is an example of the increasing application of computational methods to archeology and cultural heritages [2]. Further directions to pursue include exploring the properties of the extracted features and their relative discriminative importance in sex role and their variance under different ethnic/age populations. A statistical model of hand shapes may further improve accuracy. While the current method is general, it is not suitable for situations where people have changed sex, or have deformed hands or fingers.
The application of this approach in forensics and humancomputer interaction could lead to unexpected discoveries. For instance, law enforcement officers can use this method to profile the criminal when a handprint is available. Future computer systems with a handprint scanner can use a technique like this for user authentication.

5. ACKNOWLEDGMENTS

The research was funded by Microsoft Research. The computation equipment was provided by the National Science Foundation under grant No. 0202007. The paper is dedicated to the memory of Jim Gray, who was supportive of this work.

6. REFERENCES

[1] Y. Bulatov, S. Jambawalikar, P. Kumar, S. Sethia, “Hand recognition using geometric classifiers,” Lecture Notes in Computer Science, Vol. 3072, pp. 1-29, 2004.
[2] C.-c. Chen, H. Wactlar, J. Z. Wang, K. Kiernan, “Digital imagery for significant cultural and historical materials,” International Journal on Digital Libraries, Special Issue: Towards the New Generation Digital Libraries: Recommendations of the US-NSF/EU-DELOS Working Groups, vol. 5, no. 4, pp. 275-286, 2005.
[3] J. W. Davis, H. Gao, “An expressive three-mode principal components model for gender recognition,” J. of Vision, vol. 4, no. 5, pp. 362-377, 2004.
[4] R. G. Gunn, “Hand sizes in rock art: interpreting the measurements of hand stencils and prints,” Rock Art Research, vol. 23, pp. 97-112, 2006.
[5] R. D. Guthrie, The Nature of Paleolithic Art, University of Chicago Press, Chicago, IL, 2005.
[6] S. Gutta, J. R. J. Huang, P. J. Phillips, H. Wechsler, “Mixture of experts for classification of gender, ethnic origin, and pose of human faces,” IEEE Transactions on Neural Networks, vol. 11, pp. 948-960, 2000.
[7] J. A. Hartigan, M. A. Wong, “Algorithm AS136: a k-means clustering algorithm,” Applied Statistics, vol. 28, pp. 100-108, 1979.
[8] A. K. Jain, A. Ross, S. Pankanti, “A prototype hand geometry-based verification system,” In Proc. Intl. Conf. on Audio- and Video-Based Biometric Person Authentication, pp. 166-171, 1999.
[9] R. Kjeldsen, J. Kender, “Finding skin color images,” in Intl. Conf. on Automatic Face and Gesture Recognition, pp. 312-317, 1996.
[10] C. E. Leiserson, R. L. Rivest, C. Stein, T. H. Cormen, Introduction to Algorithms, The MIT Press, Cambridge, MA, 1990.
[11] J. T Manning, Digit Ratio, Rutgers University, New Brunswick, NJ, 2002.
[12] B. Moghaddam, M. H. Yang, “Learning gender with support faces,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 707-711, 2002.
[13] M. G. K. Ong, T. Connie, A. T. B. Jin, D. N. C. Ling, “Automated hand geometry verification system base on salient points,” in Intl. Symposium on Communications and Information Technologies, pp. 720-724, 2003.
[14] D. R. Snow, “Sexual dimorphism in Upper Palaeolithic hand stencils,” Antiquity, vol. 80, pp. 390-404, 2006.
[15] D. R. Snow et al., “Cybertools and archaeology,” Science, vol. 311, pp. 958-959, 2006.
[16] S. Theodoridis, K. Koutroumbas, Pattern Recognition, Academic Press, New York, NY, ed. 3, 2006.
[17] E. Trucco, A. Verri, Introductory Techniques for 3D Computer Vision, Prentice Hall, Upper Saddle River, NJ, 1998.
[18] J. Z. Wang, J. Li, G. Wiederhold, “SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture Libraries,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, vol. 23, no. 9, pp. 947-963, 2001.
[19] L. Walavalkar, M. Yeasin, A. M. Narasimhamurthy, R. Sharma, “Support vector learning for gender classification using audio and visual cues”, Intl. J. of
Pattern Recognition and Artificial Intelligence, vol. 17, no. 3, pp. 417-439, 2003.
[20] A. L. N. Wong, P. Shi, “Peg-free hand geometry recognition using hierarchical geometry and shape matching”, In Proc. IAPR Workshop on Machine Vision Applications, pp. 281-284, 2002.
[21] E. Y¨or¨uk, E. Konukoglu, B. Sankur, J. Darbon, “Shape-based hand recognition,” IEEE Transactions on Image Processing, vol. 15, no. 7, pp. 1803-1815, 2006.

Harvested in 2013 from the url: http://www.cse.psu.edu/~ge/publications/mm10.pdf for "A Collection of Essays on Art" by Michael B. Chang http://www.michaelchang.dk/04_words/workwords/on_art.html#_references