Abstract
In order to increase the classification performance obtained using Error-Correcting Output Codes designs (ECOC), introducing weights in the decoding phase of the ECOC has attracted a lot of interest. In this work, we present a method for ECOC designs that focuses on increasing hypothesis margin on the data samples given a base classifier. While achieving this, we implicitly reward the base classifiers with high performance, whereas punish those with low performance. The resulting objective function is of the fractional programming type and we deal with this problem through the Dinkelbach’s Algorithm. The conducted tests over well known UCI datasets show that the presented method is superior to the unweighted decoding and that it outperforms the results of the state-of-the-art weighted decoding methods in most of the performed experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Hoboken (2004)
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1, 113–141 (2001)
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. (JAIR) 2, 263–286 (1995)
Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: ICML, pp. 313–321 (1995)
Guruswami, V., Sahai, A.: Multiclass learning, boosting, and error-correcting codes. In: 12th Annual Conference Computational Learning Theory, Santa Cruz, California, pp. 145–155 (1999)
Schapire, R.E., Freund, Y.: Boosting: Foundations and Algorithms, vol. 1. MIT Press, Cambridge (2012)
Escalera, S., Pujol, O., Radeva P.: Loss-weighted decoding for error-correcting output codes. In: International Conference on Computer Vision Theory and Applications, Madeira, Portugal (2008)
Smith, R.S., Windeatt, T.: Class-separability weighting and bootstrapping in error-correcting output code ensembles. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 185–194. Springer, Heidelberg (2010)
Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, N.: Margin analysis of the LVQ algorithm. In: Proceedings of 17th Conference on Neural Information Processing Systems (2002)
Escalera, S., Pujol, O., Radeva, P.: On the decoding process in ternary error-correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 120–134 (2010)
Bajalinov, E.B.: Linear-Fractional Programming: Theory, Methods, Applications and Software, 1st edn. Kluwer Academic Publishers, New York (2003)
Zhang, X., Wu, J., Chen, Z., Lv, P.: Optimized weighted decoding for error-correcting output codes. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Kyoto, Japan (2012)
Parades, R., Vidal, E.: A class-dependent weighted dissimilarity measure for nearest neighbor classification problems. Pattern Recogn. Lett. 21(12), 1027–1036 (2000)
Sun, Y., Todorovic, S., Li, J., Wu, D.: Unifying the error-correcting output code AdaBoost within the margin framework. In: 22nd ICML, Bonn, Germany (2005)
Sniedovich, M.: Dynamic Programming Foundations and Principles, 2nd edn. CRC Press, USA (2011)
Gugat, M.: Prox-regularization methods for generalized fractional programming. J. Optim. Theory Appl. 99(3), 691–722 (1998)
Domeniconi, C., Gunopulos, D., Ma, S., Yan, B., Al-Razgan, M., Papadopoulos, D.: Locally adaptive metrics for clustering high dimensional data. J. Data Min. Knowl. Discov. 14(1), 63–67 (2007)
Smirnov, E., Moed, M., Kuyper, I.: Minimally-Sized Balanced Decomposition Schemes for Multi-class Classification. Ensembles in Machine Learning Applications. Springer, Berlin (2011)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011)
Demsar, J.: Statistical comparisons of classifiers over multiple data sets. JMLR 7, 1–30 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Ismailoglu, F., Sprinkhuizen-Kuyper, I.G., Smirnov, E., Escalera, S., Peeters, R. (2015). Fractional Programming Weighted Decoding for Error-Correcting Output Codes. In: Schwenker, F., Roli, F., Kittler, J. (eds) Multiple Classifier Systems. MCS 2015. Lecture Notes in Computer Science(), vol 9132. Springer, Cham. https://doi.org/10.1007/978-3-319-20248-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-20248-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20247-1
Online ISBN: 978-3-319-20248-8
eBook Packages: Computer ScienceComputer Science (R0)