jmlr jmlr2013 jmlr2013-12 jmlr2013-12-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Nayyar A. Zaidi, Jesús Cerquides, Mark J. Carman, Geoffrey I. Webb
Abstract: Despite the simplicity of the Naive Bayes classifier, it has continued to perform well against more sophisticated newcomers and has remained, therefore, of great interest to the machine learning community. Of numerous approaches to refining the naive Bayes classifier, attribute weighting has received less attention than it warrants. Most approaches, perhaps influenced by attribute weighting in other machine learning algorithms, use weighting to place more emphasis on highly predictive attributes than those that are less predictive. In this paper, we argue that for naive Bayes attribute weighting should instead be used to alleviate the conditional independence assumption. Based on this premise, we propose a weighted naive Bayes algorithm, called WANBIA, that selects weights to minimize either the negative conditional log likelihood or the mean squared error objective functions. We perform extensive evaluations and find that WANBIA is a competitive alternative to state of the art classifiers like Random Forest, Logistic Regression and A1DE. Keywords: classification, naive Bayes, attribute independence assumption, weighted naive Bayes classification
R. Bellman. Dynamic Programming. Princeton University Press, 1957. B. Bjerregaard, S. Brynitz, J. Holst-Christensen, E. Kalaja, J. Lund-Kristensen, J. Hilden, F. T. deDombal, and J. C. Horrocks. Computer-aided diagnosis of acute abdomen: a system from leeds used on copenhagen patients. In In Decision Making and Medical Care: Can Information Science Help. North-Holland Publishing Company, 1976. L. Breiman. Random forests. Machine Learning, 45:5–32, 2001. A. Carvalho, T. Roos, A. Oliveira, and P. Myllymaki. Discriminative learning of Bayesian networks via factorized conditional log-likelihood. Journal of Machine Learning Research, 2011. J. Cerquides and R. L. De M´ ntaras. Tan classifiers based on decomposable distributions. Machine a Learning, 59(3):323–354, 2005a. J. Cerquides and R. L. De M´ ntaras. Robust Bayesian linear classifier ensembles. In Proceedings a of the Sixteenth European Conference on Machine Learning, pages 70–81, 2005b. 1979 Z AIDI , C ERQUIDES , C ARMAN AND W EBB Mush Shuttle Pioneer Syncon Hypo Wine Anneal Pendigits Dermatology Sick Page-blocks Optdigits Bcw Segment Splice-c4.5 New-thyroid Musk2 Labor Wall-following House-votes-84 Iris Zoo Spambase Ionosphere Nursery Thyroid Musk1 Kr-vs-kp Census-income Letter-recog Car Satellite Chess Waveform-5000 Crx Sonar Adult Hungarian Hepatitis Lyn Magic Cleveland Glass3 Autos Promoters Horse-colic Pid Vowel Cylinder-bands Covtype-mod Connect-4 Ttt German Phoneme Led Balance-scale Haberman Vehicle Sign Audio Volcanoes Echocardiogram Contact-lenses Localization Post-operative Bupa Yeast Abalone Poker-hand Cmc Tae Lung-cancer Ptn Mean Mean Rank A1DE 0.0003 0.0012 0.0025 0.0033 0.0115 0.0174 0.0188 0.0197 0.0213 0.0263 0.0305 0.033 0.0371 0.0388 0.0404 0.0423 0.0438 0.0465 0.0479 0.0555 0.0577 0.0629 0.0662 0.0701 0.0744 0.0752 0.0869 0.0915 0.0986 0.1025 0.1069 0.1092 0.1318 0.1427 0.1427 0.144 0.1491 0.1592 0.1619 0.1669 0.1696 0.171 0.1724 0.1983 0.1986 0.2107 0.2193 0.2199 0.237 0.2413 0.244 0.2502 0.2535 0.263 0.265 0.2682 0.2714 0.2761 0.279 0.3288 0.3309 0.3328 0.3563 0.359 0.375 0.3843 0.4086 0.4562 0.4643 0.4791 0.5146 0.5281 0.5383 0.1781 3.7465 LR 0 0.0004 0.0079 0.0102 0.0062 0.0185 0.0117 0.0413 0.0288 0.0269 0.0368 0.0533 0.0471 0.055 0.0692 0.0514 0.0207 0.0737 0.0118 0.0493 0.057 0.0663 0.0626 0.0869 0.0747 0.0683 0.0877 0.0277 0.0433 0.1639 0.0742 0.1807 0.1233 0.147 0.1564 0.1784 0.1274 0.1811 0.1923 0.1986 0.1538 0.1863 0.2007 0.2154 0.1241 0.2726 0.2197 0.2413 0.24 0.2571 0.2425 0.0247 0.2575 0.2544 0.2694 0.2655 0.2708 0.2845 0.32 0.2677 0.3309 0.3355 0.2958 0.4584 0.425 0.3843 0.4064 0.4623 0.4988 0.447 0.5351 0.5578 0.6444 0.1817 4.5273 LR-Reg 0 0.0004 0.0079 0.0107 0.0049 0.0199 0.0094 0.0288 0.031 0.0256 0.0318 0.0396 0.0409 0.0479 0.045 0.0528 0.0143 0.0816 0.0085 0.0461 0.0573 0.0683 0.0588 0.0818 0.0747 0.0642 0.0811 0.0286 0.0433 0.1495 0.0733 0.1175 0.1242 0.1386 0.1396 0.1637 0.1274 0.181 0.17 0.1858 0.1537 0.1766 0.1776 0.2059 0.1302 0.1798 0.2198 0.2371 0.2277 0.2571 0.2425 0.0181 0.2526 0.2068 0.2659 0.2626 0.2709 0.2723 0.3204 0.2396 0.3309 0.3351 0.3604 0.4584 0.3006 0.3967 0.4059 0.4656 0.4988 0.4478 0.5334 0.5953 0.5476 0.1736 3.5000 NB 0.0261 0.004 0.0036 0.0133 0.0146 0.014 0.0361 0.1179 0.0201 0.0312 0.0609 0.0763 0.0266 0.0752 0.0463 0.0374 0.0784 0.0544 0.0957 0.0976 0.0553 0.0713 0.0979 0.0868 0.0979 0.1116 0.1325 0.1267 0.2355 0.2563 0.156 0.1751 0.1364 0.1918 0.1449 0.1519 0.159 0.1585 0.1574 0.1666 0.2169 0.1693 0.1871 0.2554 0.1387 0.2126 0.2215 0.3931 0.2559 0.3117 0.2792 0.2902 0.2532 0.3035 0.2632 0.2594 0.2714 0.3765 0.3593 0.3305 0.3309 0.3206 0.3438 0.4939 0.3728 0.3843 0.4115 0.4794 0.4988 0.4828 0.5182 0.5203 0.5388 0.2033 5.7328 WANBIACLL 0.0012 0.0014 0.0002 0.0114 0.0083 0.0079 0.0199 0.1038 0.019 0.0263 0.0373 0.0642 0.0343 0.0502 0.0411 0.0398 0.0399 0.0544 0.0219 0.0452 0.053 0.0723 0.0626 0.0779 0.0979 0.1061 0.0649 0.0696 0.0462 0.2484 0.156 0.1553 0.1338 0.1556 0.138 0.1688 0.1306 0.1701 0.1452 0.1801 0.172 0.1764 0.1757 0.23 0.1358 0.1622 0.2151 0.3617 0.2591 0.2912 0.2727 0.2731 0.257 0.2587 0.2633 0.2594 0.2714 0.3289 0.3589 0.292 0.3309 0.3298 0.3417 0.4902 0.3611 0.3843 0.4068 0.4647 0.4988 0.4695 0.5189 0.5484 0.5456 0.1886 4.2534 Table 20: Error 1980 WANBIAMSE 0.001 0.0012 0.0001 0.0118 0.0082 0.0112 0.019 0.1018 0.021 0.0264 0.0351 0.0658 0.0368 0.0505 0.04 0.0412 0.04 0.0561 0.0206 0.0463 0.0543 0.0757 0.0611 0.0746 0.0979 0.0994 0.0666 0.0622 0.0461 0.2475 0.156 0.1514 0.132 0.1565 0.1402 0.1671 0.1303 0.1692 0.149 0.1797 0.1716 0.179 0.1778 0.2302 0.1363 0.1659 0.2152 0.3646 0.2659 0.2907 0.2726 0.2714 0.2571 0.2607 0.2648 0.2594 0.2714 0.3288 0.3568 0.2942 0.3309 0.3302 0.3458 0.49 0.3528 0.3843 0.4084 0.4643 0.4988 0.4654 0.5096 0.5563 0.542 0.1885 4.2808 RF100 0 0.0009 0.0008 0.0127 0.0122 0.0211 0.0122 0.0339 0.0367 0.0263 0.0309 0.0458 0.0386 0.0413 0.0489 0.0479 0.0385 0.0939 0.0216 0.0416 0.056 0.0743 0.0575 0.0766 0.0248 0.075 0.0683 0.0128 0.0494 0.0902 0.0772 0.1085 0.1074 0.1558 0.1581 0.1704 0.1466 0.1874 0.1606 0.1909 0.1674 0.1908 0.1951 0.1937 0.1519 0.1789 0.2536 0.1674 0.2702 0.1875 0.0765 0.2684 0.1789 0.2802 0.271 0.2709 0.2742 0.2038 0.3009 0.3309 0.3489 0.3438 0.2976 0.3972 0.3817 0.421 0.4823 0.4976 0.547 0.6 0.6 0.1688 4.3356 TAN 0.0004 0.001 0.0057 0.0115 0.0187 0.0225 0.0266 0.0411 0.0434 0.0272 0.0412 0.0487 0.0513 0.0528 0.0613 0.0514 0.0494 0.0842 0.0693 0.0649 0.0587 0.0931 0.0689 0.0781 0.0686 0.0855 0.0763 0.0772 0.0574 0.151 0.081 0.1179 0.106 0.1825 0.1651 0.1683 0.1387 0.166 0.1713 0.2257 0.1619 0.1909 0.1846 0.2437 0.2325 0.2236 0.2249 0.2781 0.3761 0.2512 0.2368 0.2484 0.2838 0.3484 0.2702 0.2661 0.2722 0.2764 0.2747 0.3361 0.3309 0.3477 0.4292 0.3564 0.3706 0.3843 0.4096 0.4687 0.329 0.465 0.5344 0.4969 0.5872 0.1890 5.6232 A LLEVIATING NB ATTRIBUTE I NDEPENDENCE A SSUMPTION BY ATTRIBUTE W EIGHTING Pioneer Mush Shuttle Syncon Pendigits Hypo Dermatology Anneal Optdigits Letter-recog Thyroid Phoneme Wine Segment Page-blocks Zoo New-thyroid Splice-c4.5 Wall-following Audio Sick Nursery Iris Vowel Bcw Satellite Labor Ptn Led Musk2 House-votes-84 Car Localization Covtype-mod Autos Poker-hand Spambase Yeast Ionosphere Lyn Waveform-5000 Kr-vs-kp Musk1 Census-income Glass3 Vehicle Chess Balance-scale Adult Volcanoes Crx Sonar Connect-4 Magic Hungarian Sign Cleveland Hepatitis Pid Promoters Contact-lenses Ttt Horse-colic German Abalone Haberman Post-operative Cmc Cylinder-bands Echocardiogram Bupa Tae Lung-cancer Mean Mean Rank A1DE 0.0086 0.0136 0.0167 0.0299 0.0556 0.0713 0.0722 0.0725 0.0747 0.0754 0.0759 0.0881 0.0917 0.0957 0.0987 0.1171 0.14 0.1435 0.1461 0.1484 0.1551 0.1583 0.1628 0.169 0.1766 0.1776 0.1792 0.1816 0.1995 0.2041 0.207 0.2085 0.2091 0.2183 0.22 0.2217 0.2323 0.2333 0.2529 0.2542 0.2586 0.2715 0.2731 0.278 0.294 0.3046 0.3081 0.3229 0.3247 0.326 0.3353 0.3367 0.3382 0.3491 0.3503 0.3508 0.354 0.3544 0.3907 0.3948 0.395 0.4037 0.4115 0.4168 0.4198 0.4212 0.4281 0.4349 0.4451 0.4506 0.4863 0.5093 0.5698 0.2461 3.6712 LR 0.0156 0.007 0.0111 0.0526 0.0888 0.0549 0.0931 0.0609 0.0999 0.1001 0.0754 0.0998 0.1042 0.1234 0.1116 0.1333 0.1797 0.212 0.0734 0.146 0.1483 0.1464 0.19 0.2057 0.2143 0.2449 0.2567 0.2329 0.2025 0.14 0.2191 0.1655 0.233 0.2259 0.244 0.2382 0.2182 0.2348 0.29 0.3101 0.2647 0.1533 0.2926 0.1807 0.3453 0.3235 0.3121 0.3128 0.2967 0.326 0.3483 0.4163 0.3361 0.337 0.3781 0.3731 0.3729 0.4358 0.3888 0.3271 0.4356 0.1385 0.5142 0.4234 0.4208 0.4213 0.4777 0.4288 0.4831 0.467 0.4863 0.553 0.5945 0.2544 5.1301 LR-Reg 0.0140 0.007 0.0107 0.0551 0.0672 0.0473 0.0954 0.0537 0.0787 0.0914 0.0706 0.0782 0.1251 0.1017 0.0986 0.1279 0.1741 0.1955 0.0598 0.13 0.1445 0.1464 0.2162 0.1831 0.1844 0.1685 0.2682 0.1809 0.2067 0.11 0.1926 0.1628 0.233 0.2259 0.2202 0.2382 0.2115 0.2343 0.2649 0.2729 0.2671 0.1547 0.2568 0.1807 0.3138 0.2989 0.3017 0.3672 0.2967 0.326 0.3319 0.3586 0.3361 0.3372 0.3678 0.3734 0.3606 0.3725 0.3896 0.3281 0.431 0.1293 0.3703 0.4145 0.4368 0.4248 0.4085 0.4289 0.4161 0.4553 0.4878 0.5023 0.5689 0.2403 3.7397 NB 0.0102 0.14 0.0309 0.0632 0.1418 0.0775 0.0713 0.0958 0.1159 0.1193 0.097 0.0951 0.0819 0.1357 0.1431 0.1247 0.1327 0.1536 0.2081 0.1486 0.1681 0.1771 0.165 0.2206 0.1586 0.2374 0.1961 0.1824 0.1988 0.2766 0.2987 0.2293 0.2386 0.2494 0.2512 0.2382 0.2949 0.2341 0.2868 0.2585 0.3274 0.3049 0.3468 0.4599 0.3118 0.3867 0.3143 0.3276 0.3405 0.326 0.3414 0.3507 0.3592 0.3916 0.3659 0.3968 0.3642 0.3589 0.3949 0.333 0.3846 0.4336 0.4227 0.4202 0.4641 0.4212 0.4233 0.4463 0.4661 0.4491 0.4863 0.5135 0.564 0.2705 6.0684 WANBIACLL 0.0025 0.0403 0.0178 0.0554 0.1256 0.0642 0.069 0.0735 0.099 0.1136 0.0867 0.086 0.0656 0.1032 0.1038 0.1253 0.1411 0.1462 0.0901 0.1354 0.1458 0.1771 0.1609 0.2116 0.1649 0.1926 0.2082 0.1811 0.199 0.1733 0.185 0.2293 0.2381 0.239 0.2281 0.2382 0.22 0.2338 0.2533 0.2539 0.2697 0.269 0.2263 0.1867 0.3032 0.3238 0.3113 0.3276 0.3 0.326 0.3194 0.3365 0.3559 0.3568 0.3428 0.3899 0.353 0.3353 0.3884 0.3277 0.3845 0.4262 0.3577 0.4151 0.4206 0.4212 0.4191 0.4312 0.4587 0.4459 0.4863 0.5075 0.5707 0.2462 3.8082 Table 21: RMSE 1981 WANBIAMSE 0.0017 0.0379 0.0169 0.0578 0.1248 0.0637 0.0723 0.0741 0.1007 0.1135 0.0854 0.0865 0.0736 0.1045 0.1027 0.1264 0.1425 0.1447 0.0897 0.135 0.1452 0.1771 0.1637 0.2123 0.1737 0.1916 0.2143 0.1811 0.1991 0.1786 0.1873 0.2293 0.2381 0.2389 0.2284 0.2382 0.22 0.2338 0.2595 0.2551 0.2698 0.2673 0.2292 0.1866 0.3061 0.3239 0.3113 0.3276 0.2999 0.326 0.3213 0.3364 0.3559 0.3567 0.3436 0.3897 0.355 0.3368 0.3888 0.3253 0.3853 0.4254 0.36 0.4156 0.4204 0.4212 0.4174 0.4309 0.4621 0.4461 0.4863 0.4979 0.5711 0.2468 4.0342 RF100 0.0361 0.009 0.0142 0.1145 0.0979 0.0715 0.1303 0.0691 0.1494 0.0896 0.077 0.0731 0.1311 0.1061 0.0974 0.1279 0.156 0.2599 0.1206 0.1361 0.1487 0.101 0.1813 0.1581 0.1796 0.1682 0.2824 0.1899 0.2091 0.1752 0.1846 0.1782 0.1939 0.2041 0.2215 0.2421 0.2403 0.2701 0.3036 0.1268 0.262 0.1928 0.3146 0.3016 0.2771 0.3181 0.3274 0.326 0.3437 0.3518 0.3057 0.3571 0.369 0.3104 0.3696 0.3375 0.4247 0.3983 0.4098 0.2916 0.3762 0.4211 0.4539 0.4214 0.4399 0.4739 0.4157 0.4574 0.4862 0.4939 0.4732 0.2469 4.4657 TAN 0.0129 0.0167 0.0154 0.0557 0.0802 0.0885 0.1016 0.0797 0.0906 0.0916 0.0805 0.0986 0.1105 0.1097 0.1165 0.1381 0.1577 0.176 0.1762 0.1414 0.1499 0.1425 0.1678 0.1886 0.1996 0.1838 0.2481 0.1829 0.2034 0.2169 0.2253 0.1849 0.2095 0.2243 0.2371 0.2124 0.2396 0.2353 0.2651 0.2886 0.2941 0.2383 0.2515 0.2083 0.3034 0.3045 0.2787 0.3242 0.3091 0.326 0.354 0.3366 0.3322 0.3425 0.3441 0.3499 0.372 0.359 0.3911 0.444 0.4477 0.4098 0.4219 0.4469 0.4268 0.4213 0.4204 0.4358 0.4794 0.4627 0.4861 0.4857 0.5033 0.2528 5.0821 Z AIDI , C ERQUIDES , C ARMAN AND W EBB Mush Pioneer Shuttle Syncon Hypo Dermatology Wine Anneal Pendigits Labor Page-blocks Optdigits Sick Segment Musk2 Bcw Wall-following New-thyroid Splice-c4.5 Zoo House-votes-84 Iris Musk1 Ionosphere Thyroid Spambase Car Nursery Letter-recog Kr-vs-kp Satellite Census-income Promoters Chess Vowel Autos Waveform-5000 Sonar Crx Lyn Hepatitis Glass3 Adult Hungarian Phoneme Cleveland Cylinder-bands Magic Horse-colic Balance-scale Pid Ttt Vehicle German Contact-lenses Haberman Audio Covtype-mod Connect-4 Led Echocardiogram Sign Post-operative Localization Tae Volcanoes Bupa Abalone Lung-cancer Yeast Ptn Cmc Poker-hand Mean Mean Rank A1DE 0.0002 0.0003 0.0007 0.002 0.0084 0.0104 0.012 0.0124 0.0133 0.0168 0.0231 0.0233 0.0237 0.0253 0.0265 0.0274 0.0285 0.0303 0.0307 0.0334 0.0466 0.0466 0.0578 0.0579 0.0595 0.0601 0.0605 0.0656 0.0684 0.0716 0.0831 0.0862 0.0872 0.0943 0.0985 0.1164 0.1176 0.1179 0.1206 0.1234 0.1242 0.1378 0.1401 0.1426 0.1465 0.1505 0.1522 0.1605 0.1619 0.1721 0.1979 0.1996 0.1998 0.1998 0.209 0.2106 0.219 0.2208 0.225 0.2278 0.2447 0.257 0.2915 0.3179 0.3305 0.3309 0.3396 0.3425 0.3473 0.3672 0.3826 0.3907 0.4423 0.1366 4.7397 LR 0 0.0024 0.0002 0.005 0.0029 0.0135 0.0115 0.005 0.0167 0.0318 0.0228 0.0237 0.0224 0.0289 0.0101 0.0284 0.0059 0.0298 0.0362 0.0332 0.0271 0.0401 0.0463 0.057 0.0434 0.0452 0.0523 0.0684 0.0967 0.0192 0.0855 0.041 0.0585 0.0772 0.1 0.1087 0.1112 0.101 0.1103 0.1155 0.1007 0.1289 0.1207 0.1291 0.1264 0.1357 0.1464 0.1446 0.1403 0.17 0.1895 0.0171 0.1765 0.1932 0.158 0.2124 0.1478 0.2474 0.2346 0.2269 0.2372 0.2927 0.2715 0.4368 0.3306 0.3309 0.3396 0.3457 0.3714 0.3651 0.3667 0.3689 0.4988 0.1293 3.3561 LR-Reg 0 0.0028 0.0003 0.0054 0.0026 0.0168 0.0132 0.0042 0.0162 0.039 0.0244 0.0232 0.0219 0.0284 0.0065 0.0301 0.004 0.0306 0.0326 0.0321 0.0262 0.0442 0.0431 0.056 0.0453 0.0482 0.0519 0.0682 0.1013 0.0197 0.09 0.041 0.0613 0.0813 0.1121 0.1068 0.1134 0.1016 0.1079 0.1183 0.1012 0.1226 0.1208 0.1354 0.1264 0.1438 0.1464 0.1443 0.1344 0.1665 0.1872 0.0162 0.1821 0.1953 0.2094 0.2206 0.1424 0.2474 0.2346 0.2245 0.239 0.2924 0.2848 0.4368 0.3311 0.3309 0.3146 0.3406 0.384 0.3655 0.3681 0.3677 0.4988 0.1305 3.5958 NB 0.023 0.0011 0.003 0.0104 0.0101 0.0108 0.0118 0.0256 0.1081 0.0167 0.0525 0.0666 0.0284 0.0633 0.0718 0.0249 0.0854 0.0295 0.0382 0.0394 0.0913 0.0503 0.1165 0.0807 0.0979 0.0949 0.1076 0.0904 0.2196 0.1067 0.1684 0.2319 0.0683 0.0989 0.2287 0.1791 0.1828 0.1314 0.1253 0.1257 0.1294 0.1597 0.1544 0.1487 0.1965 0.1548 0.1814 0.2115 0.182 0.1633 0.2047 0.2493 0.3066 0.2101 0.2069 0.2106 0.2185 0.3034 0.2643 0.2262 0.2578 0.3432 0.2805 0.4717 0.3649 0.3309 0.3396 0.4201 0.352 0.3745 0.384 0.4237 0.4988 0.1677 7.0136 WANBIACLL 0.001 0 0.0011 0.0068 0.0061 0.0071 0.003 0.0135 0.0939 0.0175 0.0327 0.0513 0.0227 0.0396 0.0299 0.0269 0.0165 0.0273 0.0316 0.0363 0.0364 0.0402 0.0394 0.0592 0.0944 0.0551 0.1076 0.0904 0.2134 0.0567 0.1428 0.0454 0.0604 0.0974 0.222 0.1427 0.1404 0.1137 0.1108 0.1169 0.0958 0.1424 0.127 0.1373 0.1758 0.1423 0.142 0.1656 0.1275 0.1633 0.1873 0.2354 0.2463 0.2058 0.2015 0.2106 0.1758 0.2867 0.2628 0.2264 0.235 0.3411 0.2792 0.4693 0.3636 0.3309 0.3396 0.3795 0.3495 0.373 0.386 0.3941 0.4988 0.1486 5.0273 Table 22: Bias 1982 WANBIAMSE 0.001 0 0.0009 0.007 0.0061 0.0082 0.0039 0.012 0.0915 0.0167 0.0304 0.0503 0.0231 0.0391 0.0274 0.0275 0.0156 0.0283 0.0294 0.0384 0.0358 0.0412 0.0399 0.0542 0.0851 0.0532 0.1076 0.0904 0.2122 0.0519 0.1386 0.0453 0.0599 0.0955 0.2224 0.1384 0.1403 0.1094 0.1106 0.1132 0.095 0.1425 0.1267 0.1361 0.1763 0.1406 0.1456 0.1651 0.1293 0.1633 0.1886 0.2349 0.2446 0.2053 0.2042 0.2106 0.1744 0.2858 0.2627 0.2265 0.2329 0.3388 0.2766 0.4698 0.3583 0.3309 0.3396 0.3785 0.3525 0.3722 0.3791 0.3908 0.4988 0.1477 4.5342 RF100 0 0 0.0006 0.008 0.0083 0.019 0.01 0.006 0.0216 0.0409 0.0217 0.0294 0.0194 0.0253 0.028 0.0301 0.0122 0.0285 0.0272 0.0356 0.0327 0.0398 0.0328 0.0624 0.0516 0.0432 0.0389 0.0086 0.049 0.0063 0.0874 0.0416 0.0552 0.0548 0.0756 0.1111 0.1114 0.1045 0.117 0.1288 0.1071 0.1348 0.1109 0.1346 0.1102 0.1304 0.208 0.1244 0.1345 0.1731 0.1802 0.027 0.1827 0.197 0.1748 0.2107 0.173 0.1427 0.2278 0.2256 0.154 0.3007 0.2047 0.3315 0.3309 0.3451 0.3257 0.3804 0.336 0.3876 0.3383 0.1151 3.3424 TAN 0.0001 0.001 0.0006 0.0055 0.012 0.013 0.0102 0.0137 0.0296 0.0201 0.0336 0.0348 0.023 0.0334 0.0386 0.0254 0.0499 0.0268 0.038 0.0468 0.0444 0.0464 0.0543 0.0647 0.0634 0.0588 0.0474 0.0543 0.1041 0.0619 0.09 0.052 0.0773 0.0551 0.1069 0.1464 0.1212 0.1044 0.1185 0.1232 0.1112 0.1269 0.1263 0.1166 0.1877 0.1416 0.2912 0.147 0.1452 0.1707 0.1816 0.1701 0.196 0.1917 0.3408 0.2107 0.1662 0.2299 0.2226 0.2257 0.2356 0.2505 0.2685 0.3097 0.3385 0.3309 0.3396 0.3361 0.2834 0.3495 0.3708 0.3425 0.2356 0.1334 4.3901 A LLEVIATING NB ATTRIBUTE I NDEPENDENCE A SSUMPTION BY ATTRIBUTE W EIGHTING Volcanoes Mush Shuttle Syncon Pioneer Sick Hypo Wine Spambase Anneal Pendigits Page-blocks Nursery House-votes-84 Adult Magic Optdigits Bcw Splice-c4.5 Dermatology Iris New-thyroid Ionosphere Census-income Segment Thyroid Hungarian Musk2 Connect-4 Wall-following Kr-vs-kp Covtype-mod Cleveland Pid Poker-hand Sign Crx Waveform-5000 Satellite Sonar Musk1 Zoo Labor Letter-recog Glass3 Led Chess Hepatitis Localization Yeast Lyn Bupa Car Horse-colic Ttt German Haberman Vehicle Autos Post-operative Cylinder-bands Echocardiogram Cmc Balance-scale Audio Promoters Abalone Phoneme Vowel Contact-lenses Ptn Lung-cancer Tae Mean Mean Rank A1DE 0 0.0001 0.0005 0.0013 0.0022 0.0026 0.0031 0.0054 0.0061 0.0063 0.0063 0.0074 0.0089 0.0089 0.009 0.0091 0.0096 0.0097 0.0097 0.0109 0.011 0.012 0.0122 0.0124 0.0135 0.0157 0.0166 0.0173 0.0189 0.0194 0.0199 0.0205 0.0205 0.0214 0.022 0.022 0.0221 0.0251 0.0261 0.0261 0.0291 0.0295 0.0297 0.0341 0.0346 0.0372 0.0375 0.0378 0.041 0.0413 0.0435 0.0448 0.0464 0.0488 0.0506 0.0537 0.0608 0.0763 0.0819 0.0835 0.0848 0.0881 0.0885 0.0961 0.1097 0.1113 0.1136 0.1165 0.1214 0.1473 0.1557 0.1808 0.184 0.0415 3.8150 LR 0 0 0.0002 0.0051 0.0055 0.0045 0.0033 0.007 0.0174 0.0067 0.0247 0.014 0.0063 0.0222 0.0066 0.0092 0.0297 0.0187 0.0331 0.0153 0.0169 0.0216 0.0299 0.0023 0.0261 0.0249 0.052 0.0106 0.0079 0.0059 0.0085 0.0097 0.0506 0.0302 0 0.0273 0.0462 0.0358 0.0952 0.0773 0.0414 0.0331 0.0419 0.0672 0.0718 0.0425 0.0461 0.0915 0.0217 0.0413 0.0831 0.0448 0.0219 0.1322 0.0075 0.0643 0.0584 0.1079 0.1067 0.1535 0.0936 0.0983 0.0781 0.0955 0.1199 0.0656 0.1166 0.128 0.1413 0.1378 0.2777 0.1864 0.2045 0.0525 5.7054 LR-Reg 0 0 0.0002 0.0054 0.002 0.0037 0.0023 0.0068 0.0106 0.0051 0.0126 0.0073 0.0065 0.0199 0.0066 0.0094 0.0164 0.0109 0.0125 0.0142 0.0131 0.0222 0.0258 0.0023 0.0195 0.0189 0.0455 0.0078 0.0079 0.0045 0.009 0.0097 0.0327 0.0326 0 0.028 0.0317 0.0253 0.0275 0.0621 0.0379 0.0362 0.0426 0.0482 0.055 0.0414 0.043 0.0688 0.0216 0.0404 0.0675 0.0821 0.0214 0.0453 0.0018 0.0573 0.0503 0.0901 0.0991 0.0158 0.0813 0.0961 0.0801 0.0962 0.0972 0.0689 0.125 0.0804 0.125 0.151 0.1796 0.2113 0.2024 0.0430 4.6712 NB 0 0.0031 0.001 0.0029 0.0024 0.0028 0.0045 0.0023 0.003 0.0105 0.0097 0.0083 0.0074 0.0063 0.0045 0.0054 0.0097 0.0017 0.0081 0.0093 0.0051 0.0079 0.0061 0.0036 0.0119 0.0137 0.0098 0.0066 0.0149 0.0103 0.02 0.0082 0.0145 0.0168 0 0.0161 0.0196 0.009 0.0067 0.0206 0.016 0.0319 0.0377 0.0367 0.0275 0.037 0.0374 0.028 0.0222 0.037 0.0408 0.0448 0.0484 0.0307 0.0409 0.0431 0.0608 0.0699 0.0763 0.0923 0.0745 0.0628 0.0591 0.0962 0.112 0.0704 0.0594 0.107 0.1643 0.1368 0.1548 0.1683 0.1533 0.0357 2.8082 WANBIACLL 0 0.0002 0.0003 0.0047 0.0002 0.0036 0.0021 0.0049 0.0075 0.0064 0.0099 0.0046 0.0074 0.0088 0.0036 0.0064 0.0128 0.0074 0.0095 0.0119 0.0128 0.0124 0.0187 0.0008 0.0107 0.0117 0.0327 0.0099 0.0098 0.0054 0.0129 0.0045 0.0341 0.0278 0 0.0178 0.0272 0.0152 0.0124 0.055 0.0255 0.036 0.0369 0.035 0.0333 0.0369 0.0363 0.0493 0.0209 0.0338 0.0632 0.0448 0.0484 0.0347 0.0377 0.0511 0.0608 0.0826 0.0873 0.0819 0.117 0.0948 0.0754 0.0962 0.1162 0.0755 0.0852 0.0829 0.1397 0.1401 0.1595 0.1989 0.1552 0.0400 3.0821 Table 23: Variance 1983 WANBIAMSE 0 0 0.0003 0.0048 0.0001 0.0032 0.0021 0.0073 0.0079 0.007 0.0103 0.0047 0.0074 0.0106 0.0036 0.0065 0.0155 0.0093 0.0106 0.0128 0.0131 0.0129 0.0205 0.0008 0.0115 0.0143 0.0331 0.0126 0.0099 0.005 0.0103 0.0049 0.0384 0.0266 0 0.0179 0.0297 0.0162 0.0129 0.0577 0.0267 0.0374 0.0394 0.0353 0.0353 0.0383 0.0366 0.0541 0.0202 0.0362 0.0665 0.0448 0.0484 0.0366 0.0365 0.0517 0.0608 0.0842 0.0918 0.0761 0.1203 0.0973 0.0747 0.0962 0.1199 0.0764 0.0858 0.0844 0.1422 0.1417 0.1629 0.2037 0.1513 0.0409 3.9520 RF100 0 0 0.0003 0.0047 0.0007 0.0069 0.0039 0.0111 0.0143 0.0062 0.0124 0.0092 0.0162 0.0089 0.0357 0.043 0.0165 0.0085 0.0217 0.0178 0.0162 0.0194 0.0142 0.0078 0.016 0.0234 0.0528 0.0105 0.0448 0.0094 0.0065 0.0603 0.0734 0.0498 0.0411 0.0443 0.0211 0.066 0.0355 0.0387 0.053 0.0413 0.0603 0.0523 0.0526 0.0535 0.0929 0.0849 0.0621 0.0366 0.0383 0.0445 0.0495 0.0714 0.0602 0.0915 0.0825 0.0965 0.0622 0.1233 0.1593 0.0979 0.1279 0.0967 0.1566 0.0687 0.0918 0.169 0.2124 0.2196 0.2156 0.0537 5.6917 TAN 0 0.0003 0.0004 0.006 0.0046 0.0041 0.0067 0.0123 0.0101 0.0129 0.0114 0.0076 0.0143 0.0206 0.0124 0.0149 0.0139 0.0259 0.0234 0.0305 0.0123 0.0246 0.0134 0.0055 0.0194 0.0221 0.0494 0.0108 0.0142 0.0194 0.0153 0.0213 0.0493 0.0432 0.0935 0.0242 0.0467 0.0613 0.0279 0.0639 0.0219 0.0463 0.0641 0.0469 0.0576 0.0445 0.0509 0.0601 0.0467 0.0602 0.1025 0.0448 0.0336 0.0785 0.0783 0.0921 0.0615 0.0803 0.0973 0.1021 0.0849 0.1121 0.1225 0.0954 0.1699 0.1552 0.1327 0.1606 0.1712 0.0884 0.2164 0.2135 0.196 0.0556 6.2739 Z AIDI , C ERQUIDES , C ARMAN AND W EBB Contact-lenses Haberman Tae Echocardiogram New-thyroid Post-operative Bupa Iris Wine Labor Glass3 Zoo Cleveland Hungarian Balance-scale Hepatitis Led House-votes-84 Pid Volcanoes Yeast Lyn Ttt Horse-colic Crx Car Cmc Ptn Bcw Vowel Vehicle German Autos Ionosphere Chess Sonar Abalone Dermatology Promoters Lung-cancer Page-blocks Anneal Segment Nursery Sign Sick Hypo Kr-vs-kp Mush Audio Magic Pendigits Syncon Waveform-5000 Thyroid Phoneme Cylinder-bands Splice-c4.5 Spambase Musk1 Letter-recog Wall-following Shuttle Satellite Adult Optdigits Localization Connect-4 Covtype-mod Poker-hand Musk2 Census-income Pioneer Mean Mean Rank A1DE 0.075 0.098 0.098 0.098 0.099 0.099 0.099 0.099 0.1 0.1 0.1 0.104 0.106 0.107 0.108 0.111 0.113 0.115 0.116 0.12 0.121 0.121 0.121 0.124 0.125 0.128 0.129 0.131 0.132 0.137 0.14 0.145 0.156 0.16 0.161 0.162 0.168 0.183 0.192 0.192 0.22 0.235 0.256 0.268 0.271 0.274 0.293 0.303 0.349 0.379 0.408 0.462 0.466 0.474 0.555 0.565 0.623 0.664 0.665 0.707 0.788 0.876 0.993 1.02 1.025 1.375 1.977 4.695 10.856 15.15 19.881 20.808 309.15 5.50 2.2941 LR 0.117 0.114 0.895 0.198 0.2 0.294 0.112 0.166 0.231 0.166 0.556 0.368 0.662 0.94 0.232 0.335 5.203 0.341 0.253 2.48 33.068 0.407 0.577 1.581 4.24 6.549 6.639 181.979 0.423 18.835 37.456 3.33 1.821 0.611 5.658 1.51 16.021 1.792 0.616 1.035 77.192 4.306 11.027 43.87 97.373 18.226 7.452 27.203 4.495 31.293 84.379 37.11 6.798 107.506 1839.199 2430.18 8.705 7.369 106.084 3.631 5570.466 13.34 61.266 1534.966 720.394 50.882 6113.835 2218.933 25589.054 32887.718 56.031 11189.489 835.65 1262.10 6.4109 LR-Reg 13.765 2.67 39.861 7.756 13.392 14.513 3.486 6.629 10.128 16.176 48.909 17.779 41.424 67.985 10.895 17.759 325.834 16.883 14.32 109.075 1091.32 24.48 62.348 78.819 171.602 469.286 243.93 3127.046 20.418 1901.16 1051.993 326.224 212.885 36.326 308.214 150.955 883.628 168.475 24.846 65.012 3062.031 555.919 1455.738 2885.603 3309.138 1497.482 809.711 1483.335 587.965 6576.503 4453.728 4665.818 816.926 7163.793 42695.323 78868.508 1006.751 963.412 6027.538 837.027 892.459 2915.697 5237.291 9646.135 31035.38 6635.504 226842.296 95760.528 519389.101 751710.215 19571.881 438360.993 8835.12 14865.08 7.9726 NB 0.067 0.088 0.086 0.086 0.093 0.083 0.097 0.088 0.096 0.085 0.089 0.088 0.105 0.1 0.1 0.095 0.105 0.103 0.107 0.114 0.111 0.094 0.113 0.108 0.108 0.117 0.117 0.094 0.106 0.114 0.115 0.12 0.096 0.113 0.123 0.12 0.146 0.111 0.106 0.098 0.166 0.124 0.136 0.237 0.257 0.195 0.193 0.195 0.235 0.125 0.32 0.248 0.135 0.237 0.273 0.15 0.119 0.229 0.256 0.188 0.363 0.198 0.756 0.255 0.74 0.303 1.834 1.748 7.295 12.815 0.583 7.233 0.328 0.58 1.1369 WANBIACLL 0.063 0.08 0.111 0.108 0.121 0.111 0.084 0.115 0.189 0.132 0.159 0.193 0.2 0.269 0.091 0.2 0.341 0.246 0.225 0.111 0.598 0.262 0.202 0.376 0.668 0.123 0.535 0.965 0.312 0.921 0.729 0.556 0.675 0.363 0.65 0.356 1.953 1.84 0.395 0.311 3.405 7.262 4.472 0.476 2.234 13.786 15.659 6.525 3.97 4.846 4.507 13.162 5.481 5.051 187.224 12.683 1.788 9.889 12.773 11.641 73.676 4.933 37.254 16.169 33.376 116.538 92.014 91.768 683.568 59.807 54.759 1082.365 658.631 45.85 4.0958 Table 24: Train time 1984 WANBIAMSE 0.074 0.077 0.131 0.116 0.132 0.106 0.087 0.123 0.236 0.154 0.178 0.268 0.234 0.319 0.086 0.231 0.444 0.329 0.23 0.116 0.714 0.345 0.203 0.385 0.637 0.127 0.579 1.278 0.398 1.313 0.962 0.559 0.93 1.045 0.756 0.429 1.972 3.228 0.411 0.246 3.923 10.031 5.516 0.507 3.712 15.394 21.443 8.376 7.46 8.245 4.668 19.172 6.478 6.559 272.806 15.977 1.828 14.683 17.797 23.526 96.965 6.254 47.905 26.975 41.734 152.671 98.267 108.129 843.389 66.984 198.24 1129.399 564.602 53.01 5.1095 RF100 0.621 0.655 0.965 0.733 0.679 0.762 0.693 0.661 0.735 0.663 1.018 0.737 1.258 1.233 1.072 0.843 2.831 1.039 2.152 1.045 3.914 0.883 2.26 1.373 2.097 2.556 5.766 3.086 1.059 3.79 3.98 3.468 1.22 1.151 3.069 1.794 14.309 1.403 0.86 0.772 6.038 2.345 4.545 17.253 29.188 7.901 5.492 9.458 5.825 3.003 56.51 25.167 2.129 19.181 35.287 36.345 3.009 11.431 17.634 4.597 87.782 12.4 63.185 18.788 199.569 28.835 938.448 691.22 27.884 2307.386 296.181 71.17 5.8356 TAN 0.071 0.096 0.097 0.092 0.1 0.101 0.101 0.099 0.108 0.105 0.108 0.124 0.119 0.118 0.115 0.114 0.13 0.129 0.118 0.123 0.14 0.118 0.127 0.147 0.145 0.137 0.139 0.144 0.138 0.164 0.167 0.168 0.175 0.218 0.212 0.227 0.195 0.243 0.31 0.214 0.233 0.294 0.347 0.307 0.308 0.412 0.415 0.533 0.506 0.616 0.48 0.735 0.769 0.917 0.915 0.736 0.586 1.376 1.417 1.508 1.336 1.276 1.291 2.135 1.465 3.203 2.259 10.542 15.292 18.964 36.344 50.237 671.052 11.43 3.1438 A LLEVIATING NB ATTRIBUTE I NDEPENDENCE A SSUMPTION BY ATTRIBUTE W EIGHTING Contact-lenses Post-operative Labor Iris Tae New-thyroid Echocardiogram Bupa Hepatitis Glass3 Haberman Wine Zoo Lyn Cleveland Balance-scale Pid Hungarian Lung-cancer Bcw Volcanoes Tt House-votes-84 Autos Car Promoters Horse-colic Crx Ptn Cmc Led Vowel Yeast Ionosphere Dermatology Vehicle Sonar German Abalone Chess Cylinder-bands Page-blocks Segment Anneal Sign Nursery Sick Audio Syncon Kr-vs-kp Hypo Magic Mush Phoneme Wall-following Pendigits Waveform-5000 Musk1 Splice-c4.5 Spambase Adult Shuttle Satellite Thyroid Letter-recog Localization Optdigits Connect-4 Pioneer Covtype-mod Musk2 Poker-hand Census-income Mean Mean Rank A1DE 0.006 0.017 0.018 0.02 0.021 0.023 0.024 0.028 0.029 0.029 0.029 0.031 0.033 0.033 0.036 0.036 0.039 0.039 0.041 0.045 0.048 0.051 0.052 0.053 0.053 0.057 0.057 0.058 0.06 0.061 0.062 0.064 0.068 0.073 0.075 0.075 0.079 0.085 0.091 0.101 0.103 0.165 0.192 0.193 0.256 0.291 0.307 0.343 0.351 0.359 0.395 0.409 0.439 0.465 0.716 0.808 0.848 0.929 1.088 1.223 1.458 1.709 1.989 2.261 2.664 3.515 5.117 12.134 23.738 25.162 33.621 39.21 50.673 2.945 7.1438 LR 0.0015 0.0075 0.0045 0.0045 0.0075 0.003 0.0045 0.0075 0.006 0.009 0.006 0.006 0.0135 0.0105 0.012 0.012 0.009 0.006 0 0.0105 0.018 0.015 0.012 0.0105 0.024 0.0045 0.012 0.0165 0.0165 0.0135 0.018 0.027 0.03 0.0105 0.0105 0.009 0.003 0.0105 0.0345 0.0135 0.0105 0.066 0.039 0.0225 0.105 0.138 0.048 0.024 0.0195 0.042 0.0615 0.1455 0.093 0.3885 0.081 0.231 0.078 0.0165 0.066 0.0825 0.4635 0.831 0.114 0.372 0.906 3.1875 0.2145 1.2315 2.8455 10.251 0.2325 21.9015 4.839 0.679 2.9794 LR-Reg 0.003 0.009 0.006 0.015 0.0105 0.0135 0.015 0.024 0.012 0.021 0.0225 0.018 0.0105 0.015 0.021 0.045 0.0285 0.033 0.0015 0.0225 0.0495 0.0675 0.015 0.018 0.057 0.012 0.018 0.0375 0.201 0.0405 0.054 0.1125 0.081 0.015 0.036 0.1965 0.0255 0.093 0.1155 0.045 0.033 0.2535 0.1545 0.0645 0.2835 0.4845 0.21 0.114 0.0855 0.1365 0.1905 0.4125 0.261 1.2345 0.2115 0.6675 0.291 0.0675 0.192 0.2685 1.3635 2.4285 0.39 0.9375 0.918 8.5485 0.657 2.4525 7.3455 10.251 0.714 21.9015 10.836 1.041 5.9383 NB 0.001 0.005 0.007 0.01 0.008 0.006 0.006 0.007 0.009 0.009 0.008 0.008 0.005 0.009 0.012 0.01 0.013 0.009 0.002 0.013 0.023 0.014 0.009 0.015 0.019 0.008 0.009 0.014 0.02 0.017 0.017 0.022 0.027 0.007 0.017 0.02 0.009 0.018 0.026 0.014 0.017 0.046 0.035 0.024 0.07 0.099 0.038 0.024 0.023 0.036 0.05 0.096 0.063 0.259 0.061 0.178 0.073 0.023 0.058 0.073 0.318 0.561 0.123 0.359 0.697 1.947 0.24 0.972 1.649 7.245 0.24 15.605 3.92 0.489 2.9315 WANBIACLL 0.003 0.004 0.005 0.007 0.009 0.007 0.006 0.01 0.009 0.008 0.008 0.006 0.004 0.007 0.006 0.012 0.009 0.008 0.003 0.012 0.02 0.014 0.011 0.009 0.022 0.005 0.012 0.009 0.011 0.012 0.014 0.019 0.02 0.01 0.01 0.016 0.007 0.011 0.026 0.011 0.012 0.041 0.034 0.023 0.065 0.094 0.036 0.023 0.02 0.037 0.045 0.092 0.061 0.24 0.063 0.173 0.064 0.021 0.057 0.068 0.295 0.55 0.122 0.361 0.702 1.879 0.232 0.965 1.453 7.112 0.245 15.603 3.933 0.481 2.1027 Table 25: Test time 1985 WANBIAMSE 0.002 0.009 0.005 0.006 0.006 0.009 0.004 0.009 0.007 0.006 0.011 0.008 0.006 0.008 0.008 0.018 0.011 0.009 0.005 0.007 0.02 0.012 0.01 0.006 0.022 0.003 0.012 0.01 0.012 0.01 0.016 0.022 0.02 0.005 0.01 0.011 0.008 0.015 0.023 0.011 0.011 0.04 0.034 0.021 0.071 0.095 0.036 0.024 0.02 0.037 0.05 0.088 0.065 0.245 0.064 0.173 0.064 0.021 0.059 0.066 0.301 0.558 0.122 0.357 0.699 1.895 0.235 0.978 1.462 7.196 0.245 15.118 3.845 0.475 2.2191 RF100 0.005 0.011 0.006 0.022 0.021 0.023 0.022 0.018 0.022 0.039 0.013 0.019 0.016 0.022 0.05 0.087 0.145 0.053 0.005 0.06 0.08 0.183 0.055 0.036 0.264 0.013 0.058 0.12 0.135 0.471 0.29 0.298 0.391 0.046 0.059 0.21 0.044 0.218 1.793 0.165 0.095 0.868 0.446 0.147 5.943 3.54 0.943 0.087 0.092 1.02 0.603 10.413 0.788 3.489 1.179 4.234 1.834 0.104 0.924 1.727 22.379 7.409 1.901 5.804 11.936 93.887 2.463 47.681 7.09 1.193 131.887 5.319 7.2465 TAN 0.005 0.015 0.01 0.016 0.021 0.018 0.013 0.018 0.019 0.023 0.019 0.018 0.014 0.018 0.023 0.022 0.023 0.022 0.009 0.025 0.033 0.027 0.021 0.018 0.033 0.013 0.024 0.026 0.035 0.035 0.036 0.045 0.042 0.021 0.026 0.036 0.026 0.029 0.046 0.026 0.028 0.081 0.07 0.047 0.114 0.159 0.072 0.068 0.046 0.066 0.099 0.161 0.109 0.479 0.13 0.352 0.133 0.049 0.122 0.123 0.522 0.965 0.273 0.853 1.484 2.782 0.598 1.849 16.145 12.344 0.549 24.526 6.667 1.000 5.4383 Z AIDI , C ERQUIDES , C ARMAN AND W EBB B. Cestnik, I. Kononenko, and I. Bratko. Assistant 86: A knowledge-elicitation tool for sophisticated users. In Proceedings of the Second European Working Session on Learning, pages 31–45. Wulmslow, UK: Sigma Press, 1987. B. Damien and G. I. Webb. The need for low bias algorithms in classification learning from small data sets. In Proceedings of the Sixth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) 2002, pages 62–73, Berlin, 2002. Springer-Verlag. M. H. DeGroot and S. E. Fienbert. The comparison and evaluation of forecasters. Statistician, 32 (1):12–22, 1982. J. Demˇar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learns ing Research, 7:1–30, 2006. P. Domingos and M. J. Pazzani. Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 105–112. Morgan Kaufmann, 1996. R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley and Sons, 2006. U. M. Fayyad and B. I. Keki. On the handling of continuous-valued attributes in decision tree generation. Machine Learning, 8(1):87–102, 1992. J. T. A. S. Ferreira, D. G. T. Denison, and D. J. Hand. Weighted naive Bayes modelling for data mining, 2001. A. Frank and A. Asuncion. UCI machine learning repository, 2010. URL http://archive.ics. uci.edu/ml. J. Friedman. Flexible metric nearest neighbor classification. Technical report, Department of Statistics, Stanford University, 1994. M. Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. The American Statistical Association, 32(200):675–701, 1937. M. Friedman. A comparison of alternative tests of significance for the problem of m rankings. The American Statistical Association, 11(1):86–92, 1940. N. Friedman and M. Goldszmidt. Building classifiers using Bayesian networks. In AAAI-96, pages 1277–1284, 1996. N. Friedman, D. Geiger, and M. Goldszmidt. Bayesian network classifiers. Machine Learning, 29 (2):131–163, 1997. R. Greiner, W. Zhou, X. Su, and B. Shen. Structural extensions to logistic regression: Discriminative parameter learning of belief net classifiers. Journal of Machine Learning Research, 2004. D. Grossman and P. Domingos. Learning Bayesian network classifiers by maximizing conditional likelihood. In International Conference on Machine Learning, 2004. 1986 A LLEVIATING NB ATTRIBUTE I NDEPENDENCE A SSUMPTION BY ATTRIBUTE W EIGHTING I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh. Feature Extraction, Foundation and Applications. Springer, 2004. M. A. Hall. Correlation-based feature selection for discrete and numeric class machine learning. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 359–366. Morgan Kaufmann, 2000. M. A. Hall. A decision tree-based attribute weighting filter for naive Bayes. Knowledge-Based Systems, 20:120–126, March 2007. D. J. Hand and K. Yu. Idiot’s Bayes - not so stupid after all? International Statistical Review, 69 (3):385–398, 2001. J. Hilden and B. Bjerregaard. Computer-aided diagnosis and the atypical case. In In Decision Making and Medical Care: Can Information Science Help, pages 365–378. North-Holland Publishing Company, 1976. R. Kohavi and D. Wolpert. Bias plus variance decomposition for zero-one loss functions. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 275–283. San Francisco: Morgan Kaufmann, 1996. I. Kononenko. Comparison of inductive and naive Bayesian learning approaches to automatic knowledge acquisition. Current Trends in Knowledge Acquisition, 1990. P. Langley and S. Sage. Induction of selective Bayesian classifiers. In Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pages 399–406. Morgan Kaufmann, 1994. P. Langley, W. Iba, and K. Thompson. An analysis of Bayesian classifiers. In Proceedings of the Tenth National Conference on Artificial Intelligence, pages 223–228. AAAI Press and MIT Press, 1992. D. D. Lewis. Naive Bayes at forty: The independence assumption in information retrieval. In ECML-98: Proceedings of the Tenth European Conference on Machine Learning, pages 4–15, Berlin, April 1998. Springer. A. Murphy and R. Winkler. Reliability of subjective probability forecasts of precipitation and temperature. Applied Statistics, 26(1):41–47, 1977. F. Pernkopf and J. A. Bilmes. Efficient heuristics for discriminative structure learning of Bayesian network classifiers. Journal of Machine Learning Research, 2010. C. A. Ratanamahatana and D. Gunopulos. Feature selection for the naive Bayesian classifier using decision trees. Applied Artificial Intelligence, pages 475–487, 2003. T. Roos, H. Wettig, P. Gr¨ nwald, P. Myllym¨ ki, and H. Tirri. On discriminative Bayesian network u a classifiers and logistic regression. Machine Learning, 59(3):267–296, 2005. G. I. Webb. Multiboosting: A technique for combining boosting and wagging. Machine Learning, 40(2):159–196, 2000. 1987 Z AIDI , C ERQUIDES , C ARMAN AND W EBB G. I. Webb, J. Boughton, and Z. Wang. Not so naive Bayes: Averaged one-dependence estimators. Machine Learning, 58(1):5–24, 2005. G. I. Webb, J. Boughton, F. Zheng, K. M. Ting, and H. Salem. Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification. Machine Learning, pages 1–40, 2011. J. Wu and Z. Cai. Attribute weighting via differential evolution algorithm for attribute weighted naive Bayes (wnb). Journal of Computational Information Systems, 7(5):1672–1679, 2011. B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates. In ACM SIGKDD, International Conference on Knowledge Discovery and Data Mining, 2002. H. Zhang and S. Sheng. Learning weighted naive Bayes with accurate ranking. In Proceedings of the Fourth IEEE International Conference on Data Mining (ICDM -04), pages 567–570, 2004. F. Zheng, G. I. Webb, P. Suraweera, and L. Zhu. Subsumption resolution: An efficient and effective technique for semi-naive Bayesian learning. Machine Learning, 87(1):93–125, 2012. Z. Zheng and G.I. Webb. Lazy learning of Bayesian rules. Machine Learning, 41(1):53–84, 2000. Z. Zheng, G. I. Webb, and K.M. Ting. Lazy Bayesian Rules: A lazy semi-naive Bayesian learning technique competitive to boosting decision trees. In Proceedings of the Sixteenth International Conference on Machine Learning (ICML-99), pages 493–502. Morgan Kaufmann, 1999. C Zhu, R. H. Byrd, and J. Nocedal. L-bfgs-b: Algorithm 778: L-bfgs-b, fortran routines for large scale bound constrained optimization. ACM Transactions on Mathematical Software, 23(4):550– 560, 1997. 1988