0.25
0.5
0.75
1.25
1.5
1.75
2
Model Compression
Published on 2007-02-254549 Views
Decision trees are intelligible, but do they perform well enough that you should use them? Have SVMs replaced neural nets, or are neural nets still best for regression, and SVMs best for classificatio
Presentation
Model Compression12:45
Supervised Learning32:40:37
Normalized Scores for ES61:17:10
Ensemble Selection Works, But Is It Worth It?385:11:23
Computational Cost406:15:00
Ensemble Selection421:23:11
Best Ensembles are Big & Ugly!427:21:09
Best Ensembles are Big & Slow!444:02:05
Can’t we make the ensembles smaller, faster, and easier to use by eliminating some base-level models?473:38:41
What Models are Used in Ensembles?477:05:28
What Models are Used in Ensembles?507:03:31
Summary of Models Used by ES537:22:30
Motivation: Model Compression552:49:14
Solution: Model Compression581:37:21
Why Mimic with Neural Nets?617:02:32
Unlabeled Data?655:57:19
Synthetic Data: True Distribution673:10:28
Synthetic Data: Small Sample678:48:01
Synthetic Data: Random680:31:10
Synthetic Data: Random684:33:10
Synthetic Data: Random692:40:58
Synthetic Data: NBE712:19:55
These don’t work well enough. Had to develop a new, better method.735:53:46
These don’t work well enough. Had to develop a new, better method. Munging [1. To imperfectly transform information. 2. To modify data in a way that cannot be described succinctly.]736:17:42
Munging746:26:29
Munging748:57:31
Munging824:22:57
Munging828:23:13
Synthetic Data: Munge830:56:19
Synthetic Data: Munge837:55:10
Synthetic Data: Munge864:18:50
Synthetic Data908:18:39
Now That We Have a Method to Generate Data, Let’s Do Some Compression912:15:48
Experimental Setup: Datasets916:56:30
Experimental Setup919:33:34
Average Results by Size925:15:39
Average Results by Size932:58:30
Average Results by Size956:33:20
Average Results by Size974:12:50
Letter.P1 Results983:33:11
Hs Results988:22:30
Average Results by HU1000:19:04
Letter.P1 Results1015:26:40
Letter.P2 Results1018:30:50
Letter Results1020:45:00
It Doesn’t Always Work As Well As We’d Like, Yet!1065:28:00
Covtype Results1066:33:20
Covtype Results1074:32:53
Covtype Results1091:10:58
Covtype Results1126:42:38
Adult Results1130:44:12
Adult Results1138:06:39
Adult Results1140:44:16
RMSE Results – 400K, 256 HU1261:17:26
We’re Retaining 97% of Accuracy of Target Model, but How Are We Doing on Compression? 1277:01:00
Size of Models (MB)1278:34:43
Execution Time of Models1288:56:12
Summary of Compression Results1294:15:10
Related Work1297:15:59
Related Work1305:59:45
Related Work1329:42:57
Related Work1365:30:00
What Still Needs to Be Done?1371:50:25
Future Work: Other Mimic Models1372:16:30
Future Work: Other Target Metrics1375:07:21
Future Work: Model Complexity1395:29:38
Future Work: Munge1404:37:50
Future Work: Active Learning1467:34:46
Thank You. Questions?1500:03:58