Synth Full Bulk Times 30-Nov-2016 Initial queue size: 23476 packages built: 23334 ignored: 8 skipped: 111 failed: 23 Duration: 22:28:12 02-Dec-2016 (with spinlock changes) Initial queue size: 24496 packages built: 24371 ignored: 8 skipped: 111 failed: 6 Duration: 23:34:36 04-Dec-2016 (with extended changes) Initial queue size: 24496 packages built: 24200 ignored: 8 skipped: 286 failed: 2 Duration: 20:53:14 06-Dec-2016 (with shared lock/token changes) Initial queue size: 24496 packages built: 24370 ignored: 8 skipped: 113 failed: 5 Duration: 21:26:22 05-Jan-2017 Initial queue size: 24701 packages built: 23810 ignored: 78 skipped: 803 failed: 10 Duration: 18:57:26 Duration: 20:57:26 (approx, total, after make-up round) 07-Jan-2017 Initial queue size: 24701 (with machdep.pmap_dynamic_delete=1) packages built: 23808 ignored: 78 skipped: 801 failed: 14 Duration: 21:47:32 09-Jan-2017 Initial queue size: 24701 XEON packages built: 24385 ignored: 78 skipped: 226 failed: 12 Duration: 20:44:20 Initial queue size: 24701 MONSTER packages built: 24498 ignored: 78 skipped: 117 failed: 8 Duration: 24:34:37 10-Feb-2017 Initial queue size: 24877 XEON packages built: 24054 ignored: 79 skipped: 731 failed: 13 Duration: 21:14:39 19-Feb-2017 Initial queue size: 24888 XEON packages built: 24195 ignored: 80 skipped: 602 failed: 11 Duration: 21:02:17 22-Feb-2017 Initial queue size: 24888 XEON packages built: 24074 ignored: 80 skipped: 717 failed: 17 Duration: 20:53:27 08-Mar-2017 XEON Initial queue size: 24888 packages built: 24078 ignored: 80 skipped: 716 failed: 14 Duration: 20:48:56 15-Aug-2017 XEON (post scaling) Initial queue size: 25376 packages built: 24440 ignored: 90 skipped: 811 failed: 35 Duration: 20:55:51 03-Oct-207 XEON (with tmpfs KVABIO) (removed, too many failures due to bugs in prior version of patch, was 18:43:06 with 165 failures) 04-Oct-2017 XEON (with tmpfs KVABIO) Initial queue size: 25594 packages built: 24824 ignored: 95 skipped: 633 failed: 42 Duration: 20:39:24 04-Oct-2017 MONSTER (with tmpfs KVABIO) Initial queue size: 25594 packages built: 24852 ignored: 95 skipped: 617 failed: 30 Duration: 18:53:47 14-Oct-2017 MONSTER (tmpfs KVABIO mntlist etc) Initial queue size: 25612 packages built: - ignored: - skipped: - failed: 30-ish Duration: 18:32:00 (stuck in cargo with 6 left) 14-Oct-2017 Initial queue size: 25612 XEON (tmpfs KVABIO mntlist etc) packages built: 24868 ignored: 95 skipped: 605 failed: 44 Duration: 20:06:39 16-Oct-2017 MONSTER (95beea71d10d) Initial queue size: 25612 packages built: 24764 ignored: 95 skipped: 718 failed: 35 Duration: 18:12:03 17-Nov-2017 Initial queue size: 25612 XEON (-flto, flto partitioning ops commented packages built: 25370 out, and with INVARIANTS commented out) ignored: 95 skipped: 106 failed: 41 Duration: 21:25:56 (very good showing for pkgs built) Concurrent stat() call, xeon, 32 threads, mlstat - shared directory, distinct target file, 6 comps mtstat - distinct directory and target file, 1 component mlstat /mnt2/test/test2/test3/test4/ mtstat /mnt2/test/test2/test3/test4/ On xeon mlstat (6 components) mtstat (1 component) 17-Nov-2017 762000/sec 15.5M/sec ----------------- DPORTS WITH FLAVORS 03-May-2018 Initial queue size: 29898 RYZEN 2700X (8/16 w/DOCP 2666) packages built: 28589 ignored: 109 skipped: 1146 failed: 54 Duration: 1 day and 04:55:51 (28:55:41) Initial queue size: 29903 packages built: 28615 ignored: 109 skipped: 1127 failed: 52 Duration: 1 day and 04:09:40 (28:09:40) 18-Aug-2018 TR2 2990WX Initial queue size: 30517 packages built: 29598 ignored: 114 skipped: 735 failed: 70 Duration: 12:03:58 19-Oct-2018 TR2 2990WX 128GB@2133 ECC Initial queue size: 31453 250W at wall packages built: 28837 ignored: 87 skipped: 2422 failed: 107 Duration: 11:44:01 This is the first real full test with the 2990WX with sufficient ram to support fully loading all cores during the run. Currently need to fix the skips and run the same test with the dual-socket Xeon, but the Xeon's best time from December was around ~21 hours which puts the threadripper at nearly twice as fast with double the number of cores but only 1/2 the number of memory channels. ----------------- DPORTS WITH FLAVORS OCT 25 2018 DPORTS Sync NOV 24 2018 DFly GCC-8 23-Nov-2018 MONSTER (quad opteron, 48-cores total) Initial queue size: 31396 packages built: 31010 ignored: 68 skipped: 195 failed: 123 Duration: 26:59:20 Initial queue size: 31396 TR2 2990WX @ 265W packages built: 30523 Duration: 15:25:00 (required a restart due to a synth bug, times added together) This is the first real same-dports/same-world comparison between monster and the threadripper. Even with the typical run disparity, the TR is clearly much faster. In this run, around 1.75x the performance of monster and uses 1/4 the watts. Thus the TR has 7.0x the power efficiency of monster (the old quad-socket opteron). Initial queue size: 31396 TR2 2990WX @ unknown wattage packages built: 30035 ignored: 68 skipped: 1175 failed: 118 Duration: 14:11:28 Initial queue size: 31450 TR2 2990WX @ unknown wattage packages built: 31118 (full run, 30/30, max-load ~500 or so) ignored: 69 skipped: 139 failed: 124 Duration: 15:24:24 Initial queue size: 31450 TR2 2990WX @ unknown wattage packages built: 30548 (full run, 40/40) (tmpfs fix) ignored: 69 skipped: 720 failed: 113 Duration: 14:00:44 Initial queue size: 902 (incremental) packages built: 563 ignored: 69 skipped: 159 failed: 111 Duration: 01:01:02 Initial queue size: 29887 TR2 2990WX 26-Feb-2019 @ 270W packages built: 29746 (full run, 40/40) ignored: 27 skipped: 48 failed: 66 Duration: 15:20:15 Initial queue size: 30449 TR2 2990WX 29-Apr-2019 @ 270W packages built: 30296 (full run, 40/40) ignored: 14 skipped: 88 failed: 51 Duration: 16:27:55 Initial queue size: 30635 TR2 2990WX 21-May-2019 @ 270W packages built: 30411 (full run, 40/40, newvm) ignored: 10 skipped: 140 failed: 74 Duration: 16:11:15 Initial queue size: 30635 TR2 2990WX 28-May-2019 @ 270W packages built: 30423 (full run, 40/40, newvm, sched) ignored: 10 skipped: 122 failed: 80 Duration: 16:06:52 2780076.650u 457782.552s 16:16:37.48 5525.5% (14.13% system time) 9194+712k 1648147009+1514324io 2567900pf+0w Initial queue size: 30327 TR2 2990WX 09-Jun-2019 @ 270W packages built: 30028 (full run, 64/20, newvm, retpoline) ignored: 18 (experiment with different job mix) skipped: 206 failed: 75 Duration: 17:17:30 2840984.897u 470559.339s 17:27:05.13 5271.0% (14.20% system time) 9287+729k 1556743323+1751417io 4490413pf+0w Initial queue size: 30327 TR2 2990WX 10-Jun-2019 @ 270W packages built: 30032 (full run, 40/40, newvm, retpoline) ignored: 18 skipped: 194 failed: 83 Duration: 16:21:30 2732841.473u 474853.333s 16:30:48.82 5395.7% (14.80% system time) 9310+721k 1533259966+1504067io 2436212pf+0w Initial queue size: 30521 TR2 2990WX 14-Jun-2019 @ 270W packages built: 30251 32-core/64-thread ignored: 18 skipped: 185 failed: 67 Duration: 17:51:06 3064783.353u 513334.214s 18:00:29.81 5519.2% (14.3% system time) 9458+728k 1559999935+1639180io 3192170pf+0w Initial queue size: 30521 TR2 2990WX 24-Jun-2019 @ 270W packages built: 30255 32-core/64-thread ignored: 18 skipped: 174 failed: 74 Duration: 17:31:09 Initial queue size: 30599 RYZEN 3900X 12-Jul-2019 packages built: 30311 12-core/24-thread @ 140W (3.7Ghz/1.10V) ignored: 19 (this is comparatively insane) skipped: 172 failed: 97 Duration: 23:35:45 1501436.275u 209553.670s 23:46:07.75 1999.5% (12.2% system) 9324+708k 1554440531+1662484io 1502235pf+0w Initial queue size: 30599 RYZEN 3900X 14-Jul-2019 packages built: 30364 12-core/24-thread @ 160W (PPT limited) ignored: 19 Normal frequency management skipped: 148 (this is even more insane) failed: 68 Duration: 22:47:53 1423002.926u 201663.984s 22:56:53.92 1966.5% (12.4% system) 9122+713k 1576374422+1553830io 1604893pf+0w Initial queue size: 30599 RYZEN 3900X 16-Jul-2019 packages built: 30321 12-core/24-thread @ 150W (PPT limited ignored: 19 Normal frequency management skipped: 191 Mini-Itx mobo / 32G of ram failed: 68 Duration: 24:33:55 1449765.163u 205580.599s 24:43:14.08 1860.0% (12.4% system) 9441+713k 1584210901+1554194io 43728065pf+0w Initial queue size: 30599 RYZEN 3900X 19-Jul-2019 packages built: 30358 12-core/24-thread @ 150W (4.1GHz UC) ignored: 19 Fixed frequency skipped: 159 Mini-Itx mobo / 32G of ram failed: 63 Duration: 24:56:30 1425551.489u 214538.210s 25:06:11.96 1814.8% 9395+715k 1592099606+1535370io 32346060pf+0w Initial queue size: 30599 RYZEN 3900X 21-Jul-2019 packages built: 30340 Dynamic freq @ 160W ignored: 19 64G of ECC memory @ 2133 skipped: 177 (additional time due to slow memory) failed: 63 Duration: 26:15:04 1498001.313u 224457.406s 26:24:27.44 1811.8% 9424+714k 1568207512+1544446io 1603410pf+0w Initial queue size: 30521 MONSTER 13-Jun-2019 packages built: 30231 (full run, 32/12, newvm, retpoline) ignored: 86 48-core/48-thread skipped: 134 failed: 70 Duration: 27:25:13 (1 day + 03:25:13) Initial queue size: 30521 XEON 14-Jun-2019 packages built: 30295 (full run, 30/30, newvm, retpoline) ignored: 18 16-core/32-thread skipped: 140 failed: 68 Duration: 31:32:39 (1 day and 07:32:39) Initial queue size: 30521 RYZEN 2700X 18-Jun-2019 packages built: 30313 (full run, 15/10, newvm, retpoline) ignored: 18 8-core/16-thread. skipped: 129 failed: 61 Duration: 46:26:00 (1 day and 22:26:00) 907666.870u 116322.566s 20:05:41.43 1415.4% (11.4% system time) 9535+723k 662267949+1434694io 36998297pf+0w (user+system for last 17978 packages) Initial queue size: 30701 TR2 2990WX 04-Aug-2019 @ 270W packages built: 30359 ignored: 20 skipped: 234 failed: 88 Duration: 16:55:08 2822296.369u 475325.499s 17:04:59.63 5362.0% (14.4% system time) 9418+724k 1582086380+1555678io 3390032pf+0w Initial queue size: 30703 TR2 2990WX 24-Aug-2019 @ 270W packages built: 29440 dsynth ignored: 22 skipped: 1074 failed: 166 Duration: 15:29:40 2514583.178u 413126.968s 17:38:10.39 4611.2% (14.1% system time) 9862+762k 1530174737+1715380io 19668117pf+0w ----------------- DSYNTH WITH FLAVORS FEB 11 2020 Starting here I have a 3990X (64-core/128-thread TR3) in the racks and the first full test is to run it in approximately the same power envelope as I ran the 2990WX. And as you can see, the 3990X is able to complete the bulk build in half the time. Even though it has double the cores, it completed the operation IN THE SAME POWER ENVELOPE as the 2990WX, meaning that the 3990X is literally twice as power-efficient. Chromium building alone, -j128 takes roughly 12.5 minutes at 300W and runs at around 2.7 GHz (the compile phase takes 9:13). 12.6% system, 87.4% user. Worst case memory use is around 73GB. Chromium building alone, -j64 (cores only) takes roughly 14.5 minutes at 280W and runs at around 3.0 GHz. Same sys/user breakdown. Keeping 128 cpu threads busy with compile jobs in multiple build environments takes memory, this box has 256GB and even that is not quite enough to avoid a few periods of heavy paging to swap. This is because keeping the cpus busy with a diverse bulk builds requires over-subscribing cpu threads by 2x-4x times. This run also shows that there is also clearly room for improvement. The system time percentage is a bit higher (though only by 2-3%), probably due to SMP related issues with twice the cores and threads. Tests below are at 280W, 410W, and later 300W. 300W will be the production in-the-rack operating power. 410W is at stock settings. 280W was to try to get close to the TR2990WX for a comparative test. Initial queue size: 31465 TR3 3990X 11-Feb-2020 @ 280W (~2.7GHz) packages built: 31335 64-core/128-thread CPU ignored: 24 (Package Power Limit set to 180) skipped: 76 70/24 failed: 30 Duration: 08:10:06 2332171.079u 478533.950s 8:13:26.31 9493.6% (17.0% system time) 9751+759k 1837958440+3009487io 1781277pf+0w Initial queue size: 31465 TR3 3990X 12-Feb-2020 @ 300W (~3.0GHz) packages built: 31325 ignored: 24 skipped: 85 failed: 31 Duration: 08:18:49 2291334.883u 493646.267s 14:20:47.74 5392.2% (17.7% system time) 9825+763k 1843848991+4766190io 1659382pf+0w (again, note that total time from the shell 'time' result is not useful, only the user and system time, due to dsynth sitting at a prompt when it finishes). ----- Initial queue size: 31465 TR3 3990X 11-Feb-2020 @ 410W (~3.5GHz) packages built: 30796 ignored: 24 (Package Power Limit set to stock) skipped: 625 failed: 20 Duration: 06:41:46 <-- note followup below 1900659.224u 465526.576s 11:13:41.52 5853.7% (19.6% system time) 9431+764k 1766742872+2914923io 1733109pf+0w Initial queue size: 657 (INCREMENTAL DUE TO 625 SKIPPED) packages built: 600 (java/openjdk8 has trouble building) ignored: 24 skipped: 18 failed: 15 Duration: 00:49:47 Initial queue size: 31465 TR3 3990X 15-Feb-2020 @ 300W (~3.0GHz) packages built: 31346 ignored: 24 skipped: 75 failed: 20 Duration: 08:06:03 2369944.070u 560711.655s 13:04:15.97 6228.0% (19.1% system time) 9687+751k 1850993043+5022096io 2375969pf+0w Initial queue size: 31465 TR3 3990X 16-Feb-2020 @ 300W packages built: 31330 w/binutils & mjg mem funcs ignored: 24 skipped: 79 failed: 32 Duration: 08:07:02 2298534.489u 556441.916s 9:47:23.90 8100.6% (19.5% system time) 9688+752k 1850601005+4609216io 2678018pf+0w Initial queue size: 31258 TR3 3990X 28-Feb-2020 @ 300W packages built: 31099 w/namecache work ignored: 32 skipped: 106 failed: 21 Duration: 07:24:01 2223606.270u 485485.024s 7:29:40.99 10040.7% (17.9% system time) 9694+753k 1820648358+4794957io 2131602pf+0w Initial queue size: 31258 TR3 3990X 03-Mar-2020 @ 300W packages built: 31191 w/namecache & getattr work ignored: 32 skipped: 18 failed: 17 Duration: 08:03:13 2342610.745u 437605.204s 8:07:33.94 9503.7% (15.7% system time) 9905+767k 1843616477+5187666io 2094794pf+0w