 |
y-cruncher - A Multi-Threaded Pi-Program |
 |
From a high-school project that went a little too far...By Alexander J. Yee |
(Last updated: August 14, 2016)
Shortcuts:
The first scalable multi-threaded Pi-benchmark for multi-core systems...
How fast can your computer compute Pi?
y-cruncher is a program that can compute Pi and other constants to trillions of digits.
It is the first of its kind that is multi-threaded and scalable to multi-core systems. Ever since its launch in 2009, it has become a common benchmarking and stress-testing application for overclockers and hardware enthusiasts.
y-cruncher has been used to set several world records for the most digits of Pi ever computed.
- 13.3 trillion digits - October 2014 ("houkouonchi")
- 12.1 trillion digits - December 2013 (Shigeru Kondo)
- 10 trillion digits - October 2011 (Shigeru Kondo)
- 5 trillion digits - August 2010 (Shigeru Kondo)
Current Release:
Windows: Version 0.7.1 Build 9465 (Released: May 16, 2016)
Linux : Version 0.7.1 Build 9465 (Released: May 16, 2016)
Official HWBOT thread.
Official XtremeSystems Forums thread.
y-cruncher v0.7.1: (May 16, 2016)
This is an semi-unplanned released to address a number of critical issues with the HWBOT integration. (Most notably the reference clock skew issue.)
Other than that, there are few other user-visible features. Most of the changes since v0.6.9 are internal refactorings. Some of these were large (and dangerous) enough that it probably would've been better to wait a few more months before releasing v0.7.1. So if anything breaks, let me know.
While this version wasn't intended to have many new features, all that refactoring did lend itself to a some opportunistic stuff such as large pages and Unicode support.
GUI Benchmark Wrapper and HWBOT Integration: (April 3, 2016)
I get asked these two questions a lot:
- Why don't you add a GUI for y-cruncher?
- Why isn't y-cruncher on HWBOT?
#1 never happened because I suck at UI programming and I didn't want that mixed in with performance critical code.
#2 never happened because the HWBOT benchmark API wasn't ready.
Well, both finally happening... More details here: http://forum.hwbot.org/showthread.php?t=155079
y-cruncher has been used to set a number world record size computations.
Blue: Current World Record
Green: Former World Record
Red: Unverified computation. Does not qualify as a world record until verified using an alternate formula.
| Date Announced | Date Completed: | Source: | Who: | Constant: | Decimal Digits: | Time: | Computer: |
| August 14, 2016 | June 26, 2016 | Ron Watkins | Euler-Mascheroni Constant | 477,511,832,674 |
Not Verified |
4 x Xeon E5-4660 v3 @ 2.1 GHz 1 TB |
|
| July 11, 2016 | July 5, 2016 | "yoyo" | Golden Ratio | 10,000,000,000,000 |
Not Verified |
2 x Intel Xeon E5-2696 v4 @ 2.2 GHz 768 GB |
|
| June 28, 2016 | June 19, 2016 | Ron Watkins | Square Root of 2 | 10,000,000,000,000 | 2 x Xeon X5690 @ 3.47 GHz 141 GB |
||
| June 4, 2016 | May 29, 2015 | Ron Watkins | Lemniscate | 250,000,000,000 | 4 x Xeon E5-4660 v3 @ 2.1 GHz - 1TB 4 x Xeon X6550 @ 2 GHz - 512 GB |
||
| June 4, 2016 | May 29, 2016 | "yoyo" | e | 2,500,000,000,000 |
Not Verified |
2 x Intel Xeon E5-2696 v4 @ 2.2 GHz 768 GB |
|
| June 4, 2016 | June 2, 2016 | "yoyo" | Golden Ratio | 5,000,000,000,000 |
Not Verified |
2 x Intel Xeon E5-2696 v4 @ 2.2 GHz 768 GB |
|
| May 25, 2016 | May 18, 2016 | Ron Watkins | Euler-Mascheroni Constant | 250,000,000,000 | 2 x Xeon E5-4660 v3 @ 2.1 GHz - 1 TB 4 x Xeon X6550 @ 2.0 GHz - 512 GB |
||
| April 24, 2016 | April 18, 2016 | Ron Watkins | Log(2) | 500,000,000,000 | 4 x Xeon X5690 @ 3.47 GHz - 141 GB | ||
| April 17, 2016 | April 12, 2016 | Ron Watkins | Catalan's Constant | 250,000,000,000 | 4 x Xeon E5-4660 v3 @ 2.1 GHz 1 TB |
||
| April 9, 2016 | April 3, 2016 | Ron Watkins | Log(10) | 500,000,000,000 | 2 x Xeon X5690 @ 3.47 GHz 141 GB |
||
| February 17, 2016 | February 14, 2016 | Ron Watkins | e | 1,500,000,000,000 | 2 x Xeon X5690 @ 3.47 GHz 141 GB |
||
| February 8, 2016 | February 6, 2016 | Mike A | Catalan's Constant | 500,000,000,000 |
Not Verified |
2 x Intel Xeon E5-2697 v3 @ 2.6 GHz 128 GB |
|
| December 21, 2015 | December 21, 2015 | Dipanjan Nag | Zeta(3) - Apery's Constant | 400,000,000,000 | Xeon E5-2698B @ 2.0 GHz - 224 GB | ||
| July 24, 2015 | July 22, 2015 July 23, 2015 |
Source | Ron Watkins Dustin Kirkland |
Golden Ratio | 2,000,000,000,000 | 4 x Xeon X6550 @ 2 GHz - 512 GB Xeon E5-2676 v3 @ 2.4 GHz - 64 GB |
|
| October 8, 2014 | October 7, 2014 | "houkouonchi" | Pi | 13,300,000,000,000 |
|
2 x Xeon E5-4650L @ 2.6 GHz 192 GB DDR3 @ 1333 MHz 24 x 4 TB + 30 x 3 TB |
|
| December 28, 2013 | December 28, 2013 | Source | Shigeru Kondo | Pi | 12,100,000,000,050 | 2 x Xeon E5-2690 @ 2.9 GHz 128 GB DDR3 @ 1600 MHz 24 x 3 TB |
See the complete list including other notably large computations.
Note that for anyone attempting to set a Pi world record: Should the attempt succeed, I kindly ask that you make yourself sufficiently available for external requests to access or download the digits in its entirety (at least until it is broken again by someone else).
Pi is popular enough that people do actually want to see the digits. Everything else is less important. Though it's worth keeping the digits for large computations of the slower constants as well since they are much more difficult to compute.
- Computes Pi and other constants.
- Capable of computing trillions of digits of Pi (among other constants).
- Two algorithms are available for each major constant. One for computation and one for verification. (great for stress-testing)
- Extremely efficient even for large computations (> 1 billion digits).
- Swap Space management for large computations that require more memory than there is available.
- Multi-Threaded - Multi-threading can be used to fully utilize modern multi-core processors without significantly increasing memory usage.
For large computations, it scales almost linearly with the # of cores. - Multi-Hard Drive - Multiple hard drives can be used for faster disk swapping. (scales linearly - as fast as hardware Raid 0, with no limit to the # of drives)
- Semi-Fault Tolerant - Able to detect and correct for minor errors that may be caused by hardware instability or software bugs.
Aside from computing Pi and other constants, y-cruncher is great for stress testing 64-bit systems with lots of ram.
Sample Screenshot: 100 billion digits of Pi
|
| Core i7 4770K @ 4.0 GHz - 32GB DDR3 @ 2400 MHz - 16 HDs |
Latest Releases: (May 16, 2016)
OS Programs Download Link Size Windows
y-cruncher + HWBOT Submitter
16.8 MB Linux (Static)
y-cruncher Only
14.2 MB Linux (Dynamic)
y-cruncher Only
9.90 MB Windows
HWBOT Submitter Only
2.53 MB
The Linux version comes in both statically and dynamically linked versions. The static version should work on most Linux distributions, but lacks Cilk Plus. The dynamic version supports Cilk Plus, but is less portable due to the DLL dependency hell.
The HWBOT submitter allows y-cruncher benchmarks to be submitted to HWBOT - which is a competitive overclocking site. It is currently only available for Windows.
System Requirements:
Windows:
- Windows Vista or later.
- The HWBOT submitter requires the Java 8 Runtime.
Linux:
- 64-bit Linux is required. There is no support for 32-bit.
- The dynamic version has been tested on Ubuntu 15.10 and 16.04.
All Systems:
- An x86 or x64 processor.
Very old systems that don't meet these requirements may be able to run older versions of y-cruncher. Support goes all the way back to even before Windows XP.
Version History:
Other Downloads (for C++ programmers):
Advanced Documentation:
Functionality Issues:
- Support for Cilk Plus in Linux has limited portability due to the DLL dependency hell that is inherent to dynamic linking.
- On Windows, the program doesn't behave well when there are unbalanced processor groups. This will happen when a NUMA node has more than 64 logical cores. Disabling NUMA on a multi-socket machine with more than 64 logical cores total will also trigger this. As far as the program goes, the stress-tester, the BBP app, and the Push Pool framework make the (incorrect) assumption that processor groups are relatively balanced. So they will not be able to fully utilize the hardware if the groups are imbalanced. Computations that use the Cilk Plus framework will be okay since it's written by Intel and they know what they're doing.
Performance Issues:
- Amdahl's Law is apparent on very large machines. I recently had the opportunity to play with an 88-vcore Broadwell-EP system and it was obvious just by looking at Task Manager. This is caused by a number of unparallelized linear operations such as bignum addition and subtraction. These operations have historically been memory-bound and were largely neglected as far as optimizations go. At this point, we're entering an era where a parallelized bignum multiply may actually be faster than an unparallelized bignum add. There are currently no plans to fix this as there are bigger problems elsewhere such as NUMA.
- Performance scaling is very poor on systems that are heavily NUMA. Back in the Nehalem and Sandy Bridge days, it was only noticable on systems with 4 or more sockets. Now it's also a problem on dual-socket systems. See the FAQ for more details.
- Swap computations on the latest Ubuntu (15.10) and possibly everything else with the same kernel version have very poor performance in swap mode. This is because the OS does excessive and unnecessary disk swapping to the pagefile. The solution is to disable the swap file so that no paging is possible. It may also suffice to set the "swappiness" value to zero. y-cruncher will attempt to lock pages in memory to prevent this.
- For Intel processors, computations are noticably slower in Linux than in Windows.
- Approximately 1% of this difference is attributed to the Intel Compiler being slightly better than the GCC compiler.
- 2% of the difference is caused by the disabling of one particular optimization that gets miscompiled by the GCC compiler.
- The remaining difference is because multi-threading in Linux is less efficient than in Windows.
For 5 billion digits on a Core i7 4770K (4-cores/8-threads), the parallel speedup is 4.24x on Windows and 4.07 on Linux. The exact reason for this is unknown. But there is a lot of variation in Linux depending on the parallel framework and whether the binary is dynamically vs. statically linked. 4.07x is the best that could be achieved. Most other options are significantly worse. Furthermore, nothing seems be able to match the efficiency of the Windows thread pool. - In Linux, swap mode computations are generally 5 - 10% slower than in Windows using the same hardware. On Windows, raw I/Os using the flag FILE_FLAG_NO_BUFFERING are a true DMA. There is no overhead, and no buffering by the OS. On Linux, the same cannot be said about the O_DIRECT flag. Despite having the same sector alignment restrictions, it is not a DMA, and there is significant overhead from disk caching and other unknown sources.
So while it may be difficult to believe, Windows is currently the more suitable OS for running y-cruncher.
Comparison Chart: (Last updated: May 14, 2016)
Computations of Pi to various sizes. All times in seconds.
The timings include the time needed to convert the digits to decimal representation, but not the time needed to write out the digits to disk.
Laptops:
| Processor(s): | Core i7 3630QM | Core i7 6820HK |
| Generation: | Intel Ivy Bridge | Intel Skylake |
| Cores/Threads: | 4/8 | 4/8 |
| Processor Speed: | 2.4 GHz (3.2 GHz turbo) | 3.2 GHz |
| Memory: | 8 GB - 1600 MHz | 48 GB - 2133 MHz |
| Version: | v0.7.1 - AVX | v0.7.1 - ADX |
| 25,000,000 | 3.823 | 1.830 |
| 50,000,000 | 8.602 | 3.979 |
| 100,000,000 | 19.153 | 8.794 |
| 250,000,000 | 56.130 | 24.877 |
| 500,000,000 | 131.078 | 54.869 |
| 1,000,000,000 | 313.130 | 121.586 |
| 2,500,000,000 | 346.573 | |
| 5,000,000,000 | 765.084 | |
| 10,000,000,000 | 1665.325 |
Single-Processor Desktops:
| Processor(s): | Core 2 Quad Q6600 | Core i7 920 | FX-8350 | Core i7 4770K | Core i7 5960X | Core i7 5775C* | Core i7 6700K** |
| Generation: | Intel Core | Intel Nehalem | AMD Piledriver | Intel Haswell | Intel Haswell | Intel Broadwell | Intel Skylake |
| Cores/Threads: | 4/4 | 4/8 | 8/8 | 4/8 | 8/16 | 4/8 | 4/8 |
| Processor Speed: | 2.4 GHz | 3.5 GHz (OC) | 4.0 GHz (4.2 GHz turbo) | 4.0 GHz (OC) | 4.0 GHz (OC) | 3.8 GHz (OC) | 4.6 GHz (OC) |
| Memory: | 6 GB - 800 MHz | 12 GB - 1333 MHz | 32 GB - 1333 MHz | 32 GB - 2400 MHz | 64 GB - 2400 MHz | 16 GB - 2400 MHz | 64 GB - 2800 MHz |
| Version: | v0.7.1 - SSE3 | v0.7.1 - SSE4.1 | v0.7.1 - XOP | v0.7.1 - AVX2 | v0.6.9 - AVX2 | v0.7.1 - ADX | v0.6.9 - AVX2 |
| 25,000,000 | 10.880 | 5.597 | 3.867 | 1.740 | 1.018 | 1.730 | 1.360 |
| 50,000,000 | 23.984 | 12.223 | 8.619 | 3.867 | 2.347 | 3.940 | 3.117 |
| 100,000,000 | 53.505 | 27.254 | 19.278 | 8.579 | 4.978 | 8.739 | 6.756 |
| 250,000,000 | 156.763 | 80.000 | 54.761 | 24.227 | 13.695 | 25.073 | 19.857 |
| 500,000,000 | 354.178 | 178.933 | 121.721 | 53.865 | 30.276 | 56.343 | 45.579 |
| 1,000,000,000 | 803.675 | 406.073 | 266.824 | 119.836 | 66.807 | 125.967 | 98.891 |
| 2,500,000,000 | 765.289 | 338.446 | 189.518 | 369.738 | 284.260 | ||
| 5,000,000,000 | 1701.848 | 751.700 | 416.514 | 609.001 | |||
| 10,000,000,000 | 919.385 | 1328.204 |
*Credit to André Bachmann.
**Credit to Oliver Kruse.
Multi-Processor Workstation/Servers:
| Processor(s): | 2 x Xeon X5482 | 2 x Xeon E5-2690* | 2 x Xeon E5-2683 v3* | 2 x Xeon E5-2696 v4** |
| Generation: | Intel Penryn | Intel Sandy Bridge | Intel Haswell | Intel Broadwell |
| Cores/Threads: | 8/8 | 16/32 | 28/56 | 44/88 |
| Processor Speed: | 3.2 GHz | 3.5 GHz | 2.03 GHz | 2.2 GHz |
| Memory: | 64 GB - 800 MHz | 256 GB - ??? | 128 GB | 768 GB |
| Version: | v0.6.9 - SSE4.1 | v0.6.2/3 - AVX | v0.6.9 - AVX2 | v0.7.1 - ADX |
| 25,000,000 | 6.578 | 2.283 | 0.907 | 0.715 |
| 50,000,000 | 12.965 | 4.295 | 1.745 | 1.344 |
| 100,000,000 | 26.194 | 8.167 | 3.317 | 2.673 |
| 250,000,000 | 71.395 | 20.765 | 8.339 | 6.853 |
| 500,000,000 | 157.234 | 42.394 | 17.708 | 14.538 |
| 1,000,000,000 | 347.962 | 89.920 | 37.311 | 31.260 |
| 2,500,000,000 | 979.043 | 239.154 | 102.131 | 84.271 |
| 5,000,000,000 | 2170.838 | 520.977 | 218.917 | 192.889 |
| 10,000,000,000 | 4768.905 | 1131.809 | 471.802 | 417.322 |
| 25,000,000,000 | 3341.281 | 1511.852 | 1186.881 | |
| 50,000,000,000 | 7355.076 | 2601.476 | ||
| 100,000,000,000 | 6037.704 |
*Credit to Shigeru Kondo.
**Credit to "yoyo".
The full chart of rankings for each size can be found here:
- Rankings: 25m
- Rankings: 50m
- Rankings: 100m
- Rankings: 250m
- Rankings: 500m
- Rankings: 1b
- Rankings: 2.5b
- Rankings: 5b
- Rankings: 10b
- Rankings: 25b
- Rankings: 50b
- Rankings: 100b
- Rankings: 250b
- Rankings: 500b
- Rankings: 1t
These fastest times may include unreleased betas.
Got a faster time? Let me know: a-yee@u.northwestern.edu
Note that I usually don't respond to these emails. I simply put them into the charts which I update periodically.
Pi and other Constants:
- Where can I download the digits for Pi and other constants that are featured in the various record computations?
- Can you search for XXX in the digits of Pi for me?
- Can you add support for more constants? I want to compute ePi, Khinchin, Glaisher, etc...
Hardware and Overclocking:
Academia:
- Is there a version that can use the GPU?
- Why can't you use distributed computing to set records?
- Is there a distributed version that performs better on NUMA and HPC clusters?
Programming:
- Is there a publicly available library for the multi-threaded arithmetic that y-cruncher uses?
- Is y-cruncher open-sourced?
Program Usage:
- What's the difference between "Total Computation Time" and "Total Time"? Which is relevant for benchmarks?
- Why does y-cruncher need administrator privileges in Windows to run Swap Mode computations?
- Why is the performance so poor for small computations? The program only gets xx% CPU utilization on my xx core machine for small sizes!!!
Other:
Here's some interesting sites dedicated to the computation of Pi and other constants:
Contact me via e-mail. I'm pretty good with responding unless it gets caught in my school's junk mail filter.