FASTER PASSWORD RECOVERY WITH MODERN GPUs

Transcription

1

2 FASTER PASSWORD RECOVERY WITH MODERN GPUs Andrey Belenko ElcomSoft Co. Ltd. Security Researcher

3 WHO ARE WE Founded in 1990 Privately owned Doing password recovery (software) since 1998 HQ and development in Moscow, Russia Brought GPUs to password recovery in US patents issued, more in queue 2 are about GPU-accelerated password recovery 3

4 WHO NEEDS PASSWORD RECOVERY? Ordinary users Passwords of their own IT Departments Passwords of the employees Security auditors, consultants and penetration testers Customer/contractor passwords Law enforcement & government agencies Passwords of suspects Hackers usually don t! 4

5 WHY SPEED COUNTS? Users and IT Departments: «We needed those passwords yesterday» Auditors, consultants and pentesters: «Time is Money» Law Enforcement and investigators Legal time limits 5

6 PASSWORD RECOVERY The Loop The slow part Generate trial Transform password Validate hash/ Success password (compute hash or key encryption key) Failure Try next password 6

7 PASSWORD RECOVERY The Slow Part Designed to be slow 50ms verification time has no impact on usability but HUGE impact on password recovery performance Usually designed around well-known hash functions MD5 (old days) SHA-1 (most popular so far) SHA-2 (still exotic) Thousands to millions of hash computations per password 7

8 FAST PASSWORD RECOVERY The CPU Way Before GPGPU era most optimizations focused on: SIMD (MMX, SSE, AVX) Multi-core Distributed computing (think distributed.net) Communication overhead Difficult to manage Not power-efficient 8

9 9

10 10

11 FAST PASSWORD RECOVERY The GPU Way Password recovery constitutes embarrassingly parallel workload Each processing unit verifies own password, independently from other processing units Linear scalability in practice Done by GPU Transform password Transform password Generate trial passwords Transform password Validate hashes/ keys Success Transform password Transform password Try next password Failure 11

12 FAST PASSWORD RECOVERY The GPU Way CPU GPU Generate trial passwords PCIe Passwords[] Passwords[] Compute keys from passwords Keys[] Keys[] Validate keys 12

13 LIMITATIONS Works good for slow algorithms For fast algorithms PCIe becomes the bottleneck e.g. for SHA-1 theoretical limit is 8 Gbps / (20 bytes in + 20 bytes out) 214 million passwords per second Need to offload everything to the GPU password generation and key validation on GPU are bigger challenges than crypto itself especially so without OpenCL 13

14 ALTERNATIVE WAY CPU GPU PCIe Initial password Generate trial passwords Passwords[] Compute keys from passwords Keys[] Result Validate keys 14

15 PASSWORD RECOVERY CPU GPU Generate trial passwords PCIe Passwords[] Passwords[] Compute keys from passwords Keys[] Keys[] Validate keys 15

16 OVERLAPPING CPU AND GPU In straightforward implementation it may look like this: CPU Gen Vfy Gen Vfy Gen Vfy GPU Compute Compute Compute But CPU and GPU can work simultaneously, so overlap their operations: CPU Gen Gen Vfy Gen Vfy Vfy GPU Compute Compute Compute Profit! 16

17 PERFORMANCE PBKDF2-SHA1 x Intel i NVIDIA GTX AMD HD K 15K 30K 45K 60K Computations per second 17

18 HEY, WHY NO 100X SPEEDUP? Be fair! CPUs are not single core any more Even Atoms are not Extended instruction sets were introduced for performance reasons So why ignore them? Will usually get ~10x on comparable hardware for well-suited compute-bound tasks 18

19 IO & QPI Queue IO & QPI CPU LAYOUT 1.2 billions transistors Most are L3/L2 caches L3 Cache L3 Cache Less than 10% are in execution and/or ALU units Core Core Core Core Core Core Memory Controller 19

20 GPU LAYOUT 3 billions transistors (2.5x) About 30% are execution and/or ALU units (3x) 7.5x more transistors dedicated to execution units Core frequency is about lower (~0.4x) 3x estimated speedup In fair real-world comparison this GPU is 4x faster than CPU on compute-bound task 20

21 HEY, WHY NO 100X SPEEDUP? Be fair! CPUs are not single core any more Even Atoms are not Extended instruction sets were introduced for performance reasons So why ignore them? Will usually get ~10x on comparable hardware for well-suited tasks In our case: SSE2 code + processor-specific compiler optimizations 12 threads to fully utilize 6 cores + HT 16x over high-end CPU 21

22 PERFORMANCE PBKDF2-SHA1 x Intel i NVIDIA GTX AMD HD K 15K 30K 45K 60K Computations per second 22

23 WHY AMD IS SO FAST? Most password transformations are bounded by integer performance AMD cards exhibit awesome integer performance Many password transformations (=crypto) make heavy use of bit rotations (=cyclic shifts) There is a special instruction for this! Cyclic shift in 1 instruction instead of 3, up to 30% overall speedup in practice GPU code written in IL Utilize all GPU devices under Windows (Recent APP SDK versions allow this with OpenCL) 23

24 PERFORMANCE bitalign AMD IL Specification, section 7.13: Aligns bit data for video. This is a special instruction for multi-media video. bitalign dst, src0, src1, src2 dst = (src0 >> src2.x) (src1 << (32-src2.x)) Can be used to implement cyclic bit shift in 1 instruction VERY useful for many crypto algorithms Introduced in Evergreen Exposed at the IL level 24

25 PERFORMANCE Bitfield Insert AMD Evergreen ISA Reference, page 9-61: BFI_INT dst, src0, src1, src2 dst = (src1 & src0) (src2 & -src0) This is vector bit select dst i = (mask i!= 0 )? arg1 i : arg2 i Very useful for accelerating various crypto algorithms And especially for breaking them Introduced in Evergreen NOT exposed at the IL level OpenCL bitselect() is not using it either No documented way to emit this instruction directly 25

26 WHY INTERMEDIATE LANGUAGE We chose IL over Brook+ OpenCL has not existed yet Brook+ programming model was not quite suited for password recovery ISA provided no significant benefit over IL Early OpenCL support couldn t compete with IL either Limited support for binary (pre-compiled) kernels Limited support for multi-gpu in OpenCL (Those issues seems to be fixed in APP SDK 2.4) AMD is going to deprecate CAL in next SDK (2.5) IL will almost certainly be deprecated altogether This is very bad news for us Need to decide whether to go up (OpenCL) or down (ISA) Morning Keynote mentinoed FSAIL which seems like a great alternative! 26

27 WRITING IN INTERMEDIATE LANGUAGE IL doesn t seem to be designed to be human-friendly Use scripting languages to generate IL code And handle platform-specific optimizations (i.e. emulate bitalign on older GPUs) Compile kernels at program build time Avoids runtime compilation Solves (partially) IP problem no source code needs to be distributed Need to provide new binaries for new devices Use CAL at runtime to load, configure and launch pre-compiled kernel 27

28 SCALABILITY Not all GPUs are equally powerful Program should scale nicely with number of processing cores in installed GPU Query number of processors at runtime Partition task proportionally to number of processors Helps to reduce UI update freezes Also helps to avoid TDR 28

29 SCALABILITY 8 GPUs are not uncommon today Program should scale nicely with number of GPUs Query number of devices in system Spawn thread for each device Partition task as appropriate Speedup should be linear unless you hit PCIe bandwidth limits 29

30 Disclaimer & Attribution The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. There is no obligation to update or otherwise correct or revise this information. However, we reserve the right to revise this information and to make changes from time to time to the content hereof without obligation to notify any person of such revisions or changes. NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE CONTENTS HEREOF AND NO RESPONSIBILITY IS ASSUMED FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. ALL IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. IN NO EVENT WILL ANY LIABILITY TO ANY PERSON BE INCURRED FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. All other names used in this presentation are for informational purposes only and may be trademarks of their respective owners. The contents of this presentation were provided by individual(s) and/or company listed on the title page. The information and opinions presented in this presentation may not represent AMD s positions, strategies or opinions. Unless explicitly stated, AMD is not responsible for the content herein and no endorsements are implied. 30

31 QUESTIONS?