Using Clang to Visualize Large Codebases. Nathan Hawes and Ben Barham Oracle Labs Australia October 2014

Transcription

1 Using Clang to Visualize Large Codebases Nathan Hawes and Ben Barham Oracle Labs Australia October 2014

2 Safe Harbour The following is intended to provide some insight into a line of research in Oracle Labs. It is intended for informamon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcmonality, and should not be relied upon in making purchasing decisions. Oracle reserves the right to alter its development plans and pracmces at any Mme, and the development, release, and Mming of any features or funcmonality described in connecmon with any Oracle product or service remains at the sole discremon of Oracle. Any views expressed in this presentamon are my own and do not necessarily reflect the views of Oracle. 2

3 Code Comprehension The truth is in the source! 3

4 But what if that source is large? 10 million lines 4

5 But what if that source is large? 10 million lines 50 lines 5

6 But what if that source is large? 10 million lines 50 lines 6

7 Code Comprehension in IDEs Go to definimon, find uses, class overview, type hierarchy, etc. IDEs impracmcal to use for large C/C++ codebases Imprecise language recognimon Issues with custom build systems 7

8 Current PracMce For large C/C++ codebases Text editors and text- search tools vim, emacs grep, sed, cscope Fast and simple But imprecise à Symbol types, scopes, linking informamon, preprocessor Low- level focus static VALUE mnew( ) { data- >id = rid; } Find definimon method.h:70 node.h:244 thread_pthread.c:594 (+ 17 more) Actual definimon (14 th ) proc.c:21 8

9 Frappé Aims Provide precise dependency informamon With easy build integramon Allow users to specify higher- level queries directly Not just defs or refs Show users the broader context of the system 9

10 Frappé Overview EXTRACT QUERY DISPLAY Source Code Dependency Graph User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 10

11 Dependency Graph Natural representamon of the code Call graphs, type hierarchies, control flow graphs, etc. Nodes and edges Build system: modules, files, and linking informamon between them File system: directories and files Preprocessor: includes, macros, their expansion and interrogamons Symbols: funcmons, locals, types, and relamons between them High- level quesmons become graph queries Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 11

12 Go to DefiniMon main() writes.buf contains scope{} foo() bar() Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 12

13 Go to DefiniMon main() writes.buf contains scope{} foo() bar() Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 13

14 Go to DefiniMon main() writes.buf contains scope{} foo() bar() Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 14

15 Find References main() writes.buf contains scope{} foo() bar() Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 15

16 Find References main() writes.buf contains scope{} foo() bar() Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 16

17 Find References main() writes.buf contains scope{} foo() bar() Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 17

18 Impact EsMmaMon insert() pop() first() init() #PREV expands #FIRST #NULL Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 18

19 Impact EsMmaMon insert() pop() first() init() #PREV expands #FIRST #NULL Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 19

20 Impact EsMmaMon insert() pop() first() init() #PREV expands #FIRST #NULL Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 20

21 Impact EsMmaMon insert() pop() first() init() #PREV expands #FIRST #NULL Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 21

22 Frappé Architecture EXTRACTOR IMPORTER EDITOR PLUGINS CLI SCRIPTS WEB UI Source Code.fo Server User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 22

23 Frappé Architecture EXTRACTOR IMPORTER EDITOR PLUGINS CLI SCRIPTS WEB UI Source Code.fo Server User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 23

24 Extractor Simple Build IntegraLon Compiler Wrappers NaMve Compiler Clang + Plugin.o.fo 24

25 Extractor Simple Build IntegraLon Compiler Wrappers NaMve Compiler Clang + Plugin.o.fo 25

26 Extractor Simple Build IntegraLon Compiler Wrappers NaMve Compiler Clang + Plugin.o.fo 26

27 Extractor Clang Plugin PPCallbacks Includes, macros, their expansions and interrogamons RecursiveASTVisitor Visit all declaramons, types, and expressions Easy to use interface Provides detailed locamon informamon 27

28 Frappé Architecture EXTRACTOR IMPORTER EDITOR PLUGINS CLI SCRIPTS WEB UI Source Code.fo Server User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 28

29 Frappé Architecture EXTRACTOR IMPORTER EDITOR PLUGINS CLI SCRIPTS WEB UI Source Code.fo Server User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 29

30 Frappé Architecture EXTRACTOR IMPORTER EDITOR PLUGINS CLI SCRIPTS WEB UI Source Code.fo Server User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 30

31 Frappé Architecture EXTRACTOR IMPORTER EDITOR PLUGINS CLI SCRIPTS WEB UI Source Code.fo Server User Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle ConfidenMal - Restricted 31

32 Code Maps Visualising large codebases 32

33 Code Maps Using a cartographic map metaphor ConMnent/country/state/city à module/sub- module/file/funcmon DisMncMve shape and posimons serve as landmarks Can overlay a variety of informamon 33

34 Overlay Search Results Visual filtering Contextual search Copyright 2014 Oracle and/or its affiliates. All rights reserved. Frappé: a code comprehension tool for large codebases 34

35 Overlay Paths java.lang.runtimeexception: bad at Foobar.setup(Foobar.java:74) at Foobar.launch(Foobar.java:43) at Bar.launch(Bar.java:39) at Bar.bar(Bar.java:97) at Foo.foo(Foo.java:35) at Main.main(Main.java:104) Path in stack trace Path on map 35

36 Overlay Metrics Fine granularity Coarse granularity 36

37 But how? 37

38 ExisMng Approaches Deline, R. Staying oriented with sooware terrain maps In proc. of the workshop of visual languages and computamon, 2005 Kuhn, A.; Erni, D.; Loretan, P.; Nierstrasz, O. SoOware cartography: themalc sooware visualizalon with consistent layout Journal of Sopware Maintenance and EvoluMon,

39 Input AbstracMon hierarchy Abstracts files into higher level groupings Use directory structure by default Dependency graph Represents dependencies between files as a weighted edge Use references 39

40 Map GeneraMon Graph layout Implicit surface generamon Surface subdivision Recursive subdivision 40

41 Map GeneraMon Implicit surface generamon Graph layout Noack, A. & Lewerentz, C. A space of layout styles for hierarchical graph models of sooware systems Proceedings of the 2005 ACM symposium on Sopware visualizamon, ACM, 2005, Recursive subdivision Nocaj, A. & Brandes, U. CompuLng Voronoi Treemaps: Faster, Simpler, and ResoluLon- independent Computer Graphics Forum, Blackwell Publishing Ltd, 2012, 31,

42 DemonstraMon Linux 42

43 Future Work More detailed dependency graph Find calls where third argument is macro FLAG Find all funcmons the pointer fptr could point to More overlays Test coverage, profiling data Store mulmple versions Impact esmmamon Code map evolumon (stability) 43

44 Nathan Hawes and Ben Barham Oracle Labs Australia hsp://labs.oracle.com/locamons/australia Research Director

45