The World's Most Popular TCL Extension. D. Richard Hipp 16 Annual Tcl/Tk Conference Portland, OR

Transcription

1 The World's Most Popular TCL Extension D. Richard Hipp 16 Annual Tcl/Tk Conference Portland, OR th

2 The only SQL database engine specifically designed to work with TCL

3

4

5 SQLite is a TCL Extension

6

7 Client Client Client Client SQL Database Engine Database Files on Disk

8 Client Client Client Client SQL Database Engine Database Files on Disk

9

10 Client Client Client Client Database Files on Disk

11 Application Tcl/Tk Code High-level SQL statements Low-level disk reads & writes

12 Why SQL? Program at a higher level Programming by specification Heads-up programming Focus on your product, not on your underlying database Maintain situational awareness

13 Do not underestimate the importance of situational awareness!

14 SELECT value FROM array WHERE name='userid'; Easier to do with an array or dict: $array('userid')

15 How many lines of TCL are required to do this using just arrays and dicts? SELECT eqptid, enclosureid FROM eqpt WHERE typeid IN ( SELECT typeid FROM typespec WHERE attrid=( SELECT attrid FROM attribute WHERE name='detect_autoactuate' ) AND value=1 INTERSECT SELECT typeid FROM typespec WHERE attrid=( SELECT attrid FROM attribute WHERE name='algorithm' ) AND value IN ('sensor','wetbulb') )

16

17 SELECT h.url, h.title, f.url, (SELECT b.parent FROM moz_bookmarks b JOIN moz_bookmarks t ON t.id = b.parent AND t.parent WHERE b.type = 1 AND b.fk = h.id ORDER BY b.lastmodified DESC LIMIT 1) AS parent, (SELECT b.title FROM moz_bookmarks b JOIN moz_bookmarks t ON t.id = b.parent AND t.parent WHERE b.type = 1 AND b.fk = h.id ORDER BY b.lastmodified DESC LIMIT 1) AS bookmark, (SELECT GROUP_CONCAT(t.title, ',') FROM moz_bookmarks b JOIN moz_bookmarks t ON t.id = b.parent AND t.parent WHERE b.type = 1 AND b.fk = h.id) AS tags, h.visit_count, h.typed, h.frecency FROM moz_places_temp h LEFT OUTER JOIN moz_favicons f ON f.id = h.favicon_id WHERE h.frecency <> 0 UNION ALL SELECT h.url, h.title, f.url, (SELECT b.parent FROM moz_bookmarks b JOIN moz_bookmarks t ON t.id = b.parent AND t.parent WHERE b.type = 1 AND b.fk = h.id ORDER BY b.lastmodified DESC LIMIT 1) AS parent, (SELECT b.title FROM moz_bookmarks b JOIN moz_bookmarks t ON t.id = b.parent AND t.parent WHERE b.type = 1 AND b.fk = h.id ORDER BY b.lastmodified DESC LIMIT 1) AS bookmark, (SELECT GROUP_CONCAT(t.title, ',') FROM moz_bookmarks b JOIN moz_bookmarks t ON t.id = b.parent AND t.parent WHERE b.type = 1 AND b.fk = h.id) AS tags, h.visit_count, h.typed, h.frecency FROM moz_places h LEFT OUTER JOIN moz_favicons f ON f.id = h.favicon_id WHERE h.id NOT IN (SELECT id FROM moz_places_temp) AND h.frecency <> 0 ORDER BY 9 DESC!=?1!=?1 =?1!=?1!=?1 =?1

18 Other benefits SQL & SQLite: Persistent Transactional Cross-platform Widely known and understood Faster Fewer bugs SQL is good at doing the very few things that TCL does not already do well.

19 tcl/tk

20 Aside: How do you classify SQLite in TCL? A small language within a small language? A meta-small language?

21 % package require sqlite % sqlite3 db database.db % Ne w o b je c t fo r c o n tr o ll i n g the d a ta b a se Na m e of t h e d a t a ba se fi l e. A n ew on e i s cr ea t ed i f i t d oes n ot a l r ea d y exi st.

22 U se t h e eva l m et h od t o r u n SQL db eval { CREATE TABLE users( userid INTEGER, first_name VARCHAR(30), last_name VARCHAR(40) ); } user first_name last_name Sem i col on sepa r a t es m u l t i pl e SQL st a t em en t s. Fi n a l sem i col on i s opt i on a l.

23 db eval { CREATE TABLE users( userid INTEGER, first_name VARCHAR(30), last_name VARCHAR(40) ); } user first_name last_name

24 db eval { CREATE TABLE users( userid INTEGER, first_name VARCHAR(30), last_name VARCHAR(40) ); } user first_name last_name D a t a t ypes a r e i g n or ed, m ost l y

25 Traditional SQL Rigid typing Types declared on containers Exceptions if type rules are violated

26 Traditional SQL Rigid typing Types declared on containers Exceptions if type rules are violated No types - everything is a string Internal dual representation Very flexible

27 Traditional SQL Rigid typing Types declared on containers Exceptions if type rules are violated No types - everything is a string Internal dual representation Very flexible

28 Traditional SQL Rigid typing Types declared on containers Exceptions if type rules are violated Type associated with values Containers have a suggested type All types accepted by every container No types - everything is a string Internal dual representation Very flexible

29 Va l u es con ver t ed t o i n t eg er i f t h ey ca n be. Ot h er wi se st or ed a s t h ey a r e. Va l u es con ver t ed t o st r i n g s. Len g t h r est r i ct i on s a r e i g n or ed user first_name db eval { CREATE TABLE users( userid INTEGER, first_name VARCHAR(30), last_name VARCHAR(40) ); } last_name Ad d i t i on a l i n for m a t i on a t h t t p:// www. sql i t e. or g / d a t a t ype 3. h t m l

30 U se a n I NSERT st a t em en t t o a d d d a t a db eval { INSERT INTO users VALUES(1, 'D. Richard', 'Hipp') }

31 db eval { INSERT INTO users VALUES(1, 'D. Richard', 'Hipp') } user first_name 1 D. Richard last_name Hipp

32 U se a SELECT st a t em en t t o ext r a ct d a t a fr om t h e d a t a ba se db eval { SELECT user, first_name, last_name FROM users } 1 {D. Richard} Hipp

33 db eval { SELECT user, first_name, last_name FROM users } 1 {D. Richard} Hipp D a t a r et u r n ed i n a TCL l i st

34 db eval { INSERT INTO users VALUES(2, 'Ginger', 'Wyrick') } user first_name last_name 1 D. Richard Hipp 2 Ginger Wyrick

35 db eval { SELECT * FROM users } 1 {D. Richard} Hipp 2 Ginger Wyrick Ad d i t i on a l r ows of d a t a ju st m a ke t h e r et u r n ed l i st l on g er

36 sqlite3 db database.db db eval {SELECT * FROM user} { puts userid=$userid puts name=$first_name $last_name } userid=1 name=d. Richard Hipp userid=2 Col u m n name=ginger Wyrick Scr i pt r u n s on ce for ea ch r ow i n r esu l t set con t en t s st or e i n TCL va r i a bl es

37 sqlite3 db database.db db eval {SELECT * FROM user} { puts userid=$userid puts name=$first_name $last_name break } userid=1 br ea k a n d name=d. Richard Hipp con t i n u e wor k i n t h e u su a l wa y

38 db eval {SELECT * FROM user} break set userid 1 set first_name Va r i a bl es per si st a ft er D. Richard t h e l a st i t er a t i on of t h e l oop set last_name Hipp

39 db eval { ALTER TABLE user ADD COLUMN picture; } user first_name 1 D. Richard last_name picture Hipp Ne w Co l u m n Ad d e d

40 set in [open drh.jpg] fconfigure $in -translation binary set drhphoto [read $in] close $in db eval { UPDATE user SET picture=$drhphoto WHERE user=1 } Not e t h e u se of cu r l y- br a ces, n ot d ou bl e- qu ot es

41 set in [open drh.jpg] fconfigure $in -translation binary set drhphoto [read $in] close $in db eval { UPDATE user SET picture=$drhphoto WHERE user=1 } No u n n ecessa r y copyi n g or qu ot i n g of l a r g e object s. I m m u n e t o SQL i n ject i on a t t a cks

42 set in [open drh.jpg] fconfigure $in -translation binary set drhphoto [read $in] close $in db eval { UPDATE user SET picture=@drhphoto WHERE user=1 i n st ea d of $ t o for ce t h e u se of t h e Byt ear r a y r epr esen t a t i on

43 db transaction { db eval {...} # other TCL code... db eval {...} } St a r t a t r a n sa ct i on COM M I T on su ccess ROLLBACK on a n y er r or

44 D efi n e a n ew TCL fu n ct i on proc sqrtfunc {x} { return [expr {sqrt($x)}] } db function sqrt sqrtfunc db eval { SELECT sqrt(id) FROM user } Reg i st er t h e fu n ct i on wi t h SQLi t e U se t h e TCL fu n ct i on i n a n SQLi t e qu er y

45 proc sqlitecon::_edit {original_text} { # Code here to implement a GUI editor # for $original_text and return the result. } db function edit ::sqlite::_edit Cr ea t e a n ew SQL fu n ct i on n a m ed ed i t i m pl em en t ed by t h e TCL pr oc :: sql i t e :: _ed i t

46

47 Go H er e Get Th i s

48 % package require Tk % source sqlitecon.txt % sqlitecon::create.console {sqlite> } test.db db %

49

50

51

52 Are you still not convinced that SQLite is a TCL extension?

53 SQLite Written Mostly In TCL

54 TCL must be installed on the development system in order to build SQLite. TCL is required to test SQLite All SQLite documentation is generated by TCL scripts.

55 select-core { stack {line SELECT {or nil DISTINCT ALL} {loop result-column,}} {optx FROM join-source} {optx WHERE expr} {optx GROUP BY {loop ordering-term,} {optx HAVING expr}} } Se e h tt p : //w i k i. tc l. tk / fo r a d d i ti o n a l i n fo r m a ti o n

56 SQLite would not exist without TCL

57 SQLite is different

58 is zero administration

59 Database Administrators

60

61

62 is to as is to

63 SQLite does not compete with Oracle

64 SQLite does not compete with Oracle SQLite competes with fopen()

65 Portable File Format A database is a single ordinary disk file No special naming conventions or required file suffixes Cross-platform: big/little-endian and 32/64-bit Backwards compatible through Promise to keep it compatible moving forward Not tied to any particular programming language.

66

67 Zero-Administration & Portable File Format means makes a great Application File Format

68 <xml/> c,s,v home grown

69

70 SQLite as file format freebies No parsing and generating code to write Atomic updates Fast, built-in searching Access via third-party tools Simplified upgrade migration Cross-platform file format High-level query language

71 Small Footprint gcc -Os -DSQLITE_THREADSAFE=0 293 KiB gcc -O3 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_RTREE=1 845 KiB Sizes Include TCL language bindings Values current as of

72 Single Source Code File The amalgamation source code file: sqlite3.c or tclsqlite3.c About 68,000 lines of ANSI C code 3.9 MB Few dependencies: libc and libtcl Very simple to add to a larger C program Very simple to build as a tclsh loadable library

73

74 The generic folder in the TEA distribution of SQLite contains exactly one file: tclsqlite3.c

75 Building a tclsh loadable library bash> gcc -shared tclsqlite3.c -o tclsqlite3.so bash> tclsh % load./tclsqlite3.so % sqlite3 db :memory: % db eval {SELECT sqlite_version()} Note Add -DSQLITE_THREADSAFE=0 for non-threadsafe tclsh

76 Other Features Of SQLite Gigibyte size BLOBs and strings Tebibyte size databases 100% branch test coverage Nested transactions Full text search R-Trees ATTACH DATABASE Robust against power loss, malloc() failures, and I/O errors. Referential integrity

77 Many companies and organizations use SQLite...

78 Adobe Photoshop Lightroom

79 Adobe Reader

80 Mozilla Firefox

81 Symbian/Nokia

82 Google Android

83 iphone

84 ipod & itunes

85 istuff

86 Blackberry

87 Palm webos

88 Skype

89 Sony Playstation

90 ... and so forth

91 Open Source

92 Have you looked at SQLite lately... Faster CHECK constraints SAVEPOINT and nested transactions Enhanced query planner 100% branch test coverage Recursive triggers FOREIGN KEY constraints Sources managed using Fossil

93 Closing Thoughts??? Shouldn't you be using SQLite instead of [open]? Why isn't tclsqlite3.c part of the TCL core? Can we get a TCL amalgamation?

94 tcl/tk